* [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver
@ 2024-07-09 13:28 Daniel Scally
2024-07-09 13:28 ` [PATCH v6 01/18] media: mc-entity: Record number of video devices in a pipeline Daniel Scally
` (17 more replies)
0 siblings, 18 replies; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:28 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
Hello all
This patchset introduces a driver for Arm's Mali-C55 Image Signal Processor.
The driver uses the V4L2 / media controller API and implements both of the ISP's
capture pipelines allowing a range of output formats plus downscaling and
cropping. The capture pipelines are named "Full resolution" and "Downscale" and
so abbreviated FR and DS throughout the driver.
The driver exposes 4 V4L2 subdevices:
- mali-c55 isp: input data formatting
- mali-c55 tpg: test pattern generator (modeled as a camera sensor entity)
- mali-c55 resizer fr: downscale / crop and format setting for the FR pipe
- mali-c55 resizer ds: downscale / crop and format setting for the DS pipe
Along with 4 V4L2 Video devices:
- mali-c55 fr: Capture device for the full resolution pipe
- mali-c55 ds: Capture device for the downscale pipe
- mali-c55 3a stats: Capture device for statistics to support 3A algorithms
- mali-c55 3a params: Output device for parameter buffers to configure the ISP
Support is implemented in the parameters video device code for many of the ISP'S
hardware blocks, but not yet all of them. The buffer format is (as far as I am
aware anyway) a novel design that we intend to be extensible so that support for
the C55's remaining hardware blocks can be added later.
Patches 1, 4, 5, 6 and 7 have already had versions 1-4 on the mailing list...I
decided to post the additional work on the driver as extra patches rather than
merge them all into the existing series as it's already a lot of code to review
and I hoped that that might make it a little easier...if I'm wrong and that's
not liked I can just squash them into a much smaller series.
The rest of this message comprises the v4l2-compliance report for the driver.
The tool reports a single failure and two warnings, but they're all from the
imx415 driver (with which I'm testing the code)
v4l2-compliance 1.27.0-5220, 64 bits, 64-bit time_t
v4l2-compliance SHA: 8387e3673837 2024-07-01 11:09:32
Compliance test for mali-c55 device /dev/media0:
Media Driver Info:
Driver name : mali-c55
Model : ARM Mali-C55 ISP
Serial :
Bus info : platform:60400000.isp
Media version : 6.9.0
Hardware revision: 0x01d982d6 (31032022)
Driver version : 6.9.0
Required ioctls:
test MEDIA_IOC_DEVICE_INFO: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/media0 open: OK
test MEDIA_IOC_DEVICE_INFO: OK
test for unlimited opens: OK
Media Controller ioctls:
test MEDIA_IOC_G_TOPOLOGY: OK
Entities: 10 Interfaces: 10 Pads: 18 Links: 20
test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
test MEDIA_IOC_SETUP_LINK: OK
Total for mali-c55 device /dev/media0: 8, Succeeded: 8, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mali-c55 device /dev/video0:
Driver Info:
Driver name : mali-c55
Card type : ARM Mali-C55 ISP
Bus info : platform:60400000.isp
Driver version : 6.9.0
Capabilities : 0xa4201000
Video Capture Multiplanar
I/O MC
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x24201000
Video Capture Multiplanar
I/O MC
Streaming
Extended Pix Format
Media Driver Info:
Driver name : mali-c55
Model : ARM Mali-C55 ISP
Serial :
Bus info : platform:60400000.isp
Media version : 6.9.0
Hardware revision: 0x01d982d6 (31032022)
Driver version : 6.9.0
Interface Info:
ID : 0x03000012
Type : V4L Video
Entity Info:
ID : 0x00000010 (16)
Name : mali-c55 fr
Function : V4L2 I/O
Pad 0x01000011 : 0: Sink
Link 0x02000026: from remote pad 0x100000b of entity 'mali-c55 resizer fr' (Video Scaler): Data, Enabled
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 1 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls (Input 0):
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
test VIDIOC_QUERYCTRL: OK (Not Supported)
test VIDIOC_G/S_CTRL: OK (Not Supported)
test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 0 Private Controls: 0
Format ioctls (Input 0):
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK
Codec ioctls (Input 0):
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls (Input 0):
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
Total for mali-c55 device /dev/video0: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mali-c55 device /dev/video1:
Driver Info:
Driver name : mali-c55
Card type : ARM Mali-C55 ISP
Bus info : platform:60400000.isp
Driver version : 6.9.0
Capabilities : 0xa4201000
Video Capture Multiplanar
I/O MC
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x24201000
Video Capture Multiplanar
I/O MC
Streaming
Extended Pix Format
Media Driver Info:
Driver name : mali-c55
Model : ARM Mali-C55 ISP
Serial :
Bus info : platform:60400000.isp
Media version : 6.9.0
Hardware revision: 0x01d982d6 (31032022)
Driver version : 6.9.0
Interface Info:
ID : 0x03000016
Type : V4L Video
Entity Info:
ID : 0x00000014 (20)
Name : mali-c55 ds
Function : V4L2 I/O
Pad 0x01000015 : 0: Sink
Link 0x0200002a: from remote pad 0x100000f of entity 'mali-c55 resizer ds' (Video Scaler): Data, Enabled
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video1 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 1 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls (Input 0):
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
test VIDIOC_QUERYCTRL: OK (Not Supported)
test VIDIOC_G/S_CTRL: OK (Not Supported)
test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 0 Private Controls: 0
Format ioctls (Input 0):
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK
Codec ioctls (Input 0):
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls (Input 0):
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
Total for mali-c55 device /dev/video1: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mali-c55 device /dev/video2:
Driver Info:
Driver name : mali-c55
Card type : ARM Mali-C55 ISP
Bus info : platform:60400000.isp
Driver version : 6.9.0
Capabilities : 0x8c200000
Metadata Output
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x0c200000
Metadata Output
Streaming
Extended Pix Format
Media Driver Info:
Driver name : mali-c55
Model : ARM Mali-C55 ISP
Serial :
Bus info : platform:60400000.isp
Media version : 6.9.0
Hardware revision: 0x01d982d6 (31032022)
Driver version : 6.9.0
Interface Info:
ID : 0x0300001a
Type : V4L Video
Entity Info:
ID : 0x00000018 (24)
Name : mali-c55 3a params
Function : V4L2 I/O
Pad 0x01000019 : 0: Source
Link 0x0200002e: to remote pad 0x1000008 of entity 'mali-c55 isp' (Image Signal Processor): Data, Enabled
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video2 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
test VIDIOC_QUERYCTRL: OK (Not Supported)
test VIDIOC_G/S_CTRL: OK (Not Supported)
test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 0 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
Total for mali-c55 device /dev/video2: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mali-c55 device /dev/video3:
Driver Info:
Driver name : mali-c55
Card type : ARM Mali-C55 ISP
Bus info : platform:60400000.isp
Driver version : 6.9.0
Capabilities : 0x84a00000
Metadata Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04a00000
Metadata Capture
Streaming
Extended Pix Format
Media Driver Info:
Driver name : mali-c55
Model : ARM Mali-C55 ISP
Serial :
Bus info : platform:60400000.isp
Media version : 6.9.0
Hardware revision: 0x01d982d6 (31032022)
Driver version : 6.9.0
Interface Info:
ID : 0x0300001e
Type : V4L Video
Entity Info:
ID : 0x0000001c (28)
Name : mali-c55 3a stats
Function : V4L2 I/O
Pad 0x0100001d : 0: Sink
Link 0x0200002c: from remote pad 0x1000007 of entity 'mali-c55 isp' (Image Signal Processor): Data, Enabled
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video3 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
test VIDIOC_QUERYCTRL: OK (Not Supported)
test VIDIOC_G/S_CTRL: OK (Not Supported)
test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 0 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
Total for mali-c55 device /dev/video3: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mali-c55 device /dev/v4l-subdev0:
Driver Info:
Driver version : 6.9.0
Capabilities : 0x00000000
Client Capabilities: 0x0000000000000003
streams interval-uses-which Media Driver Info:
Driver name : mali-c55
Model : ARM Mali-C55 ISP
Serial :
Bus info : platform:60400000.isp
Media version : 6.9.0
Hardware revision: 0x01d982d6 (31032022)
Driver version : 6.9.0
Interface Info:
ID : 0x03000039
Type : V4L Sub-Device
Entity Info:
ID : 0x00000001 (1)
Name : mali-c55 tpg
Function : Camera Sensor
Pad 0x01000002 : 0: Source
Link 0x02000020: to remote pad 0x1000004 of entity 'mali-c55 isp' (Image Signal Processor): Data
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_SUDBEV_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/v4l-subdev0 open: OK
test VIDIOC_SUBDEV_QUERYCAP: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Sub-Device ioctls (Source Pad 0):
Try Stream 0
test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Try VIDIOC_SUBDEV_G/S_FMT: OK
test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
Active Stream 0
test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Active VIDIOC_SUBDEV_G/S_FMT: OK
test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 6 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK (Not Supported)
test VIDIOC_TRY_FMT: OK (Not Supported)
test VIDIOC_S_FMT: OK (Not Supported)
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK (Not Supported)
test Requests: OK (Not Supported)
Total for mali-c55 device /dev/v4l-subdev0: 53, Succeeded: 53, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mali-c55 device /dev/v4l-subdev1:
Driver Info:
Driver version : 6.9.0
Capabilities : 0x00000000
Client Capabilities: 0x0000000000000003
streams interval-uses-which Media Driver Info:
Driver name : mali-c55
Model : ARM Mali-C55 ISP
Serial :
Bus info : platform:60400000.isp
Media version : 6.9.0
Hardware revision: 0x01d982d6 (31032022)
Driver version : 6.9.0
Interface Info:
ID : 0x0300003b
Type : V4L Sub-Device
Entity Info:
ID : 0x00000003 (3)
Name : mali-c55 isp
Function : Image Signal Processor
Pad 0x01000004 : 0: Sink, Must Connect
Link 0x02000020: from remote pad 0x1000002 of entity 'mali-c55 tpg' (Camera Sensor): Data
Link 0x02000033: from remote pad 0x1000032 of entity 'lte-csi2-rx' (Video Interface Bridge): Data, Enabled
Pad 0x01000005 : 1: Source
Link 0x02000022: to remote pad 0x100000a of entity 'mali-c55 resizer fr' (Video Scaler): Data, Enabled, Immutable
Link 0x02000028: to remote pad 0x100000e of entity 'mali-c55 resizer ds' (Video Scaler): Data, Enabled, Immutable
Pad 0x01000006 : 2: Source
Link 0x02000024: to remote pad 0x100000c of entity 'mali-c55 resizer fr' (Video Scaler): Data, Enabled, Immutable
Pad 0x01000007 : 3: Source
Link 0x0200002c: to remote pad 0x100001d of entity 'mali-c55 3a stats' (V4L2 I/O): Data, Enabled
Pad 0x01000008 : 4: Sink
Link 0x0200002e: from remote pad 0x1000019 of entity 'mali-c55 3a params' (V4L2 I/O): Data, Enabled
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_SUDBEV_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/v4l-subdev1 open: OK
test VIDIOC_SUBDEV_QUERYCAP: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Sub-Device ioctls (Sink Pad 0):
Try Stream 0
test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Try VIDIOC_SUBDEV_G/S_FMT: OK
test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK
Active Stream 0
test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Active VIDIOC_SUBDEV_G/S_FMT: OK
test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK
test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
Sub-Device ioctls (Source Pad 1):
Try Stream 0
test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Try VIDIOC_SUBDEV_G/S_FMT: OK
test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
Active Stream 0
test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Active VIDIOC_SUBDEV_G/S_FMT: OK
test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
Sub-Device ioctls (Source Pad 2):
Try Stream 0
test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Try VIDIOC_SUBDEV_G/S_FMT: OK
test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
Active Stream 0
test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Active VIDIOC_SUBDEV_G/S_FMT: OK
test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
Sub-Device ioctls (Source Pad 3):
Try Stream 0
test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Try VIDIOC_SUBDEV_G/S_FMT: OK
test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
Active Stream 0
test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Active VIDIOC_SUBDEV_G/S_FMT: OK
test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
Sub-Device ioctls (Sink Pad 4):
Try Stream 0
test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Try VIDIOC_SUBDEV_G/S_FMT: OK
test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
Active Stream 0
test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Active VIDIOC_SUBDEV_G/S_FMT: OK
test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
test VIDIOC_QUERYCTRL: OK (Not Supported)
test VIDIOC_G/S_CTRL: OK (Not Supported)
test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 0 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK (Not Supported)
test VIDIOC_TRY_FMT: OK (Not Supported)
test VIDIOC_S_FMT: OK (Not Supported)
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK (Not Supported)
test Requests: OK (Not Supported)
Total for mali-c55 device /dev/v4l-subdev1: 81, Succeeded: 81, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mali-c55 device /dev/v4l-subdev2:
Driver Info:
Driver version : 6.9.0
Capabilities : 0x00000002
Streams Support
Client Capabilities: 0x0000000000000003
streams interval-uses-which Media Driver Info:
Driver name : mali-c55
Model : ARM Mali-C55 ISP
Serial :
Bus info : platform:60400000.isp
Media version : 6.9.0
Hardware revision: 0x01d982d6 (31032022)
Driver version : 6.9.0
Interface Info:
ID : 0x0300003d
Type : V4L Sub-Device
Entity Info:
ID : 0x00000009 (9)
Name : mali-c55 resizer fr
Function : Video Scaler
Pad 0x0100000a : 0: Sink
Link 0x02000022: from remote pad 0x1000005 of entity 'mali-c55 isp' (Image Signal Processor): Data, Enabled, Immutable
Pad 0x0100000b : 1: Source
Link 0x02000026: to remote pad 0x1000011 of entity 'mali-c55 fr' (V4L2 I/O): Data, Enabled
Pad 0x0100000c : 2: Sink
Link 0x02000024: from remote pad 0x1000006 of entity 'mali-c55 isp' (Image Signal Processor): Data, Enabled, Immutable
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_SUDBEV_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/v4l-subdev2 open: OK
test VIDIOC_SUBDEV_QUERYCAP: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Sub-Device routing ioctls:
test Try VIDIOC_SUBDEV_G_ROUTING/VIDIOC_SUBDEV_S_ROUTING: OK
test Active VIDIOC_SUBDEV_G_ROUTING/VIDIOC_SUBDEV_S_ROUTING: OK
Sub-Device ioctls (Sink Pad 0):
Try Stream 0
test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Try VIDIOC_SUBDEV_G/S_FMT: OK
test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK
Active Stream 0
test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Active VIDIOC_SUBDEV_G/S_FMT: OK
test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK
test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
Sub-Device ioctls (Source Pad 1):
Try Stream 0
test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Try VIDIOC_SUBDEV_G/S_FMT: OK
test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
Active Stream 0
test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Active VIDIOC_SUBDEV_G/S_FMT: OK
test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
Sub-Device ioctls (Sink Pad 2):
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
test VIDIOC_QUERYCTRL: OK (Not Supported)
test VIDIOC_G/S_CTRL: OK (Not Supported)
test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 0 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK (Not Supported)
test VIDIOC_TRY_FMT: OK (Not Supported)
test VIDIOC_S_FMT: OK (Not Supported)
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK (Not Supported)
test Requests: OK (Not Supported)
Total for mali-c55 device /dev/v4l-subdev2: 62, Succeeded: 62, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mali-c55 device /dev/v4l-subdev3:
Driver Info:
Driver version : 6.9.0
Capabilities : 0x00000002
Streams Support
Client Capabilities: 0x0000000000000003
streams interval-uses-which Media Driver Info:
Driver name : mali-c55
Model : ARM Mali-C55 ISP
Serial :
Bus info : platform:60400000.isp
Media version : 6.9.0
Hardware revision: 0x01d982d6 (31032022)
Driver version : 6.9.0
Interface Info:
ID : 0x0300003f
Type : V4L Sub-Device
Entity Info:
ID : 0x0000000d (13)
Name : mali-c55 resizer ds
Function : Video Scaler
Pad 0x0100000e : 0: Sink
Link 0x02000028: from remote pad 0x1000005 of entity 'mali-c55 isp' (Image Signal Processor): Data, Enabled, Immutable
Pad 0x0100000f : 1: Source
Link 0x0200002a: to remote pad 0x1000015 of entity 'mali-c55 ds' (V4L2 I/O): Data, Enabled
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_SUDBEV_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/v4l-subdev3 open: OK
test VIDIOC_SUBDEV_QUERYCAP: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Sub-Device routing ioctls:
test Try VIDIOC_SUBDEV_G_ROUTING/VIDIOC_SUBDEV_S_ROUTING: OK
test Active VIDIOC_SUBDEV_G_ROUTING/VIDIOC_SUBDEV_S_ROUTING: OK
Sub-Device ioctls (Sink Pad 0):
Try Stream 0
test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Try VIDIOC_SUBDEV_G/S_FMT: OK
test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK
Active Stream 0
test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Active VIDIOC_SUBDEV_G/S_FMT: OK
test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK
test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
Sub-Device ioctls (Source Pad 1):
Try Stream 0
test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Try VIDIOC_SUBDEV_G/S_FMT: OK
test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
Active Stream 0
test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Active VIDIOC_SUBDEV_G/S_FMT: OK
test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
test VIDIOC_QUERYCTRL: OK (Not Supported)
test VIDIOC_G/S_CTRL: OK (Not Supported)
test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 0 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK (Not Supported)
test VIDIOC_TRY_FMT: OK (Not Supported)
test VIDIOC_S_FMT: OK (Not Supported)
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK (Not Supported)
test Requests: OK (Not Supported)
Total for mali-c55 device /dev/v4l-subdev3: 62, Succeeded: 62, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mali-c55 device /dev/v4l-subdev4:
Driver Info:
Driver version : 6.9.0
Capabilities : 0x00000000
Client Capabilities: 0x0000000000000003
streams interval-uses-which Media Driver Info:
Driver name : mali-c55
Model : ARM Mali-C55 ISP
Serial :
Bus info : platform:60400000.isp
Media version : 6.9.0
Hardware revision: 0x01d982d6 (31032022)
Driver version : 6.9.0
Interface Info:
ID : 0x03000041
Type : V4L Sub-Device
Entity Info:
ID : 0x00000030 (48)
Name : lte-csi2-rx
Function : Video Interface Bridge
Pad 0x01000031 : 0: Sink, Must Connect
Link 0x02000037: from remote pad 0x1000036 of entity 'imx415 1-001a' (Camera Sensor): Data, Enabled, Immutable
Pad 0x01000032 : 1: Source, Must Connect
Link 0x02000033: to remote pad 0x1000004 of entity 'mali-c55 isp' (Image Signal Processor): Data, Enabled
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_SUDBEV_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/v4l-subdev4 open: OK
test VIDIOC_SUBDEV_QUERYCAP: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Sub-Device ioctls (Sink Pad 0):
Try Stream 0
test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
test Try VIDIOC_SUBDEV_G/S_FMT: OK
test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
Active Stream 0
test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
test Active VIDIOC_SUBDEV_G/S_FMT: OK
test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
Sub-Device ioctls (Source Pad 1):
Try Stream 0
test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
test Try VIDIOC_SUBDEV_G/S_FMT: OK
test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
Active Stream 0
test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
test Active VIDIOC_SUBDEV_G/S_FMT: OK
test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
test VIDIOC_QUERYCTRL: OK (Not Supported)
test VIDIOC_G/S_CTRL: OK (Not Supported)
test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 0 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK (Not Supported)
test VIDIOC_TRY_FMT: OK (Not Supported)
test VIDIOC_S_FMT: OK (Not Supported)
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK (Not Supported)
test Requests: OK (Not Supported)
Total for mali-c55 device /dev/v4l-subdev4: 60, Succeeded: 60, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mali-c55 device /dev/v4l-subdev5:
Driver Info:
Driver version : 6.9.0
Capabilities : 0x00000000
Client Capabilities: 0x0000000000000003
streams interval-uses-which Media Driver Info:
Driver name : mali-c55
Model : ARM Mali-C55 ISP
Serial :
Bus info : platform:60400000.isp
Media version : 6.9.0
Hardware revision: 0x01d982d6 (31032022)
Driver version : 6.9.0
Interface Info:
ID : 0x03000043
Type : V4L Sub-Device
Entity Info:
ID : 0x00000035 (53)
Name : imx415 1-001a
Function : Camera Sensor
Pad 0x01000036 : 0: Source
Link 0x02000037: to remote pad 0x1000031 of entity 'lte-csi2-rx' (Video Interface Bridge): Data, Enabled, Immutable
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_SUDBEV_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/v4l-subdev5 open: OK
test VIDIOC_SUBDEV_QUERYCAP: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Sub-Device ioctls (Source Pad 0):
Try Stream 0
test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Try VIDIOC_SUBDEV_G/S_FMT: OK
warn: ../utils/v4l2-compliance/v4l2-test-subdevs.cpp(566): VIDIOC_SUBDEV_G_SELECTION is supported for target 0 but not VIDIOC_SUBDEV_S_SELECTION
test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK
Active Stream 0
test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
test Active VIDIOC_SUBDEV_G/S_FMT: OK
warn: ../utils/v4l2-compliance/v4l2-test-subdevs.cpp(566): VIDIOC_SUBDEV_G_SELECTION is supported for target 0 but not VIDIOC_SUBDEV_S_SELECTION
test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK
test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
fail: ../utils/v4l2-compliance/v4l2-test-controls.cpp(1108): subscribe event for control 'User Controls' failed
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 14 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK (Not Supported)
test VIDIOC_TRY_FMT: OK (Not Supported)
test VIDIOC_S_FMT: OK (Not Supported)
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK (Not Supported)
test Requests: OK (Not Supported)
Total for mali-c55 device /dev/v4l-subdev5: 53, Succeeded: 52, Failed: 1, Warnings: 2
Grand Total for mali-c55 device /dev/media0: 571, Succeeded: 570, Failed: 1, Warnings: 2
Thanks
Dan
Daniel Scally (17):
media: mc-entity: Record number of video devices in a pipeline
media: uapi: Add MEDIA_BUS_FMT_RGB202020_1X60 format code
media: uapi: Add 20-bit bayer formats
media: v4l2-common: Add RAW16 format info
media: v4l2-common: Add RAW14 format info
dt-bindings: media: Add bindings for ARM mali-c55
media: mali-c55: Add Mali-C55 ISP driver
media: Documentation: Add Mali-C55 ISP Documentation
MAINTAINERS: Add entry for mali-c55 driver
media: Add MALI_C55_3A_STATS meta format
media: uapi: Add 3a stats buffer for mali-c55
media: platform: Add mali-c55 3a stats devnode
media: platform: Fill stats buffer on ISP_START
Documentation: mali-c55: Add Statistics documentation
media: uapi: Add parameters structs to mali-c55-config.h
media: platform: Add mali-c55 parameters video node
Documentation: mali-c55: Document the mali-c55 parameter setting
Jacopo Mondi (1):
media: mali-c55: Add image formats for Mali-C55 parameters buffer
.../admin-guide/media/mali-c55-graph.dot | 19 +
Documentation/admin-guide/media/mali-c55.rst | 413 +++++++
.../admin-guide/media/v4l-drivers.rst | 1 +
.../bindings/media/arm,mali-c55.yaml | 66 +
.../userspace-api/media/v4l/meta-formats.rst | 1 +
.../media/v4l/metafmt-arm-mali-c55.rst | 89 ++
.../media/v4l/subdev-formats.rst | 420 ++++++-
MAINTAINERS | 13 +
drivers/media/mc/mc-entity.c | 5 +
drivers/media/platform/Kconfig | 1 +
drivers/media/platform/Makefile | 1 +
drivers/media/platform/arm/Kconfig | 5 +
drivers/media/platform/arm/Makefile | 2 +
drivers/media/platform/arm/mali-c55/Kconfig | 17 +
drivers/media/platform/arm/mali-c55/Makefile | 11 +
.../platform/arm/mali-c55/mali-c55-capture.c | 961 +++++++++++++++
.../platform/arm/mali-c55/mali-c55-common.h | 296 +++++
.../platform/arm/mali-c55/mali-c55-core.c | 949 ++++++++++++++
.../platform/arm/mali-c55/mali-c55-isp.c | 561 +++++++++
.../platform/arm/mali-c55/mali-c55-params.c | 671 ++++++++++
.../arm/mali-c55/mali-c55-registers.h | 444 +++++++
.../platform/arm/mali-c55/mali-c55-resizer.c | 1096 +++++++++++++++++
.../platform/arm/mali-c55/mali-c55-stats.c | 373 ++++++
.../platform/arm/mali-c55/mali-c55-tpg.c | 438 +++++++
drivers/media/v4l2-core/v4l2-common.c | 8 +
drivers/media/v4l2-core/v4l2-ioctl.c | 2 +
include/media/media-entity.h | 2 +
include/uapi/linux/media-bus-format.h | 9 +-
.../uapi/linux/media/arm/mali-c55-config.h | 945 ++++++++++++++
include/uapi/linux/videodev2.h | 3 +
30 files changed, 7818 insertions(+), 4 deletions(-)
create mode 100644 Documentation/admin-guide/media/mali-c55-graph.dot
create mode 100644 Documentation/admin-guide/media/mali-c55.rst
create mode 100644 Documentation/devicetree/bindings/media/arm,mali-c55.yaml
create mode 100644 Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst
create mode 100644 drivers/media/platform/arm/Kconfig
create mode 100644 drivers/media/platform/arm/Makefile
create mode 100644 drivers/media/platform/arm/mali-c55/Kconfig
create mode 100644 drivers/media/platform/arm/mali-c55/Makefile
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-capture.c
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-common.h
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-core.c
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-isp.c
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-params.c
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-registers.h
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-stats.c
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-tpg.c
create mode 100644 include/uapi/linux/media/arm/mali-c55-config.h
--
2.34.1
^ permalink raw reply [flat|nested] 41+ messages in thread
* [PATCH v6 01/18] media: mc-entity: Record number of video devices in a pipeline
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
@ 2024-07-09 13:28 ` Daniel Scally
2024-07-30 15:09 ` Laurent Pinchart
2024-07-09 13:28 ` [PATCH v6 02/18] media: uapi: Add MEDIA_BUS_FMT_RGB202020_1X60 format code Daniel Scally
` (16 subsequent siblings)
17 siblings, 1 reply; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:28 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
Record the number of video devices in a pipeline so that we can track
in a central location how many need to be started before drivers must
actually begin streaming.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- New patch. This is intended to support Sakari's requirement for the
driver not to start streaming before all of the video devices have
called streamon(). This was the cleanest way I could think to acheive
the goal, and lets us just check for start_count == required_count
before streaming.
drivers/media/mc/mc-entity.c | 5 +++++
include/media/media-entity.h | 2 ++
2 files changed, 7 insertions(+)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 96dd0f6ccd0d..1e8186b13b55 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -596,6 +596,9 @@ static int media_pipeline_add_pad(struct media_pipeline *pipe,
list_add_tail(&ppad->list, &pipe->pads);
+ if (pad->entity->obj_type == MEDIA_ENTITY_TYPE_VIDEO_DEVICE)
+ pipe->required_count++;
+
dev_dbg(pad->graph_obj.mdev->dev,
"media pipeline: added pad '%s':%u\n",
pad->entity->name, pad->index);
@@ -713,6 +716,8 @@ static void media_pipeline_cleanup(struct media_pipeline *pipe)
list_del(&ppad->list);
kfree(ppad);
}
+
+ pipe->required_count = 0;
}
static int media_pipeline_populate(struct media_pipeline *pipe,
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 0393b23129eb..ab84458b40dc 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -104,12 +104,14 @@ struct media_graph {
* @mdev: The media device the pipeline is part of
* @pads: List of media_pipeline_pad
* @start_count: Media pipeline start - stop count
+ * @required_count: Number of starts required to be "running"
*/
struct media_pipeline {
bool allocated;
struct media_device *mdev;
struct list_head pads;
int start_count;
+ int required_count;
};
/**
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 02/18] media: uapi: Add MEDIA_BUS_FMT_RGB202020_1X60 format code
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
2024-07-09 13:28 ` [PATCH v6 01/18] media: mc-entity: Record number of video devices in a pipeline Daniel Scally
@ 2024-07-09 13:28 ` Daniel Scally
2024-07-09 13:28 ` [PATCH v6 03/18] media: uapi: Add 20-bit bayer formats Daniel Scally
` (15 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:28 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
The Mali-C55 ISP by ARM requires 20-bits per colour channel input on
the bus. Add a new media bus format code to represent it.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- Fixed the table formatting
Changes in v5:
- none
Changes in v4:
- None
Changes in v3:
- None
Changes in v2:
- none
.../media/v4l/subdev-formats.rst | 168 ++++++++++++++++++
include/uapi/linux/media-bus-format.h | 3 +-
2 files changed, 170 insertions(+), 1 deletion(-)
diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
index d2a6cd2e1eb2..5dbf8c9b18fb 100644
--- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
+++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
@@ -2224,6 +2224,174 @@ The following table list existing packed 48bit wide RGB formats.
\endgroup
+The following table list existing packed 60bit wide RGB formats.
+
+.. tabularcolumns:: |p{4.0cm}|p{0.7cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|
+
+.. _v4l2-mbus-pixelcode-rgb-60:
+
+.. raw:: latex
+
+ \begingroup
+ \tiny
+ \setlength{\tabcolsep}{2pt}
+
+.. flat-table:: 60bit RGB formats
+ :header-rows: 3
+ :stub-columns: 0
+ :widths: 36 7 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
+
+ * - Identifier
+ - Code
+ -
+ - :cspan:`31` Data organization
+ * -
+ -
+ - Bit
+ -
+ -
+ -
+ -
+ - 59
+ - 58
+ - 57
+ - 56
+ - 55
+ - 54
+ - 53
+ - 52
+ - 51
+ - 50
+ - 49
+ - 48
+ - 47
+ - 46
+ - 45
+ - 44
+ - 43
+ - 42
+ - 41
+ - 40
+ - 39
+ - 38
+ - 37
+ - 36
+ - 35
+ - 34
+ - 33
+ - 32
+ * -
+ -
+ -
+ - 31
+ - 30
+ - 29
+ - 28
+ - 27
+ - 26
+ - 25
+ - 24
+ - 23
+ - 22
+ - 21
+ - 20
+ - 19
+ - 18
+ - 17
+ - 16
+ - 15
+ - 14
+ - 13
+ - 12
+ - 11
+ - 10
+ - 9
+ - 8
+ - 7
+ - 6
+ - 5
+ - 4
+ - 3
+ - 2
+ - 1
+ - 0
+ * .. _MEDIA-BUS-FMT-RGB202020-1X60:
+
+ - MEDIA_BUS_FMT_RGB202020_1X60
+ - 0x1026
+ -
+ -
+ -
+ -
+ -
+ - r\ :sub:`19`
+ - r\ :sub:`18`
+ - r\ :sub:`17`
+ - r\ :sub:`16`
+ - r\ :sub:`15`
+ - r\ :sub:`14`
+ - r\ :sub:`13`
+ - r\ :sub:`12`
+ - r\ :sub:`11`
+ - r\ :sub:`10`
+ - r\ :sub:`9`
+ - r\ :sub:`8`
+ - r\ :sub:`7`
+ - r\ :sub:`6`
+ - r\ :sub:`5`
+ - r\ :sub:`4`
+ - r\ :sub:`3`
+ - r\ :sub:`2`
+ - r\ :sub:`1`
+ - r\ :sub:`0`
+ - g\ :sub:`19`
+ - g\ :sub:`18`
+ - g\ :sub:`17`
+ - g\ :sub:`16`
+ - g\ :sub:`15`
+ - g\ :sub:`14`
+ - g\ :sub:`13`
+ - g\ :sub:`12`
+ * -
+ -
+ -
+ - g\ :sub:`11`
+ - g\ :sub:`10`
+ - g\ :sub:`9`
+ - g\ :sub:`8`
+ - g\ :sub:`7`
+ - g\ :sub:`6`
+ - g\ :sub:`5`
+ - g\ :sub:`4`
+ - g\ :sub:`3`
+ - g\ :sub:`2`
+ - g\ :sub:`1`
+ - g\ :sub:`0`
+ - b\ :sub:`19`
+ - b\ :sub:`18`
+ - b\ :sub:`17`
+ - b\ :sub:`16`
+ - b\ :sub:`15`
+ - b\ :sub:`14`
+ - b\ :sub:`13`
+ - b\ :sub:`12`
+ - b\ :sub:`11`
+ - b\ :sub:`10`
+ - b\ :sub:`9`
+ - b\ :sub:`8`
+ - b\ :sub:`7`
+ - b\ :sub:`6`
+ - b\ :sub:`5`
+ - b\ :sub:`4`
+ - b\ :sub:`3`
+ - b\ :sub:`2`
+ - b\ :sub:`1`
+ - b\ :sub:`0`
+
+.. raw:: latex
+
+ \endgroup
+
On LVDS buses, usually each sample is transferred serialized in seven
time slots per pixel clock, on three (18-bit) or four (24-bit)
differential data pairs at the same time. The remaining bits are used
diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h
index d4c1d991014b..49be328d9a3b 100644
--- a/include/uapi/linux/media-bus-format.h
+++ b/include/uapi/linux/media-bus-format.h
@@ -34,7 +34,7 @@
#define MEDIA_BUS_FMT_FIXED 0x0001
-/* RGB - next is 0x1026 */
+/* RGB - next is 0x1027 */
#define MEDIA_BUS_FMT_RGB444_1X12 0x1016
#define MEDIA_BUS_FMT_RGB444_2X8_PADHI_BE 0x1001
#define MEDIA_BUS_FMT_RGB444_2X8_PADHI_LE 0x1002
@@ -72,6 +72,7 @@
#define MEDIA_BUS_FMT_RGB888_1X36_CPADLO 0x1021
#define MEDIA_BUS_FMT_RGB121212_1X36 0x1019
#define MEDIA_BUS_FMT_RGB161616_1X48 0x101a
+#define MEDIA_BUS_FMT_RGB202020_1X60 0x1026
/* YUV (including grey) - next is 0x202f */
#define MEDIA_BUS_FMT_Y8_1X8 0x2001
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 03/18] media: uapi: Add 20-bit bayer formats
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
2024-07-09 13:28 ` [PATCH v6 01/18] media: mc-entity: Record number of video devices in a pipeline Daniel Scally
2024-07-09 13:28 ` [PATCH v6 02/18] media: uapi: Add MEDIA_BUS_FMT_RGB202020_1X60 format code Daniel Scally
@ 2024-07-09 13:28 ` Daniel Scally
2024-07-22 22:13 ` Laurent Pinchart
2024-07-09 13:28 ` [PATCH v6 04/18] media: v4l2-common: Add RAW16 format info Daniel Scally
` (14 subsequent siblings)
17 siblings, 1 reply; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:28 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
The Mali-C55 requires input data be in 20-bit format, MSB aligned.
Add some new media bus format macros to represent that input format.
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- Fixed the table formatting
Changes in v5:
- New patch
.../media/v4l/subdev-formats.rst | 252 +++++++++++++++++-
include/uapi/linux/media-bus-format.h | 6 +-
2 files changed, 255 insertions(+), 3 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
index 5dbf8c9b18fb..ff328eb485e8 100644
--- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
+++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
@@ -2664,7 +2664,7 @@ organization is given as an example for the first pixel only.
\tiny
\setlength{\tabcolsep}{2pt}
-.. tabularcolumns:: |p{6.0cm}|p{0.7cm}|p{0.3cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|
+.. tabularcolumns:: |p{6.0cm}|p{0.7cm}|p{0.3cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|
.. _v4l2-mbus-pixelcode-bayer:
@@ -2677,10 +2677,14 @@ organization is given as an example for the first pixel only.
* - Identifier
- Code
-
- - :cspan:`15` Data organization
+ - :cspan:`19` Data organization
* -
-
- Bit
+ - 19
+ - 18
+ - 17
+ - 16
- 15
- 14
- 13
@@ -2710,6 +2714,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- b\ :sub:`7`
- b\ :sub:`6`
- b\ :sub:`5`
@@ -2731,6 +2739,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- g\ :sub:`7`
- g\ :sub:`6`
- g\ :sub:`5`
@@ -2752,6 +2764,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- g\ :sub:`7`
- g\ :sub:`6`
- g\ :sub:`5`
@@ -2773,6 +2789,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- r\ :sub:`7`
- r\ :sub:`6`
- r\ :sub:`5`
@@ -2794,6 +2814,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- b\ :sub:`7`
- b\ :sub:`6`
- b\ :sub:`5`
@@ -2815,6 +2839,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- g\ :sub:`7`
- g\ :sub:`6`
- g\ :sub:`5`
@@ -2836,6 +2864,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- g\ :sub:`7`
- g\ :sub:`6`
- g\ :sub:`5`
@@ -2857,6 +2889,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- r\ :sub:`7`
- r\ :sub:`6`
- r\ :sub:`5`
@@ -2878,6 +2914,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- b\ :sub:`7`
- b\ :sub:`6`
- b\ :sub:`5`
@@ -2899,6 +2939,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- g\ :sub:`7`
- g\ :sub:`6`
- g\ :sub:`5`
@@ -2920,6 +2964,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- g\ :sub:`7`
- g\ :sub:`6`
- g\ :sub:`5`
@@ -2941,6 +2989,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- r\ :sub:`7`
- r\ :sub:`6`
- r\ :sub:`5`
@@ -2962,6 +3014,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- 0
- 0
- 0
@@ -2981,6 +3037,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- b\ :sub:`7`
- b\ :sub:`6`
- b\ :sub:`5`
@@ -3002,6 +3062,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- b\ :sub:`7`
- b\ :sub:`6`
- b\ :sub:`5`
@@ -3021,6 +3085,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- 0
- 0
- 0
@@ -3042,6 +3110,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- b\ :sub:`9`
- b\ :sub:`8`
- b\ :sub:`7`
@@ -3061,6 +3133,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- b\ :sub:`1`
- b\ :sub:`0`
- 0
@@ -3082,6 +3158,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- b\ :sub:`1`
- b\ :sub:`0`
- 0
@@ -3101,6 +3181,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- b\ :sub:`9`
- b\ :sub:`8`
- b\ :sub:`7`
@@ -3120,6 +3204,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- b\ :sub:`9`
- b\ :sub:`8`
- b\ :sub:`7`
@@ -3141,6 +3229,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- g\ :sub:`9`
- g\ :sub:`8`
- g\ :sub:`7`
@@ -3162,6 +3254,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- g\ :sub:`9`
- g\ :sub:`8`
- g\ :sub:`7`
@@ -3183,6 +3279,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- r\ :sub:`9`
- r\ :sub:`8`
- r\ :sub:`7`
@@ -3202,6 +3302,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- b\ :sub:`11`
- b\ :sub:`10`
- b\ :sub:`9`
@@ -3223,6 +3327,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- g\ :sub:`11`
- g\ :sub:`10`
- g\ :sub:`9`
@@ -3244,6 +3352,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- g\ :sub:`11`
- g\ :sub:`10`
- g\ :sub:`9`
@@ -3265,6 +3377,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- r\ :sub:`11`
- r\ :sub:`10`
- r\ :sub:`9`
@@ -3284,6 +3400,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- b\ :sub:`13`
- b\ :sub:`12`
- b\ :sub:`11`
@@ -3305,6 +3425,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- g\ :sub:`13`
- g\ :sub:`12`
- g\ :sub:`11`
@@ -3326,6 +3450,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- g\ :sub:`13`
- g\ :sub:`12`
- g\ :sub:`11`
@@ -3347,6 +3475,10 @@ organization is given as an example for the first pixel only.
-
-
-
+ -
+ -
+ -
+ -
- r\ :sub:`13`
- r\ :sub:`12`
- r\ :sub:`11`
@@ -3366,6 +3498,10 @@ organization is given as an example for the first pixel only.
- MEDIA_BUS_FMT_SBGGR16_1X16
- 0x301d
-
+ -
+ -
+ -
+ -
- b\ :sub:`15`
- b\ :sub:`14`
- b\ :sub:`13`
@@ -3387,6 +3523,10 @@ organization is given as an example for the first pixel only.
- MEDIA_BUS_FMT_SGBRG16_1X16
- 0x301e
-
+ -
+ -
+ -
+ -
- g\ :sub:`15`
- g\ :sub:`14`
- g\ :sub:`13`
@@ -3408,6 +3548,10 @@ organization is given as an example for the first pixel only.
- MEDIA_BUS_FMT_SGRBG16_1X16
- 0x301f
-
+ -
+ -
+ -
+ -
- g\ :sub:`15`
- g\ :sub:`14`
- g\ :sub:`13`
@@ -3429,6 +3573,110 @@ organization is given as an example for the first pixel only.
- MEDIA_BUS_FMT_SRGGB16_1X16
- 0x3020
-
+ -
+ -
+ -
+ -
+ - r\ :sub:`15`
+ - r\ :sub:`14`
+ - r\ :sub:`13`
+ - r\ :sub:`12`
+ - r\ :sub:`11`
+ - r\ :sub:`10`
+ - r\ :sub:`9`
+ - r\ :sub:`8`
+ - r\ :sub:`7`
+ - r\ :sub:`6`
+ - r\ :sub:`5`
+ - r\ :sub:`4`
+ - r\ :sub:`3`
+ - r\ :sub:`2`
+ - r\ :sub:`1`
+ - r\ :sub:`0`
+ * .. _MEDIA-BUS-FMT-SBGGR20-1X20:
+
+ - MEDIA_BUS_FMT_SBGGR20_1X20
+ - 0x3021
+ -
+ - b\ :sub:`19`
+ - b\ :sub:`18`
+ - b\ :sub:`17`
+ - b\ :sub:`16`
+ - b\ :sub:`15`
+ - b\ :sub:`14`
+ - b\ :sub:`13`
+ - b\ :sub:`12`
+ - b\ :sub:`11`
+ - b\ :sub:`10`
+ - b\ :sub:`9`
+ - b\ :sub:`8`
+ - b\ :sub:`7`
+ - b\ :sub:`6`
+ - b\ :sub:`5`
+ - b\ :sub:`4`
+ - b\ :sub:`3`
+ - b\ :sub:`2`
+ - b\ :sub:`1`
+ - b\ :sub:`0`
+ * .. _MEDIA-BUS-FMT-SGBRG20-1X20:
+
+ - MEDIA_BUS_FMT_SGBRG20_1X20
+ - 0x3022
+ -
+ - g\ :sub:`19`
+ - g\ :sub:`18`
+ - g\ :sub:`17`
+ - g\ :sub:`16`
+ - g\ :sub:`15`
+ - g\ :sub:`14`
+ - g\ :sub:`13`
+ - g\ :sub:`12`
+ - g\ :sub:`11`
+ - g\ :sub:`10`
+ - g\ :sub:`9`
+ - g\ :sub:`8`
+ - g\ :sub:`7`
+ - g\ :sub:`6`
+ - g\ :sub:`5`
+ - g\ :sub:`4`
+ - g\ :sub:`3`
+ - g\ :sub:`2`
+ - g\ :sub:`1`
+ - g\ :sub:`0`
+ * .. _MEDIA-BUS-FMT-SGRBG20-1X20:
+
+ - MEDIA_BUS_FMT_SGRBG20_1X20
+ - 0x3023
+ -
+ - g\ :sub:`19`
+ - g\ :sub:`18`
+ - g\ :sub:`17`
+ - g\ :sub:`16`
+ - g\ :sub:`15`
+ - g\ :sub:`14`
+ - g\ :sub:`13`
+ - g\ :sub:`12`
+ - g\ :sub:`11`
+ - g\ :sub:`10`
+ - g\ :sub:`9`
+ - g\ :sub:`8`
+ - g\ :sub:`7`
+ - g\ :sub:`6`
+ - g\ :sub:`5`
+ - g\ :sub:`4`
+ - g\ :sub:`3`
+ - g\ :sub:`2`
+ - g\ :sub:`1`
+ - g\ :sub:`0`
+ * .. _MEDIA-BUS-FMT-SRGGB20-1X20:
+
+ - MEDIA_BUS_FMT_SRGGB20_1X20
+ - 0x3024
+ -
+ - r\ :sub:`19`
+ - r\ :sub:`18`
+ - r\ :sub:`17`
+ - r\ :sub:`16`
- r\ :sub:`15`
- r\ :sub:`14`
- r\ :sub:`13`
diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h
index 49be328d9a3b..b6acf8c8e383 100644
--- a/include/uapi/linux/media-bus-format.h
+++ b/include/uapi/linux/media-bus-format.h
@@ -122,7 +122,7 @@
#define MEDIA_BUS_FMT_YUV16_1X48 0x202a
#define MEDIA_BUS_FMT_UYYVYY16_0_5X48 0x202b
-/* Bayer - next is 0x3021 */
+/* Bayer - next is 0x3025 */
#define MEDIA_BUS_FMT_SBGGR8_1X8 0x3001
#define MEDIA_BUS_FMT_SGBRG8_1X8 0x3013
#define MEDIA_BUS_FMT_SGRBG8_1X8 0x3002
@@ -155,6 +155,10 @@
#define MEDIA_BUS_FMT_SGBRG16_1X16 0x301e
#define MEDIA_BUS_FMT_SGRBG16_1X16 0x301f
#define MEDIA_BUS_FMT_SRGGB16_1X16 0x3020
+#define MEDIA_BUS_FMT_SBGGR20_1X20 0x3021
+#define MEDIA_BUS_FMT_SGBRG20_1X20 0x3022
+#define MEDIA_BUS_FMT_SGRBG20_1X20 0x3023
+#define MEDIA_BUS_FMT_SRGGB20_1X20 0x3024
/* JPEG compressed formats - next is 0x4002 */
#define MEDIA_BUS_FMT_JPEG_1X8 0x4001
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 04/18] media: v4l2-common: Add RAW16 format info
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
` (2 preceding siblings ...)
2024-07-09 13:28 ` [PATCH v6 03/18] media: uapi: Add 20-bit bayer formats Daniel Scally
@ 2024-07-09 13:28 ` Daniel Scally
2024-07-09 13:28 ` [PATCH v6 05/18] media: v4l2-common: Add RAW14 " Daniel Scally
` (13 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:28 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
Add entries to v4l2_format_info describing the 16-bit bayer
formats so that they can be used in drivers.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- New patch
Changes in v5:
- New patch
drivers/media/v4l2-core/v4l2-common.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index 4165c815faef..c5d5704af5ee 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -331,6 +331,10 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
{ .format = V4L2_PIX_FMT_SGBRG12, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_SGRBG12, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_SRGGB12, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
+ { .format = V4L2_PIX_FMT_SBGGR16, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
+ { .format = V4L2_PIX_FMT_SGBRG16, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
+ { .format = V4L2_PIX_FMT_SGRBG16, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
+ { .format = V4L2_PIX_FMT_SRGGB16, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
};
unsigned int i;
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 05/18] media: v4l2-common: Add RAW14 format info
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
` (3 preceding siblings ...)
2024-07-09 13:28 ` [PATCH v6 04/18] media: v4l2-common: Add RAW16 format info Daniel Scally
@ 2024-07-09 13:28 ` Daniel Scally
2024-07-22 22:14 ` Laurent Pinchart
2024-07-09 13:28 ` [PATCH v6 06/18] dt-bindings: media: Add bindings for ARM mali-c55 Daniel Scally
` (12 subsequent siblings)
17 siblings, 1 reply; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:28 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
Add entries to v4l2_format_info describing the 14-bit bayer
formats so that they can be used in drivers.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- New patch
drivers/media/v4l2-core/v4l2-common.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index c5d5704af5ee..fe33221e6c02 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -331,6 +331,10 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
{ .format = V4L2_PIX_FMT_SGBRG12, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_SGRBG12, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_SRGGB12, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
+ { .format = V4L2_PIX_FMT_SBGGR14, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
+ { .format = V4L2_PIX_FMT_SGBRG14, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
+ { .format = V4L2_PIX_FMT_SGRBG14, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
+ { .format = V4L2_PIX_FMT_SRGGB14, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_SBGGR16, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_SGBRG16, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_SGRBG16, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 06/18] dt-bindings: media: Add bindings for ARM mali-c55
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
` (4 preceding siblings ...)
2024-07-09 13:28 ` [PATCH v6 05/18] media: v4l2-common: Add RAW14 " Daniel Scally
@ 2024-07-09 13:28 ` Daniel Scally
2024-07-09 13:28 ` [PATCH v6 07/18] media: mali-c55: Add Mali-C55 ISP driver Daniel Scally
` (11 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:28 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally,
Krzysztof Kozlowski
Add the yaml binding for ARM's Mali-C55 Image Signal Processor.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- None
Changes in v5:
- None
Changes in v4:
- Switched to port instead of ports
Changes in v3:
- Dropped the video clock as suggested by Laurent. I didn't retain it
for the purposes of the refcount since this driver will call .s_stream()
for the sensor driver which will refcount the clock anyway.
- Clarified that the port is a parallel input port rather (Sakari)
Changes in v2:
- Added clocks information
- Fixed the warnings raised by Rob
.../bindings/media/arm,mali-c55.yaml | 66 +++++++++++++++++++
1 file changed, 66 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/arm,mali-c55.yaml
diff --git a/Documentation/devicetree/bindings/media/arm,mali-c55.yaml b/Documentation/devicetree/bindings/media/arm,mali-c55.yaml
new file mode 100644
index 000000000000..9cc2481f2da3
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/arm,mali-c55.yaml
@@ -0,0 +1,66 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/arm,mali-c55.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: ARM Mali-C55 Image Signal Processor
+
+maintainers:
+ - Daniel Scally <dan.scally@ideasonboard.com>
+ - Jacopo Mondi <jacopo.mondi@ideasonboard.com>
+
+properties:
+ compatible:
+ const: arm,mali-c55
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ clocks:
+ items:
+ - description: ISP AXI clock
+ - description: ISP AHB-lite clock
+
+ clock-names:
+ items:
+ - const: aclk
+ - const: hclk
+
+ port:
+ $ref: /schemas/graph.yaml#/properties/port
+ description: Input parallel video bus
+
+ properties:
+ endpoint:
+ $ref: /schemas/graph.yaml#/properties/endpoint
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+ - clock-names
+ - port
+
+additionalProperties: false
+
+examples:
+ - |
+ mali_c55: isp@400000 {
+ compatible = "arm,mali-c55";
+ reg = <0x400000 0x200000>;
+ clocks = <&clk 0>, <&clk 1>;
+ clock-names = "aclk", "hclk";
+ interrupts = <0>;
+
+ port {
+ isp_in: endpoint {
+ remote-endpoint = <&csi2_rx_out>;
+ };
+ };
+ };
+...
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 07/18] media: mali-c55: Add Mali-C55 ISP driver
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
` (5 preceding siblings ...)
2024-07-09 13:28 ` [PATCH v6 06/18] dt-bindings: media: Add bindings for ARM mali-c55 Daniel Scally
@ 2024-07-09 13:28 ` Daniel Scally
2024-07-30 21:23 ` Laurent Pinchart
2024-07-09 13:28 ` [PATCH v6 08/18] media: Documentation: Add Mali-C55 ISP Documentation Daniel Scally
` (10 subsequent siblings)
17 siblings, 1 reply; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:28 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
Add a driver for Arm's Mali-C55 Image Signal Processor. The driver is
V4L2 and Media Controller compliant and creates subdevices to manage
the ISP itself, its internal test pattern generator as well as the
crop, scaler and output format functionality for each of its two
output devices.
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- Reverted the rework of mali_c55_update_bits() so it matches how the
regmap_update_bits() function works
- Global rename of "rzr" abbreviation for "resizer" to "rsz"
- Added separate functions to write to hardware and register buffer.
- Honoured custom strides from userspace
- Used .enable_streams(), .disable_streams() and
v4l2_subdev_[en|dis]able_streams() throughout
- Only call streamon for the sensor when all video devices included in
the pipeline through enabled links are started.
- Don't write registers until streamon() in the capture devices
- Commented mutex/spinlocks
- Moved v4l2 async notifier from isp.c to core.c
- Reconfigured hardware in pm_runtime_resume() callback to make sure
they are not lost on power-off
- A lot of other changes, which are mostly stylistic in nature. THe full
list is too large to include but I can send a diff between the version
if needed.
Changes in v5:
- Reworked input formats - previously we allowed representing input data
as any 8-16 bit format. Now we only allow input data to be represented
by the new 20-bit bayer formats, which is corrected to the equivalent
16-bit format in RAW bypass mode.
- Stopped bypassing blocks that we haven't added supporting parameters
for yet.
- Addressed most of Sakari's comments from the list
Changes not yet made in v5:
- The output pipelines can still be started and stopped independently of
one another - I'd like to discuss that more.
- the TPG subdev still uses .s_stream() - I need to rebase onto a tree
with working .enable_streams() for a single-source-pad subdevice.
Changes in v4:
- Reworked mali_c55_update_bits() to internally perform the bit-shift
- Reworked the resizer to allow cropping during streaming
- Fixed a bug in NV12 output
Changes in v3:
- Mostly minor fixes suggested by Sakari
- Fixed the sequencing of vb2 buffers to be synchronised across the two
capture devices.
Changes in v2:
- Clock handling
- Fixed the warnings raised by the kernel test robot
drivers/media/platform/Kconfig | 1 +
drivers/media/platform/Makefile | 1 +
drivers/media/platform/arm/Kconfig | 5 +
drivers/media/platform/arm/Makefile | 2 +
drivers/media/platform/arm/mali-c55/Kconfig | 17 +
drivers/media/platform/arm/mali-c55/Makefile | 9 +
.../platform/arm/mali-c55/mali-c55-capture.c | 961 +++++++++++++++
.../platform/arm/mali-c55/mali-c55-common.h | 247 ++++
.../platform/arm/mali-c55/mali-c55-core.c | 893 ++++++++++++++
.../platform/arm/mali-c55/mali-c55-isp.c | 531 ++++++++
.../arm/mali-c55/mali-c55-registers.h | 313 +++++
.../platform/arm/mali-c55/mali-c55-resizer.c | 1096 +++++++++++++++++
.../platform/arm/mali-c55/mali-c55-tpg.c | 438 +++++++
13 files changed, 4514 insertions(+)
create mode 100644 drivers/media/platform/arm/Kconfig
create mode 100644 drivers/media/platform/arm/Makefile
create mode 100644 drivers/media/platform/arm/mali-c55/Kconfig
create mode 100644 drivers/media/platform/arm/mali-c55/Makefile
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-capture.c
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-common.h
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-core.c
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-isp.c
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-registers.h
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-tpg.c
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 2d79bfc68c15..c929169766aa 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -65,6 +65,7 @@ config VIDEO_MUX
source "drivers/media/platform/allegro-dvt/Kconfig"
source "drivers/media/platform/amlogic/Kconfig"
source "drivers/media/platform/amphion/Kconfig"
+source "drivers/media/platform/arm/Kconfig"
source "drivers/media/platform/aspeed/Kconfig"
source "drivers/media/platform/atmel/Kconfig"
source "drivers/media/platform/broadcom/Kconfig"
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index da17301f7439..9a647abd5218 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -8,6 +8,7 @@
obj-y += allegro-dvt/
obj-y += amlogic/
obj-y += amphion/
+obj-y += arm/
obj-y += aspeed/
obj-y += atmel/
obj-y += broadcom/
diff --git a/drivers/media/platform/arm/Kconfig b/drivers/media/platform/arm/Kconfig
new file mode 100644
index 000000000000..4f0764c329c7
--- /dev/null
+++ b/drivers/media/platform/arm/Kconfig
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+comment "ARM media platform drivers"
+
+source "drivers/media/platform/arm/mali-c55/Kconfig"
diff --git a/drivers/media/platform/arm/Makefile b/drivers/media/platform/arm/Makefile
new file mode 100644
index 000000000000..8cc4918725ef
--- /dev/null
+++ b/drivers/media/platform/arm/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-y += mali-c55/
diff --git a/drivers/media/platform/arm/mali-c55/Kconfig b/drivers/media/platform/arm/mali-c55/Kconfig
new file mode 100644
index 000000000000..6ba70e765b8d
--- /dev/null
+++ b/drivers/media/platform/arm/mali-c55/Kconfig
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config VIDEO_MALI_C55
+ tristate "ARM Mali-C55 Image Signal Processor driver"
+ depends on ARCH_VEXPRESS || COMPILE_TEST
+ depends on V4L_PLATFORM_DRIVERS
+ depends on VIDEO_DEV && OF
+ select GENERIC_PHY_MIPI_DPHY
+ select MEDIA_CONTROLLER
+ select V4L2_FWNODE
+ select VIDEO_V4L2_SUBDEV_API
+ select VIDEOBUF2_DMA_CONTIG
+ select VIDEOBUF2_VMALLOC
+ help
+ Enable this to support Arm's Mali-C55 Image Signal Processor.
+
+ To compile this driver as a module, choose M here: the module
+ will be called mali-c55.
diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile
new file mode 100644
index 000000000000..9178ac35e50e
--- /dev/null
+++ b/drivers/media/platform/arm/mali-c55/Makefile
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0
+
+mali-c55-y := mali-c55-capture.o \
+ mali-c55-core.o \
+ mali-c55-isp.o \
+ mali-c55-resizer.o \
+ mali-c55-tpg.o
+
+obj-$(CONFIG_VIDEO_MALI_C55) += mali-c55.o
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-capture.c b/drivers/media/platform/arm/mali-c55/mali-c55-capture.c
new file mode 100644
index 000000000000..508245f06a09
--- /dev/null
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-capture.c
@@ -0,0 +1,961 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM Mali-C55 ISP Driver - Video capture devices
+ *
+ * Copyright (C) 2024 Ideas on Board Oy
+ */
+
+#include <linux/cleanup.h>
+#include <linux/minmax.h>
+#include <linux/pm_runtime.h>
+#include <linux/string.h>
+#include <linux/videodev2.h>
+
+#include <media/v4l2-dev.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mali-c55-common.h"
+#include "mali-c55-registers.h"
+
+static const struct mali_c55_format_info mali_c55_fmts[] = {
+ /*
+ * This table is missing some entries which need further work or
+ * investigation:
+ *
+ * Base mode 1 is a backwards V4L2_PIX_FMT_XRGB32 with no V4L2 equivalent
+ * Base mode 5 is "Generic Data"
+ * Base mode 8 is a backwards V4L2_PIX_FMT_XYUV32 - no V4L2 equivalent
+ * Base mode 9 seems to have no V4L2 equivalent
+ * Base mode 17, 19 and 20 describe formats which seem to have no V4L2
+ * equivalent
+ */
+ {
+ .fourcc = V4L2_PIX_FMT_ARGB2101010,
+ .mbus_codes = {
+ MEDIA_BUS_FMT_RGB121212_1X36,
+ MEDIA_BUS_FMT_RGB202020_1X60,
+ },
+ .is_raw = false,
+ .registers = {
+ .base_mode = MALI_C55_OUTPUT_A2R10G10B10,
+ .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
+ }
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_RGB565,
+ .mbus_codes = {
+ MEDIA_BUS_FMT_RGB121212_1X36,
+ MEDIA_BUS_FMT_RGB202020_1X60,
+ },
+ .is_raw = false,
+ .registers = {
+ .base_mode = MALI_C55_OUTPUT_RGB565,
+ .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
+ }
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_BGR24,
+ .mbus_codes = {
+ MEDIA_BUS_FMT_RGB121212_1X36,
+ MEDIA_BUS_FMT_RGB202020_1X60,
+ },
+ .is_raw = false,
+ .registers = {
+ .base_mode = MALI_C55_OUTPUT_RGB24,
+ .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
+ }
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_YUYV,
+ .mbus_codes = {
+ MEDIA_BUS_FMT_YUV10_1X30,
+ },
+ .is_raw = false,
+ .registers = {
+ .base_mode = MALI_C55_OUTPUT_YUY2,
+ .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
+ }
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_UYVY,
+ .mbus_codes = {
+ MEDIA_BUS_FMT_YUV10_1X30,
+ },
+ .is_raw = false,
+ .registers = {
+ .base_mode = MALI_C55_OUTPUT_UYVY,
+ .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
+ }
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_Y210,
+ .mbus_codes = {
+ MEDIA_BUS_FMT_YUV10_1X30,
+ },
+ .is_raw = false,
+ .registers = {
+ .base_mode = MALI_C55_OUTPUT_Y210,
+ .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
+ }
+ },
+ /*
+ * This is something of a hack, the ISP thinks it's running NV12M but
+ * by setting uv_plane = 0 we simply discard that planes and only output
+ * the Y-plane.
+ */
+ {
+ .fourcc = V4L2_PIX_FMT_GREY,
+ .mbus_codes = {
+ MEDIA_BUS_FMT_YUV10_1X30,
+ },
+ .is_raw = false,
+ .registers = {
+ .base_mode = MALI_C55_OUTPUT_NV12_21,
+ .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
+ }
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_NV12M,
+ .mbus_codes = {
+ MEDIA_BUS_FMT_YUV10_1X30,
+ },
+ .is_raw = false,
+ .registers = {
+ .base_mode = MALI_C55_OUTPUT_NV12_21,
+ .uv_plane = MALI_C55_OUTPUT_PLANE_ALT1
+ }
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_NV21M,
+ .mbus_codes = {
+ MEDIA_BUS_FMT_YUV10_1X30,
+ },
+ .is_raw = false,
+ .registers = {
+ .base_mode = MALI_C55_OUTPUT_NV12_21,
+ .uv_plane = MALI_C55_OUTPUT_PLANE_ALT2
+ }
+ },
+ /*
+ * RAW uncompressed formats are all packed in 16 bpp.
+ * TODO: Expand this list to encompass all possible RAW formats.
+ */
+ {
+ .fourcc = V4L2_PIX_FMT_SRGGB16,
+ .mbus_codes = {
+ MEDIA_BUS_FMT_SRGGB16_1X16,
+ },
+ .is_raw = true,
+ .registers = {
+ .base_mode = MALI_C55_OUTPUT_RAW16,
+ .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
+ }
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SBGGR16,
+ .mbus_codes = {
+ MEDIA_BUS_FMT_SBGGR16_1X16,
+ },
+ .is_raw = true,
+ .registers = {
+ .base_mode = MALI_C55_OUTPUT_RAW16,
+ .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
+ }
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGBRG16,
+ .mbus_codes = {
+ MEDIA_BUS_FMT_SGBRG16_1X16,
+ },
+ .is_raw = true,
+ .registers = {
+ .base_mode = MALI_C55_OUTPUT_RAW16,
+ .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
+ }
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGRBG16,
+ .mbus_codes = {
+ MEDIA_BUS_FMT_SGRBG16_1X16,
+ },
+ .is_raw = true,
+ .registers = {
+ .base_mode = MALI_C55_OUTPUT_RAW16,
+ .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
+ }
+ },
+};
+
+void mali_c55_cap_dev_write(struct mali_c55_cap_dev *cap_dev, unsigned int addr,
+ u32 val)
+{
+ mali_c55_ctx_write(cap_dev->mali_c55, addr + cap_dev->reg_offset, val);
+}
+
+static u32 mali_c55_cap_dev_read(struct mali_c55_cap_dev *cap_dev, unsigned int addr)
+{
+ return mali_c55_ctx_read(cap_dev->mali_c55, addr + cap_dev->reg_offset);
+}
+
+static void mali_c55_cap_dev_update_bits(struct mali_c55_cap_dev *cap_dev,
+ unsigned int addr, u32 mask, u32 val)
+{
+ u32 orig, tmp;
+
+ orig = mali_c55_cap_dev_read(cap_dev, addr);
+
+ tmp = orig & ~mask;
+ tmp |= val & mask;
+
+ if (tmp != orig)
+ mali_c55_cap_dev_write(cap_dev, addr, tmp);
+}
+
+static bool
+mali_c55_mbus_code_can_produce_fmt(const struct mali_c55_format_info *fmt,
+ u32 code)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(fmt->mbus_codes); i++) {
+ if (fmt->mbus_codes[i] == code)
+ return true;
+ }
+
+ return false;
+}
+
+bool mali_c55_format_is_raw(unsigned int mbus_code)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(mali_c55_fmts); i++) {
+ if (mali_c55_fmts[i].mbus_codes[0] == mbus_code)
+ return mali_c55_fmts[i].is_raw;
+ }
+
+ return false;
+}
+
+static const struct mali_c55_format_info *
+mali_c55_format_from_pix(const u32 pixelformat)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(mali_c55_fmts); i++) {
+ if (mali_c55_fmts[i].fourcc == pixelformat)
+ return &mali_c55_fmts[i];
+ }
+
+ /*
+ * If we find no matching pixelformat, we'll just default to the first
+ * one for now.
+ */
+
+ return &mali_c55_fmts[0];
+}
+
+static const char * const capture_device_names[] = {
+ "mali-c55 fr",
+ "mali-c55 ds",
+};
+
+static int mali_c55_link_validate(struct media_link *link)
+{
+ struct video_device *vdev =
+ media_entity_to_video_device(link->sink->entity);
+ struct mali_c55_cap_dev *cap_dev = video_get_drvdata(vdev);
+ struct v4l2_subdev *sd =
+ media_entity_to_v4l2_subdev(link->source->entity);
+ const struct v4l2_pix_format_mplane *pix_mp;
+ const struct mali_c55_format_info *cap_fmt;
+ struct v4l2_subdev_format sd_fmt = {
+ .which = V4L2_SUBDEV_FORMAT_ACTIVE,
+ .pad = link->source->index,
+ };
+ int ret;
+
+ ret = v4l2_subdev_call(sd, pad, get_fmt, NULL, &sd_fmt);
+ if (ret)
+ return ret;
+
+ pix_mp = &cap_dev->format.format;
+ cap_fmt = cap_dev->format.info;
+
+ if (sd_fmt.format.width != pix_mp->width ||
+ sd_fmt.format.height != pix_mp->height) {
+ dev_dbg(cap_dev->mali_c55->dev,
+ "link '%s':%u -> '%s':%u not valid: %ux%u != %ux%u\n",
+ link->source->entity->name, link->source->index,
+ link->sink->entity->name, link->sink->index,
+ sd_fmt.format.width, sd_fmt.format.height,
+ pix_mp->width, pix_mp->height);
+ return -EPIPE;
+ }
+
+ if (!mali_c55_mbus_code_can_produce_fmt(cap_fmt, sd_fmt.format.code)) {
+ dev_dbg(cap_dev->mali_c55->dev,
+ "link '%s':%u -> '%s':%u not valid: mbus_code 0x%04x cannot produce pixel format %p4cc\n",
+ link->source->entity->name, link->source->index,
+ link->sink->entity->name, link->sink->index,
+ sd_fmt.format.code, &pix_mp->pixelformat);
+ return -EPIPE;
+ }
+
+ return 0;
+}
+
+static const struct media_entity_operations mali_c55_media_ops = {
+ .link_validate = mali_c55_link_validate,
+};
+
+static int mali_c55_vb2_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
+ unsigned int *num_planes, unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct mali_c55_cap_dev *cap_dev = q->drv_priv;
+ unsigned int i;
+
+ if (*num_planes) {
+ if (*num_planes != cap_dev->format.format.num_planes)
+ return -EINVAL;
+
+ for (i = 0; i < cap_dev->format.format.num_planes; i++)
+ if (sizes[i] < cap_dev->format.format.plane_fmt[i].sizeimage)
+ return -EINVAL;
+ } else {
+ *num_planes = cap_dev->format.format.num_planes;
+ for (i = 0; i < cap_dev->format.format.num_planes; i++)
+ sizes[i] = cap_dev->format.format.plane_fmt[i].sizeimage;
+ }
+
+ return 0;
+}
+
+static void mali_c55_buf_queue(struct vb2_buffer *vb)
+{
+ struct mali_c55_cap_dev *cap_dev = vb2_get_drv_priv(vb->vb2_queue);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct mali_c55_buffer *buf = container_of(vbuf,
+ struct mali_c55_buffer, vb);
+ unsigned int i;
+
+ buf->planes_pending = cap_dev->format.format.num_planes;
+
+ for (i = 0; i < cap_dev->format.format.num_planes; i++) {
+ unsigned long size = cap_dev->format.format.plane_fmt[i].sizeimage;
+
+ vb2_set_plane_payload(vb, i, size);
+ }
+
+ buf->vb.field = V4L2_FIELD_NONE;
+
+ spin_lock(&cap_dev->buffers.lock);
+ list_add_tail(&buf->queue, &cap_dev->buffers.queue);
+ spin_unlock(&cap_dev->buffers.lock);
+}
+
+static int mali_c55_buf_init(struct vb2_buffer *vb)
+{
+ struct mali_c55_cap_dev *cap_dev = vb2_get_drv_priv(vb->vb2_queue);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct mali_c55_buffer *buf = container_of(vbuf,
+ struct mali_c55_buffer, vb);
+ unsigned int i;
+
+ for (i = 0; i < cap_dev->format.format.num_planes; i++)
+ buf->addrs[i] = vb2_dma_contig_plane_dma_addr(vb, i);
+
+ return 0;
+}
+
+void mali_c55_set_next_buffer(struct mali_c55_cap_dev *cap_dev)
+{
+ guard(spinlock)(&cap_dev->buffers.lock);
+
+ cap_dev->buffers.curr = cap_dev->buffers.next;
+ cap_dev->buffers.next = NULL;
+
+ if (!list_empty(&cap_dev->buffers.queue)) {
+ struct v4l2_pix_format_mplane *pix_mp;
+ const struct v4l2_format_info *info;
+ dma_addr_t *addrs;
+
+ pix_mp = &cap_dev->format.format;
+ info = v4l2_format_info(pix_mp->pixelformat);
+
+ mali_c55_cap_dev_update_bits(cap_dev,
+ MALI_C55_REG_Y_WRITER_MODE,
+ MALI_C55_WRITER_FRAME_WRITE_MASK,
+ MALI_C55_WRITER_FRAME_WRITE_ENABLE);
+ if (cap_dev->format.info->registers.uv_plane)
+ mali_c55_cap_dev_update_bits(cap_dev,
+ MALI_C55_REG_UV_WRITER_MODE,
+ MALI_C55_WRITER_FRAME_WRITE_MASK,
+ MALI_C55_WRITER_FRAME_WRITE_ENABLE);
+
+ cap_dev->buffers.next = list_first_entry(&cap_dev->buffers.queue,
+ struct mali_c55_buffer,
+ queue);
+ list_del(&cap_dev->buffers.next->queue);
+
+ addrs = cap_dev->buffers.next->addrs;
+ mali_c55_cap_dev_write(cap_dev,
+ MALI_C55_REG_Y_WRITER_BANKS_BASE,
+ addrs[MALI_C55_PLANE_Y]);
+ mali_c55_cap_dev_write(cap_dev,
+ MALI_C55_REG_UV_WRITER_BANKS_BASE,
+ addrs[MALI_C55_PLANE_UV]);
+ mali_c55_cap_dev_write(cap_dev,
+ MALI_C55_REG_Y_WRITER_OFFSET,
+ pix_mp->width * info->bpp[MALI_C55_PLANE_Y]);
+ mali_c55_cap_dev_write(cap_dev,
+ MALI_C55_REG_UV_WRITER_OFFSET,
+ pix_mp->width * info->bpp[MALI_C55_PLANE_UV]
+ / info->hdiv);
+ } else {
+ /*
+ * If we underflow then we can tell the ISP that we don't want
+ * to write out the next frame.
+ */
+ mali_c55_cap_dev_update_bits(cap_dev,
+ MALI_C55_REG_Y_WRITER_MODE,
+ MALI_C55_WRITER_FRAME_WRITE_MASK,
+ 0x00);
+ mali_c55_cap_dev_update_bits(cap_dev,
+ MALI_C55_REG_UV_WRITER_MODE,
+ MALI_C55_WRITER_FRAME_WRITE_MASK,
+ 0x00);
+ }
+}
+
+static void mali_c55_handle_buffer(struct mali_c55_buffer *curr_buf,
+ unsigned int framecount)
+{
+ curr_buf->vb.vb2_buf.timestamp = ktime_get_boottime_ns();
+ curr_buf->vb.sequence = framecount;
+ vb2_buffer_done(&curr_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
+}
+
+/**
+ * mali_c55_set_plane_done - mark the plane as written and process the buffer if
+ * both planes are finished.
+ * @cap_dev: pointer to the fr or ds pipe output
+ * @plane: the plane to mark as completed
+ *
+ * The Mali C55 ISP has muliplanar outputs for some formats that come with two
+ * separate "buffer write completed" interrupts - we need to flag each plane's
+ * completion and check whether both planes are done - if so, complete the buf
+ * in vb2.
+ */
+void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
+ enum mali_c55_planes plane)
+{
+ struct mali_c55_isp *isp = &cap_dev->mali_c55->isp;
+ struct mali_c55_buffer *curr_buf;
+
+ guard(spinlock)(&cap_dev->buffers.lock);
+ curr_buf = cap_dev->buffers.curr;
+
+ /*
+ * This _should_ never happen. If no buffer was available from vb2 then
+ * we tell the ISP not to bother writing the next frame, which means the
+ * interrupts that call this function should never trigger. If it does
+ * happen then one of our assumptions is horribly wrong - complain
+ * loudly and do nothing.
+ */
+ if (!curr_buf) {
+ dev_err(cap_dev->mali_c55->dev, "%s null buffer in %s()\n",
+ cap_dev->vdev.name, __func__);
+ return;
+ }
+
+ /* If the other plane is also done... */
+ if (!--curr_buf->planes_pending) {
+ mali_c55_handle_buffer(curr_buf, isp->frame_sequence);
+ cap_dev->buffers.curr = NULL;
+ isp->frame_sequence++;
+ }
+}
+
+static void mali_c55_cap_dev_stream_disable(struct mali_c55_cap_dev *cap_dev)
+{
+ mali_c55_cap_dev_update_bits(cap_dev, MALI_C55_REG_Y_WRITER_MODE,
+ MALI_C55_WRITER_FRAME_WRITE_MASK, 0x00);
+ mali_c55_cap_dev_update_bits(cap_dev, MALI_C55_REG_UV_WRITER_MODE,
+ MALI_C55_WRITER_FRAME_WRITE_MASK, 0x00);
+}
+
+static void mali_c55_cap_dev_stream_enable(struct mali_c55_cap_dev *cap_dev)
+{
+ /*
+ * The Mali ISP can hold up to 5 buffer addresses and simply cycle
+ * through them, but it's not clear to me that the vb2 queue _guarantees_
+ * it will queue buffers to the driver in a fixed order, and ensuring
+ * we call vb2_buffer_done() for the right buffer seems to me to add
+ * pointless complexity given in multi-context mode we'd need to
+ * re-write those registers every frame anyway...so we tell the ISP to
+ * use a single register and update it for each frame.
+ */
+ mali_c55_cap_dev_update_bits(cap_dev,
+ MALI_C55_REG_Y_WRITER_BANKS_CONFIG,
+ MALI_C55_REG_Y_WRITER_MAX_BANKS_MASK, 0);
+ mali_c55_cap_dev_update_bits(cap_dev,
+ MALI_C55_REG_UV_WRITER_BANKS_CONFIG,
+ MALI_C55_REG_UV_WRITER_MAX_BANKS_MASK, 0);
+
+ mali_c55_set_next_buffer(cap_dev);
+}
+
+static void mali_c55_cap_dev_return_buffers(struct mali_c55_cap_dev *cap_dev,
+ enum vb2_buffer_state state)
+{
+ struct mali_c55_buffer *buf, *tmp;
+
+ guard(spinlock)(&cap_dev->buffers.lock);
+
+ if (cap_dev->buffers.curr) {
+ vb2_buffer_done(&cap_dev->buffers.curr->vb.vb2_buf,
+ state);
+ cap_dev->buffers.curr = NULL;
+ }
+
+ if (cap_dev->buffers.next) {
+ vb2_buffer_done(&cap_dev->buffers.next->vb.vb2_buf,
+ state);
+ cap_dev->buffers.next = NULL;
+ }
+
+ list_for_each_entry_safe(buf, tmp, &cap_dev->buffers.queue, queue) {
+ list_del(&buf->queue);
+ vb2_buffer_done(&buf->vb.vb2_buf, state);
+ }
+}
+
+static void mali_c55_cap_dev_format_configure(struct mali_c55_cap_dev *cap_dev)
+{
+ const struct mali_c55_format_info *capture_format = cap_dev->format.info;
+ struct v4l2_pix_format_mplane *pix_mp = &cap_dev->format.format;
+ const struct v4l2_format_info *info;
+
+ info = v4l2_format_info(pix_mp->pixelformat);
+ if (WARN_ON(!info))
+ return;
+
+ mali_c55_cap_dev_write(cap_dev, MALI_C55_REG_Y_WRITER_MODE,
+ capture_format->registers.base_mode);
+ mali_c55_cap_dev_write(cap_dev, MALI_C55_REG_ACTIVE_OUT_Y_SIZE,
+ MALI_C55_REG_ACTIVE_OUT_SIZE_W(pix_mp->width) |
+ MALI_C55_REG_ACTIVE_OUT_SIZE_H(pix_mp->height));
+
+ if (info->mem_planes > 1) {
+ mali_c55_cap_dev_write(cap_dev, MALI_C55_REG_UV_WRITER_MODE,
+ capture_format->registers.base_mode);
+ mali_c55_cap_dev_update_bits(cap_dev,
+ MALI_C55_REG_UV_WRITER_MODE,
+ MALI_C55_WRITER_SUBMODE_MASK,
+ MALI_C55_WRITER_SUBMODE(capture_format->registers.uv_plane));
+
+ mali_c55_cap_dev_write(cap_dev, MALI_C55_REG_ACTIVE_OUT_UV_SIZE,
+ MALI_C55_REG_ACTIVE_OUT_SIZE_W(pix_mp->width) |
+ MALI_C55_REG_ACTIVE_OUT_SIZE_H(pix_mp->height));
+ }
+
+ if (info->pixel_enc == V4L2_PIXEL_ENC_YUV) {
+ mali_c55_cap_dev_write(cap_dev, MALI_C55_REG_CS_CONV_CONFIG,
+ MALI_C55_CS_CONV_MATRIX_MASK);
+
+ if (info->hdiv > 1)
+ mali_c55_cap_dev_update_bits(cap_dev,
+ MALI_C55_REG_CS_CONV_CONFIG,
+ MALI_C55_CS_CONV_HORZ_DOWNSAMPLE_MASK,
+ MALI_C55_CS_CONV_HORZ_DOWNSAMPLE_ENABLE);
+ if (info->vdiv > 1)
+ mali_c55_cap_dev_update_bits(cap_dev,
+ MALI_C55_REG_CS_CONV_CONFIG,
+ MALI_C55_CS_CONV_VERT_DOWNSAMPLE_MASK,
+ MALI_C55_CS_CONV_VERT_DOWNSAMPLE_ENABLE);
+ if (info->hdiv > 1 || info->vdiv > 1)
+ mali_c55_cap_dev_update_bits(cap_dev,
+ MALI_C55_REG_CS_CONV_CONFIG,
+ MALI_C55_CS_CONV_FILTER_MASK,
+ MALI_C55_CS_CONV_FILTER_ENABLE);
+ }
+}
+
+static int mali_c55_vb2_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+ struct mali_c55_cap_dev *cap_dev = q->drv_priv;
+ struct mali_c55 *mali_c55 = cap_dev->mali_c55;
+ struct mali_c55_resizer *rsz = cap_dev->rsz;
+ struct mali_c55_isp *isp = &mali_c55->isp;
+ int ret;
+
+ guard(mutex)(&isp->capture_lock);
+
+ ret = pm_runtime_resume_and_get(mali_c55->dev);
+ if (ret)
+ return ret;
+
+ mali_c55_cap_dev_format_configure(cap_dev);
+
+ ret = video_device_pipeline_start(&cap_dev->vdev,
+ &cap_dev->mali_c55->pipe);
+ if (ret) {
+ dev_dbg(mali_c55->dev, "%s failed to start media pipeline\n",
+ cap_dev->vdev.name);
+ goto err_pm_put;
+ }
+
+ mali_c55_cap_dev_stream_enable(cap_dev);
+
+ ret = v4l2_subdev_enable_streams(&rsz->sd, MALI_C55_RSZ_SOURCE_PAD,
+ BIT(0));
+ if (ret)
+ goto err_disable_cap_dev;
+
+ if (mali_c55->pipe.start_count == mali_c55->pipe.required_count) {
+ /*
+ * The ISP has a "processed" and "bypass" source pad, but they
+ * don't have a separate start process so we'll just start the
+ * processed pad unconditionally.
+ */
+ ret = v4l2_subdev_enable_streams(&mali_c55->isp.sd,
+ MALI_C55_ISP_PAD_SOURCE_VIDEO,
+ BIT(0));
+ if (ret)
+ goto err_disable_rsz;
+ }
+
+ return 0;
+
+err_disable_rsz:
+ v4l2_subdev_disable_streams(&rsz->sd, MALI_C55_RSZ_SOURCE_PAD, BIT(0));
+err_disable_cap_dev:
+ mali_c55_cap_dev_stream_disable(cap_dev);
+ video_device_pipeline_stop(&cap_dev->vdev);
+err_pm_put:
+ pm_runtime_put(mali_c55->dev);
+ mali_c55_cap_dev_return_buffers(cap_dev, VB2_BUF_STATE_QUEUED);
+
+ return ret;
+}
+
+static void mali_c55_vb2_stop_streaming(struct vb2_queue *q)
+{
+ struct mali_c55_cap_dev *cap_dev = q->drv_priv;
+ struct mali_c55 *mali_c55 = cap_dev->mali_c55;
+ struct mali_c55_resizer *rsz = cap_dev->rsz;
+ struct mali_c55_isp *isp = &mali_c55->isp;
+
+ guard(mutex)(&isp->capture_lock);
+
+ v4l2_subdev_disable_streams(&mali_c55->isp.sd,
+ MALI_C55_ISP_PAD_SOURCE_VIDEO, BIT(0));
+ v4l2_subdev_disable_streams(&rsz->sd, MALI_C55_RSZ_SOURCE_PAD, BIT(0));
+ mali_c55_cap_dev_stream_disable(cap_dev);
+ mali_c55_cap_dev_return_buffers(cap_dev, VB2_BUF_STATE_ERROR);
+ video_device_pipeline_stop(&cap_dev->vdev);
+ pm_runtime_put_autosuspend(mali_c55->dev);
+}
+
+static const struct vb2_ops mali_c55_vb2_ops = {
+ .queue_setup = &mali_c55_vb2_queue_setup,
+ .buf_queue = &mali_c55_buf_queue,
+ .buf_init = &mali_c55_buf_init,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+ .start_streaming = &mali_c55_vb2_start_streaming,
+ .stop_streaming = &mali_c55_vb2_stop_streaming,
+};
+
+static const struct v4l2_file_operations mali_c55_v4l2_fops = {
+ .owner = THIS_MODULE,
+ .unlocked_ioctl = video_ioctl2,
+ .open = v4l2_fh_open,
+ .release = vb2_fop_release,
+ .poll = vb2_fop_poll,
+ .mmap = vb2_fop_mmap,
+};
+
+static void mali_c55_try_fmt(struct v4l2_pix_format_mplane *pix_mp)
+{
+ const struct mali_c55_format_info *capture_format;
+ const struct v4l2_format_info *info;
+ struct v4l2_plane_pix_format *plane, *y_plane;
+ unsigned int padding;
+ unsigned int i;
+
+ capture_format = mali_c55_format_from_pix(pix_mp->pixelformat);
+ pix_mp->pixelformat = capture_format->fourcc;
+
+ pix_mp->width = clamp(pix_mp->width, MALI_C55_MIN_WIDTH,
+ MALI_C55_MAX_WIDTH);
+ pix_mp->height = clamp(pix_mp->height, MALI_C55_MIN_HEIGHT,
+ MALI_C55_MAX_HEIGHT);
+
+ pix_mp->field = V4L2_FIELD_NONE;
+ pix_mp->colorspace = V4L2_COLORSPACE_DEFAULT;
+ pix_mp->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ pix_mp->quantization = V4L2_QUANTIZATION_DEFAULT;
+
+ info = v4l2_format_info(pix_mp->pixelformat);
+ pix_mp->num_planes = info->mem_planes;
+ memset(pix_mp->plane_fmt, 0, sizeof(pix_mp->plane_fmt));
+
+ y_plane = &pix_mp->plane_fmt[0];
+ y_plane->bytesperline = clamp(y_plane->bytesperline,
+ info->bpp[0] * pix_mp->width, 65535U);
+ y_plane->sizeimage = y_plane->bytesperline * pix_mp->height;
+
+ padding = y_plane->bytesperline - (pix_mp->width * info->bpp[0]);
+
+ for (i = 1; i < info->comp_planes; i++) {
+ plane = &pix_mp->plane_fmt[i];
+
+ plane->bytesperline = DIV_ROUND_UP(info->bpp[i] * pix_mp->width,
+ info->hdiv) + padding;
+ plane->sizeimage = DIV_ROUND_UP(
+ plane->bytesperline * pix_mp->height,
+ info->vdiv);
+ }
+
+ if (info->mem_planes == 1) {
+ for (i = 1; i < info->comp_planes; i++) {
+ plane = &pix_mp->plane_fmt[i];
+ y_plane->sizeimage += plane->sizeimage;
+ }
+ }
+}
+
+static int mali_c55_try_fmt_vid_cap_mplane(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ mali_c55_try_fmt(&f->fmt.pix_mp);
+
+ return 0;
+}
+
+static void mali_c55_set_format(struct mali_c55_cap_dev *cap_dev,
+ struct v4l2_pix_format_mplane *pix_mp)
+{
+ mali_c55_try_fmt(pix_mp);
+
+ cap_dev->format.format = *pix_mp;
+ cap_dev->format.info = mali_c55_format_from_pix(pix_mp->pixelformat);
+}
+
+static int mali_c55_s_fmt_vid_cap_mplane(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct mali_c55_cap_dev *cap_dev = video_drvdata(file);
+
+ if (vb2_is_busy(&cap_dev->queue))
+ return -EBUSY;
+
+ mali_c55_set_format(cap_dev, &f->fmt.pix_mp);
+
+ return 0;
+}
+
+static int mali_c55_g_fmt_vid_cap_mplane(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct mali_c55_cap_dev *cap_dev = video_drvdata(file);
+
+ f->fmt.pix_mp = cap_dev->format.format;
+
+ return 0;
+}
+
+static int mali_c55_enum_fmt_vid_cap_mplane(struct file *file, void *fh,
+ struct v4l2_fmtdesc *f)
+{
+ struct mali_c55_cap_dev *cap_dev = video_drvdata(file);
+ unsigned int j = 0;
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(mali_c55_fmts); i++) {
+ if (f->mbus_code &&
+ !mali_c55_mbus_code_can_produce_fmt(&mali_c55_fmts[i],
+ f->mbus_code))
+ continue;
+
+ /* Downscale pipe can't output RAW formats */
+ if (mali_c55_fmts[i].is_raw &&
+ cap_dev->reg_offset == MALI_C55_CAP_DEV_DS_REG_OFFSET)
+ continue;
+
+ if (j++ == f->index) {
+ f->pixelformat = mali_c55_fmts[i].fourcc;
+ return 0;
+ }
+ }
+
+ return -EINVAL;
+}
+
+static int mali_c55_querycap(struct file *file, void *fh,
+ struct v4l2_capability *cap)
+{
+ strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver));
+ strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card));
+
+ return 0;
+}
+
+static const struct v4l2_ioctl_ops mali_c55_v4l2_ioctl_ops = {
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+ .vidioc_try_fmt_vid_cap_mplane = mali_c55_try_fmt_vid_cap_mplane,
+ .vidioc_s_fmt_vid_cap_mplane = mali_c55_s_fmt_vid_cap_mplane,
+ .vidioc_g_fmt_vid_cap_mplane = mali_c55_g_fmt_vid_cap_mplane,
+ .vidioc_enum_fmt_vid_cap = mali_c55_enum_fmt_vid_cap_mplane,
+ .vidioc_querycap = mali_c55_querycap,
+ .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static int mali_c55_register_cap_dev(struct mali_c55 *mali_c55,
+ enum mali_c55_cap_devs cap_dev_id)
+{
+ struct mali_c55_cap_dev *cap_dev = &mali_c55->cap_devs[cap_dev_id];
+ struct v4l2_pix_format_mplane pix_mp;
+ struct video_device *vdev;
+ struct vb2_queue *vb2q;
+ int ret;
+
+ vdev = &cap_dev->vdev;
+ vb2q = &cap_dev->queue;
+
+ cap_dev->mali_c55 = mali_c55;
+ mutex_init(&cap_dev->lock);
+ INIT_LIST_HEAD(&cap_dev->buffers.queue);
+
+ switch (cap_dev_id) {
+ case MALI_C55_CAP_DEV_FR:
+ cap_dev->rsz = &mali_c55->resizers[MALI_C55_RSZ_FR];
+ cap_dev->reg_offset = MALI_C55_CAP_DEV_FR_REG_OFFSET;
+ break;
+ case MALI_C55_CAP_DEV_DS:
+ cap_dev->rsz = &mali_c55->resizers[MALI_C55_RSZ_DS];
+ cap_dev->reg_offset = MALI_C55_CAP_DEV_DS_REG_OFFSET;
+ break;
+ default:
+ ret = -EINVAL;
+ goto err_destroy_mutex;
+ }
+
+ cap_dev->pad.flags = MEDIA_PAD_FL_SINK;
+ ret = media_entity_pads_init(&cap_dev->vdev.entity, 1, &cap_dev->pad);
+ if (ret) {
+ mutex_destroy(&cap_dev->lock);
+ goto err_destroy_mutex;
+ }
+
+ vb2q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ vb2q->io_modes = VB2_MMAP | VB2_DMABUF;
+ vb2q->drv_priv = cap_dev;
+ vb2q->mem_ops = &vb2_dma_contig_memops;
+ vb2q->ops = &mali_c55_vb2_ops;
+ vb2q->buf_struct_size = sizeof(struct mali_c55_buffer);
+ vb2q->min_queued_buffers = 1;
+ vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+ vb2q->lock = &cap_dev->lock;
+ vb2q->dev = mali_c55->dev;
+
+ ret = vb2_queue_init(vb2q);
+ if (ret) {
+ dev_err(mali_c55->dev, "%s vb2 queue init failed\n",
+ cap_dev->vdev.name);
+ goto err_cleanup_media_entity;
+ }
+
+ strscpy(cap_dev->vdev.name, capture_device_names[cap_dev_id],
+ sizeof(cap_dev->vdev.name));
+ vdev->release = video_device_release_empty;
+ vdev->fops = &mali_c55_v4l2_fops;
+ vdev->ioctl_ops = &mali_c55_v4l2_ioctl_ops;
+ vdev->lock = &cap_dev->lock;
+ vdev->v4l2_dev = &mali_c55->v4l2_dev;
+ vdev->queue = &cap_dev->queue;
+ vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE_MPLANE |
+ V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
+ vdev->entity.ops = &mali_c55_media_ops;
+ video_set_drvdata(vdev, cap_dev);
+
+ memset(&pix_mp, 0, sizeof(pix_mp));
+ pix_mp.pixelformat = V4L2_PIX_FMT_RGB565;
+ pix_mp.width = MALI_C55_DEFAULT_WIDTH;
+ pix_mp.height = MALI_C55_DEFAULT_HEIGHT;
+ mali_c55_set_format(cap_dev, &pix_mp);
+
+ ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+ if (ret) {
+ dev_err(mali_c55->dev,
+ "%s failed to register video device\n",
+ cap_dev->vdev.name);
+ goto err_release_vb2q;
+ }
+
+ return 0;
+
+err_release_vb2q:
+ vb2_queue_release(vb2q);
+err_cleanup_media_entity:
+ media_entity_cleanup(&cap_dev->vdev.entity);
+err_destroy_mutex:
+ mutex_destroy(&cap_dev->lock);
+
+ return ret;
+}
+
+int mali_c55_register_capture_devs(struct mali_c55 *mali_c55)
+{
+ int ret;
+
+ ret = mali_c55_register_cap_dev(mali_c55, MALI_C55_CAP_DEV_FR);
+ if (ret)
+ return ret;
+
+ if (mali_c55->capabilities & MALI_C55_GPS_DS_PIPE_FITTED) {
+ ret = mali_c55_register_cap_dev(mali_c55, MALI_C55_CAP_DEV_DS);
+ if (ret) {
+ mali_c55_unregister_capture_devs(mali_c55);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static void mali_c55_unregister_cap_dev(struct mali_c55 *mali_c55,
+ enum mali_c55_cap_devs cap_dev_id)
+{
+ struct mali_c55_cap_dev *cap_dev = &mali_c55->cap_devs[cap_dev_id];
+
+ if (!video_is_registered(&cap_dev->vdev))
+ return;
+
+ vb2_video_unregister_device(&cap_dev->vdev);
+ media_entity_cleanup(&cap_dev->vdev.entity);
+ mutex_destroy(&cap_dev->lock);
+}
+
+void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55)
+{
+ mali_c55_unregister_cap_dev(mali_c55, MALI_C55_CAP_DEV_FR);
+ if (mali_c55->capabilities & MALI_C55_GPS_DS_PIPE_FITTED)
+ mali_c55_unregister_cap_dev(mali_c55, MALI_C55_CAP_DEV_DS);
+}
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
new file mode 100644
index 000000000000..f7764a938e9f
--- /dev/null
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
@@ -0,0 +1,247 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * ARM Mali-C55 ISP Driver - Common definitions
+ *
+ * Copyright (C) 2024 Ideas on Board Oy
+ */
+
+#ifndef _MALI_C55_COMMON_H
+#define _MALI_C55_COMMON_H
+
+#include <linux/clk.h>
+#include <linux/io.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+
+#include <media/media-device.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-dev.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#define MALI_C55_DRIVER_NAME "mali-c55"
+
+/* min and max values for the image sizes */
+#define MALI_C55_MIN_WIDTH 640U
+#define MALI_C55_MIN_HEIGHT 480U
+#define MALI_C55_MAX_WIDTH 8192U
+#define MALI_C55_MAX_HEIGHT 8192U
+#define MALI_C55_DEFAULT_WIDTH 1920U
+#define MALI_C55_DEFAULT_HEIGHT 1080U
+
+#define MALI_C55_DEFAULT_MEDIA_BUS_FMT MEDIA_BUS_FMT_RGB121212_1X36
+
+#define MALI_C55_NUM_CLKS 2
+
+struct device;
+struct dma_chan;
+struct mali_c55;
+struct mali_c55_cap_dev;
+struct platform_device;
+struct resource;
+
+enum mali_c55_isp_pads {
+ MALI_C55_ISP_PAD_SINK_VIDEO,
+ MALI_C55_ISP_PAD_SOURCE_VIDEO,
+ MALI_C55_ISP_PAD_SOURCE_BYPASS,
+ MALI_C55_ISP_NUM_PADS,
+};
+
+struct mali_c55_tpg {
+ struct mali_c55 *mali_c55;
+ struct v4l2_subdev sd;
+ struct media_pad pad;
+ struct mali_c55_tpg_ctrls {
+ struct v4l2_ctrl_handler handler;
+ struct v4l2_ctrl *vblank;
+ } ctrls;
+};
+
+struct mali_c55_isp {
+ struct mali_c55 *mali_c55;
+ struct v4l2_subdev sd;
+ struct media_pad pads[MALI_C55_ISP_NUM_PADS];
+ struct media_pad *remote_src;
+ /* Mutex to guard vb2 start/stop streaming */
+ struct mutex capture_lock;
+ unsigned int frame_sequence;
+};
+
+enum mali_c55_resizer_ids {
+ MALI_C55_RSZ_FR,
+ MALI_C55_RSZ_DS,
+ MALI_C55_NUM_RSZS,
+};
+
+enum mali_c55_rsz_pads {
+ MALI_C55_RSZ_SINK_PAD,
+ MALI_C55_RSZ_SOURCE_PAD,
+ MALI_C55_RSZ_SINK_BYPASS_PAD,
+ MALI_C55_RSZ_NUM_PADS
+};
+
+struct mali_c55_resizer {
+ struct mali_c55 *mali_c55;
+ struct mali_c55_cap_dev *cap_dev;
+ enum mali_c55_resizer_ids id;
+ struct v4l2_subdev sd;
+ struct media_pad pads[MALI_C55_RSZ_NUM_PADS];
+ unsigned int num_routes;
+ bool streaming;
+};
+
+enum mali_c55_cap_devs {
+ MALI_C55_CAP_DEV_FR,
+ MALI_C55_CAP_DEV_DS,
+ MALI_C55_NUM_CAP_DEVS
+};
+
+struct mali_c55_format_info {
+ u32 fourcc;
+ /*
+ * The output formats can be produced by a couple of different media bus
+ * formats, depending on how the ISP is configured.
+ */
+ unsigned int mbus_codes[2];
+ bool is_raw;
+ struct {
+ u32 base_mode;
+ u32 uv_plane;
+ } registers;
+};
+
+struct mali_c55_isp_fmt {
+ u32 code;
+ bool bypass;
+ u32 order;
+};
+
+enum mali_c55_planes {
+ MALI_C55_PLANE_Y,
+ MALI_C55_PLANE_UV,
+ MALI_C55_NUM_PLANES
+};
+
+struct mali_c55_buffer {
+ struct vb2_v4l2_buffer vb;
+ unsigned int planes_pending;
+ struct list_head queue;
+ dma_addr_t addrs[MALI_C55_NUM_PLANES];
+};
+
+struct mali_c55_cap_dev {
+ struct mali_c55 *mali_c55;
+ struct mali_c55_resizer *rsz;
+ struct video_device vdev;
+ struct media_pad pad;
+ struct vb2_queue queue;
+ /* Mutex to provide to vb2 */
+ struct mutex lock;
+ unsigned int reg_offset;
+
+ struct {
+ const struct mali_c55_format_info *info;
+ struct v4l2_pix_format_mplane format;
+ } format;
+
+ struct {
+ /* Spinlock to guard buffer queue */
+ spinlock_t lock;
+ struct list_head queue;
+ struct mali_c55_buffer *curr;
+ struct mali_c55_buffer *next;
+ } buffers;
+
+ bool streaming;
+};
+
+enum mali_c55_config_spaces {
+ MALI_C55_CONFIG_PING,
+ MALI_C55_CONFIG_PONG,
+ MALI_C55_NUM_CONFIG_SPACES
+};
+
+/**
+ * struct mali_c55_context - Fields relating to a single camera context
+ *
+ * @mali_c55: Pointer to the main struct mali_c55
+ * @registers: A pointer to some allocated memory holding register
+ * values to be written to the hardware at frame interrupt
+ * @base: Base address of the config space in the hardware
+ * @lock: A spinlock to protect against writes to @registers whilst that
+ * space is being copied to the hardware
+ * @list: A list head to facilitate a context queue
+ */
+struct mali_c55_context {
+ struct mali_c55 *mali_c55;
+ void *registers;
+ phys_addr_t base;
+ /* Spinlock to prevent simultaneous access of register space */
+ spinlock_t lock;
+ struct list_head list;
+};
+
+struct mali_c55 {
+ struct device *dev;
+ void __iomem *base;
+ struct dma_chan *channel;
+ struct clk_bulk_data clks[MALI_C55_NUM_CLKS];
+
+ u16 capabilities;
+ struct media_device media_dev;
+ struct v4l2_device v4l2_dev;
+ struct v4l2_async_notifier notifier;
+ struct media_pipeline pipe;
+
+ struct mali_c55_tpg tpg;
+ struct mali_c55_isp isp;
+ struct mali_c55_resizer resizers[MALI_C55_NUM_RSZS];
+ struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS];
+
+ struct mali_c55_context context;
+ enum mali_c55_config_spaces next_config;
+};
+
+void mali_c55_write(struct mali_c55 *mali_c55, unsigned int addr, u32 val);
+void mali_c55_cap_dev_write(struct mali_c55_cap_dev *cap_dev, unsigned int addr,
+ u32 val);
+void mali_c55_update_bits(struct mali_c55 *mali_c55, unsigned int addr,
+ u32 mask, u32 val);
+u32 mali_c55_read(struct mali_c55 *mali_c55, unsigned int addr);
+void mali_c55_ctx_write(struct mali_c55 *mali_c55, unsigned int addr, u32 val);
+u32 mali_c55_ctx_read(struct mali_c55 *mali_c55, unsigned int addr);
+void mali_c55_ctx_update_bits(struct mali_c55 *mali_c55, unsigned int addr,
+ u32 mask, u32 val);
+
+int mali_c55_config_write(struct mali_c55_context *ctx,
+ enum mali_c55_config_spaces cfg_space);
+
+int mali_c55_register_isp(struct mali_c55 *mali_c55);
+int mali_c55_register_tpg(struct mali_c55 *mali_c55);
+void mali_c55_unregister_tpg(struct mali_c55 *mali_c55);
+void mali_c55_unregister_isp(struct mali_c55 *mali_c55);
+int mali_c55_register_resizers(struct mali_c55 *mali_c55);
+void mali_c55_unregister_resizers(struct mali_c55 *mali_c55);
+int mali_c55_register_capture_devs(struct mali_c55 *mali_c55);
+void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55);
+struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55);
+void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
+ enum mali_c55_planes plane);
+void mali_c55_set_next_buffer(struct mali_c55_cap_dev *cap_dev);
+void mali_c55_isp_queue_event_sof(struct mali_c55 *mali_c55);
+
+bool mali_c55_format_is_raw(unsigned int mbus_code);
+
+const struct mali_c55_isp_fmt *
+mali_c55_isp_fmt_next(const struct mali_c55_isp_fmt *fmt);
+const struct mali_c55_isp_fmt *
+mali_c55_isp_get_mbus_config_by_code(u32 code);
+const struct mali_c55_isp_fmt *
+mali_c55_isp_get_mbus_config_by_index(u32 index);
+
+#endif /* _MALI_C55_COMMON_H */
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
new file mode 100644
index 000000000000..dbc07d12d3a3
--- /dev/null
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
@@ -0,0 +1,893 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM Mali-C55 ISP Driver - Core driver code
+ *
+ * Copyright (C) 2024 Ideas on Board Oy
+ */
+
+#include <linux/bitops.h>
+#include <linux/cleanup.h>
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/dmaengine.h>
+#include <linux/dma-mapping.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/ioport.h>
+#include <linux/mod_devicetable.h>
+#include <linux/of.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+
+#include <media/media-entity.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-mc.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mali-c55-common.h"
+#include "mali-c55-registers.h"
+
+static const char * const mali_c55_interrupt_names[] = {
+ [MALI_C55_IRQ_ISP_START] = "ISP start",
+ [MALI_C55_IRQ_ISP_DONE] = "ISP done",
+ [MALI_C55_IRQ_MCM_ERROR] = "Multi-context management error",
+ [MALI_C55_IRQ_BROKEN_FRAME_ERROR] = "Broken frame error",
+ [MALI_C55_IRQ_MET_AF_DONE] = "AF metering done",
+ [MALI_C55_IRQ_MET_AEXP_DONE] = "AEXP metering done",
+ [MALI_C55_IRQ_MET_AWB_DONE] = "AWB metering done",
+ [MALI_C55_IRQ_AEXP_1024_DONE] = "AEXP 1024-bit histogram done",
+ [MALI_C55_IRQ_IRIDIX_MET_DONE] = "Iridix metering done",
+ [MALI_C55_IRQ_LUT_INIT_DONE] = "LUT memory init done",
+ [MALI_C55_IRQ_FR_Y_DONE] = "Full resolution Y plane DMA done",
+ [MALI_C55_IRQ_FR_UV_DONE] = "Full resolution U/V plane DMA done",
+ [MALI_C55_IRQ_DS_Y_DONE] = "Downscale Y plane DMA done",
+ [MALI_C55_IRQ_DS_UV_DONE] = "Downscale U/V plane DMA done",
+ [MALI_C55_IRQ_LINEARIZATION_DONE] = "Linearisation done",
+ [MALI_C55_IRQ_RAW_FRONTEND_DONE] = "Raw frontend processing done",
+ [MALI_C55_IRQ_NOISE_REDUCTION_DONE] = "Noise reduction done",
+ [MALI_C55_IRQ_IRIDIX_DONE] = "Iridix done",
+ [MALI_C55_IRQ_BAYER2RGB_DONE] = "Bayer to RGB conversion done",
+ [MALI_C55_IRQ_WATCHDOG_TIMER] = "Watchdog timer timed out",
+ [MALI_C55_IRQ_FRAME_COLLISION] = "Frame collision error",
+ [MALI_C55_IRQ_UNUSED] = "IRQ bit unused",
+ [MALI_C55_IRQ_DMA_ERROR] = "DMA error",
+ [MALI_C55_IRQ_INPUT_STOPPED] = "Input port safely stopped",
+ [MALI_C55_IRQ_MET_AWB_TARGET1_HIT] = "AWB metering target 1 address hit",
+ [MALI_C55_IRQ_MET_AWB_TARGET2_HIT] = "AWB metering target 2 address hit"
+};
+
+static const unsigned int config_space_addrs[] = {
+ [MALI_C55_CONFIG_PING] = 0x0ab6c,
+ [MALI_C55_CONFIG_PONG] = 0x22b2c,
+};
+
+static const char * const mali_c55_clk_names[] = {
+ "aclk",
+ "hclk",
+};
+
+/*
+ * System IO
+ *
+ * The Mali-C55 ISP has up to two configuration register spaces (called 'ping'
+ * and 'pong'), with the expectation that the 'active' space will be left
+ * untouched whilst a frame is being processed and the 'inactive' space
+ * configured ready to be switched to during the blanking period before the next
+ * frame processing starts. These spaces should ideally be set via DMA transfer
+ * from a buffer rather than through individual register set operations. There
+ * is also a shared global register space which should be set normally. Of
+ * course, the ISP might be included in a system which lacks a suitable DMA
+ * engine, and the second configuration space might not be fitted at all, which
+ * means we need to support four scenarios:
+ *
+ * 1. Multi config space, with DMA engine.
+ * 2. Multi config space, no DMA engine.
+ * 3. Single config space, with DMA engine.
+ * 4. Single config space, no DMA engine.
+ *
+ * The first case is very easy, but the rest present annoying problems. The best
+ * way to solve them seems to be simply to replicate the concept of DMAing over
+ * the configuration buffer even if there's no DMA engine on the board, for
+ * which we rely on memcpy. To facilitate this any read/write call that is made
+ * to an address within those config spaces should infact be directed to a
+ * buffer that was allocated to hold them rather than the IO memory itself. The
+ * actual copy of that buffer to IO mem will happen on interrupt.
+ */
+
+void mali_c55_write(struct mali_c55 *mali_c55, unsigned int addr, u32 val)
+{
+ WARN_ON(addr >= MALI_C55_REG_CONFIG_SPACES_OFFSET);
+
+ writel(val, mali_c55->base + addr);
+}
+
+u32 mali_c55_read(struct mali_c55 *mali_c55, unsigned int addr)
+{
+ WARN_ON(addr >= MALI_C55_REG_CONFIG_SPACES_OFFSET);
+
+ return readl(mali_c55->base + addr);
+}
+
+void mali_c55_update_bits(struct mali_c55 *mali_c55, unsigned int addr,
+ u32 mask, u32 val)
+{
+ u32 orig, tmp;
+
+ orig = mali_c55_read(mali_c55, addr);
+
+ tmp = orig & ~mask;
+ tmp |= val & mask;
+
+ if (tmp != orig)
+ mali_c55_write(mali_c55, addr, tmp);
+}
+
+static void __mali_c55_ctx_write(struct mali_c55_context *ctx,
+ unsigned int addr, u32 val)
+{
+ spin_lock(&ctx->lock);
+ addr = (addr - MALI_C55_REG_CONFIG_SPACES_OFFSET) / 4;
+ ((u32 *)ctx->registers)[addr] = val;
+ spin_unlock(&ctx->lock);
+}
+
+void mali_c55_ctx_write(struct mali_c55 *mali_c55, unsigned int addr, u32 val)
+{
+ struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
+
+ WARN_ON(addr < MALI_C55_REG_CONFIG_SPACES_OFFSET);
+ __mali_c55_ctx_write(ctx, addr, val);
+}
+
+static u32 __mali_c55_ctx_read(struct mali_c55_context *ctx, unsigned int addr)
+{
+ u32 val;
+
+ spin_lock(&ctx->lock);
+ addr = (addr - MALI_C55_REG_CONFIG_SPACES_OFFSET) / 4;
+ val = ((u32 *)ctx->registers)[addr];
+ spin_unlock(&ctx->lock);
+
+ return val;
+}
+
+u32 mali_c55_ctx_read(struct mali_c55 *mali_c55, unsigned int addr)
+{
+ struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
+
+ WARN_ON(addr < MALI_C55_REG_CONFIG_SPACES_OFFSET);
+ return __mali_c55_ctx_read(ctx, addr);
+}
+
+void mali_c55_ctx_update_bits(struct mali_c55 *mali_c55, unsigned int addr,
+ u32 mask, u32 val)
+{
+ struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
+ u32 orig, tmp;
+
+ WARN_ON(addr < MALI_C55_REG_CONFIG_SPACES_OFFSET);
+
+ orig = __mali_c55_ctx_read(ctx, addr);
+
+ tmp = orig & ~mask;
+ tmp |= val & mask;
+
+ if (tmp != orig)
+ __mali_c55_ctx_write(ctx, addr, tmp);
+}
+
+static void mali_c55_dma_xfer_complete(void *param,
+ const struct dmaengine_result *result)
+{
+ struct mali_c55 *mali_c55 = param;
+
+ if (result->result != DMA_TRANS_NOERROR)
+ dev_err(mali_c55->dev, "Failed to DMA xfer ISP config\n");
+}
+
+static int mali_c55_dma_xfer(struct mali_c55_context *ctx, dma_addr_t src,
+ dma_addr_t dst)
+{
+ struct mali_c55 *mali_c55 = ctx->mali_c55;
+ struct dma_async_tx_descriptor *tx;
+ dma_cookie_t cookie;
+
+ tx = dmaengine_prep_dma_memcpy(mali_c55->channel, dst, src,
+ MALI_C55_CONFIG_SPACE_SIZE, 0);
+ if (!tx) {
+ dev_err(mali_c55->dev, "failed to prep DMA memcpy\n");
+ return -EIO;
+ }
+
+ tx->callback_result = mali_c55_dma_xfer_complete;
+ tx->callback_param = mali_c55;
+
+ cookie = dmaengine_submit(tx);
+ if (dma_submit_error(cookie)) {
+ dev_err(mali_c55->dev, "error submitting dma transfer\n");
+ return -EIO;
+ }
+
+ dma_async_issue_pending(mali_c55->channel);
+
+ return 0;
+}
+
+static int mali_c55_dma_write(struct mali_c55_context *ctx,
+ enum mali_c55_config_spaces cfg_space)
+{
+ struct mali_c55 *mali_c55 = ctx->mali_c55;
+ struct device *dma_dev = mali_c55->channel->device->dev;
+ dma_addr_t dst = ctx->base + config_space_addrs[cfg_space];
+ dma_addr_t src;
+ int ret;
+
+ guard(spinlock)(&ctx->lock);
+
+ src = dma_map_single(dma_dev, ctx->registers,
+ MALI_C55_CONFIG_SPACE_SIZE, DMA_TO_DEVICE);
+ if (dma_mapping_error(dma_dev, src)) {
+ dev_err(mali_c55->dev, "failed to map DMA addr\n");
+ return -EIO;
+ }
+
+ ret = mali_c55_dma_xfer(ctx, src, dst);
+ dma_unmap_single(dma_dev, src, MALI_C55_CONFIG_SPACE_SIZE,
+ DMA_TO_DEVICE);
+
+ return ret;
+}
+
+int mali_c55_config_write(struct mali_c55_context *ctx,
+ enum mali_c55_config_spaces cfg_space)
+{
+ struct mali_c55 *mali_c55 = ctx->mali_c55;
+
+ if (mali_c55->channel)
+ return mali_c55_dma_write(ctx, cfg_space);
+
+ memcpy_toio(mali_c55->base + config_space_addrs[cfg_space],
+ ctx->registers, MALI_C55_CONFIG_SPACE_SIZE);
+
+ return 0;
+}
+
+struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55)
+{
+ return &mali_c55->context;
+}
+
+static int mali_c55_parse_endpoint(struct mali_c55 *mali_c55)
+{
+ struct v4l2_async_connection *asc;
+ struct fwnode_handle *ep;
+
+ /*
+ * The ISP should have a single endpoint pointing to some flavour of
+ * CSI-2 receiver...but for now at least we do want everything to work
+ * normally even with no sensors connected, as we have the TPG. If we
+ * don't find a sensor just warn and return success.
+ */
+ ep = fwnode_graph_get_endpoint_by_id(dev_fwnode(mali_c55->dev),
+ 0, 0, 0);
+ if (!ep) {
+ dev_warn(mali_c55->dev, "no local endpoint found\n");
+ return 0;
+ }
+
+ asc = v4l2_async_nf_add_fwnode_remote(&mali_c55->notifier, ep,
+ struct v4l2_async_connection);
+ fwnode_handle_put(ep);
+ if (IS_ERR(asc)) {
+ dev_err(mali_c55->dev, "failed to add remote fwnode\n");
+ return PTR_ERR(asc);
+ }
+
+ return 0;
+}
+
+static void mali_c55_remove_links(struct mali_c55 *mali_c55)
+{
+ unsigned int i;
+
+ media_entity_remove_links(&mali_c55->tpg.sd.entity);
+ media_entity_remove_links(&mali_c55->isp.sd.entity);
+
+ for (i = 0; i < MALI_C55_NUM_RSZS; i++)
+ media_entity_remove_links(&mali_c55->resizers[i].sd.entity);
+
+ for (i = 0; i < MALI_C55_NUM_CAP_DEVS; i++)
+ media_entity_remove_links(&mali_c55->cap_devs[i].vdev.entity);
+}
+
+static int mali_c55_create_links(struct mali_c55 *mali_c55)
+{
+ int ret;
+
+ /* Test pattern generator to ISP */
+ ret = media_create_pad_link(&mali_c55->tpg.sd.entity, 0,
+ &mali_c55->isp.sd.entity,
+ MALI_C55_ISP_PAD_SINK_VIDEO, 0);
+ if (ret) {
+ dev_err(mali_c55->dev, "failed to link TPG and ISP\n");
+ goto err_remove_links;
+ }
+
+ /* Full resolution resizer pipe. */
+ ret = media_create_pad_link(&mali_c55->isp.sd.entity,
+ MALI_C55_ISP_PAD_SOURCE_VIDEO,
+ &mali_c55->resizers[MALI_C55_RSZ_FR].sd.entity,
+ MALI_C55_RSZ_SINK_PAD,
+ MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE);
+ if (ret) {
+ dev_err(mali_c55->dev, "failed to link ISP and FR resizer\n");
+ goto err_remove_links;
+ }
+
+ /* Full resolution bypass. */
+ ret = media_create_pad_link(&mali_c55->isp.sd.entity,
+ MALI_C55_ISP_PAD_SOURCE_BYPASS,
+ &mali_c55->resizers[MALI_C55_RSZ_FR].sd.entity,
+ MALI_C55_RSZ_SINK_BYPASS_PAD,
+ MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE);
+ if (ret) {
+ dev_err(mali_c55->dev, "failed to link ISP and FR resizer\n");
+ goto err_remove_links;
+ }
+
+ /* Resizer pipe to video capture nodes. */
+ ret = media_create_pad_link(&mali_c55->resizers[0].sd.entity,
+ MALI_C55_RSZ_SOURCE_PAD,
+ &mali_c55->cap_devs[MALI_C55_CAP_DEV_FR].vdev.entity,
+ 0, MEDIA_LNK_FL_ENABLED);
+ if (ret) {
+ dev_err(mali_c55->dev,
+ "failed to link FR resizer and video device\n");
+ goto err_remove_links;
+ }
+
+ /* The downscale pipe is an optional hardware block */
+ if (mali_c55->capabilities & MALI_C55_GPS_DS_PIPE_FITTED) {
+ ret = media_create_pad_link(&mali_c55->isp.sd.entity,
+ MALI_C55_ISP_PAD_SOURCE_VIDEO,
+ &mali_c55->resizers[MALI_C55_RSZ_DS].sd.entity,
+ MALI_C55_RSZ_SINK_PAD,
+ MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE);
+ if (ret) {
+ dev_err(mali_c55->dev,
+ "failed to link ISP and DS resizer\n");
+ goto err_remove_links;
+ }
+
+ ret = media_create_pad_link(&mali_c55->resizers[1].sd.entity,
+ MALI_C55_RSZ_SOURCE_PAD,
+ &mali_c55->cap_devs[MALI_C55_CAP_DEV_DS].vdev.entity,
+ 0, MEDIA_LNK_FL_ENABLED);
+ if (ret) {
+ dev_err(mali_c55->dev,
+ "failed to link DS resizer and video device\n");
+ goto err_remove_links;
+ }
+ }
+
+ return 0;
+
+err_remove_links:
+ mali_c55_remove_links(mali_c55);
+ return ret;
+}
+
+static void mali_c55_unregister_entities(struct mali_c55 *mali_c55)
+{
+ mali_c55_remove_links(mali_c55);
+ mali_c55_unregister_tpg(mali_c55);
+ mali_c55_unregister_isp(mali_c55);
+ mali_c55_unregister_resizers(mali_c55);
+ mali_c55_unregister_capture_devs(mali_c55);
+}
+
+static int mali_c55_register_entities(struct mali_c55 *mali_c55)
+{
+ int ret;
+
+ ret = mali_c55_register_tpg(mali_c55);
+ if (ret)
+ return ret;
+
+ ret = mali_c55_register_isp(mali_c55);
+ if (ret)
+ goto err_unregister_entities;
+
+ ret = mali_c55_register_resizers(mali_c55);
+ if (ret)
+ goto err_unregister_entities;
+
+ ret = mali_c55_register_capture_devs(mali_c55);
+ if (ret)
+ goto err_unregister_entities;
+
+ ret = mali_c55_create_links(mali_c55);
+ if (ret)
+ goto err_unregister_entities;
+
+ return 0;
+
+err_unregister_entities:
+ mali_c55_unregister_entities(mali_c55);
+
+ return ret;
+}
+
+static int mali_c55_notifier_bound(struct v4l2_async_notifier *notifier,
+ struct v4l2_subdev *subdev,
+ struct v4l2_async_connection *asc)
+{
+ struct mali_c55 *mali_c55 = container_of(notifier,
+ struct mali_c55, notifier);
+ struct media_pad *pad = &mali_c55->isp.pads[MALI_C55_ISP_PAD_SINK_VIDEO];
+ int ret;
+
+ /*
+ * By default we'll flag this link enabled and the TPG disabled, but
+ * no immutable flag because we need to be able to switch between the
+ * two.
+ */
+ ret = v4l2_create_fwnode_links_to_pad(subdev, pad,
+ MEDIA_LNK_FL_ENABLED);
+ if (ret)
+ dev_err(mali_c55->dev, "failed to create link for %s\n",
+ subdev->name);
+
+ return ret;
+}
+
+static int mali_c55_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+ struct mali_c55 *mali_c55 = container_of(notifier,
+ struct mali_c55, notifier);
+
+ return v4l2_device_register_subdev_nodes(&mali_c55->v4l2_dev);
+}
+
+static const struct v4l2_async_notifier_operations mali_c55_notifier_ops = {
+ .bound = mali_c55_notifier_bound,
+ .complete = mali_c55_notifier_complete,
+};
+
+static int mali_c55_media_frameworks_init(struct mali_c55 *mali_c55)
+{
+ int ret;
+
+ media_device_init(&mali_c55->media_dev);
+ ret = media_device_register(&mali_c55->media_dev);
+ if (ret)
+ goto err_cleanup_media_device;
+
+ mali_c55->v4l2_dev.mdev = &mali_c55->media_dev;
+ ret = v4l2_device_register(mali_c55->dev, &mali_c55->v4l2_dev);
+ if (ret) {
+ dev_err(mali_c55->dev, "failed to register V4L2 device\n");
+ goto err_unregister_media_device;
+ };
+
+ mali_c55->notifier.ops = &mali_c55_notifier_ops;
+ v4l2_async_nf_init(&mali_c55->notifier, &mali_c55->v4l2_dev);
+
+ ret = mali_c55_register_entities(mali_c55);
+ if (ret) {
+ dev_err(mali_c55->dev, "failed to register entities\n");
+ goto err_cleanup_nf;
+ }
+
+ ret = mali_c55_parse_endpoint(mali_c55);
+ if (ret)
+ goto err_cleanup_nf;
+
+ ret = v4l2_async_nf_register(&mali_c55->notifier);
+ if (ret) {
+ dev_err(mali_c55->dev, "failed to register notifier\n");
+ goto err_unregister_entities;
+ }
+
+ return 0;
+
+err_unregister_entities:
+ mali_c55_unregister_entities(mali_c55);
+err_cleanup_nf:
+ v4l2_async_nf_cleanup(&mali_c55->notifier);
+ v4l2_device_unregister(&mali_c55->v4l2_dev);
+err_unregister_media_device:
+ media_device_unregister(&mali_c55->media_dev);
+err_cleanup_media_device:
+ media_device_cleanup(&mali_c55->media_dev);
+
+ return ret;
+}
+
+static void mali_c55_media_frameworks_deinit(struct mali_c55 *mali_c55)
+{
+ v4l2_async_nf_unregister(&mali_c55->notifier);
+ mali_c55_unregister_entities(mali_c55);
+ v4l2_async_nf_cleanup(&mali_c55->notifier);
+ v4l2_device_unregister(&mali_c55->v4l2_dev);
+ media_device_unregister(&mali_c55->media_dev);
+ media_device_cleanup(&mali_c55->media_dev);
+}
+
+static int mali_c55_check_hwcfg(struct mali_c55 *mali_c55)
+{
+ u32 product, version, revision, capabilities;
+
+ product = mali_c55_read(mali_c55, MALI_C55_REG_PRODUCT);
+ version = mali_c55_read(mali_c55, MALI_C55_REG_VERSION);
+ revision = mali_c55_read(mali_c55, MALI_C55_REG_REVISION);
+
+ mali_c55->media_dev.hw_revision = version;
+
+ dev_info(mali_c55->dev, "Detected Mali-C55 ISP %u.%u.%u\n",
+ product, version, revision);
+
+ capabilities = mali_c55_read(mali_c55,
+ MALI_C55_REG_GLOBAL_PARAMETER_STATUS);
+
+ /*
+ * In its current iteration, the driver only supports inline mode. Given
+ * we cannot control input data timing in this mode, we cannot guarantee
+ * that the vertical blanking periods between frames will be long enough
+ * for us to write configuration data to the ISP during them. For that
+ * reason we can't really support single config space configuration
+ * until memory input mode is implemented.
+ */
+ if (!capabilities & MALI_C55_GPS_PONG_FITTED) {
+ dev_err(mali_c55->dev, "Pong config space not fitted.\n");
+ return -EINVAL;
+ }
+
+ mali_c55->capabilities = (capabilities & 0xffff);
+
+ return 0;
+}
+
+static void mali_c55_swap_next_config(struct mali_c55 *mali_c55)
+{
+ struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
+ u32 curr_config, next_config;
+
+ curr_config = mali_c55_read(mali_c55, MALI_C55_REG_PING_PONG_READ);
+ curr_config = (curr_config & MALI_C55_REG_PING_PONG_READ_MASK)
+ >> (ffs(MALI_C55_REG_PING_PONG_READ_MASK) - 1);
+ next_config = curr_config ^ 1;
+
+ mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG,
+ MALI_C55_REG_MCU_CONFIG_WRITE_MASK,
+ MALI_C55_MCU_CONFIG_WRITE(next_config));
+ mali_c55_config_write(ctx, next_config ?
+ MALI_C55_CONFIG_PING : MALI_C55_CONFIG_PONG);
+}
+
+static irqreturn_t mali_c55_isr(int irq, void *context)
+{
+ struct device *dev = context;
+ struct mali_c55 *mali_c55 = dev_get_drvdata(dev);
+ u32 interrupt_status;
+ unsigned int i, j;
+
+ interrupt_status = mali_c55_read(mali_c55,
+ MALI_C55_REG_INTERRUPT_STATUS_VECTOR);
+ if (!interrupt_status)
+ return IRQ_NONE;
+
+ mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR_VECTOR,
+ interrupt_status);
+ mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR, 0);
+ mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR, 1);
+
+ for (i = 0; i < MALI_C55_NUM_IRQ_BITS; i++) {
+ if (!(interrupt_status & (1 << i)))
+ continue;
+
+ switch (i) {
+ case MALI_C55_IRQ_ISP_START:
+ mali_c55_isp_queue_event_sof(mali_c55);
+
+ for (j = i; j < MALI_C55_NUM_CAP_DEVS; j++)
+ mali_c55_set_next_buffer(&mali_c55->cap_devs[j]);
+
+ mali_c55_swap_next_config(mali_c55);
+
+ break;
+ case MALI_C55_IRQ_ISP_DONE:
+ /*
+ * TODO: Where the ISP has no Pong config fitted, we'd
+ * have to do the mali_c55_swap_next_config() call here.
+ */
+ break;
+ case MALI_C55_IRQ_FR_Y_DONE:
+ mali_c55_set_plane_done(
+ &mali_c55->cap_devs[MALI_C55_CAP_DEV_FR],
+ MALI_C55_PLANE_Y);
+ break;
+ case MALI_C55_IRQ_FR_UV_DONE:
+ mali_c55_set_plane_done(
+ &mali_c55->cap_devs[MALI_C55_CAP_DEV_FR],
+ MALI_C55_PLANE_UV);
+ break;
+ case MALI_C55_IRQ_DS_Y_DONE:
+ mali_c55_set_plane_done(
+ &mali_c55->cap_devs[MALI_C55_CAP_DEV_DS],
+ MALI_C55_PLANE_Y);
+ break;
+ case MALI_C55_IRQ_DS_UV_DONE:
+ mali_c55_set_plane_done(
+ &mali_c55->cap_devs[MALI_C55_CAP_DEV_DS],
+ MALI_C55_PLANE_UV);
+ break;
+ default:
+ /*
+ * Only the above interrupts are currently unmasked. If
+ * we receive anything else here then something weird
+ * has gone on.
+ */
+ dev_err(dev, "masked interrupt %s triggered\n",
+ mali_c55_interrupt_names[i]);
+ }
+ }
+
+ return IRQ_HANDLED;
+}
+
+static int mali_c55_init_context(struct mali_c55 *mali_c55,
+ struct resource *res)
+{
+ struct mali_c55_context *ctx = &mali_c55->context;
+
+ ctx->base = res->start;
+ ctx->mali_c55 = mali_c55;
+
+ ctx->registers = kzalloc(MALI_C55_CONFIG_SPACE_SIZE,
+ GFP_KERNEL | GFP_DMA);
+ if (!ctx->registers)
+ return -ENOMEM;
+
+ /*
+ * The allocated memory is empty, we need to load the default
+ * register settings. We just read Ping; it's identical to Pong.
+ */
+ memcpy_fromio(ctx->registers,
+ mali_c55->base + config_space_addrs[MALI_C55_CONFIG_PING],
+ MALI_C55_CONFIG_SPACE_SIZE);
+
+ /*
+ * Some features of the ISP need to be disabled by default and only
+ * enabled at the same time as they're configured by a parameters buffer
+ */
+
+ /* Bypass the sqrt and square compression and expansion modules */
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BYPASS_1,
+ MALI_C55_REG_BYPASS_1_FE_SQRT,
+ MALI_C55_REG_BYPASS_1_FE_SQRT);
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BYPASS_3,
+ MALI_C55_REG_BYPASS_3_SQUARE_BE,
+ MALI_C55_REG_BYPASS_3_SQUARE_BE);
+
+ /* Bypass the temper module */
+ mali_c55_ctx_write(mali_c55, MALI_C55_REG_BYPASS_2,
+ MALI_C55_REG_BYPASS_2_TEMPER);
+
+ /* Bypass the colour noise reduction */
+ mali_c55_ctx_write(mali_c55, MALI_C55_REG_BYPASS_4,
+ MALI_C55_REG_BYPASS_4_CNR);
+
+ /* Disable the sinter module */
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_SINTER_CONFIG,
+ MALI_C55_SINTER_ENABLE_MASK, 0x00);
+
+ /* Disable the RGB Gamma module for each output */
+ mali_c55_ctx_write(mali_c55, MALI_C55_REG_FR_GAMMA_RGB_ENABLE, 0x00);
+ mali_c55_ctx_write(mali_c55, MALI_C55_REG_DS_GAMMA_RGB_ENABLE, 0x00);
+
+ /* Disable the colour correction matrix */
+ mali_c55_ctx_write(mali_c55, MALI_C55_REG_CCM_ENABLE, 0x00);
+
+ return 0;
+}
+
+static int mali_c55_runtime_resume(struct device *dev)
+{
+ struct mali_c55 *mali_c55 = dev_get_drvdata(dev);
+ int ret;
+
+ ret = clk_bulk_prepare_enable(ARRAY_SIZE(mali_c55->clks),
+ mali_c55->clks);
+ if (ret) {
+ dev_err(mali_c55->dev, "failed to enable clocks\n");
+ return ret;
+ }
+
+ /*
+ * Mask the interrupts and clear any that were set, then unmask the ones
+ * that we actually want to handle.
+ */
+ mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_MASK_VECTOR,
+ MALI_C55_INTERRUPT_MASK_ALL);
+ mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR_VECTOR,
+ MALI_C55_INTERRUPT_MASK_ALL);
+ mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR, 0x01);
+ mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR, 0x00);
+
+ mali_c55_update_bits(mali_c55, MALI_C55_REG_INTERRUPT_MASK_VECTOR,
+ MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_ISP_START) |
+ MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_ISP_DONE) |
+ MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_FR_Y_DONE) |
+ MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_FR_UV_DONE) |
+ MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_DS_Y_DONE) |
+ MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_DS_UV_DONE),
+ 0x00);
+
+ return 0;
+}
+
+static int mali_c55_runtime_suspend(struct device *dev)
+{
+ struct mali_c55 *mali_c55 = dev_get_drvdata(dev);
+
+ clk_bulk_disable_unprepare(ARRAY_SIZE(mali_c55->clks), mali_c55->clks);
+ return 0;
+}
+
+static const struct dev_pm_ops mali_c55_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+ pm_runtime_force_resume)
+ SET_RUNTIME_PM_OPS(mali_c55_runtime_suspend, mali_c55_runtime_resume,
+ NULL)
+};
+
+static int mali_c55_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct mali_c55 *mali_c55;
+ struct resource *res;
+ dma_cap_mask_t mask;
+ int ret;
+ u32 val;
+
+ mali_c55 = devm_kzalloc(dev, sizeof(*mali_c55), GFP_KERNEL);
+ if (!mali_c55)
+ return -ENOMEM;
+
+ mali_c55->dev = dev;
+ platform_set_drvdata(pdev, mali_c55);
+
+ mali_c55->base = devm_platform_get_and_ioremap_resource(pdev, 0,
+ &res);
+ if (IS_ERR(mali_c55->base))
+ return dev_err_probe(dev, PTR_ERR(mali_c55->base),
+ "failed to map IO memory\n");
+
+ for (unsigned int i = 0; i < ARRAY_SIZE(mali_c55_clk_names); i++)
+ mali_c55->clks[i].id = mali_c55_clk_names[i];
+
+ ret = devm_clk_bulk_get(dev, ARRAY_SIZE(mali_c55->clks), mali_c55->clks);
+ if (ret)
+ return dev_err_probe(dev, ret, "failed to acquire clocks\n");
+
+ of_reserved_mem_device_init(dev);
+ vb2_dma_contig_set_max_seg_size(dev, UINT_MAX);
+
+ dma_cap_zero(mask);
+ dma_cap_set(DMA_MEMCPY, mask);
+
+ pm_runtime_set_autosuspend_delay(&pdev->dev, 2000);
+ pm_runtime_use_autosuspend(&pdev->dev);
+ pm_runtime_enable(&pdev->dev);
+
+ ret = pm_runtime_resume_and_get(&pdev->dev);
+ if (ret)
+ goto err_pm_runtime_disable;
+
+ ret = mali_c55_check_hwcfg(mali_c55);
+ if (ret)
+ goto err_pm_runtime_put;
+
+ /* Use "software only" context management. */
+ mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG,
+ MALI_C55_REG_MCU_CONFIG_OVERRIDE_MASK, 0x01);
+
+ /*
+ * No error check, because we will just fallback on memcpy if there is
+ * no usable DMA channel on the system.
+ */
+ mali_c55->channel = dma_request_channel(mask, NULL, NULL);
+
+ ret = mali_c55_init_context(mali_c55, res);
+ if (ret)
+ goto err_release_dma_channel;
+
+ mali_c55->media_dev.dev = dev;
+ strscpy(mali_c55->media_dev.model, "ARM Mali-C55 ISP",
+ sizeof(mali_c55->media_dev.model));
+
+ ret = mali_c55_media_frameworks_init(mali_c55);
+ if (ret)
+ goto err_free_context_registers;
+
+ /* Set safe stop to ensure we're in a non-streaming state */
+ mali_c55_write(mali_c55, MALI_C55_REG_INPUT_MODE_REQUEST,
+ MALI_C55_INPUT_SAFE_STOP);
+ readl_poll_timeout(mali_c55->base + MALI_C55_REG_MODE_STATUS,
+ val, !val, 10 * USEC_PER_MSEC, 250 * USEC_PER_MSEC);
+
+ pm_runtime_put(&pdev->dev);
+
+ ret = platform_get_irq(pdev, 0);
+ if (ret < 0) {
+ dev_err(dev, "failed to get interrupt\n");
+ goto err_deinit_media_frameworks;
+ }
+
+ /*
+ * The driver needs to transfer large amounts of register settings to
+ * the ISP each frame, using either a DMA transfer or memcpy. We use a
+ * threaded IRQ to avoid disabling interrupts the entire time that's
+ * happening.
+ */
+ ret = devm_request_threaded_irq(&pdev->dev, ret, NULL,
+ mali_c55_isr, IRQF_ONESHOT,
+ dev_driver_string(&pdev->dev),
+ &pdev->dev);
+ if (ret) {
+ dev_err(dev, "failed to request irq\n");
+ goto err_deinit_media_frameworks;
+ }
+
+ return 0;
+
+err_deinit_media_frameworks:
+ mali_c55_media_frameworks_deinit(mali_c55);
+err_free_context_registers:
+ kfree(mali_c55->context.registers);
+err_release_dma_channel:
+ dma_release_channel(mali_c55->channel);
+err_pm_runtime_put:
+ pm_runtime_put(&pdev->dev);
+err_pm_runtime_disable:
+ pm_runtime_disable(&pdev->dev);
+
+ return ret;
+}
+
+static void mali_c55_remove(struct platform_device *pdev)
+{
+ struct mali_c55 *mali_c55 = platform_get_drvdata(pdev);
+
+ kfree(mali_c55->context.registers);
+ mali_c55_media_frameworks_deinit(mali_c55);
+ dma_release_channel(mali_c55->channel);
+}
+
+static const struct of_device_id mali_c55_of_match[] = {
+ { .compatible = "arm,mali-c55", },
+ { /* Sentinel */},
+};
+MODULE_DEVICE_TABLE(of, mali_c55_of_match);
+
+static struct platform_driver mali_c55_driver = {
+ .driver = {
+ .name = "mali-c55",
+ .of_match_table = mali_c55_of_match,
+ .pm = &mali_c55_pm_ops,
+ },
+ .probe = mali_c55_probe,
+ .remove_new = mali_c55_remove,
+};
+
+module_platform_driver(mali_c55_driver);
+
+MODULE_AUTHOR("Daniel Scally <dan.scally@ideasonboard.com>");
+MODULE_AUTHOR("Jacopo Mondi <jacopo.mondi@ideasonboard.com>");
+MODULE_DESCRIPTION("ARM Mali-C55 ISP platform driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
new file mode 100644
index 000000000000..f784983a4ccc
--- /dev/null
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
@@ -0,0 +1,531 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM Mali-C55 ISP Driver - Image signal processor
+ *
+ * Copyright (C) 2024 Ideas on Board Oy
+ */
+
+#include <linux/delay.h>
+#include <linux/iopoll.h>
+#include <linux/property.h>
+#include <linux/string.h>
+
+#include <media/media-entity.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+
+#include "mali-c55-common.h"
+#include "mali-c55-registers.h"
+
+static const struct mali_c55_isp_fmt mali_c55_isp_fmts[] = {
+ {
+ .code = MEDIA_BUS_FMT_SRGGB20_1X20,
+ .order = MALI_C55_BAYER_ORDER_RGGB,
+ .bypass = false,
+ },
+ {
+ .code = MEDIA_BUS_FMT_SGRBG20_1X20,
+ .order = MALI_C55_BAYER_ORDER_GRBG,
+ .bypass = false,
+ },
+ {
+ .code = MEDIA_BUS_FMT_SGBRG20_1X20,
+ .order = MALI_C55_BAYER_ORDER_GBRG,
+ .bypass = false,
+ },
+ {
+ .code = MEDIA_BUS_FMT_SBGGR20_1X20,
+ .order = MALI_C55_BAYER_ORDER_BGGR,
+ .bypass = false,
+ },
+ {
+ .code = MEDIA_BUS_FMT_RGB202020_1X60,
+ .order = 0, /* Not relevant for this format */
+ .bypass = true,
+ }
+ /*
+ * TODO: Support MEDIA_BUS_FMT_YUV20_1X60 here. This is so that we can
+ * also support YUV input from a sensor passed-through to the output. At
+ * present we have no mechanism to test that though so it may have to
+ * wait a while...
+ */
+};
+
+const struct mali_c55_isp_fmt *
+mali_c55_isp_get_mbus_config_by_index(u32 index)
+{
+ if (index < ARRAY_SIZE(mali_c55_isp_fmts))
+ return &mali_c55_isp_fmts[index];
+
+ return NULL;
+}
+
+const struct mali_c55_isp_fmt *
+mali_c55_isp_get_mbus_config_by_code(u32 code)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(mali_c55_isp_fmts); i++) {
+ if (mali_c55_isp_fmts[i].code == code)
+ return &mali_c55_isp_fmts[i];
+ }
+
+ return NULL;
+}
+
+static void mali_c55_isp_stop(struct mali_c55 *mali_c55)
+{
+ u32 val;
+
+ mali_c55_write(mali_c55, MALI_C55_REG_INPUT_MODE_REQUEST,
+ MALI_C55_INPUT_SAFE_STOP);
+ readl_poll_timeout(mali_c55->base + MALI_C55_REG_MODE_STATUS,
+ val, !val, 10 * USEC_PER_MSEC, 250 * USEC_PER_MSEC);
+}
+
+static int mali_c55_isp_start(struct mali_c55 *mali_c55)
+{
+ struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
+ struct v4l2_subdev *sd = &mali_c55->isp.sd;
+ const struct v4l2_mbus_framefmt *format;
+ const struct mali_c55_isp_fmt *cfg;
+ struct v4l2_subdev_state *state;
+ const struct v4l2_rect *crop;
+ u32 val;
+ int ret;
+
+ mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG,
+ MALI_C55_REG_MCU_CONFIG_WRITE_MASK,
+ MALI_C55_REG_MCU_CONFIG_WRITE_PING);
+
+ /* Apply input windowing */
+ state = v4l2_subdev_get_locked_active_state(sd);
+ crop = v4l2_subdev_state_get_crop(state, MALI_C55_ISP_PAD_SINK_VIDEO);
+ format = v4l2_subdev_state_get_format(state,
+ MALI_C55_ISP_PAD_SINK_VIDEO);
+ cfg = mali_c55_isp_get_mbus_config_by_code(format->code);
+
+ mali_c55_write(mali_c55, MALI_C55_REG_HC_START,
+ MALI_C55_HC_START(crop->left));
+ mali_c55_write(mali_c55, MALI_C55_REG_HC_SIZE,
+ MALI_C55_HC_SIZE(crop->width));
+ mali_c55_write(mali_c55, MALI_C55_REG_VC_START_SIZE,
+ MALI_C55_VC_START(crop->top) |
+ MALI_C55_VC_SIZE(crop->height));
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BASE_ADDR,
+ MALI_C55_REG_ACTIVE_WIDTH_MASK, format->width);
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BASE_ADDR,
+ MALI_C55_REG_ACTIVE_HEIGHT_MASK,
+ format->height << 16);
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BAYER_ORDER,
+ MALI_C55_BAYER_ORDER_MASK, cfg->order);
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_INPUT_WIDTH,
+ MALI_C55_INPUT_WIDTH_MASK,
+ MALI_C55_INPUT_WIDTH_20BIT);
+
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_ISP_RAW_BYPASS,
+ MALI_C55_ISP_RAW_BYPASS_BYPASS_MASK,
+ cfg->bypass ? MALI_C55_ISP_RAW_BYPASS_BYPASS_MASK :
+ 0x00);
+
+ ret = mali_c55_config_write(ctx, MALI_C55_CONFIG_PING);
+ if (ret) {
+ dev_err(mali_c55->dev, "failed to write ISP config\n");
+ return ret;
+ }
+
+ mali_c55_write(mali_c55, MALI_C55_REG_INPUT_MODE_REQUEST,
+ MALI_C55_INPUT_SAFE_START);
+ readl_poll_timeout(mali_c55->base + MALI_C55_REG_MODE_STATUS, val, !val,
+ 10 * USEC_PER_MSEC, 250 * USEC_PER_MSEC);
+
+ return 0;
+}
+
+static int mali_c55_isp_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ /*
+ * Only the internal RGB processed format is allowed on the regular
+ * processing source pad.
+ */
+ if (code->pad == MALI_C55_ISP_PAD_SOURCE_VIDEO) {
+ if (code->index)
+ return -EINVAL;
+
+ code->code = MEDIA_BUS_FMT_RGB121212_1X36;
+ return 0;
+ }
+
+ /* On the sink and bypass pads all the supported formats are allowed. */
+ if (code->index >= ARRAY_SIZE(mali_c55_isp_fmts))
+ return -EINVAL;
+
+ code->code = mali_c55_isp_fmts[code->index].code;
+
+ return 0;
+}
+
+static int mali_c55_isp_enum_frame_size(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_frame_size_enum *fse)
+{
+ const struct mali_c55_isp_fmt *cfg;
+
+ if (fse->index > 0)
+ return -EINVAL;
+
+ /*
+ * Only the internal RGB processed format is allowed on the regular
+ * processing source pad.
+ *
+ * On the sink and bypass pads all the supported formats are allowed.
+ */
+ if (fse->pad == MALI_C55_ISP_PAD_SOURCE_VIDEO) {
+ if (fse->code != MEDIA_BUS_FMT_RGB121212_1X36)
+ return -EINVAL;
+ } else {
+ cfg = mali_c55_isp_get_mbus_config_by_code(fse->code);
+ if (!cfg)
+ return -EINVAL;
+ }
+
+ fse->min_width = MALI_C55_MIN_WIDTH;
+ fse->min_height = MALI_C55_MIN_HEIGHT;
+ fse->max_width = MALI_C55_MAX_WIDTH;
+ fse->max_height = MALI_C55_MAX_HEIGHT;
+
+ return 0;
+}
+
+static int mali_c55_isp_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format)
+{
+ struct v4l2_mbus_framefmt *fmt = &format->format;
+ struct v4l2_mbus_framefmt *src_fmt, *sink_fmt;
+ const struct mali_c55_isp_fmt *cfg;
+ struct v4l2_rect *crop;
+
+ /*
+ * Disallow set_fmt on the source pads; format is fixed and the sizes
+ * are the result of applying the sink crop rectangle to the sink
+ * format.
+ */
+ if (format->pad != MALI_C55_ISP_PAD_SINK_VIDEO)
+ return v4l2_subdev_get_fmt(sd, state, format);
+
+ sink_fmt = v4l2_subdev_state_get_format(state,
+ MALI_C55_ISP_PAD_SINK_VIDEO);
+
+ cfg = mali_c55_isp_get_mbus_config_by_code(fmt->code);
+ sink_fmt->code = cfg ? fmt->code : MEDIA_BUS_FMT_SRGGB20_1X20;
+
+ /*
+ * Clamp sizes in the accepted limits and clamp the crop rectangle in
+ * the new sizes.
+ */
+ sink_fmt->width = clamp(fmt->width, MALI_C55_MIN_WIDTH,
+ MALI_C55_MAX_WIDTH);
+ sink_fmt->height = clamp(fmt->height, MALI_C55_MIN_HEIGHT,
+ MALI_C55_MAX_HEIGHT);
+
+ *fmt = *sink_fmt;
+
+ crop = v4l2_subdev_state_get_crop(state, MALI_C55_ISP_PAD_SINK_VIDEO);
+ crop->left = 0;
+ crop->top = 0;
+ crop->width = sink_fmt->width;
+ crop->height = sink_fmt->height;
+
+ /*
+ * Propagate format to source pads. On the 'regular' output pad use
+ * the internal RGB processed format, while on the bypass pad simply
+ * replicate the ISP sink format. The sizes on both pads are the same as
+ * the ISP sink crop rectangle. The "field" and "colorspace" fields are
+ * set in .init_state() and fixed for both source pads, as is the "code"
+ * field for the processed data source pad.
+ */
+ src_fmt = v4l2_subdev_state_get_format(state,
+ MALI_C55_ISP_PAD_SOURCE_VIDEO);
+ src_fmt->width = crop->width;
+ src_fmt->height = crop->height;
+
+ src_fmt = v4l2_subdev_state_get_format(state,
+ MALI_C55_ISP_PAD_SOURCE_BYPASS);
+ src_fmt->code = sink_fmt->code;
+ src_fmt->width = crop->width;
+ src_fmt->height = crop->height;
+
+ return 0;
+}
+
+static int mali_c55_isp_get_selection(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_selection *sel)
+{
+ if (sel->pad != MALI_C55_ISP_PAD_SINK_VIDEO ||
+ sel->target != V4L2_SEL_TGT_CROP)
+ return -EINVAL;
+
+ sel->r = *v4l2_subdev_state_get_crop(state, MALI_C55_ISP_PAD_SINK_VIDEO);
+
+ return 0;
+}
+
+static int mali_c55_isp_set_selection(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_selection *sel)
+{
+ struct v4l2_mbus_framefmt *src_fmt;
+ const struct v4l2_mbus_framefmt *fmt;
+ struct v4l2_rect *crop;
+
+ if (sel->pad != MALI_C55_ISP_PAD_SINK_VIDEO ||
+ sel->target != V4L2_SEL_TGT_CROP)
+ return -EINVAL;
+
+ fmt = v4l2_subdev_state_get_format(state, MALI_C55_ISP_PAD_SINK_VIDEO);
+
+ sel->r.left = clamp_t(unsigned int, sel->r.left, 0, fmt->width);
+ sel->r.top = clamp_t(unsigned int, sel->r.top, 0, fmt->height);
+ sel->r.width = clamp_t(unsigned int, sel->r.width, MALI_C55_MIN_WIDTH,
+ fmt->width - sel->r.left);
+ sel->r.height = clamp_t(unsigned int, sel->r.height,
+ MALI_C55_MIN_HEIGHT,
+ fmt->height - sel->r.top);
+
+ crop = v4l2_subdev_state_get_crop(state, MALI_C55_ISP_PAD_SINK_VIDEO);
+ *crop = sel->r;
+
+ /*
+ * Propagate the crop rectangle sizes to the source pad format. The crop
+ * isn't propagated to the bypass source pad, because the bypassed data
+ * cannot be cropped.
+ */
+ src_fmt = v4l2_subdev_state_get_format(state,
+ MALI_C55_ISP_PAD_SOURCE_VIDEO);
+ src_fmt->width = crop->width;
+ src_fmt->height = crop->height;
+
+ return 0;
+}
+
+static int mali_c55_isp_enable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask)
+{
+ struct mali_c55_isp *isp = container_of(sd, struct mali_c55_isp, sd);
+ struct mali_c55 *mali_c55 = isp->mali_c55;
+ struct v4l2_subdev *src_sd;
+ struct media_pad *sink_pad;
+ int ret;
+
+ /*
+ * We have two source pads, both of which have only a single stream. The
+ * core v4l2 code already validated those parameters so we can just get
+ * on with starting the ISP.
+ */
+
+ sink_pad = &isp->pads[MALI_C55_ISP_PAD_SINK_VIDEO];
+ isp->remote_src = media_pad_remote_pad_unique(sink_pad);
+ if (IS_ERR(isp->remote_src)) {
+ dev_err(mali_c55->dev, "Failed to get source for ISP\n");
+ return PTR_ERR(isp->remote_src);
+ }
+
+ src_sd = media_entity_to_v4l2_subdev(isp->remote_src->entity);
+
+ isp->frame_sequence = 0;
+ ret = mali_c55_isp_start(mali_c55);
+ if (ret) {
+ dev_err(mali_c55->dev, "Failed to start ISP\n");
+ isp->remote_src = NULL;
+ return ret;
+ }
+
+ /*
+ * We only support a single input stream, so we can just enable the 1st
+ * entry in the streams mask.
+ */
+ ret = v4l2_subdev_enable_streams(src_sd, isp->remote_src->index, BIT(0));
+ if (ret) {
+ dev_err(mali_c55->dev, "Failed to start ISP source\n");
+ mali_c55_isp_stop(mali_c55);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int mali_c55_isp_disable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask)
+{
+ struct mali_c55_isp *isp = container_of(sd, struct mali_c55_isp, sd);
+ struct mali_c55 *mali_c55 = isp->mali_c55;
+ struct v4l2_subdev *src_sd;
+
+ if (isp->remote_src) {
+ src_sd = media_entity_to_v4l2_subdev(isp->remote_src->entity);
+ v4l2_subdev_disable_streams(src_sd, isp->remote_src->index,
+ BIT(0));
+ }
+ isp->remote_src = NULL;
+
+ mali_c55_isp_stop(mali_c55);
+
+ return 0;
+}
+
+static const struct v4l2_subdev_pad_ops mali_c55_isp_pad_ops = {
+ .enum_mbus_code = mali_c55_isp_enum_mbus_code,
+ .enum_frame_size = mali_c55_isp_enum_frame_size,
+ .get_fmt = v4l2_subdev_get_fmt,
+ .set_fmt = mali_c55_isp_set_fmt,
+ .get_selection = mali_c55_isp_get_selection,
+ .set_selection = mali_c55_isp_set_selection,
+ .link_validate = v4l2_subdev_link_validate_default,
+ .enable_streams = mali_c55_isp_enable_streams,
+ .disable_streams = mali_c55_isp_disable_streams,
+};
+
+void mali_c55_isp_queue_event_sof(struct mali_c55 *mali_c55)
+{
+ struct v4l2_event event = {
+ .type = V4L2_EVENT_FRAME_SYNC,
+ };
+
+ event.u.frame_sync.frame_sequence = mali_c55->isp.frame_sequence;
+ v4l2_event_queue(mali_c55->isp.sd.devnode, &event);
+}
+
+static int
+mali_c55_isp_subscribe_event(struct v4l2_subdev *sd, struct v4l2_fh *fh,
+ struct v4l2_event_subscription *sub)
+{
+ if (sub->type != V4L2_EVENT_FRAME_SYNC)
+ return -EINVAL;
+
+ /* V4L2_EVENT_FRAME_SYNC doesn't require an id, so zero should be set */
+ if (sub->id != 0)
+ return -EINVAL;
+
+ return v4l2_event_subscribe(fh, sub, 0, NULL);
+}
+
+static const struct v4l2_subdev_core_ops mali_c55_isp_core_ops = {
+ .subscribe_event = mali_c55_isp_subscribe_event,
+ .unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_ops mali_c55_isp_ops = {
+ .pad = &mali_c55_isp_pad_ops,
+ .core = &mali_c55_isp_core_ops,
+};
+
+static int mali_c55_isp_init_state(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state)
+{
+ struct v4l2_mbus_framefmt *sink_fmt, *src_fmt;
+ struct v4l2_rect *in_crop;
+
+ sink_fmt = v4l2_subdev_state_get_format(state,
+ MALI_C55_ISP_PAD_SINK_VIDEO);
+ src_fmt = v4l2_subdev_state_get_format(state,
+ MALI_C55_ISP_PAD_SOURCE_VIDEO);
+ in_crop = v4l2_subdev_state_get_crop(state,
+ MALI_C55_ISP_PAD_SINK_VIDEO);
+
+ sink_fmt->width = MALI_C55_DEFAULT_WIDTH;
+ sink_fmt->height = MALI_C55_DEFAULT_HEIGHT;
+ sink_fmt->field = V4L2_FIELD_NONE;
+ sink_fmt->code = MEDIA_BUS_FMT_SRGGB20_1X20;
+ sink_fmt->colorspace = V4L2_COLORSPACE_RAW;
+
+ *v4l2_subdev_state_get_format(state,
+ MALI_C55_ISP_PAD_SOURCE_BYPASS) = *sink_fmt;
+
+ src_fmt->width = MALI_C55_DEFAULT_WIDTH;
+ src_fmt->height = MALI_C55_DEFAULT_HEIGHT;
+ src_fmt->field = V4L2_FIELD_NONE;
+ src_fmt->code = MEDIA_BUS_FMT_RGB121212_1X36;
+ src_fmt->colorspace = V4L2_COLORSPACE_SRGB;
+
+ in_crop->top = 0;
+ in_crop->left = 0;
+ in_crop->width = MALI_C55_DEFAULT_WIDTH;
+ in_crop->height = MALI_C55_DEFAULT_HEIGHT;
+
+ return 0;
+}
+
+static const struct v4l2_subdev_internal_ops mali_c55_isp_internal_ops = {
+ .init_state = mali_c55_isp_init_state,
+};
+
+static const struct media_entity_operations mali_c55_isp_media_ops = {
+ .link_validate = v4l2_subdev_link_validate,
+};
+
+int mali_c55_register_isp(struct mali_c55 *mali_c55)
+{
+ struct mali_c55_isp *isp = &mali_c55->isp;
+ struct v4l2_subdev *sd = &isp->sd;
+ int ret;
+
+ isp->mali_c55 = mali_c55;
+
+ v4l2_subdev_init(sd, &mali_c55_isp_ops);
+ sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
+ sd->entity.ops = &mali_c55_isp_media_ops;
+ sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_ISP;
+ sd->internal_ops = &mali_c55_isp_internal_ops;
+ strscpy(sd->name, MALI_C55_DRIVER_NAME " isp", sizeof(sd->name));
+
+ isp->pads[MALI_C55_ISP_PAD_SINK_VIDEO].flags = MEDIA_PAD_FL_SINK |
+ MEDIA_PAD_FL_MUST_CONNECT;
+ isp->pads[MALI_C55_ISP_PAD_SOURCE_VIDEO].flags = MEDIA_PAD_FL_SOURCE;
+ isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE;
+
+ ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS,
+ isp->pads);
+ if (ret)
+ return ret;
+
+ ret = v4l2_subdev_init_finalize(sd);
+ if (ret)
+ goto err_cleanup_media_entity;
+
+ ret = v4l2_device_register_subdev(&mali_c55->v4l2_dev, sd);
+ if (ret)
+ goto err_cleanup_subdev;
+
+ mutex_init(&isp->capture_lock);
+
+ return 0;
+
+err_cleanup_subdev:
+ v4l2_subdev_cleanup(sd);
+err_cleanup_media_entity:
+ media_entity_cleanup(&sd->entity);
+ isp->mali_c55 = NULL;
+
+ return ret;
+}
+
+void mali_c55_unregister_isp(struct mali_c55 *mali_c55)
+{
+ struct mali_c55_isp *isp = &mali_c55->isp;
+
+ if (!isp->mali_c55)
+ return;
+
+ mutex_destroy(&isp->capture_lock);
+ v4l2_device_unregister_subdev(&isp->sd);
+ v4l2_subdev_cleanup(&isp->sd);
+ media_entity_cleanup(&isp->sd.entity);
+}
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
new file mode 100644
index 000000000000..0a391f8a043e
--- /dev/null
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
@@ -0,0 +1,313 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * ARM Mali-C55 ISP Driver - Register definitions
+ *
+ * Copyright (C) 2024 Ideas on Board Oy
+ */
+
+#ifndef _MALI_C55_REGISTERS_H
+#define _MALI_C55_REGISTERS_H
+
+#include <linux/bits.h>
+
+/* ISP Common 0x00000 - 0x000ff */
+
+#define MALI_C55_REG_API 0x00000
+#define MALI_C55_REG_PRODUCT 0x00004
+#define MALI_C55_REG_VERSION 0x00008
+#define MALI_C55_REG_REVISION 0x0000c
+#define MALI_C55_REG_PULSE_MODE 0x0003c
+#define MALI_C55_REG_INPUT_MODE_REQUEST 0x0009c
+#define MALI_C55_INPUT_SAFE_STOP 0x00
+#define MALI_C55_INPUT_SAFE_START 0x01
+#define MALI_C55_REG_MODE_STATUS 0x000a0
+#define MALI_C55_REG_INTERRUPT_MASK_VECTOR 0x00030
+#define MALI_C55_INTERRUPT_MASK_ALL GENMASK(31, 0)
+
+#define MALI_C55_REG_GLOBAL_MONITOR 0x00050
+
+#define MALI_C55_REG_GEN_VIDEO 0x00080
+#define MALI_C55_REG_GEN_VIDEO_ON_MASK BIT(0)
+#define MALI_C55_REG_GEN_VIDEO_MULTI_MASK BIT(1)
+#define MALI_C55_REG_GEN_PREFETCH_MASK GENMASK(31, 16)
+
+#define MALI_C55_REG_MCU_CONFIG 0x00020
+#define MALI_C55_REG_MCU_CONFIG_OVERRIDE_MASK BIT(0)
+#define MALI_C55_REG_MCU_CONFIG_WRITE_MASK BIT(1)
+#define MALI_C55_MCU_CONFIG_WRITE(x) ((x) << 1)
+#define MALI_C55_REG_MCU_CONFIG_WRITE_PING BIT(1)
+#define MALI_C55_REG_MCU_CONFIG_WRITE_PONG 0x00
+#define MALI_C55_REG_MULTI_CONTEXT_MODE_MASK BIT(8)
+#define MALI_C55_REG_PING_PONG_READ 0x00024
+#define MALI_C55_REG_PING_PONG_READ_MASK BIT(2)
+
+#define MALI_C55_REG_INTERRUPT_CLEAR_VECTOR 0x00034
+#define MALI_C55_REG_INTERRUPT_CLEAR 0x00040
+#define MALI_C55_REG_INTERRUPT_STATUS_VECTOR 0x00044
+
+enum mali_c55_interrupts {
+ MALI_C55_IRQ_ISP_START,
+ MALI_C55_IRQ_ISP_DONE,
+ MALI_C55_IRQ_MCM_ERROR,
+ MALI_C55_IRQ_BROKEN_FRAME_ERROR,
+ MALI_C55_IRQ_MET_AF_DONE,
+ MALI_C55_IRQ_MET_AEXP_DONE,
+ MALI_C55_IRQ_MET_AWB_DONE,
+ MALI_C55_IRQ_AEXP_1024_DONE,
+ MALI_C55_IRQ_IRIDIX_MET_DONE,
+ MALI_C55_IRQ_LUT_INIT_DONE,
+ MALI_C55_IRQ_FR_Y_DONE,
+ MALI_C55_IRQ_FR_UV_DONE,
+ MALI_C55_IRQ_DS_Y_DONE,
+ MALI_C55_IRQ_DS_UV_DONE,
+ MALI_C55_IRQ_LINEARIZATION_DONE,
+ MALI_C55_IRQ_RAW_FRONTEND_DONE,
+ MALI_C55_IRQ_NOISE_REDUCTION_DONE,
+ MALI_C55_IRQ_IRIDIX_DONE,
+ MALI_C55_IRQ_BAYER2RGB_DONE,
+ MALI_C55_IRQ_WATCHDOG_TIMER,
+ MALI_C55_IRQ_FRAME_COLLISION,
+ MALI_C55_IRQ_UNUSED,
+ MALI_C55_IRQ_DMA_ERROR,
+ MALI_C55_IRQ_INPUT_STOPPED,
+ MALI_C55_IRQ_MET_AWB_TARGET1_HIT,
+ MALI_C55_IRQ_MET_AWB_TARGET2_HIT,
+ MALI_C55_NUM_IRQ_BITS
+};
+
+#define MALI_C55_INTERRUPT_BIT(x) (0x01 << (x))
+
+#define MALI_C55_REG_GLOBAL_PARAMETER_STATUS 0x00068
+#define MALI_C55_GPS_PONG_FITTED BIT(0)
+#define MALI_C55_GPS_WDR_FITTED BIT(1)
+#define MALI_C55_GPS_COMPRESSION_FITTED BIT(2)
+#define MALI_C55_GPS_TEMPER_FITTED BIT(3)
+#define MALI_C55_GPS_SINTER_LITE_FITTED BIT(4)
+#define MALI_C55_GPS_SINTER_FITTED BIT(5)
+#define MALI_C55_GPS_IRIDIX_LTM_FITTED BIT(6)
+#define MALI_C55_GPS_IRIDIX_GTM_FITTED BIT(7)
+#define MALI_C55_GPS_CNR_FITTED BIT(8)
+#define MALI_C55_GPS_FRSCALER_FITTED BIT(9)
+#define MALI_C55_GPS_DS_PIPE_FITTED BIT(10)
+
+#define MALI_C55_REG_BLANKING 0x00084
+#define MALI_C55_REG_HBLANK_MASK GENMASK(15, 0)
+#define MALI_C55_REG_VBLANK_MASK GENMASK(31, 16)
+#define MALI_C55_VBLANK(x) ((x) << 16)
+
+#define MALI_C55_REG_HC_START 0x00088
+#define MALI_C55_HC_START(h) (((h) & 0xffff) << 16)
+#define MALI_C55_REG_HC_SIZE 0x0008c
+#define MALI_C55_HC_SIZE(h) ((h) & 0xffff)
+#define MALI_C55_REG_VC_START_SIZE 0x00094
+#define MALI_C55_VC_START(v) ((v) & 0xffff)
+#define MALI_C55_VC_SIZE(v) (((v) & 0xffff) << 16)
+
+/* Ping/Pong Configuration Space */
+#define MALI_C55_REG_BASE_ADDR 0x18e88
+#define MALI_C55_REG_BYPASS_0 0x18eac
+#define MALI_C55_REG_BYPASS_0_VIDEO_TEST BIT(0)
+#define MALI_C55_REG_BYPASS_0_INPUT_FMT BIT(1)
+#define MALI_C55_REG_BYPASS_0_DECOMPANDER BIT(2)
+#define MALI_C55_REG_BYPASS_0_SENSOR_OFFSET_WDR BIT(3)
+#define MALI_C55_REG_BYPASS_0_GAIN_WDR BIT(4)
+#define MALI_C55_REG_BYPASS_0_FRAME_STITCH BIT(5)
+#define MALI_C55_REG_BYPASS_1 0x18eb0
+#define MALI_C55_REG_BYPASS_1_DIGI_GAIN BIT(0)
+#define MALI_C55_REG_BYPASS_1_FE_SENSOR_OFFS BIT(1)
+#define MALI_C55_REG_BYPASS_1_FE_SQRT BIT(2)
+#define MALI_C55_REG_BYPASS_1_RAW_FE BIT(3)
+#define MALI_C55_REG_BYPASS_2 0x18eb8
+#define MALI_C55_REG_BYPASS_2_SINTER BIT(0)
+#define MALI_C55_REG_BYPASS_2_TEMPER BIT(1)
+#define MALI_C55_REG_BYPASS_3 0x18ebc
+#define MALI_C55_REG_BYPASS_3_SQUARE_BE BIT(0)
+#define MALI_C55_REG_BYPASS_3_SENSOR_OFFSET_PRE_SH BIT(1)
+#define MALI_C55_REG_BYPASS_3_MESH_SHADING BIT(3)
+#define MALI_C55_REG_BYPASS_3_WHITE_BALANCE BIT(4)
+#define MALI_C55_REG_BYPASS_3_IRIDIX BIT(5)
+#define MALI_C55_REG_BYPASS_3_IRIDIX_GAIN BIT(6)
+#define MALI_C55_REG_BYPASS_4 0x18ec0
+#define MALI_C55_REG_BYPASS_4_DEMOSAIC_RGB BIT(1)
+#define MALI_C55_REG_BYPASS_4_PF_CORRECTION BIT(3)
+#define MALI_C55_REG_BYPASS_4_CCM BIT(4)
+#define MALI_C55_REG_BYPASS_4_CNR BIT(5)
+#define MALI_C55_REG_FR_BYPASS 0x18ec4
+#define MALI_C55_REG_DS_BYPASS 0x18ec8
+#define MALI_C55_BYPASS_CROP BIT(0)
+#define MALI_C55_BYPASS_SCALER BIT(1)
+#define MALI_C55_BYPASS_GAMMA_RGB BIT(2)
+#define MALI_C55_BYPASS_SHARPEN BIT(3)
+#define MALI_C55_BYPASS_CS_CONV BIT(4)
+#define MALI_C55_REG_ISP_RAW_BYPASS 0x18ecc
+#define MALI_C55_ISP_RAW_BYPASS_BYPASS_MASK BIT(0)
+#define MALI_C55_ISP_RAW_BYPASS_FR_BYPASS_MASK GENMASK(9, 8)
+#define MALI_C55_ISP_RAW_BYPASS_RAW_FR_BYPASS (2 << 8)
+#define MALI_C55_ISP_RAW_BYPASS_RGB_FR_BYPASS (1 << 8)
+#define MALI_C55_ISP_RAW_BYPASS_DS_PIPE_DISABLE BIT(1)
+#define MALI_C55_ISP_RAW_BYPASS_RAW_BYPASS BIT(0)
+
+#define MALI_C55_REG_ACTIVE_WIDTH_MASK 0xffff
+#define MALI_C55_REG_ACTIVE_HEIGHT_MASK 0xffff0000
+#define MALI_C55_REG_BAYER_ORDER 0x18e8c
+#define MALI_C55_BAYER_ORDER_MASK GENMASK(1, 0)
+#define MALI_C55_BAYER_ORDER_RGGB 0
+#define MALI_C55_BAYER_ORDER_GRBG 1
+#define MALI_C55_BAYER_ORDER_GBRG 2
+#define MALI_C55_BAYER_ORDER_BGGR 3
+
+#define MALI_C55_REG_TPG_CH0 0x18ed8
+#define MALI_C55_TEST_PATTERN_ON_OFF BIT(0)
+#define MALI_C55_TEST_PATTERN_RGB_MASK BIT(1)
+#define MALI_C55_TEST_PATTERN_RGB(x) ((x) << 1)
+#define MALI_C55_REG_TPG_R_BACKGROUND 0x18ee0
+#define MALI_C55_REG_TPG_G_BACKGROUND 0x18ee4
+#define MALI_C55_REG_TPG_B_BACKGROUND 0x18ee8
+#define MALI_C55_TPG_BACKGROUND_MAX 0xfffff
+#define MALI_C55_REG_INPUT_WIDTH 0x18f98
+#define MALI_C55_INPUT_WIDTH_MASK GENMASK(18, 16)
+#define MALI_C55_INPUT_WIDTH_8BIT (0 << 16)
+#define MALI_C55_INPUT_WIDTH_10BIT (1 << 16)
+#define MALI_C55_INPUT_WIDTH_12BIT (2 << 16)
+#define MALI_C55_INPUT_WIDTH_14BIT (3 << 16)
+#define MALI_C55_INPUT_WIDTH_16BIT (4 << 16)
+#define MALI_C55_INPUT_WIDTH_20BIT (5 << 16)
+#define MALI_C55_REG_SPACE_SIZE 0x4000
+#define MALI_C55_REG_CONFIG_SPACES_OFFSET 0x0ab6c
+#define MALI_C55_CONFIG_SPACE_SIZE 0x1231c
+
+#define MALI_C55_REG_SINTER_CONFIG 0x19348
+#define MALI_C55_SINTER_VIEW_FILTER_MASK GENMASK(1, 0)
+#define MALI_C55_SINTER_SCALE_MODE_MASK GENMASK(3, 2)
+#define MALI_C55_SINTER_ENABLE_MASK BIT(4)
+#define MALI_C55_SINTER_FILTER_SELECT_MASK BIT(5)
+#define MALI_C55_SINTER_INT_SELECT_MASK BIT(6)
+#define MALI_C55_SINTER_RM_ENABLE_MASK BIT(7)
+
+/* Colour Correction Matrix Configuration */
+#define MALI_C55_REG_CCM_ENABLE 0x1b07c
+#define MALI_C55_CCM_ENABLE_MASK BIT(0)
+#define MALI_C55_REG_CCM_COEF_R_R 0x1b080
+#define MALI_C55_REG_CCM_COEF_R_G 0x1b084
+#define MALI_C55_REG_CCM_COEF_R_B 0x1b088
+#define MALI_C55_REG_CCM_COEF_G_R 0x1b090
+#define MALI_C55_REG_CCM_COEF_G_G 0x1b094
+#define MALI_C55_REG_CCM_COEF_G_B 0x1b098
+#define MALI_C55_REG_CCM_COEF_B_R 0x1b0a0
+#define MALI_C55_REG_CCM_COEF_B_G 0x1b0a4
+#define MALI_C55_REG_CCM_COEF_B_B 0x1b0a8
+#define MALI_C55_CCM_COEF_MASK GENMASK(12, 0)
+#define MALI_C55_REG_CCM_ANTIFOG_GAIN_R 0x1b0b0
+#define MALI_C55_REG_CCM_ANTIFOG_GAIN_G 0x1b0b4
+#define MALI_C55_REG_CCM_ANTIFOG_GAIN_B 0x1b0b8
+#define MALI_C55_CCM_ANTIFOG_GAIN_MASK GENMASK(11, 0)
+#define MALI_C55_REG_CCM_ANTIFOG_OFFSET_R 0x1b0c0
+#define MALI_C55_REG_CCM_ANTIFOG_OFFSET_G 0x1b0c4
+#define MALI_C55_REG_CCM_ANTIFOG_OFFSET_B 0x1b0c8
+#define MALI_C55_CCM_ANTIFOG_OFFSET_MASK GENMASK(11, 0)
+
+/*
+ * The Mali-C55 ISP has up to two output pipes; known as full resolution and
+ * down scaled. The register space for these is laid out identically, but offset
+ * by 372 bytes.
+ */
+#define MALI_C55_CAP_DEV_FR_REG_OFFSET 0x0
+#define MALI_C55_CAP_DEV_DS_REG_OFFSET 0x174
+
+#define MALI_C55_REG_CS_CONV_CONFIG 0x1c098
+#define MALI_C55_CS_CONV_MATRIX_MASK BIT(0)
+#define MALI_C55_CS_CONV_FILTER_MASK BIT(1)
+#define MALI_C55_CS_CONV_HORZ_DOWNSAMPLE_MASK BIT(2)
+#define MALI_C55_CS_CONV_VERT_DOWNSAMPLE_MASK BIT(3)
+#define MALI_C55_CS_CONV_FILTER_ENABLE (0x01 << 1)
+#define MALI_C55_CS_CONV_HORZ_DOWNSAMPLE_ENABLE (0x01 << 2)
+#define MALI_C55_CS_CONV_VERT_DOWNSAMPLE_ENABLE (0x01 << 3)
+#define MALI_C55_REG_Y_WRITER_MODE 0x1c0ec
+#define MALI_C55_REG_UV_WRITER_MODE 0x1c144
+#define MALI_C55_WRITER_MODE_MASK GENMASK(4, 0)
+#define MALI_C55_OUTPUT_DISABLED 0
+#define MALI_C55_OUTPUT_RGB32 1
+#define MALI_C55_OUTPUT_A2R10G10B10 2
+#define MALI_C55_OUTPUT_RGB565 3
+#define MALI_C55_OUTPUT_RGB24 4
+#define MALI_C55_OUTPUT_GEN32 5
+#define MALI_C55_OUTPUT_RAW16 6
+#define MALI_C55_OUTPUT_AYUV 8
+#define MALI_C55_OUTPUT_Y410 9
+#define MALI_C55_OUTPUT_YUY2 10
+#define MALI_C55_OUTPUT_UYVY 11
+#define MALI_C55_OUTPUT_Y210 12
+#define MALI_C55_OUTPUT_NV12_21 13
+#define MALI_C55_OUTPUT_YUV_420_422 17
+#define MALI_C55_OUTPUT_P210_P010 19
+#define MALI_C55_OUTPUT_YUV422 20
+#define MALI_C55_WRITER_SUBMODE_MASK GENMASK(7, 6)
+#define MALI_C55_WRITER_SUBMODE(x) ((x) << 6)
+#define MALI_C55_OUTPUT_PLANE_ALT0 0
+#define MALI_C55_OUTPUT_PLANE_ALT1 1
+#define MALI_C55_OUTPUT_PLANE_ALT2 2
+#define MALI_C55_WRITER_FRAME_WRITE_MASK BIT(9)
+#define MALI_C55_WRITER_FRAME_WRITE_ENABLE (0x01 << 9)
+#define MALI_C55_REG_ACTIVE_OUT_Y_SIZE 0x1c0f0
+#define MALI_C55_REG_ACTIVE_OUT_UV_SIZE 0x1c148
+#define MALI_C55_REG_ACTIVE_OUT_SIZE_W(w) ((w) << 0)
+#define MALI_C55_REG_ACTIVE_OUT_SIZE_H(h) ((h) << 16)
+#define MALI_C55_REG_Y_WRITER_BANKS_BASE 0x1c0f4
+#define MALI_C55_REG_Y_WRITER_BANKS_CONFIG 0x1c108
+#define MALI_C55_REG_Y_WRITER_MAX_BANKS_MASK GENMASK(2, 0)
+#define MALI_C55_REG_Y_WRITER_BANKS_RESTART BIT(3)
+#define MALI_C55_REG_Y_WRITER_OFFSET 0x1c10c
+#define MALI_C55_REG_UV_WRITER_BANKS_BASE 0x1c14c
+#define MALI_C55_REG_UV_WRITER_BANKS_CONFIG 0x1c160
+#define MALI_C55_REG_UV_WRITER_MAX_BANKS_MASK GENMASK(2, 0)
+#define MALI_C55_REG_UV_WRITER_BANKS_RESTART BIT(3)
+#define MALI_C55_REG_UV_WRITER_OFFSET 0x1c164
+
+#define MALI_C55_REG_TEST_GEN_CH0_OFF_ON
+#define MALI_C55_REG_TEST_GEN_CH0_PATTERN_TYPE 0x18edc
+
+#define MALI_C55_REG_CROP_EN 0x1c028
+#define MALI_C55_CROP_ENABLE BIT(0)
+#define MALI_C55_REG_CROP_X_START 0x1c02c
+#define MALI_C55_REG_CROP_Y_START 0x1c030
+#define MALI_C55_REG_CROP_X_SIZE 0x1c034
+#define MALI_C55_REG_CROP_Y_SIZE 0x1c038
+#define MALI_C55_REG_SCALER_TIMEOUT_EN 0x1c040
+#define MALI_C55_SCALER_TIMEOUT_EN BIT(4)
+#define MALI_C55_SCALER_TIMEOUT(t) ((t) << 16)
+#define MALI_C55_REG_SCALER_IN_WIDTH 0x1c044
+#define MALI_C55_REG_SCALER_IN_HEIGHT 0x1c048
+#define MALI_C55_REG_SCALER_OUT_WIDTH 0x1c04c
+#define MALI_C55_REG_SCALER_OUT_HEIGHT 0x1c050
+#define MALI_C55_REG_SCALER_HFILT_TINC 0x1c054
+#define MALI_C55_REG_SCALER_HFILT_COEF 0x1c058
+#define MALI_C55_REG_SCALER_VFILT_TINC 0x1c05c
+#define MALI_C55_REG_SCALER_VFILT_COEF 0x1c060
+
+#define MALI_C55_REG_GAMMA_RGB_ENABLE 0x1c064
+#define MALI_C55_GAMMA_ENABLE_MASK BIT(0)
+#define MALI_C55_REG_GAMMA_GAINS_1 0x1c068
+#define MALI_C55_GAMMA_GAIN_R_MASK GENMASK(11, 0)
+#define MALI_C55_GAMMA_GAIN_G_MASK GENMASK(27, 16)
+#define MALI_C55_REG_GAMMA_GAINS_2 0x1c06c
+#define MALI_C55_GAMMA_GAIN_B_MASK GENMASK(11, 0)
+#define MALI_C55_REG_GAMMA_OFFSETS_1 0x1c070
+#define MALI_C55_GAMMA_OFFSET_R_MASK GENMASK(11, 0)
+#define MALI_C55_GAMMA_OFFSET_G_MASK GENMASK(27, 16)
+#define MALI_C55_REG_GAMMA_OFFSETS_2 0x1c074
+#define MALI_C55_GAMMA_OFFSET_B_MASK GENMASK(11, 0)
+
+/*
+ * A re-definition of an above register. These will usually be written on a per
+ * capture device basis and handled with mali_c55_cap_dev_write(), but on
+ * startup is written by core.c
+ */
+#define MALI_C55_REG_FR_GAMMA_RGB_ENABLE 0x1c064
+#define MALI_C55_REG_DS_GAMMA_RGB_ENABLE 0x1c1d8
+
+#define MALI_C55_REG_FR_SCALER_HFILT 0x34a8
+#define MALI_C55_REG_FR_SCALER_VFILT 0x44a8
+#define MALI_C55_REG_DS_SCALER_HFILT 0x14a8
+#define MALI_C55_REG_DS_SCALER_VFILT 0x24a8
+
+#endif /* _MALI_C55_REGISTERS_H */
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c b/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
new file mode 100644
index 000000000000..b63a94cbe2fc
--- /dev/null
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
@@ -0,0 +1,1096 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM Mali-C55 ISP Driver - Image signal processor
+ *
+ * Copyright (C) 2024 Ideas on Board Oy
+ */
+
+#include <linux/math.h>
+#include <linux/minmax.h>
+
+#include <media/media-entity.h>
+#include <media/v4l2-subdev.h>
+
+#include "mali-c55-common.h"
+#include "mali-c55-registers.h"
+
+/* Scaling factor in Q4.20 format. */
+#define MALI_C55_RSZ_SCALER_FACTOR (1U << 20)
+
+#define MALI_C55_RSZ_COEFS_BANKS 8
+#define MALI_C55_RSZ_COEFS_ENTRIES 64
+
+static inline struct mali_c55_resizer *
+sd_to_mali_c55_rsz(struct v4l2_subdev *sd)
+{
+ return container_of(sd, struct mali_c55_resizer, sd);
+}
+
+static const unsigned int
+mali_c55_rsz_filter_coeffs_h[MALI_C55_RSZ_COEFS_BANKS]
+ [MALI_C55_RSZ_COEFS_ENTRIES] = {
+ { /* Bank 0 */
+ 0x24fc0000, 0x0000fc24, 0x27fc0000, 0x0000fc21,
+ 0x28fc0000, 0x0000fd1f, 0x2cfb0000, 0x0000fd1c,
+ 0x2efb0000, 0x0000fd1a, 0x30fb0000, 0x0000fe17,
+ 0x32fb0000, 0x0000fe15, 0x35fb0000, 0x0000fe12,
+ 0x35fc0000, 0x0000ff10, 0x37fc0000, 0x0000ff0e,
+ 0x39fc0000, 0x0000ff0c, 0x3afd0000, 0x0000ff0a,
+ 0x3afe0000, 0x00000008, 0x3cfe0000, 0x00000006,
+ 0x3dff0000, 0x00000004, 0x3d000000, 0x00000003,
+ 0x3c020000, 0x00000002, 0x3d030000, 0x00000000,
+ 0x3d040000, 0x000000ff, 0x3c060000, 0x000000fe,
+ 0x3a080000, 0x000000fe, 0x3a0aff00, 0x000000fd,
+ 0x390cff00, 0x000000fc, 0x370eff00, 0x000000fc,
+ 0x3510ff00, 0x000000fc, 0x3512fe00, 0x000000fb,
+ 0x3215fe00, 0x000000fb, 0x3017fe00, 0x000000fb,
+ 0x2e1afd00, 0x000000fb, 0x2c1cfd00, 0x000000fb,
+ 0x281ffd00, 0x000000fc, 0x2721fc00, 0x000000fc,
+ },
+ { /* Bank 1 */
+ 0x25fb0000, 0x0000fb25, 0x27fb0000, 0x0000fb23,
+ 0x29fb0000, 0x0000fb21, 0x2afc0000, 0x0000fb1f,
+ 0x2cfc0000, 0x0000fb1d, 0x2efc0000, 0x0000fb1b,
+ 0x2ffd0000, 0x0000fb19, 0x2ffe0000, 0x0000fc17,
+ 0x31fe0000, 0x0000fc15, 0x32ff0000, 0x0000fc13,
+ 0x3400ff00, 0x0000fc11, 0x3301ff00, 0x0000fd10,
+ 0x3402ff00, 0x0000fd0e, 0x3503ff00, 0x0000fd0c,
+ 0x3505ff00, 0x0000fd0a, 0x3506fe00, 0x0000fe09,
+ 0x3607fe00, 0x0000fe07, 0x3509fe00, 0x0000fe06,
+ 0x350afd00, 0x0000ff05, 0x350cfd00, 0x0000ff03,
+ 0x340efd00, 0x0000ff02, 0x3310fd00, 0x0000ff01,
+ 0x3411fc00, 0x0000ff00, 0x3213fc00, 0x000000ff,
+ 0x3115fc00, 0x000000fe, 0x2f17fc00, 0x000000fe,
+ 0x2f19fb00, 0x000000fd, 0x2e1bfb00, 0x000000fc,
+ 0x2c1dfb00, 0x000000fc, 0x2a1ffb00, 0x000000fc,
+ 0x2921fb00, 0x000000fb, 0x2723fb00, 0x000000fb,
+ },
+ { /* Bank 2 */
+ 0x1f010000, 0x0000011f, 0x21010000, 0x0000001e,
+ 0x21020000, 0x0000001d, 0x22020000, 0x0000001c,
+ 0x23030000, 0x0000ff1b, 0x2404ff00, 0x0000ff1a,
+ 0x2504ff00, 0x0000ff19, 0x2505ff00, 0x0000ff18,
+ 0x2606ff00, 0x0000fe17, 0x2607ff00, 0x0000fe16,
+ 0x2708ff00, 0x0000fe14, 0x2709ff00, 0x0000fe13,
+ 0x270aff00, 0x0000fe12, 0x280bfe00, 0x0000fe11,
+ 0x280cfe00, 0x0000fe10, 0x280dfe00, 0x0000fe0f,
+ 0x280efe00, 0x0000fe0e, 0x280ffe00, 0x0000fe0d,
+ 0x2810fe00, 0x0000fe0c, 0x2811fe00, 0x0000fe0b,
+ 0x2712fe00, 0x0000ff0a, 0x2713fe00, 0x0000ff09,
+ 0x2714fe00, 0x0000ff08, 0x2616fe00, 0x0000ff07,
+ 0x2617fe00, 0x0000ff06, 0x2518ff00, 0x0000ff05,
+ 0x2519ff00, 0x0000ff04, 0x241aff00, 0x0000ff04,
+ 0x231bff00, 0x00000003, 0x221c0000, 0x00000002,
+ 0x211d0000, 0x00000002, 0x211e0000, 0x00000001,
+ },
+ { /* Bank 3 */
+ 0x1b06ff00, 0x00ff061b, 0x1b07ff00, 0x00ff061a,
+ 0x1c07ff00, 0x00ff051a, 0x1c08ff00, 0x00ff0519,
+ 0x1c09ff00, 0x00ff0419, 0x1d09ff00, 0x00ff0418,
+ 0x1e0aff00, 0x00ff0317, 0x1e0aff00, 0x00ff0317,
+ 0x1e0bff00, 0x00ff0316, 0x1f0cff00, 0x00ff0215,
+ 0x1e0cff00, 0x00000215, 0x1e0dff00, 0x00000214,
+ 0x1e0e0000, 0x00000113, 0x1e0e0000, 0x00000113,
+ 0x1e0f0000, 0x00000112, 0x1f100000, 0x00000011,
+ 0x20100000, 0x00000010, 0x1f110000, 0x00000010,
+ 0x1e120100, 0x0000000f, 0x1e130100, 0x0000000e,
+ 0x1e130100, 0x0000000e, 0x1e140200, 0x0000ff0d,
+ 0x1e150200, 0x0000ff0c, 0x1f1502ff, 0x0000ff0c,
+ 0x1e1603ff, 0x0000ff0b, 0x1e1703ff, 0x0000ff0a,
+ 0x1e1703ff, 0x0000ff0a, 0x1d1804ff, 0x0000ff09,
+ 0x1c1904ff, 0x0000ff09, 0x1c1905ff, 0x0000ff08,
+ 0x1c1a05ff, 0x0000ff07, 0x1b1a06ff, 0x0000ff07,
+ },
+ { /* Bank 4 */
+ 0x17090000, 0x00000917, 0x18090000, 0x00000916,
+ 0x170a0100, 0x00000816, 0x170a0100, 0x00000816,
+ 0x180b0100, 0x00000715, 0x180b0100, 0x00000715,
+ 0x170c0100, 0x00000715, 0x190c0100, 0x00000614,
+ 0x180d0100, 0x00000614, 0x190d0200, 0x00000513,
+ 0x180e0200, 0x00000513, 0x180e0200, 0x00000513,
+ 0x1a0e0200, 0x00000412, 0x190f0200, 0x00000412,
+ 0x190f0300, 0x00000411, 0x18100300, 0x00000411,
+ 0x1a100300, 0x00000310, 0x18110400, 0x00000310,
+ 0x19110400, 0x0000030f, 0x19120400, 0x0000020f,
+ 0x1a120400, 0x0000020e, 0x18130500, 0x0000020e,
+ 0x18130500, 0x0000020e, 0x19130500, 0x0000020d,
+ 0x18140600, 0x0000010d, 0x19140600, 0x0000010c,
+ 0x17150700, 0x0000010c, 0x18150700, 0x0000010b,
+ 0x18150700, 0x0000010b, 0x17160800, 0x0000010a,
+ 0x17160800, 0x0000010a, 0x18160900, 0x00000009,
+ },
+ { /* Bank 5 */
+ 0x120b0300, 0x00030b12, 0x120c0300, 0x00030b11,
+ 0x110c0400, 0x00030b11, 0x110c0400, 0x00030b11,
+ 0x130c0400, 0x00020a11, 0x120d0400, 0x00020a11,
+ 0x110d0500, 0x00020a11, 0x110d0500, 0x00020a11,
+ 0x130d0500, 0x00010911, 0x130e0500, 0x00010910,
+ 0x120e0600, 0x00010910, 0x120e0600, 0x00010910,
+ 0x130e0600, 0x00010810, 0x120f0600, 0x00010810,
+ 0x120f0700, 0x00000810, 0x130f0700, 0x0000080f,
+ 0x140f0700, 0x0000070f, 0x130f0800, 0x0000070f,
+ 0x12100800, 0x0000070f, 0x12100801, 0x0000060f,
+ 0x13100801, 0x0000060e, 0x12100901, 0x0000060e,
+ 0x12100901, 0x0000060e, 0x13100901, 0x0000050e,
+ 0x13110901, 0x0000050d, 0x11110a02, 0x0000050d,
+ 0x11110a02, 0x0000050d, 0x12110a02, 0x0000040d,
+ 0x13110a02, 0x0000040c, 0x11110b03, 0x0000040c,
+ 0x11110b03, 0x0000040c, 0x12110b03, 0x0000030c,
+ },
+ { /* Bank 6 */
+ 0x0b0a0805, 0x00080a0c, 0x0b0a0805, 0x00080a0c,
+ 0x0c0a0805, 0x00080a0b, 0x0c0a0805, 0x00080a0b,
+ 0x0d0a0805, 0x00070a0b, 0x0d0a0805, 0x00070a0b,
+ 0x0d0a0805, 0x00070a0b, 0x0c0a0806, 0x00070a0b,
+ 0x0b0b0806, 0x00070a0b, 0x0c0b0806, 0x0007090b,
+ 0x0b0b0906, 0x0007090b, 0x0b0b0906, 0x0007090b,
+ 0x0b0b0906, 0x0007090b, 0x0b0b0906, 0x0007090b,
+ 0x0b0b0906, 0x0007090b, 0x0c0b0906, 0x0006090b,
+ 0x0c0b0906, 0x0006090b, 0x0c0b0906, 0x0006090b,
+ 0x0b0b0907, 0x0006090b, 0x0b0b0907, 0x0006090b,
+ 0x0b0b0907, 0x0006090b, 0x0b0b0907, 0x0006090b,
+ 0x0b0b0907, 0x0006090b, 0x0c0b0907, 0x0006080b,
+ 0x0b0b0a07, 0x0006080b, 0x0c0b0a07, 0x0006080a,
+ 0x0d0b0a07, 0x0005080a, 0x0d0b0a07, 0x0005080a,
+ 0x0d0b0a07, 0x0005080a, 0x0c0b0a08, 0x0005080a,
+ 0x0c0b0a08, 0x0005080a, 0x0c0b0a08, 0x0005080a,
+ },
+ { /* Bank 7 */
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
+ }
+};
+
+static const unsigned int
+mali_c55_rsz_filter_coeffs_v[MALI_C55_RSZ_COEFS_BANKS]
+ [MALI_C55_RSZ_COEFS_ENTRIES] = {
+ { /* Bank 0 */
+ 0x2424fc00, 0x000000fc, 0x2721fc00, 0x000000fc,
+ 0x281ffd00, 0x000000fc, 0x2c1cfd00, 0x000000fb,
+ 0x2e1afd00, 0x000000fb, 0x3017fe00, 0x000000fb,
+ 0x3215fe00, 0x000000fb, 0x3512fe00, 0x000000fb,
+ 0x3510ff00, 0x000000fc, 0x370eff00, 0x000000fc,
+ 0x390cff00, 0x000000fc, 0x3a0aff00, 0x000000fd,
+ 0x3a080000, 0x000000fe, 0x3c060000, 0x000000fe,
+ 0x3d040000, 0x000000ff, 0x3d030000, 0x00000000,
+ 0x3c020000, 0x00000002, 0x3d000000, 0x00000003,
+ 0x3dff0000, 0x00000004, 0x3cfe0000, 0x00000006,
+ 0x3afe0000, 0x00000008, 0x3afd0000, 0x0000ff0a,
+ 0x39fc0000, 0x0000ff0c, 0x37fc0000, 0x0000ff0e,
+ 0x35fc0000, 0x0000ff10, 0x35fb0000, 0x0000fe12,
+ 0x32fb0000, 0x0000fe15, 0x30fb0000, 0x0000fe17,
+ 0x2efb0000, 0x0000fd1a, 0x2cfb0000, 0x0000fd1c,
+ 0x28fc0000, 0x0000fd1f, 0x27fc0000, 0x0000fc21,
+ },
+ { /* Bank 1 */
+ 0x2525fb00, 0x000000fb, 0x2723fb00, 0x000000fb,
+ 0x2921fb00, 0x000000fb, 0x2a1ffb00, 0x000000fc,
+ 0x2c1dfb00, 0x000000fc, 0x2e1bfb00, 0x000000fc,
+ 0x2f19fb00, 0x000000fd, 0x2f17fc00, 0x000000fe,
+ 0x3115fc00, 0x000000fe, 0x3213fc00, 0x000000ff,
+ 0x3411fc00, 0x0000ff00, 0x3310fd00, 0x0000ff01,
+ 0x340efd00, 0x0000ff02, 0x350cfd00, 0x0000ff03,
+ 0x350afd00, 0x0000ff05, 0x3509fe00, 0x0000fe06,
+ 0x3607fe00, 0x0000fe07, 0x3506fe00, 0x0000fe09,
+ 0x3505ff00, 0x0000fd0a, 0x3503ff00, 0x0000fd0c,
+ 0x3402ff00, 0x0000fd0e, 0x3301ff00, 0x0000fd10,
+ 0x3400ff00, 0x0000fc11, 0x32ff0000, 0x0000fc13,
+ 0x31fe0000, 0x0000fc15, 0x2ffe0000, 0x0000fc17,
+ 0x2ffd0000, 0x0000fb19, 0x2efc0000, 0x0000fb1b,
+ 0x2cfc0000, 0x0000fb1d, 0x2afc0000, 0x0000fb1f,
+ 0x29fb0000, 0x0000fb21, 0x27fb0000, 0x0000fb23,
+ },
+ { /* Bank 2 */
+ 0x1f1f0100, 0x00000001, 0x211e0000, 0x00000001,
+ 0x211d0000, 0x00000002, 0x221c0000, 0x00000002,
+ 0x231bff00, 0x00000003, 0x241aff00, 0x0000ff04,
+ 0x2519ff00, 0x0000ff04, 0x2518ff00, 0x0000ff05,
+ 0x2617fe00, 0x0000ff06, 0x2616fe00, 0x0000ff07,
+ 0x2714fe00, 0x0000ff08, 0x2713fe00, 0x0000ff09,
+ 0x2712fe00, 0x0000ff0a, 0x2811fe00, 0x0000fe0b,
+ 0x2810fe00, 0x0000fe0c, 0x280ffe00, 0x0000fe0d,
+ 0x280efe00, 0x0000fe0e, 0x280dfe00, 0x0000fe0f,
+ 0x280cfe00, 0x0000fe10, 0x280bfe00, 0x0000fe11,
+ 0x270aff00, 0x0000fe12, 0x2709ff00, 0x0000fe13,
+ 0x2708ff00, 0x0000fe14, 0x2607ff00, 0x0000fe16,
+ 0x2606ff00, 0x0000fe17, 0x2505ff00, 0x0000ff18,
+ 0x2504ff00, 0x0000ff19, 0x2404ff00, 0x0000ff1a,
+ 0x23030000, 0x0000ff1b, 0x22020000, 0x0000001c,
+ 0x21020000, 0x0000001d, 0x21010000, 0x0000001e,
+ },
+ { /* Bank 3 */
+ 0x1b1b06ff, 0x0000ff06, 0x1b1a06ff, 0x0000ff07,
+ 0x1c1a05ff, 0x0000ff07, 0x1c1905ff, 0x0000ff08,
+ 0x1c1904ff, 0x0000ff09, 0x1d1804ff, 0x0000ff09,
+ 0x1e1703ff, 0x0000ff0a, 0x1e1703ff, 0x0000ff0a,
+ 0x1e1603ff, 0x0000ff0b, 0x1f1502ff, 0x0000ff0c,
+ 0x1e150200, 0x0000ff0c, 0x1e140200, 0x0000ff0d,
+ 0x1e130100, 0x0000000e, 0x1e130100, 0x0000000e,
+ 0x1e120100, 0x0000000f, 0x1f110000, 0x00000010,
+ 0x20100000, 0x00000010, 0x1f100000, 0x00000011,
+ 0x1e0f0000, 0x00000112, 0x1e0e0000, 0x00000113,
+ 0x1e0e0000, 0x00000113, 0x1e0dff00, 0x00000214,
+ 0x1e0cff00, 0x00000215, 0x1f0cff00, 0x00ff0215,
+ 0x1e0bff00, 0x00ff0316, 0x1e0aff00, 0x00ff0317,
+ 0x1e0aff00, 0x00ff0317, 0x1d09ff00, 0x00ff0418,
+ 0x1c09ff00, 0x00ff0419, 0x1c08ff00, 0x00ff0519,
+ 0x1c07ff00, 0x00ff051a, 0x1b07ff00, 0x00ff061a,
+ },
+ { /* Bank 4 */
+ 0x17170900, 0x00000009, 0x18160900, 0x00000009,
+ 0x17160800, 0x0000010a, 0x17160800, 0x0000010a,
+ 0x18150700, 0x0000010b, 0x18150700, 0x0000010b,
+ 0x17150700, 0x0000010c, 0x19140600, 0x0000010c,
+ 0x18140600, 0x0000010d, 0x19130500, 0x0000020d,
+ 0x18130500, 0x0000020e, 0x18130500, 0x0000020e,
+ 0x1a120400, 0x0000020e, 0x19120400, 0x0000020f,
+ 0x19110400, 0x0000030f, 0x18110400, 0x00000310,
+ 0x1a100300, 0x00000310, 0x18100300, 0x00000411,
+ 0x190f0300, 0x00000411, 0x190f0200, 0x00000412,
+ 0x1a0e0200, 0x00000412, 0x180e0200, 0x00000513,
+ 0x180e0200, 0x00000513, 0x190d0200, 0x00000513,
+ 0x180d0100, 0x00000614, 0x190c0100, 0x00000614,
+ 0x170c0100, 0x00000715, 0x180b0100, 0x00000715,
+ 0x180b0100, 0x00000715, 0x170a0100, 0x00000816,
+ 0x170a0100, 0x00000816, 0x18090000, 0x00000916,
+ },
+ { /* Bank 5 */
+ 0x12120b03, 0x0000030b, 0x12110b03, 0x0000030c,
+ 0x11110b03, 0x0000040c, 0x11110b03, 0x0000040c,
+ 0x13110a02, 0x0000040c, 0x12110a02, 0x0000040d,
+ 0x11110a02, 0x0000050d, 0x11110a02, 0x0000050d,
+ 0x13110901, 0x0000050d, 0x13100901, 0x0000050e,
+ 0x12100901, 0x0000060e, 0x12100901, 0x0000060e,
+ 0x13100801, 0x0000060e, 0x12100801, 0x0000060f,
+ 0x12100800, 0x0000070f, 0x130f0800, 0x0000070f,
+ 0x140f0700, 0x0000070f, 0x130f0700, 0x0000080f,
+ 0x120f0700, 0x00000810, 0x120f0600, 0x00010810,
+ 0x130e0600, 0x00010810, 0x120e0600, 0x00010910,
+ 0x120e0600, 0x00010910, 0x130e0500, 0x00010910,
+ 0x130d0500, 0x00010911, 0x110d0500, 0x00020a11,
+ 0x110d0500, 0x00020a11, 0x120d0400, 0x00020a11,
+ 0x130c0400, 0x00020a11, 0x110c0400, 0x00030b11,
+ 0x110c0400, 0x00030b11, 0x120c0300, 0x00030b11,
+ },
+ { /* Bank 6 */
+ 0x0b0c0a08, 0x0005080a, 0x0b0c0a08, 0x0005080a,
+ 0x0c0b0a08, 0x0005080a, 0x0c0b0a08, 0x0005080a,
+ 0x0d0b0a07, 0x0005080a, 0x0d0b0a07, 0x0005080a,
+ 0x0d0b0a07, 0x0005080a, 0x0c0b0a07, 0x0006080a,
+ 0x0b0b0a07, 0x0006080b, 0x0c0b0907, 0x0006080b,
+ 0x0b0b0907, 0x0006090b, 0x0b0b0907, 0x0006090b,
+ 0x0b0b0907, 0x0006090b, 0x0b0b0907, 0x0006090b,
+ 0x0b0b0907, 0x0006090b, 0x0c0b0906, 0x0006090b,
+ 0x0c0b0906, 0x0006090b, 0x0c0b0906, 0x0006090b,
+ 0x0b0b0906, 0x0007090b, 0x0b0b0906, 0x0007090b,
+ 0x0b0b0906, 0x0007090b, 0x0b0b0906, 0x0007090b,
+ 0x0b0b0906, 0x0007090b, 0x0c0b0806, 0x0007090b,
+ 0x0b0b0806, 0x00070a0b, 0x0c0a0806, 0x00070a0b,
+ 0x0d0a0805, 0x00070a0b, 0x0d0a0805, 0x00070a0b,
+ 0x0d0a0805, 0x00070a0b, 0x0c0a0805, 0x00080a0b,
+ 0x0c0a0805, 0x00080a0b, 0x0c0a0805, 0x00080a0b,
+ },
+ { /* Bank 7 */
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
+ }
+};
+
+static const struct mali_c55_rsz_coef_bank {
+ unsigned int top;
+ unsigned int bottom;
+} mali_c55_rsz_coef_banks[] = {
+ {
+ .top = 1000,
+ .bottom = 770,
+ }, {
+ .top = 769,
+ .bottom = 600,
+ }, {
+ .top = 599,
+ .bottom = 460,
+ }, {
+ .top = 459,
+ .bottom = 354,
+ }, {
+ .top = 353,
+ .bottom = 273,
+ }, {
+ .top = 272,
+ .bottom = 210,
+ }, {
+ .top = 209,
+ .bottom = 162,
+ }, {
+ .top = 161,
+ .bottom = 125,
+ },
+};
+
+/*
+ * Select the right filter coefficients bank based on the scaler input and the
+ * scaler output sizes ratio, set by the v4l2 crop and scale selection
+ * rectangles respectively.
+ */
+static unsigned int mali_c55_rsz_calculate_bank(struct mali_c55 *mali_c55,
+ unsigned int rsz_in,
+ unsigned int rsz_out)
+{
+ unsigned int rsz_ratio = (rsz_out * 1000U) / rsz_in;
+
+ for (unsigned int i = 0; i < ARRAY_SIZE(mali_c55_rsz_coef_banks); i++) {
+ if (rsz_ratio >= mali_c55_rsz_coef_banks[i].bottom &&
+ rsz_ratio <= mali_c55_rsz_coef_banks[i].top)
+ return i;
+ }
+
+ /*
+ * We shouldn't ever get here, in theory. As we have no good choices
+ * simply warn the user and use the first bank of coefficients.
+ */
+ dev_warn(mali_c55->dev, "scaling factor outside defined bounds\n");
+ return 0;
+}
+
+static const u32 rsz_non_bypass_src_fmts[] = {
+ MEDIA_BUS_FMT_RGB121212_1X36,
+ MEDIA_BUS_FMT_YUV10_1X30
+};
+
+static void mali_c55_resizer_program_coefficients(struct mali_c55_resizer *rsz)
+{
+ struct mali_c55 *mali_c55 = rsz->mali_c55;
+ unsigned int haddr = rsz->id == MALI_C55_RSZ_FR ?
+ MALI_C55_REG_FR_SCALER_HFILT :
+ MALI_C55_REG_DS_SCALER_HFILT;
+ unsigned int vaddr = rsz->id == MALI_C55_RSZ_FR ?
+ MALI_C55_REG_FR_SCALER_VFILT :
+ MALI_C55_REG_DS_SCALER_VFILT;
+
+ for (unsigned int i = 0; i < MALI_C55_RSZ_COEFS_BANKS; i++) {
+ for (unsigned int j = 0; j < MALI_C55_RSZ_COEFS_ENTRIES; j++) {
+ mali_c55_write(mali_c55, haddr,
+ mali_c55_rsz_filter_coeffs_h[i][j]);
+ mali_c55_write(mali_c55, vaddr,
+ mali_c55_rsz_filter_coeffs_v[i][j]);
+
+ haddr += sizeof(u32);
+ vaddr += sizeof(u32);
+ }
+ }
+}
+
+static int mali_c55_rsz_program_crop(struct mali_c55_resizer *rsz,
+ struct v4l2_subdev_state *state)
+{
+ const struct v4l2_mbus_framefmt *fmt;
+ const struct v4l2_rect *crop;
+
+ /* Verify if crop should be enabled. */
+ fmt = v4l2_subdev_state_get_format(state, MALI_C55_RSZ_SINK_PAD, 0);
+ crop = v4l2_subdev_state_get_crop(state, MALI_C55_RSZ_SINK_PAD, 0);
+
+ if (fmt->width == crop->width && fmt->height == crop->height)
+ return MALI_C55_BYPASS_CROP;
+
+ mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_CROP_X_START,
+ crop->left);
+ mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_CROP_Y_START,
+ crop->top);
+ mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_CROP_X_SIZE,
+ crop->width);
+ mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_CROP_Y_SIZE,
+ crop->height);
+
+ mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_CROP_EN,
+ MALI_C55_CROP_ENABLE);
+
+ return 0;
+}
+
+static int mali_c55_rsz_program_resizer(struct mali_c55_resizer *rsz,
+ struct v4l2_subdev_state *state)
+{
+ struct mali_c55 *mali_c55 = rsz->mali_c55;
+ const struct v4l2_rect *crop, *scale;
+ unsigned int h_bank, v_bank;
+ u64 h_scale, v_scale;
+
+ /* Verify if scaling should be enabled. */
+ crop = v4l2_subdev_state_get_crop(state, MALI_C55_RSZ_SINK_PAD, 0);
+ scale = v4l2_subdev_state_get_compose(state, MALI_C55_RSZ_SINK_PAD, 0);
+
+ if (crop->width == scale->width && crop->height == scale->height)
+ return MALI_C55_BYPASS_SCALER;
+
+ /* Program the scaler coefficients if the scaler is in use. */
+ mali_c55_resizer_program_coefficients(rsz);
+
+ /* Program the V/H scaling factor in Q4.20 format. */
+ h_scale = crop->width * MALI_C55_RSZ_SCALER_FACTOR;
+ v_scale = crop->height * MALI_C55_RSZ_SCALER_FACTOR;
+
+ do_div(h_scale, scale->width);
+ do_div(v_scale, scale->height);
+
+ mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_IN_WIDTH,
+ crop->width);
+ mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_IN_HEIGHT,
+ crop->height);
+
+ mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_OUT_WIDTH,
+ scale->width);
+ mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_OUT_HEIGHT,
+ scale->height);
+
+ mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_HFILT_TINC,
+ h_scale);
+ mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_VFILT_TINC,
+ v_scale);
+
+ /* Select the scaler coefficients bank to use. */
+ h_bank = mali_c55_rsz_calculate_bank(mali_c55, crop->width,
+ scale->width);
+ mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_HFILT_COEF,
+ h_bank);
+
+ v_bank = mali_c55_rsz_calculate_bank(mali_c55, crop->height,
+ scale->height);
+ mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_VFILT_COEF,
+ v_bank);
+
+ return 0;
+}
+
+static void mali_c55_rsz_program(struct mali_c55_resizer *rsz,
+ struct v4l2_subdev_state *state)
+{
+ struct mali_c55 *mali_c55 = rsz->mali_c55;
+ u32 bypass = 0;
+
+ /* Verify if cropping and scaling should be enabled. */
+ bypass |= mali_c55_rsz_program_crop(rsz, state);
+ bypass |= mali_c55_rsz_program_resizer(rsz, state);
+
+ mali_c55_ctx_update_bits(mali_c55, rsz->id == MALI_C55_RSZ_FR ?
+ MALI_C55_REG_FR_BYPASS : MALI_C55_REG_DS_BYPASS,
+ MALI_C55_BYPASS_CROP | MALI_C55_BYPASS_SCALER,
+ bypass);
+}
+
+/*
+ * Inspect the routing table to know which of the two (mutually exclusive)
+ * routes is enabled and return the sink pad id of the active route.
+ */
+static unsigned int mali_c55_rsz_get_active_sink(struct v4l2_subdev_state *state)
+{
+ struct v4l2_subdev_krouting *routing = &state->routing;
+ struct v4l2_subdev_route *route;
+
+ /* A single route is enabled at a time. */
+ for_each_active_route(routing, route)
+ return route->sink_pad;
+
+ return MALI_C55_RSZ_SINK_PAD;
+}
+
+/*
+ * When operating in bypass mode, the ISP takes input in a 20-bit format, but
+ * can only output 16-bit RAW bayer data (with the 4 least significant bits from
+ * the input being lost). Return the 16-bit version of the 20-bit input formats.
+ */
+static u32 mali_c55_rsz_shift_mbus_code(u32 mbus_code)
+{
+ switch (mbus_code) {
+ case MEDIA_BUS_FMT_SBGGR20_1X20:
+ return MEDIA_BUS_FMT_SBGGR16_1X16;
+ case MEDIA_BUS_FMT_SGBRG20_1X20:
+ return MEDIA_BUS_FMT_SGBRG16_1X16;
+ case MEDIA_BUS_FMT_SGRBG20_1X20:
+ return MEDIA_BUS_FMT_SGRBG16_1X16;
+ case MEDIA_BUS_FMT_SRGGB20_1X20:
+ return MEDIA_BUS_FMT_SRGGB16_1X16;
+ }
+
+ return 0;
+}
+
+static int __mali_c55_rsz_set_routing(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ const struct v4l2_subdev_krouting *routing)
+{
+ struct mali_c55_resizer *rsz = sd_to_mali_c55_rsz(sd);
+ unsigned int active_sink = UINT_MAX;
+ struct v4l2_mbus_framefmt *src_fmt;
+ struct v4l2_rect *crop, *compose;
+ struct v4l2_subdev_route *route;
+ unsigned int active_routes = 0;
+ struct v4l2_mbus_framefmt *fmt;
+ int ret;
+
+ ret = v4l2_subdev_routing_validate(sd, routing, 0);
+ if (ret)
+ return ret;
+
+ /* Only a single route can be enabled at a time. */
+ for_each_active_route(routing, route) {
+ if (++active_routes > 1) {
+ dev_dbg(rsz->mali_c55->dev,
+ "Only one route can be active");
+ return -EINVAL;
+ }
+
+ active_sink = route->sink_pad;
+ }
+ if (active_sink == UINT_MAX) {
+ dev_dbg(rsz->mali_c55->dev, "One route has to be active");
+ return -EINVAL;
+ }
+
+ ret = v4l2_subdev_set_routing(sd, state, routing);
+ if (ret) {
+ dev_dbg(rsz->mali_c55->dev, "Failed to set routing\n");
+ return ret;
+ }
+
+ fmt = v4l2_subdev_state_get_format(state, active_sink, 0);
+ crop = v4l2_subdev_state_get_crop(state, active_sink, 0);
+ compose = v4l2_subdev_state_get_compose(state, active_sink, 0);
+
+ fmt->width = MALI_C55_DEFAULT_WIDTH;
+ fmt->height = MALI_C55_DEFAULT_HEIGHT;
+ fmt->colorspace = V4L2_COLORSPACE_SRGB;
+ fmt->field = V4L2_FIELD_NONE;
+
+ if (active_sink == MALI_C55_RSZ_SINK_PAD) {
+ fmt->code = MEDIA_BUS_FMT_RGB121212_1X36;
+
+ crop->left = 0;
+ crop->top = 0;
+ crop->width = MALI_C55_DEFAULT_WIDTH;
+ crop->height = MALI_C55_DEFAULT_HEIGHT;
+
+ *compose = *crop;
+ } else {
+ fmt->code = MEDIA_BUS_FMT_SRGGB20_1X20;
+ }
+
+ /* Propagate the format to the source pad */
+ src_fmt = v4l2_subdev_state_get_format(state, MALI_C55_RSZ_SOURCE_PAD,
+ 0);
+ *src_fmt = *fmt;
+
+ /* In the event this is the bypass pad the mbus code needs correcting */
+ if (active_sink == MALI_C55_RSZ_SINK_BYPASS_PAD)
+ src_fmt->code = mali_c55_rsz_shift_mbus_code(src_fmt->code);
+
+ return 0;
+}
+
+static int mali_c55_rsz_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ struct v4l2_mbus_framefmt *sink_fmt;
+ const struct mali_c55_isp_fmt *fmt;
+ u32 sink_pad;
+
+ switch (code->pad) {
+ case MALI_C55_RSZ_SINK_PAD:
+ if (code->index)
+ return -EINVAL;
+
+ code->code = MEDIA_BUS_FMT_RGB121212_1X36;
+
+ return 0;
+ case MALI_C55_RSZ_SOURCE_PAD:
+ sink_pad = mali_c55_rsz_get_active_sink(state);
+ sink_fmt = v4l2_subdev_state_get_format(state, sink_pad, 0);
+
+ /*
+ * If the active route is from the Bypass sink pad, then the
+ * source pad is a simple passthrough of the sink format,
+ * downshifted to 16-bits.
+ */
+
+ if (sink_pad == MALI_C55_RSZ_SINK_BYPASS_PAD) {
+ if (code->index)
+ return -EINVAL;
+
+ code->code = mali_c55_rsz_shift_mbus_code(sink_fmt->code);
+ if (!code->code)
+ return -EINVAL;
+
+ return 0;
+ }
+
+ /*
+ * If the active route is from the non-bypass sink then we can
+ * select either RGB or conversion to YUV.
+ */
+
+ if (code->index >= ARRAY_SIZE(rsz_non_bypass_src_fmts))
+ return -EINVAL;
+
+ code->code = rsz_non_bypass_src_fmts[code->index];
+
+ return 0;
+ case MALI_C55_RSZ_SINK_BYPASS_PAD:
+ fmt = mali_c55_isp_get_mbus_config_by_index(code->index);
+ if (fmt) {
+ code->code = fmt->code;
+ return 0;
+ }
+
+ break;
+ }
+
+ return -EINVAL;
+}
+
+static int mali_c55_rsz_enum_frame_size(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_frame_size_enum *fse)
+{
+ if (fse->index)
+ return -EINVAL;
+
+ fse->max_width = MALI_C55_MAX_WIDTH;
+ fse->max_height = MALI_C55_MAX_HEIGHT;
+ fse->min_width = MALI_C55_MIN_WIDTH;
+ fse->min_height = MALI_C55_MIN_HEIGHT;
+
+ return 0;
+}
+
+static int mali_c55_rsz_set_sink_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format)
+{
+ struct v4l2_mbus_framefmt *fmt = &format->format;
+ unsigned int active_sink;
+ struct v4l2_rect *rect;
+
+ /*
+ * Clamp to min/max and then reset crop and compose rectangles to the
+ * newly applied size.
+ */
+ fmt->width = clamp_t(unsigned int, fmt->width, MALI_C55_MIN_WIDTH,
+ MALI_C55_MAX_WIDTH);
+ fmt->height = clamp_t(unsigned int, fmt->height, MALI_C55_MIN_HEIGHT,
+ MALI_C55_MAX_HEIGHT);
+
+ rect = v4l2_subdev_state_get_crop(state, format->pad);
+ rect->left = 0;
+ rect->top = 0;
+ rect->width = fmt->width;
+ rect->height = fmt->height;
+
+ rect = v4l2_subdev_state_get_compose(state, format->pad);
+ rect->left = 0;
+ rect->top = 0;
+ rect->width = fmt->width;
+ rect->height = fmt->height;
+
+ if (format->pad == MALI_C55_RSZ_SINK_BYPASS_PAD) {
+ /*
+ * Make sure the media bus code is one of the supported
+ * ISP input media bus codes. Default it to SRGGB otherwise.
+ */
+ if (!mali_c55_isp_get_mbus_config_by_code(fmt->code))
+ fmt->code = MEDIA_BUS_FMT_SRGGB20_1X20;
+ } else {
+ fmt->code = MEDIA_BUS_FMT_RGB121212_1X36;
+ }
+
+ *v4l2_subdev_state_get_format(state, format->pad, 0) = *fmt;
+
+ /* If format->pad is routed to the source pad, propagate the format. */
+ active_sink = mali_c55_rsz_get_active_sink(state);
+ if (active_sink == format->pad) {
+ /* If the bypass route is used, downshift the code to 16bpp. */
+ if (active_sink == MALI_C55_RSZ_SINK_BYPASS_PAD)
+ fmt->code = mali_c55_rsz_shift_mbus_code(fmt->code);
+
+ *v4l2_subdev_state_get_format(state,
+ MALI_C55_RSZ_SOURCE_PAD, 0) = *fmt;
+ }
+
+ return 0;
+}
+
+static int mali_c55_rsz_set_source_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format)
+{
+ struct v4l2_mbus_framefmt *fmt = &format->format;
+ struct v4l2_mbus_framefmt *sink_fmt;
+ struct v4l2_rect *sink_compose;
+ unsigned int active_sink;
+
+ active_sink = mali_c55_rsz_get_active_sink(state);
+ sink_fmt = v4l2_subdev_state_get_format(state, active_sink, 0);
+ sink_compose = v4l2_subdev_state_get_compose(state, active_sink, 0);
+
+ /*
+ * The source pad format sizes come directly from the active sink pad
+ * compose rectangle.
+ */
+ fmt->width = sink_compose->width;
+ fmt->height = sink_compose->height;
+
+ if (active_sink == MALI_C55_RSZ_SINK_PAD) {
+ /*
+ * Regular processing pipe: RGB121212 can be color-space
+ * converted to YUV101010.
+ */
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(rsz_non_bypass_src_fmts); i++) {
+ if (fmt->code == rsz_non_bypass_src_fmts[i])
+ break;
+ }
+
+ if (i == ARRAY_SIZE(rsz_non_bypass_src_fmts))
+ fmt->code = MEDIA_BUS_FMT_RGB121212_1X36;
+ } else {
+ /*
+ * Bypass pipe: the source format is the same as the bypass
+ * sink pad downshifted to 16bpp.
+ */
+ fmt->code = mali_c55_rsz_shift_mbus_code(sink_fmt->code);
+ }
+
+ *v4l2_subdev_state_get_format(state, MALI_C55_RSZ_SOURCE_PAD) = *fmt;
+
+ return 0;
+}
+
+static int mali_c55_rsz_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format)
+{
+ /*
+ * On sink pads fmt is either fixed for the 'regular' processing
+ * pad or a RAW format or 20-bit wide RGB/YUV format for the FR bypass
+ * pad.
+ *
+ * On source pad sizes are the result of crop+compose on the sink
+ * pad sizes, while the format depends on the active route.
+ */
+
+ if (format->pad == MALI_C55_RSZ_SINK_PAD ||
+ format->pad == MALI_C55_RSZ_SINK_BYPASS_PAD)
+ return mali_c55_rsz_set_sink_fmt(sd, state, format);
+
+ return mali_c55_rsz_set_source_fmt(sd, state, format);
+}
+
+static int mali_c55_rsz_get_selection(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_selection *sel)
+{
+ if (sel->pad != MALI_C55_RSZ_SINK_PAD)
+ return -EINVAL;
+
+ if (sel->target != V4L2_SEL_TGT_CROP &&
+ sel->target != V4L2_SEL_TGT_COMPOSE)
+ return -EINVAL;
+
+ sel->r = sel->target == V4L2_SEL_TGT_CROP
+ ? *v4l2_subdev_state_get_crop(state, MALI_C55_RSZ_SINK_PAD)
+ : *v4l2_subdev_state_get_compose(state, MALI_C55_RSZ_SINK_PAD);
+
+ return 0;
+}
+
+static int mali_c55_rsz_set_selection(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_selection *sel)
+{
+ struct mali_c55_resizer *rsz = sd_to_mali_c55_rsz(sd);
+ struct v4l2_mbus_framefmt *sink_fmt;
+ struct v4l2_rect *crop, *compose;
+
+ if (sel->pad != MALI_C55_RSZ_SINK_PAD)
+ return -EINVAL;
+
+ if (sel->target != V4L2_SEL_TGT_CROP &&
+ sel->target != V4L2_SEL_TGT_COMPOSE)
+ return -EINVAL;
+
+ sink_fmt = v4l2_subdev_state_get_format(state, MALI_C55_RSZ_SINK_PAD);
+ crop = v4l2_subdev_state_get_crop(state, MALI_C55_RSZ_SINK_PAD);
+ compose = v4l2_subdev_state_get_compose(state, MALI_C55_RSZ_SINK_PAD);
+
+ /* During streaming, it is allowed to only change the crop rectangle. */
+ if (rsz->streaming && sel->target != V4L2_SEL_TGT_CROP)
+ return -EBUSY;
+
+ /*
+ * Update the desired target and then clamp the crop rectangle to the
+ * sink format sizes and the compose size to the crop rectangle.
+ */
+ if (sel->target == V4L2_SEL_TGT_CROP)
+ *crop = sel->r;
+ else
+ *compose = sel->r;
+
+ crop->left = clamp_t(unsigned int, crop->left, 0, sink_fmt->width);
+ crop->top = clamp_t(unsigned int, crop->top, 0, sink_fmt->height);
+ crop->width = clamp_t(unsigned int, crop->width, MALI_C55_MIN_WIDTH,
+ sink_fmt->width - crop->left);
+ crop->height = clamp_t(unsigned int, crop->height, MALI_C55_MIN_HEIGHT,
+ sink_fmt->height - crop->top);
+
+ if (rsz->streaming) {
+ /*
+ * It is not valid to apply at runtime a crop rectangle smaller
+ * than the compose one, as it requires re-programming the
+ * scaler output sizes and the output buffer sizes.
+ *
+ * Adjust the crop rectangle to be at least as large as the
+ * compose one if we're streaming and apply it immediately.
+ */
+ if (crop->width < compose->width)
+ crop->width = compose->width;
+ if (crop->height < compose->height)
+ crop->height = compose->height;
+
+ sel->r = *crop;
+
+ mali_c55_rsz_program(rsz, state);
+
+ return 0;
+ }
+
+ compose->left = 0;
+ compose->top = 0;
+ compose->width = clamp_t(unsigned int, compose->width, crop->width / 8,
+ crop->width);
+ compose->height = clamp_t(unsigned int, compose->height,
+ crop->height / 8, crop->height);
+
+ sel->r = sel->target == V4L2_SEL_TGT_CROP ? *crop : *compose;
+
+ return 0;
+}
+
+static int mali_c55_rsz_set_routing(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ enum v4l2_subdev_format_whence which,
+ struct v4l2_subdev_krouting *routing)
+{
+ if (which == V4L2_SUBDEV_FORMAT_ACTIVE &&
+ media_entity_is_streaming(&sd->entity))
+ return -EBUSY;
+
+ return __mali_c55_rsz_set_routing(sd, state, routing);
+}
+
+static int mali_c55_rsz_enable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask)
+{
+ struct mali_c55_resizer *rsz = sd_to_mali_c55_rsz(sd);
+ struct mali_c55 *mali_c55 = rsz->mali_c55;
+ unsigned int sink_pad;
+
+ sink_pad = mali_c55_rsz_get_active_sink(state);
+ if (sink_pad == MALI_C55_RSZ_SINK_BYPASS_PAD) {
+ /* Bypass FR pipe processing if the bypass route is active. */
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_ISP_RAW_BYPASS,
+ MALI_C55_ISP_RAW_BYPASS_FR_BYPASS_MASK,
+ MALI_C55_ISP_RAW_BYPASS_RAW_FR_BYPASS);
+ rsz->streaming = true;
+ return 0;
+ }
+
+ /* Disable bypass and use regular processing. */
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_ISP_RAW_BYPASS,
+ MALI_C55_ISP_RAW_BYPASS_FR_BYPASS_MASK, 0);
+ mali_c55_rsz_program(rsz, state);
+
+ rsz->streaming = true;
+
+ return 0;
+}
+
+static int mali_c55_rsz_disable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask)
+{
+ struct mali_c55_resizer *rsz = sd_to_mali_c55_rsz(sd);
+
+ rsz->streaming = false;
+ return 0;
+}
+
+static const struct v4l2_subdev_pad_ops mali_c55_resizer_pad_ops = {
+ .enum_mbus_code = mali_c55_rsz_enum_mbus_code,
+ .enum_frame_size = mali_c55_rsz_enum_frame_size,
+ .get_fmt = v4l2_subdev_get_fmt,
+ .set_fmt = mali_c55_rsz_set_fmt,
+ .get_selection = mali_c55_rsz_get_selection,
+ .set_selection = mali_c55_rsz_set_selection,
+ .set_routing = mali_c55_rsz_set_routing,
+ .enable_streams = mali_c55_rsz_enable_streams,
+ .disable_streams = mali_c55_rsz_disable_streams,
+};
+
+static const struct v4l2_subdev_ops mali_c55_resizer_ops = {
+ .pad = &mali_c55_resizer_pad_ops,
+};
+
+static int mali_c55_rsz_init_state(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state)
+{
+ struct mali_c55_resizer *rsz = sd_to_mali_c55_rsz(sd);
+ struct v4l2_subdev_route routes[2] = {
+ {
+ .sink_pad = MALI_C55_RSZ_SINK_PAD,
+ .source_pad = MALI_C55_RSZ_SOURCE_PAD,
+ .flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+ }, {
+ .sink_pad = MALI_C55_RSZ_SINK_BYPASS_PAD,
+ .source_pad = MALI_C55_RSZ_SOURCE_PAD,
+ },
+ };
+ struct v4l2_subdev_krouting routing = {
+ .num_routes = rsz->num_routes,
+ .routes = routes,
+ };
+
+ return __mali_c55_rsz_set_routing(sd, state, &routing);
+}
+
+static const struct v4l2_subdev_internal_ops mali_c55_resizer_internal_ops = {
+ .init_state = mali_c55_rsz_init_state,
+};
+
+static int mali_c55_register_resizer(struct mali_c55 *mali_c55,
+ struct mali_c55_resizer *rsz)
+{
+ struct v4l2_subdev *sd = &rsz->sd;
+ unsigned int num_pads;
+ int ret;
+
+ rsz->streaming = false;
+
+ v4l2_subdev_init(sd, &mali_c55_resizer_ops);
+ sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_STREAMS;
+ sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
+ sd->internal_ops = &mali_c55_resizer_internal_ops;
+
+ rsz->pads[MALI_C55_RSZ_SINK_PAD].flags = MEDIA_PAD_FL_SINK;
+ rsz->pads[MALI_C55_RSZ_SOURCE_PAD].flags = MEDIA_PAD_FL_SOURCE;
+
+ if (rsz->id == MALI_C55_RSZ_FR) {
+ num_pads = MALI_C55_RSZ_NUM_PADS;
+ rsz->num_routes = 2;
+
+ rsz->pads[MALI_C55_RSZ_SINK_BYPASS_PAD].flags =
+ MEDIA_PAD_FL_SINK;
+
+ snprintf(sd->name, ARRAY_SIZE(sd->name), "%s resizer fr",
+ MALI_C55_DRIVER_NAME);
+
+ } else {
+ num_pads = MALI_C55_RSZ_NUM_PADS - 1;
+ rsz->num_routes = 1;
+
+ snprintf(sd->name, ARRAY_SIZE(sd->name), "%s resizer ds",
+ MALI_C55_DRIVER_NAME);
+ }
+
+ ret = media_entity_pads_init(&sd->entity, num_pads, rsz->pads);
+ if (ret)
+ return ret;
+
+ ret = v4l2_subdev_init_finalize(sd);
+ if (ret)
+ goto err_media_cleanup;
+
+ ret = v4l2_device_register_subdev(&mali_c55->v4l2_dev, sd);
+ if (ret)
+ goto err_subdev_cleanup;
+
+ return 0;
+
+err_subdev_cleanup:
+ v4l2_subdev_cleanup(sd);
+err_media_cleanup:
+ media_entity_cleanup(&sd->entity);
+
+ return ret;
+}
+
+static void mali_c55_unregister_resizer(struct mali_c55_resizer *rsz)
+{
+ if (!rsz->mali_c55)
+ return;
+
+ v4l2_device_unregister_subdev(&rsz->sd);
+ v4l2_subdev_cleanup(&rsz->sd);
+ media_entity_cleanup(&rsz->sd.entity);
+}
+
+int mali_c55_register_resizers(struct mali_c55 *mali_c55)
+{
+ unsigned int i;
+ int ret;
+
+ for (i = 0; i < MALI_C55_NUM_RSZS; ++i) {
+ struct mali_c55_resizer *rsz = &mali_c55->resizers[i];
+
+ rsz->id = i;
+ ret = mali_c55_register_resizer(mali_c55, rsz);
+ if (ret)
+ goto err_cleanup;
+
+ rsz->cap_dev = &mali_c55->cap_devs[i];
+ rsz->mali_c55 = mali_c55;
+ }
+
+ return 0;
+
+err_cleanup:
+ for (; i >= 0; --i)
+ mali_c55_unregister_resizer(&mali_c55->resizers[i]);
+
+ return ret;
+}
+
+void mali_c55_unregister_resizers(struct mali_c55 *mali_c55)
+{
+ for (unsigned int i = 0; i < MALI_C55_NUM_RSZS; i++)
+ mali_c55_unregister_resizer(&mali_c55->resizers[i]);
+}
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-tpg.c b/drivers/media/platform/arm/mali-c55/mali-c55-tpg.c
new file mode 100644
index 000000000000..56bb617eb5a4
--- /dev/null
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-tpg.c
@@ -0,0 +1,438 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM Mali-C55 ISP Driver - Test pattern generator
+ *
+ * Copyright (C) 2024 Ideas on Board Oy
+ */
+
+#include <linux/minmax.h>
+#include <linux/pm_runtime.h>
+#include <linux/string.h>
+
+#include <media/media-entity.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-subdev.h>
+
+#include "mali-c55-common.h"
+#include "mali-c55-registers.h"
+
+#define MALI_C55_TPG_SRC_PAD 0
+#define MALI_C55_TPG_FIXED_HBLANK 0x20
+#define MALI_C55_TPG_DEFAULT_MIN_VBLANK 66
+#define MALI_C55_TPG_DEFAULT_DEF_VBLANK 626
+#define MALI_C55_TPG_MAX_VBLANK 0xffff
+#define MALI_C55_TPG_PIXEL_RATE 100000000
+
+static const char * const mali_c55_tpg_test_pattern_menu[] = {
+ "Flat field",
+ "Horizontal gradient",
+ "Vertical gradient",
+ "Vertical bars",
+ "Arbitrary rectangle",
+ "White frame on black field"
+};
+
+static const u32 mali_c55_tpg_mbus_codes[] = {
+ MEDIA_BUS_FMT_SRGGB20_1X20,
+ MEDIA_BUS_FMT_RGB202020_1X60,
+};
+
+static void mali_c55_tpg_update_vblank(struct mali_c55_tpg *tpg,
+ struct v4l2_mbus_framefmt *format)
+{
+ unsigned int def_vblank;
+ unsigned int min_vblank;
+ unsigned int hts;
+ int tgt_fps;
+
+ hts = format->width + MALI_C55_TPG_FIXED_HBLANK;
+
+ /*
+ * The ISP has minimum vertical blanking requirements that must be
+ * adhered to by the TPG. The minimum is a function of the Iridix blocks
+ * clocking requirements and the width of the image and horizontal
+ * blanking, but if we assume the worst case iVariance and sVariance
+ * values then it boils down to the below (plus one to the numerator to
+ * ensure the answer is rounded up).
+ */
+ min_vblank = 15 + (120501 / hts);
+
+ /*
+ * We need to set a sensible default vblank for whatever format height
+ * we happen to be given from set_fmt(). This function just targets
+ * an even multiple of 15fps. If we can't get 15fps, let's target 5fps.
+ * If we can't get 5fps we'll take whatever the minimum vblank gives us.
+ */
+ tgt_fps = MALI_C55_TPG_PIXEL_RATE / hts / (format->height + min_vblank);
+
+ if (tgt_fps < 5)
+ def_vblank = min_vblank;
+ else
+ def_vblank = (MALI_C55_TPG_PIXEL_RATE / hts
+ / max(rounddown(tgt_fps, 15), 5)) - format->height;
+
+ def_vblank = ALIGN_DOWN(def_vblank, 2);
+
+ __v4l2_ctrl_modify_range(tpg->ctrls.vblank, min_vblank,
+ MALI_C55_TPG_MAX_VBLANK, 1, def_vblank);
+ __v4l2_ctrl_s_ctrl(tpg->ctrls.vblank, def_vblank);
+}
+
+static int mali_c55_tpg_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct mali_c55_tpg *tpg = container_of(ctrl->handler,
+ struct mali_c55_tpg,
+ ctrls.handler);
+ struct mali_c55 *mali_c55 = container_of(tpg, struct mali_c55, tpg);
+
+ if (!pm_runtime_get_if_in_use(mali_c55->dev))
+ return 0;
+
+ switch (ctrl->id) {
+ case V4L2_CID_TEST_PATTERN:
+ mali_c55_ctx_write(mali_c55,
+ MALI_C55_REG_TEST_GEN_CH0_PATTERN_TYPE,
+ ctrl->val);
+ break;
+ case V4L2_CID_VBLANK:
+ mali_c55_update_bits(mali_c55, MALI_C55_REG_BLANKING,
+ MALI_C55_REG_VBLANK_MASK,
+ MALI_C55_VBLANK(ctrl->val));
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ pm_runtime_put(mali_c55->dev);
+
+ return 0;
+}
+
+static const struct v4l2_ctrl_ops mali_c55_tpg_ctrl_ops = {
+ .s_ctrl = &mali_c55_tpg_s_ctrl,
+};
+
+static void mali_c55_tpg_configure(struct mali_c55_tpg *tpg)
+{
+ struct v4l2_subdev *sd = &tpg->sd;
+ struct v4l2_subdev_state *state;
+ struct v4l2_mbus_framefmt *fmt;
+ u32 test_pattern_format;
+
+ /*
+ * hblank needs setting, but is a read-only control and thus won't be
+ * called during __v4l2_ctrl_handler_setup(). Do it here instead.
+ */
+ mali_c55_update_bits(tpg->mali_c55, MALI_C55_REG_BLANKING,
+ MALI_C55_REG_HBLANK_MASK,
+ MALI_C55_TPG_FIXED_HBLANK);
+ mali_c55_update_bits(tpg->mali_c55, MALI_C55_REG_GEN_VIDEO,
+ MALI_C55_REG_GEN_VIDEO_MULTI_MASK,
+ MALI_C55_REG_GEN_VIDEO_MULTI_MASK);
+
+ state = v4l2_subdev_get_locked_active_state(sd);
+ fmt = v4l2_subdev_state_get_format(state, MALI_C55_TPG_SRC_PAD);
+
+ test_pattern_format = fmt->code == MEDIA_BUS_FMT_RGB202020_1X60 ?
+ 0x01 : 0x0;
+
+ mali_c55_ctx_update_bits(tpg->mali_c55, MALI_C55_REG_TPG_CH0,
+ MALI_C55_TEST_PATTERN_RGB_MASK,
+ MALI_C55_TEST_PATTERN_RGB(test_pattern_format));
+
+ v4l2_subdev_unlock_state(state);
+}
+
+static int mali_c55_tpg_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ if (code->index >= ARRAY_SIZE(mali_c55_tpg_mbus_codes))
+ return -EINVAL;
+
+ code->code = mali_c55_tpg_mbus_codes[code->index];
+
+ return 0;
+}
+
+static int mali_c55_tpg_enum_frame_size(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_frame_size_enum *fse)
+{
+ unsigned int i;
+
+ if (fse->index > 0)
+ return -EINVAL;
+
+ for (i = 0; i < ARRAY_SIZE(mali_c55_tpg_mbus_codes); i++) {
+ if (fse->code == mali_c55_tpg_mbus_codes[i])
+ break;
+ }
+
+ if (i == ARRAY_SIZE(mali_c55_tpg_mbus_codes))
+ return -EINVAL;
+
+ fse->min_width = MALI_C55_MIN_WIDTH;
+ fse->max_width = MALI_C55_MAX_WIDTH;
+ fse->min_height = MALI_C55_MIN_HEIGHT;
+ fse->max_height = MALI_C55_MAX_HEIGHT;
+
+ return 0;
+}
+
+static int mali_c55_tpg_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format)
+{
+ struct mali_c55_tpg *tpg = container_of(sd, struct mali_c55_tpg, sd);
+ struct v4l2_mbus_framefmt *fmt;
+ unsigned int i;
+
+ fmt = v4l2_subdev_state_get_format(state, MALI_C55_TPG_SRC_PAD);
+ fmt->code = format->format.code;
+
+ for (i = 0; i < ARRAY_SIZE(mali_c55_tpg_mbus_codes); i++) {
+ if (fmt->code == mali_c55_tpg_mbus_codes[i])
+ break;
+ }
+
+ if (i == ARRAY_SIZE(mali_c55_tpg_mbus_codes))
+ fmt->code = MEDIA_BUS_FMT_SRGGB20_1X20;
+
+ /*
+ * The TPG says that the test frame timing generation logic expects a
+ * minimum framesize of 4x4 pixels, but given the rest of the ISP can't
+ * handle anything smaller than 128x128 it seems pointless to allow a
+ * smaller frame.
+ */
+ fmt->width = clamp(fmt->width, MALI_C55_MIN_WIDTH, MALI_C55_MAX_WIDTH);
+ fmt->height = clamp(fmt->height, MALI_C55_MIN_HEIGHT,
+ MALI_C55_MAX_HEIGHT);
+
+ format->format = *fmt;
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY)
+ return 0;
+
+ mali_c55_tpg_update_vblank(tpg, fmt);
+
+ return 0;
+}
+
+static int mali_c55_tpg_enable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask)
+{
+ struct mali_c55_tpg *tpg = container_of(sd, struct mali_c55_tpg, sd);
+ struct mali_c55 *mali_c55 = container_of(tpg, struct mali_c55, tpg);
+
+ /*
+ * We only have a source pad and a single stream, and v4l2-core already
+ * validated both so we don't need to do that. One might reasonably
+ * expect the framesize to be set here given it's configurable in
+ * .set_fmt(), but it's done in the ISP subdevice's .enable_streams()
+ * instead, as the same register is also used to indicate the size of
+ * the data coming from the sensor.
+ */
+ mali_c55_tpg_configure(tpg);
+ __v4l2_ctrl_handler_setup(sd->ctrl_handler);
+
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_TPG_CH0,
+ MALI_C55_TEST_PATTERN_ON_OFF,
+ MALI_C55_TEST_PATTERN_ON_OFF);
+ mali_c55_update_bits(mali_c55, MALI_C55_REG_GEN_VIDEO,
+ MALI_C55_REG_GEN_VIDEO_ON_MASK,
+ MALI_C55_REG_GEN_VIDEO_ON_MASK);
+
+ return 0;
+}
+
+static int mali_c55_tpg_disable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask)
+{
+ struct mali_c55_tpg *tpg = container_of(sd, struct mali_c55_tpg, sd);
+ struct mali_c55 *mali_c55 = container_of(tpg, struct mali_c55, tpg);
+
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_TPG_CH0,
+ MALI_C55_TEST_PATTERN_ON_OFF, 0x00);
+ mali_c55_update_bits(mali_c55, MALI_C55_REG_GEN_VIDEO,
+ MALI_C55_REG_GEN_VIDEO_ON_MASK, 0x00);
+
+ return 0;
+}
+
+static const struct v4l2_subdev_pad_ops mali_c55_tpg_pad_ops = {
+ .enum_mbus_code = mali_c55_tpg_enum_mbus_code,
+ .enum_frame_size = mali_c55_tpg_enum_frame_size,
+ .get_fmt = v4l2_subdev_get_fmt,
+ .set_fmt = mali_c55_tpg_set_fmt,
+ .enable_streams = mali_c55_tpg_enable_streams,
+ .disable_streams = mali_c55_tpg_disable_streams,
+};
+
+static const struct v4l2_subdev_core_ops mali_c55_isp_core_ops = {
+ .subscribe_event = v4l2_ctrl_subdev_subscribe_event,
+ .unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_ops mali_c55_tpg_ops = {
+ .core = &mali_c55_isp_core_ops,
+ .pad = &mali_c55_tpg_pad_ops,
+};
+
+static int mali_c55_tpg_init_state(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state)
+{
+ struct v4l2_mbus_framefmt *fmt =
+ v4l2_subdev_state_get_format(state, MALI_C55_TPG_SRC_PAD);
+
+ fmt->width = MALI_C55_DEFAULT_WIDTH;
+ fmt->height = MALI_C55_DEFAULT_HEIGHT;
+ fmt->field = V4L2_FIELD_NONE;
+ fmt->code = MEDIA_BUS_FMT_SRGGB20_1X20;
+ fmt->colorspace = V4L2_COLORSPACE_RAW;
+
+ return 0;
+}
+
+static const struct v4l2_subdev_internal_ops mali_c55_tpg_internal_ops = {
+ .init_state = mali_c55_tpg_init_state,
+};
+
+static int mali_c55_tpg_init_controls(struct mali_c55 *mali_c55)
+{
+ struct mali_c55_tpg_ctrls *ctrls = &mali_c55->tpg.ctrls;
+ struct v4l2_ctrl *pixel_rate;
+ struct v4l2_ctrl *hblank;
+ int ret;
+
+ ret = v4l2_ctrl_handler_init(&ctrls->handler, 4);
+ if (ret)
+ return ret;
+
+ v4l2_ctrl_new_std_menu_items(&ctrls->handler,
+ &mali_c55_tpg_ctrl_ops, V4L2_CID_TEST_PATTERN,
+ ARRAY_SIZE(mali_c55_tpg_test_pattern_menu) - 1,
+ 0, 3, mali_c55_tpg_test_pattern_menu);
+
+ /*
+ * We fix hblank at the minimum allowed value and control framerate
+ * solely through the vblank control.
+ */
+ hblank = v4l2_ctrl_new_std(&ctrls->handler, &mali_c55_tpg_ctrl_ops,
+ V4L2_CID_HBLANK, MALI_C55_TPG_FIXED_HBLANK,
+ MALI_C55_TPG_FIXED_HBLANK, 1,
+ MALI_C55_TPG_FIXED_HBLANK);
+ if (hblank)
+ hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
+
+ ctrls->vblank = v4l2_ctrl_new_std(&ctrls->handler,
+ &mali_c55_tpg_ctrl_ops,
+ V4L2_CID_VBLANK,
+ MALI_C55_TPG_DEFAULT_MIN_VBLANK,
+ MALI_C55_TPG_MAX_VBLANK, 1,
+ MALI_C55_TPG_DEFAULT_DEF_VBLANK);
+
+ pixel_rate = v4l2_ctrl_new_std(&ctrls->handler, &mali_c55_tpg_ctrl_ops,
+ V4L2_CID_PIXEL_RATE,
+ MALI_C55_TPG_PIXEL_RATE,
+ MALI_C55_TPG_PIXEL_RATE, 1,
+ MALI_C55_TPG_PIXEL_RATE);
+ if (pixel_rate)
+ pixel_rate->flags |= V4L2_CTRL_FLAG_READ_ONLY;
+
+ if (ctrls->handler.error) {
+ dev_err(mali_c55->dev, "Error during v4l2 controls init\n");
+ ret = ctrls->handler.error;
+ goto err_free_handler;
+ }
+
+ mali_c55->tpg.sd.ctrl_handler = &ctrls->handler;
+ mali_c55->tpg.sd.state_lock = ctrls->handler.lock;
+
+ return 0;
+
+err_free_handler:
+ v4l2_ctrl_handler_free(&ctrls->handler);
+
+ return ret;
+}
+
+int mali_c55_register_tpg(struct mali_c55 *mali_c55)
+{
+ struct mali_c55_tpg *tpg = &mali_c55->tpg;
+ struct v4l2_subdev *sd = &tpg->sd;
+ struct media_pad *pad = &tpg->pad;
+ int ret;
+
+ v4l2_subdev_init(sd, &mali_c55_tpg_ops);
+ sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
+ sd->entity.function = MEDIA_ENT_F_CAM_SENSOR;
+ sd->internal_ops = &mali_c55_tpg_internal_ops;
+ strscpy(sd->name, MALI_C55_DRIVER_NAME " tpg", sizeof(sd->name));
+
+ pad->flags = MEDIA_PAD_FL_SOURCE;
+ ret = media_entity_pads_init(&sd->entity, 1, pad);
+ if (ret) {
+ dev_err(mali_c55->dev,
+ "Failed to initialize media entity pads\n");
+ return ret;
+ }
+
+ ret = mali_c55_tpg_init_controls(mali_c55);
+ if (ret) {
+ dev_err(mali_c55->dev,
+ "Error initialising controls\n");
+ goto err_cleanup_media_entity;
+ }
+
+ ret = v4l2_subdev_init_finalize(sd);
+ if (ret)
+ goto err_free_ctrl_handler;
+
+ ret = v4l2_device_register_subdev(&mali_c55->v4l2_dev, sd);
+ if (ret) {
+ dev_err(mali_c55->dev, "Failed to register tpg subdev\n");
+ goto err_cleanup_subdev;
+ }
+
+ /*
+ * By default the colour settings lead to a very dim image that is
+ * nearly indistinguishable from black on some monitor settings. Ramp
+ * them up a bit so the image is brighter.
+ */
+ mali_c55_ctx_write(mali_c55, MALI_C55_REG_TPG_R_BACKGROUND,
+ MALI_C55_TPG_BACKGROUND_MAX);
+ mali_c55_ctx_write(mali_c55, MALI_C55_REG_TPG_G_BACKGROUND,
+ MALI_C55_TPG_BACKGROUND_MAX);
+ mali_c55_ctx_write(mali_c55, MALI_C55_REG_TPG_B_BACKGROUND,
+ MALI_C55_TPG_BACKGROUND_MAX);
+
+ tpg->mali_c55 = mali_c55;
+
+ return 0;
+
+err_cleanup_subdev:
+ v4l2_subdev_cleanup(sd);
+err_free_ctrl_handler:
+ v4l2_ctrl_handler_free(&tpg->ctrls.handler);
+err_cleanup_media_entity:
+ media_entity_cleanup(&sd->entity);
+
+ return ret;
+}
+
+void mali_c55_unregister_tpg(struct mali_c55 *mali_c55)
+{
+ struct mali_c55_tpg *tpg = &mali_c55->tpg;
+
+ if (!tpg->mali_c55)
+ return;
+
+ v4l2_device_unregister_subdev(&tpg->sd);
+ v4l2_ctrl_handler_free(&tpg->ctrls.handler);
+ v4l2_subdev_cleanup(&tpg->sd);
+ media_entity_cleanup(&tpg->sd.entity);
+}
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 08/18] media: Documentation: Add Mali-C55 ISP Documentation
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
` (6 preceding siblings ...)
2024-07-09 13:28 ` [PATCH v6 07/18] media: mali-c55: Add Mali-C55 ISP driver Daniel Scally
@ 2024-07-09 13:28 ` Daniel Scally
2024-07-30 15:15 ` Laurent Pinchart
2024-07-09 13:28 ` [PATCH v6 09/18] MAINTAINERS: Add entry for mali-c55 driver Daniel Scally
` (9 subsequent siblings)
17 siblings, 1 reply; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:28 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
Add a documentation page for the mali-c55 driver, which gives a brief
overview of the hardware and explains how to use the driver's capture
devices and the crop/scaler functions.
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- Minor formatting changes
Changes in v5:
- None
Changes in v4:
- None
Changes in v3:
- Documented the synchronised buffer sequence numbers (Sakari)
- Clarified that the downscale pipe cannot output raw data, the ISP'S
resolution limits and choice of media bus format code (Kieran)
Changes in v2:
- none
.../admin-guide/media/mali-c55-graph.dot | 19 +
Documentation/admin-guide/media/mali-c55.rst | 340 ++++++++++++++++++
.../admin-guide/media/v4l-drivers.rst | 1 +
3 files changed, 360 insertions(+)
create mode 100644 Documentation/admin-guide/media/mali-c55-graph.dot
create mode 100644 Documentation/admin-guide/media/mali-c55.rst
diff --git a/Documentation/admin-guide/media/mali-c55-graph.dot b/Documentation/admin-guide/media/mali-c55-graph.dot
new file mode 100644
index 000000000000..0775ba42bf4c
--- /dev/null
+++ b/Documentation/admin-guide/media/mali-c55-graph.dot
@@ -0,0 +1,19 @@
+digraph board {
+ rankdir=TB
+ n00000001 [label="{{} | mali-c55 tpg\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000001:port0 -> n00000003:port0 [style=dashed]
+ n00000003 [label="{{<port0> 0} | mali-c55 isp\n/dev/v4l-subdev1 | {<port1> 1 | <port2> 2}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000003:port1 -> n00000007:port0 [style=bold]
+ n00000003:port2 -> n00000007:port2 [style=bold]
+ n00000003:port1 -> n0000000b:port0 [style=bold]
+ n00000007 [label="{{<port0> 0 | <port2> 2} | mali-c55 resizer fr\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000007:port1 -> n0000000e [style=bold]
+ n0000000b [label="{{<port0> 0} | mali-c55 resizer ds\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
+ n0000000b:port1 -> n00000012 [style=bold]
+ n0000000e [label="mali-c55 fr\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
+ n00000012 [label="mali-c55 ds\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
+ n00000022 [label="{{<port0> 0} | csi2-rx\n/dev/v4l-subdev4 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000022:port1 -> n00000003:port0
+ n00000027 [label="{{} | imx415 1-001a\n/dev/v4l-subdev5 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000027:port0 -> n00000022:port0 [style=bold]
+}
\ No newline at end of file
diff --git a/Documentation/admin-guide/media/mali-c55.rst b/Documentation/admin-guide/media/mali-c55.rst
new file mode 100644
index 000000000000..72cdded507b3
--- /dev/null
+++ b/Documentation/admin-guide/media/mali-c55.rst
@@ -0,0 +1,340 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==========================================
+ARM Mali-C55 Image Signal Processor driver
+==========================================
+
+Introduction
+============
+
+This file documents the driver for ARM's Mali-C55 Image Signal Processor. The
+driver is located under drivers/media/platform/arm/mali-c55.
+
+The Mali-C55 ISP receives data in either raw Bayer format or RGB/YUV format from
+sensors through either a parallel interface or a memory bus before processing it
+and outputting it through an internal DMA engine. Two output pipelines are
+possible (though one may not be fitted, depending on the implementation). These
+are referred to as "Full resolution" and "Downscale", but the naming is historic
+and both pipes are capable of cropping/scaling operations. The full resolution
+pipe is also capable of outputting RAW data, bypassing much of the ISP's
+processing. The downscale pipe cannot output RAW data. An integrated test
+pattern generator can be used to drive the ISP and produce image data in the
+absence of a connected camera sensor. The driver module is named mali_c55, and
+is enabled through the CONFIG_VIDEO_MALI_C55 config option.
+
+The driver implements V4L2, Media Controller and V4L2 Subdevice interfaces and
+expects camera sensors connected to the ISP to have V4L2 subdevice interfaces.
+
+Mali-C55 ISP hardware
+=====================
+
+A high level functional view of the Mali-C55 ISP is presented below. The ISP
+takes input from either a live source or through a DMA engine for memory input,
+depending on the SoC integration.::
+
+ +---------+ +----------+ +--------+
+ | Sensor |--->| CSI-2 Rx | "Full Resolution" | DMA |
+ +---------+ +----------+ |\ Output +--->| Writer |
+ | | \ | +--------+
+ | | \ +----------+ +------+---> Streaming I/O
+ +------------+ +------->| | | | |
+ | | | |-->| Mali-C55 |--+
+ | DMA Reader |--------------->| | | ISP | |
+ | | | / | | | +---> Streaming I/O
+ +------------+ | / +----------+ | |
+ |/ +------+
+ | +--------+
+ +--->| DMA |
+ "Downscaled" | Writer |
+ Output +--------+
+
+Media Controller Topology
+=========================
+
+An example of the ISP's topology (as implemented in a system with an IMX415
+camera sensor and generic CSI-2 receiver) is below:
+
+
+.. kernel-figure:: mali-c55-graph.dot
+ :alt: mali-c55-graph.dot
+ :align: center
+
+The driver has 4 V4L2 subdevices:
+
+- `mali_c55 isp`: Responsible for configuring input crop and color space
+ conversion
+- `mali_c55 tpg`: The test pattern generator, emulating a camera sensor.
+- `mali_c55 resizer fr`: The Full-Resolution pipe resizer
+- `mali_c55 resizer ds`: The Downscale pipe resizer
+
+The driver has 2 V4L2 video devices:
+
+- `mali-c55 fr`: The full-resolution pipe's capture device
+- `mali-c55 ds`: The downscale pipe's capture device
+
+Frame sequences are synchronised across to two capture devices, meaning if one
+pipe is started later than the other the sequence numbers returned in its
+buffers will match those of the other pipe rather than starting from zero.
+
+Idiosyncrasies
+--------------
+
+**mali-c55 isp**
+The `mali-c55 isp` subdevice has a single sink pad to which all sources of data
+should be connected. The active source is selected by enabling the appropriate
+media link and disabling all others. The ISP has two source pads, reflecting the
+different paths through which it can internally route data. Tap points within
+the ISP allow users to divert data to avoid processing by some or all of the
+hardware's processing steps. The diagram below is intended only to highlight how
+the bypassing works and is not a true reflection of those processing steps; for
+a high-level functional block diagram see ARM's developer page for the
+ISP [3]_::
+
+ +--------------------------------------------------------------+
+ | Possible Internal ISP Data Routes |
+ | +------------+ +----------+ +------------+ |
+ +---+ | | | | | Colour | +---+
+ | 0 |--+-->| Processing |->| Demosaic |->| Space |--->| 1 |
+ +---+ | | | | | | Conversion | +---+
+ | | +------------+ +----------+ +------------+ |
+ | | +---+
+ | +---------------------------------------------------| 2 |
+ | +---+
+ | |
+ +--------------------------------------------------------------+
+
+
+.. flat-table::
+ :header-rows: 1
+
+ * - Pad
+ - Direction
+ - Purpose
+
+ * - 0
+ - sink
+ - Data input, connected to the TPG and camera sensors
+
+ * - 1
+ - source
+ - RGB/YUV data, connected to the FR and DS V4L2 subdevices
+
+ * - 2
+ - source
+ - RAW bayer data, connected to the FR V4L2 subdevices
+
+The ISP is limited to both input and output resolutions between 640x480 and
+8192x8192, and this is reflected in the ISP and resizer subdevice's .set_fmt()
+operations.
+
+**mali-c55 resizer fr**
+The `mali-c55 resizer fr` subdevice has two _sink_ pads to reflect the different
+insertion points in the hardware (either RAW or demosaiced data):
+
+.. flat-table::
+ :header-rows: 1
+
+ * - Pad
+ - Direction
+ - Purpose
+
+ * - 0
+ - sink
+ - Data input connected to the ISP's demosaiced stream.
+
+ * - 1
+ - source
+ - Data output connected to the capture video device
+
+ * - 2
+ - sink
+ - Data input connected to the ISP's raw data stream
+
+The data source in use is selected through the routing API; two routes each of a
+single stream are available:
+
+.. flat-table::
+ :header-rows: 1
+
+ * - Sink Pad
+ - Source Pad
+ - Purpose
+
+ * - 0
+ - 1
+ - Demosaiced data route
+
+ * - 2
+ - 1
+ - Raw data route
+
+
+If the demosaiced route is active then the FR pipe is only capable of output
+in RGB/YUV formats. If the raw route is active then the output reflects the
+input (which may be either Bayer or RGB/YUV data).
+
+Using the driver to capture video
+=================================
+
+Using the media controller APIs we can configure the input source and ISP to
+capture images in a variety of formats. In the examples below, configuring the
+media graph is done with the v4l-utils [1]_ package's media-ctl utility.
+Capturing the images is done with yavta [2]_.
+
+Configuring the input source
+----------------------------
+
+The first step is to set the input source that we wish by enabling the correct
+media link. Using the example topology above, we can select the TPG as follows:
+
+.. code-block:: none
+
+ media-ctl -l "'lte-csi2-rx':1->'mali-c55 isp':0[0]"
+ media-ctl -l "'mali-c55 tpg':0->'mali-c55 isp':0[1]"
+
+Configuring which video devices will stream data
+------------------------------------------------
+
+The driver will wait for all video devices to have their VIDIOC_STREAMON ioctl
+called before it tells the sensor to start streaming. To facilitate this we need
+to enable links to the video devices that we want to use. In the example below
+we enable the links to both of the image capture video devices
+
+.. code-block:: none
+
+ media-ctl -l "'mali-c55 resizer fr':1->'mali-c55 fr':0[1]"
+ media-ctl -l "'mali-c55 resizer ds':1->'mali-c55 ds':0[1]"
+
+Capturing bayer data from the source and processing to RGB/YUV
+--------------------------------------------------------------
+
+To capture 1920x1080 bayer data from the source and push it through the ISP's
+full processing pipeline, we configure the data formats appropriately on the
+source, ISP and resizer subdevices and set the FR resizer's routing to select
+processed data. The media bus format on the resizer's source pad will be either
+RGB121212_1X36 or YUV10_1X30, depending on whether you want to capture RGB or
+YUV. The ISP's debayering block outputs RGB data natively, setting the source
+pad format to YUV10_1X30 enables the colour space conversion block.
+
+In this example we target RGB565 output, so select RGB121212_1X36 as the resizer
+source pad's format:
+
+.. code-block:: none
+
+ # Set formats on the TPG and ISP
+ media-ctl -V "'mali-c55 tpg':0[fmt:SRGGB20_1X20/1920x1080]"
+ media-ctl -V "'mali-c55 isp':0[fmt:SRGGB20_1X20/1920x1080]"
+ media-ctl -V "'mali-c55 isp':1[fmt:SRGGB20_1X20/1920x1080]"
+
+ # Set routing on the FR resizer
+ media-ctl -R "'mali-c55 resizer fr'[0/0->1/0[1],2/0->1/0[0]]"
+
+ # Set format on the resizer, must be done AFTER the routing.
+ media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/1920x1080]"
+
+The downscale output can also be used to stream data at the same time. In this
+case since only processed data can be captured through the downscale output no
+routing need be set:
+
+.. code-block:: none
+
+ # Set format on the resizer
+ media-ctl -V "'mali-c55 resizer ds':1[fmt:RGB121212_1X36/1920x1080]"
+
+Following which images can be captured from both the FR and DS output's video
+devices (simultaneously, if desired):
+
+.. code-block:: none
+
+ yavta -f RGB565 -s 1920x1080 -c10 /dev/video0
+ yavta -f RGB565 -s 1920x1080 -c10 /dev/video1
+
+Cropping the image
+~~~~~~~~~~~~~~~~~~
+
+Both the full resolution and downscale pipes can crop to a minimum resolution of
+640x480. To crop the image simply configure the resizer's sink pad's crop and
+compose rectangles and set the format on the video device:
+
+.. code-block:: none
+
+ media-ctl -V "'mali-c55 resizer fr':0[fmt:RGB121212_1X36/1920x1080 crop:(480,270)/640x480 compose:(0,0)/640x480]"
+ media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/640x480]"
+ yavta -f RGB565 -s 640x480 -c10 /dev/video0
+
+Downscaling the image
+~~~~~~~~~~~~~~~~~~~~~
+
+Both the full resolution and downscale pipes can downscale the image by up to 8x
+provided the minimum 640x480 output resolution is adhered to. For the best image
+result the scaling ratio for each direction should be the same. To configure
+scaling we use the compose rectangle on the resizer's sink pad:
+
+.. code-block:: none
+
+ media-ctl -V "'mali-c55 resizer fr':0[fmt:RGB121212_1X36/1920x1080 crop:(0,0)/1920x1080 compose:(0,0)/640x480]"
+ media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/640x480]"
+ yavta -f RGB565 -s 640x480 -c10 /dev/video0
+
+Capturing images in YUV formats
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If we need to output YUV data rather than RGB the color space conversion block
+needs to be active, which is achieved by setting MEDIA_BUS_FMT_YUV10_1X30 on the
+resizer's source pad. We can then configure a capture format like NV12 (here in
+its multi-planar variant)
+
+.. code-block:: none
+
+ media-ctl -V "'mali-c55 resizer fr':1[fmt:YUV10_1X30/1920x1080]"
+ yavta -f NV12M -s 1920x1080 -c10 /dev/video0
+
+Capturing RGB data from the source and processing it with the resizers
+----------------------------------------------------------------------
+
+The Mali-C55 ISP can work with sensors capable of outputting RGB data. In this
+case although none of the image quality blocks would be used it can still
+crop/scale the data in the usual way. For this reason RGB data input to the ISP
+still goes through the ISP subdevice's pad 1 to the resizer.
+
+To achieve this, the ISP's sink pad's format is set to
+MEDIA_BUS_FMT_RGB202020_1X60 - this reflects the format that data must be in to
+work with the ISP. Converting the camera sensor's output to that format is the
+responsibility of external hardware.
+
+In this example we ask the test pattern generator to give us RGB data instead of
+bayer.
+
+.. code-block:: none
+
+ media-ctl -V "'mali-c55 tpg':0[fmt:RGB202020_1X60/1920x1080]"
+ media-ctl -V "'mali-c55 isp':0[fmt:RGB202020_1X60/1920x1080]"
+
+Cropping or scaling the data can be done in exactly the same way as outlined
+earlier.
+
+Capturing raw data from the source and outputting it unmodified
+-----------------------------------------------------------------
+
+The ISP can additionally capture raw data from the source and output it on the
+full resolution pipe only, completely unmodified. In this case the downscale
+pipe can still process the data normally and be used at the same time.
+
+To configure raw bypass the FR resizer's subdevice's routing table needs to be
+configured, followed by formats in the appropriate places:
+
+.. code-block:: none
+
+ media-ctl -R "'mali-c55 resizer fr'[0/0->1/0[0],2/0->1/0[1]]"
+ media-ctl -V "'mali-c55 isp':0[fmt:RGB202020_1X60/1920x1080]"
+ media-ctl -V "'mali-c55 resizer fr':2[fmt:RGB202020_1X60/1920x1080]"
+ media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB202020_1X60/1920x1080]"
+
+ # Set format on the video device and stream
+ yavta -f RGB565 -s 1920x1080 -c10 /dev/video0
+
+References
+==========
+.. [1] https://git.linuxtv.org/v4l-utils.git/
+.. [2] https://git.ideasonboard.org/yavta.git
+.. [3] https://developer.arm.com/Processors/Mali-C55
diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst
index 4120eded9a13..1d9485860d93 100644
--- a/Documentation/admin-guide/media/v4l-drivers.rst
+++ b/Documentation/admin-guide/media/v4l-drivers.rst
@@ -18,6 +18,7 @@ Video4Linux (V4L) driver-specific documentation
ipu3
ipu6-isys
ivtv
+ mali-c55
mgb4
omap3isp
omap4_camera
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 09/18] MAINTAINERS: Add entry for mali-c55 driver
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
` (7 preceding siblings ...)
2024-07-09 13:28 ` [PATCH v6 08/18] media: Documentation: Add Mali-C55 ISP Documentation Daniel Scally
@ 2024-07-09 13:28 ` Daniel Scally
2024-07-09 13:28 ` [PATCH v6 10/18] media: Add MALI_C55_3A_STATS meta format Daniel Scally
` (8 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:28 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
Add a MAINTAINERS entry for the mali-c55 driver and its associated
documentation.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- None
Changes in v5:
- None
Changes in v4:
- None
Changes in v3:
- none
Changes in v2:
- none
MAINTAINERS | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 9caf669234f2..04726aaaefd4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1755,6 +1755,17 @@ F: Documentation/devicetree/bindings/gpu/arm,mali-valhall-csf.yaml
F: drivers/gpu/drm/panthor/
F: include/uapi/drm/panthor_drm.h
+ARM MALI-C55 ISP DRIVER
+M: Daniel Scally <dan.scally@ideasonboard.com>
+M: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
+L: linux-media@vger.kernel.org
+S: Maintained
+T: git git://linuxtv.org/media_tree.git
+F: Documentation/admin-guide/media/mali-c55-graph.dot
+F: Documentation/admin-guide/media/mali-c55.rst
+F: Documentation/devicetree/bindings/media/arm,mali-c55.yaml
+F: drivers/media/platform/arm/mali-c55/
+
ARM MALI-DP DRM DRIVER
M: Liviu Dudau <liviu.dudau@arm.com>
S: Supported
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 10/18] media: Add MALI_C55_3A_STATS meta format
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
` (8 preceding siblings ...)
2024-07-09 13:28 ` [PATCH v6 09/18] MAINTAINERS: Add entry for mali-c55 driver Daniel Scally
@ 2024-07-09 13:28 ` Daniel Scally
2024-07-09 13:28 ` [PATCH v6 11/18] media: uapi: Add 3a stats buffer for mali-c55 Daniel Scally
` (7 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:28 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
Add a new meta format for the Mali-C55 ISP's 3A Statistics along
with a new descriptor entry.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- New patch
Changes in v5:
- New patch
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/videodev2.h | 2 ++
2 files changed, 3 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 4c76d17b4629..7e49a5a43077 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1456,6 +1456,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_META_FMT_VIVID: descr = "Vivid Metadata"; break;
case V4L2_META_FMT_RK_ISP1_PARAMS: descr = "Rockchip ISP1 3A Parameters"; break;
case V4L2_META_FMT_RK_ISP1_STAT_3A: descr = "Rockchip ISP1 3A Statistics"; break;
+ case V4L2_META_FMT_MALI_C55_STATS: descr = "ARM Mali-C55 ISP 3A Statistics"; break;
case V4L2_PIX_FMT_NV12_8L128: descr = "NV12 (8x128 Linear)"; break;
case V4L2_PIX_FMT_NV12M_8L128: descr = "NV12M (8x128 Linear)"; break;
case V4L2_PIX_FMT_NV12_10BE_8L128: descr = "10-bit NV12 (8x128 Linear, BE)"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index fe6b67e83751..4fb09ef85772 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -841,6 +841,8 @@ struct v4l2_pix_format {
#define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
#define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
+#define V4L2_META_FMT_MALI_C55_STATS v4l2_fourcc('C', '5', '5', 'S') /* ARM Mali-C55 3A Statistics */
+
#ifdef __KERNEL__
/*
* Line-based metadata formats. Remember to update v4l_fill_fmtdesc() when
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 11/18] media: uapi: Add 3a stats buffer for mali-c55
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
` (9 preceding siblings ...)
2024-07-09 13:28 ` [PATCH v6 10/18] media: Add MALI_C55_3A_STATS meta format Daniel Scally
@ 2024-07-09 13:28 ` Daniel Scally
2024-07-09 13:29 ` [PATCH v6 12/18] media: platform: Add mali-c55 3a stats devnode Daniel Scally
` (6 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:28 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
Add a header that describes the format of the 3A statistics buffers
for the mali-c55 ISP.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- New patch
Changes in v5:
- New patch
MAINTAINERS | 1 +
.../uapi/linux/media/arm/mali-c55-config.h | 182 ++++++++++++++++++
2 files changed, 183 insertions(+)
create mode 100644 include/uapi/linux/media/arm/mali-c55-config.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 04726aaaefd4..3b5c39b820cc 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1765,6 +1765,7 @@ F: Documentation/admin-guide/media/mali-c55-graph.dot
F: Documentation/admin-guide/media/mali-c55.rst
F: Documentation/devicetree/bindings/media/arm,mali-c55.yaml
F: drivers/media/platform/arm/mali-c55/
+F: include/uapi/linux/media/arm/mali-c55-config.h
ARM MALI-DP DRM DRIVER
M: Liviu Dudau <liviu.dudau@arm.com>
diff --git a/include/uapi/linux/media/arm/mali-c55-config.h b/include/uapi/linux/media/arm/mali-c55-config.h
new file mode 100644
index 000000000000..21b453bdee5d
--- /dev/null
+++ b/include/uapi/linux/media/arm/mali-c55-config.h
@@ -0,0 +1,182 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * ARM Mali-C55 ISP Driver - Userspace API
+ *
+ * Copyright (C) 2023 Ideas on Board Oy
+ */
+
+#ifndef __UAPI_MALI_C55_CONFIG_H
+#define __UAPI_MALI_C55_CONFIG_H
+
+#include <linux/types.h>
+
+/*
+ * Frames are split into zones of almost equal width and height - a zone is a
+ * rectangular tile of a frame. The metering blocks within the ISP collect
+ * aggregated statistics per zone. A maximum of 15x15 zones can be configured,
+ * and so the statistics buffer within the hardware is sized to accommodate
+ * that.
+ *
+ * The utilised number of zones is runtime configurable.
+ */
+#define MALI_C55_MAX_ZONES (15 * 15)
+
+/**
+ * struct mali_c55_ae_1024bin_hist - Auto Exposure 1024-bin histogram statistics
+ *
+ * @bins: 1024 element array of 16-bit pixel counts.
+ *
+ * The 1024-bin histogram module collects image-global but zone-weighted
+ * intensity distributions of pixels in fixed-width bins. The modules can be
+ * configured into different "plane modes" which affect the contents of the
+ * collected statistics. In plane mode 0, pixel intensities are taken regardless
+ * of colour plane into a single 1024-bin histogram with a bin width of 4. In
+ * plane mode 1, four 256-bin histograms with a bin width of 16 are collected -
+ * one for each CFA colour plane. In plane modes 4, 5, 6 and 7 two 512-bin
+ * histograms with a bin width of 8 are collected - in each mode one of the
+ * colour planes is collected into the first histogram and all the others are
+ * combined into the second. The histograms are stored consecutively in the bins
+ * array.
+ *
+ * The 16-bit pixel counts are stored as a 4-bit exponent in the most
+ * significant bits followed by a 12-bit mantissa. Conversion to a usable
+ * format can be done according to the following pseudo-code::
+ *
+ * if (e == 0) {
+ * bin = m * 2;
+ * } else {
+ * bin = (m + 4096) * 2^e
+ * }
+ *
+ * where
+ * e is the exponent value in range 0..15
+ * m is the mantissa value in range 0..4095
+ *
+ * The pixels used in calculating the statistics can be masked using three
+ * methods:
+ *
+ * 1. Pixels can be skipped in X and Y directions independently.
+ * 2. Minimum/Maximum intensities can be configured
+ * 3. Zones can be differentially weighted, including 0 weighted to mask them
+ *
+ * The data for this histogram can be collected from different tap points in the
+ * ISP depending on configuration - after the white balance or digital gain
+ * blocks, or immediately after the input crossbar.
+ */
+struct mali_c55_ae_1024bin_hist {
+ __u16 bins[1024];
+} __attribute__((packed));
+
+/**
+ * struct mali_c55_ae_5bin_hist - Auto Exposure 5-bin histogram statistics
+ *
+ * @hist0: 16-bit normalised pixel count for the 0th intensity bin
+ * @hist1: 16-bit normalised pixel count for the 1st intensity bin
+ * @hist3: 16-bit normalised pixel count for the 3rd intensity bin
+ * @hist4: 16-bit normalised pixel count for the 4th intensity bin
+ *
+ * The ISP generates a 5-bin histogram of normalised pixel counts within bins of
+ * pixel intensity for each of 225 possible zones within a frame. The centre bin
+ * of the histogram for each zone is not available from the hardware and must be
+ * calculated by subtracting the values of hist0, hist1, hist3 and hist4 from
+ * 0xffff as in the following equation:
+ *
+ * hist2 = 0xffff - (hist0 + hist1 + hist3 + hist4)
+ */
+struct mali_c55_ae_5bin_hist {
+ __u16 hist0;
+ __u16 hist1;
+ __u16 hist3;
+ __u16 hist4;
+} __attribute__((packed));
+
+/**
+ * struct mali_c55_awb_average_ratios - Auto White Balance colour ratios
+ *
+ * @avg_rg_gr: Average R/G or G/R ratio in Q4.8 format.
+ * @avg_bg_br: Average B/G or B/R ratio in Q4.8 format.
+ * @num_pixels: The number of pixels used in the AWB calculation
+ *
+ * The ISP calculates and collects average colour ratios for each zone in an
+ * image and stores them in Q4.8 format (the lowest 8 bits are fractional, with
+ * bits [11:8] representing the integer). The exact ratios collected (either
+ * R/G, B/G or G/R, B/R) are configurable through the parameters buffer. The
+ * value of the 4 high bits is undefined.
+ */
+struct mali_c55_awb_average_ratios {
+ __u16 avg_rg_gr;
+ __u16 avg_bg_br;
+ __u32 num_pixels;
+} __attribute__((packed));
+
+/**
+ * struct mali_c55_af_statistics - Auto Focus edge and intensity statistics
+ *
+ * @intensity_stats: Packed mantissa and exponent value for pixel intensity
+ * @edge_stats: Packed mantissa and exponent values for edge intensity
+ *
+ * The ISP collects the squared sum of pixel intensities for each zone within a
+ * configurable Region of Interest on the frame. Additionally, the same data are
+ * collected after being passed through a bandpass filter which removes high and
+ * low frequency components - these are referred to as the edge statistics.
+ *
+ * The intensity and edge statistics for a zone can be used to calculate the
+ * contrast information for a zone
+ *
+ * C = E2 / I2
+ *
+ * Where I2 is the intensity statistic for a zone and E2 is the edge statistic
+ * for that zone. Optimum focus is reached when C is at its maximum.
+ *
+ * The intensity and edge statistics are stored packed into a non-standard 16
+ * bit floating point format, where the 7 most significant bits represent the
+ * exponent and the 9 least significant bits the mantissa. This format can be
+ * unpacked with the following pseudocode::
+ *
+ * if (e == 0) {
+ * x = m;
+ * } else {
+ * x = 2^e-1 * (m + 2^9)
+ * }
+ *
+ * where
+ * e is the exponent value in range 0..127
+ * m is the mantissa value in range 0..511
+ */
+struct mali_c55_af_statistics {
+ __u16 intensity_stats;
+ __u16 edge_stats;
+} __attribute__((packed));
+
+/**
+ * struct mali_c55_stats_buffer - 3A statistics for the mali-c55 ISP
+ *
+ * @ae_1024bin_hist: 1024-bin frame-global pixel intensity histogram
+ * @iridix_1024bin_hist: Post-Iridix block 1024-bin histogram
+ * @ae_5bin_hists: 5-bin pixel intensity histograms for AEC
+ * @reserved1: Undefined buffer space
+ * @awb_ratios: Color balance ratios for Auto White Balance
+ * @reserved2: Undefined buffer space
+ * @af_statistics: Pixel intensity statistics for Auto Focus
+ * @reserved3: Undefined buffer space
+ *
+ * This struct describes the metering statistics space in the Mali-C55 ISP's
+ * hardware in its entirety. The space between each defined area is marked as
+ * "unknown" and may not be 0, but should not be used. The @ae_5bin_hists,
+ * @awb_ratios and @af_statistics members are arrays of statistics per-zone.
+ * The zones are arranged in the array in raster order starting from the top
+ * left corner of the image.
+ */
+
+struct mali_c55_stats_buffer {
+ struct mali_c55_ae_1024bin_hist ae_1024bin_hist;
+ struct mali_c55_ae_1024bin_hist iridix_1024bin_hist;
+ struct mali_c55_ae_5bin_hist ae_5bin_hists[MALI_C55_MAX_ZONES];
+ __u32 reserved1[14];
+ struct mali_c55_awb_average_ratios awb_ratios[MALI_C55_MAX_ZONES];
+ __u32 reserved2[14];
+ struct mali_c55_af_statistics af_statistics[MALI_C55_MAX_ZONES];
+ __u32 reserved3[15];
+} __attribute__((packed));
+
+#endif /* __UAPI_MALI_C55_CONFIG_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 12/18] media: platform: Add mali-c55 3a stats devnode
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
` (10 preceding siblings ...)
2024-07-09 13:28 ` [PATCH v6 11/18] media: uapi: Add 3a stats buffer for mali-c55 Daniel Scally
@ 2024-07-09 13:29 ` Daniel Scally
2024-07-22 14:50 ` Laurent Pinchart
2024-07-09 13:29 ` [PATCH v6 13/18] media: platform: Fill stats buffer on ISP_START Daniel Scally
` (5 subsequent siblings)
17 siblings, 1 reply; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:29 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
Add a new code file to govern the 3a statistics capture node.
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- Fixed mising includes
- Minor renames and formatting
- Reworked mali_c55_stats_metering_complete() so it could only return
buffers when both halves of the DMA read were done
- Terminate dma transfers on streamoff
Changes in v5:
- New patch
drivers/media/platform/arm/mali-c55/Makefile | 1 +
.../platform/arm/mali-c55/mali-c55-common.h | 29 ++
.../platform/arm/mali-c55/mali-c55-core.c | 15 +
.../platform/arm/mali-c55/mali-c55-isp.c | 11 +
.../arm/mali-c55/mali-c55-registers.h | 3 +
.../platform/arm/mali-c55/mali-c55-stats.c | 373 ++++++++++++++++++
6 files changed, 432 insertions(+)
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-stats.c
diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile
index 9178ac35e50e..b5a22d414479 100644
--- a/drivers/media/platform/arm/mali-c55/Makefile
+++ b/drivers/media/platform/arm/mali-c55/Makefile
@@ -4,6 +4,7 @@ mali-c55-y := mali-c55-capture.o \
mali-c55-core.o \
mali-c55-isp.o \
mali-c55-resizer.o \
+ mali-c55-stats.o \
mali-c55-tpg.o
obj-$(CONFIG_VIDEO_MALI_C55) += mali-c55.o
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
index f7764a938e9f..136c785c68ba 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-common.h
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
@@ -49,6 +49,7 @@ enum mali_c55_isp_pads {
MALI_C55_ISP_PAD_SINK_VIDEO,
MALI_C55_ISP_PAD_SOURCE_VIDEO,
MALI_C55_ISP_PAD_SOURCE_BYPASS,
+ MALI_C55_ISP_PAD_SOURCE_STATS,
MALI_C55_ISP_NUM_PADS,
};
@@ -160,6 +161,29 @@ struct mali_c55_cap_dev {
bool streaming;
};
+struct mali_c55_stats_buf {
+ struct vb2_v4l2_buffer vb;
+ unsigned int segments_remaining;
+ struct list_head queue;
+ bool failed;
+};
+
+struct mali_c55_stats {
+ struct mali_c55 *mali_c55;
+ struct video_device vdev;
+ struct dma_chan *channel;
+ struct vb2_queue queue;
+ struct media_pad pad;
+ /* Mutex to provide to vb2 */
+ struct mutex lock;
+
+ struct {
+ /* Spinlock to guard buffer queue */
+ spinlock_t lock;
+ struct list_head queue;
+ } buffers;
+};
+
enum mali_c55_config_spaces {
MALI_C55_CONFIG_PING,
MALI_C55_CONFIG_PONG,
@@ -202,6 +226,7 @@ struct mali_c55 {
struct mali_c55_isp isp;
struct mali_c55_resizer resizers[MALI_C55_NUM_RSZS];
struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS];
+ struct mali_c55_stats stats;
struct mali_c55_context context;
enum mali_c55_config_spaces next_config;
@@ -229,6 +254,8 @@ int mali_c55_register_resizers(struct mali_c55 *mali_c55);
void mali_c55_unregister_resizers(struct mali_c55 *mali_c55);
int mali_c55_register_capture_devs(struct mali_c55 *mali_c55);
void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55);
+int mali_c55_register_stats(struct mali_c55 *mali_c55);
+void mali_c55_unregister_stats(struct mali_c55 *mali_c55);
struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55);
void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
enum mali_c55_planes plane);
@@ -243,5 +270,7 @@ const struct mali_c55_isp_fmt *
mali_c55_isp_get_mbus_config_by_code(u32 code);
const struct mali_c55_isp_fmt *
mali_c55_isp_get_mbus_config_by_index(u32 index);
+void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
+ enum mali_c55_config_spaces cfg_space);
#endif /* _MALI_C55_COMMON_H */
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
index dbc07d12d3a3..eedc8f450184 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
@@ -374,6 +374,16 @@ static int mali_c55_create_links(struct mali_c55 *mali_c55)
}
}
+ ret = media_create_pad_link(&mali_c55->isp.sd.entity,
+ MALI_C55_ISP_PAD_SOURCE_STATS,
+ &mali_c55->stats.vdev.entity, 0,
+ MEDIA_LNK_FL_ENABLED);
+ if (ret) {
+ dev_err(mali_c55->dev,
+ "failed to link ISP and 3a stats node\n");
+ goto err_remove_links;
+ }
+
return 0;
err_remove_links:
@@ -388,6 +398,7 @@ static void mali_c55_unregister_entities(struct mali_c55 *mali_c55)
mali_c55_unregister_isp(mali_c55);
mali_c55_unregister_resizers(mali_c55);
mali_c55_unregister_capture_devs(mali_c55);
+ mali_c55_unregister_stats(mali_c55);
}
static int mali_c55_register_entities(struct mali_c55 *mali_c55)
@@ -410,6 +421,10 @@ static int mali_c55_register_entities(struct mali_c55 *mali_c55)
if (ret)
goto err_unregister_entities;
+ ret = mali_c55_register_stats(mali_c55);
+ if (ret)
+ goto err_unregister_entities;
+
ret = mali_c55_create_links(mali_c55);
if (ret)
goto err_unregister_entities;
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
index f784983a4ccc..2f450c00300a 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
@@ -5,6 +5,8 @@
* Copyright (C) 2024 Ideas on Board Oy
*/
+#include <linux/media/arm/mali-c55-config.h>
+
#include <linux/delay.h>
#include <linux/iopoll.h>
#include <linux/property.h>
@@ -460,6 +462,14 @@ static int mali_c55_isp_init_state(struct v4l2_subdev *sd,
in_crop->width = MALI_C55_DEFAULT_WIDTH;
in_crop->height = MALI_C55_DEFAULT_HEIGHT;
+ src_fmt = v4l2_subdev_state_get_format(state,
+ MALI_C55_ISP_PAD_SOURCE_STATS);
+
+ src_fmt->width = sizeof(struct mali_c55_stats_buffer);
+ src_fmt->height = 1;
+ src_fmt->field = V4L2_FIELD_NONE;
+ src_fmt->code = MEDIA_BUS_FMT_METADATA_FIXED;
+
return 0;
}
@@ -490,6 +500,7 @@ int mali_c55_register_isp(struct mali_c55 *mali_c55)
MEDIA_PAD_FL_MUST_CONNECT;
isp->pads[MALI_C55_ISP_PAD_SOURCE_VIDEO].flags = MEDIA_PAD_FL_SOURCE;
isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE;
+ isp->pads[MALI_C55_ISP_PAD_SOURCE_STATS].flags = MEDIA_PAD_FL_SOURCE;
ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS,
isp->pads);
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
index 0a391f8a043e..e72e749b81d5 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
@@ -103,6 +103,9 @@ enum mali_c55_interrupts {
#define MALI_C55_VC_START(v) ((v) & 0xffff)
#define MALI_C55_VC_SIZE(v) (((v) & 0xffff) << 16)
+#define MALI_C55_REG_1024BIN_HIST 0x054a8
+#define MALI_C55_1024BIN_HIST_SIZE 4096
+
/* Ping/Pong Configuration Space */
#define MALI_C55_REG_BASE_ADDR 0x18e88
#define MALI_C55_REG_BYPASS_0 0x18eac
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-stats.c b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
new file mode 100644
index 000000000000..38a17fb5d052
--- /dev/null
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
@@ -0,0 +1,373 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM Mali-C55 ISP Driver - 3A Statistics capture device
+ *
+ * Copyright (C) 2023 Ideas on Board Oy
+ */
+
+#include <linux/container_of.h>
+#include <linux/dev_printk.h>
+#include <linux/dmaengine.h>
+#include <linux/list.h>
+#include <linux/media/arm/mali-c55-config.h>
+#include <linux/mutex.h>
+#include <linux/spinlock.h>
+#include <linux/string.h>
+
+#include <media/media-entity.h>
+#include <media/v4l2-dev.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fh.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mali-c55-common.h"
+#include "mali-c55-registers.h"
+
+static const unsigned int metering_space_addrs[] = {
+ [MALI_C55_CONFIG_PING] = 0x095ac,
+ [MALI_C55_CONFIG_PONG] = 0x2156c,
+};
+
+static int mali_c55_stats_enum_fmt_meta_cap(struct file *file, void *fh,
+ struct v4l2_fmtdesc *f)
+{
+ if (f->index)
+ return -EINVAL;
+
+ f->pixelformat = V4L2_META_FMT_MALI_C55_STATS;
+
+ return 0;
+}
+
+static int mali_c55_stats_g_fmt_meta_cap(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ static const struct v4l2_meta_format mfmt = {
+ .dataformat = V4L2_META_FMT_MALI_C55_STATS,
+ .buffersize = sizeof(struct mali_c55_stats_buffer)
+ };
+
+ f->fmt.meta = mfmt;
+
+ return 0;
+}
+
+static int mali_c55_stats_querycap(struct file *file,
+ void *priv, struct v4l2_capability *cap)
+{
+ strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver));
+ strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card));
+
+ return 0;
+}
+
+static const struct v4l2_ioctl_ops mali_c55_stats_v4l2_ioctl_ops = {
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+ .vidioc_enum_fmt_meta_cap = mali_c55_stats_enum_fmt_meta_cap,
+ .vidioc_g_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
+ .vidioc_s_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
+ .vidioc_try_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
+ .vidioc_querycap = mali_c55_stats_querycap,
+ .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_file_operations mali_c55_stats_v4l2_fops = {
+ .owner = THIS_MODULE,
+ .unlocked_ioctl = video_ioctl2,
+ .open = v4l2_fh_open,
+ .release = vb2_fop_release,
+ .poll = vb2_fop_poll,
+ .mmap = vb2_fop_mmap,
+};
+
+static int
+mali_c55_stats_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
+ unsigned int *num_planes, unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct mali_c55_stats *stats = vb2_get_drv_priv(q);
+
+ if (*num_planes && *num_planes > 1)
+ return -EINVAL;
+
+ if (sizes[0] && sizes[0] < sizeof(struct mali_c55_stats_buffer))
+ return -EINVAL;
+
+ *num_planes = 1;
+
+ if (!sizes[0])
+ sizes[0] = sizeof(struct mali_c55_stats_buffer);
+
+ if (stats->channel)
+ alloc_devs[0] = stats->channel->device->dev;
+
+ return 0;
+}
+
+static void mali_c55_stats_buf_queue(struct vb2_buffer *vb)
+{
+ struct mali_c55_stats *stats = vb2_get_drv_priv(vb->vb2_queue);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct mali_c55_stats_buf *buf = container_of(vbuf,
+ struct mali_c55_stats_buf, vb);
+
+ vb2_set_plane_payload(vb, 0, sizeof(struct mali_c55_stats_buffer));
+ buf->segments_remaining = 2;
+ buf->failed = false;
+
+ spin_lock(&stats->buffers.lock);
+ list_add_tail(&buf->queue, &stats->buffers.queue);
+ spin_unlock(&stats->buffers.lock);
+}
+
+static int mali_c55_stats_start_streaming(struct vb2_queue *q,
+ unsigned int count)
+{
+ struct mali_c55_stats *stats = vb2_get_drv_priv(q);
+ struct mali_c55 *mali_c55 = stats->mali_c55;
+ int ret;
+
+ ret = video_device_pipeline_start(&stats->vdev,
+ &stats->mali_c55->pipe);
+ if (ret)
+ return ret;
+
+ if (mali_c55->pipe.start_count == mali_c55->pipe.required_count) {
+ ret = v4l2_subdev_enable_streams(&mali_c55->isp.sd,
+ MALI_C55_ISP_PAD_SOURCE_VIDEO,
+ BIT(0));
+ if (ret)
+ goto err_stop_pipeline;
+ }
+
+ return 0;
+
+err_stop_pipeline:
+ video_device_pipeline_stop(&stats->vdev);
+
+ return ret;
+}
+
+static void mali_c55_stats_stop_streaming(struct vb2_queue *q)
+{
+ struct mali_c55_stats *stats = vb2_get_drv_priv(q);
+ struct mali_c55_stats_buf *buf, *tmp;
+
+ dmaengine_terminate_sync(stats->channel);
+
+ spin_lock(&stats->buffers.lock);
+
+ list_for_each_entry_safe(buf, tmp, &stats->buffers.queue, queue) {
+ list_del(&buf->queue);
+ vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+ }
+
+ spin_unlock(&stats->buffers.lock);
+
+ video_device_pipeline_stop(&stats->vdev);
+}
+
+static const struct vb2_ops mali_c55_stats_vb2_ops = {
+ .queue_setup = mali_c55_stats_queue_setup,
+ .buf_queue = mali_c55_stats_buf_queue,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+ .start_streaming = mali_c55_stats_start_streaming,
+ .stop_streaming = mali_c55_stats_stop_streaming,
+};
+
+static void
+mali_c55_stats_metering_complete(void *param,
+ const struct dmaengine_result *result)
+{
+ struct mali_c55_stats_buf *buf = param;
+
+ if (result->result != DMA_TRANS_NOERROR)
+ buf->failed = true;
+
+ if (!--buf->segments_remaining)
+ vb2_buffer_done(&buf->vb.vb2_buf, buf->failed ?
+ VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE);
+}
+
+static int mali_c55_stats_dma_xfer(struct mali_c55_stats *stats, dma_addr_t src,
+ dma_addr_t dst,
+ struct mali_c55_stats_buf *buf,
+ size_t length)
+{
+ struct dma_async_tx_descriptor *tx;
+ dma_cookie_t cookie;
+
+ tx = dmaengine_prep_dma_memcpy(stats->channel, dst, src, length, 0);
+ if (!tx) {
+ dev_err(stats->mali_c55->dev, "failed to prep stats DMA\n");
+ return -EIO;
+ }
+
+ tx->callback_result = mali_c55_stats_metering_complete;
+ tx->callback_param = buf;
+
+ cookie = dmaengine_submit(tx);
+ if (dma_submit_error(cookie)) {
+ dev_err(stats->mali_c55->dev, "failed to submit stats DMA\n");
+ return -EIO;
+ }
+
+ dma_async_issue_pending(stats->channel);
+ return 0;
+}
+
+void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
+ enum mali_c55_config_spaces cfg_space)
+{
+ struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
+ struct mali_c55_stats *stats = &mali_c55->stats;
+ struct mali_c55_stats_buf *buf = NULL;
+ dma_addr_t src, dst;
+ size_t length;
+ int ret;
+
+ spin_lock(&stats->buffers.lock);
+ if (!list_empty(&stats->buffers.queue)) {
+ buf = list_first_entry(&stats->buffers.queue,
+ struct mali_c55_stats_buf, queue);
+ list_del(&buf->queue);
+ }
+ spin_unlock(&stats->buffers.lock);
+
+ if (!buf)
+ return;
+
+ buf->vb.sequence = mali_c55->isp.frame_sequence;
+ buf->vb.vb2_buf.timestamp = ktime_get_boottime_ns();
+
+ /*
+ * There are in fact two noncontiguous sections of the ISP's
+ * memory space that hold statistics for 3a algorithms to use: A
+ * section in each config space and a global section holding
+ * histograms which is double buffered and so holds data for the
+ * last frame. We need to read both.
+ */
+ src = ctx->base + MALI_C55_REG_1024BIN_HIST;
+ dst = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
+
+ ret = mali_c55_stats_dma_xfer(stats, src, dst, buf,
+ MALI_C55_1024BIN_HIST_SIZE);
+ if (ret)
+ goto err_fail_buffer;
+
+ src = ctx->base + metering_space_addrs[cfg_space];
+ dst += MALI_C55_1024BIN_HIST_SIZE;
+
+ length = sizeof(struct mali_c55_stats_buffer) - MALI_C55_1024BIN_HIST_SIZE;
+ ret = mali_c55_stats_dma_xfer(stats, src, dst, buf, length);
+ if (ret) {
+ dmaengine_terminate_sync(stats->channel);
+ goto err_fail_buffer;
+ }
+
+ return;
+
+err_fail_buffer:
+ vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+}
+
+void mali_c55_unregister_stats(struct mali_c55 *mali_c55)
+{
+ struct mali_c55_stats *stats = &mali_c55->stats;
+
+ if (!video_is_registered(&stats->vdev))
+ return;
+
+ vb2_video_unregister_device(&stats->vdev);
+ media_entity_cleanup(&stats->vdev.entity);
+ dma_release_channel(stats->channel);
+ mutex_destroy(&stats->lock);
+}
+
+int mali_c55_register_stats(struct mali_c55 *mali_c55)
+{
+ struct mali_c55_stats *stats = &mali_c55->stats;
+ struct video_device *vdev = &stats->vdev;
+ struct vb2_queue *vb2q = &stats->queue;
+ dma_cap_mask_t mask;
+ int ret;
+
+ mutex_init(&stats->lock);
+ INIT_LIST_HEAD(&stats->buffers.queue);
+
+ dma_cap_zero(mask);
+ dma_cap_set(DMA_MEMCPY, mask);
+
+ stats->channel = dma_request_channel(mask, 0, NULL);
+ if (!stats->channel) {
+ ret = -ENODEV;
+ goto err_destroy_mutex;
+ }
+
+ stats->pad.flags = MEDIA_PAD_FL_SINK;
+ ret = media_entity_pads_init(&stats->vdev.entity, 1, &stats->pad);
+ if (ret)
+ goto err_release_dma_channel;
+
+ vb2q->type = V4L2_BUF_TYPE_META_CAPTURE;
+ vb2q->io_modes = VB2_MMAP | VB2_DMABUF;
+ vb2q->drv_priv = stats;
+ vb2q->mem_ops = &vb2_dma_contig_memops;
+ vb2q->ops = &mali_c55_stats_vb2_ops;
+ vb2q->buf_struct_size = sizeof(struct mali_c55_stats_buf);
+ vb2q->min_queued_buffers = 1;
+ vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+ vb2q->lock = &stats->lock;
+ vb2q->dev = stats->channel->device->dev;
+
+ ret = vb2_queue_init(vb2q);
+ if (ret) {
+ dev_err(mali_c55->dev, "stats vb2 queue init failed\n");
+ goto err_cleanup_entity;
+ }
+
+ strscpy(stats->vdev.name, "mali-c55 3a stats", sizeof(stats->vdev.name));
+ vdev->release = video_device_release_empty;
+ vdev->fops = &mali_c55_stats_v4l2_fops;
+ vdev->ioctl_ops = &mali_c55_stats_v4l2_ioctl_ops;
+ vdev->lock = &stats->lock;
+ vdev->v4l2_dev = &mali_c55->v4l2_dev;
+ vdev->queue = &stats->queue;
+ vdev->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
+ vdev->vfl_dir = VFL_DIR_RX;
+ video_set_drvdata(vdev, stats);
+
+ ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+ if (ret) {
+ dev_err(mali_c55->dev,
+ "failed to register stats video device\n");
+ goto err_release_vb2q;
+ }
+
+ stats->mali_c55 = mali_c55;
+
+ return 0;
+
+err_release_vb2q:
+ vb2_queue_release(vb2q);
+err_cleanup_entity:
+ media_entity_cleanup(&stats->vdev.entity);
+err_release_dma_channel:
+ dma_release_channel(stats->channel);
+err_destroy_mutex:
+ mutex_destroy(&stats->lock);
+
+ return ret;
+}
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 13/18] media: platform: Fill stats buffer on ISP_START
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
` (11 preceding siblings ...)
2024-07-09 13:29 ` [PATCH v6 12/18] media: platform: Add mali-c55 3a stats devnode Daniel Scally
@ 2024-07-09 13:29 ` Daniel Scally
2024-07-30 21:46 ` Laurent Pinchart
2024-07-09 13:29 ` [PATCH v6 14/18] Documentation: mali-c55: Add Statistics documentation Daniel Scally
` (4 subsequent siblings)
17 siblings, 1 reply; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:29 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
On ISP_START, fill the stats buffer by reading out the metering space
in the ISP's memory. This is done for the non-active config just as
the dma transfer of the registers is. To acheive that, move the
checking of the current config outside of mali_c55_swap_next_config()
so we can use it for both functions.
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- None
Changes in v5:
- New patch
.../platform/arm/mali-c55/mali-c55-core.c | 34 ++++++++++++++-----
1 file changed, 26 insertions(+), 8 deletions(-)
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
index eedc8f450184..ed0db34767a4 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
@@ -567,15 +567,9 @@ static int mali_c55_check_hwcfg(struct mali_c55 *mali_c55)
return 0;
}
-static void mali_c55_swap_next_config(struct mali_c55 *mali_c55)
+static void mali_c55_swap_next_config(struct mali_c55 *mali_c55, u32 next_config)
{
struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
- u32 curr_config, next_config;
-
- curr_config = mali_c55_read(mali_c55, MALI_C55_REG_PING_PONG_READ);
- curr_config = (curr_config & MALI_C55_REG_PING_PONG_READ_MASK)
- >> (ffs(MALI_C55_REG_PING_PONG_READ_MASK) - 1);
- next_config = curr_config ^ 1;
mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG,
MALI_C55_REG_MCU_CONFIG_WRITE_MASK,
@@ -588,6 +582,7 @@ static irqreturn_t mali_c55_isr(int irq, void *context)
{
struct device *dev = context;
struct mali_c55 *mali_c55 = dev_get_drvdata(dev);
+ u32 curr_config, next_config;
u32 interrupt_status;
unsigned int i, j;
@@ -612,7 +607,30 @@ static irqreturn_t mali_c55_isr(int irq, void *context)
for (j = i; j < MALI_C55_NUM_CAP_DEVS; j++)
mali_c55_set_next_buffer(&mali_c55->cap_devs[j]);
- mali_c55_swap_next_config(mali_c55);
+ /*
+ * When the ISP starts a frame we have some work to do:
+ *
+ * 1. Copy over the config for the **next** frame
+ * 2. Read out the metering stats for the **last** frame
+ */
+
+ curr_config = mali_c55_read(mali_c55,
+ MALI_C55_REG_PING_PONG_READ);
+ curr_config &= MALI_C55_REG_PING_PONG_READ_MASK;
+ curr_config >>= ffs(MALI_C55_REG_PING_PONG_READ_MASK) - 1;
+ next_config = curr_config ^ 1;
+
+ /*
+ * The ordering of these two is currently important as
+ * mali_c55_stats_fill_buffer() is asynchronous whereas
+ * mali_c55_swap_next_config() is not.
+ *
+ * TODO: Should mali_c55_swap_next_config() be async?
+ */
+ mali_c55_stats_fill_buffer(mali_c55,
+ next_config ? MALI_C55_CONFIG_PING :
+ MALI_C55_CONFIG_PONG);
+ mali_c55_swap_next_config(mali_c55, next_config);
break;
case MALI_C55_IRQ_ISP_DONE:
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 14/18] Documentation: mali-c55: Add Statistics documentation
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
` (12 preceding siblings ...)
2024-07-09 13:29 ` [PATCH v6 13/18] media: platform: Fill stats buffer on ISP_START Daniel Scally
@ 2024-07-09 13:29 ` Daniel Scally
2024-07-09 13:29 ` [PATCH v6 15/18] media: mali-c55: Add image formats for Mali-C55 parameters buffer Daniel Scally
` (3 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:29 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
Add documentation explaining the ability to capture statistics from
the mali-c55 driver's new V4L2 device, as well as the various tap
points from which those statistics can be drawn in the ISP's
processing flow. Additionally add a page detailing the new V4L2
meta format for the mali-c55 statistics.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- None
Changes in v5:
- New patch
Documentation/admin-guide/media/mali-c55.rst | 60 ++++++++++++++++++-
.../userspace-api/media/v4l/meta-formats.rst | 1 +
.../media/v4l/metafmt-arm-mali-c55.rst | 29 +++++++++
MAINTAINERS | 1 +
4 files changed, 90 insertions(+), 1 deletion(-)
create mode 100644 Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst
diff --git a/Documentation/admin-guide/media/mali-c55.rst b/Documentation/admin-guide/media/mali-c55.rst
index 72cdded507b3..7eaeac63ddf7 100644
--- a/Documentation/admin-guide/media/mali-c55.rst
+++ b/Documentation/admin-guide/media/mali-c55.rst
@@ -67,10 +67,11 @@ The driver has 4 V4L2 subdevices:
- `mali_c55 resizer fr`: The Full-Resolution pipe resizer
- `mali_c55 resizer ds`: The Downscale pipe resizer
-The driver has 2 V4L2 video devices:
+The driver has 3 V4L2 video devices:
- `mali-c55 fr`: The full-resolution pipe's capture device
- `mali-c55 ds`: The downscale pipe's capture device
+- `mali-c55 3a stats`: The 3A statistics capture device
Frame sequences are synchronised across to two capture devices, meaning if one
pipe is started later than the other the sequence numbers returned in its
@@ -333,6 +334,63 @@ configured, followed by formats in the appropriate places:
# Set format on the video device and stream
yavta -f RGB565 -s 1920x1080 -c10 /dev/video0
+.. _mali-c55-3a-stats:
+
+Capturing ISP Statistics
+========================
+
+The ISP is capable of producing statistics for consumption by image processing
+algorithms running in userspace. These statistics can be captured by queueing
+buffers to the `mali-c55 3a stats` V4L2 Device whilst the ISP is streaming. Only
+the :ref:`V4L2_META_FMT_MALI_C55_STATS <v4l2-meta-fmt-mali-c55-3a-stats>`
+format is supported, so no format-setting need be done:
+
+.. code-block:: none
+
+ # We assume the media graph has been configured to support RGB565 capture
+ # from the mali-c55 fr V4L2 Device, which is at /dev/video0. The statistics
+ # V4L2 device is at /dev/video3
+
+ yavta -f RGB565 -s 1920x1080 -c32 /dev/video0 && \
+ yavta -c10 -F /dev/video3
+
+The layout of the buffer is described by :c:type:`mali_c55_stats_buffer`,
+but broadly statistics are generated to support three image processing
+algorithms; AEXP (Auto-Exposure), AWB (Auto-White Balance) and AF (Auto-Focus).
+These stats can be drawn from various places in the Mali C55 ISP pipeline, known
+as "tap points". This high-level block diagram is intended to explain where in
+the processing flow the statistics can be drawn from::
+
+ +--> AEXP-2 +----> AEXP-1 +--> AF-0
+ | +----> AF-1 |
+ | | |
+ +---------+ | +--------------+ | +--------------+ |
+ | Input +-+-->+ Digital Gain +---+-->+ Black Level +---+---+
+ +---------+ +--------------+ +--------------+ |
+ +-----------------------------------------------------------------+
+ |
+ | +--------------+ +---------+ +----------------+
+ +-->| Sinter Noise +-+ White +--+--->| Lens Shading +--+---------------+
+ | Reduction | | Balance | | | | | |
+ +--------------+ +---------+ | +----------------+ | |
+ +---> AEXP-0 (A) +--> AEXP-0 (B) |
+ +--------------------------------------------------------------------------+
+ |
+ | +----------------+ +--------------+ +----------------+
+ +-->| Tone mapping +-+--->| Demosaicing +->+ Purple Fringe +-+-----------+
+ | | | +--------------+ | Correction | | |
+ +----------------+ +-> AEXP-IRIDIX +----------------+ +---> AWB-0 |
+ +----------------------------------------------------------------------------+
+ | +-------------+ +-------------+
+ +------------------->| Colour +---+--->| Output |
+ | Correction | | | Pipelines |
+ +-------------+ | +-------------+
+ +--> AWB-1
+
+At present all statistics are drawn from the 0th tap point for each algorithm;
+I.E. AEXP statistics from AEXP-0 (A), AWB statistics from AWB-0 and AF
+statistics from AF-0. In the future this will be configurable.
+
References
==========
.. [1] https://git.linuxtv.org/v4l-utils.git/
diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst
index c23aac823d2c..319ca7b91f6b 100644
--- a/Documentation/userspace-api/media/v4l/meta-formats.rst
+++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
@@ -12,6 +12,7 @@ These formats are used for the :ref:`metadata` interface only.
.. toctree::
:maxdepth: 1
+ metafmt-arm-mali-c55
metafmt-d4xx
metafmt-generic
metafmt-intel-ipu3
diff --git a/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst b/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst
new file mode 100644
index 000000000000..186e0deb9ece
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst
@@ -0,0 +1,29 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. _v4l2-meta-fmt-mali-c55-3a-stats:
+
+*************************************
+V4L2_META_FMT_MALI_C55_STATS ('C55S')
+*************************************
+
+3A Statistics
+=============
+
+The ISP device collects different statistics over an input bayer frame. Those
+statistics can be obtained by userspace from the
+:ref:`mali-c55 3a stats <mali-c55-3a-stats>` metadata capture video node, using
+the :c:type:`v4l2_meta_format` interface. The buffer contains a single instance
+of the C structure :c:type:`mali_c55_stats_buffer` defined in
+``mali-c55-config.h``, so the structure can be obtained from the buffer by:
+
+.. code-block:: C
+
+ struct mali_c55_stats_buffer *stats =
+ (struct mali_c55_stats_buffer *)buf;
+
+For details of the statistics see :c:type:`mali_c55_stats_buffer`.
+
+Arm Mali-C55 uAPI data types
+============================
+
+.. kernel-doc:: include/uapi/linux/media/arm/mali-c55-config.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 3b5c39b820cc..c8b20a1a1c1c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1764,6 +1764,7 @@ T: git git://linuxtv.org/media_tree.git
F: Documentation/admin-guide/media/mali-c55-graph.dot
F: Documentation/admin-guide/media/mali-c55.rst
F: Documentation/devicetree/bindings/media/arm,mali-c55.yaml
+F: Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst
F: drivers/media/platform/arm/mali-c55/
F: include/uapi/linux/media/arm/mali-c55-config.h
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 15/18] media: mali-c55: Add image formats for Mali-C55 parameters buffer
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
` (13 preceding siblings ...)
2024-07-09 13:29 ` [PATCH v6 14/18] Documentation: mali-c55: Add Statistics documentation Daniel Scally
@ 2024-07-09 13:29 ` Daniel Scally
2024-07-09 13:29 ` [PATCH v6 16/18] media: uapi: Add parameters structs to mali-c55-config.h Daniel Scally
` (2 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:29 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
From: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Add a new V4L2 meta format code for the Mali-C55 parameters.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- None
Changes in v5:
- New patch
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/videodev2.h | 1 +
2 files changed, 2 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 7e49a5a43077..54f3710ad79b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1456,6 +1456,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_META_FMT_VIVID: descr = "Vivid Metadata"; break;
case V4L2_META_FMT_RK_ISP1_PARAMS: descr = "Rockchip ISP1 3A Parameters"; break;
case V4L2_META_FMT_RK_ISP1_STAT_3A: descr = "Rockchip ISP1 3A Statistics"; break;
+ case V4L2_META_FMT_MALI_C55_PARAMS: descr = "ARM Mali-C55 ISP Parameters"; break;
case V4L2_META_FMT_MALI_C55_STATS: descr = "ARM Mali-C55 ISP 3A Statistics"; break;
case V4L2_PIX_FMT_NV12_8L128: descr = "NV12 (8x128 Linear)"; break;
case V4L2_PIX_FMT_NV12M_8L128: descr = "NV12M (8x128 Linear)"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 4fb09ef85772..d480a958614a 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -841,6 +841,7 @@ struct v4l2_pix_format {
#define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
#define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
+#define V4L2_META_FMT_MALI_C55_PARAMS v4l2_fourcc('C', '5', '5', 'P') /* ARM Mali-C55 Parameters */
#define V4L2_META_FMT_MALI_C55_STATS v4l2_fourcc('C', '5', '5', 'S') /* ARM Mali-C55 3A Statistics */
#ifdef __KERNEL__
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 16/18] media: uapi: Add parameters structs to mali-c55-config.h
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
` (14 preceding siblings ...)
2024-07-09 13:29 ` [PATCH v6 15/18] media: mali-c55: Add image formats for Mali-C55 parameters buffer Daniel Scally
@ 2024-07-09 13:29 ` Daniel Scally
2024-07-22 23:48 ` Laurent Pinchart
2024-07-09 13:29 ` [PATCH v6 17/18] media: platform: Add mali-c55 parameters video node Daniel Scally
2024-07-09 13:29 ` [PATCH v6 18/18] Documentation: mali-c55: Document the mali-c55 parameter setting Daniel Scally
17 siblings, 1 reply; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:29 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
Add structures describing the ISP parameters to mali-c55-config.h
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- Flagged which struct goes with which enum value from
enum mali_c55_param_block_type
- Used only types with well defined sizes in the structs
- Expanded the documentation for skipping in the AEXP histogram config
- Aligned the header struct to 64 bits
- Added a new union type to hold pointers to the parameter structs
Changes in v5:
- New patch
.../uapi/linux/media/arm/mali-c55-config.h | 763 ++++++++++++++++++
1 file changed, 763 insertions(+)
diff --git a/include/uapi/linux/media/arm/mali-c55-config.h b/include/uapi/linux/media/arm/mali-c55-config.h
index 21b453bdee5d..ea9872e9076f 100644
--- a/include/uapi/linux/media/arm/mali-c55-config.h
+++ b/include/uapi/linux/media/arm/mali-c55-config.h
@@ -179,4 +179,767 @@ struct mali_c55_stats_buffer {
__u32 reserved3[15];
} __attribute__((packed));
+/**
+ * enum mali_c55_param_buffer_version - Mali-C55 parameters block versioning
+ *
+ * @MALI_C55_PARAM_BUFFER_V1: First version of Mali-C55 parameters block
+ */
+enum mali_c55_param_buffer_version {
+ MALI_C55_PARAM_BUFFER_V1,
+};
+
+/**
+ * enum mali_c55_param_block_type - Enumeration of Mali-C55 parameter blocks
+ *
+ * This enumeration defines the types of Mali-C55 parameters block. Each block
+ * configures a specific processing block of the Mali-C55 ISP. The block
+ * type allows the driver to correctly interpret the parameters block data.
+ *
+ * It is the responsibility of userspace to correctly set the type of each
+ * parameters block.
+ *
+ * @MALI_C55_PARAM_BLOCK_SENSOR_OFFS: Sensor pre-shading black level offset
+ * @MALI_C55_PARAM_BLOCK_AEXP_HIST: Auto-exposure 1024-bin histogram
+ * configuration
+ * @MALI_C55_PARAM_BLOCK_AEXP_IHIST: Post-Iridix auto-exposure 1024-bin
+ * histogram configuration
+ * @MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS: Auto-exposure 1024-bin histogram
+ * weighting
+ * @MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS: Post-Iridix auto-exposure 1024-bin
+ * histogram weighting
+ * @MALI_C55_PARAM_BLOCK_DIGITAL_GAIN: Digital gain
+ * @MALI_C55_PARAM_BLOCK_AWB_GAINS: Auto-white balance gains
+ * @MALI_C55_PARAM_BLOCK_AWB_CONFIG: Auto-white balance statistics config
+ * @MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP: Auto-white balance gains for AEXP-0 tap
+ * @MALI_C55_PARAM_MESH_SHADING_CONFIG : Mesh shading tables configuration
+ * @MALI_C55_PARAM_MESH_SHADING_SELECTION: Mesh shading table selection
+ * @MALI_C55_PARAM_BLOCK_SENTINEL: First non-valid block index
+ */
+enum mali_c55_param_block_type {
+ MALI_C55_PARAM_BLOCK_SENSOR_OFFS,
+ MALI_C55_PARAM_BLOCK_AEXP_HIST,
+ MALI_C55_PARAM_BLOCK_AEXP_IHIST,
+ MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS,
+ MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS,
+ MALI_C55_PARAM_BLOCK_DIGITAL_GAIN,
+ MALI_C55_PARAM_BLOCK_AWB_GAINS,
+ MALI_C55_PARAM_BLOCK_AWB_CONFIG,
+ MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP,
+ MALI_C55_PARAM_MESH_SHADING_CONFIG,
+ MALI_C55_PARAM_MESH_SHADING_SELECTION,
+ MALI_C55_PARAM_BLOCK_SENTINEL,
+};
+
+/**
+ * struct mali_c55_params_block_header - Mali-C55 parameter block header
+ *
+ * This structure represents the common part of all the ISP configuration
+ * blocks. Each parameters block embeds an instance of this structure type
+ * as its first member, followed by the block-specific configuration data. The
+ * driver inspects this common header to discern the block type and its size and
+ * properly handle the block content by casting it to the correct block-specific
+ * type.
+ *
+ * The @type field is one of the values enumerated by
+ * :c:type:`mali_c55_param_block_type` and specifies how the data should be
+ * interpreted by the driver. The @size field specifies the size of the
+ * parameters block and is used by the driver for validation purposes. The
+ * @enabled field specifies if the ISP block should be enabled (and configured
+ * according to the provided parameters) or disabled.
+ *
+ * Userspace is responsible for correctly populating the parameters block header
+ * fields (@type, @enabled and @size) and correctly populate the block-specific
+ * parameters.
+ *
+ * For example:
+ *
+ * .. code-block:: c
+ *
+ * void populate_sensor_offs(struct mali_c55_params_block_header *block) {
+ * block->type = MALI_C55_PARAM_BLOCK_SENSOR_OFFS;
+ * block->enabled = true;
+ * block->size = sizeof(struct mali_c55_params_sensor_off_preshading);
+ *
+ * struct mali_c55_params_sensor_off_preshading *sensor_offs =
+ * (struct mali_c55_params_sensor_off_preshading *)block;
+ *
+ * sensor_offs->chan00 = offset00;
+ * sensor_offs->chan01 = offset01;
+ * sensor_offs->chan10 = offset10;
+ * sensor_offs->chan11 = offset11;
+ * }
+ *
+ * @type: The parameters block type from :c:type:`mali_c55_param_block_type`
+ * @enabled: Block enabled/disabled flag
+ * @size: Size (in bytes) of the parameters block
+ */
+struct mali_c55_params_block_header {
+ __u8 type;
+ __u8 enabled;
+ __u32 size;
+} __attribute__((aligned(8)));
+
+/**
+ * struct mali_c55_params_sensor_off_preshading - offset subtraction for each
+ * color channel
+ *
+ * Provides removal of the sensor black level from the sensor data. Separate
+ * offsets are provided for each of the four Bayer component color channels
+ * which are defaulted to R, Gr, Gb, B.
+ *
+ * header.type should be set to MALI_C55_PARAM_BLOCK_SENSOR_OFFS from
+ * :c:type:`mali_c55_param_block_type` for this block.
+ *
+ * @header: The Mali-C55 parameters block header
+ * @chan00: Offset for color channel 00 (default: R)
+ * @chan01: Offset for color channel 01 (default: Gr)
+ * @chan10: Offset for color channel 10 (default: Gb)
+ * @chan11: Offset for color channel 11 (default: B)
+ */
+struct mali_c55_params_sensor_off_preshading {
+ struct mali_c55_params_block_header header;
+ __u32 chan00;
+ __u32 chan01;
+ __u32 chan10;
+ __u32 chan11;
+};
+
+/**
+ * enum mali_c55_aexp_hist_tap_points - Tap points for the AEXP histogram
+ * @MALI_C55_AEXP_HIST_TAP_WB: After static white balance
+ * @MALI_C55_AEXP_HIST_TAP_FS: After WDR Frame Stitch
+ * @MALI_C55_AEXP_HIST_TAP_TPG: After the test pattern generator
+ */
+enum mali_c55_aexp_hist_tap_points {
+ MALI_C55_AEXP_HIST_TAP_WB = 0,
+ MALI_C55_AEXP_HIST_TAP_FS,
+ MALI_C55_AEXP_HIST_TAP_TPG,
+};
+
+/**
+ * enum mali_c55_aexp_skip_x - Horizontal pixel skipping
+ * @MALI_C55_AEXP_SKIP_X_EVERY_2ND: Collect every 2nd pixel horizontally
+ * @MALI_C55_AEXP_SKIP_X_EVERY_3RD: Collect every 3rd pixel horizontally
+ * @MALI_C55_AEXP_SKIP_X_EVERY_4TH: Collect every 4th pixel horizontally
+ * @MALI_C55_AEXP_SKIP_X_EVERY_5TH: Collect every 5th pixel horizontally
+ * @MALI_C55_AEXP_SKIP_X_EVERY_8TH: Collect every 8th pixel horizontally
+ * @MALI_C55_AEXP_SKIP_X_EVERY_9TH: Collect every 9th pixel horizontally
+ */
+enum mali_c55_aexp_skip_x {
+ MALI_C55_AEXP_SKIP_X_EVERY_2ND,
+ MALI_C55_AEXP_SKIP_X_EVERY_3RD,
+ MALI_C55_AEXP_SKIP_X_EVERY_4TH,
+ MALI_C55_AEXP_SKIP_X_EVERY_5TH,
+ MALI_C55_AEXP_SKIP_X_EVERY_8TH,
+ MALI_C55_AEXP_SKIP_X_EVERY_9TH
+};
+
+/**
+ * enum mali_c55_aexp_skip_y - Vertical pixel skipping
+ * @MALI_C55_AEXP_SKIP_Y_ALL: Collect every single pixel vertically
+ * @MALI_C55_AEXP_SKIP_Y_EVERY_2ND: Collect every 2nd pixel vertically
+ * @MALI_C55_AEXP_SKIP_Y_EVERY_3RD: Collect every 3rd pixel vertically
+ * @MALI_C55_AEXP_SKIP_Y_EVERY_4TH: Collect every 4th pixel vertically
+ * @MALI_C55_AEXP_SKIP_Y_EVERY_5TH: Collect every 5th pixel vertically
+ * @MALI_C55_AEXP_SKIP_Y_EVERY_8TH: Collect every 8th pixel vertically
+ * @MALI_C55_AEXP_SKIP_Y_EVERY_9TH: Collect every 9th pixel vertically
+ */
+enum mali_c55_aexp_skip_y {
+ MALI_C55_AEXP_SKIP_Y_ALL,
+ MALI_C55_AEXP_SKIP_Y_EVERY_2ND,
+ MALI_C55_AEXP_SKIP_Y_EVERY_3RD,
+ MALI_C55_AEXP_SKIP_Y_EVERY_4TH,
+ MALI_C55_AEXP_SKIP_Y_EVERY_5TH,
+ MALI_C55_AEXP_SKIP_Y_EVERY_8TH,
+ MALI_C55_AEXP_SKIP_Y_EVERY_9TH
+};
+
+/**
+ * enum mali_c55_aexp_row_column_offset - Start from the first or second row or
+ * column
+ * @MALI_C55_AEXP_FIRST_ROW_OR_COL: Start from the first row / column
+ * @MALI_C55_AEXP_SECOND_ROW_OR_COL: Start from the second row / column
+ */
+enum mali_c55_aexp_row_column_offset {
+ MALI_C55_AEXP_FIRST_ROW_OR_COL = 1,
+ MALI_C55_AEXP_SECOND_ROW_OR_COL = 2,
+};
+
+/**
+ * enum mali_c55_aexp_hist_plane_mode - Mode for the AEXP Histograms
+ * @MALI_C55_AEXP_HIST_COMBINED: All color planes in one 1024-bin histogram
+ * @MALI_C55_AEXP_HIST_SEPARATE: Each color plane in one 256-bin histogram with a bin width of 16
+ * @MALI_C55_AEXP_HIST_FOCUS_00: Top left plane in the first bank, rest in second bank
+ * @MALI_C55_AEXP_HIST_FOCUS_01: Top right plane in the first bank, rest in second bank
+ * @MALI_C55_AEXP_HIST_FOCUS_10: Bottom left plane in the first bank, rest in second bank
+ * @MALI_C55_AEXP_HIST_FOCUS_11: Bottom right plane in the first bank, rest in second bank
+ *
+ * In the "focus" modes statistics are collected into two 512-bin histograms
+ * with a bin width of 8. One colour plane is in the first histogram with the
+ * remainder combined into the second. The four options represent which of the
+ * four positions in a bayer pattern are the focused plane.
+ */
+enum mali_c55_aexp_hist_plane_mode {
+ MALI_C55_AEXP_HIST_COMBINED = 0,
+ MALI_C55_AEXP_HIST_SEPARATE = 1,
+ MALI_C55_AEXP_HIST_FOCUS_00 = 4,
+ MALI_C55_AEXP_HIST_FOCUS_01 = 5,
+ MALI_C55_AEXP_HIST_FOCUS_10 = 6,
+ MALI_C55_AEXP_HIST_FOCUS_11 = 7,
+};
+
+/**
+ * struct mali_c55_params_aexp_hist - configuration for AEXP metering hists
+ *
+ * This struct allows users to configure the 1024-bin AEXP histograms. Broadly
+ * speaking the parameters allow you to mask particular regions of the image and
+ * to select different kinds of histogram.
+ *
+ * The skip_x, offset_x, skip_y and offset_y fields allow users to ignore or
+ * mask pixels in the frame by their position relative to the top left pixel.
+ * First, the skip_y, offset_x and offset_y fields define a 2x2 pixel pattern of
+ * which pixels covered by the pattern will be counted in the statistics.
+ *
+ * If skip_y == 0 then two pixels from each coveredregion will be counted. If
+ * both offset_x and offset_y are zero, then the two left-most pixels in each
+ * 2x2 pixel region covered by the pattern will be counted. Setting offset_x = 1
+ * will discount the top left pixel and count the top right pixel. Setting
+ * offset_y = 1 will discount the bottom left pixel and count the bottom right
+ * pixel.
+ *
+ * If skip_y != 0 then only a single pixel from each region covered by the
+ * pattern will be counted. In this case offset_x controls whether the pixel
+ * that's counted is in the left (if offset_x == 0) or right (if offset_x == 1)
+ * column and offset_y controls whether the pixel that's counted is in the top
+ * (if offset_y == 0) or bottom (if offset_y == 1) row.
+ *
+ * The skip_x and skip_y fields control how the pixel masking pattern is
+ * repeated across the image data. The first instance of the pattern is always
+ * in the top left of the image data. The skip_x field controls how many pixels
+ * are ignored in the x direction before the pixel masking pattern is repeated.
+ * The skip_y field controls how many pixels are ignored in the y direction
+ * before the pixel masking pattern is repeated.
+ *
+ * These fields can be used to reduce the number of pixels counted for the
+ * statistics, but it's important to be careful to configure them correctly.
+ * Some combinations of values will result in colour components from the input
+ * data being ignored entirely, for example in the following configuration:
+ *
+ * skip_x = 0
+ * offset_x = 0
+ * skip_y = 0
+ * offset_y = 0
+ *
+ * Only the R and Gb components of RGGB data that was input would be collected.
+ * Similarly in the following configuration:
+ *
+ * skip_x = 0
+ * offset_x = 0
+ * skip_y = 1
+ * offset_y = 1
+ *
+ * Only the Gb component of RGGB data that was input would be collected. To
+ * correct things such that all 4 colour components were included it would be
+ * necessary to set the skip_x and skip_y fields in a way that resulted in all
+ * four colour components being collected:
+ *
+ * skip_x = 1
+ * offset_x = 0
+ * skip_y = 1
+ * offset_y = 1
+ *
+ * header.type should be set to one of either MALI_C55_PARAM_BLOCK_AEXP_HIST or
+ * MALI_C55_PARAM_BLOCK_AEXP_IHIST from :c:type:`mali_c55_param_block_type`.
+ *
+ * @header: The Mali-C55 parameters block header
+ * @skip_x: Horizontal decimation. See enum mali_c55_aexp_skip_x
+ * @offset_x: Skip the first column, or not. See enum mali_c55_aexp_row_column_offset
+ * @skip_y: Vertical decimation. See enum mali_c55_aexp_skip_y
+ * @offset_y: Skip the first row, or not. See enum mali_c55_aexp_row_column_offset
+ * @scale_bottom: Scale pixels in bottom half of intensity range: 0=1x ,1=2x, 2=4x, 4=8x, 4=16x
+ * @scale_top: scale pixels in top half of intensity range: 0=1x ,1=2x, 2=4x, 4=8x, 4=16x
+ * @plane_mode: Plane separation mode. See enum mali_c55_aexp_hist_plane_mode
+ * @tap_point: Tap point for histogram from enum mali_c55_aexp_hist_tap_points.
+ * This parameter is unused for the post-Iridix Histogram
+ */
+struct mali_c55_params_aexp_hist {
+ struct mali_c55_params_block_header header;
+ __u8 skip_x;
+ __u8 offset_x;
+ __u8 skip_y;
+ __u8 offset_y;
+ __u8 scale_bottom;
+ __u8 scale_top;
+ __u8 plane_mode;
+ __u8 tap_point;
+};
+
+/**
+ * struct mali_c55_params_aexp_weights - Array of weights for AEXP metering
+ *
+ * This struct allows users to configure the weighting for both of the 1024-bin
+ * AEXP histograms. The pixel data collected for each zone is multiplied by the
+ * corresponding weight from this array, which may be zero if the intention is
+ * to mask off the zone entirely.
+ *
+ * header.type should be set to one of either MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS
+ * or MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS from :c:type:`mali_c55_param_block_type`.
+ *
+ * @header: The Mali-C55 parameters block header
+ * @nodes_used_horiz: Number of active zones horizontally [0..15]
+ * @nodes_used_vert: Number of active zones vertically [0..15]
+ * @zone_weights: Zone weighting. Index is row*col where 0,0 is the top
+ * left zone continuing in raster order. Each zone can be
+ * weighted in the range [0..15]. The number of rows and
+ * columns is defined by @nodes_used_vert and
+ * @nodes_used_horiz
+ */
+struct mali_c55_params_aexp_weights {
+ struct mali_c55_params_block_header header;
+ __u8 nodes_used_horiz;
+ __u8 nodes_used_vert;
+ __u8 zone_weights[MALI_C55_MAX_ZONES];
+};
+
+/**
+ * struct mali_c55_params_digital_gain - Digital gain value
+ *
+ * This struct carries a digital gain value to set in the ISP.
+ *
+ * header.type should be set to MALI_C55_PARAM_BLOCK_DIGITAL_GAIN from
+ * :c:type:`mali_c55_param_block_type` for this block.
+ *
+ * @header: The Mali-C55 parameters block header
+ * @gain: The digital gain value to apply, in Q5.8 format.
+ */
+struct mali_c55_params_digital_gain {
+ struct mali_c55_params_block_header header;
+ __u16 gain;
+};
+
+/**
+ * enum mali_c55_awb_stats_mode - Statistics mode for AWB
+ * @MALI_C55_AWB_MODE_GRBR: Statistics collected as Green/Red and Blue/Red ratios
+ * @MALI_C55_AWB_MODE_RGBG: Statistics collected as Red/Green and Blue/Green ratios
+ */
+enum mali_c55_awb_stats_mode {
+ MALI_C55_AWB_MODE_GRBR = 0,
+ MALI_C55_AWB_MODE_RGBG,
+};
+
+/**
+ * struct mali_c55_params_awb_gains - Gain settings for auto white balance
+ *
+ * This struct allows users to configure the gains for auto-white balance. There
+ * are four gain settings corresponding to each colour channel in the bayer
+ * domain. Although named generically, the association between the gain applied
+ * and the colour channel is done automatically within the ISP depending on the
+ * input format, and so the following mapping always holds true::
+ *
+ * gain00 = R
+ * gain01 = Gr
+ * gain10 = Gb
+ * gain11 = B
+ *
+ * All of the gains are stored in Q4.8 format.
+ *
+ * header.type should be set to one of either MALI_C55_PARAM_BLOCK_AWB_GAINS or
+ * MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP from :c:type:`mali_c55_param_block_type`.
+ *
+ * @header: The Mali-C55 parameters block header
+ * @gain00: Multiplier for colour channel 00
+ * @gain01: Multiplier for colour channel 01
+ * @gain10: Multiplier for colour channel 10
+ * @gain11: Multiplier for colour channel 11
+ */
+struct mali_c55_params_awb_gains {
+ struct mali_c55_params_block_header header;
+ __u16 gain00;
+ __u16 gain01;
+ __u16 gain10;
+ __u16 gain11;
+};
+
+/**
+ * enum mali_c55_params_awb_tap_points - Tap points for the AWB statistics
+ * @MALI_C55_AWB_STATS_TAP_PF: Immediately after the Purple Fringe block
+ * @MALI_C55_AWB_STATS_TAP_CNR: Immediately after the CNR block
+ */
+enum mali_c55_params_awb_tap_points {
+ MALI_C55_AWB_STATS_TAP_PF = 0,
+ MALI_C55_AWB_STATS_TAP_CNR,
+};
+
+/**
+ * struct mali_c55_params_awb_config - Stats settings for auto-white balance
+ *
+ * This struct allows the configuration of the statistics generated for auto
+ * white balance. Pixel intensity limits can be set to exclude overly bright or
+ * dark regions of an image from the statistics entirely. Colour ratio minima
+ * and maxima can be set to discount pixels who's ratios fall outside the
+ * defined boundaries; there are two sets of registers to do this - the
+ * "min/max" ratios which bound a region and the "high/low" ratios which further
+ * trim the upper and lower ratios. For example with the boundaries configured
+ * as follows, only pixels whos colour ratios falls into the region marked "A"
+ * would be counted::
+ *
+ * cr_high
+ * 2.0 | |
+ * | cb_max --> _________________________v_____
+ * 1.8 | | \ |
+ * | | \ |
+ * 1.6 | | \ |
+ * | | \ |
+ * c 1.4 | cb_low -->|\ A \|<-- cb_high
+ * b | | \ |
+ * 1.2 | | \ |
+ * r | | \ |
+ * a 1.0 | cb_min --> |____\_________________________|
+ * t | ^ ^ ^
+ * i 0.8 | | | |
+ * o | cr_min | cr_max
+ * s 0.6 | |
+ * | cr_low
+ * 0.4 |
+ * |
+ * 0.2 |
+ * |
+ * 0.0 |_______________________________________________________________
+ * 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0
+ * cr ratios
+ *
+ * header.type should be set to MALI_C55_PARAM_BLOCK_AWB_CONFIG from
+ * :c:type:`mali_c55_param_block_type` for this block.
+ *
+ * @header: The Mali-C55 parameters block header
+ * @tap_point: The tap point from enum mali_c55_params_awb_tap_points
+ * @stats_mode: AWB statistics collection mode, see :c:type:`mali_c55_awb_stats_mode`
+ * @white_level: Upper pixel intensity (I.E. raw pixel values) limit
+ * @black_level: Lower pixel intensity (I.E. raw pixel values) limit
+ * @cr_max: Maximum R/G ratio (Q4.8 format)
+ * @cr_min: Minimum R/G ratio (Q4.8 format)
+ * @cb_max: Maximum B/G ratio (Q4.8 format)
+ * @cb_min: Minimum B/G ratio (Q4.8 format)
+ * @nodes_used_horiz: Number of active zones horizontally [0..15]
+ * @nodes_used_vert: Number of active zones vertically [0..15]
+ * @cr_high: R/G ratio trim high (Q4.8 format)
+ * @cr_low: R/G ratio trim low (Q4.8 format)
+ * @cb_high: B/G ratio trim high (Q4.8 format)
+ * @cb_low: B/G ratio trim low (Q4.8 format)
+ */
+struct mali_c55_params_awb_config {
+ struct mali_c55_params_block_header header;
+ __u8 tap_point;
+ __u8 stats_mode;
+ __u16 white_level;
+ __u16 black_level;
+ __u16 cr_max;
+ __u16 cr_min;
+ __u16 cb_max;
+ __u16 cb_min;
+ __u8 nodes_used_horiz;
+ __u8 nodes_used_vert;
+ __u16 cr_high;
+ __u16 cr_low;
+ __u16 cb_high;
+ __u16 cb_low;
+};
+
+#define MALI_C55_NUM_MESH_SHADING_ELEMENTS 3072
+
+/**
+ * struct mali_c55_params_mesh_shading_config - Mesh shading configuration
+ *
+ * The mesh shading correction module allows programming a separate table of
+ * either 16x16 or 32x32 node coefficients for 3 different light sources. The
+ * final correction coefficients applied are computed by blending the
+ * coefficients from two tables together.
+ *
+ * A page of 1024 32-bit integers is associated to each colour channel, with
+ * pages stored consecutively in memory. Each 32-bit integer packs 3 8-bit
+ * correction coefficients for a single node, one for each of the three light
+ * sources. The 8 most significant bits are unused. The following table
+ * describes the layout::
+ *
+ * +----------- Page (Colour Plane) 0 -------------+
+ * | @mesh[i] | Mesh Point | Bits | Light Source |
+ * +-----------+------------+-------+--------------+
+ * | 0 | 0,0 | 16,23 | LS2 |
+ * | | | 08-15 | LS1 |
+ * | | | 00-07 | LS0 |
+ * +-----------+------------+-------+--------------+
+ * | 1 | 0,1 | 16,23 | LS2 |
+ * | | | 08-15 | LS1 |
+ * | | | 00-07 | LS0 |
+ * +-----------+------------+-------+--------------+
+ * | ... | ... | ... | ... |
+ * +-----------+------------+-------+--------------+
+ * | 1023 | 31,31 | 16,23 | LS2 |
+ * | | | 08-15 | LS1 |
+ * | | | 00-07 | LS0 |
+ * +----------- Page (Colour Plane) 1 -------------+
+ * | @mesh[i] | Mesh Point | Bits | Light Source |
+ * +-----------+------------+-------+--------------+
+ * | 1024 | 0,0 | 16,23 | LS2 |
+ * | | | 08-15 | LS1 |
+ * | | | 00-07 | LS0 |
+ * +-----------+------------+-------+--------------+
+ * | 1025 | 0,1 | 16,23 | LS2 |
+ * | | | 08-15 | LS1 |
+ * | | | 00-07 | LS0 |
+ * +-----------+------------+-------+--------------+
+ * | ... | ... | ... | ... |
+ * +-----------+------------+-------+--------------+
+ * | 2047 | 31,31 | 16,23 | LS2 |
+ * | | | 08-15 | LS1 |
+ * | | | 00-07 | LS0 |
+ * +----------- Page (Colour Plane) 2 -------------+
+ * | @mesh[i] | Mesh Point | Bits | Light Source |
+ * +-----------+------------+-------+--------------+
+ * | 2048 | 0,0 | 16,23 | LS2 |
+ * | | | 08-15 | LS1 |
+ * | | | 00-07 | LS0 |
+ * +-----------+------------+-------+--------------+
+ * | 2049 | 0,1 | 16,23 | LS2 |
+ * | | | 08-15 | LS1 |
+ * | | | 00-07 | LS0 |
+ * +-----------+------------+-------+--------------+
+ * | ... | ... | ... | ... |
+ * +-----------+------------+-------+--------------+
+ * | 3071 | 31,31 | 16,23 | LS2 |
+ * | | | 08-15 | LS1 |
+ * | | | 00-07 | LS0 |
+ * +-----------+------------+-------+--------------+
+ *
+ * The @mesh_scale member determines the precision and minimum and maximum gain.
+ * For example if @mesh_scale is 0 and therefore selects 0 - 2x gain, a value of
+ * 0 in a coefficient means 0.0 gain, a value of 128 means 1.0 gain and 255
+ * means 2.0 gain.
+ *
+ * header.type should be set to MALI_C55_PARAM_MESH_SHADING_CONFIG from
+ * :c:type:`mali_c55_param_block_type` for this block.
+ *
+ * @header: The Mali-C55 parameters block header
+ * @mesh_show: Output the mesh data rather than image data
+ * @mesh_scale: Set the precision and maximum gain range of mesh shading
+ * - 0 = 0-2x gain
+ * - 1 = 0-4x gain
+ * - 2 = 0-8x gain
+ * - 3 = 0-16x gain
+ * - 4 = 1-2x gain
+ * - 5 = 1-3x gain
+ * - 6 = 1-5x gain
+ * - 7 = 1-9x gain
+ * @mesh_page_r: Mesh page select for red colour plane [0..2]
+ * @mesh_page_g: Mesh page select for green colour plane [0..2]
+ * @mesh_page_b: Mesh page select for blue colour plane [0..2]
+ * @mesh_width: Number of horizontal nodes minus 1 [15,31]
+ * @mesh_height: Number of vertical nodes minus 1 [15,31]
+ * @mesh: Mesh shading correction tables
+ */
+struct mali_c55_params_mesh_shading_config {
+ struct mali_c55_params_block_header header;
+ __u8 mesh_show;
+ __u8 mesh_scale;
+ __u8 mesh_page_r;
+ __u8 mesh_page_g;
+ __u8 mesh_page_b;
+ __u8 mesh_width;
+ __u8 mesh_height;
+ __u32 mesh[MALI_C55_NUM_MESH_SHADING_ELEMENTS];
+};
+
+/** enum mali_c55_params_mesh_alpha_bank - Mesh shading table bank selection
+ * @MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS1 - Select Light Sources 0 and 1
+ * @MALI_C55_MESH_ALPHA_BANK_LS1_AND_LS2 - Select Light Sources 1 and 2
+ * @MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS2 - Select Light Sources 0 and 2
+ */
+enum mali_c55_params_mesh_alpha_bank {
+ MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS1 = 0,
+ MALI_C55_MESH_ALPHA_BANK_LS1_AND_LS2 = 1,
+ MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS2 = 4
+};
+
+/**
+ * struct mali_c55_params_mesh_shading_selection - Mesh table selection
+ *
+ * The module computes the final correction coefficients by blending the ones
+ * from two light source tables, which are selected (independently for each
+ * colour channel) by the @mesh_alpha_bank_r/g/b fields.
+ *
+ * The final blended coefficients for each node are calculated using the
+ * following equation:
+ *
+ * Final coefficient = (a * LS\ :sub:`b`\ + (256 - a) * LS\ :sub:`a`\) / 256
+ *
+ * Where a is the @mesh_alpha_r/g/b value, and LS\ :sub:`a`\ and LS\ :sub:`b`\
+ * are the node cofficients for the two tables selected by the
+ * @mesh_alpha_bank_r/g/b value.
+ *
+ * The scale of the applied correction may also be controlled by tuning the
+ * @mesh_strength member. This is a modifier to the final coefficients which can
+ * be used to globally reduce the gains applied.
+ *
+ * header.type should be set to MALI_C55_PARAM_MESH_SHADING_SELECTION from
+ * :c:type:`mali_c55_param_block_type` for this block.
+ *
+ * @header: The Mali-C55 parameters block header
+ * @mesh_alpha_bank_r: Red mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`)
+ * @mesh_alpha_bank_g: Green mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`)
+ * @mesh_alpha_bank_b: Blue mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`)
+ * @mesh_alpha_r: Blend coefficient for R [0..255]
+ * @mesh_alpha_g: Blend coefficient for G [0..255]
+ * @mesh_alpha_b: Blend coefficient for B [0..255]
+ * @mesh_strength: Mesh strength in Q4.12 format [0..4096]
+ */
+struct mali_c55_params_mesh_shading_selection {
+ struct mali_c55_params_block_header header;
+ __u8 mesh_alpha_bank_r;
+ __u8 mesh_alpha_bank_g;
+ __u8 mesh_alpha_bank_b;
+ __u8 mesh_alpha_r;
+ __u8 mesh_alpha_g;
+ __u8 mesh_alpha_b;
+ __u16 mesh_strength;
+};
+
+/**
+ * union mali_c55_params_block - Generalisation of a parameter block
+ *
+ * This union allows the driver to treat a block as a generic pointer to this
+ * union and safely access the header and block-specific struct without having
+ * to resort to casting. The header member is accessed first, and the type field
+ * checked which allows the driver to determine which of the other members
+ * should be used. The data member at the end allows a pointer to an address
+ * within the data member of :c:type:`mali_c55_params_buffer` to initialise a
+ * union variable.
+ *
+ * @header: Pointer to the shared header struct embedded as the
+ * first member of all the possible other members (except
+ * @data). This member would be accessed first and the type
+ * field checked to determine which of the other members
+ * should be accessed.
+ * @sensor_offs: For header->type == MALI_C55_PARAM_BLOCK_SENSOR_OFFS
+ * @aexp_hist: For header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST and
+ * header->type == MALI_C55_PARAM_BLOCK_AEXP_IHIST
+ * @aexp_weights: For header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS
+ * and header->type = MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS
+ * @digital_gain: For header->type == MALI_C55_PARAM_BLOCK_DIGITAL_GAIN
+ * @awb_gains: For header->type == MALI_C55_PARAM_BLOCK_AWB_GAINS and
+ * header->type = MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP
+ * @awb_config: For header->type == MALI_C55_PARAM_MESH_SHADING_CONFIG
+ * @shading_config: For header->type == MALI_C55_PARAM_MESH_SHADING_SELECTION
+ * @shading_selection: For header->type == MALI_C55_PARAM_BLOCK_SENSOR_OFFS
+ * @data: Allows easy initialisation of a union variable with a
+ * pointer into a __u8 array.
+ */
+union mali_c55_params_block {
+ struct mali_c55_params_block_header *header;
+ struct mali_c55_params_sensor_off_preshading *sensor_offs;
+ struct mali_c55_params_aexp_hist *aexp_hist;
+ struct mali_c55_params_aexp_weights *aexp_weights;
+ struct mali_c55_params_digital_gain *digital_gain;
+ struct mali_c55_params_awb_gains *awb_gains;
+ struct mali_c55_params_awb_config *awb_config;
+ struct mali_c55_params_mesh_shading_config *shading_config;
+ struct mali_c55_params_mesh_shading_selection *shading_selection;
+ __u8 *data;
+};
+
+/**
+ * define MALI_C55_PARAMS_MAX_SIZE - Maximum size of all Mali C55 Parameters
+ *
+ * Though the parameters for the Mali-C55 are passed as optional blocks, the
+ * driver still needs to know the absolute maximum size so that it can allocate
+ * a buffer sized appropriately to accommodate userspace attempting to set all
+ * possible parameters in a single frame.
+ *
+ * Some structs are in this list multiple times. Where that's the case, it just
+ * reflects the fact that the same struct can be used with multiple different
+ * header types from :c:type:`mali_c55_param_block_type`.
+ */
+#define MALI_C55_PARAMS_MAX_SIZE \
+ (sizeof(struct mali_c55_params_sensor_off_preshading) + \
+ sizeof(struct mali_c55_params_aexp_hist) + \
+ sizeof(struct mali_c55_params_aexp_weights) + \
+ sizeof(struct mali_c55_params_aexp_hist) + \
+ sizeof(struct mali_c55_params_aexp_weights) + \
+ sizeof(struct mali_c55_params_digital_gain) + \
+ sizeof(struct mali_c55_params_awb_gains) + \
+ sizeof(struct mali_c55_params_awb_config) + \
+ sizeof(struct mali_c55_params_awb_gains) + \
+ sizeof(struct mali_c55_params_mesh_shading_config) + \
+ sizeof(struct mali_c55_params_mesh_shading_selection))
+
+/**
+ * struct mali_c55_params_buffer - 3A configuration parameters
+ *
+ * This struct contains the configuration parameters of the Mali-C55 ISP
+ * algorithms, serialized by userspace into a data buffer. Each configuration
+ * parameter block is represented by a block-specific structure which contains a
+ * :c:type:`mali_c55_params_block_header` entry as first member. Userspace
+ * populates the @data buffer with configuration parameters for the blocks that
+ * it intends to configure. As a consequence, the data buffer effective size
+ * changes according to the number of ISP blocks that userspace intends to
+ * configure.
+ *
+ * The parameters buffer is versioned by the @version field to allow modifying
+ * and extending its definition. Userspace shall populate the @version field to
+ * inform the driver about the version it intends to use. The driver will parse
+ * and handle the @data buffer according to the data layout specific to the
+ * indicated version and return an error if the desired version is not
+ * supported.
+ *
+ * For each ISP block that userspace wants to configure, a block-specific
+ * structure is appended to the @data buffer, one after the other without gaps
+ * in between nor overlaps. Userspace shall populate the @total_size field with
+ * the effective size, in bytes, of the @data buffer.
+ *
+ * The expected memory layout of the parameters buffer is::
+ *
+ * +-------------------- struct mali_c55_params_buffer ------------------+
+ * | version = MALI_C55_PARAM_BUFFER_V1; |
+ * | total_size = sizeof(struct mali_c55_params_sensor_off_preshading) |
+ * | sizeof(struct mali_c55_params_aexp_hist); |
+ * | +------------------------- data ---------------------------------+ |
+ * | | +--------- struct mali_c55_params_sensor_off_preshading ------+ | |
+ * | | | +-------- struct mali_c55_params_block_header header -----+ | | |
+ * | | | | type = MALI_C55_PARAM_BLOCK_SENSOR_OFFS; | | | |
+ * | | | | enabled = 1; | | | |
+ * | | | | size = | | | |
+ * | | | | sizeof(struct mali_c55_params_sensor_off_preshading);| | | |
+ * | | | +---------------------------------------------------------+ | | |
+ * | | | chan00 = ...; | | |
+ * | | | chan01 = ...; | | |
+ * | | | chan10 = ...; | | |
+ * | | | chan11 = ...; | | |
+ * | | +------------ struct mali_c55_params_aexp_hist ---------------+ | |
+ * | | | +-------- struct mali_c55_params_block_header header -----+ | | |
+ * | | | | type = MALI_C55_PARAM_BLOCK_AEXP_HIST; | | | |
+ * | | | | enabled = 1; | | | |
+ * | | | | size = sizeof(struct mali_c55_params_aexp_hist); | | | |
+ * | | | +---------------------------------------------------------+ | | |
+ * | | | skip_x = ...; | | |
+ * | | | offset_x = ...; | | |
+ * | | | skip_y = ...; | | |
+ * | | | offset_y = ...; | | |
+ * | | | scale_bottom = ...; | | |
+ * | | | scale_top = ...; | | |
+ * | | | plane_mode = ...; | | |
+ * | | | tap_point = ...; | | |
+ * | | +-------------------------------------------------------------+ | |
+ * | +-----------------------------------------------------------------+ |
+ * +---------------------------------------------------------------------+
+ *
+ * @version: The version from :c:type:`mali_c55_param_buffer_version`
+ * @total_size: The Mali-C55 configuration data effective size, excluding this
+ * header
+ * @data: The Mali-C55 configuration blocks data
+ */
+struct mali_c55_params_buffer {
+ __u8 version;
+ __u32 total_size;
+ __u8 data[MALI_C55_PARAMS_MAX_SIZE];
+};
+
#endif /* __UAPI_MALI_C55_CONFIG_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 17/18] media: platform: Add mali-c55 parameters video node
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
` (15 preceding siblings ...)
2024-07-09 13:29 ` [PATCH v6 16/18] media: uapi: Add parameters structs to mali-c55-config.h Daniel Scally
@ 2024-07-09 13:29 ` Daniel Scally
2024-07-30 22:16 ` Laurent Pinchart
2024-07-09 13:29 ` [PATCH v6 18/18] Documentation: mali-c55: Document the mali-c55 parameter setting Daniel Scally
17 siblings, 1 reply; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:29 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
Add a new code file to the mali-c55 driver that registers an output
video node for userspace to queue buffers of parameters to. Handlers
are included to program the statistics generation plus the white
balance, black level correction and mesh shading correction blocks.
Update the rest of the driver to register and link the new video node
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Changes in v6:
- Used a union to generalise the block pointer rather than resorting to
casting everywhere - fantastic idea Sakari, this made it much cleaner.
- Reworked the loop in mali_c55_params_write_config() so that we can be
sure there's remaining space for the next block header.
Changes in v5:
- New patch
drivers/media/platform/arm/mali-c55/Makefile | 1 +
.../platform/arm/mali-c55/mali-c55-common.h | 20 +
.../platform/arm/mali-c55/mali-c55-core.c | 23 +
.../platform/arm/mali-c55/mali-c55-isp.c | 21 +-
.../platform/arm/mali-c55/mali-c55-params.c | 671 ++++++++++++++++++
.../arm/mali-c55/mali-c55-registers.h | 128 ++++
6 files changed, 863 insertions(+), 1 deletion(-)
create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-params.c
diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile
index b5a22d414479..d5718b0b23e0 100644
--- a/drivers/media/platform/arm/mali-c55/Makefile
+++ b/drivers/media/platform/arm/mali-c55/Makefile
@@ -3,6 +3,7 @@
mali-c55-y := mali-c55-capture.o \
mali-c55-core.o \
mali-c55-isp.o \
+ mali-c55-params.o \
mali-c55-resizer.o \
mali-c55-stats.o \
mali-c55-tpg.o
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
index 136c785c68ba..66a46a7c0547 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-common.h
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
@@ -50,6 +50,7 @@ enum mali_c55_isp_pads {
MALI_C55_ISP_PAD_SOURCE_VIDEO,
MALI_C55_ISP_PAD_SOURCE_BYPASS,
MALI_C55_ISP_PAD_SOURCE_STATS,
+ MALI_C55_ISP_PAD_SINK_PARAMS,
MALI_C55_ISP_NUM_PADS,
};
@@ -184,6 +185,21 @@ struct mali_c55_stats {
} buffers;
};
+struct mali_c55_params {
+ struct mali_c55 *mali_c55;
+ struct video_device vdev;
+ struct vb2_queue queue;
+ struct media_pad pad;
+ /* Mutex to provide to vb2 */
+ struct mutex lock;
+
+ struct {
+ /* Spinlock to guard buffer queue */
+ spinlock_t lock;
+ struct list_head queue;
+ } buffers;
+};
+
enum mali_c55_config_spaces {
MALI_C55_CONFIG_PING,
MALI_C55_CONFIG_PONG,
@@ -226,6 +242,7 @@ struct mali_c55 {
struct mali_c55_isp isp;
struct mali_c55_resizer resizers[MALI_C55_NUM_RSZS];
struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS];
+ struct mali_c55_params params;
struct mali_c55_stats stats;
struct mali_c55_context context;
@@ -256,6 +273,8 @@ int mali_c55_register_capture_devs(struct mali_c55 *mali_c55);
void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55);
int mali_c55_register_stats(struct mali_c55 *mali_c55);
void mali_c55_unregister_stats(struct mali_c55 *mali_c55);
+int mali_c55_register_params(struct mali_c55 *mali_c55);
+void mali_c55_unregister_params(struct mali_c55 *mali_c55);
struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55);
void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
enum mali_c55_planes plane);
@@ -272,5 +291,6 @@ const struct mali_c55_isp_fmt *
mali_c55_isp_get_mbus_config_by_index(u32 index);
void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
enum mali_c55_config_spaces cfg_space);
+void mali_c55_params_write_config(struct mali_c55 *mali_c55);
#endif /* _MALI_C55_COMMON_H */
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
index ed0db34767a4..55b3cbf53791 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
@@ -384,6 +384,16 @@ static int mali_c55_create_links(struct mali_c55 *mali_c55)
goto err_remove_links;
}
+ ret = media_create_pad_link(&mali_c55->params.vdev.entity, 0,
+ &mali_c55->isp.sd.entity,
+ MALI_C55_ISP_PAD_SINK_PARAMS,
+ MEDIA_LNK_FL_ENABLED);
+ if (ret) {
+ dev_err(mali_c55->dev,
+ "failed to link ISP and parameters video node\n");
+ goto err_remove_links;
+ }
+
return 0;
err_remove_links:
@@ -398,6 +408,7 @@ static void mali_c55_unregister_entities(struct mali_c55 *mali_c55)
mali_c55_unregister_isp(mali_c55);
mali_c55_unregister_resizers(mali_c55);
mali_c55_unregister_capture_devs(mali_c55);
+ mali_c55_unregister_params(mali_c55);
mali_c55_unregister_stats(mali_c55);
}
@@ -421,6 +432,10 @@ static int mali_c55_register_entities(struct mali_c55 *mali_c55)
if (ret)
goto err_unregister_entities;
+ ret = mali_c55_register_params(mali_c55);
+ if (ret)
+ goto err_unregister_entities;
+
ret = mali_c55_register_stats(mali_c55);
if (ret)
goto err_unregister_entities;
@@ -620,6 +635,14 @@ static irqreturn_t mali_c55_isr(int irq, void *context)
curr_config >>= ffs(MALI_C55_REG_PING_PONG_READ_MASK) - 1;
next_config = curr_config ^ 1;
+ /*
+ * Write the configuration parameters received from
+ * userspace into the configuration buffer, which will
+ * be transferred to the 'next' active config space at
+ * by mali_c55_swap_next_config().
+ */
+ mali_c55_params_write_config(mali_c55);
+
/*
* The ordering of these two is currently important as
* mali_c55_stats_fill_buffer() is asynchronous whereas
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
index 2f450c00300a..40d7ef6eda22 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
@@ -132,6 +132,7 @@ static int mali_c55_isp_start(struct mali_c55 *mali_c55)
cfg->bypass ? MALI_C55_ISP_RAW_BYPASS_BYPASS_MASK :
0x00);
+ mali_c55_params_write_config(mali_c55);
ret = mali_c55_config_write(ctx, MALI_C55_CONFIG_PING);
if (ret) {
dev_err(mali_c55->dev, "failed to write ISP config\n");
@@ -464,12 +465,17 @@ static int mali_c55_isp_init_state(struct v4l2_subdev *sd,
src_fmt = v4l2_subdev_state_get_format(state,
MALI_C55_ISP_PAD_SOURCE_STATS);
+ sink_fmt = v4l2_subdev_state_get_format(state,
+ MALI_C55_ISP_PAD_SINK_PARAMS);
src_fmt->width = sizeof(struct mali_c55_stats_buffer);
src_fmt->height = 1;
src_fmt->field = V4L2_FIELD_NONE;
src_fmt->code = MEDIA_BUS_FMT_METADATA_FIXED;
+ *sink_fmt = *src_fmt;
+ sink_fmt->width = sizeof(struct mali_c55_params_buffer);
+
return 0;
}
@@ -477,8 +483,20 @@ static const struct v4l2_subdev_internal_ops mali_c55_isp_internal_ops = {
.init_state = mali_c55_isp_init_state,
};
+static int mali_c55_subdev_link_validate(struct media_link *link)
+{
+ /*
+ * Skip validation for the parameters sink pad, as the source is not
+ * a subdevice.
+ */
+ if (link->sink->index == MALI_C55_ISP_PAD_SINK_PARAMS)
+ return 0;
+
+ return v4l2_subdev_link_validate(link);
+}
+
static const struct media_entity_operations mali_c55_isp_media_ops = {
- .link_validate = v4l2_subdev_link_validate,
+ .link_validate = mali_c55_subdev_link_validate,
};
int mali_c55_register_isp(struct mali_c55 *mali_c55)
@@ -501,6 +519,7 @@ int mali_c55_register_isp(struct mali_c55 *mali_c55)
isp->pads[MALI_C55_ISP_PAD_SOURCE_VIDEO].flags = MEDIA_PAD_FL_SOURCE;
isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE;
isp->pads[MALI_C55_ISP_PAD_SOURCE_STATS].flags = MEDIA_PAD_FL_SOURCE;
+ isp->pads[MALI_C55_ISP_PAD_SINK_PARAMS].flags = MEDIA_PAD_FL_SINK;
ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS,
isp->pads);
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-params.c b/drivers/media/platform/arm/mali-c55/mali-c55-params.c
new file mode 100644
index 000000000000..c0ca4a759653
--- /dev/null
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-params.c
@@ -0,0 +1,671 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM Mali-C55 ISP Driver - Configuration parameters output device
+ *
+ * Copyright (C) 2024 Ideas on Board Oy
+ */
+#include <linux/media/arm/mali-c55-config.h>
+
+#include <media/media-entity.h>
+#include <media/v4l2-dev.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fh.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mali-c55-common.h"
+#include "mali-c55-registers.h"
+
+typedef void (*mali_c55_block_handler)(struct mali_c55 *mali_c55,
+ union mali_c55_params_block block);
+
+struct mali_c55_block_handler {
+ size_t size;
+ mali_c55_block_handler handler;
+};
+
+static void mali_c55_params_sensor_offs(struct mali_c55 *mali_c55,
+ union mali_c55_params_block block)
+{
+ struct mali_c55_params_sensor_off_preshading *p = block.sensor_offs;
+ __u32 global_offset;
+
+ if (!block.header->enabled)
+ return;
+
+ if (!(p->chan00 || p->chan01 || p->chan10 || p->chan11))
+ return;
+
+ mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_00,
+ p->chan00 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
+ mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_01,
+ p->chan01 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
+ mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_10,
+ p->chan10 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
+ mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_11,
+ p->chan11 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
+
+ /*
+ * The average offset is applied as a global offset for the digital
+ * gain block
+ */
+ global_offset = (p->chan00 + p->chan01 + p->chan10 + p->chan11) >> 2;
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_DIGITAL_GAIN_OFFSET,
+ MALI_C55_DIGITAL_GAIN_OFFSET_MASK,
+ global_offset);
+
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BYPASS_3,
+ MALI_C55_REG_BYPASS_3_SENSOR_OFFSET_PRE_SH,
+ 0x00);
+}
+
+static void mali_c55_params_aexp_hist(struct mali_c55 *mali_c55,
+ union mali_c55_params_block block)
+{
+ u32 disable_mask;
+ u32 disable_val;
+ u32 base;
+
+ if (block.header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST) {
+ disable_mask = MALI_C55_AEXP_HIST_DISABLE_MASK;
+ disable_val = MALI_C55_AEXP_HIST_DISABLE;
+ base = MALI_C55_REG_AEXP_HIST_BASE;
+ } else {
+ disable_mask = MALI_C55_AEXP_IHIST_DISABLE_MASK;
+ disable_val = MALI_C55_AEXP_IHIST_DISABLE;
+ base = MALI_C55_REG_AEXP_IHIST_BASE;
+ }
+ struct mali_c55_params_aexp_hist *params = block.aexp_hist;
+
+ if (!block.header->enabled) {
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
+ disable_mask, disable_val);
+ return;
+ }
+
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
+ disable_mask, false);
+
+ mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
+ MALI_C55_AEXP_HIST_SKIP_X_MASK, params->skip_x);
+ mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
+ MALI_C55_AEXP_HIST_OFFSET_X_MASK,
+ MALI_C55_AEXP_HIST_OFFSET_X(params->offset_x));
+ mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
+ MALI_C55_AEXP_HIST_SKIP_Y_MASK,
+ MALI_C55_AEXP_HIST_SKIP_Y(params->skip_y));
+ mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
+ MALI_C55_AEXP_HIST_OFFSET_Y_MASK,
+ MALI_C55_AEXP_HIST_OFFSET_Y(params->offset_y));
+
+ mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SCALE_OFFSET,
+ MALI_C55_AEXP_HIST_SCALE_BOTTOM_MASK,
+ params->scale_bottom);
+ mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SCALE_OFFSET,
+ MALI_C55_AEXP_HIST_SCALE_TOP_MASK,
+ MALI_C55_AEXP_HIST_SCALE_TOP(params->scale_top));
+
+ mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_PLANE_MODE_OFFSET,
+ MALI_C55_AEXP_HIST_PLANE_MODE_MASK,
+ params->plane_mode);
+
+ if (block.header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST)
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
+ MALI_C55_AEXP_HIST_SWITCH_MASK,
+ MALI_C55_AEXP_HIST_SWITCH(params->tap_point));
+}
+
+static void
+mali_c55_params_aexp_hist_weights(struct mali_c55 *mali_c55,
+ union mali_c55_params_block block)
+{
+ struct mali_c55_params_aexp_weights *params = block.aexp_weights;
+ u32 base;
+
+ if (!block.header->enabled)
+ return;
+
+ base = block.header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS ?
+ MALI_C55_REG_AEXP_HIST_BASE :
+ MALI_C55_REG_AEXP_IHIST_BASE;
+
+ mali_c55_ctx_update_bits(mali_c55,
+ base + MALI_C55_AEXP_HIST_NODES_USED_OFFSET,
+ MALI_C55_AEXP_HIST_NODES_USED_HORIZ_MASK,
+ params->nodes_used_horiz);
+ mali_c55_ctx_update_bits(mali_c55,
+ base + MALI_C55_AEXP_HIST_NODES_USED_OFFSET,
+ MALI_C55_AEXP_HIST_NODES_USED_VERT_MASK,
+ MALI_C55_AEXP_HIST_NODES_USED_VERT(params->nodes_used_vert));
+
+ /*
+ * The zone weights array is a 225-element array of u8 values, but that
+ * is a bit annoying to handle given the ISP expects 32-bit writes. We
+ * just reinterpret it as a 57-element array of 32-bit values for the
+ * purposes of this transaction (the 3 bytes of additional space at the
+ * end of the write is just padding for the array of weights in the ISP
+ * memory space anyway, so there's no risk of overwriting other
+ * registers).
+ */
+ for (unsigned int i = 0; i < 57; i++) {
+ u32 val = ((u32 *)params->zone_weights)[i]
+ & MALI_C55_AEXP_HIST_ZONE_WEIGHT_MASK;
+ u32 addr = base + MALI_C55_AEXP_HIST_ZONE_WEIGHTS_OFFSET + (4 * i);
+
+ mali_c55_ctx_write(mali_c55, addr, val);
+ }
+}
+
+static void mali_c55_params_digital_gain(struct mali_c55 *mali_c55,
+ union mali_c55_params_block block)
+{
+ struct mali_c55_params_digital_gain *dgain = block.digital_gain;
+
+ /*
+ * If the block is flagged as disabled we write a gain of 1.0, which in
+ * Q5.8 format is 256.
+ */
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_DIGITAL_GAIN,
+ MALI_C55_DIGITAL_GAIN_MASK,
+ block.header->enabled ? dgain->gain : 256);
+}
+
+static void mali_c55_params_awb_gains(struct mali_c55 *mali_c55,
+ union mali_c55_params_block block)
+{
+ struct mali_c55_params_awb_gains *gains = block.awb_gains;
+
+ /*
+ * There are two places AWB gains can be set in the ISP; one affects the
+ * image output data and the other affects the statistics for the
+ * AEXP-0 tap point.
+ */
+ u32 addr1 = block.header->type == MALI_C55_PARAM_BLOCK_AWB_GAINS ?
+ MALI_C55_REG_AWB_GAINS1 :
+ MALI_C55_REG_AWB_GAINS1_AEXP;
+ u32 addr2 = block.header->type == MALI_C55_PARAM_BLOCK_AWB_GAINS ?
+ MALI_C55_REG_AWB_GAINS2 :
+ MALI_C55_REG_AWB_GAINS2_AEXP;
+
+ mali_c55_ctx_update_bits(mali_c55, addr1, MALI_C55_AWB_GAIN00_MASK,
+ gains->gain00);
+ mali_c55_ctx_update_bits(mali_c55, addr1, MALI_C55_AWB_GAIN01_MASK,
+ MALI_C55_AWB_GAIN01(gains->gain01));
+ mali_c55_ctx_update_bits(mali_c55, addr2, MALI_C55_AWB_GAIN10_MASK,
+ gains->gain10);
+ mali_c55_ctx_update_bits(mali_c55, addr2, MALI_C55_AWB_GAIN11_MASK,
+ MALI_C55_AWB_GAIN11(gains->gain11));
+}
+
+static void mali_c55_params_awb_config(struct mali_c55 *mali_c55,
+ union mali_c55_params_block block)
+{
+ struct mali_c55_params_awb_config *params = block.awb_config;
+
+ if (!block.header->enabled) {
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
+ MALI_C55_AWB_DISABLE_MASK,
+ MALI_C55_AWB_DISABLE_MASK);
+ return;
+ }
+
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
+ MALI_C55_AWB_DISABLE_MASK, false);
+
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_STATS_MODE,
+ MALI_C55_AWB_STATS_MODE_MASK, params->stats_mode);
+
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_WHITE_LEVEL,
+ MALI_C55_AWB_WHITE_LEVEL_MASK, params->white_level);
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_BLACK_LEVEL,
+ MALI_C55_AWB_BLACK_LEVEL_MASK, params->black_level);
+
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_MAX,
+ MALI_C55_AWB_CR_MAX_MASK, params->cr_max);
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_MIN,
+ MALI_C55_AWB_CR_MIN_MASK, params->cr_min);
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_MAX,
+ MALI_C55_AWB_CB_MAX_MASK, params->cb_max);
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_MIN,
+ MALI_C55_AWB_CB_MIN_MASK, params->cb_min);
+
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_NODES_USED,
+ MALI_C55_AWB_NODES_USED_HORIZ_MASK,
+ params->nodes_used_horiz);
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_NODES_USED,
+ MALI_C55_AWB_NODES_USED_VERT_MASK,
+ MALI_C55_AWB_NODES_USED_VERT(params->nodes_used_vert));
+
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_HIGH,
+ MALI_C55_AWB_CR_HIGH_MASK, params->cr_high);
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_LOW,
+ MALI_C55_AWB_CR_LOW_MASK, params->cr_low);
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_HIGH,
+ MALI_C55_AWB_CB_HIGH_MASK, params->cb_high);
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_LOW,
+ MALI_C55_AWB_CB_LOW_MASK, params->cb_low);
+
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
+ MALI_C55_AWB_SWITCH_MASK,
+ MALI_C55_AWB_SWITCH(params->tap_point));
+}
+
+static void mali_c55_params_lsc_config(struct mali_c55 *mali_c55,
+ union mali_c55_params_block block)
+{
+ struct mali_c55_params_mesh_shading_config *params = block.shading_config;
+ unsigned int i;
+ u32 addr;
+
+ if (!block.header->enabled) {
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
+ MALI_C55_MESH_SHADING_ENABLE_MASK,
+ false);
+ return;
+ }
+
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
+ MALI_C55_MESH_SHADING_ENABLE_MASK, true);
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
+ MALI_C55_MESH_SHADING_MESH_SHOW_MASK,
+ MALI_C55_MESH_SHADING_MESH_SHOW(params->mesh_show));
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
+ MALI_C55_MESH_SHADING_SCALE_MASK,
+ MALI_C55_MESH_SHADING_SCALE(params->mesh_scale));
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
+ MALI_C55_MESH_SHADING_PAGE_R_MASK,
+ MALI_C55_MESH_SHADING_PAGE_R(params->mesh_page_r));
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
+ MALI_C55_MESH_SHADING_PAGE_G_MASK,
+ MALI_C55_MESH_SHADING_PAGE_G(params->mesh_page_g));
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
+ MALI_C55_MESH_SHADING_PAGE_B_MASK,
+ MALI_C55_MESH_SHADING_PAGE_B(params->mesh_page_b));
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
+ MALI_C55_MESH_SHADING_MESH_WIDTH_MASK,
+ MALI_C55_MESH_SHADING_MESH_WIDTH(params->mesh_width));
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
+ MALI_C55_MESH_SHADING_MESH_HEIGHT_MASK,
+ MALI_C55_MESH_SHADING_MESH_HEIGHT(params->mesh_height));
+
+ for (i = 0; i < MALI_C55_NUM_MESH_SHADING_ELEMENTS; i++) {
+ addr = MALI_C55_REG_MESH_SHADING_TABLES + (i * 4);
+ mali_c55_ctx_write(mali_c55, addr, params->mesh[i]);
+ }
+}
+
+static void mali_c55_params_lsc_selection(struct mali_c55 *mali_c55,
+ union mali_c55_params_block block)
+{
+ struct mali_c55_params_mesh_shading_selection *params =
+ block.shading_selection;
+
+ if (!block.header->enabled)
+ return;
+
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK,
+ MALI_C55_MESH_SHADING_ALPHA_BANK_R_MASK,
+ params->mesh_alpha_bank_r);
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK,
+ MALI_C55_MESH_SHADING_ALPHA_BANK_G_MASK,
+ MALI_C55_MESH_SHADING_ALPHA_BANK_G(params->mesh_alpha_bank_g));
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK,
+ MALI_C55_MESH_SHADING_ALPHA_BANK_B_MASK,
+ MALI_C55_MESH_SHADING_ALPHA_BANK_B(params->mesh_alpha_bank_b));
+
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA,
+ MALI_C55_MESH_SHADING_ALPHA_R_MASK,
+ params->mesh_alpha_r);
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA,
+ MALI_C55_MESH_SHADING_ALPHA_G_MASK,
+ MALI_C55_MESH_SHADING_ALPHA_G(params->mesh_alpha_g));
+ mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA,
+ MALI_C55_MESH_SHADING_ALPHA_B_MASK,
+ MALI_C55_MESH_SHADING_ALPHA_B(params->mesh_alpha_b));
+
+ mali_c55_ctx_update_bits(mali_c55,
+ MALI_C55_REG_MESH_SHADING_MESH_STRENGTH,
+ MALI_c55_MESH_STRENGTH_MASK,
+ params->mesh_strength);
+}
+
+static const struct mali_c55_block_handler mali_c55_block_handlers[] = {
+ [MALI_C55_PARAM_BLOCK_SENSOR_OFFS] = {
+ .size = sizeof(struct mali_c55_params_sensor_off_preshading),
+ .handler = &mali_c55_params_sensor_offs,
+ },
+ [MALI_C55_PARAM_BLOCK_AEXP_HIST] = {
+ .size = sizeof(struct mali_c55_params_aexp_hist),
+ .handler = &mali_c55_params_aexp_hist,
+ },
+ [MALI_C55_PARAM_BLOCK_AEXP_IHIST] = {
+ .size = sizeof(struct mali_c55_params_aexp_hist),
+ .handler = &mali_c55_params_aexp_hist,
+ },
+ [MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS] = {
+ .size = sizeof(struct mali_c55_params_aexp_weights),
+ .handler = &mali_c55_params_aexp_hist_weights,
+ },
+ [MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS] = {
+ .size = sizeof(struct mali_c55_params_aexp_weights),
+ .handler = &mali_c55_params_aexp_hist_weights,
+ },
+ [MALI_C55_PARAM_BLOCK_DIGITAL_GAIN] = {
+ .size = sizeof(struct mali_c55_params_digital_gain),
+ .handler = &mali_c55_params_digital_gain,
+ },
+ [MALI_C55_PARAM_BLOCK_AWB_GAINS] = {
+ .size = sizeof(struct mali_c55_params_awb_gains),
+ .handler = &mali_c55_params_awb_gains,
+ },
+ [MALI_C55_PARAM_BLOCK_AWB_CONFIG] = {
+ .size = sizeof(struct mali_c55_params_awb_config),
+ .handler = &mali_c55_params_awb_config,
+ },
+ [MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP] = {
+ .size = sizeof(struct mali_c55_params_awb_gains),
+ .handler = &mali_c55_params_awb_gains,
+ },
+ [MALI_C55_PARAM_MESH_SHADING_CONFIG] = {
+ .size = sizeof(struct mali_c55_params_mesh_shading_config),
+ .handler = &mali_c55_params_lsc_config,
+ },
+ [MALI_C55_PARAM_MESH_SHADING_SELECTION] = {
+ .size = sizeof(struct mali_c55_params_mesh_shading_selection),
+ .handler = &mali_c55_params_lsc_selection,
+ },
+};
+
+static int mali_c55_params_enum_fmt_meta_out(struct file *file, void *fh,
+ struct v4l2_fmtdesc *f)
+{
+ if (f->index)
+ return -EINVAL;
+
+ f->pixelformat = V4L2_META_FMT_MALI_C55_PARAMS;
+
+ return 0;
+}
+
+static int mali_c55_params_g_fmt_meta_out(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ static const struct v4l2_meta_format mfmt = {
+ .dataformat = V4L2_META_FMT_MALI_C55_PARAMS,
+ .buffersize = sizeof(struct mali_c55_params_buffer),
+ };
+
+ f->fmt.meta = mfmt;
+
+ return 0;
+}
+
+static int mali_c55_params_querycap(struct file *file,
+ void *priv, struct v4l2_capability *cap)
+{
+ strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver));
+ strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card));
+
+ return 0;
+}
+
+static const struct v4l2_ioctl_ops mali_c55_params_v4l2_ioctl_ops = {
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+ .vidioc_enum_fmt_meta_out = mali_c55_params_enum_fmt_meta_out,
+ .vidioc_g_fmt_meta_out = mali_c55_params_g_fmt_meta_out,
+ .vidioc_s_fmt_meta_out = mali_c55_params_g_fmt_meta_out,
+ .vidioc_try_fmt_meta_out = mali_c55_params_g_fmt_meta_out,
+ .vidioc_querycap = mali_c55_params_querycap,
+ .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_file_operations mali_c55_params_v4l2_fops = {
+ .owner = THIS_MODULE,
+ .unlocked_ioctl = video_ioctl2,
+ .open = v4l2_fh_open,
+ .release = vb2_fop_release,
+ .poll = vb2_fop_poll,
+ .mmap = vb2_fop_mmap,
+};
+
+static int
+mali_c55_params_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
+ unsigned int *num_planes, unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ if (*num_planes && *num_planes > 1)
+ return -EINVAL;
+
+ if (sizes[0] && sizes[0] < sizeof(struct mali_c55_params_buffer))
+ return -EINVAL;
+
+ *num_planes = 1;
+
+ if (!sizes[0])
+ sizes[0] = sizeof(struct mali_c55_params_buffer);
+
+ return 0;
+}
+
+static void mali_c55_params_buf_queue(struct vb2_buffer *vb)
+{
+ struct mali_c55_params *params = vb2_get_drv_priv(vb->vb2_queue);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct mali_c55_buffer *buf = container_of(vbuf,
+ struct mali_c55_buffer, vb);
+
+ vb2_set_plane_payload(vb, 0, sizeof(struct mali_c55_params_buffer));
+
+ spin_lock(¶ms->buffers.lock);
+ list_add_tail(&buf->queue, ¶ms->buffers.queue);
+ spin_unlock(¶ms->buffers.lock);
+}
+
+static int mali_c55_params_start_streaming(struct vb2_queue *q,
+ unsigned int count)
+{
+ struct mali_c55_params *params = vb2_get_drv_priv(q);
+ struct mali_c55 *mali_c55 = params->mali_c55;
+ int ret;
+
+ ret = video_device_pipeline_start(¶ms->vdev,
+ ¶ms->mali_c55->pipe);
+ if (ret)
+ return ret;
+
+ if (mali_c55->pipe.start_count == mali_c55->pipe.required_count) {
+ ret = v4l2_subdev_enable_streams(&mali_c55->isp.sd,
+ MALI_C55_ISP_PAD_SOURCE_VIDEO,
+ BIT(0));
+ if (ret)
+ goto err_stop_pipeline;
+ }
+
+ return 0;
+
+err_stop_pipeline:
+ video_device_pipeline_stop(¶ms->vdev);
+
+ return ret;
+}
+
+static void mali_c55_params_stop_streaming(struct vb2_queue *q)
+{
+ struct mali_c55_params *params = vb2_get_drv_priv(q);
+ struct mali_c55_buffer *buf, *tmp;
+
+ spin_lock(¶ms->buffers.lock);
+
+ list_for_each_entry_safe(buf, tmp, ¶ms->buffers.queue, queue) {
+ list_del(&buf->queue);
+ vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+ }
+
+ spin_unlock(¶ms->buffers.lock);
+
+ video_device_pipeline_stop(¶ms->vdev);
+}
+
+static const struct vb2_ops mali_c55_params_vb2_ops = {
+ .queue_setup = mali_c55_params_queue_setup,
+ .buf_queue = mali_c55_params_buf_queue,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+ .start_streaming = mali_c55_params_start_streaming,
+ .stop_streaming = mali_c55_params_stop_streaming,
+};
+
+void mali_c55_params_write_config(struct mali_c55 *mali_c55)
+{
+ struct mali_c55_params *params = &mali_c55->params;
+ enum vb2_buffer_state state = VB2_BUF_STATE_DONE;
+ struct mali_c55_params_buffer *config;
+ struct mali_c55_buffer *buf;
+ size_t block_offset = 0;
+ size_t max_offset;
+
+ spin_lock(¶ms->buffers.lock);
+
+ buf = list_first_entry_or_null(¶ms->buffers.queue,
+ struct mali_c55_buffer, queue);
+ if (buf)
+ list_del(&buf->queue);
+ spin_unlock(¶ms->buffers.lock);
+
+ if (!buf)
+ return;
+
+ buf->vb.sequence = mali_c55->isp.frame_sequence;
+ config = vb2_plane_vaddr(&buf->vb.vb2_buf, 0);
+
+ if (config->total_size > MALI_C55_PARAMS_MAX_SIZE) {
+ dev_dbg(mali_c55->dev, "Invalid parameters buffer size %u\n",
+ config->total_size);
+ state = VB2_BUF_STATE_ERROR;
+ goto err_buffer_done;
+ }
+
+ max_offset = config->total_size - sizeof(struct mali_c55_params_block_header);
+
+ /* Walk the list of parameter blocks and process them. */
+ while (block_offset < max_offset) {
+ const struct mali_c55_block_handler *block_handler;
+ union mali_c55_params_block block;
+
+ block = (union mali_c55_params_block)
+ &config->data[block_offset];
+
+ if (block.header->type >= MALI_C55_PARAM_BLOCK_SENTINEL) {
+ dev_dbg(mali_c55->dev, "Invalid parameters block type\n");
+ state = VB2_BUF_STATE_ERROR;
+ goto err_buffer_done;
+ }
+
+ if (block_offset + block.header->size >= config->total_size) {
+ dev_dbg(mali_c55->dev, "Parameters block too large\n");
+ state = VB2_BUF_STATE_ERROR;
+ goto err_buffer_done;
+ }
+
+ block_handler = &mali_c55_block_handlers[block.header->type];
+ if (block.header->size != block_handler->size) {
+ dev_dbg(mali_c55->dev, "Invalid parameters block size\n");
+ state = VB2_BUF_STATE_ERROR;
+ goto err_buffer_done;
+ }
+
+ block_handler->handler(mali_c55, block);
+
+ block_offset += block.header->size;
+ }
+
+err_buffer_done:
+ vb2_buffer_done(&buf->vb.vb2_buf, state);
+}
+
+void mali_c55_unregister_params(struct mali_c55 *mali_c55)
+{
+ struct mali_c55_params *params = &mali_c55->params;
+
+ if (!video_is_registered(¶ms->vdev))
+ return;
+
+ vb2_video_unregister_device(¶ms->vdev);
+ media_entity_cleanup(¶ms->vdev.entity);
+ mutex_destroy(¶ms->lock);
+}
+
+int mali_c55_register_params(struct mali_c55 *mali_c55)
+{
+ struct mali_c55_params *params = &mali_c55->params;
+ struct video_device *vdev = ¶ms->vdev;
+ struct vb2_queue *vb2q = ¶ms->queue;
+ int ret;
+
+ mutex_init(¶ms->lock);
+ INIT_LIST_HEAD(¶ms->buffers.queue);
+
+ params->pad.flags = MEDIA_PAD_FL_SOURCE;
+ ret = media_entity_pads_init(¶ms->vdev.entity, 1, ¶ms->pad);
+ if (ret)
+ goto err_destroy_mutex;
+
+ vb2q->type = V4L2_BUF_TYPE_META_OUTPUT;
+ vb2q->io_modes = VB2_MMAP | VB2_DMABUF;
+ vb2q->drv_priv = params;
+ vb2q->mem_ops = &vb2_dma_contig_memops;
+ vb2q->ops = &mali_c55_params_vb2_ops;
+ vb2q->buf_struct_size = sizeof(struct mali_c55_buffer);
+ vb2q->min_queued_buffers = 1;
+ vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+ vb2q->lock = ¶ms->lock;
+ vb2q->dev = mali_c55->dev;
+
+ ret = vb2_queue_init(vb2q);
+ if (ret) {
+ dev_err(mali_c55->dev, "params vb2 queue init failed\n");
+ goto err_cleanup_entity;
+ }
+
+ strscpy(params->vdev.name, "mali-c55 3a params",
+ sizeof(params->vdev.name));
+ vdev->release = video_device_release_empty;
+ vdev->fops = &mali_c55_params_v4l2_fops;
+ vdev->ioctl_ops = &mali_c55_params_v4l2_ioctl_ops;
+ vdev->lock = ¶ms->lock;
+ vdev->v4l2_dev = &mali_c55->v4l2_dev;
+ vdev->queue = ¶ms->queue;
+ vdev->device_caps = V4L2_CAP_META_OUTPUT | V4L2_CAP_STREAMING;
+ vdev->vfl_dir = VFL_DIR_TX;
+ video_set_drvdata(vdev, params);
+
+ ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+ if (ret) {
+ dev_err(mali_c55->dev,
+ "failed to register params video device\n");
+ goto err_release_vb2q;
+ }
+
+ params->mali_c55 = mali_c55;
+
+ return 0;
+
+err_release_vb2q:
+ vb2_queue_release(vb2q);
+err_cleanup_entity:
+ media_entity_cleanup(¶ms->vdev.entity);
+err_destroy_mutex:
+ mutex_destroy(¶ms->lock);
+
+ return ret;
+}
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
index e72e749b81d5..f2cad402492c 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
@@ -159,6 +159,23 @@ enum mali_c55_interrupts {
#define MALI_C55_BAYER_ORDER_GBRG 2
#define MALI_C55_BAYER_ORDER_BGGR 3
+#define MALI_C55_REG_METERING_CONFIG 0x18ed0
+#define MALI_C55_5BIN_HIST_DISABLE_MASK BIT(0)
+#define MALI_C55_5BIN_HIST_SWITCH_MASK GENMASK(2, 1)
+#define MALI_C55_5BIN_HIST_SWITCH(x) 1
+#define MALI_C55_AF_DISABLE_MASK BIT(4)
+#define MALI_C55_AF_SWITCH_MASK BIT(5)
+#define MALI_C55_AWB_DISABLE_MASK BIT(8)
+#define MALI_C55_AWB_SWITCH_MASK BIT(9)
+#define MALI_C55_AWB_SWITCH(x) ((x) << 9)
+#define MALI_C55_AEXP_HIST_DISABLE_MASK BIT(12)
+#define MALI_C55_AEXP_HIST_DISABLE (0x01 << 12)
+#define MALI_C55_AEXP_HIST_SWITCH_MASK GENMASK(14, 13)
+#define MALI_C55_AEXP_HIST_SWITCH(x) ((x) << 13)
+#define MALI_C55_AEXP_IHIST_DISABLE_MASK BIT(16)
+#define MALI_C55_AEXP_IHIST_DISABLE (0x01 << 12)
+#define MALI_C55_AEXP_SRC_MASK BIT(24)
+
#define MALI_C55_REG_TPG_CH0 0x18ed8
#define MALI_C55_TEST_PATTERN_ON_OFF BIT(0)
#define MALI_C55_TEST_PATTERN_RGB_MASK BIT(1)
@@ -179,6 +196,11 @@ enum mali_c55_interrupts {
#define MALI_C55_REG_CONFIG_SPACES_OFFSET 0x0ab6c
#define MALI_C55_CONFIG_SPACE_SIZE 0x1231c
+#define MALI_C55_REG_DIGITAL_GAIN 0x1926c
+#define MALI_C55_DIGITAL_GAIN_MASK GENMASK(12, 0)
+#define MALI_C55_REG_DIGITAL_GAIN_OFFSET 0x19270
+#define MALI_C55_DIGITAL_GAIN_OFFSET_MASK GENMASK(19, 0)
+
#define MALI_C55_REG_SINTER_CONFIG 0x19348
#define MALI_C55_SINTER_VIEW_FILTER_MASK GENMASK(1, 0)
#define MALI_C55_SINTER_SCALE_MODE_MASK GENMASK(3, 2)
@@ -187,6 +209,59 @@ enum mali_c55_interrupts {
#define MALI_C55_SINTER_INT_SELECT_MASK BIT(6)
#define MALI_C55_SINTER_RM_ENABLE_MASK BIT(7)
+/* Black Level Correction Configuration */
+#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_00 0x1abcc
+#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_01 0x1abd0
+#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_10 0x1abd4
+#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_11 0x1abd8
+#define MALI_C55_SENSOR_OFF_PRE_SHA_MASK 0xfffff
+
+/* Lens Mesh Shading Configuration */
+#define MALI_C55_REG_MESH_SHADING_TABLES 0x13074
+#define MALI_C55_REG_MESH_SHADING_CONFIG 0x1abfc
+#define MALI_C55_MESH_SHADING_ENABLE_MASK BIT(0)
+#define MALI_C55_MESH_SHADING_MESH_SHOW_MASK BIT(1)
+#define MALI_C55_MESH_SHADING_MESH_SHOW(x) ((x) << 1)
+#define MALI_C55_MESH_SHADING_SCALE_MASK GENMASK(4, 2)
+#define MALI_C55_MESH_SHADING_SCALE(x) ((x) << 2)
+#define MALI_C55_MESH_SHADING_PAGE_R_MASK GENMASK(9, 8)
+#define MALI_C55_MESH_SHADING_PAGE_R(x) ((x) << 8)
+#define MALI_C55_MESH_SHADING_PAGE_G_MASK GENMASK(11, 10)
+#define MALI_C55_MESH_SHADING_PAGE_G(x) ((x) << 10)
+#define MALI_C55_MESH_SHADING_PAGE_B_MASK GENMASK(13, 12)
+#define MALI_C55_MESH_SHADING_PAGE_B(x) ((x) << 12)
+#define MALI_C55_MESH_SHADING_MESH_WIDTH_MASK GENMASK(21, 16)
+#define MALI_C55_MESH_SHADING_MESH_WIDTH(x) ((x) << 16)
+#define MALI_C55_MESH_SHADING_MESH_HEIGHT_MASK GENMASK(29, 24)
+#define MALI_C55_MESH_SHADING_MESH_HEIGHT(x) ((x) << 24)
+
+#define MALI_C55_REG_MESH_SHADING_ALPHA_BANK 0x1ac04
+#define MALI_C55_MESH_SHADING_ALPHA_BANK_R_MASK GENMASK(2, 0)
+#define MALI_C55_MESH_SHADING_ALPHA_BANK_G_MASK GENMASK(5, 3)
+#define MALI_C55_MESH_SHADING_ALPHA_BANK_G(x) ((x) << 3)
+#define MALI_C55_MESH_SHADING_ALPHA_BANK_B_MASK GENMASK(8, 6)
+#define MALI_C55_MESH_SHADING_ALPHA_BANK_B(x) ((x) << 6)
+#define MALI_C55_REG_MESH_SHADING_ALPHA 0x1ac08
+#define MALI_C55_MESH_SHADING_ALPHA_R_MASK GENMASK(7, 0)
+#define MALI_C55_MESH_SHADING_ALPHA_G_MASK GENMASK(15, 8)
+#define MALI_C55_MESH_SHADING_ALPHA_G(x) ((x) << 8)
+#define MALI_C55_MESH_SHADING_ALPHA_B_MASK GENMASK(23, 16)
+#define MALI_C55_MESH_SHADING_ALPHA_B(x) ((x) << 16)
+#define MALI_C55_REG_MESH_SHADING_MESH_STRENGTH 0x1ac0c
+#define MALI_c55_MESH_STRENGTH_MASK GENMASK(15, 0)
+
+/* AWB Gains Configuration */
+#define MALI_C55_REG_AWB_GAINS1 0x1ac10
+#define MALI_C55_AWB_GAIN00_MASK GENMASK(11, 0)
+#define MALI_C55_AWB_GAIN01_MASK GENMASK(27, 16)
+#define MALI_C55_AWB_GAIN01(x) ((x) << 16)
+#define MALI_C55_REG_AWB_GAINS2 0x1ac14
+#define MALI_C55_AWB_GAIN10_MASK GENMASK(11, 0)
+#define MALI_C55_AWB_GAIN11_MASK GENMASK(27, 16)
+#define MALI_C55_AWB_GAIN11(x) ((x) << 16)
+#define MALI_C55_REG_AWB_GAINS1_AEXP 0x1ac18
+#define MALI_C55_REG_AWB_GAINS2_AEXP 0x1ac1c
+
/* Colour Correction Matrix Configuration */
#define MALI_C55_REG_CCM_ENABLE 0x1b07c
#define MALI_C55_CCM_ENABLE_MASK BIT(0)
@@ -209,6 +284,59 @@ enum mali_c55_interrupts {
#define MALI_C55_REG_CCM_ANTIFOG_OFFSET_B 0x1b0c8
#define MALI_C55_CCM_ANTIFOG_OFFSET_MASK GENMASK(11, 0)
+/* AWB Statistics Configuration */
+#define MALI_C55_REG_AWB_STATS_MODE 0x1b29c
+#define MALI_C55_AWB_STATS_MODE_MASK BIT(0)
+#define MALI_C55_REG_AWB_WHITE_LEVEL 0x1b2a0
+#define MALI_C55_AWB_WHITE_LEVEL_MASK GENMASK(9, 0)
+#define MALI_C55_REG_AWB_BLACK_LEVEL 0x1b2a4
+#define MALI_C55_AWB_BLACK_LEVEL_MASK GENMASK(9, 0)
+#define MALI_C55_REG_AWB_CR_MAX 0x1b2a8
+#define MALI_C55_AWB_CR_MAX_MASK GENMASK(11, 0)
+#define MALI_C55_REG_AWB_CR_MIN 0x1b2ac
+#define MALI_C55_AWB_CR_MIN_MASK GENMASK(11, 0)
+#define MALI_C55_REG_AWB_CB_MAX 0x1b2b0
+#define MALI_C55_AWB_CB_MAX_MASK GENMASK(11, 0)
+#define MALI_C55_REG_AWB_CB_MIN 0x1b2b4
+#define MALI_C55_AWB_CB_MIN_MASK GENMASK(11, 0)
+#define MALI_C55_REG_AWB_NODES_USED 0x1b2c4
+#define MALI_C55_AWB_NODES_USED_HORIZ_MASK GENMASK(7, 0)
+#define MALI_C55_AWB_NODES_USED_VERT_MASK GENMASK(15, 8)
+#define MALI_C55_AWB_NODES_USED_VERT(x) ((x) << 8)
+#define MALI_C55_REG_AWB_CR_HIGH 0x1b2c8
+#define MALI_C55_AWB_CR_HIGH_MASK GENMASK(11, 0)
+#define MALI_C55_REG_AWB_CR_LOW 0x1b2cc
+#define MALI_C55_AWB_CR_LOW_MASK GENMASK(11, 0)
+#define MALI_C55_REG_AWB_CB_HIGH 0x1b2d0
+#define MALI_C55_AWB_CB_HIGH_MASK GENMASK(11, 0)
+#define MALI_C55_REG_AWB_CB_LOW 0x1b2d4
+#define MALI_C55_AWB_CB_LOW_MASK GENMASK(11, 0)
+
+/* AEXP Metering Histogram Configuration */
+#define MALI_C55_REG_AEXP_HIST_BASE 0x1b730
+#define MALI_C55_REG_AEXP_IHIST_BASE 0x1bbac
+#define MALI_C55_AEXP_HIST_SKIP_OFFSET 0
+#define MALI_C55_AEXP_HIST_SKIP_X_MASK GENMASK(2, 0)
+#define MALI_c55_AEXP_HIST_SKIP_X(x) 0
+#define MALI_C55_AEXP_HIST_OFFSET_X_MASK BIT(3)
+#define MALI_C55_AEXP_HIST_OFFSET_X(x) ((x) << 3)
+#define MALI_C55_AEXP_HIST_SKIP_Y_MASK GENMASK(6, 4)
+#define MALI_C55_AEXP_HIST_SKIP_Y(x) ((x) << 4)
+#define MALI_C55_AEXP_HIST_OFFSET_Y_MASK BIT(7)
+#define MALI_C55_AEXP_HIST_OFFSET_Y(x) ((x) << 7)
+#define MALI_C55_AEXP_HIST_SCALE_OFFSET 4
+#define MALI_C55_AEXP_HIST_SCALE_BOTTOM_MASK GENMASK(3, 0)
+#define MALI_C55_AEXP_HIST_SCALE_TOP_MASK GENMASK(7, 4)
+#define MALI_C55_AEXP_HIST_SCALE_TOP(x) ((x) << 4)
+#define MALI_C55_AEXP_HIST_PLANE_MODE_OFFSET 16
+#define MALI_C55_AEXP_HIST_PLANE_MODE_MASK GENMASK(2, 0)
+#define MALI_C55_AEXP_HIST_NODES_USED_OFFSET 52
+#define MALI_C55_AEXP_HIST_NODES_USED_HORIZ_MASK GENMASK(7, 0)
+#define MALI_C55_AEXP_HIST_NODES_USED_VERT_MASK GENMASK(15, 8)
+#define MALI_C55_AEXP_HIST_NODES_USED_VERT(x) ((x) << 8)
+#define MALI_C55_AEXP_HIST_ZONE_WEIGHTS_OFFSET 56
+#define MALI_C55_AEXP_HIST_ZONE_WEIGHT_MASK 0x0f0f0f0f
+
/*
* The Mali-C55 ISP has up to two output pipes; known as full resolution and
* down scaled. The register space for these is laid out identically, but offset
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v6 18/18] Documentation: mali-c55: Document the mali-c55 parameter setting
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
` (16 preceding siblings ...)
2024-07-09 13:29 ` [PATCH v6 17/18] media: platform: Add mali-c55 parameters video node Daniel Scally
@ 2024-07-09 13:29 ` Daniel Scally
17 siblings, 0 replies; 41+ messages in thread
From: Daniel Scally @ 2024-07-09 13:29 UTC (permalink / raw)
To: linux-media, devicetree, linux-arm-kernel
Cc: jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, laurent.pinchart, sakari.ailus, Daniel Scally
Document the mali-c55 parameter setting by expanding the relevant
pages in both admin-guide/ and userspace-api/.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
---
Documentation/admin-guide/media/mali-c55.rst | 19 +++++-
.../media/v4l/metafmt-arm-mali-c55.rst | 66 ++++++++++++++++++-
2 files changed, 80 insertions(+), 5 deletions(-)
diff --git a/Documentation/admin-guide/media/mali-c55.rst b/Documentation/admin-guide/media/mali-c55.rst
index 7eaeac63ddf7..dbed5179d5f8 100644
--- a/Documentation/admin-guide/media/mali-c55.rst
+++ b/Documentation/admin-guide/media/mali-c55.rst
@@ -387,9 +387,24 @@ the processing flow the statistics can be drawn from::
+-------------+ | +-------------+
+--> AWB-1
-At present all statistics are drawn from the 0th tap point for each algorithm;
+By default all statistics are drawn from the 0th tap point for each algorithm;
I.E. AEXP statistics from AEXP-0 (A), AWB statistics from AWB-0 and AF
-statistics from AF-0. In the future this will be configurable.
+statistics from AF-0. This is configurable for AEXP and AWB statsistics through
+programming the ISP's parameters.
+
+.. _mali-c55-3a-params:
+
+Programming ISP Parameters
+==========================
+
+The ISP can be programmed with various parameters from userspace to apply to the
+hardware before and during video stream. This allows userspace to dynamically
+change values such as black level, white balance and lens shading gains and so
+on.
+
+The buffer format and how to populate it are described by the
+:ref:`V4L2_META_FMT_MALI_C55_PARAMS <v4l2-meta-fmt-mali-c55-params>` format,
+which should be set as the data format for the `mali-c55 3a params` video node.
References
==========
diff --git a/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst b/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst
index 186e0deb9ece..c0948b41fb0c 100644
--- a/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst
+++ b/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst
@@ -1,10 +1,11 @@
.. SPDX-License-Identifier: GPL-2.0
+.. _v4l2-meta-fmt-mali-c55-params:
.. _v4l2-meta-fmt-mali-c55-3a-stats:
-*************************************
-V4L2_META_FMT_MALI_C55_STATS ('C55S')
-*************************************
+*****************************************************************************
+V4L2_META_FMT_MALI_C55_STATS ('C55S'), V4L2_META_FMT_MALI_C55_PARAMS ('C55P')
+*****************************************************************************
3A Statistics
=============
@@ -23,6 +24,65 @@ of the C structure :c:type:`mali_c55_stats_buffer` defined in
For details of the statistics see :c:type:`mali_c55_stats_buffer`.
+Configuration Parameters
+========================
+
+The configuration parameters are passed to the
+:ref:`mali-c55 3a params <mali-c55-3a-params>` metadata output video node, using
+the :c:type:`v4l2_meta_format` interface. Rather than a single struct containing
+sub-structs for each configurable area of the ISP, parameters for the Mali-C55
+are defined as distinct structs or "blocks" which may be added to the data
+member of :c:type:`mali_c55_params_buffer`. Userspace is responsible for
+populating the data member with the blocks that need to be configured by the driver, but
+need not populate it with **all** the blocks, or indeed with any at all if there
+are no configuration changes to make. Populated blocks **must** be consecutive
+in the buffer. To assist both userspace and the driver in identifying the
+blocks each block-specific struct embeds
+:c:type:`mali_c55_params_block_header` as its first member and userspace
+must populate the type member with a value from
+:c:type:`mali_c55_param_block_type`. Once the blocks have been populated
+into the data buffer, the combined size of all populated blocks shall be set in
+the total_size member of :c:type:`mali_c55_params_buffer`. For example:
+
+.. code-block:: c
+
+ struct mali_c55_params_buffer *params =
+ (struct mali_c55_params_buffer *)buffer;
+
+ params->version = MALI_C55_PARAM_BUFFER_V1;
+ params->total_size = 0;
+
+ void *data = (void *)params->data;
+
+ struct mali_c55_params_awb_gains *gains =
+ (struct mali_c55_params_awb_gains *)data;
+
+ gains->header.type = MALI_C55_PARAM_BLOCK_AWB_GAINS;
+ gains->header.enabled = true;
+ gains->header.size = sizeof(struct mali_c55_params_awb_gains);
+
+ gains->gain00 = 256;
+ gains->gain00 = 256;
+ gains->gain00 = 256;
+ gains->gain00 = 256;
+
+ data += sizeof(struct mali_c55_params_awb_gains);
+ params->total_size += sizeof(struct mali_c55_params_awb_gains);
+
+ struct mali_c55_params_sensor_off_preshading *blc =
+ (struct mali_c55_params_sensor_off_preshading *)data;
+
+ blc->header.type = MALI_C55_PARAM_BLOCK_SENSOR_OFFS;
+ blc->header.enabled = true;
+ blc->header.size = sizeof(struct mali_c55_params_sensor_off_preshading);
+
+ blc->chan00 = 51200;
+ blc->chan01 = 51200;
+ blc->chan10 = 51200;
+ blc->chan11 = 51200;
+
+ params->total_size += sizeof(struct mali_c55_params_sensor_off_preshading);
+
Arm Mali-C55 uAPI data types
============================
--
2.34.1
^ permalink raw reply related [flat|nested] 41+ messages in thread
* Re: [PATCH v6 12/18] media: platform: Add mali-c55 3a stats devnode
2024-07-09 13:29 ` [PATCH v6 12/18] media: platform: Add mali-c55 3a stats devnode Daniel Scally
@ 2024-07-22 14:50 ` Laurent Pinchart
2024-07-22 22:55 ` Laurent Pinchart
` (2 more replies)
0 siblings, 3 replies; 41+ messages in thread
From: Laurent Pinchart @ 2024-07-22 14:50 UTC (permalink / raw)
To: Daniel Scally
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Dan,
Thank you for the patch.
On Tue, Jul 09, 2024 at 02:29:00PM +0100, Daniel Scally wrote:
> Add a new code file to govern the 3a statistics capture node.
>
> Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
> ---
> Changes in v6:
>
> - Fixed mising includes
> - Minor renames and formatting
> - Reworked mali_c55_stats_metering_complete() so it could only return
> buffers when both halves of the DMA read were done
> - Terminate dma transfers on streamoff
>
> Changes in v5:
>
> - New patch
>
> drivers/media/platform/arm/mali-c55/Makefile | 1 +
> .../platform/arm/mali-c55/mali-c55-common.h | 29 ++
> .../platform/arm/mali-c55/mali-c55-core.c | 15 +
> .../platform/arm/mali-c55/mali-c55-isp.c | 11 +
> .../arm/mali-c55/mali-c55-registers.h | 3 +
> .../platform/arm/mali-c55/mali-c55-stats.c | 373 ++++++++++++++++++
> 6 files changed, 432 insertions(+)
> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-stats.c
>
> diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile
> index 9178ac35e50e..b5a22d414479 100644
> --- a/drivers/media/platform/arm/mali-c55/Makefile
> +++ b/drivers/media/platform/arm/mali-c55/Makefile
> @@ -4,6 +4,7 @@ mali-c55-y := mali-c55-capture.o \
> mali-c55-core.o \
> mali-c55-isp.o \
> mali-c55-resizer.o \
> + mali-c55-stats.o \
> mali-c55-tpg.o
>
> obj-$(CONFIG_VIDEO_MALI_C55) += mali-c55.o
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> index f7764a938e9f..136c785c68ba 100644
> --- a/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> @@ -49,6 +49,7 @@ enum mali_c55_isp_pads {
> MALI_C55_ISP_PAD_SINK_VIDEO,
> MALI_C55_ISP_PAD_SOURCE_VIDEO,
> MALI_C55_ISP_PAD_SOURCE_BYPASS,
> + MALI_C55_ISP_PAD_SOURCE_STATS,
> MALI_C55_ISP_NUM_PADS,
> };
>
> @@ -160,6 +161,29 @@ struct mali_c55_cap_dev {
> bool streaming;
> };
>
> +struct mali_c55_stats_buf {
> + struct vb2_v4l2_buffer vb;
> + unsigned int segments_remaining;
> + struct list_head queue;
> + bool failed;
> +};
> +
> +struct mali_c55_stats {
> + struct mali_c55 *mali_c55;
> + struct video_device vdev;
> + struct dma_chan *channel;
> + struct vb2_queue queue;
> + struct media_pad pad;
> + /* Mutex to provide to vb2 */
> + struct mutex lock;
> +
> + struct {
> + /* Spinlock to guard buffer queue */
> + spinlock_t lock;
> + struct list_head queue;
> + } buffers;
> +};
> +
> enum mali_c55_config_spaces {
> MALI_C55_CONFIG_PING,
> MALI_C55_CONFIG_PONG,
> @@ -202,6 +226,7 @@ struct mali_c55 {
> struct mali_c55_isp isp;
> struct mali_c55_resizer resizers[MALI_C55_NUM_RSZS];
> struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS];
> + struct mali_c55_stats stats;
>
> struct mali_c55_context context;
> enum mali_c55_config_spaces next_config;
> @@ -229,6 +254,8 @@ int mali_c55_register_resizers(struct mali_c55 *mali_c55);
> void mali_c55_unregister_resizers(struct mali_c55 *mali_c55);
> int mali_c55_register_capture_devs(struct mali_c55 *mali_c55);
> void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55);
> +int mali_c55_register_stats(struct mali_c55 *mali_c55);
> +void mali_c55_unregister_stats(struct mali_c55 *mali_c55);
> struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55);
> void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
> enum mali_c55_planes plane);
> @@ -243,5 +270,7 @@ const struct mali_c55_isp_fmt *
> mali_c55_isp_get_mbus_config_by_code(u32 code);
> const struct mali_c55_isp_fmt *
> mali_c55_isp_get_mbus_config_by_index(u32 index);
> +void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
> + enum mali_c55_config_spaces cfg_space);
>
> #endif /* _MALI_C55_COMMON_H */
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> index dbc07d12d3a3..eedc8f450184 100644
> --- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> @@ -374,6 +374,16 @@ static int mali_c55_create_links(struct mali_c55 *mali_c55)
> }
> }
>
> + ret = media_create_pad_link(&mali_c55->isp.sd.entity,
> + MALI_C55_ISP_PAD_SOURCE_STATS,
> + &mali_c55->stats.vdev.entity, 0,
> + MEDIA_LNK_FL_ENABLED);
> + if (ret) {
> + dev_err(mali_c55->dev,
> + "failed to link ISP and 3a stats node\n");
> + goto err_remove_links;
> + }
> +
> return 0;
>
> err_remove_links:
> @@ -388,6 +398,7 @@ static void mali_c55_unregister_entities(struct mali_c55 *mali_c55)
> mali_c55_unregister_isp(mali_c55);
> mali_c55_unregister_resizers(mali_c55);
> mali_c55_unregister_capture_devs(mali_c55);
> + mali_c55_unregister_stats(mali_c55);
> }
>
> static int mali_c55_register_entities(struct mali_c55 *mali_c55)
> @@ -410,6 +421,10 @@ static int mali_c55_register_entities(struct mali_c55 *mali_c55)
> if (ret)
> goto err_unregister_entities;
>
> + ret = mali_c55_register_stats(mali_c55);
> + if (ret)
> + goto err_unregister_entities;
> +
> ret = mali_c55_create_links(mali_c55);
> if (ret)
> goto err_unregister_entities;
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> index f784983a4ccc..2f450c00300a 100644
> --- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> @@ -5,6 +5,8 @@
> * Copyright (C) 2024 Ideas on Board Oy
> */
>
> +#include <linux/media/arm/mali-c55-config.h>
> +
> #include <linux/delay.h>
> #include <linux/iopoll.h>
> #include <linux/property.h>
> @@ -460,6 +462,14 @@ static int mali_c55_isp_init_state(struct v4l2_subdev *sd,
> in_crop->width = MALI_C55_DEFAULT_WIDTH;
> in_crop->height = MALI_C55_DEFAULT_HEIGHT;
>
> + src_fmt = v4l2_subdev_state_get_format(state,
> + MALI_C55_ISP_PAD_SOURCE_STATS);
> +
> + src_fmt->width = sizeof(struct mali_c55_stats_buffer);
> + src_fmt->height = 1;
According to
https://docs.kernel.org/userspace-api/media/v4l/subdev-formats.html#metadata-formats,
width and height should be set to 0 for MEDIA_BUS_FMT_METADATA_FIXED. I
haven't checked if v4l2-compliance expects this, we may have
discrepancies between the API documentation and the existing
implementations in the kernel and userspace. In any case, this should be
sorted out, either by fixing the kernel code and enforcing the
requirement in v4l2-compliance, or fixing the documentation.
> + src_fmt->field = V4L2_FIELD_NONE;
> + src_fmt->code = MEDIA_BUS_FMT_METADATA_FIXED;
> +
> return 0;
> }
>
> @@ -490,6 +500,7 @@ int mali_c55_register_isp(struct mali_c55 *mali_c55)
> MEDIA_PAD_FL_MUST_CONNECT;
> isp->pads[MALI_C55_ISP_PAD_SOURCE_VIDEO].flags = MEDIA_PAD_FL_SOURCE;
> isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE;
> + isp->pads[MALI_C55_ISP_PAD_SOURCE_STATS].flags = MEDIA_PAD_FL_SOURCE;
>
> ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS,
> isp->pads);
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
> index 0a391f8a043e..e72e749b81d5 100644
> --- a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
> @@ -103,6 +103,9 @@ enum mali_c55_interrupts {
> #define MALI_C55_VC_START(v) ((v) & 0xffff)
> #define MALI_C55_VC_SIZE(v) (((v) & 0xffff) << 16)
>
> +#define MALI_C55_REG_1024BIN_HIST 0x054a8
> +#define MALI_C55_1024BIN_HIST_SIZE 4096
> +
> /* Ping/Pong Configuration Space */
> #define MALI_C55_REG_BASE_ADDR 0x18e88
> #define MALI_C55_REG_BYPASS_0 0x18eac
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-stats.c b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
> new file mode 100644
> index 000000000000..38a17fb5d052
> --- /dev/null
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
> @@ -0,0 +1,373 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * ARM Mali-C55 ISP Driver - 3A Statistics capture device
> + *
> + * Copyright (C) 2023 Ideas on Board Oy
> + */
> +
> +#include <linux/container_of.h>
> +#include <linux/dev_printk.h>
> +#include <linux/dmaengine.h>
> +#include <linux/list.h>
> +#include <linux/media/arm/mali-c55-config.h>
> +#include <linux/mutex.h>
> +#include <linux/spinlock.h>
> +#include <linux/string.h>
> +
> +#include <media/media-entity.h>
> +#include <media/v4l2-dev.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-fh.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mali-c55-common.h"
> +#include "mali-c55-registers.h"
> +
> +static const unsigned int metering_space_addrs[] = {
> + [MALI_C55_CONFIG_PING] = 0x095ac,
> + [MALI_C55_CONFIG_PONG] = 0x2156c,
> +};
> +
> +static int mali_c55_stats_enum_fmt_meta_cap(struct file *file, void *fh,
> + struct v4l2_fmtdesc *f)
> +{
> + if (f->index)
> + return -EINVAL;
> +
> + f->pixelformat = V4L2_META_FMT_MALI_C55_STATS;
> +
> + return 0;
> +}
> +
> +static int mali_c55_stats_g_fmt_meta_cap(struct file *file, void *fh,
> + struct v4l2_format *f)
> +{
> + static const struct v4l2_meta_format mfmt = {
> + .dataformat = V4L2_META_FMT_MALI_C55_STATS,
> + .buffersize = sizeof(struct mali_c55_stats_buffer)
> + };
> +
> + f->fmt.meta = mfmt;
> +
> + return 0;
> +}
> +
> +static int mali_c55_stats_querycap(struct file *file,
> + void *priv, struct v4l2_capability *cap)
> +{
> + strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver));
> + strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card));
> +
> + return 0;
> +}
> +
> +static const struct v4l2_ioctl_ops mali_c55_stats_v4l2_ioctl_ops = {
> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> + .vidioc_querybuf = vb2_ioctl_querybuf,
> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> + .vidioc_qbuf = vb2_ioctl_qbuf,
> + .vidioc_expbuf = vb2_ioctl_expbuf,
> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> + .vidioc_streamon = vb2_ioctl_streamon,
> + .vidioc_streamoff = vb2_ioctl_streamoff,
> + .vidioc_enum_fmt_meta_cap = mali_c55_stats_enum_fmt_meta_cap,
> + .vidioc_g_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
> + .vidioc_s_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
> + .vidioc_try_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
> + .vidioc_querycap = mali_c55_stats_querycap,
> + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static const struct v4l2_file_operations mali_c55_stats_v4l2_fops = {
> + .owner = THIS_MODULE,
> + .unlocked_ioctl = video_ioctl2,
> + .open = v4l2_fh_open,
> + .release = vb2_fop_release,
> + .poll = vb2_fop_poll,
> + .mmap = vb2_fop_mmap,
> +};
> +
> +static int
> +mali_c55_stats_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
> + unsigned int *num_planes, unsigned int sizes[],
> + struct device *alloc_devs[])
> +{
> + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
> +
> + if (*num_planes && *num_planes > 1)
> + return -EINVAL;
> +
> + if (sizes[0] && sizes[0] < sizeof(struct mali_c55_stats_buffer))
> + return -EINVAL;
> +
> + *num_planes = 1;
> +
> + if (!sizes[0])
> + sizes[0] = sizeof(struct mali_c55_stats_buffer);
> +
> + if (stats->channel)
> + alloc_devs[0] = stats->channel->device->dev;
> +
> + return 0;
> +}
> +
> +static void mali_c55_stats_buf_queue(struct vb2_buffer *vb)
> +{
> + struct mali_c55_stats *stats = vb2_get_drv_priv(vb->vb2_queue);
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct mali_c55_stats_buf *buf = container_of(vbuf,
> + struct mali_c55_stats_buf, vb);
> +
> + vb2_set_plane_payload(vb, 0, sizeof(struct mali_c55_stats_buffer));
> + buf->segments_remaining = 2;
> + buf->failed = false;
> +
> + spin_lock(&stats->buffers.lock);
> + list_add_tail(&buf->queue, &stats->buffers.queue);
> + spin_unlock(&stats->buffers.lock);
> +}
> +
> +static int mali_c55_stats_start_streaming(struct vb2_queue *q,
> + unsigned int count)
> +{
> + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
> + struct mali_c55 *mali_c55 = stats->mali_c55;
> + int ret;
> +
> + ret = video_device_pipeline_start(&stats->vdev,
> + &stats->mali_c55->pipe);
> + if (ret)
> + return ret;
> +
> + if (mali_c55->pipe.start_count == mali_c55->pipe.required_count) {
> + ret = v4l2_subdev_enable_streams(&mali_c55->isp.sd,
> + MALI_C55_ISP_PAD_SOURCE_VIDEO,
> + BIT(0));
> + if (ret)
> + goto err_stop_pipeline;
> + }
> +
> + return 0;
> +
> +err_stop_pipeline:
> + video_device_pipeline_stop(&stats->vdev);
> +
> + return ret;
> +}
> +
> +static void mali_c55_stats_stop_streaming(struct vb2_queue *q)
> +{
> + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
> + struct mali_c55_stats_buf *buf, *tmp;
> +
> + dmaengine_terminate_sync(stats->channel);
> +
> + spin_lock(&stats->buffers.lock);
> +
> + list_for_each_entry_safe(buf, tmp, &stats->buffers.queue, queue) {
> + list_del(&buf->queue);
> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> + }
> +
> + spin_unlock(&stats->buffers.lock);
> +
> + video_device_pipeline_stop(&stats->vdev);
> +}
> +
> +static const struct vb2_ops mali_c55_stats_vb2_ops = {
> + .queue_setup = mali_c55_stats_queue_setup,
> + .buf_queue = mali_c55_stats_buf_queue,
> + .wait_prepare = vb2_ops_wait_prepare,
> + .wait_finish = vb2_ops_wait_finish,
> + .start_streaming = mali_c55_stats_start_streaming,
> + .stop_streaming = mali_c55_stats_stop_streaming,
> +};
> +
> +static void
> +mali_c55_stats_metering_complete(void *param,
> + const struct dmaengine_result *result)
> +{
> + struct mali_c55_stats_buf *buf = param;
> +
> + if (result->result != DMA_TRANS_NOERROR)
> + buf->failed = true;
> +
> + if (!--buf->segments_remaining)
> + vb2_buffer_done(&buf->vb.vb2_buf, buf->failed ?
> + VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE);
> +}
> +
> +static int mali_c55_stats_dma_xfer(struct mali_c55_stats *stats, dma_addr_t src,
> + dma_addr_t dst,
> + struct mali_c55_stats_buf *buf,
> + size_t length)
> +{
> + struct dma_async_tx_descriptor *tx;
> + dma_cookie_t cookie;
> +
> + tx = dmaengine_prep_dma_memcpy(stats->channel, dst, src, length, 0);
> + if (!tx) {
> + dev_err(stats->mali_c55->dev, "failed to prep stats DMA\n");
> + return -EIO;
> + }
> +
> + tx->callback_result = mali_c55_stats_metering_complete;
> + tx->callback_param = buf;
> +
> + cookie = dmaengine_submit(tx);
> + if (dma_submit_error(cookie)) {
> + dev_err(stats->mali_c55->dev, "failed to submit stats DMA\n");
> + return -EIO;
> + }
> +
> + dma_async_issue_pending(stats->channel);
> + return 0;
> +}
> +
> +void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
> + enum mali_c55_config_spaces cfg_space)
> +{
> + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
> + struct mali_c55_stats *stats = &mali_c55->stats;
> + struct mali_c55_stats_buf *buf = NULL;
> + dma_addr_t src, dst;
> + size_t length;
> + int ret;
> +
> + spin_lock(&stats->buffers.lock);
> + if (!list_empty(&stats->buffers.queue)) {
> + buf = list_first_entry(&stats->buffers.queue,
> + struct mali_c55_stats_buf, queue);
> + list_del(&buf->queue);
> + }
> + spin_unlock(&stats->buffers.lock);
> +
> + if (!buf)
> + return;
> +
> + buf->vb.sequence = mali_c55->isp.frame_sequence;
> + buf->vb.vb2_buf.timestamp = ktime_get_boottime_ns();
> +
> + /*
> + * There are in fact two noncontiguous sections of the ISP's
> + * memory space that hold statistics for 3a algorithms to use: A
> + * section in each config space and a global section holding
> + * histograms which is double buffered and so holds data for the
> + * last frame. We need to read both.
> + */
> + src = ctx->base + MALI_C55_REG_1024BIN_HIST;
> + dst = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> +
> + ret = mali_c55_stats_dma_xfer(stats, src, dst, buf,
> + MALI_C55_1024BIN_HIST_SIZE);
> + if (ret)
> + goto err_fail_buffer;
> +
> + src = ctx->base + metering_space_addrs[cfg_space];
> + dst += MALI_C55_1024BIN_HIST_SIZE;
> +
> + length = sizeof(struct mali_c55_stats_buffer) - MALI_C55_1024BIN_HIST_SIZE;
> + ret = mali_c55_stats_dma_xfer(stats, src, dst, buf, length);
> + if (ret) {
> + dmaengine_terminate_sync(stats->channel);
> + goto err_fail_buffer;
> + }
> +
> + return;
> +
> +err_fail_buffer:
> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> +}
> +
> +void mali_c55_unregister_stats(struct mali_c55 *mali_c55)
> +{
> + struct mali_c55_stats *stats = &mali_c55->stats;
> +
> + if (!video_is_registered(&stats->vdev))
> + return;
> +
> + vb2_video_unregister_device(&stats->vdev);
> + media_entity_cleanup(&stats->vdev.entity);
> + dma_release_channel(stats->channel);
> + mutex_destroy(&stats->lock);
> +}
> +
> +int mali_c55_register_stats(struct mali_c55 *mali_c55)
> +{
> + struct mali_c55_stats *stats = &mali_c55->stats;
> + struct video_device *vdev = &stats->vdev;
> + struct vb2_queue *vb2q = &stats->queue;
> + dma_cap_mask_t mask;
> + int ret;
> +
> + mutex_init(&stats->lock);
> + INIT_LIST_HEAD(&stats->buffers.queue);
> +
> + dma_cap_zero(mask);
> + dma_cap_set(DMA_MEMCPY, mask);
> +
> + stats->channel = dma_request_channel(mask, 0, NULL);
> + if (!stats->channel) {
> + ret = -ENODEV;
> + goto err_destroy_mutex;
> + }
> +
> + stats->pad.flags = MEDIA_PAD_FL_SINK;
> + ret = media_entity_pads_init(&stats->vdev.entity, 1, &stats->pad);
> + if (ret)
> + goto err_release_dma_channel;
> +
> + vb2q->type = V4L2_BUF_TYPE_META_CAPTURE;
> + vb2q->io_modes = VB2_MMAP | VB2_DMABUF;
> + vb2q->drv_priv = stats;
> + vb2q->mem_ops = &vb2_dma_contig_memops;
> + vb2q->ops = &mali_c55_stats_vb2_ops;
> + vb2q->buf_struct_size = sizeof(struct mali_c55_stats_buf);
> + vb2q->min_queued_buffers = 1;
> + vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> + vb2q->lock = &stats->lock;
> + vb2q->dev = stats->channel->device->dev;
> +
> + ret = vb2_queue_init(vb2q);
> + if (ret) {
> + dev_err(mali_c55->dev, "stats vb2 queue init failed\n");
> + goto err_cleanup_entity;
> + }
> +
> + strscpy(stats->vdev.name, "mali-c55 3a stats", sizeof(stats->vdev.name));
> + vdev->release = video_device_release_empty;
> + vdev->fops = &mali_c55_stats_v4l2_fops;
> + vdev->ioctl_ops = &mali_c55_stats_v4l2_ioctl_ops;
> + vdev->lock = &stats->lock;
> + vdev->v4l2_dev = &mali_c55->v4l2_dev;
> + vdev->queue = &stats->queue;
> + vdev->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
> + vdev->vfl_dir = VFL_DIR_RX;
> + video_set_drvdata(vdev, stats);
> +
> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> + if (ret) {
> + dev_err(mali_c55->dev,
> + "failed to register stats video device\n");
> + goto err_release_vb2q;
> + }
> +
> + stats->mali_c55 = mali_c55;
> +
> + return 0;
> +
> +err_release_vb2q:
> + vb2_queue_release(vb2q);
> +err_cleanup_entity:
> + media_entity_cleanup(&stats->vdev.entity);
> +err_release_dma_channel:
> + dma_release_channel(stats->channel);
> +err_destroy_mutex:
> + mutex_destroy(&stats->lock);
> +
> + return ret;
> +}
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 03/18] media: uapi: Add 20-bit bayer formats
2024-07-09 13:28 ` [PATCH v6 03/18] media: uapi: Add 20-bit bayer formats Daniel Scally
@ 2024-07-22 22:13 ` Laurent Pinchart
0 siblings, 0 replies; 41+ messages in thread
From: Laurent Pinchart @ 2024-07-22 22:13 UTC (permalink / raw)
To: Daniel Scally
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Dan,
Thank you for the patch.
On Tue, Jul 09, 2024 at 02:28:51PM +0100, Daniel Scally wrote:
> The Mali-C55 requires input data be in 20-bit format, MSB aligned.
> Add some new media bus format macros to represent that input format.
>
> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Changes in v6:
>
> - Fixed the table formatting
>
> Changes in v5:
>
> - New patch
>
> .../media/v4l/subdev-formats.rst | 252 +++++++++++++++++-
> include/uapi/linux/media-bus-format.h | 6 +-
> 2 files changed, 255 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> index 5dbf8c9b18fb..ff328eb485e8 100644
> --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> @@ -2664,7 +2664,7 @@ organization is given as an example for the first pixel only.
> \tiny
> \setlength{\tabcolsep}{2pt}
>
> -.. tabularcolumns:: |p{6.0cm}|p{0.7cm}|p{0.3cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|
> +.. tabularcolumns:: |p{6.0cm}|p{0.7cm}|p{0.3cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|
>
> .. _v4l2-mbus-pixelcode-bayer:
>
> @@ -2677,10 +2677,14 @@ organization is given as an example for the first pixel only.
> * - Identifier
> - Code
> -
> - - :cspan:`15` Data organization
> + - :cspan:`19` Data organization
> * -
> -
> - Bit
> + - 19
> + - 18
> + - 17
> + - 16
> - 15
> - 14
> - 13
> @@ -2710,6 +2714,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - b\ :sub:`7`
> - b\ :sub:`6`
> - b\ :sub:`5`
> @@ -2731,6 +2739,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - g\ :sub:`7`
> - g\ :sub:`6`
> - g\ :sub:`5`
> @@ -2752,6 +2764,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - g\ :sub:`7`
> - g\ :sub:`6`
> - g\ :sub:`5`
> @@ -2773,6 +2789,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - r\ :sub:`7`
> - r\ :sub:`6`
> - r\ :sub:`5`
> @@ -2794,6 +2814,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - b\ :sub:`7`
> - b\ :sub:`6`
> - b\ :sub:`5`
> @@ -2815,6 +2839,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - g\ :sub:`7`
> - g\ :sub:`6`
> - g\ :sub:`5`
> @@ -2836,6 +2864,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - g\ :sub:`7`
> - g\ :sub:`6`
> - g\ :sub:`5`
> @@ -2857,6 +2889,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - r\ :sub:`7`
> - r\ :sub:`6`
> - r\ :sub:`5`
> @@ -2878,6 +2914,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - b\ :sub:`7`
> - b\ :sub:`6`
> - b\ :sub:`5`
> @@ -2899,6 +2939,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - g\ :sub:`7`
> - g\ :sub:`6`
> - g\ :sub:`5`
> @@ -2920,6 +2964,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - g\ :sub:`7`
> - g\ :sub:`6`
> - g\ :sub:`5`
> @@ -2941,6 +2989,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - r\ :sub:`7`
> - r\ :sub:`6`
> - r\ :sub:`5`
> @@ -2962,6 +3014,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - 0
> - 0
> - 0
> @@ -2981,6 +3037,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - b\ :sub:`7`
> - b\ :sub:`6`
> - b\ :sub:`5`
> @@ -3002,6 +3062,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - b\ :sub:`7`
> - b\ :sub:`6`
> - b\ :sub:`5`
> @@ -3021,6 +3085,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - 0
> - 0
> - 0
> @@ -3042,6 +3110,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - b\ :sub:`9`
> - b\ :sub:`8`
> - b\ :sub:`7`
> @@ -3061,6 +3133,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - b\ :sub:`1`
> - b\ :sub:`0`
> - 0
> @@ -3082,6 +3158,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - b\ :sub:`1`
> - b\ :sub:`0`
> - 0
> @@ -3101,6 +3181,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - b\ :sub:`9`
> - b\ :sub:`8`
> - b\ :sub:`7`
> @@ -3120,6 +3204,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - b\ :sub:`9`
> - b\ :sub:`8`
> - b\ :sub:`7`
> @@ -3141,6 +3229,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - g\ :sub:`9`
> - g\ :sub:`8`
> - g\ :sub:`7`
> @@ -3162,6 +3254,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - g\ :sub:`9`
> - g\ :sub:`8`
> - g\ :sub:`7`
> @@ -3183,6 +3279,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - r\ :sub:`9`
> - r\ :sub:`8`
> - r\ :sub:`7`
> @@ -3202,6 +3302,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - b\ :sub:`11`
> - b\ :sub:`10`
> - b\ :sub:`9`
> @@ -3223,6 +3327,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - g\ :sub:`11`
> - g\ :sub:`10`
> - g\ :sub:`9`
> @@ -3244,6 +3352,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - g\ :sub:`11`
> - g\ :sub:`10`
> - g\ :sub:`9`
> @@ -3265,6 +3377,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - r\ :sub:`11`
> - r\ :sub:`10`
> - r\ :sub:`9`
> @@ -3284,6 +3400,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - b\ :sub:`13`
> - b\ :sub:`12`
> - b\ :sub:`11`
> @@ -3305,6 +3425,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - g\ :sub:`13`
> - g\ :sub:`12`
> - g\ :sub:`11`
> @@ -3326,6 +3450,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - g\ :sub:`13`
> - g\ :sub:`12`
> - g\ :sub:`11`
> @@ -3347,6 +3475,10 @@ organization is given as an example for the first pixel only.
> -
> -
> -
> + -
> + -
> + -
> + -
> - r\ :sub:`13`
> - r\ :sub:`12`
> - r\ :sub:`11`
> @@ -3366,6 +3498,10 @@ organization is given as an example for the first pixel only.
> - MEDIA_BUS_FMT_SBGGR16_1X16
> - 0x301d
> -
> + -
> + -
> + -
> + -
> - b\ :sub:`15`
> - b\ :sub:`14`
> - b\ :sub:`13`
> @@ -3387,6 +3523,10 @@ organization is given as an example for the first pixel only.
> - MEDIA_BUS_FMT_SGBRG16_1X16
> - 0x301e
> -
> + -
> + -
> + -
> + -
> - g\ :sub:`15`
> - g\ :sub:`14`
> - g\ :sub:`13`
> @@ -3408,6 +3548,10 @@ organization is given as an example for the first pixel only.
> - MEDIA_BUS_FMT_SGRBG16_1X16
> - 0x301f
> -
> + -
> + -
> + -
> + -
> - g\ :sub:`15`
> - g\ :sub:`14`
> - g\ :sub:`13`
> @@ -3429,6 +3573,110 @@ organization is given as an example for the first pixel only.
> - MEDIA_BUS_FMT_SRGGB16_1X16
> - 0x3020
> -
> + -
> + -
> + -
> + -
> + - r\ :sub:`15`
> + - r\ :sub:`14`
> + - r\ :sub:`13`
> + - r\ :sub:`12`
> + - r\ :sub:`11`
> + - r\ :sub:`10`
> + - r\ :sub:`9`
> + - r\ :sub:`8`
> + - r\ :sub:`7`
> + - r\ :sub:`6`
> + - r\ :sub:`5`
> + - r\ :sub:`4`
> + - r\ :sub:`3`
> + - r\ :sub:`2`
> + - r\ :sub:`1`
> + - r\ :sub:`0`
> + * .. _MEDIA-BUS-FMT-SBGGR20-1X20:
> +
> + - MEDIA_BUS_FMT_SBGGR20_1X20
> + - 0x3021
> + -
> + - b\ :sub:`19`
> + - b\ :sub:`18`
> + - b\ :sub:`17`
> + - b\ :sub:`16`
> + - b\ :sub:`15`
> + - b\ :sub:`14`
> + - b\ :sub:`13`
> + - b\ :sub:`12`
> + - b\ :sub:`11`
> + - b\ :sub:`10`
> + - b\ :sub:`9`
> + - b\ :sub:`8`
> + - b\ :sub:`7`
> + - b\ :sub:`6`
> + - b\ :sub:`5`
> + - b\ :sub:`4`
> + - b\ :sub:`3`
> + - b\ :sub:`2`
> + - b\ :sub:`1`
> + - b\ :sub:`0`
> + * .. _MEDIA-BUS-FMT-SGBRG20-1X20:
> +
> + - MEDIA_BUS_FMT_SGBRG20_1X20
> + - 0x3022
> + -
> + - g\ :sub:`19`
> + - g\ :sub:`18`
> + - g\ :sub:`17`
> + - g\ :sub:`16`
> + - g\ :sub:`15`
> + - g\ :sub:`14`
> + - g\ :sub:`13`
> + - g\ :sub:`12`
> + - g\ :sub:`11`
> + - g\ :sub:`10`
> + - g\ :sub:`9`
> + - g\ :sub:`8`
> + - g\ :sub:`7`
> + - g\ :sub:`6`
> + - g\ :sub:`5`
> + - g\ :sub:`4`
> + - g\ :sub:`3`
> + - g\ :sub:`2`
> + - g\ :sub:`1`
> + - g\ :sub:`0`
> + * .. _MEDIA-BUS-FMT-SGRBG20-1X20:
> +
> + - MEDIA_BUS_FMT_SGRBG20_1X20
> + - 0x3023
> + -
> + - g\ :sub:`19`
> + - g\ :sub:`18`
> + - g\ :sub:`17`
> + - g\ :sub:`16`
> + - g\ :sub:`15`
> + - g\ :sub:`14`
> + - g\ :sub:`13`
> + - g\ :sub:`12`
> + - g\ :sub:`11`
> + - g\ :sub:`10`
> + - g\ :sub:`9`
> + - g\ :sub:`8`
> + - g\ :sub:`7`
> + - g\ :sub:`6`
> + - g\ :sub:`5`
> + - g\ :sub:`4`
> + - g\ :sub:`3`
> + - g\ :sub:`2`
> + - g\ :sub:`1`
> + - g\ :sub:`0`
> + * .. _MEDIA-BUS-FMT-SRGGB20-1X20:
> +
> + - MEDIA_BUS_FMT_SRGGB20_1X20
> + - 0x3024
> + -
> + - r\ :sub:`19`
> + - r\ :sub:`18`
> + - r\ :sub:`17`
> + - r\ :sub:`16`
> - r\ :sub:`15`
> - r\ :sub:`14`
> - r\ :sub:`13`
> diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h
> index 49be328d9a3b..b6acf8c8e383 100644
> --- a/include/uapi/linux/media-bus-format.h
> +++ b/include/uapi/linux/media-bus-format.h
> @@ -122,7 +122,7 @@
> #define MEDIA_BUS_FMT_YUV16_1X48 0x202a
> #define MEDIA_BUS_FMT_UYYVYY16_0_5X48 0x202b
>
> -/* Bayer - next is 0x3021 */
> +/* Bayer - next is 0x3025 */
> #define MEDIA_BUS_FMT_SBGGR8_1X8 0x3001
> #define MEDIA_BUS_FMT_SGBRG8_1X8 0x3013
> #define MEDIA_BUS_FMT_SGRBG8_1X8 0x3002
> @@ -155,6 +155,10 @@
> #define MEDIA_BUS_FMT_SGBRG16_1X16 0x301e
> #define MEDIA_BUS_FMT_SGRBG16_1X16 0x301f
> #define MEDIA_BUS_FMT_SRGGB16_1X16 0x3020
> +#define MEDIA_BUS_FMT_SBGGR20_1X20 0x3021
> +#define MEDIA_BUS_FMT_SGBRG20_1X20 0x3022
> +#define MEDIA_BUS_FMT_SGRBG20_1X20 0x3023
> +#define MEDIA_BUS_FMT_SRGGB20_1X20 0x3024
>
> /* JPEG compressed formats - next is 0x4002 */
> #define MEDIA_BUS_FMT_JPEG_1X8 0x4001
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 05/18] media: v4l2-common: Add RAW14 format info
2024-07-09 13:28 ` [PATCH v6 05/18] media: v4l2-common: Add RAW14 " Daniel Scally
@ 2024-07-22 22:14 ` Laurent Pinchart
0 siblings, 0 replies; 41+ messages in thread
From: Laurent Pinchart @ 2024-07-22 22:14 UTC (permalink / raw)
To: Daniel Scally
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Dan,
Thank you for the patch.
On Tue, Jul 09, 2024 at 02:28:53PM +0100, Daniel Scally wrote:
> Add entries to v4l2_format_info describing the 14-bit bayer
> formats so that they can be used in drivers.
>
> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Changes in v6:
>
> - New patch
>
> drivers/media/v4l2-core/v4l2-common.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> index c5d5704af5ee..fe33221e6c02 100644
> --- a/drivers/media/v4l2-core/v4l2-common.c
> +++ b/drivers/media/v4l2-core/v4l2-common.c
> @@ -331,6 +331,10 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
> { .format = V4L2_PIX_FMT_SGBRG12, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
> { .format = V4L2_PIX_FMT_SGRBG12, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
> { .format = V4L2_PIX_FMT_SRGGB12, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
> + { .format = V4L2_PIX_FMT_SBGGR14, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
> + { .format = V4L2_PIX_FMT_SGBRG14, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
> + { .format = V4L2_PIX_FMT_SGRBG14, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
> + { .format = V4L2_PIX_FMT_SRGGB14, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
> { .format = V4L2_PIX_FMT_SBGGR16, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
> { .format = V4L2_PIX_FMT_SGBRG16, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
> { .format = V4L2_PIX_FMT_SGRBG16, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
> --
> 2.34.1
>
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 12/18] media: platform: Add mali-c55 3a stats devnode
2024-07-22 14:50 ` Laurent Pinchart
@ 2024-07-22 22:55 ` Laurent Pinchart
2024-08-13 13:13 ` Dan Scally
2024-07-29 10:53 ` Dan Scally
2024-08-13 13:51 ` Dan Scally
2 siblings, 1 reply; 41+ messages in thread
From: Laurent Pinchart @ 2024-07-22 22:55 UTC (permalink / raw)
To: Daniel Scally
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
On Mon, Jul 22, 2024 at 05:51:00PM +0300, Laurent Pinchart wrote:
> Hi Dan,
>
> Thank you for the patch.
>
> On Tue, Jul 09, 2024 at 02:29:00PM +0100, Daniel Scally wrote:
> > Add a new code file to govern the 3a statistics capture node.
> >
> > Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
> > Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
> > ---
> > Changes in v6:
> >
> > - Fixed mising includes
> > - Minor renames and formatting
> > - Reworked mali_c55_stats_metering_complete() so it could only return
> > buffers when both halves of the DMA read were done
> > - Terminate dma transfers on streamoff
> >
> > Changes in v5:
> >
> > - New patch
> >
> > drivers/media/platform/arm/mali-c55/Makefile | 1 +
> > .../platform/arm/mali-c55/mali-c55-common.h | 29 ++
> > .../platform/arm/mali-c55/mali-c55-core.c | 15 +
> > .../platform/arm/mali-c55/mali-c55-isp.c | 11 +
> > .../arm/mali-c55/mali-c55-registers.h | 3 +
> > .../platform/arm/mali-c55/mali-c55-stats.c | 373 ++++++++++++++++++
> > 6 files changed, 432 insertions(+)
> > create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-stats.c
> >
> > diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile
> > index 9178ac35e50e..b5a22d414479 100644
> > --- a/drivers/media/platform/arm/mali-c55/Makefile
> > +++ b/drivers/media/platform/arm/mali-c55/Makefile
> > @@ -4,6 +4,7 @@ mali-c55-y := mali-c55-capture.o \
> > mali-c55-core.o \
> > mali-c55-isp.o \
> > mali-c55-resizer.o \
> > + mali-c55-stats.o \
> > mali-c55-tpg.o
> >
> > obj-$(CONFIG_VIDEO_MALI_C55) += mali-c55.o
> > diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> > index f7764a938e9f..136c785c68ba 100644
> > --- a/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> > +++ b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> > @@ -49,6 +49,7 @@ enum mali_c55_isp_pads {
> > MALI_C55_ISP_PAD_SINK_VIDEO,
> > MALI_C55_ISP_PAD_SOURCE_VIDEO,
> > MALI_C55_ISP_PAD_SOURCE_BYPASS,
> > + MALI_C55_ISP_PAD_SOURCE_STATS,
> > MALI_C55_ISP_NUM_PADS,
> > };
> >
> > @@ -160,6 +161,29 @@ struct mali_c55_cap_dev {
> > bool streaming;
> > };
> >
> > +struct mali_c55_stats_buf {
> > + struct vb2_v4l2_buffer vb;
> > + unsigned int segments_remaining;
> > + struct list_head queue;
> > + bool failed;
> > +};
> > +
> > +struct mali_c55_stats {
> > + struct mali_c55 *mali_c55;
> > + struct video_device vdev;
> > + struct dma_chan *channel;
> > + struct vb2_queue queue;
> > + struct media_pad pad;
> > + /* Mutex to provide to vb2 */
> > + struct mutex lock;
> > +
> > + struct {
> > + /* Spinlock to guard buffer queue */
> > + spinlock_t lock;
> > + struct list_head queue;
> > + } buffers;
> > +};
> > +
> > enum mali_c55_config_spaces {
> > MALI_C55_CONFIG_PING,
> > MALI_C55_CONFIG_PONG,
> > @@ -202,6 +226,7 @@ struct mali_c55 {
> > struct mali_c55_isp isp;
> > struct mali_c55_resizer resizers[MALI_C55_NUM_RSZS];
> > struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS];
> > + struct mali_c55_stats stats;
> >
> > struct mali_c55_context context;
> > enum mali_c55_config_spaces next_config;
> > @@ -229,6 +254,8 @@ int mali_c55_register_resizers(struct mali_c55 *mali_c55);
> > void mali_c55_unregister_resizers(struct mali_c55 *mali_c55);
> > int mali_c55_register_capture_devs(struct mali_c55 *mali_c55);
> > void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55);
> > +int mali_c55_register_stats(struct mali_c55 *mali_c55);
> > +void mali_c55_unregister_stats(struct mali_c55 *mali_c55);
> > struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55);
> > void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
> > enum mali_c55_planes plane);
> > @@ -243,5 +270,7 @@ const struct mali_c55_isp_fmt *
> > mali_c55_isp_get_mbus_config_by_code(u32 code);
> > const struct mali_c55_isp_fmt *
> > mali_c55_isp_get_mbus_config_by_index(u32 index);
> > +void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
> > + enum mali_c55_config_spaces cfg_space);
> >
> > #endif /* _MALI_C55_COMMON_H */
> > diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> > index dbc07d12d3a3..eedc8f450184 100644
> > --- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> > +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> > @@ -374,6 +374,16 @@ static int mali_c55_create_links(struct mali_c55 *mali_c55)
> > }
> > }
> >
> > + ret = media_create_pad_link(&mali_c55->isp.sd.entity,
> > + MALI_C55_ISP_PAD_SOURCE_STATS,
> > + &mali_c55->stats.vdev.entity, 0,
> > + MEDIA_LNK_FL_ENABLED);
> > + if (ret) {
> > + dev_err(mali_c55->dev,
> > + "failed to link ISP and 3a stats node\n");
> > + goto err_remove_links;
> > + }
> > +
> > return 0;
> >
> > err_remove_links:
> > @@ -388,6 +398,7 @@ static void mali_c55_unregister_entities(struct mali_c55 *mali_c55)
> > mali_c55_unregister_isp(mali_c55);
> > mali_c55_unregister_resizers(mali_c55);
> > mali_c55_unregister_capture_devs(mali_c55);
> > + mali_c55_unregister_stats(mali_c55);
> > }
> >
> > static int mali_c55_register_entities(struct mali_c55 *mali_c55)
> > @@ -410,6 +421,10 @@ static int mali_c55_register_entities(struct mali_c55 *mali_c55)
> > if (ret)
> > goto err_unregister_entities;
> >
> > + ret = mali_c55_register_stats(mali_c55);
> > + if (ret)
> > + goto err_unregister_entities;
> > +
> > ret = mali_c55_create_links(mali_c55);
> > if (ret)
> > goto err_unregister_entities;
> > diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> > index f784983a4ccc..2f450c00300a 100644
> > --- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> > +++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> > @@ -5,6 +5,8 @@
> > * Copyright (C) 2024 Ideas on Board Oy
> > */
> >
> > +#include <linux/media/arm/mali-c55-config.h>
> > +
> > #include <linux/delay.h>
> > #include <linux/iopoll.h>
> > #include <linux/property.h>
> > @@ -460,6 +462,14 @@ static int mali_c55_isp_init_state(struct v4l2_subdev *sd,
> > in_crop->width = MALI_C55_DEFAULT_WIDTH;
> > in_crop->height = MALI_C55_DEFAULT_HEIGHT;
> >
> > + src_fmt = v4l2_subdev_state_get_format(state,
> > + MALI_C55_ISP_PAD_SOURCE_STATS);
> > +
> > + src_fmt->width = sizeof(struct mali_c55_stats_buffer);
> > + src_fmt->height = 1;
>
> According to
> https://docs.kernel.org/userspace-api/media/v4l/subdev-formats.html#metadata-formats,
> width and height should be set to 0 for MEDIA_BUS_FMT_METADATA_FIXED. I
> haven't checked if v4l2-compliance expects this, we may have
> discrepancies between the API documentation and the existing
> implementations in the kernel and userspace. In any case, this should be
> sorted out, either by fixing the kernel code and enforcing the
> requirement in v4l2-compliance, or fixing the documentation.
>
> > + src_fmt->field = V4L2_FIELD_NONE;
> > + src_fmt->code = MEDIA_BUS_FMT_METADATA_FIXED;
> > +
> > return 0;
> > }
> >
> > @@ -490,6 +500,7 @@ int mali_c55_register_isp(struct mali_c55 *mali_c55)
> > MEDIA_PAD_FL_MUST_CONNECT;
> > isp->pads[MALI_C55_ISP_PAD_SOURCE_VIDEO].flags = MEDIA_PAD_FL_SOURCE;
> > isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE;
> > + isp->pads[MALI_C55_ISP_PAD_SOURCE_STATS].flags = MEDIA_PAD_FL_SOURCE;
> >
> > ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS,
> > isp->pads);
> > diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
> > index 0a391f8a043e..e72e749b81d5 100644
> > --- a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
> > +++ b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
> > @@ -103,6 +103,9 @@ enum mali_c55_interrupts {
> > #define MALI_C55_VC_START(v) ((v) & 0xffff)
> > #define MALI_C55_VC_SIZE(v) (((v) & 0xffff) << 16)
> >
> > +#define MALI_C55_REG_1024BIN_HIST 0x054a8
> > +#define MALI_C55_1024BIN_HIST_SIZE 4096
> > +
> > /* Ping/Pong Configuration Space */
> > #define MALI_C55_REG_BASE_ADDR 0x18e88
> > #define MALI_C55_REG_BYPASS_0 0x18eac
> > diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-stats.c b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
> > new file mode 100644
> > index 000000000000..38a17fb5d052
> > --- /dev/null
> > +++ b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
> > @@ -0,0 +1,373 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * ARM Mali-C55 ISP Driver - 3A Statistics capture device
> > + *
> > + * Copyright (C) 2023 Ideas on Board Oy
> > + */
> > +
> > +#include <linux/container_of.h>
> > +#include <linux/dev_printk.h>
> > +#include <linux/dmaengine.h>
> > +#include <linux/list.h>
> > +#include <linux/media/arm/mali-c55-config.h>
> > +#include <linux/mutex.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/string.h>
> > +
> > +#include <media/media-entity.h>
> > +#include <media/v4l2-dev.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-fh.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mali-c55-common.h"
> > +#include "mali-c55-registers.h"
> > +
> > +static const unsigned int metering_space_addrs[] = {
> > + [MALI_C55_CONFIG_PING] = 0x095ac,
> > + [MALI_C55_CONFIG_PONG] = 0x2156c,
> > +};
> > +
> > +static int mali_c55_stats_enum_fmt_meta_cap(struct file *file, void *fh,
> > + struct v4l2_fmtdesc *f)
> > +{
> > + if (f->index)
> > + return -EINVAL;
> > +
> > + f->pixelformat = V4L2_META_FMT_MALI_C55_STATS;
> > +
> > + return 0;
> > +}
> > +
> > +static int mali_c55_stats_g_fmt_meta_cap(struct file *file, void *fh,
> > + struct v4l2_format *f)
> > +{
> > + static const struct v4l2_meta_format mfmt = {
> > + .dataformat = V4L2_META_FMT_MALI_C55_STATS,
> > + .buffersize = sizeof(struct mali_c55_stats_buffer)
> > + };
> > +
> > + f->fmt.meta = mfmt;
> > +
> > + return 0;
> > +}
> > +
> > +static int mali_c55_stats_querycap(struct file *file,
> > + void *priv, struct v4l2_capability *cap)
> > +{
> > + strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver));
> > + strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card));
> > +
> > + return 0;
> > +}
> > +
> > +static const struct v4l2_ioctl_ops mali_c55_stats_v4l2_ioctl_ops = {
> > + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> > + .vidioc_querybuf = vb2_ioctl_querybuf,
> > + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> > + .vidioc_qbuf = vb2_ioctl_qbuf,
> > + .vidioc_expbuf = vb2_ioctl_expbuf,
> > + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> > + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > + .vidioc_streamon = vb2_ioctl_streamon,
> > + .vidioc_streamoff = vb2_ioctl_streamoff,
> > + .vidioc_enum_fmt_meta_cap = mali_c55_stats_enum_fmt_meta_cap,
> > + .vidioc_g_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
> > + .vidioc_s_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
> > + .vidioc_try_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
> > + .vidioc_querycap = mali_c55_stats_querycap,
> > + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> > + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_file_operations mali_c55_stats_v4l2_fops = {
> > + .owner = THIS_MODULE,
> > + .unlocked_ioctl = video_ioctl2,
> > + .open = v4l2_fh_open,
> > + .release = vb2_fop_release,
> > + .poll = vb2_fop_poll,
> > + .mmap = vb2_fop_mmap,
> > +};
> > +
> > +static int
> > +mali_c55_stats_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
> > + unsigned int *num_planes, unsigned int sizes[],
> > + struct device *alloc_devs[])
> > +{
> > + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
> > +
> > + if (*num_planes && *num_planes > 1)
> > + return -EINVAL;
> > +
> > + if (sizes[0] && sizes[0] < sizeof(struct mali_c55_stats_buffer))
> > + return -EINVAL;
> > +
> > + *num_planes = 1;
> > +
> > + if (!sizes[0])
> > + sizes[0] = sizeof(struct mali_c55_stats_buffer);
> > +
> > + if (stats->channel)
> > + alloc_devs[0] = stats->channel->device->dev;
> > +
> > + return 0;
> > +}
> > +
> > +static void mali_c55_stats_buf_queue(struct vb2_buffer *vb)
> > +{
> > + struct mali_c55_stats *stats = vb2_get_drv_priv(vb->vb2_queue);
> > + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > + struct mali_c55_stats_buf *buf = container_of(vbuf,
> > + struct mali_c55_stats_buf, vb);
> > +
> > + vb2_set_plane_payload(vb, 0, sizeof(struct mali_c55_stats_buffer));
> > + buf->segments_remaining = 2;
> > + buf->failed = false;
> > +
> > + spin_lock(&stats->buffers.lock);
> > + list_add_tail(&buf->queue, &stats->buffers.queue);
> > + spin_unlock(&stats->buffers.lock);
> > +}
> > +
> > +static int mali_c55_stats_start_streaming(struct vb2_queue *q,
> > + unsigned int count)
> > +{
> > + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
> > + struct mali_c55 *mali_c55 = stats->mali_c55;
> > + int ret;
> > +
> > + ret = video_device_pipeline_start(&stats->vdev,
> > + &stats->mali_c55->pipe);
> > + if (ret)
> > + return ret;
> > +
> > + if (mali_c55->pipe.start_count == mali_c55->pipe.required_count) {
> > + ret = v4l2_subdev_enable_streams(&mali_c55->isp.sd,
> > + MALI_C55_ISP_PAD_SOURCE_VIDEO,
> > + BIT(0));
> > + if (ret)
> > + goto err_stop_pipeline;
> > + }
> > +
> > + return 0;
> > +
> > +err_stop_pipeline:
> > + video_device_pipeline_stop(&stats->vdev);
> > +
> > + return ret;
> > +}
> > +
> > +static void mali_c55_stats_stop_streaming(struct vb2_queue *q)
> > +{
> > + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
> > + struct mali_c55_stats_buf *buf, *tmp;
> > +
> > + dmaengine_terminate_sync(stats->channel);
> > +
> > + spin_lock(&stats->buffers.lock);
> > +
> > + list_for_each_entry_safe(buf, tmp, &stats->buffers.queue, queue) {
> > + list_del(&buf->queue);
> > + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> > + }
> > +
> > + spin_unlock(&stats->buffers.lock);
> > +
> > + video_device_pipeline_stop(&stats->vdev);
There's a lack of symmetry here, you call v4l2_subdev_enable_streams()
in mali_c55_stats_start_streaming() but you don't call
v4l2_subdev_disable_streams() anywhere. Is that intentional ?
> > +}
> > +
> > +static const struct vb2_ops mali_c55_stats_vb2_ops = {
> > + .queue_setup = mali_c55_stats_queue_setup,
> > + .buf_queue = mali_c55_stats_buf_queue,
> > + .wait_prepare = vb2_ops_wait_prepare,
> > + .wait_finish = vb2_ops_wait_finish,
> > + .start_streaming = mali_c55_stats_start_streaming,
> > + .stop_streaming = mali_c55_stats_stop_streaming,
> > +};
> > +
> > +static void
> > +mali_c55_stats_metering_complete(void *param,
> > + const struct dmaengine_result *result)
> > +{
> > + struct mali_c55_stats_buf *buf = param;
> > +
> > + if (result->result != DMA_TRANS_NOERROR)
> > + buf->failed = true;
> > +
> > + if (!--buf->segments_remaining)
> > + vb2_buffer_done(&buf->vb.vb2_buf, buf->failed ?
> > + VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE);
> > +}
> > +
> > +static int mali_c55_stats_dma_xfer(struct mali_c55_stats *stats, dma_addr_t src,
> > + dma_addr_t dst,
> > + struct mali_c55_stats_buf *buf,
> > + size_t length)
> > +{
> > + struct dma_async_tx_descriptor *tx;
> > + dma_cookie_t cookie;
> > +
> > + tx = dmaengine_prep_dma_memcpy(stats->channel, dst, src, length, 0);
> > + if (!tx) {
> > + dev_err(stats->mali_c55->dev, "failed to prep stats DMA\n");
> > + return -EIO;
> > + }
> > +
> > + tx->callback_result = mali_c55_stats_metering_complete;
> > + tx->callback_param = buf;
> > +
> > + cookie = dmaengine_submit(tx);
> > + if (dma_submit_error(cookie)) {
> > + dev_err(stats->mali_c55->dev, "failed to submit stats DMA\n");
> > + return -EIO;
> > + }
> > +
> > + dma_async_issue_pending(stats->channel);
You could possibly lower the overhead a bit by calling
dma_async_issue_pending() only once after submitting the two transfers.
It may not make any real difference though, I don't recall the details.
> > + return 0;
> > +}
> > +
> > +void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
> > + enum mali_c55_config_spaces cfg_space)
> > +{
> > + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
> > + struct mali_c55_stats *stats = &mali_c55->stats;
> > + struct mali_c55_stats_buf *buf = NULL;
> > + dma_addr_t src, dst;
> > + size_t length;
> > + int ret;
> > +
> > + spin_lock(&stats->buffers.lock);
> > + if (!list_empty(&stats->buffers.queue)) {
> > + buf = list_first_entry(&stats->buffers.queue,
> > + struct mali_c55_stats_buf, queue);
> > + list_del(&buf->queue);
> > + }
> > + spin_unlock(&stats->buffers.lock);
> > +
> > + if (!buf)
> > + return;
> > +
> > + buf->vb.sequence = mali_c55->isp.frame_sequence;
> > + buf->vb.vb2_buf.timestamp = ktime_get_boottime_ns();
> > +
> > + /*
> > + * There are in fact two noncontiguous sections of the ISP's
> > + * memory space that hold statistics for 3a algorithms to use: A
> > + * section in each config space and a global section holding
> > + * histograms which is double buffered and so holds data for the
> > + * last frame. We need to read both.
> > + */
> > + src = ctx->base + MALI_C55_REG_1024BIN_HIST;
> > + dst = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> > +
> > + ret = mali_c55_stats_dma_xfer(stats, src, dst, buf,
> > + MALI_C55_1024BIN_HIST_SIZE);
> > + if (ret)
> > + goto err_fail_buffer;
> > +
> > + src = ctx->base + metering_space_addrs[cfg_space];
> > + dst += MALI_C55_1024BIN_HIST_SIZE;
> > +
> > + length = sizeof(struct mali_c55_stats_buffer) - MALI_C55_1024BIN_HIST_SIZE;
> > + ret = mali_c55_stats_dma_xfer(stats, src, dst, buf, length);
> > + if (ret) {
> > + dmaengine_terminate_sync(stats->channel);
> > + goto err_fail_buffer;
> > + }
> > +
> > + return;
> > +
> > +err_fail_buffer:
> > + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> > +}
> > +
> > +void mali_c55_unregister_stats(struct mali_c55 *mali_c55)
> > +{
> > + struct mali_c55_stats *stats = &mali_c55->stats;
> > +
> > + if (!video_is_registered(&stats->vdev))
> > + return;
> > +
> > + vb2_video_unregister_device(&stats->vdev);
> > + media_entity_cleanup(&stats->vdev.entity);
> > + dma_release_channel(stats->channel);
> > + mutex_destroy(&stats->lock);
> > +}
> > +
> > +int mali_c55_register_stats(struct mali_c55 *mali_c55)
> > +{
> > + struct mali_c55_stats *stats = &mali_c55->stats;
> > + struct video_device *vdev = &stats->vdev;
> > + struct vb2_queue *vb2q = &stats->queue;
> > + dma_cap_mask_t mask;
> > + int ret;
> > +
> > + mutex_init(&stats->lock);
> > + INIT_LIST_HEAD(&stats->buffers.queue);
> > +
> > + dma_cap_zero(mask);
> > + dma_cap_set(DMA_MEMCPY, mask);
> > +
> > + stats->channel = dma_request_channel(mask, 0, NULL);
> > + if (!stats->channel) {
> > + ret = -ENODEV;
> > + goto err_destroy_mutex;
> > + }
> > +
> > + stats->pad.flags = MEDIA_PAD_FL_SINK;
> > + ret = media_entity_pads_init(&stats->vdev.entity, 1, &stats->pad);
> > + if (ret)
> > + goto err_release_dma_channel;
> > +
> > + vb2q->type = V4L2_BUF_TYPE_META_CAPTURE;
> > + vb2q->io_modes = VB2_MMAP | VB2_DMABUF;
> > + vb2q->drv_priv = stats;
> > + vb2q->mem_ops = &vb2_dma_contig_memops;
> > + vb2q->ops = &mali_c55_stats_vb2_ops;
> > + vb2q->buf_struct_size = sizeof(struct mali_c55_stats_buf);
> > + vb2q->min_queued_buffers = 1;
> > + vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > + vb2q->lock = &stats->lock;
> > + vb2q->dev = stats->channel->device->dev;
> > +
> > + ret = vb2_queue_init(vb2q);
> > + if (ret) {
> > + dev_err(mali_c55->dev, "stats vb2 queue init failed\n");
> > + goto err_cleanup_entity;
> > + }
> > +
> > + strscpy(stats->vdev.name, "mali-c55 3a stats", sizeof(stats->vdev.name));
> > + vdev->release = video_device_release_empty;
Not a good idea at all, but I won't ask you to fix object lifetime
management in V4L2 as a prerequisite to merging this driver :-)
With those issues addressed,
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > + vdev->fops = &mali_c55_stats_v4l2_fops;
> > + vdev->ioctl_ops = &mali_c55_stats_v4l2_ioctl_ops;
> > + vdev->lock = &stats->lock;
> > + vdev->v4l2_dev = &mali_c55->v4l2_dev;
> > + vdev->queue = &stats->queue;
> > + vdev->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
> > + vdev->vfl_dir = VFL_DIR_RX;
> > + video_set_drvdata(vdev, stats);
> > +
> > + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> > + if (ret) {
> > + dev_err(mali_c55->dev,
> > + "failed to register stats video device\n");
> > + goto err_release_vb2q;
> > + }
> > +
> > + stats->mali_c55 = mali_c55;
> > +
> > + return 0;
> > +
> > +err_release_vb2q:
> > + vb2_queue_release(vb2q);
> > +err_cleanup_entity:
> > + media_entity_cleanup(&stats->vdev.entity);
> > +err_release_dma_channel:
> > + dma_release_channel(stats->channel);
> > +err_destroy_mutex:
> > + mutex_destroy(&stats->lock);
> > +
> > + return ret;
> > +}
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 16/18] media: uapi: Add parameters structs to mali-c55-config.h
2024-07-09 13:29 ` [PATCH v6 16/18] media: uapi: Add parameters structs to mali-c55-config.h Daniel Scally
@ 2024-07-22 23:48 ` Laurent Pinchart
2024-07-29 10:51 ` Dan Scally
0 siblings, 1 reply; 41+ messages in thread
From: Laurent Pinchart @ 2024-07-22 23:48 UTC (permalink / raw)
To: Daniel Scally
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Dan,
Thank you for the patch.
On Tue, Jul 09, 2024 at 02:29:04PM +0100, Daniel Scally wrote:
> Add structures describing the ISP parameters to mali-c55-config.h
>
> Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
> ---
> Changes in v6:
>
> - Flagged which struct goes with which enum value from
> enum mali_c55_param_block_type
> - Used only types with well defined sizes in the structs
> - Expanded the documentation for skipping in the AEXP histogram config
> - Aligned the header struct to 64 bits
> - Added a new union type to hold pointers to the parameter structs
>
> Changes in v5:
>
> - New patch
>
> .../uapi/linux/media/arm/mali-c55-config.h | 763 ++++++++++++++++++
> 1 file changed, 763 insertions(+)
>
> diff --git a/include/uapi/linux/media/arm/mali-c55-config.h b/include/uapi/linux/media/arm/mali-c55-config.h
> index 21b453bdee5d..ea9872e9076f 100644
> --- a/include/uapi/linux/media/arm/mali-c55-config.h
> +++ b/include/uapi/linux/media/arm/mali-c55-config.h
> @@ -179,4 +179,767 @@ struct mali_c55_stats_buffer {
> __u32 reserved3[15];
> } __attribute__((packed));
>
> +/**
> + * enum mali_c55_param_buffer_version - Mali-C55 parameters block versioning
> + *
> + * @MALI_C55_PARAM_BUFFER_V1: First version of Mali-C55 parameters block
> + */
> +enum mali_c55_param_buffer_version {
> + MALI_C55_PARAM_BUFFER_V1,
> +};
> +
> +/**
> + * enum mali_c55_param_block_type - Enumeration of Mali-C55 parameter blocks
> + *
> + * This enumeration defines the types of Mali-C55 parameters block. Each block
> + * configures a specific processing block of the Mali-C55 ISP. The block
> + * type allows the driver to correctly interpret the parameters block data.
> + *
> + * It is the responsibility of userspace to correctly set the type of each
> + * parameters block.
> + *
> + * @MALI_C55_PARAM_BLOCK_SENSOR_OFFS: Sensor pre-shading black level offset
> + * @MALI_C55_PARAM_BLOCK_AEXP_HIST: Auto-exposure 1024-bin histogram
> + * configuration
> + * @MALI_C55_PARAM_BLOCK_AEXP_IHIST: Post-Iridix auto-exposure 1024-bin
> + * histogram configuration
> + * @MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS: Auto-exposure 1024-bin histogram
> + * weighting
> + * @MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS: Post-Iridix auto-exposure 1024-bin
> + * histogram weighting
> + * @MALI_C55_PARAM_BLOCK_DIGITAL_GAIN: Digital gain
> + * @MALI_C55_PARAM_BLOCK_AWB_GAINS: Auto-white balance gains
> + * @MALI_C55_PARAM_BLOCK_AWB_CONFIG: Auto-white balance statistics config
> + * @MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP: Auto-white balance gains for AEXP-0 tap
> + * @MALI_C55_PARAM_MESH_SHADING_CONFIG : Mesh shading tables configuration
> + * @MALI_C55_PARAM_MESH_SHADING_SELECTION: Mesh shading table selection
> + * @MALI_C55_PARAM_BLOCK_SENTINEL: First non-valid block index
The sentinel value will change when adding new types. That may affect
userspace. We managed to get rid of the sentinel in the rkisp1 driver,
it would be nice to do the same here.
> + */
> +enum mali_c55_param_block_type {
> + MALI_C55_PARAM_BLOCK_SENSOR_OFFS,
> + MALI_C55_PARAM_BLOCK_AEXP_HIST,
> + MALI_C55_PARAM_BLOCK_AEXP_IHIST,
> + MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS,
> + MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS,
> + MALI_C55_PARAM_BLOCK_DIGITAL_GAIN,
> + MALI_C55_PARAM_BLOCK_AWB_GAINS,
> + MALI_C55_PARAM_BLOCK_AWB_CONFIG,
> + MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP,
> + MALI_C55_PARAM_MESH_SHADING_CONFIG,
> + MALI_C55_PARAM_MESH_SHADING_SELECTION,
> + MALI_C55_PARAM_BLOCK_SENTINEL,
> +};
> +
> +/**
> + * struct mali_c55_params_block_header - Mali-C55 parameter block header
> + *
> + * This structure represents the common part of all the ISP configuration
> + * blocks. Each parameters block embeds an instance of this structure type
> + * as its first member, followed by the block-specific configuration data. The
> + * driver inspects this common header to discern the block type and its size and
> + * properly handle the block content by casting it to the correct block-specific
> + * type.
> + *
> + * The @type field is one of the values enumerated by
> + * :c:type:`mali_c55_param_block_type` and specifies how the data should be
> + * interpreted by the driver. The @size field specifies the size of the
> + * parameters block and is used by the driver for validation purposes. The
> + * @enabled field specifies if the ISP block should be enabled (and configured
> + * according to the provided parameters) or disabled.
> + *
> + * Userspace is responsible for correctly populating the parameters block header
> + * fields (@type, @enabled and @size) and correctly populate the block-specific
> + * parameters.
> + *
> + * For example:
> + *
> + * .. code-block:: c
> + *
> + * void populate_sensor_offs(struct mali_c55_params_block_header *block) {
> + * block->type = MALI_C55_PARAM_BLOCK_SENSOR_OFFS;
> + * block->enabled = true;
> + * block->size = sizeof(struct mali_c55_params_sensor_off_preshading);
> + *
> + * struct mali_c55_params_sensor_off_preshading *sensor_offs =
> + * (struct mali_c55_params_sensor_off_preshading *)block;
> + *
> + * sensor_offs->chan00 = offset00;
> + * sensor_offs->chan01 = offset01;
> + * sensor_offs->chan10 = offset10;
> + * sensor_offs->chan11 = offset11;
> + * }
> + *
> + * @type: The parameters block type from :c:type:`mali_c55_param_block_type`
> + * @enabled: Block enabled/disabled flag
> + * @size: Size (in bytes) of the parameters block
> + */
> +struct mali_c55_params_block_header {
> + __u8 type;
> + __u8 enabled;
> + __u32 size;
> +} __attribute__((aligned(8)));
> +
> +/**
> + * struct mali_c55_params_sensor_off_preshading - offset subtraction for each
> + * color channel
> + *
> + * Provides removal of the sensor black level from the sensor data. Separate
> + * offsets are provided for each of the four Bayer component color channels
> + * which are defaulted to R, Gr, Gb, B.
> + *
> + * header.type should be set to MALI_C55_PARAM_BLOCK_SENSOR_OFFS from
> + * :c:type:`mali_c55_param_block_type` for this block.
> + *
> + * @header: The Mali-C55 parameters block header
> + * @chan00: Offset for color channel 00 (default: R)
> + * @chan01: Offset for color channel 01 (default: Gr)
> + * @chan10: Offset for color channel 10 (default: Gb)
> + * @chan11: Offset for color channel 11 (default: B)
> + */
> +struct mali_c55_params_sensor_off_preshading {
> + struct mali_c55_params_block_header header;
> + __u32 chan00;
> + __u32 chan01;
> + __u32 chan10;
> + __u32 chan11;
> +};
> +
> +/**
> + * enum mali_c55_aexp_hist_tap_points - Tap points for the AEXP histogram
> + * @MALI_C55_AEXP_HIST_TAP_WB: After static white balance
> + * @MALI_C55_AEXP_HIST_TAP_FS: After WDR Frame Stitch
> + * @MALI_C55_AEXP_HIST_TAP_TPG: After the test pattern generator
> + */
> +enum mali_c55_aexp_hist_tap_points {
> + MALI_C55_AEXP_HIST_TAP_WB = 0,
> + MALI_C55_AEXP_HIST_TAP_FS,
> + MALI_C55_AEXP_HIST_TAP_TPG,
> +};
> +
> +/**
> + * enum mali_c55_aexp_skip_x - Horizontal pixel skipping
> + * @MALI_C55_AEXP_SKIP_X_EVERY_2ND: Collect every 2nd pixel horizontally
> + * @MALI_C55_AEXP_SKIP_X_EVERY_3RD: Collect every 3rd pixel horizontally
> + * @MALI_C55_AEXP_SKIP_X_EVERY_4TH: Collect every 4th pixel horizontally
> + * @MALI_C55_AEXP_SKIP_X_EVERY_5TH: Collect every 5th pixel horizontally
> + * @MALI_C55_AEXP_SKIP_X_EVERY_8TH: Collect every 8th pixel horizontally
> + * @MALI_C55_AEXP_SKIP_X_EVERY_9TH: Collect every 9th pixel horizontally
> + */
> +enum mali_c55_aexp_skip_x {
> + MALI_C55_AEXP_SKIP_X_EVERY_2ND,
> + MALI_C55_AEXP_SKIP_X_EVERY_3RD,
> + MALI_C55_AEXP_SKIP_X_EVERY_4TH,
> + MALI_C55_AEXP_SKIP_X_EVERY_5TH,
> + MALI_C55_AEXP_SKIP_X_EVERY_8TH,
> + MALI_C55_AEXP_SKIP_X_EVERY_9TH
> +};
> +
> +/**
> + * enum mali_c55_aexp_skip_y - Vertical pixel skipping
> + * @MALI_C55_AEXP_SKIP_Y_ALL: Collect every single pixel vertically
> + * @MALI_C55_AEXP_SKIP_Y_EVERY_2ND: Collect every 2nd pixel vertically
> + * @MALI_C55_AEXP_SKIP_Y_EVERY_3RD: Collect every 3rd pixel vertically
> + * @MALI_C55_AEXP_SKIP_Y_EVERY_4TH: Collect every 4th pixel vertically
> + * @MALI_C55_AEXP_SKIP_Y_EVERY_5TH: Collect every 5th pixel vertically
> + * @MALI_C55_AEXP_SKIP_Y_EVERY_8TH: Collect every 8th pixel vertically
> + * @MALI_C55_AEXP_SKIP_Y_EVERY_9TH: Collect every 9th pixel vertically
> + */
> +enum mali_c55_aexp_skip_y {
> + MALI_C55_AEXP_SKIP_Y_ALL,
> + MALI_C55_AEXP_SKIP_Y_EVERY_2ND,
> + MALI_C55_AEXP_SKIP_Y_EVERY_3RD,
> + MALI_C55_AEXP_SKIP_Y_EVERY_4TH,
> + MALI_C55_AEXP_SKIP_Y_EVERY_5TH,
> + MALI_C55_AEXP_SKIP_Y_EVERY_8TH,
> + MALI_C55_AEXP_SKIP_Y_EVERY_9TH
> +};
> +
> +/**
> + * enum mali_c55_aexp_row_column_offset - Start from the first or second row or
> + * column
> + * @MALI_C55_AEXP_FIRST_ROW_OR_COL: Start from the first row / column
> + * @MALI_C55_AEXP_SECOND_ROW_OR_COL: Start from the second row / column
> + */
> +enum mali_c55_aexp_row_column_offset {
> + MALI_C55_AEXP_FIRST_ROW_OR_COL = 1,
> + MALI_C55_AEXP_SECOND_ROW_OR_COL = 2,
> +};
> +
> +/**
> + * enum mali_c55_aexp_hist_plane_mode - Mode for the AEXP Histograms
> + * @MALI_C55_AEXP_HIST_COMBINED: All color planes in one 1024-bin histogram
> + * @MALI_C55_AEXP_HIST_SEPARATE: Each color plane in one 256-bin histogram with a bin width of 16
> + * @MALI_C55_AEXP_HIST_FOCUS_00: Top left plane in the first bank, rest in second bank
> + * @MALI_C55_AEXP_HIST_FOCUS_01: Top right plane in the first bank, rest in second bank
> + * @MALI_C55_AEXP_HIST_FOCUS_10: Bottom left plane in the first bank, rest in second bank
> + * @MALI_C55_AEXP_HIST_FOCUS_11: Bottom right plane in the first bank, rest in second bank
> + *
> + * In the "focus" modes statistics are collected into two 512-bin histograms
> + * with a bin width of 8. One colour plane is in the first histogram with the
> + * remainder combined into the second. The four options represent which of the
> + * four positions in a bayer pattern are the focused plane.
> + */
> +enum mali_c55_aexp_hist_plane_mode {
> + MALI_C55_AEXP_HIST_COMBINED = 0,
> + MALI_C55_AEXP_HIST_SEPARATE = 1,
> + MALI_C55_AEXP_HIST_FOCUS_00 = 4,
> + MALI_C55_AEXP_HIST_FOCUS_01 = 5,
> + MALI_C55_AEXP_HIST_FOCUS_10 = 6,
> + MALI_C55_AEXP_HIST_FOCUS_11 = 7,
> +};
> +
> +/**
> + * struct mali_c55_params_aexp_hist - configuration for AEXP metering hists
> + *
> + * This struct allows users to configure the 1024-bin AEXP histograms. Broadly
> + * speaking the parameters allow you to mask particular regions of the image and
> + * to select different kinds of histogram.
> + *
> + * The skip_x, offset_x, skip_y and offset_y fields allow users to ignore or
> + * mask pixels in the frame by their position relative to the top left pixel.
> + * First, the skip_y, offset_x and offset_y fields define a 2x2 pixel pattern of
> + * which pixels covered by the pattern will be counted in the statistics.
I have a really hard time parsing the second sentence. Let me try to
rephrase it.
* First, the skip_y, offset_x and offset_y fields define which pixels will be
* counted in the statistics within a 2x2 pixels region.
Is that what you meant ?
> + *
> + * If skip_y == 0 then two pixels from each coveredregion will be counted. If
s/coveredregion/region/
> + * both offset_x and offset_y are zero, then the two left-most pixels in each
> + * 2x2 pixel region covered by the pattern will be counted. Setting offset_x = 1
I'd drop "covered by the pattern" as I'm not sure what that means.
> + * will discount the top left pixel and count the top right pixel. Setting
> + * offset_y = 1 will discount the bottom left pixel and count the bottom right
> + * pixel.
That sounds strange, I would have expected offset_y to control something
vertically, not control the horizontal position on the second line. If
that's how the hardware operates, why not.
> + *
> + * If skip_y != 0 then only a single pixel from each region covered by the
> + * pattern will be counted. In this case offset_x controls whether the pixel
> + * that's counted is in the left (if offset_x == 0) or right (if offset_x == 1)
> + * column and offset_y controls whether the pixel that's counted is in the top
> + * (if offset_y == 0) or bottom (if offset_y == 1) row.
> + *
> + * The skip_x and skip_y fields control how the pixel masking pattern is
> + * repeated across the image data. The first instance of the pattern is always
And here maybe "control how the 2x2 pixel region is repeated ...".
s/pattern/region/ in the rest.
This is otherwise a great explanation, thank you.
> + * in the top left of the image data. The skip_x field controls how many pixels
> + * are ignored in the x direction before the pixel masking pattern is repeated.
> + * The skip_y field controls how many pixels are ignored in the y direction
> + * before the pixel masking pattern is repeated.
> + *
> + * These fields can be used to reduce the number of pixels counted for the
> + * statistics, but it's important to be careful to configure them correctly.
> + * Some combinations of values will result in colour components from the input
> + * data being ignored entirely, for example in the following configuration:
> + *
> + * skip_x = 0
> + * offset_x = 0
> + * skip_y = 0
> + * offset_y = 0
> + *
> + * Only the R and Gb components of RGGB data that was input would be collected.
> + * Similarly in the following configuration:
> + *
> + * skip_x = 0
> + * offset_x = 0
> + * skip_y = 1
> + * offset_y = 1
> + *
> + * Only the Gb component of RGGB data that was input would be collected. To
> + * correct things such that all 4 colour components were included it would be
> + * necessary to set the skip_x and skip_y fields in a way that resulted in all
> + * four colour components being collected:
> + *
> + * skip_x = 1
> + * offset_x = 0
> + * skip_y = 1
> + * offset_y = 1
> + *
> + * header.type should be set to one of either MALI_C55_PARAM_BLOCK_AEXP_HIST or
> + * MALI_C55_PARAM_BLOCK_AEXP_IHIST from :c:type:`mali_c55_param_block_type`.
> + *
> + * @header: The Mali-C55 parameters block header
> + * @skip_x: Horizontal decimation. See enum mali_c55_aexp_skip_x
> + * @offset_x: Skip the first column, or not. See enum mali_c55_aexp_row_column_offset
> + * @skip_y: Vertical decimation. See enum mali_c55_aexp_skip_y
> + * @offset_y: Skip the first row, or not. See enum mali_c55_aexp_row_column_offset
> + * @scale_bottom: Scale pixels in bottom half of intensity range: 0=1x ,1=2x, 2=4x, 4=8x, 4=16x
> + * @scale_top: scale pixels in top half of intensity range: 0=1x ,1=2x, 2=4x, 4=8x, 4=16x
> + * @plane_mode: Plane separation mode. See enum mali_c55_aexp_hist_plane_mode
> + * @tap_point: Tap point for histogram from enum mali_c55_aexp_hist_tap_points.
> + * This parameter is unused for the post-Iridix Histogram
> + */
> +struct mali_c55_params_aexp_hist {
> + struct mali_c55_params_block_header header;
> + __u8 skip_x;
> + __u8 offset_x;
> + __u8 skip_y;
> + __u8 offset_y;
> + __u8 scale_bottom;
> + __u8 scale_top;
> + __u8 plane_mode;
> + __u8 tap_point;
> +};
> +
> +/**
> + * struct mali_c55_params_aexp_weights - Array of weights for AEXP metering
> + *
> + * This struct allows users to configure the weighting for both of the 1024-bin
> + * AEXP histograms. The pixel data collected for each zone is multiplied by the
> + * corresponding weight from this array, which may be zero if the intention is
> + * to mask off the zone entirely.
> + *
> + * header.type should be set to one of either MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS
> + * or MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS from :c:type:`mali_c55_param_block_type`.
> + *
> + * @header: The Mali-C55 parameters block header
> + * @nodes_used_horiz: Number of active zones horizontally [0..15]
> + * @nodes_used_vert: Number of active zones vertically [0..15]
> + * @zone_weights: Zone weighting. Index is row*col where 0,0 is the top
> + * left zone continuing in raster order. Each zone can be
> + * weighted in the range [0..15]. The number of rows and
> + * columns is defined by @nodes_used_vert and
> + * @nodes_used_horiz
> + */
> +struct mali_c55_params_aexp_weights {
> + struct mali_c55_params_block_header header;
> + __u8 nodes_used_horiz;
> + __u8 nodes_used_vert;
> + __u8 zone_weights[MALI_C55_MAX_ZONES];
> +};
> +
> +/**
> + * struct mali_c55_params_digital_gain - Digital gain value
> + *
> + * This struct carries a digital gain value to set in the ISP.
> + *
> + * header.type should be set to MALI_C55_PARAM_BLOCK_DIGITAL_GAIN from
> + * :c:type:`mali_c55_param_block_type` for this block.
> + *
> + * @header: The Mali-C55 parameters block header
> + * @gain: The digital gain value to apply, in Q5.8 format.
> + */
> +struct mali_c55_params_digital_gain {
> + struct mali_c55_params_block_header header;
> + __u16 gain;
> +};
> +
> +/**
> + * enum mali_c55_awb_stats_mode - Statistics mode for AWB
> + * @MALI_C55_AWB_MODE_GRBR: Statistics collected as Green/Red and Blue/Red ratios
> + * @MALI_C55_AWB_MODE_RGBG: Statistics collected as Red/Green and Blue/Green ratios
> + */
> +enum mali_c55_awb_stats_mode {
> + MALI_C55_AWB_MODE_GRBR = 0,
> + MALI_C55_AWB_MODE_RGBG,
> +};
> +
> +/**
> + * struct mali_c55_params_awb_gains - Gain settings for auto white balance
> + *
> + * This struct allows users to configure the gains for auto-white balance. There
> + * are four gain settings corresponding to each colour channel in the bayer
> + * domain. Although named generically, the association between the gain applied
> + * and the colour channel is done automatically within the ISP depending on the
> + * input format, and so the following mapping always holds true::
> + *
> + * gain00 = R
> + * gain01 = Gr
> + * gain10 = Gb
> + * gain11 = B
> + *
> + * All of the gains are stored in Q4.8 format.
> + *
> + * header.type should be set to one of either MALI_C55_PARAM_BLOCK_AWB_GAINS or
> + * MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP from :c:type:`mali_c55_param_block_type`.
> + *
> + * @header: The Mali-C55 parameters block header
> + * @gain00: Multiplier for colour channel 00
> + * @gain01: Multiplier for colour channel 01
> + * @gain10: Multiplier for colour channel 10
> + * @gain11: Multiplier for colour channel 11
> + */
> +struct mali_c55_params_awb_gains {
> + struct mali_c55_params_block_header header;
> + __u16 gain00;
> + __u16 gain01;
> + __u16 gain10;
> + __u16 gain11;
> +};
> +
> +/**
> + * enum mali_c55_params_awb_tap_points - Tap points for the AWB statistics
> + * @MALI_C55_AWB_STATS_TAP_PF: Immediately after the Purple Fringe block
> + * @MALI_C55_AWB_STATS_TAP_CNR: Immediately after the CNR block
> + */
> +enum mali_c55_params_awb_tap_points {
> + MALI_C55_AWB_STATS_TAP_PF = 0,
> + MALI_C55_AWB_STATS_TAP_CNR,
> +};
> +
> +/**
> + * struct mali_c55_params_awb_config - Stats settings for auto-white balance
> + *
> + * This struct allows the configuration of the statistics generated for auto
> + * white balance. Pixel intensity limits can be set to exclude overly bright or
> + * dark regions of an image from the statistics entirely. Colour ratio minima
> + * and maxima can be set to discount pixels who's ratios fall outside the
> + * defined boundaries; there are two sets of registers to do this - the
> + * "min/max" ratios which bound a region and the "high/low" ratios which further
> + * trim the upper and lower ratios. For example with the boundaries configured
> + * as follows, only pixels whos colour ratios falls into the region marked "A"
> + * would be counted::
> + *
> + * cr_high
> + * 2.0 | |
> + * | cb_max --> _________________________v_____
> + * 1.8 | | \ |
> + * | | \ |
> + * 1.6 | | \ |
> + * | | \ |
> + * c 1.4 | cb_low -->|\ A \|<-- cb_high
> + * b | | \ |
> + * 1.2 | | \ |
> + * r | | \ |
> + * a 1.0 | cb_min --> |____\_________________________|
> + * t | ^ ^ ^
> + * i 0.8 | | | |
> + * o | cr_min | cr_max
> + * s 0.6 | |
> + * | cr_low
> + * 0.4 |
> + * |
> + * 0.2 |
> + * |
> + * 0.0 |_______________________________________________________________
> + * 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0
> + * cr ratios
> + *
> + * header.type should be set to MALI_C55_PARAM_BLOCK_AWB_CONFIG from
> + * :c:type:`mali_c55_param_block_type` for this block.
> + *
> + * @header: The Mali-C55 parameters block header
> + * @tap_point: The tap point from enum mali_c55_params_awb_tap_points
> + * @stats_mode: AWB statistics collection mode, see :c:type:`mali_c55_awb_stats_mode`
> + * @white_level: Upper pixel intensity (I.E. raw pixel values) limit
> + * @black_level: Lower pixel intensity (I.E. raw pixel values) limit
> + * @cr_max: Maximum R/G ratio (Q4.8 format)
> + * @cr_min: Minimum R/G ratio (Q4.8 format)
> + * @cb_max: Maximum B/G ratio (Q4.8 format)
> + * @cb_min: Minimum B/G ratio (Q4.8 format)
> + * @nodes_used_horiz: Number of active zones horizontally [0..15]
> + * @nodes_used_vert: Number of active zones vertically [0..15]
> + * @cr_high: R/G ratio trim high (Q4.8 format)
> + * @cr_low: R/G ratio trim low (Q4.8 format)
> + * @cb_high: B/G ratio trim high (Q4.8 format)
> + * @cb_low: B/G ratio trim low (Q4.8 format)
> + */
> +struct mali_c55_params_awb_config {
> + struct mali_c55_params_block_header header;
> + __u8 tap_point;
> + __u8 stats_mode;
> + __u16 white_level;
> + __u16 black_level;
> + __u16 cr_max;
> + __u16 cr_min;
> + __u16 cb_max;
> + __u16 cb_min;
> + __u8 nodes_used_horiz;
> + __u8 nodes_used_vert;
> + __u16 cr_high;
> + __u16 cr_low;
> + __u16 cb_high;
> + __u16 cb_low;
> +};
> +
> +#define MALI_C55_NUM_MESH_SHADING_ELEMENTS 3072
> +
> +/**
> + * struct mali_c55_params_mesh_shading_config - Mesh shading configuration
> + *
> + * The mesh shading correction module allows programming a separate table of
> + * either 16x16 or 32x32 node coefficients for 3 different light sources. The
> + * final correction coefficients applied are computed by blending the
> + * coefficients from two tables together.
> + *
> + * A page of 1024 32-bit integers is associated to each colour channel, with
> + * pages stored consecutively in memory. Each 32-bit integer packs 3 8-bit
> + * correction coefficients for a single node, one for each of the three light
> + * sources. The 8 most significant bits are unused. The following table
> + * describes the layout::
> + *
> + * +----------- Page (Colour Plane) 0 -------------+
> + * | @mesh[i] | Mesh Point | Bits | Light Source |
> + * +-----------+------------+-------+--------------+
> + * | 0 | 0,0 | 16,23 | LS2 |
> + * | | | 08-15 | LS1 |
> + * | | | 00-07 | LS0 |
> + * +-----------+------------+-------+--------------+
> + * | 1 | 0,1 | 16,23 | LS2 |
> + * | | | 08-15 | LS1 |
> + * | | | 00-07 | LS0 |
> + * +-----------+------------+-------+--------------+
> + * | ... | ... | ... | ... |
> + * +-----------+------------+-------+--------------+
> + * | 1023 | 31,31 | 16,23 | LS2 |
> + * | | | 08-15 | LS1 |
> + * | | | 00-07 | LS0 |
> + * +----------- Page (Colour Plane) 1 -------------+
> + * | @mesh[i] | Mesh Point | Bits | Light Source |
> + * +-----------+------------+-------+--------------+
> + * | 1024 | 0,0 | 16,23 | LS2 |
> + * | | | 08-15 | LS1 |
> + * | | | 00-07 | LS0 |
> + * +-----------+------------+-------+--------------+
> + * | 1025 | 0,1 | 16,23 | LS2 |
> + * | | | 08-15 | LS1 |
> + * | | | 00-07 | LS0 |
> + * +-----------+------------+-------+--------------+
> + * | ... | ... | ... | ... |
> + * +-----------+------------+-------+--------------+
> + * | 2047 | 31,31 | 16,23 | LS2 |
> + * | | | 08-15 | LS1 |
> + * | | | 00-07 | LS0 |
> + * +----------- Page (Colour Plane) 2 -------------+
> + * | @mesh[i] | Mesh Point | Bits | Light Source |
> + * +-----------+------------+-------+--------------+
> + * | 2048 | 0,0 | 16,23 | LS2 |
> + * | | | 08-15 | LS1 |
> + * | | | 00-07 | LS0 |
> + * +-----------+------------+-------+--------------+
> + * | 2049 | 0,1 | 16,23 | LS2 |
> + * | | | 08-15 | LS1 |
> + * | | | 00-07 | LS0 |
> + * +-----------+------------+-------+--------------+
> + * | ... | ... | ... | ... |
> + * +-----------+------------+-------+--------------+
> + * | 3071 | 31,31 | 16,23 | LS2 |
> + * | | | 08-15 | LS1 |
> + * | | | 00-07 | LS0 |
> + * +-----------+------------+-------+--------------+
> + *
> + * The @mesh_scale member determines the precision and minimum and maximum gain.
> + * For example if @mesh_scale is 0 and therefore selects 0 - 2x gain, a value of
> + * 0 in a coefficient means 0.0 gain, a value of 128 means 1.0 gain and 255
> + * means 2.0 gain.
> + *
> + * header.type should be set to MALI_C55_PARAM_MESH_SHADING_CONFIG from
> + * :c:type:`mali_c55_param_block_type` for this block.
> + *
> + * @header: The Mali-C55 parameters block header
> + * @mesh_show: Output the mesh data rather than image data
> + * @mesh_scale: Set the precision and maximum gain range of mesh shading
> + * - 0 = 0-2x gain
> + * - 1 = 0-4x gain
> + * - 2 = 0-8x gain
> + * - 3 = 0-16x gain
> + * - 4 = 1-2x gain
> + * - 5 = 1-3x gain
> + * - 6 = 1-5x gain
> + * - 7 = 1-9x gain
> + * @mesh_page_r: Mesh page select for red colour plane [0..2]
> + * @mesh_page_g: Mesh page select for green colour plane [0..2]
> + * @mesh_page_b: Mesh page select for blue colour plane [0..2]
> + * @mesh_width: Number of horizontal nodes minus 1 [15,31]
> + * @mesh_height: Number of vertical nodes minus 1 [15,31]
> + * @mesh: Mesh shading correction tables
> + */
> +struct mali_c55_params_mesh_shading_config {
> + struct mali_c55_params_block_header header;
> + __u8 mesh_show;
> + __u8 mesh_scale;
> + __u8 mesh_page_r;
> + __u8 mesh_page_g;
> + __u8 mesh_page_b;
> + __u8 mesh_width;
> + __u8 mesh_height;
> + __u32 mesh[MALI_C55_NUM_MESH_SHADING_ELEMENTS];
> +};
> +
> +/** enum mali_c55_params_mesh_alpha_bank - Mesh shading table bank selection
> + * @MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS1 - Select Light Sources 0 and 1
> + * @MALI_C55_MESH_ALPHA_BANK_LS1_AND_LS2 - Select Light Sources 1 and 2
> + * @MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS2 - Select Light Sources 0 and 2
> + */
> +enum mali_c55_params_mesh_alpha_bank {
> + MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS1 = 0,
> + MALI_C55_MESH_ALPHA_BANK_LS1_AND_LS2 = 1,
> + MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS2 = 4
> +};
> +
> +/**
> + * struct mali_c55_params_mesh_shading_selection - Mesh table selection
> + *
> + * The module computes the final correction coefficients by blending the ones
> + * from two light source tables, which are selected (independently for each
> + * colour channel) by the @mesh_alpha_bank_r/g/b fields.
> + *
> + * The final blended coefficients for each node are calculated using the
> + * following equation:
> + *
> + * Final coefficient = (a * LS\ :sub:`b`\ + (256 - a) * LS\ :sub:`a`\) / 256
> + *
> + * Where a is the @mesh_alpha_r/g/b value, and LS\ :sub:`a`\ and LS\ :sub:`b`\
> + * are the node cofficients for the two tables selected by the
> + * @mesh_alpha_bank_r/g/b value.
> + *
> + * The scale of the applied correction may also be controlled by tuning the
> + * @mesh_strength member. This is a modifier to the final coefficients which can
> + * be used to globally reduce the gains applied.
> + *
> + * header.type should be set to MALI_C55_PARAM_MESH_SHADING_SELECTION from
> + * :c:type:`mali_c55_param_block_type` for this block.
> + *
> + * @header: The Mali-C55 parameters block header
> + * @mesh_alpha_bank_r: Red mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`)
> + * @mesh_alpha_bank_g: Green mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`)
> + * @mesh_alpha_bank_b: Blue mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`)
> + * @mesh_alpha_r: Blend coefficient for R [0..255]
> + * @mesh_alpha_g: Blend coefficient for G [0..255]
> + * @mesh_alpha_b: Blend coefficient for B [0..255]
> + * @mesh_strength: Mesh strength in Q4.12 format [0..4096]
> + */
> +struct mali_c55_params_mesh_shading_selection {
> + struct mali_c55_params_block_header header;
> + __u8 mesh_alpha_bank_r;
> + __u8 mesh_alpha_bank_g;
> + __u8 mesh_alpha_bank_b;
> + __u8 mesh_alpha_r;
> + __u8 mesh_alpha_g;
> + __u8 mesh_alpha_b;
> + __u16 mesh_strength;
> +};
> +
> +/**
> + * union mali_c55_params_block - Generalisation of a parameter block
> + *
> + * This union allows the driver to treat a block as a generic pointer to this
> + * union and safely access the header and block-specific struct without having
> + * to resort to casting. The header member is accessed first, and the type field
If this is for driver usage only, I would move the structure to the
driver. Let's minimize the UAPI surface.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> + * checked which allows the driver to determine which of the other members
> + * should be used. The data member at the end allows a pointer to an address
> + * within the data member of :c:type:`mali_c55_params_buffer` to initialise a
> + * union variable.
> + *
> + * @header: Pointer to the shared header struct embedded as the
> + * first member of all the possible other members (except
> + * @data). This member would be accessed first and the type
> + * field checked to determine which of the other members
> + * should be accessed.
> + * @sensor_offs: For header->type == MALI_C55_PARAM_BLOCK_SENSOR_OFFS
> + * @aexp_hist: For header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST and
> + * header->type == MALI_C55_PARAM_BLOCK_AEXP_IHIST
> + * @aexp_weights: For header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS
> + * and header->type = MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS
> + * @digital_gain: For header->type == MALI_C55_PARAM_BLOCK_DIGITAL_GAIN
> + * @awb_gains: For header->type == MALI_C55_PARAM_BLOCK_AWB_GAINS and
> + * header->type = MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP
> + * @awb_config: For header->type == MALI_C55_PARAM_MESH_SHADING_CONFIG
> + * @shading_config: For header->type == MALI_C55_PARAM_MESH_SHADING_SELECTION
> + * @shading_selection: For header->type == MALI_C55_PARAM_BLOCK_SENSOR_OFFS
> + * @data: Allows easy initialisation of a union variable with a
> + * pointer into a __u8 array.
> + */
> +union mali_c55_params_block {
> + struct mali_c55_params_block_header *header;
> + struct mali_c55_params_sensor_off_preshading *sensor_offs;
> + struct mali_c55_params_aexp_hist *aexp_hist;
> + struct mali_c55_params_aexp_weights *aexp_weights;
> + struct mali_c55_params_digital_gain *digital_gain;
> + struct mali_c55_params_awb_gains *awb_gains;
> + struct mali_c55_params_awb_config *awb_config;
> + struct mali_c55_params_mesh_shading_config *shading_config;
> + struct mali_c55_params_mesh_shading_selection *shading_selection;
> + __u8 *data;
> +};
> +
> +/**
> + * define MALI_C55_PARAMS_MAX_SIZE - Maximum size of all Mali C55 Parameters
> + *
> + * Though the parameters for the Mali-C55 are passed as optional blocks, the
> + * driver still needs to know the absolute maximum size so that it can allocate
> + * a buffer sized appropriately to accommodate userspace attempting to set all
> + * possible parameters in a single frame.
> + *
> + * Some structs are in this list multiple times. Where that's the case, it just
> + * reflects the fact that the same struct can be used with multiple different
> + * header types from :c:type:`mali_c55_param_block_type`.
> + */
> +#define MALI_C55_PARAMS_MAX_SIZE \
> + (sizeof(struct mali_c55_params_sensor_off_preshading) + \
> + sizeof(struct mali_c55_params_aexp_hist) + \
> + sizeof(struct mali_c55_params_aexp_weights) + \
> + sizeof(struct mali_c55_params_aexp_hist) + \
> + sizeof(struct mali_c55_params_aexp_weights) + \
> + sizeof(struct mali_c55_params_digital_gain) + \
> + sizeof(struct mali_c55_params_awb_gains) + \
> + sizeof(struct mali_c55_params_awb_config) + \
> + sizeof(struct mali_c55_params_awb_gains) + \
> + sizeof(struct mali_c55_params_mesh_shading_config) + \
> + sizeof(struct mali_c55_params_mesh_shading_selection))
> +
> +/**
> + * struct mali_c55_params_buffer - 3A configuration parameters
> + *
> + * This struct contains the configuration parameters of the Mali-C55 ISP
> + * algorithms, serialized by userspace into a data buffer. Each configuration
> + * parameter block is represented by a block-specific structure which contains a
> + * :c:type:`mali_c55_params_block_header` entry as first member. Userspace
> + * populates the @data buffer with configuration parameters for the blocks that
> + * it intends to configure. As a consequence, the data buffer effective size
> + * changes according to the number of ISP blocks that userspace intends to
> + * configure.
> + *
> + * The parameters buffer is versioned by the @version field to allow modifying
> + * and extending its definition. Userspace shall populate the @version field to
> + * inform the driver about the version it intends to use. The driver will parse
> + * and handle the @data buffer according to the data layout specific to the
> + * indicated version and return an error if the desired version is not
> + * supported.
> + *
> + * For each ISP block that userspace wants to configure, a block-specific
> + * structure is appended to the @data buffer, one after the other without gaps
> + * in between nor overlaps. Userspace shall populate the @total_size field with
> + * the effective size, in bytes, of the @data buffer.
> + *
> + * The expected memory layout of the parameters buffer is::
> + *
> + * +-------------------- struct mali_c55_params_buffer ------------------+
> + * | version = MALI_C55_PARAM_BUFFER_V1; |
> + * | total_size = sizeof(struct mali_c55_params_sensor_off_preshading) |
> + * | sizeof(struct mali_c55_params_aexp_hist); |
> + * | +------------------------- data ---------------------------------+ |
> + * | | +--------- struct mali_c55_params_sensor_off_preshading ------+ | |
> + * | | | +-------- struct mali_c55_params_block_header header -----+ | | |
> + * | | | | type = MALI_C55_PARAM_BLOCK_SENSOR_OFFS; | | | |
> + * | | | | enabled = 1; | | | |
> + * | | | | size = | | | |
> + * | | | | sizeof(struct mali_c55_params_sensor_off_preshading);| | | |
> + * | | | +---------------------------------------------------------+ | | |
> + * | | | chan00 = ...; | | |
> + * | | | chan01 = ...; | | |
> + * | | | chan10 = ...; | | |
> + * | | | chan11 = ...; | | |
> + * | | +------------ struct mali_c55_params_aexp_hist ---------------+ | |
> + * | | | +-------- struct mali_c55_params_block_header header -----+ | | |
> + * | | | | type = MALI_C55_PARAM_BLOCK_AEXP_HIST; | | | |
> + * | | | | enabled = 1; | | | |
> + * | | | | size = sizeof(struct mali_c55_params_aexp_hist); | | | |
> + * | | | +---------------------------------------------------------+ | | |
> + * | | | skip_x = ...; | | |
> + * | | | offset_x = ...; | | |
> + * | | | skip_y = ...; | | |
> + * | | | offset_y = ...; | | |
> + * | | | scale_bottom = ...; | | |
> + * | | | scale_top = ...; | | |
> + * | | | plane_mode = ...; | | |
> + * | | | tap_point = ...; | | |
> + * | | +-------------------------------------------------------------+ | |
> + * | +-----------------------------------------------------------------+ |
> + * +---------------------------------------------------------------------+
> + *
> + * @version: The version from :c:type:`mali_c55_param_buffer_version`
> + * @total_size: The Mali-C55 configuration data effective size, excluding this
> + * header
> + * @data: The Mali-C55 configuration blocks data
> + */
> +struct mali_c55_params_buffer {
> + __u8 version;
> + __u32 total_size;
> + __u8 data[MALI_C55_PARAMS_MAX_SIZE];
> +};
> +
> #endif /* __UAPI_MALI_C55_CONFIG_H */
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 16/18] media: uapi: Add parameters structs to mali-c55-config.h
2024-07-22 23:48 ` Laurent Pinchart
@ 2024-07-29 10:51 ` Dan Scally
2024-07-29 11:03 ` Laurent Pinchart
0 siblings, 1 reply; 41+ messages in thread
From: Dan Scally @ 2024-07-29 10:51 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Laurent - thanks for the review
On 23/07/2024 00:48, Laurent Pinchart wrote:
> Hi Dan,
>
> Thank you for the patch.
>
> On Tue, Jul 09, 2024 at 02:29:04PM +0100, Daniel Scally wrote:
>> Add structures describing the ISP parameters to mali-c55-config.h
>>
>> Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
>> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
>> ---
>> Changes in v6:
>>
>> - Flagged which struct goes with which enum value from
>> enum mali_c55_param_block_type
>> - Used only types with well defined sizes in the structs
>> - Expanded the documentation for skipping in the AEXP histogram config
>> - Aligned the header struct to 64 bits
>> - Added a new union type to hold pointers to the parameter structs
>>
>> Changes in v5:
>>
>> - New patch
>>
>> .../uapi/linux/media/arm/mali-c55-config.h | 763 ++++++++++++++++++
>> 1 file changed, 763 insertions(+)
>>
>> diff --git a/include/uapi/linux/media/arm/mali-c55-config.h b/include/uapi/linux/media/arm/mali-c55-config.h
>> index 21b453bdee5d..ea9872e9076f 100644
>> --- a/include/uapi/linux/media/arm/mali-c55-config.h
>> +++ b/include/uapi/linux/media/arm/mali-c55-config.h
>> @@ -179,4 +179,767 @@ struct mali_c55_stats_buffer {
>> __u32 reserved3[15];
>> } __attribute__((packed));
>>
>> +/**
>> + * enum mali_c55_param_buffer_version - Mali-C55 parameters block versioning
>> + *
>> + * @MALI_C55_PARAM_BUFFER_V1: First version of Mali-C55 parameters block
>> + */
>> +enum mali_c55_param_buffer_version {
>> + MALI_C55_PARAM_BUFFER_V1,
>> +};
>> +
>> +/**
>> + * enum mali_c55_param_block_type - Enumeration of Mali-C55 parameter blocks
>> + *
>> + * This enumeration defines the types of Mali-C55 parameters block. Each block
>> + * configures a specific processing block of the Mali-C55 ISP. The block
>> + * type allows the driver to correctly interpret the parameters block data.
>> + *
>> + * It is the responsibility of userspace to correctly set the type of each
>> + * parameters block.
>> + *
>> + * @MALI_C55_PARAM_BLOCK_SENSOR_OFFS: Sensor pre-shading black level offset
>> + * @MALI_C55_PARAM_BLOCK_AEXP_HIST: Auto-exposure 1024-bin histogram
>> + * configuration
>> + * @MALI_C55_PARAM_BLOCK_AEXP_IHIST: Post-Iridix auto-exposure 1024-bin
>> + * histogram configuration
>> + * @MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS: Auto-exposure 1024-bin histogram
>> + * weighting
>> + * @MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS: Post-Iridix auto-exposure 1024-bin
>> + * histogram weighting
>> + * @MALI_C55_PARAM_BLOCK_DIGITAL_GAIN: Digital gain
>> + * @MALI_C55_PARAM_BLOCK_AWB_GAINS: Auto-white balance gains
>> + * @MALI_C55_PARAM_BLOCK_AWB_CONFIG: Auto-white balance statistics config
>> + * @MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP: Auto-white balance gains for AEXP-0 tap
>> + * @MALI_C55_PARAM_MESH_SHADING_CONFIG : Mesh shading tables configuration
>> + * @MALI_C55_PARAM_MESH_SHADING_SELECTION: Mesh shading table selection
>> + * @MALI_C55_PARAM_BLOCK_SENTINEL: First non-valid block index
> The sentinel value will change when adding new types. That may affect
> userspace. We managed to get rid of the sentinel in the rkisp1 driver,
> it would be nice to do the same here.
Ack - I'm sure it'll be possible.
>
>> + */
>> +enum mali_c55_param_block_type {
>> + MALI_C55_PARAM_BLOCK_SENSOR_OFFS,
>> + MALI_C55_PARAM_BLOCK_AEXP_HIST,
>> + MALI_C55_PARAM_BLOCK_AEXP_IHIST,
>> + MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS,
>> + MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS,
>> + MALI_C55_PARAM_BLOCK_DIGITAL_GAIN,
>> + MALI_C55_PARAM_BLOCK_AWB_GAINS,
>> + MALI_C55_PARAM_BLOCK_AWB_CONFIG,
>> + MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP,
>> + MALI_C55_PARAM_MESH_SHADING_CONFIG,
>> + MALI_C55_PARAM_MESH_SHADING_SELECTION,
>> + MALI_C55_PARAM_BLOCK_SENTINEL,
>> +};
>> +
>> +/**
>> + * struct mali_c55_params_block_header - Mali-C55 parameter block header
>> + *
>> + * This structure represents the common part of all the ISP configuration
>> + * blocks. Each parameters block embeds an instance of this structure type
>> + * as its first member, followed by the block-specific configuration data. The
>> + * driver inspects this common header to discern the block type and its size and
>> + * properly handle the block content by casting it to the correct block-specific
>> + * type.
>> + *
>> + * The @type field is one of the values enumerated by
>> + * :c:type:`mali_c55_param_block_type` and specifies how the data should be
>> + * interpreted by the driver. The @size field specifies the size of the
>> + * parameters block and is used by the driver for validation purposes. The
>> + * @enabled field specifies if the ISP block should be enabled (and configured
>> + * according to the provided parameters) or disabled.
>> + *
>> + * Userspace is responsible for correctly populating the parameters block header
>> + * fields (@type, @enabled and @size) and correctly populate the block-specific
>> + * parameters.
>> + *
>> + * For example:
>> + *
>> + * .. code-block:: c
>> + *
>> + * void populate_sensor_offs(struct mali_c55_params_block_header *block) {
>> + * block->type = MALI_C55_PARAM_BLOCK_SENSOR_OFFS;
>> + * block->enabled = true;
>> + * block->size = sizeof(struct mali_c55_params_sensor_off_preshading);
>> + *
>> + * struct mali_c55_params_sensor_off_preshading *sensor_offs =
>> + * (struct mali_c55_params_sensor_off_preshading *)block;
>> + *
>> + * sensor_offs->chan00 = offset00;
>> + * sensor_offs->chan01 = offset01;
>> + * sensor_offs->chan10 = offset10;
>> + * sensor_offs->chan11 = offset11;
>> + * }
>> + *
>> + * @type: The parameters block type from :c:type:`mali_c55_param_block_type`
>> + * @enabled: Block enabled/disabled flag
>> + * @size: Size (in bytes) of the parameters block
>> + */
>> +struct mali_c55_params_block_header {
>> + __u8 type;
>> + __u8 enabled;
>> + __u32 size;
>> +} __attribute__((aligned(8)));
>> +
>> +/**
>> + * struct mali_c55_params_sensor_off_preshading - offset subtraction for each
>> + * color channel
>> + *
>> + * Provides removal of the sensor black level from the sensor data. Separate
>> + * offsets are provided for each of the four Bayer component color channels
>> + * which are defaulted to R, Gr, Gb, B.
>> + *
>> + * header.type should be set to MALI_C55_PARAM_BLOCK_SENSOR_OFFS from
>> + * :c:type:`mali_c55_param_block_type` for this block.
>> + *
>> + * @header: The Mali-C55 parameters block header
>> + * @chan00: Offset for color channel 00 (default: R)
>> + * @chan01: Offset for color channel 01 (default: Gr)
>> + * @chan10: Offset for color channel 10 (default: Gb)
>> + * @chan11: Offset for color channel 11 (default: B)
>> + */
>> +struct mali_c55_params_sensor_off_preshading {
>> + struct mali_c55_params_block_header header;
>> + __u32 chan00;
>> + __u32 chan01;
>> + __u32 chan10;
>> + __u32 chan11;
>> +};
>> +
>> +/**
>> + * enum mali_c55_aexp_hist_tap_points - Tap points for the AEXP histogram
>> + * @MALI_C55_AEXP_HIST_TAP_WB: After static white balance
>> + * @MALI_C55_AEXP_HIST_TAP_FS: After WDR Frame Stitch
>> + * @MALI_C55_AEXP_HIST_TAP_TPG: After the test pattern generator
>> + */
>> +enum mali_c55_aexp_hist_tap_points {
>> + MALI_C55_AEXP_HIST_TAP_WB = 0,
>> + MALI_C55_AEXP_HIST_TAP_FS,
>> + MALI_C55_AEXP_HIST_TAP_TPG,
>> +};
>> +
>> +/**
>> + * enum mali_c55_aexp_skip_x - Horizontal pixel skipping
>> + * @MALI_C55_AEXP_SKIP_X_EVERY_2ND: Collect every 2nd pixel horizontally
>> + * @MALI_C55_AEXP_SKIP_X_EVERY_3RD: Collect every 3rd pixel horizontally
>> + * @MALI_C55_AEXP_SKIP_X_EVERY_4TH: Collect every 4th pixel horizontally
>> + * @MALI_C55_AEXP_SKIP_X_EVERY_5TH: Collect every 5th pixel horizontally
>> + * @MALI_C55_AEXP_SKIP_X_EVERY_8TH: Collect every 8th pixel horizontally
>> + * @MALI_C55_AEXP_SKIP_X_EVERY_9TH: Collect every 9th pixel horizontally
>> + */
>> +enum mali_c55_aexp_skip_x {
>> + MALI_C55_AEXP_SKIP_X_EVERY_2ND,
>> + MALI_C55_AEXP_SKIP_X_EVERY_3RD,
>> + MALI_C55_AEXP_SKIP_X_EVERY_4TH,
>> + MALI_C55_AEXP_SKIP_X_EVERY_5TH,
>> + MALI_C55_AEXP_SKIP_X_EVERY_8TH,
>> + MALI_C55_AEXP_SKIP_X_EVERY_9TH
>> +};
>> +
>> +/**
>> + * enum mali_c55_aexp_skip_y - Vertical pixel skipping
>> + * @MALI_C55_AEXP_SKIP_Y_ALL: Collect every single pixel vertically
>> + * @MALI_C55_AEXP_SKIP_Y_EVERY_2ND: Collect every 2nd pixel vertically
>> + * @MALI_C55_AEXP_SKIP_Y_EVERY_3RD: Collect every 3rd pixel vertically
>> + * @MALI_C55_AEXP_SKIP_Y_EVERY_4TH: Collect every 4th pixel vertically
>> + * @MALI_C55_AEXP_SKIP_Y_EVERY_5TH: Collect every 5th pixel vertically
>> + * @MALI_C55_AEXP_SKIP_Y_EVERY_8TH: Collect every 8th pixel vertically
>> + * @MALI_C55_AEXP_SKIP_Y_EVERY_9TH: Collect every 9th pixel vertically
>> + */
>> +enum mali_c55_aexp_skip_y {
>> + MALI_C55_AEXP_SKIP_Y_ALL,
>> + MALI_C55_AEXP_SKIP_Y_EVERY_2ND,
>> + MALI_C55_AEXP_SKIP_Y_EVERY_3RD,
>> + MALI_C55_AEXP_SKIP_Y_EVERY_4TH,
>> + MALI_C55_AEXP_SKIP_Y_EVERY_5TH,
>> + MALI_C55_AEXP_SKIP_Y_EVERY_8TH,
>> + MALI_C55_AEXP_SKIP_Y_EVERY_9TH
>> +};
>> +
>> +/**
>> + * enum mali_c55_aexp_row_column_offset - Start from the first or second row or
>> + * column
>> + * @MALI_C55_AEXP_FIRST_ROW_OR_COL: Start from the first row / column
>> + * @MALI_C55_AEXP_SECOND_ROW_OR_COL: Start from the second row / column
>> + */
>> +enum mali_c55_aexp_row_column_offset {
>> + MALI_C55_AEXP_FIRST_ROW_OR_COL = 1,
>> + MALI_C55_AEXP_SECOND_ROW_OR_COL = 2,
>> +};
>> +
>> +/**
>> + * enum mali_c55_aexp_hist_plane_mode - Mode for the AEXP Histograms
>> + * @MALI_C55_AEXP_HIST_COMBINED: All color planes in one 1024-bin histogram
>> + * @MALI_C55_AEXP_HIST_SEPARATE: Each color plane in one 256-bin histogram with a bin width of 16
>> + * @MALI_C55_AEXP_HIST_FOCUS_00: Top left plane in the first bank, rest in second bank
>> + * @MALI_C55_AEXP_HIST_FOCUS_01: Top right plane in the first bank, rest in second bank
>> + * @MALI_C55_AEXP_HIST_FOCUS_10: Bottom left plane in the first bank, rest in second bank
>> + * @MALI_C55_AEXP_HIST_FOCUS_11: Bottom right plane in the first bank, rest in second bank
>> + *
>> + * In the "focus" modes statistics are collected into two 512-bin histograms
>> + * with a bin width of 8. One colour plane is in the first histogram with the
>> + * remainder combined into the second. The four options represent which of the
>> + * four positions in a bayer pattern are the focused plane.
>> + */
>> +enum mali_c55_aexp_hist_plane_mode {
>> + MALI_C55_AEXP_HIST_COMBINED = 0,
>> + MALI_C55_AEXP_HIST_SEPARATE = 1,
>> + MALI_C55_AEXP_HIST_FOCUS_00 = 4,
>> + MALI_C55_AEXP_HIST_FOCUS_01 = 5,
>> + MALI_C55_AEXP_HIST_FOCUS_10 = 6,
>> + MALI_C55_AEXP_HIST_FOCUS_11 = 7,
>> +};
>> +
>> +/**
>> + * struct mali_c55_params_aexp_hist - configuration for AEXP metering hists
>> + *
>> + * This struct allows users to configure the 1024-bin AEXP histograms. Broadly
>> + * speaking the parameters allow you to mask particular regions of the image and
>> + * to select different kinds of histogram.
>> + *
>> + * The skip_x, offset_x, skip_y and offset_y fields allow users to ignore or
>> + * mask pixels in the frame by their position relative to the top left pixel.
>> + * First, the skip_y, offset_x and offset_y fields define a 2x2 pixel pattern of
>> + * which pixels covered by the pattern will be counted in the statistics.
> I have a really hard time parsing the second sentence. Let me try to
> rephrase it.
>
> * First, the skip_y, offset_x and offset_y fields define which pixels will be
> * counted in the statistics within a 2x2 pixels region.
>
> Is that what you meant ?
Yes, although I feel it's a bit iffy...perhaps "...which of the pixels within each 2x2 region will
be counted in the statistics"?
>
>> + *
>> + * If skip_y == 0 then two pixels from each coveredregion will be counted. If
> s/coveredregion/region/
>
>> + * both offset_x and offset_y are zero, then the two left-most pixels in each
>> + * 2x2 pixel region covered by the pattern will be counted. Setting offset_x = 1
> I'd drop "covered by the pattern" as I'm not sure what that means.
OK, I think it reads fine that way.
>
>> + * will discount the top left pixel and count the top right pixel. Setting
>> + * offset_y = 1 will discount the bottom left pixel and count the bottom right
>> + * pixel.
> That sounds strange, I would have expected offset_y to control something
> vertically, not control the horizontal position on the second line. If
> that's how the hardware operates, why not.
Yes it is strange, but that's what the documentation says.
>
>> + *
>> + * If skip_y != 0 then only a single pixel from each region covered by the
>> + * pattern will be counted. In this case offset_x controls whether the pixel
>> + * that's counted is in the left (if offset_x == 0) or right (if offset_x == 1)
>> + * column and offset_y controls whether the pixel that's counted is in the top
>> + * (if offset_y == 0) or bottom (if offset_y == 1) row.
>> + *
>> + * The skip_x and skip_y fields control how the pixel masking pattern is
>> + * repeated across the image data. The first instance of the pattern is always
> And here maybe "control how the 2x2 pixel region is repeated ...".
> s/pattern/region/ in the rest.
Okedokey
>
> This is otherwise a great explanation, thank you.
Thanks!
>> + * in the top left of the image data. The skip_x field controls how many pixels
>> + * are ignored in the x direction before the pixel masking pattern is repeated.
>> + * The skip_y field controls how many pixels are ignored in the y direction
>> + * before the pixel masking pattern is repeated.
>> + *
>> + * These fields can be used to reduce the number of pixels counted for the
>> + * statistics, but it's important to be careful to configure them correctly.
>> + * Some combinations of values will result in colour components from the input
>> + * data being ignored entirely, for example in the following configuration:
>> + *
>> + * skip_x = 0
>> + * offset_x = 0
>> + * skip_y = 0
>> + * offset_y = 0
>> + *
>> + * Only the R and Gb components of RGGB data that was input would be collected.
>> + * Similarly in the following configuration:
>> + *
>> + * skip_x = 0
>> + * offset_x = 0
>> + * skip_y = 1
>> + * offset_y = 1
>> + *
>> + * Only the Gb component of RGGB data that was input would be collected. To
>> + * correct things such that all 4 colour components were included it would be
>> + * necessary to set the skip_x and skip_y fields in a way that resulted in all
>> + * four colour components being collected:
>> + *
>> + * skip_x = 1
>> + * offset_x = 0
>> + * skip_y = 1
>> + * offset_y = 1
>> + *
>> + * header.type should be set to one of either MALI_C55_PARAM_BLOCK_AEXP_HIST or
>> + * MALI_C55_PARAM_BLOCK_AEXP_IHIST from :c:type:`mali_c55_param_block_type`.
>> + *
>> + * @header: The Mali-C55 parameters block header
>> + * @skip_x: Horizontal decimation. See enum mali_c55_aexp_skip_x
>> + * @offset_x: Skip the first column, or not. See enum mali_c55_aexp_row_column_offset
>> + * @skip_y: Vertical decimation. See enum mali_c55_aexp_skip_y
>> + * @offset_y: Skip the first row, or not. See enum mali_c55_aexp_row_column_offset
>> + * @scale_bottom: Scale pixels in bottom half of intensity range: 0=1x ,1=2x, 2=4x, 4=8x, 4=16x
>> + * @scale_top: scale pixels in top half of intensity range: 0=1x ,1=2x, 2=4x, 4=8x, 4=16x
>> + * @plane_mode: Plane separation mode. See enum mali_c55_aexp_hist_plane_mode
>> + * @tap_point: Tap point for histogram from enum mali_c55_aexp_hist_tap_points.
>> + * This parameter is unused for the post-Iridix Histogram
>> + */
>> +struct mali_c55_params_aexp_hist {
>> + struct mali_c55_params_block_header header;
>> + __u8 skip_x;
>> + __u8 offset_x;
>> + __u8 skip_y;
>> + __u8 offset_y;
>> + __u8 scale_bottom;
>> + __u8 scale_top;
>> + __u8 plane_mode;
>> + __u8 tap_point;
>> +};
>> +
>> +/**
>> + * struct mali_c55_params_aexp_weights - Array of weights for AEXP metering
>> + *
>> + * This struct allows users to configure the weighting for both of the 1024-bin
>> + * AEXP histograms. The pixel data collected for each zone is multiplied by the
>> + * corresponding weight from this array, which may be zero if the intention is
>> + * to mask off the zone entirely.
>> + *
>> + * header.type should be set to one of either MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS
>> + * or MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS from :c:type:`mali_c55_param_block_type`.
>> + *
>> + * @header: The Mali-C55 parameters block header
>> + * @nodes_used_horiz: Number of active zones horizontally [0..15]
>> + * @nodes_used_vert: Number of active zones vertically [0..15]
>> + * @zone_weights: Zone weighting. Index is row*col where 0,0 is the top
>> + * left zone continuing in raster order. Each zone can be
>> + * weighted in the range [0..15]. The number of rows and
>> + * columns is defined by @nodes_used_vert and
>> + * @nodes_used_horiz
>> + */
>> +struct mali_c55_params_aexp_weights {
>> + struct mali_c55_params_block_header header;
>> + __u8 nodes_used_horiz;
>> + __u8 nodes_used_vert;
>> + __u8 zone_weights[MALI_C55_MAX_ZONES];
>> +};
>> +
>> +/**
>> + * struct mali_c55_params_digital_gain - Digital gain value
>> + *
>> + * This struct carries a digital gain value to set in the ISP.
>> + *
>> + * header.type should be set to MALI_C55_PARAM_BLOCK_DIGITAL_GAIN from
>> + * :c:type:`mali_c55_param_block_type` for this block.
>> + *
>> + * @header: The Mali-C55 parameters block header
>> + * @gain: The digital gain value to apply, in Q5.8 format.
>> + */
>> +struct mali_c55_params_digital_gain {
>> + struct mali_c55_params_block_header header;
>> + __u16 gain;
>> +};
>> +
>> +/**
>> + * enum mali_c55_awb_stats_mode - Statistics mode for AWB
>> + * @MALI_C55_AWB_MODE_GRBR: Statistics collected as Green/Red and Blue/Red ratios
>> + * @MALI_C55_AWB_MODE_RGBG: Statistics collected as Red/Green and Blue/Green ratios
>> + */
>> +enum mali_c55_awb_stats_mode {
>> + MALI_C55_AWB_MODE_GRBR = 0,
>> + MALI_C55_AWB_MODE_RGBG,
>> +};
>> +
>> +/**
>> + * struct mali_c55_params_awb_gains - Gain settings for auto white balance
>> + *
>> + * This struct allows users to configure the gains for auto-white balance. There
>> + * are four gain settings corresponding to each colour channel in the bayer
>> + * domain. Although named generically, the association between the gain applied
>> + * and the colour channel is done automatically within the ISP depending on the
>> + * input format, and so the following mapping always holds true::
>> + *
>> + * gain00 = R
>> + * gain01 = Gr
>> + * gain10 = Gb
>> + * gain11 = B
>> + *
>> + * All of the gains are stored in Q4.8 format.
>> + *
>> + * header.type should be set to one of either MALI_C55_PARAM_BLOCK_AWB_GAINS or
>> + * MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP from :c:type:`mali_c55_param_block_type`.
>> + *
>> + * @header: The Mali-C55 parameters block header
>> + * @gain00: Multiplier for colour channel 00
>> + * @gain01: Multiplier for colour channel 01
>> + * @gain10: Multiplier for colour channel 10
>> + * @gain11: Multiplier for colour channel 11
>> + */
>> +struct mali_c55_params_awb_gains {
>> + struct mali_c55_params_block_header header;
>> + __u16 gain00;
>> + __u16 gain01;
>> + __u16 gain10;
>> + __u16 gain11;
>> +};
>> +
>> +/**
>> + * enum mali_c55_params_awb_tap_points - Tap points for the AWB statistics
>> + * @MALI_C55_AWB_STATS_TAP_PF: Immediately after the Purple Fringe block
>> + * @MALI_C55_AWB_STATS_TAP_CNR: Immediately after the CNR block
>> + */
>> +enum mali_c55_params_awb_tap_points {
>> + MALI_C55_AWB_STATS_TAP_PF = 0,
>> + MALI_C55_AWB_STATS_TAP_CNR,
>> +};
>> +
>> +/**
>> + * struct mali_c55_params_awb_config - Stats settings for auto-white balance
>> + *
>> + * This struct allows the configuration of the statistics generated for auto
>> + * white balance. Pixel intensity limits can be set to exclude overly bright or
>> + * dark regions of an image from the statistics entirely. Colour ratio minima
>> + * and maxima can be set to discount pixels who's ratios fall outside the
>> + * defined boundaries; there are two sets of registers to do this - the
>> + * "min/max" ratios which bound a region and the "high/low" ratios which further
>> + * trim the upper and lower ratios. For example with the boundaries configured
>> + * as follows, only pixels whos colour ratios falls into the region marked "A"
>> + * would be counted::
>> + *
>> + * cr_high
>> + * 2.0 | |
>> + * | cb_max --> _________________________v_____
>> + * 1.8 | | \ |
>> + * | | \ |
>> + * 1.6 | | \ |
>> + * | | \ |
>> + * c 1.4 | cb_low -->|\ A \|<-- cb_high
>> + * b | | \ |
>> + * 1.2 | | \ |
>> + * r | | \ |
>> + * a 1.0 | cb_min --> |____\_________________________|
>> + * t | ^ ^ ^
>> + * i 0.8 | | | |
>> + * o | cr_min | cr_max
>> + * s 0.6 | |
>> + * | cr_low
>> + * 0.4 |
>> + * |
>> + * 0.2 |
>> + * |
>> + * 0.0 |_______________________________________________________________
>> + * 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0
>> + * cr ratios
>> + *
>> + * header.type should be set to MALI_C55_PARAM_BLOCK_AWB_CONFIG from
>> + * :c:type:`mali_c55_param_block_type` for this block.
>> + *
>> + * @header: The Mali-C55 parameters block header
>> + * @tap_point: The tap point from enum mali_c55_params_awb_tap_points
>> + * @stats_mode: AWB statistics collection mode, see :c:type:`mali_c55_awb_stats_mode`
>> + * @white_level: Upper pixel intensity (I.E. raw pixel values) limit
>> + * @black_level: Lower pixel intensity (I.E. raw pixel values) limit
>> + * @cr_max: Maximum R/G ratio (Q4.8 format)
>> + * @cr_min: Minimum R/G ratio (Q4.8 format)
>> + * @cb_max: Maximum B/G ratio (Q4.8 format)
>> + * @cb_min: Minimum B/G ratio (Q4.8 format)
>> + * @nodes_used_horiz: Number of active zones horizontally [0..15]
>> + * @nodes_used_vert: Number of active zones vertically [0..15]
>> + * @cr_high: R/G ratio trim high (Q4.8 format)
>> + * @cr_low: R/G ratio trim low (Q4.8 format)
>> + * @cb_high: B/G ratio trim high (Q4.8 format)
>> + * @cb_low: B/G ratio trim low (Q4.8 format)
>> + */
>> +struct mali_c55_params_awb_config {
>> + struct mali_c55_params_block_header header;
>> + __u8 tap_point;
>> + __u8 stats_mode;
>> + __u16 white_level;
>> + __u16 black_level;
>> + __u16 cr_max;
>> + __u16 cr_min;
>> + __u16 cb_max;
>> + __u16 cb_min;
>> + __u8 nodes_used_horiz;
>> + __u8 nodes_used_vert;
>> + __u16 cr_high;
>> + __u16 cr_low;
>> + __u16 cb_high;
>> + __u16 cb_low;
>> +};
>> +
>> +#define MALI_C55_NUM_MESH_SHADING_ELEMENTS 3072
>> +
>> +/**
>> + * struct mali_c55_params_mesh_shading_config - Mesh shading configuration
>> + *
>> + * The mesh shading correction module allows programming a separate table of
>> + * either 16x16 or 32x32 node coefficients for 3 different light sources. The
>> + * final correction coefficients applied are computed by blending the
>> + * coefficients from two tables together.
>> + *
>> + * A page of 1024 32-bit integers is associated to each colour channel, with
>> + * pages stored consecutively in memory. Each 32-bit integer packs 3 8-bit
>> + * correction coefficients for a single node, one for each of the three light
>> + * sources. The 8 most significant bits are unused. The following table
>> + * describes the layout::
>> + *
>> + * +----------- Page (Colour Plane) 0 -------------+
>> + * | @mesh[i] | Mesh Point | Bits | Light Source |
>> + * +-----------+------------+-------+--------------+
>> + * | 0 | 0,0 | 16,23 | LS2 |
>> + * | | | 08-15 | LS1 |
>> + * | | | 00-07 | LS0 |
>> + * +-----------+------------+-------+--------------+
>> + * | 1 | 0,1 | 16,23 | LS2 |
>> + * | | | 08-15 | LS1 |
>> + * | | | 00-07 | LS0 |
>> + * +-----------+------------+-------+--------------+
>> + * | ... | ... | ... | ... |
>> + * +-----------+------------+-------+--------------+
>> + * | 1023 | 31,31 | 16,23 | LS2 |
>> + * | | | 08-15 | LS1 |
>> + * | | | 00-07 | LS0 |
>> + * +----------- Page (Colour Plane) 1 -------------+
>> + * | @mesh[i] | Mesh Point | Bits | Light Source |
>> + * +-----------+------------+-------+--------------+
>> + * | 1024 | 0,0 | 16,23 | LS2 |
>> + * | | | 08-15 | LS1 |
>> + * | | | 00-07 | LS0 |
>> + * +-----------+------------+-------+--------------+
>> + * | 1025 | 0,1 | 16,23 | LS2 |
>> + * | | | 08-15 | LS1 |
>> + * | | | 00-07 | LS0 |
>> + * +-----------+------------+-------+--------------+
>> + * | ... | ... | ... | ... |
>> + * +-----------+------------+-------+--------------+
>> + * | 2047 | 31,31 | 16,23 | LS2 |
>> + * | | | 08-15 | LS1 |
>> + * | | | 00-07 | LS0 |
>> + * +----------- Page (Colour Plane) 2 -------------+
>> + * | @mesh[i] | Mesh Point | Bits | Light Source |
>> + * +-----------+------------+-------+--------------+
>> + * | 2048 | 0,0 | 16,23 | LS2 |
>> + * | | | 08-15 | LS1 |
>> + * | | | 00-07 | LS0 |
>> + * +-----------+------------+-------+--------------+
>> + * | 2049 | 0,1 | 16,23 | LS2 |
>> + * | | | 08-15 | LS1 |
>> + * | | | 00-07 | LS0 |
>> + * +-----------+------------+-------+--------------+
>> + * | ... | ... | ... | ... |
>> + * +-----------+------------+-------+--------------+
>> + * | 3071 | 31,31 | 16,23 | LS2 |
>> + * | | | 08-15 | LS1 |
>> + * | | | 00-07 | LS0 |
>> + * +-----------+------------+-------+--------------+
>> + *
>> + * The @mesh_scale member determines the precision and minimum and maximum gain.
>> + * For example if @mesh_scale is 0 and therefore selects 0 - 2x gain, a value of
>> + * 0 in a coefficient means 0.0 gain, a value of 128 means 1.0 gain and 255
>> + * means 2.0 gain.
>> + *
>> + * header.type should be set to MALI_C55_PARAM_MESH_SHADING_CONFIG from
>> + * :c:type:`mali_c55_param_block_type` for this block.
>> + *
>> + * @header: The Mali-C55 parameters block header
>> + * @mesh_show: Output the mesh data rather than image data
>> + * @mesh_scale: Set the precision and maximum gain range of mesh shading
>> + * - 0 = 0-2x gain
>> + * - 1 = 0-4x gain
>> + * - 2 = 0-8x gain
>> + * - 3 = 0-16x gain
>> + * - 4 = 1-2x gain
>> + * - 5 = 1-3x gain
>> + * - 6 = 1-5x gain
>> + * - 7 = 1-9x gain
>> + * @mesh_page_r: Mesh page select for red colour plane [0..2]
>> + * @mesh_page_g: Mesh page select for green colour plane [0..2]
>> + * @mesh_page_b: Mesh page select for blue colour plane [0..2]
>> + * @mesh_width: Number of horizontal nodes minus 1 [15,31]
>> + * @mesh_height: Number of vertical nodes minus 1 [15,31]
>> + * @mesh: Mesh shading correction tables
>> + */
>> +struct mali_c55_params_mesh_shading_config {
>> + struct mali_c55_params_block_header header;
>> + __u8 mesh_show;
>> + __u8 mesh_scale;
>> + __u8 mesh_page_r;
>> + __u8 mesh_page_g;
>> + __u8 mesh_page_b;
>> + __u8 mesh_width;
>> + __u8 mesh_height;
>> + __u32 mesh[MALI_C55_NUM_MESH_SHADING_ELEMENTS];
>> +};
>> +
>> +/** enum mali_c55_params_mesh_alpha_bank - Mesh shading table bank selection
>> + * @MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS1 - Select Light Sources 0 and 1
>> + * @MALI_C55_MESH_ALPHA_BANK_LS1_AND_LS2 - Select Light Sources 1 and 2
>> + * @MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS2 - Select Light Sources 0 and 2
>> + */
>> +enum mali_c55_params_mesh_alpha_bank {
>> + MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS1 = 0,
>> + MALI_C55_MESH_ALPHA_BANK_LS1_AND_LS2 = 1,
>> + MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS2 = 4
>> +};
>> +
>> +/**
>> + * struct mali_c55_params_mesh_shading_selection - Mesh table selection
>> + *
>> + * The module computes the final correction coefficients by blending the ones
>> + * from two light source tables, which are selected (independently for each
>> + * colour channel) by the @mesh_alpha_bank_r/g/b fields.
>> + *
>> + * The final blended coefficients for each node are calculated using the
>> + * following equation:
>> + *
>> + * Final coefficient = (a * LS\ :sub:`b`\ + (256 - a) * LS\ :sub:`a`\) / 256
>> + *
>> + * Where a is the @mesh_alpha_r/g/b value, and LS\ :sub:`a`\ and LS\ :sub:`b`\
>> + * are the node cofficients for the two tables selected by the
>> + * @mesh_alpha_bank_r/g/b value.
>> + *
>> + * The scale of the applied correction may also be controlled by tuning the
>> + * @mesh_strength member. This is a modifier to the final coefficients which can
>> + * be used to globally reduce the gains applied.
>> + *
>> + * header.type should be set to MALI_C55_PARAM_MESH_SHADING_SELECTION from
>> + * :c:type:`mali_c55_param_block_type` for this block.
>> + *
>> + * @header: The Mali-C55 parameters block header
>> + * @mesh_alpha_bank_r: Red mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`)
>> + * @mesh_alpha_bank_g: Green mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`)
>> + * @mesh_alpha_bank_b: Blue mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`)
>> + * @mesh_alpha_r: Blend coefficient for R [0..255]
>> + * @mesh_alpha_g: Blend coefficient for G [0..255]
>> + * @mesh_alpha_b: Blend coefficient for B [0..255]
>> + * @mesh_strength: Mesh strength in Q4.12 format [0..4096]
>> + */
>> +struct mali_c55_params_mesh_shading_selection {
>> + struct mali_c55_params_block_header header;
>> + __u8 mesh_alpha_bank_r;
>> + __u8 mesh_alpha_bank_g;
>> + __u8 mesh_alpha_bank_b;
>> + __u8 mesh_alpha_r;
>> + __u8 mesh_alpha_g;
>> + __u8 mesh_alpha_b;
>> + __u16 mesh_strength;
>> +};
>> +
>> +/**
>> + * union mali_c55_params_block - Generalisation of a parameter block
>> + *
>> + * This union allows the driver to treat a block as a generic pointer to this
>> + * union and safely access the header and block-specific struct without having
>> + * to resort to casting. The header member is accessed first, and the type field
> If this is for driver usage only, I would move the structure to the
> driver. Let's minimize the UAPI surface.
I have actually used the union in libcamera too, to clean up the code there handling filling the
blocks. We could redefine it within libcamera of course but given it's useful in userspace too, I
thought it made sense to keep it in the header...what do you think?
>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>
>> + * checked which allows the driver to determine which of the other members
>> + * should be used. The data member at the end allows a pointer to an address
>> + * within the data member of :c:type:`mali_c55_params_buffer` to initialise a
>> + * union variable.
>> + *
>> + * @header: Pointer to the shared header struct embedded as the
>> + * first member of all the possible other members (except
>> + * @data). This member would be accessed first and the type
>> + * field checked to determine which of the other members
>> + * should be accessed.
>> + * @sensor_offs: For header->type == MALI_C55_PARAM_BLOCK_SENSOR_OFFS
>> + * @aexp_hist: For header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST and
>> + * header->type == MALI_C55_PARAM_BLOCK_AEXP_IHIST
>> + * @aexp_weights: For header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS
>> + * and header->type = MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS
>> + * @digital_gain: For header->type == MALI_C55_PARAM_BLOCK_DIGITAL_GAIN
>> + * @awb_gains: For header->type == MALI_C55_PARAM_BLOCK_AWB_GAINS and
>> + * header->type = MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP
>> + * @awb_config: For header->type == MALI_C55_PARAM_MESH_SHADING_CONFIG
>> + * @shading_config: For header->type == MALI_C55_PARAM_MESH_SHADING_SELECTION
>> + * @shading_selection: For header->type == MALI_C55_PARAM_BLOCK_SENSOR_OFFS
>> + * @data: Allows easy initialisation of a union variable with a
>> + * pointer into a __u8 array.
>> + */
>> +union mali_c55_params_block {
>> + struct mali_c55_params_block_header *header;
>> + struct mali_c55_params_sensor_off_preshading *sensor_offs;
>> + struct mali_c55_params_aexp_hist *aexp_hist;
>> + struct mali_c55_params_aexp_weights *aexp_weights;
>> + struct mali_c55_params_digital_gain *digital_gain;
>> + struct mali_c55_params_awb_gains *awb_gains;
>> + struct mali_c55_params_awb_config *awb_config;
>> + struct mali_c55_params_mesh_shading_config *shading_config;
>> + struct mali_c55_params_mesh_shading_selection *shading_selection;
>> + __u8 *data;
>> +};
>> +
>> +/**
>> + * define MALI_C55_PARAMS_MAX_SIZE - Maximum size of all Mali C55 Parameters
>> + *
>> + * Though the parameters for the Mali-C55 are passed as optional blocks, the
>> + * driver still needs to know the absolute maximum size so that it can allocate
>> + * a buffer sized appropriately to accommodate userspace attempting to set all
>> + * possible parameters in a single frame.
>> + *
>> + * Some structs are in this list multiple times. Where that's the case, it just
>> + * reflects the fact that the same struct can be used with multiple different
>> + * header types from :c:type:`mali_c55_param_block_type`.
>> + */
>> +#define MALI_C55_PARAMS_MAX_SIZE \
>> + (sizeof(struct mali_c55_params_sensor_off_preshading) + \
>> + sizeof(struct mali_c55_params_aexp_hist) + \
>> + sizeof(struct mali_c55_params_aexp_weights) + \
>> + sizeof(struct mali_c55_params_aexp_hist) + \
>> + sizeof(struct mali_c55_params_aexp_weights) + \
>> + sizeof(struct mali_c55_params_digital_gain) + \
>> + sizeof(struct mali_c55_params_awb_gains) + \
>> + sizeof(struct mali_c55_params_awb_config) + \
>> + sizeof(struct mali_c55_params_awb_gains) + \
>> + sizeof(struct mali_c55_params_mesh_shading_config) + \
>> + sizeof(struct mali_c55_params_mesh_shading_selection))
>> +
>> +/**
>> + * struct mali_c55_params_buffer - 3A configuration parameters
>> + *
>> + * This struct contains the configuration parameters of the Mali-C55 ISP
>> + * algorithms, serialized by userspace into a data buffer. Each configuration
>> + * parameter block is represented by a block-specific structure which contains a
>> + * :c:type:`mali_c55_params_block_header` entry as first member. Userspace
>> + * populates the @data buffer with configuration parameters for the blocks that
>> + * it intends to configure. As a consequence, the data buffer effective size
>> + * changes according to the number of ISP blocks that userspace intends to
>> + * configure.
>> + *
>> + * The parameters buffer is versioned by the @version field to allow modifying
>> + * and extending its definition. Userspace shall populate the @version field to
>> + * inform the driver about the version it intends to use. The driver will parse
>> + * and handle the @data buffer according to the data layout specific to the
>> + * indicated version and return an error if the desired version is not
>> + * supported.
>> + *
>> + * For each ISP block that userspace wants to configure, a block-specific
>> + * structure is appended to the @data buffer, one after the other without gaps
>> + * in between nor overlaps. Userspace shall populate the @total_size field with
>> + * the effective size, in bytes, of the @data buffer.
>> + *
>> + * The expected memory layout of the parameters buffer is::
>> + *
>> + * +-------------------- struct mali_c55_params_buffer ------------------+
>> + * | version = MALI_C55_PARAM_BUFFER_V1; |
>> + * | total_size = sizeof(struct mali_c55_params_sensor_off_preshading) |
>> + * | sizeof(struct mali_c55_params_aexp_hist); |
>> + * | +------------------------- data ---------------------------------+ |
>> + * | | +--------- struct mali_c55_params_sensor_off_preshading ------+ | |
>> + * | | | +-------- struct mali_c55_params_block_header header -----+ | | |
>> + * | | | | type = MALI_C55_PARAM_BLOCK_SENSOR_OFFS; | | | |
>> + * | | | | enabled = 1; | | | |
>> + * | | | | size = | | | |
>> + * | | | | sizeof(struct mali_c55_params_sensor_off_preshading);| | | |
>> + * | | | +---------------------------------------------------------+ | | |
>> + * | | | chan00 = ...; | | |
>> + * | | | chan01 = ...; | | |
>> + * | | | chan10 = ...; | | |
>> + * | | | chan11 = ...; | | |
>> + * | | +------------ struct mali_c55_params_aexp_hist ---------------+ | |
>> + * | | | +-------- struct mali_c55_params_block_header header -----+ | | |
>> + * | | | | type = MALI_C55_PARAM_BLOCK_AEXP_HIST; | | | |
>> + * | | | | enabled = 1; | | | |
>> + * | | | | size = sizeof(struct mali_c55_params_aexp_hist); | | | |
>> + * | | | +---------------------------------------------------------+ | | |
>> + * | | | skip_x = ...; | | |
>> + * | | | offset_x = ...; | | |
>> + * | | | skip_y = ...; | | |
>> + * | | | offset_y = ...; | | |
>> + * | | | scale_bottom = ...; | | |
>> + * | | | scale_top = ...; | | |
>> + * | | | plane_mode = ...; | | |
>> + * | | | tap_point = ...; | | |
>> + * | | +-------------------------------------------------------------+ | |
>> + * | +-----------------------------------------------------------------+ |
>> + * +---------------------------------------------------------------------+
>> + *
>> + * @version: The version from :c:type:`mali_c55_param_buffer_version`
>> + * @total_size: The Mali-C55 configuration data effective size, excluding this
>> + * header
>> + * @data: The Mali-C55 configuration blocks data
>> + */
>> +struct mali_c55_params_buffer {
>> + __u8 version;
>> + __u32 total_size;
>> + __u8 data[MALI_C55_PARAMS_MAX_SIZE];
>> +};
>> +
>> #endif /* __UAPI_MALI_C55_CONFIG_H */
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 12/18] media: platform: Add mali-c55 3a stats devnode
2024-07-22 14:50 ` Laurent Pinchart
2024-07-22 22:55 ` Laurent Pinchart
@ 2024-07-29 10:53 ` Dan Scally
2024-08-13 13:51 ` Dan Scally
2 siblings, 0 replies; 41+ messages in thread
From: Dan Scally @ 2024-07-29 10:53 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Laurent
On 22/07/2024 15:50, Laurent Pinchart wrote:
> Hi Dan,
>
> Thank you for the patch.
>
> On Tue, Jul 09, 2024 at 02:29:00PM +0100, Daniel Scally wrote:
>> Add a new code file to govern the 3a statistics capture node.
>>
>> Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
>> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
>> ---
>> Changes in v6:
>>
>> - Fixed mising includes
>> - Minor renames and formatting
>> - Reworked mali_c55_stats_metering_complete() so it could only return
>> buffers when both halves of the DMA read were done
>> - Terminate dma transfers on streamoff
>>
>> Changes in v5:
>>
>> - New patch
>>
>> drivers/media/platform/arm/mali-c55/Makefile | 1 +
>> .../platform/arm/mali-c55/mali-c55-common.h | 29 ++
>> .../platform/arm/mali-c55/mali-c55-core.c | 15 +
>> .../platform/arm/mali-c55/mali-c55-isp.c | 11 +
>> .../arm/mali-c55/mali-c55-registers.h | 3 +
>> .../platform/arm/mali-c55/mali-c55-stats.c | 373 ++++++++++++++++++
>> 6 files changed, 432 insertions(+)
>> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-stats.c
>>
>> diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile
>> index 9178ac35e50e..b5a22d414479 100644
>> --- a/drivers/media/platform/arm/mali-c55/Makefile
>> +++ b/drivers/media/platform/arm/mali-c55/Makefile
>> @@ -4,6 +4,7 @@ mali-c55-y := mali-c55-capture.o \
>> mali-c55-core.o \
>> mali-c55-isp.o \
>> mali-c55-resizer.o \
>> + mali-c55-stats.o \
>> mali-c55-tpg.o
>>
>> obj-$(CONFIG_VIDEO_MALI_C55) += mali-c55.o
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
>> index f7764a938e9f..136c785c68ba 100644
>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-common.h
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
>> @@ -49,6 +49,7 @@ enum mali_c55_isp_pads {
>> MALI_C55_ISP_PAD_SINK_VIDEO,
>> MALI_C55_ISP_PAD_SOURCE_VIDEO,
>> MALI_C55_ISP_PAD_SOURCE_BYPASS,
>> + MALI_C55_ISP_PAD_SOURCE_STATS,
>> MALI_C55_ISP_NUM_PADS,
>> };
>>
>> @@ -160,6 +161,29 @@ struct mali_c55_cap_dev {
>> bool streaming;
>> };
>>
>> +struct mali_c55_stats_buf {
>> + struct vb2_v4l2_buffer vb;
>> + unsigned int segments_remaining;
>> + struct list_head queue;
>> + bool failed;
>> +};
>> +
>> +struct mali_c55_stats {
>> + struct mali_c55 *mali_c55;
>> + struct video_device vdev;
>> + struct dma_chan *channel;
>> + struct vb2_queue queue;
>> + struct media_pad pad;
>> + /* Mutex to provide to vb2 */
>> + struct mutex lock;
>> +
>> + struct {
>> + /* Spinlock to guard buffer queue */
>> + spinlock_t lock;
>> + struct list_head queue;
>> + } buffers;
>> +};
>> +
>> enum mali_c55_config_spaces {
>> MALI_C55_CONFIG_PING,
>> MALI_C55_CONFIG_PONG,
>> @@ -202,6 +226,7 @@ struct mali_c55 {
>> struct mali_c55_isp isp;
>> struct mali_c55_resizer resizers[MALI_C55_NUM_RSZS];
>> struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS];
>> + struct mali_c55_stats stats;
>>
>> struct mali_c55_context context;
>> enum mali_c55_config_spaces next_config;
>> @@ -229,6 +254,8 @@ int mali_c55_register_resizers(struct mali_c55 *mali_c55);
>> void mali_c55_unregister_resizers(struct mali_c55 *mali_c55);
>> int mali_c55_register_capture_devs(struct mali_c55 *mali_c55);
>> void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55);
>> +int mali_c55_register_stats(struct mali_c55 *mali_c55);
>> +void mali_c55_unregister_stats(struct mali_c55 *mali_c55);
>> struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55);
>> void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
>> enum mali_c55_planes plane);
>> @@ -243,5 +270,7 @@ const struct mali_c55_isp_fmt *
>> mali_c55_isp_get_mbus_config_by_code(u32 code);
>> const struct mali_c55_isp_fmt *
>> mali_c55_isp_get_mbus_config_by_index(u32 index);
>> +void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
>> + enum mali_c55_config_spaces cfg_space);
>>
>> #endif /* _MALI_C55_COMMON_H */
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
>> index dbc07d12d3a3..eedc8f450184 100644
>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
>> @@ -374,6 +374,16 @@ static int mali_c55_create_links(struct mali_c55 *mali_c55)
>> }
>> }
>>
>> + ret = media_create_pad_link(&mali_c55->isp.sd.entity,
>> + MALI_C55_ISP_PAD_SOURCE_STATS,
>> + &mali_c55->stats.vdev.entity, 0,
>> + MEDIA_LNK_FL_ENABLED);
>> + if (ret) {
>> + dev_err(mali_c55->dev,
>> + "failed to link ISP and 3a stats node\n");
>> + goto err_remove_links;
>> + }
>> +
>> return 0;
>>
>> err_remove_links:
>> @@ -388,6 +398,7 @@ static void mali_c55_unregister_entities(struct mali_c55 *mali_c55)
>> mali_c55_unregister_isp(mali_c55);
>> mali_c55_unregister_resizers(mali_c55);
>> mali_c55_unregister_capture_devs(mali_c55);
>> + mali_c55_unregister_stats(mali_c55);
>> }
>>
>> static int mali_c55_register_entities(struct mali_c55 *mali_c55)
>> @@ -410,6 +421,10 @@ static int mali_c55_register_entities(struct mali_c55 *mali_c55)
>> if (ret)
>> goto err_unregister_entities;
>>
>> + ret = mali_c55_register_stats(mali_c55);
>> + if (ret)
>> + goto err_unregister_entities;
>> +
>> ret = mali_c55_create_links(mali_c55);
>> if (ret)
>> goto err_unregister_entities;
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>> index f784983a4ccc..2f450c00300a 100644
>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>> @@ -5,6 +5,8 @@
>> * Copyright (C) 2024 Ideas on Board Oy
>> */
>>
>> +#include <linux/media/arm/mali-c55-config.h>
>> +
>> #include <linux/delay.h>
>> #include <linux/iopoll.h>
>> #include <linux/property.h>
>> @@ -460,6 +462,14 @@ static int mali_c55_isp_init_state(struct v4l2_subdev *sd,
>> in_crop->width = MALI_C55_DEFAULT_WIDTH;
>> in_crop->height = MALI_C55_DEFAULT_HEIGHT;
>>
>> + src_fmt = v4l2_subdev_state_get_format(state,
>> + MALI_C55_ISP_PAD_SOURCE_STATS);
>> +
>> + src_fmt->width = sizeof(struct mali_c55_stats_buffer);
>> + src_fmt->height = 1;
> According to
> https://docs.kernel.org/userspace-api/media/v4l/subdev-formats.html#metadata-formats,
> width and height should be set to 0 for MEDIA_BUS_FMT_METADATA_FIXED. I
> haven't checked if v4l2-compliance expects this, we may have
> discrepancies between the API documentation and the existing
> implementations in the kernel and userspace. In any case, this should be
> sorted out, either by fixing the kernel code and enforcing the
> requirement in v4l2-compliance, or fixing the documentation.
Ah. I can't completely remember why I set these to other-than-zero. When I refresh the series I'll
figure out what spurred this, and then we can address v4l2-compliance / the documentation.
>
>> + src_fmt->field = V4L2_FIELD_NONE;
>> + src_fmt->code = MEDIA_BUS_FMT_METADATA_FIXED;
>> +
>> return 0;
>> }
>>
>> @@ -490,6 +500,7 @@ int mali_c55_register_isp(struct mali_c55 *mali_c55)
>> MEDIA_PAD_FL_MUST_CONNECT;
>> isp->pads[MALI_C55_ISP_PAD_SOURCE_VIDEO].flags = MEDIA_PAD_FL_SOURCE;
>> isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE;
>> + isp->pads[MALI_C55_ISP_PAD_SOURCE_STATS].flags = MEDIA_PAD_FL_SOURCE;
>>
>> ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS,
>> isp->pads);
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>> index 0a391f8a043e..e72e749b81d5 100644
>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>> @@ -103,6 +103,9 @@ enum mali_c55_interrupts {
>> #define MALI_C55_VC_START(v) ((v) & 0xffff)
>> #define MALI_C55_VC_SIZE(v) (((v) & 0xffff) << 16)
>>
>> +#define MALI_C55_REG_1024BIN_HIST 0x054a8
>> +#define MALI_C55_1024BIN_HIST_SIZE 4096
>> +
>> /* Ping/Pong Configuration Space */
>> #define MALI_C55_REG_BASE_ADDR 0x18e88
>> #define MALI_C55_REG_BYPASS_0 0x18eac
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-stats.c b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
>> new file mode 100644
>> index 000000000000..38a17fb5d052
>> --- /dev/null
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
>> @@ -0,0 +1,373 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * ARM Mali-C55 ISP Driver - 3A Statistics capture device
>> + *
>> + * Copyright (C) 2023 Ideas on Board Oy
>> + */
>> +
>> +#include <linux/container_of.h>
>> +#include <linux/dev_printk.h>
>> +#include <linux/dmaengine.h>
>> +#include <linux/list.h>
>> +#include <linux/media/arm/mali-c55-config.h>
>> +#include <linux/mutex.h>
>> +#include <linux/spinlock.h>
>> +#include <linux/string.h>
>> +
>> +#include <media/media-entity.h>
>> +#include <media/v4l2-dev.h>
>> +#include <media/v4l2-event.h>
>> +#include <media/v4l2-fh.h>
>> +#include <media/v4l2-ioctl.h>
>> +#include <media/videobuf2-core.h>
>> +#include <media/videobuf2-dma-contig.h>
>> +
>> +#include "mali-c55-common.h"
>> +#include "mali-c55-registers.h"
>> +
>> +static const unsigned int metering_space_addrs[] = {
>> + [MALI_C55_CONFIG_PING] = 0x095ac,
>> + [MALI_C55_CONFIG_PONG] = 0x2156c,
>> +};
>> +
>> +static int mali_c55_stats_enum_fmt_meta_cap(struct file *file, void *fh,
>> + struct v4l2_fmtdesc *f)
>> +{
>> + if (f->index)
>> + return -EINVAL;
>> +
>> + f->pixelformat = V4L2_META_FMT_MALI_C55_STATS;
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_stats_g_fmt_meta_cap(struct file *file, void *fh,
>> + struct v4l2_format *f)
>> +{
>> + static const struct v4l2_meta_format mfmt = {
>> + .dataformat = V4L2_META_FMT_MALI_C55_STATS,
>> + .buffersize = sizeof(struct mali_c55_stats_buffer)
>> + };
>> +
>> + f->fmt.meta = mfmt;
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_stats_querycap(struct file *file,
>> + void *priv, struct v4l2_capability *cap)
>> +{
>> + strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver));
>> + strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card));
>> +
>> + return 0;
>> +}
>> +
>> +static const struct v4l2_ioctl_ops mali_c55_stats_v4l2_ioctl_ops = {
>> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
>> + .vidioc_querybuf = vb2_ioctl_querybuf,
>> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
>> + .vidioc_qbuf = vb2_ioctl_qbuf,
>> + .vidioc_expbuf = vb2_ioctl_expbuf,
>> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
>> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>> + .vidioc_streamon = vb2_ioctl_streamon,
>> + .vidioc_streamoff = vb2_ioctl_streamoff,
>> + .vidioc_enum_fmt_meta_cap = mali_c55_stats_enum_fmt_meta_cap,
>> + .vidioc_g_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
>> + .vidioc_s_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
>> + .vidioc_try_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
>> + .vidioc_querycap = mali_c55_stats_querycap,
>> + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
>> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
>> +};
>> +
>> +static const struct v4l2_file_operations mali_c55_stats_v4l2_fops = {
>> + .owner = THIS_MODULE,
>> + .unlocked_ioctl = video_ioctl2,
>> + .open = v4l2_fh_open,
>> + .release = vb2_fop_release,
>> + .poll = vb2_fop_poll,
>> + .mmap = vb2_fop_mmap,
>> +};
>> +
>> +static int
>> +mali_c55_stats_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
>> + unsigned int *num_planes, unsigned int sizes[],
>> + struct device *alloc_devs[])
>> +{
>> + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
>> +
>> + if (*num_planes && *num_planes > 1)
>> + return -EINVAL;
>> +
>> + if (sizes[0] && sizes[0] < sizeof(struct mali_c55_stats_buffer))
>> + return -EINVAL;
>> +
>> + *num_planes = 1;
>> +
>> + if (!sizes[0])
>> + sizes[0] = sizeof(struct mali_c55_stats_buffer);
>> +
>> + if (stats->channel)
>> + alloc_devs[0] = stats->channel->device->dev;
>> +
>> + return 0;
>> +}
>> +
>> +static void mali_c55_stats_buf_queue(struct vb2_buffer *vb)
>> +{
>> + struct mali_c55_stats *stats = vb2_get_drv_priv(vb->vb2_queue);
>> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>> + struct mali_c55_stats_buf *buf = container_of(vbuf,
>> + struct mali_c55_stats_buf, vb);
>> +
>> + vb2_set_plane_payload(vb, 0, sizeof(struct mali_c55_stats_buffer));
>> + buf->segments_remaining = 2;
>> + buf->failed = false;
>> +
>> + spin_lock(&stats->buffers.lock);
>> + list_add_tail(&buf->queue, &stats->buffers.queue);
>> + spin_unlock(&stats->buffers.lock);
>> +}
>> +
>> +static int mali_c55_stats_start_streaming(struct vb2_queue *q,
>> + unsigned int count)
>> +{
>> + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
>> + struct mali_c55 *mali_c55 = stats->mali_c55;
>> + int ret;
>> +
>> + ret = video_device_pipeline_start(&stats->vdev,
>> + &stats->mali_c55->pipe);
>> + if (ret)
>> + return ret;
>> +
>> + if (mali_c55->pipe.start_count == mali_c55->pipe.required_count) {
>> + ret = v4l2_subdev_enable_streams(&mali_c55->isp.sd,
>> + MALI_C55_ISP_PAD_SOURCE_VIDEO,
>> + BIT(0));
>> + if (ret)
>> + goto err_stop_pipeline;
>> + }
>> +
>> + return 0;
>> +
>> +err_stop_pipeline:
>> + video_device_pipeline_stop(&stats->vdev);
>> +
>> + return ret;
>> +}
>> +
>> +static void mali_c55_stats_stop_streaming(struct vb2_queue *q)
>> +{
>> + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
>> + struct mali_c55_stats_buf *buf, *tmp;
>> +
>> + dmaengine_terminate_sync(stats->channel);
>> +
>> + spin_lock(&stats->buffers.lock);
>> +
>> + list_for_each_entry_safe(buf, tmp, &stats->buffers.queue, queue) {
>> + list_del(&buf->queue);
>> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
>> + }
>> +
>> + spin_unlock(&stats->buffers.lock);
>> +
>> + video_device_pipeline_stop(&stats->vdev);
>> +}
>> +
>> +static const struct vb2_ops mali_c55_stats_vb2_ops = {
>> + .queue_setup = mali_c55_stats_queue_setup,
>> + .buf_queue = mali_c55_stats_buf_queue,
>> + .wait_prepare = vb2_ops_wait_prepare,
>> + .wait_finish = vb2_ops_wait_finish,
>> + .start_streaming = mali_c55_stats_start_streaming,
>> + .stop_streaming = mali_c55_stats_stop_streaming,
>> +};
>> +
>> +static void
>> +mali_c55_stats_metering_complete(void *param,
>> + const struct dmaengine_result *result)
>> +{
>> + struct mali_c55_stats_buf *buf = param;
>> +
>> + if (result->result != DMA_TRANS_NOERROR)
>> + buf->failed = true;
>> +
>> + if (!--buf->segments_remaining)
>> + vb2_buffer_done(&buf->vb.vb2_buf, buf->failed ?
>> + VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE);
>> +}
>> +
>> +static int mali_c55_stats_dma_xfer(struct mali_c55_stats *stats, dma_addr_t src,
>> + dma_addr_t dst,
>> + struct mali_c55_stats_buf *buf,
>> + size_t length)
>> +{
>> + struct dma_async_tx_descriptor *tx;
>> + dma_cookie_t cookie;
>> +
>> + tx = dmaengine_prep_dma_memcpy(stats->channel, dst, src, length, 0);
>> + if (!tx) {
>> + dev_err(stats->mali_c55->dev, "failed to prep stats DMA\n");
>> + return -EIO;
>> + }
>> +
>> + tx->callback_result = mali_c55_stats_metering_complete;
>> + tx->callback_param = buf;
>> +
>> + cookie = dmaengine_submit(tx);
>> + if (dma_submit_error(cookie)) {
>> + dev_err(stats->mali_c55->dev, "failed to submit stats DMA\n");
>> + return -EIO;
>> + }
>> +
>> + dma_async_issue_pending(stats->channel);
>> + return 0;
>> +}
>> +
>> +void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
>> + enum mali_c55_config_spaces cfg_space)
>> +{
>> + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
>> + struct mali_c55_stats *stats = &mali_c55->stats;
>> + struct mali_c55_stats_buf *buf = NULL;
>> + dma_addr_t src, dst;
>> + size_t length;
>> + int ret;
>> +
>> + spin_lock(&stats->buffers.lock);
>> + if (!list_empty(&stats->buffers.queue)) {
>> + buf = list_first_entry(&stats->buffers.queue,
>> + struct mali_c55_stats_buf, queue);
>> + list_del(&buf->queue);
>> + }
>> + spin_unlock(&stats->buffers.lock);
>> +
>> + if (!buf)
>> + return;
>> +
>> + buf->vb.sequence = mali_c55->isp.frame_sequence;
>> + buf->vb.vb2_buf.timestamp = ktime_get_boottime_ns();
>> +
>> + /*
>> + * There are in fact two noncontiguous sections of the ISP's
>> + * memory space that hold statistics for 3a algorithms to use: A
>> + * section in each config space and a global section holding
>> + * histograms which is double buffered and so holds data for the
>> + * last frame. We need to read both.
>> + */
>> + src = ctx->base + MALI_C55_REG_1024BIN_HIST;
>> + dst = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
>> +
>> + ret = mali_c55_stats_dma_xfer(stats, src, dst, buf,
>> + MALI_C55_1024BIN_HIST_SIZE);
>> + if (ret)
>> + goto err_fail_buffer;
>> +
>> + src = ctx->base + metering_space_addrs[cfg_space];
>> + dst += MALI_C55_1024BIN_HIST_SIZE;
>> +
>> + length = sizeof(struct mali_c55_stats_buffer) - MALI_C55_1024BIN_HIST_SIZE;
>> + ret = mali_c55_stats_dma_xfer(stats, src, dst, buf, length);
>> + if (ret) {
>> + dmaengine_terminate_sync(stats->channel);
>> + goto err_fail_buffer;
>> + }
>> +
>> + return;
>> +
>> +err_fail_buffer:
>> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
>> +}
>> +
>> +void mali_c55_unregister_stats(struct mali_c55 *mali_c55)
>> +{
>> + struct mali_c55_stats *stats = &mali_c55->stats;
>> +
>> + if (!video_is_registered(&stats->vdev))
>> + return;
>> +
>> + vb2_video_unregister_device(&stats->vdev);
>> + media_entity_cleanup(&stats->vdev.entity);
>> + dma_release_channel(stats->channel);
>> + mutex_destroy(&stats->lock);
>> +}
>> +
>> +int mali_c55_register_stats(struct mali_c55 *mali_c55)
>> +{
>> + struct mali_c55_stats *stats = &mali_c55->stats;
>> + struct video_device *vdev = &stats->vdev;
>> + struct vb2_queue *vb2q = &stats->queue;
>> + dma_cap_mask_t mask;
>> + int ret;
>> +
>> + mutex_init(&stats->lock);
>> + INIT_LIST_HEAD(&stats->buffers.queue);
>> +
>> + dma_cap_zero(mask);
>> + dma_cap_set(DMA_MEMCPY, mask);
>> +
>> + stats->channel = dma_request_channel(mask, 0, NULL);
>> + if (!stats->channel) {
>> + ret = -ENODEV;
>> + goto err_destroy_mutex;
>> + }
>> +
>> + stats->pad.flags = MEDIA_PAD_FL_SINK;
>> + ret = media_entity_pads_init(&stats->vdev.entity, 1, &stats->pad);
>> + if (ret)
>> + goto err_release_dma_channel;
>> +
>> + vb2q->type = V4L2_BUF_TYPE_META_CAPTURE;
>> + vb2q->io_modes = VB2_MMAP | VB2_DMABUF;
>> + vb2q->drv_priv = stats;
>> + vb2q->mem_ops = &vb2_dma_contig_memops;
>> + vb2q->ops = &mali_c55_stats_vb2_ops;
>> + vb2q->buf_struct_size = sizeof(struct mali_c55_stats_buf);
>> + vb2q->min_queued_buffers = 1;
>> + vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
>> + vb2q->lock = &stats->lock;
>> + vb2q->dev = stats->channel->device->dev;
>> +
>> + ret = vb2_queue_init(vb2q);
>> + if (ret) {
>> + dev_err(mali_c55->dev, "stats vb2 queue init failed\n");
>> + goto err_cleanup_entity;
>> + }
>> +
>> + strscpy(stats->vdev.name, "mali-c55 3a stats", sizeof(stats->vdev.name));
>> + vdev->release = video_device_release_empty;
>> + vdev->fops = &mali_c55_stats_v4l2_fops;
>> + vdev->ioctl_ops = &mali_c55_stats_v4l2_ioctl_ops;
>> + vdev->lock = &stats->lock;
>> + vdev->v4l2_dev = &mali_c55->v4l2_dev;
>> + vdev->queue = &stats->queue;
>> + vdev->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
>> + vdev->vfl_dir = VFL_DIR_RX;
>> + video_set_drvdata(vdev, stats);
>> +
>> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>> + if (ret) {
>> + dev_err(mali_c55->dev,
>> + "failed to register stats video device\n");
>> + goto err_release_vb2q;
>> + }
>> +
>> + stats->mali_c55 = mali_c55;
>> +
>> + return 0;
>> +
>> +err_release_vb2q:
>> + vb2_queue_release(vb2q);
>> +err_cleanup_entity:
>> + media_entity_cleanup(&stats->vdev.entity);
>> +err_release_dma_channel:
>> + dma_release_channel(stats->channel);
>> +err_destroy_mutex:
>> + mutex_destroy(&stats->lock);
>> +
>> + return ret;
>> +}
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 16/18] media: uapi: Add parameters structs to mali-c55-config.h
2024-07-29 10:51 ` Dan Scally
@ 2024-07-29 11:03 ` Laurent Pinchart
0 siblings, 0 replies; 41+ messages in thread
From: Laurent Pinchart @ 2024-07-29 11:03 UTC (permalink / raw)
To: Dan Scally
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Dan,
On Mon, Jul 29, 2024 at 11:51:43AM +0100, Daniel Scally wrote:
> On 23/07/2024 00:48, Laurent Pinchart wrote:
> > On Tue, Jul 09, 2024 at 02:29:04PM +0100, Daniel Scally wrote:
> >> Add structures describing the ISP parameters to mali-c55-config.h
> >>
> >> Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
> >> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> >> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> >> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
> >> ---
> >> Changes in v6:
> >>
> >> - Flagged which struct goes with which enum value from
> >> enum mali_c55_param_block_type
> >> - Used only types with well defined sizes in the structs
> >> - Expanded the documentation for skipping in the AEXP histogram config
> >> - Aligned the header struct to 64 bits
> >> - Added a new union type to hold pointers to the parameter structs
> >>
> >> Changes in v5:
> >>
> >> - New patch
> >>
> >> .../uapi/linux/media/arm/mali-c55-config.h | 763 ++++++++++++++++++
> >> 1 file changed, 763 insertions(+)
> >>
> >> diff --git a/include/uapi/linux/media/arm/mali-c55-config.h b/include/uapi/linux/media/arm/mali-c55-config.h
> >> index 21b453bdee5d..ea9872e9076f 100644
> >> --- a/include/uapi/linux/media/arm/mali-c55-config.h
> >> +++ b/include/uapi/linux/media/arm/mali-c55-config.h
> >> @@ -179,4 +179,767 @@ struct mali_c55_stats_buffer {
> >> __u32 reserved3[15];
> >> } __attribute__((packed));
> >>
> >> +/**
> >> + * enum mali_c55_param_buffer_version - Mali-C55 parameters block versioning
> >> + *
> >> + * @MALI_C55_PARAM_BUFFER_V1: First version of Mali-C55 parameters block
> >> + */
> >> +enum mali_c55_param_buffer_version {
> >> + MALI_C55_PARAM_BUFFER_V1,
> >> +};
> >> +
> >> +/**
> >> + * enum mali_c55_param_block_type - Enumeration of Mali-C55 parameter blocks
> >> + *
> >> + * This enumeration defines the types of Mali-C55 parameters block. Each block
> >> + * configures a specific processing block of the Mali-C55 ISP. The block
> >> + * type allows the driver to correctly interpret the parameters block data.
> >> + *
> >> + * It is the responsibility of userspace to correctly set the type of each
> >> + * parameters block.
> >> + *
> >> + * @MALI_C55_PARAM_BLOCK_SENSOR_OFFS: Sensor pre-shading black level offset
> >> + * @MALI_C55_PARAM_BLOCK_AEXP_HIST: Auto-exposure 1024-bin histogram
> >> + * configuration
> >> + * @MALI_C55_PARAM_BLOCK_AEXP_IHIST: Post-Iridix auto-exposure 1024-bin
> >> + * histogram configuration
> >> + * @MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS: Auto-exposure 1024-bin histogram
> >> + * weighting
> >> + * @MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS: Post-Iridix auto-exposure 1024-bin
> >> + * histogram weighting
> >> + * @MALI_C55_PARAM_BLOCK_DIGITAL_GAIN: Digital gain
> >> + * @MALI_C55_PARAM_BLOCK_AWB_GAINS: Auto-white balance gains
> >> + * @MALI_C55_PARAM_BLOCK_AWB_CONFIG: Auto-white balance statistics config
> >> + * @MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP: Auto-white balance gains for AEXP-0 tap
> >> + * @MALI_C55_PARAM_MESH_SHADING_CONFIG : Mesh shading tables configuration
> >> + * @MALI_C55_PARAM_MESH_SHADING_SELECTION: Mesh shading table selection
> >> + * @MALI_C55_PARAM_BLOCK_SENTINEL: First non-valid block index
> >
> > The sentinel value will change when adding new types. That may affect
> > userspace. We managed to get rid of the sentinel in the rkisp1 driver,
> > it would be nice to do the same here.
>
> Ack - I'm sure it'll be possible.
>
> >> + */
> >> +enum mali_c55_param_block_type {
> >> + MALI_C55_PARAM_BLOCK_SENSOR_OFFS,
> >> + MALI_C55_PARAM_BLOCK_AEXP_HIST,
> >> + MALI_C55_PARAM_BLOCK_AEXP_IHIST,
> >> + MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS,
> >> + MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS,
> >> + MALI_C55_PARAM_BLOCK_DIGITAL_GAIN,
> >> + MALI_C55_PARAM_BLOCK_AWB_GAINS,
> >> + MALI_C55_PARAM_BLOCK_AWB_CONFIG,
> >> + MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP,
> >> + MALI_C55_PARAM_MESH_SHADING_CONFIG,
> >> + MALI_C55_PARAM_MESH_SHADING_SELECTION,
> >> + MALI_C55_PARAM_BLOCK_SENTINEL,
> >> +};
> >> +
> >> +/**
> >> + * struct mali_c55_params_block_header - Mali-C55 parameter block header
> >> + *
> >> + * This structure represents the common part of all the ISP configuration
> >> + * blocks. Each parameters block embeds an instance of this structure type
> >> + * as its first member, followed by the block-specific configuration data. The
> >> + * driver inspects this common header to discern the block type and its size and
> >> + * properly handle the block content by casting it to the correct block-specific
> >> + * type.
> >> + *
> >> + * The @type field is one of the values enumerated by
> >> + * :c:type:`mali_c55_param_block_type` and specifies how the data should be
> >> + * interpreted by the driver. The @size field specifies the size of the
> >> + * parameters block and is used by the driver for validation purposes. The
> >> + * @enabled field specifies if the ISP block should be enabled (and configured
> >> + * according to the provided parameters) or disabled.
> >> + *
> >> + * Userspace is responsible for correctly populating the parameters block header
> >> + * fields (@type, @enabled and @size) and correctly populate the block-specific
> >> + * parameters.
> >> + *
> >> + * For example:
> >> + *
> >> + * .. code-block:: c
> >> + *
> >> + * void populate_sensor_offs(struct mali_c55_params_block_header *block) {
> >> + * block->type = MALI_C55_PARAM_BLOCK_SENSOR_OFFS;
> >> + * block->enabled = true;
> >> + * block->size = sizeof(struct mali_c55_params_sensor_off_preshading);
> >> + *
> >> + * struct mali_c55_params_sensor_off_preshading *sensor_offs =
> >> + * (struct mali_c55_params_sensor_off_preshading *)block;
> >> + *
> >> + * sensor_offs->chan00 = offset00;
> >> + * sensor_offs->chan01 = offset01;
> >> + * sensor_offs->chan10 = offset10;
> >> + * sensor_offs->chan11 = offset11;
> >> + * }
> >> + *
> >> + * @type: The parameters block type from :c:type:`mali_c55_param_block_type`
> >> + * @enabled: Block enabled/disabled flag
> >> + * @size: Size (in bytes) of the parameters block
> >> + */
> >> +struct mali_c55_params_block_header {
> >> + __u8 type;
> >> + __u8 enabled;
> >> + __u32 size;
> >> +} __attribute__((aligned(8)));
> >> +
> >> +/**
> >> + * struct mali_c55_params_sensor_off_preshading - offset subtraction for each
> >> + * color channel
> >> + *
> >> + * Provides removal of the sensor black level from the sensor data. Separate
> >> + * offsets are provided for each of the four Bayer component color channels
> >> + * which are defaulted to R, Gr, Gb, B.
> >> + *
> >> + * header.type should be set to MALI_C55_PARAM_BLOCK_SENSOR_OFFS from
> >> + * :c:type:`mali_c55_param_block_type` for this block.
> >> + *
> >> + * @header: The Mali-C55 parameters block header
> >> + * @chan00: Offset for color channel 00 (default: R)
> >> + * @chan01: Offset for color channel 01 (default: Gr)
> >> + * @chan10: Offset for color channel 10 (default: Gb)
> >> + * @chan11: Offset for color channel 11 (default: B)
> >> + */
> >> +struct mali_c55_params_sensor_off_preshading {
> >> + struct mali_c55_params_block_header header;
> >> + __u32 chan00;
> >> + __u32 chan01;
> >> + __u32 chan10;
> >> + __u32 chan11;
> >> +};
> >> +
> >> +/**
> >> + * enum mali_c55_aexp_hist_tap_points - Tap points for the AEXP histogram
> >> + * @MALI_C55_AEXP_HIST_TAP_WB: After static white balance
> >> + * @MALI_C55_AEXP_HIST_TAP_FS: After WDR Frame Stitch
> >> + * @MALI_C55_AEXP_HIST_TAP_TPG: After the test pattern generator
> >> + */
> >> +enum mali_c55_aexp_hist_tap_points {
> >> + MALI_C55_AEXP_HIST_TAP_WB = 0,
> >> + MALI_C55_AEXP_HIST_TAP_FS,
> >> + MALI_C55_AEXP_HIST_TAP_TPG,
> >> +};
> >> +
> >> +/**
> >> + * enum mali_c55_aexp_skip_x - Horizontal pixel skipping
> >> + * @MALI_C55_AEXP_SKIP_X_EVERY_2ND: Collect every 2nd pixel horizontally
> >> + * @MALI_C55_AEXP_SKIP_X_EVERY_3RD: Collect every 3rd pixel horizontally
> >> + * @MALI_C55_AEXP_SKIP_X_EVERY_4TH: Collect every 4th pixel horizontally
> >> + * @MALI_C55_AEXP_SKIP_X_EVERY_5TH: Collect every 5th pixel horizontally
> >> + * @MALI_C55_AEXP_SKIP_X_EVERY_8TH: Collect every 8th pixel horizontally
> >> + * @MALI_C55_AEXP_SKIP_X_EVERY_9TH: Collect every 9th pixel horizontally
> >> + */
> >> +enum mali_c55_aexp_skip_x {
> >> + MALI_C55_AEXP_SKIP_X_EVERY_2ND,
> >> + MALI_C55_AEXP_SKIP_X_EVERY_3RD,
> >> + MALI_C55_AEXP_SKIP_X_EVERY_4TH,
> >> + MALI_C55_AEXP_SKIP_X_EVERY_5TH,
> >> + MALI_C55_AEXP_SKIP_X_EVERY_8TH,
> >> + MALI_C55_AEXP_SKIP_X_EVERY_9TH
> >> +};
> >> +
> >> +/**
> >> + * enum mali_c55_aexp_skip_y - Vertical pixel skipping
> >> + * @MALI_C55_AEXP_SKIP_Y_ALL: Collect every single pixel vertically
> >> + * @MALI_C55_AEXP_SKIP_Y_EVERY_2ND: Collect every 2nd pixel vertically
> >> + * @MALI_C55_AEXP_SKIP_Y_EVERY_3RD: Collect every 3rd pixel vertically
> >> + * @MALI_C55_AEXP_SKIP_Y_EVERY_4TH: Collect every 4th pixel vertically
> >> + * @MALI_C55_AEXP_SKIP_Y_EVERY_5TH: Collect every 5th pixel vertically
> >> + * @MALI_C55_AEXP_SKIP_Y_EVERY_8TH: Collect every 8th pixel vertically
> >> + * @MALI_C55_AEXP_SKIP_Y_EVERY_9TH: Collect every 9th pixel vertically
> >> + */
> >> +enum mali_c55_aexp_skip_y {
> >> + MALI_C55_AEXP_SKIP_Y_ALL,
> >> + MALI_C55_AEXP_SKIP_Y_EVERY_2ND,
> >> + MALI_C55_AEXP_SKIP_Y_EVERY_3RD,
> >> + MALI_C55_AEXP_SKIP_Y_EVERY_4TH,
> >> + MALI_C55_AEXP_SKIP_Y_EVERY_5TH,
> >> + MALI_C55_AEXP_SKIP_Y_EVERY_8TH,
> >> + MALI_C55_AEXP_SKIP_Y_EVERY_9TH
> >> +};
> >> +
> >> +/**
> >> + * enum mali_c55_aexp_row_column_offset - Start from the first or second row or
> >> + * column
> >> + * @MALI_C55_AEXP_FIRST_ROW_OR_COL: Start from the first row / column
> >> + * @MALI_C55_AEXP_SECOND_ROW_OR_COL: Start from the second row / column
> >> + */
> >> +enum mali_c55_aexp_row_column_offset {
> >> + MALI_C55_AEXP_FIRST_ROW_OR_COL = 1,
> >> + MALI_C55_AEXP_SECOND_ROW_OR_COL = 2,
> >> +};
> >> +
> >> +/**
> >> + * enum mali_c55_aexp_hist_plane_mode - Mode for the AEXP Histograms
> >> + * @MALI_C55_AEXP_HIST_COMBINED: All color planes in one 1024-bin histogram
> >> + * @MALI_C55_AEXP_HIST_SEPARATE: Each color plane in one 256-bin histogram with a bin width of 16
> >> + * @MALI_C55_AEXP_HIST_FOCUS_00: Top left plane in the first bank, rest in second bank
> >> + * @MALI_C55_AEXP_HIST_FOCUS_01: Top right plane in the first bank, rest in second bank
> >> + * @MALI_C55_AEXP_HIST_FOCUS_10: Bottom left plane in the first bank, rest in second bank
> >> + * @MALI_C55_AEXP_HIST_FOCUS_11: Bottom right plane in the first bank, rest in second bank
> >> + *
> >> + * In the "focus" modes statistics are collected into two 512-bin histograms
> >> + * with a bin width of 8. One colour plane is in the first histogram with the
> >> + * remainder combined into the second. The four options represent which of the
> >> + * four positions in a bayer pattern are the focused plane.
> >> + */
> >> +enum mali_c55_aexp_hist_plane_mode {
> >> + MALI_C55_AEXP_HIST_COMBINED = 0,
> >> + MALI_C55_AEXP_HIST_SEPARATE = 1,
> >> + MALI_C55_AEXP_HIST_FOCUS_00 = 4,
> >> + MALI_C55_AEXP_HIST_FOCUS_01 = 5,
> >> + MALI_C55_AEXP_HIST_FOCUS_10 = 6,
> >> + MALI_C55_AEXP_HIST_FOCUS_11 = 7,
> >> +};
> >> +
> >> +/**
> >> + * struct mali_c55_params_aexp_hist - configuration for AEXP metering hists
> >> + *
> >> + * This struct allows users to configure the 1024-bin AEXP histograms. Broadly
> >> + * speaking the parameters allow you to mask particular regions of the image and
> >> + * to select different kinds of histogram.
> >> + *
> >> + * The skip_x, offset_x, skip_y and offset_y fields allow users to ignore or
> >> + * mask pixels in the frame by their position relative to the top left pixel.
> >> + * First, the skip_y, offset_x and offset_y fields define a 2x2 pixel pattern of
> >> + * which pixels covered by the pattern will be counted in the statistics.
> >
> > I have a really hard time parsing the second sentence. Let me try to
> > rephrase it.
> >
> > * First, the skip_y, offset_x and offset_y fields define which pixels will be
> > * counted in the statistics within a 2x2 pixels region.
> >
> > Is that what you meant ?
>
> Yes, although I feel it's a bit iffy...perhaps "...which of the pixels within each 2x2 region will
> be counted in the statistics"?
Sounds good to me.
> >> + *
> >> + * If skip_y == 0 then two pixels from each coveredregion will be counted. If
> >
> > s/coveredregion/region/
> >
> >> + * both offset_x and offset_y are zero, then the two left-most pixels in each
> >> + * 2x2 pixel region covered by the pattern will be counted. Setting offset_x = 1
> >
> > I'd drop "covered by the pattern" as I'm not sure what that means.
>
> OK, I think it reads fine that way.
>
> >> + * will discount the top left pixel and count the top right pixel. Setting
> >> + * offset_y = 1 will discount the bottom left pixel and count the bottom right
> >> + * pixel.
> >
> > That sounds strange, I would have expected offset_y to control something
> > vertically, not control the horizontal position on the second line. If
> > that's how the hardware operates, why not.
>
> Yes it is strange, but that's what the documentation says.
>
> >> + *
> >> + * If skip_y != 0 then only a single pixel from each region covered by the
> >> + * pattern will be counted. In this case offset_x controls whether the pixel
> >> + * that's counted is in the left (if offset_x == 0) or right (if offset_x == 1)
> >> + * column and offset_y controls whether the pixel that's counted is in the top
> >> + * (if offset_y == 0) or bottom (if offset_y == 1) row.
> >> + *
> >> + * The skip_x and skip_y fields control how the pixel masking pattern is
> >> + * repeated across the image data. The first instance of the pattern is always
> >
> > And here maybe "control how the 2x2 pixel region is repeated ...".
> > s/pattern/region/ in the rest.
>
> Okedokey
>
> > This is otherwise a great explanation, thank you.
>
> Thanks!
>
> >> + * in the top left of the image data. The skip_x field controls how many pixels
> >> + * are ignored in the x direction before the pixel masking pattern is repeated.
> >> + * The skip_y field controls how many pixels are ignored in the y direction
> >> + * before the pixel masking pattern is repeated.
> >> + *
> >> + * These fields can be used to reduce the number of pixels counted for the
> >> + * statistics, but it's important to be careful to configure them correctly.
> >> + * Some combinations of values will result in colour components from the input
> >> + * data being ignored entirely, for example in the following configuration:
> >> + *
> >> + * skip_x = 0
> >> + * offset_x = 0
> >> + * skip_y = 0
> >> + * offset_y = 0
> >> + *
> >> + * Only the R and Gb components of RGGB data that was input would be collected.
> >> + * Similarly in the following configuration:
> >> + *
> >> + * skip_x = 0
> >> + * offset_x = 0
> >> + * skip_y = 1
> >> + * offset_y = 1
> >> + *
> >> + * Only the Gb component of RGGB data that was input would be collected. To
> >> + * correct things such that all 4 colour components were included it would be
> >> + * necessary to set the skip_x and skip_y fields in a way that resulted in all
> >> + * four colour components being collected:
> >> + *
> >> + * skip_x = 1
> >> + * offset_x = 0
> >> + * skip_y = 1
> >> + * offset_y = 1
> >> + *
> >> + * header.type should be set to one of either MALI_C55_PARAM_BLOCK_AEXP_HIST or
> >> + * MALI_C55_PARAM_BLOCK_AEXP_IHIST from :c:type:`mali_c55_param_block_type`.
> >> + *
> >> + * @header: The Mali-C55 parameters block header
> >> + * @skip_x: Horizontal decimation. See enum mali_c55_aexp_skip_x
> >> + * @offset_x: Skip the first column, or not. See enum mali_c55_aexp_row_column_offset
> >> + * @skip_y: Vertical decimation. See enum mali_c55_aexp_skip_y
> >> + * @offset_y: Skip the first row, or not. See enum mali_c55_aexp_row_column_offset
> >> + * @scale_bottom: Scale pixels in bottom half of intensity range: 0=1x ,1=2x, 2=4x, 4=8x, 4=16x
> >> + * @scale_top: scale pixels in top half of intensity range: 0=1x ,1=2x, 2=4x, 4=8x, 4=16x
> >> + * @plane_mode: Plane separation mode. See enum mali_c55_aexp_hist_plane_mode
> >> + * @tap_point: Tap point for histogram from enum mali_c55_aexp_hist_tap_points.
> >> + * This parameter is unused for the post-Iridix Histogram
> >> + */
> >> +struct mali_c55_params_aexp_hist {
> >> + struct mali_c55_params_block_header header;
> >> + __u8 skip_x;
> >> + __u8 offset_x;
> >> + __u8 skip_y;
> >> + __u8 offset_y;
> >> + __u8 scale_bottom;
> >> + __u8 scale_top;
> >> + __u8 plane_mode;
> >> + __u8 tap_point;
> >> +};
> >> +
> >> +/**
> >> + * struct mali_c55_params_aexp_weights - Array of weights for AEXP metering
> >> + *
> >> + * This struct allows users to configure the weighting for both of the 1024-bin
> >> + * AEXP histograms. The pixel data collected for each zone is multiplied by the
> >> + * corresponding weight from this array, which may be zero if the intention is
> >> + * to mask off the zone entirely.
> >> + *
> >> + * header.type should be set to one of either MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS
> >> + * or MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS from :c:type:`mali_c55_param_block_type`.
> >> + *
> >> + * @header: The Mali-C55 parameters block header
> >> + * @nodes_used_horiz: Number of active zones horizontally [0..15]
> >> + * @nodes_used_vert: Number of active zones vertically [0..15]
> >> + * @zone_weights: Zone weighting. Index is row*col where 0,0 is the top
> >> + * left zone continuing in raster order. Each zone can be
> >> + * weighted in the range [0..15]. The number of rows and
> >> + * columns is defined by @nodes_used_vert and
> >> + * @nodes_used_horiz
> >> + */
> >> +struct mali_c55_params_aexp_weights {
> >> + struct mali_c55_params_block_header header;
> >> + __u8 nodes_used_horiz;
> >> + __u8 nodes_used_vert;
> >> + __u8 zone_weights[MALI_C55_MAX_ZONES];
> >> +};
> >> +
> >> +/**
> >> + * struct mali_c55_params_digital_gain - Digital gain value
> >> + *
> >> + * This struct carries a digital gain value to set in the ISP.
> >> + *
> >> + * header.type should be set to MALI_C55_PARAM_BLOCK_DIGITAL_GAIN from
> >> + * :c:type:`mali_c55_param_block_type` for this block.
> >> + *
> >> + * @header: The Mali-C55 parameters block header
> >> + * @gain: The digital gain value to apply, in Q5.8 format.
> >> + */
> >> +struct mali_c55_params_digital_gain {
> >> + struct mali_c55_params_block_header header;
> >> + __u16 gain;
> >> +};
> >> +
> >> +/**
> >> + * enum mali_c55_awb_stats_mode - Statistics mode for AWB
> >> + * @MALI_C55_AWB_MODE_GRBR: Statistics collected as Green/Red and Blue/Red ratios
> >> + * @MALI_C55_AWB_MODE_RGBG: Statistics collected as Red/Green and Blue/Green ratios
> >> + */
> >> +enum mali_c55_awb_stats_mode {
> >> + MALI_C55_AWB_MODE_GRBR = 0,
> >> + MALI_C55_AWB_MODE_RGBG,
> >> +};
> >> +
> >> +/**
> >> + * struct mali_c55_params_awb_gains - Gain settings for auto white balance
> >> + *
> >> + * This struct allows users to configure the gains for auto-white balance. There
> >> + * are four gain settings corresponding to each colour channel in the bayer
> >> + * domain. Although named generically, the association between the gain applied
> >> + * and the colour channel is done automatically within the ISP depending on the
> >> + * input format, and so the following mapping always holds true::
> >> + *
> >> + * gain00 = R
> >> + * gain01 = Gr
> >> + * gain10 = Gb
> >> + * gain11 = B
> >> + *
> >> + * All of the gains are stored in Q4.8 format.
> >> + *
> >> + * header.type should be set to one of either MALI_C55_PARAM_BLOCK_AWB_GAINS or
> >> + * MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP from :c:type:`mali_c55_param_block_type`.
> >> + *
> >> + * @header: The Mali-C55 parameters block header
> >> + * @gain00: Multiplier for colour channel 00
> >> + * @gain01: Multiplier for colour channel 01
> >> + * @gain10: Multiplier for colour channel 10
> >> + * @gain11: Multiplier for colour channel 11
> >> + */
> >> +struct mali_c55_params_awb_gains {
> >> + struct mali_c55_params_block_header header;
> >> + __u16 gain00;
> >> + __u16 gain01;
> >> + __u16 gain10;
> >> + __u16 gain11;
> >> +};
> >> +
> >> +/**
> >> + * enum mali_c55_params_awb_tap_points - Tap points for the AWB statistics
> >> + * @MALI_C55_AWB_STATS_TAP_PF: Immediately after the Purple Fringe block
> >> + * @MALI_C55_AWB_STATS_TAP_CNR: Immediately after the CNR block
> >> + */
> >> +enum mali_c55_params_awb_tap_points {
> >> + MALI_C55_AWB_STATS_TAP_PF = 0,
> >> + MALI_C55_AWB_STATS_TAP_CNR,
> >> +};
> >> +
> >> +/**
> >> + * struct mali_c55_params_awb_config - Stats settings for auto-white balance
> >> + *
> >> + * This struct allows the configuration of the statistics generated for auto
> >> + * white balance. Pixel intensity limits can be set to exclude overly bright or
> >> + * dark regions of an image from the statistics entirely. Colour ratio minima
> >> + * and maxima can be set to discount pixels who's ratios fall outside the
> >> + * defined boundaries; there are two sets of registers to do this - the
> >> + * "min/max" ratios which bound a region and the "high/low" ratios which further
> >> + * trim the upper and lower ratios. For example with the boundaries configured
> >> + * as follows, only pixels whos colour ratios falls into the region marked "A"
> >> + * would be counted::
> >> + *
> >> + * cr_high
> >> + * 2.0 | |
> >> + * | cb_max --> _________________________v_____
> >> + * 1.8 | | \ |
> >> + * | | \ |
> >> + * 1.6 | | \ |
> >> + * | | \ |
> >> + * c 1.4 | cb_low -->|\ A \|<-- cb_high
> >> + * b | | \ |
> >> + * 1.2 | | \ |
> >> + * r | | \ |
> >> + * a 1.0 | cb_min --> |____\_________________________|
> >> + * t | ^ ^ ^
> >> + * i 0.8 | | | |
> >> + * o | cr_min | cr_max
> >> + * s 0.6 | |
> >> + * | cr_low
> >> + * 0.4 |
> >> + * |
> >> + * 0.2 |
> >> + * |
> >> + * 0.0 |_______________________________________________________________
> >> + * 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0
> >> + * cr ratios
> >> + *
> >> + * header.type should be set to MALI_C55_PARAM_BLOCK_AWB_CONFIG from
> >> + * :c:type:`mali_c55_param_block_type` for this block.
> >> + *
> >> + * @header: The Mali-C55 parameters block header
> >> + * @tap_point: The tap point from enum mali_c55_params_awb_tap_points
> >> + * @stats_mode: AWB statistics collection mode, see :c:type:`mali_c55_awb_stats_mode`
> >> + * @white_level: Upper pixel intensity (I.E. raw pixel values) limit
> >> + * @black_level: Lower pixel intensity (I.E. raw pixel values) limit
> >> + * @cr_max: Maximum R/G ratio (Q4.8 format)
> >> + * @cr_min: Minimum R/G ratio (Q4.8 format)
> >> + * @cb_max: Maximum B/G ratio (Q4.8 format)
> >> + * @cb_min: Minimum B/G ratio (Q4.8 format)
> >> + * @nodes_used_horiz: Number of active zones horizontally [0..15]
> >> + * @nodes_used_vert: Number of active zones vertically [0..15]
> >> + * @cr_high: R/G ratio trim high (Q4.8 format)
> >> + * @cr_low: R/G ratio trim low (Q4.8 format)
> >> + * @cb_high: B/G ratio trim high (Q4.8 format)
> >> + * @cb_low: B/G ratio trim low (Q4.8 format)
> >> + */
> >> +struct mali_c55_params_awb_config {
> >> + struct mali_c55_params_block_header header;
> >> + __u8 tap_point;
> >> + __u8 stats_mode;
> >> + __u16 white_level;
> >> + __u16 black_level;
> >> + __u16 cr_max;
> >> + __u16 cr_min;
> >> + __u16 cb_max;
> >> + __u16 cb_min;
> >> + __u8 nodes_used_horiz;
> >> + __u8 nodes_used_vert;
> >> + __u16 cr_high;
> >> + __u16 cr_low;
> >> + __u16 cb_high;
> >> + __u16 cb_low;
> >> +};
> >> +
> >> +#define MALI_C55_NUM_MESH_SHADING_ELEMENTS 3072
> >> +
> >> +/**
> >> + * struct mali_c55_params_mesh_shading_config - Mesh shading configuration
> >> + *
> >> + * The mesh shading correction module allows programming a separate table of
> >> + * either 16x16 or 32x32 node coefficients for 3 different light sources. The
> >> + * final correction coefficients applied are computed by blending the
> >> + * coefficients from two tables together.
> >> + *
> >> + * A page of 1024 32-bit integers is associated to each colour channel, with
> >> + * pages stored consecutively in memory. Each 32-bit integer packs 3 8-bit
> >> + * correction coefficients for a single node, one for each of the three light
> >> + * sources. The 8 most significant bits are unused. The following table
> >> + * describes the layout::
> >> + *
> >> + * +----------- Page (Colour Plane) 0 -------------+
> >> + * | @mesh[i] | Mesh Point | Bits | Light Source |
> >> + * +-----------+------------+-------+--------------+
> >> + * | 0 | 0,0 | 16,23 | LS2 |
> >> + * | | | 08-15 | LS1 |
> >> + * | | | 00-07 | LS0 |
> >> + * +-----------+------------+-------+--------------+
> >> + * | 1 | 0,1 | 16,23 | LS2 |
> >> + * | | | 08-15 | LS1 |
> >> + * | | | 00-07 | LS0 |
> >> + * +-----------+------------+-------+--------------+
> >> + * | ... | ... | ... | ... |
> >> + * +-----------+------------+-------+--------------+
> >> + * | 1023 | 31,31 | 16,23 | LS2 |
> >> + * | | | 08-15 | LS1 |
> >> + * | | | 00-07 | LS0 |
> >> + * +----------- Page (Colour Plane) 1 -------------+
> >> + * | @mesh[i] | Mesh Point | Bits | Light Source |
> >> + * +-----------+------------+-------+--------------+
> >> + * | 1024 | 0,0 | 16,23 | LS2 |
> >> + * | | | 08-15 | LS1 |
> >> + * | | | 00-07 | LS0 |
> >> + * +-----------+------------+-------+--------------+
> >> + * | 1025 | 0,1 | 16,23 | LS2 |
> >> + * | | | 08-15 | LS1 |
> >> + * | | | 00-07 | LS0 |
> >> + * +-----------+------------+-------+--------------+
> >> + * | ... | ... | ... | ... |
> >> + * +-----------+------------+-------+--------------+
> >> + * | 2047 | 31,31 | 16,23 | LS2 |
> >> + * | | | 08-15 | LS1 |
> >> + * | | | 00-07 | LS0 |
> >> + * +----------- Page (Colour Plane) 2 -------------+
> >> + * | @mesh[i] | Mesh Point | Bits | Light Source |
> >> + * +-----------+------------+-------+--------------+
> >> + * | 2048 | 0,0 | 16,23 | LS2 |
> >> + * | | | 08-15 | LS1 |
> >> + * | | | 00-07 | LS0 |
> >> + * +-----------+------------+-------+--------------+
> >> + * | 2049 | 0,1 | 16,23 | LS2 |
> >> + * | | | 08-15 | LS1 |
> >> + * | | | 00-07 | LS0 |
> >> + * +-----------+------------+-------+--------------+
> >> + * | ... | ... | ... | ... |
> >> + * +-----------+------------+-------+--------------+
> >> + * | 3071 | 31,31 | 16,23 | LS2 |
> >> + * | | | 08-15 | LS1 |
> >> + * | | | 00-07 | LS0 |
> >> + * +-----------+------------+-------+--------------+
> >> + *
> >> + * The @mesh_scale member determines the precision and minimum and maximum gain.
> >> + * For example if @mesh_scale is 0 and therefore selects 0 - 2x gain, a value of
> >> + * 0 in a coefficient means 0.0 gain, a value of 128 means 1.0 gain and 255
> >> + * means 2.0 gain.
> >> + *
> >> + * header.type should be set to MALI_C55_PARAM_MESH_SHADING_CONFIG from
> >> + * :c:type:`mali_c55_param_block_type` for this block.
> >> + *
> >> + * @header: The Mali-C55 parameters block header
> >> + * @mesh_show: Output the mesh data rather than image data
> >> + * @mesh_scale: Set the precision and maximum gain range of mesh shading
> >> + * - 0 = 0-2x gain
> >> + * - 1 = 0-4x gain
> >> + * - 2 = 0-8x gain
> >> + * - 3 = 0-16x gain
> >> + * - 4 = 1-2x gain
> >> + * - 5 = 1-3x gain
> >> + * - 6 = 1-5x gain
> >> + * - 7 = 1-9x gain
> >> + * @mesh_page_r: Mesh page select for red colour plane [0..2]
> >> + * @mesh_page_g: Mesh page select for green colour plane [0..2]
> >> + * @mesh_page_b: Mesh page select for blue colour plane [0..2]
> >> + * @mesh_width: Number of horizontal nodes minus 1 [15,31]
> >> + * @mesh_height: Number of vertical nodes minus 1 [15,31]
> >> + * @mesh: Mesh shading correction tables
> >> + */
> >> +struct mali_c55_params_mesh_shading_config {
> >> + struct mali_c55_params_block_header header;
> >> + __u8 mesh_show;
> >> + __u8 mesh_scale;
> >> + __u8 mesh_page_r;
> >> + __u8 mesh_page_g;
> >> + __u8 mesh_page_b;
> >> + __u8 mesh_width;
> >> + __u8 mesh_height;
> >> + __u32 mesh[MALI_C55_NUM_MESH_SHADING_ELEMENTS];
> >> +};
> >> +
> >> +/** enum mali_c55_params_mesh_alpha_bank - Mesh shading table bank selection
> >> + * @MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS1 - Select Light Sources 0 and 1
> >> + * @MALI_C55_MESH_ALPHA_BANK_LS1_AND_LS2 - Select Light Sources 1 and 2
> >> + * @MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS2 - Select Light Sources 0 and 2
> >> + */
> >> +enum mali_c55_params_mesh_alpha_bank {
> >> + MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS1 = 0,
> >> + MALI_C55_MESH_ALPHA_BANK_LS1_AND_LS2 = 1,
> >> + MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS2 = 4
> >> +};
> >> +
> >> +/**
> >> + * struct mali_c55_params_mesh_shading_selection - Mesh table selection
> >> + *
> >> + * The module computes the final correction coefficients by blending the ones
> >> + * from two light source tables, which are selected (independently for each
> >> + * colour channel) by the @mesh_alpha_bank_r/g/b fields.
> >> + *
> >> + * The final blended coefficients for each node are calculated using the
> >> + * following equation:
> >> + *
> >> + * Final coefficient = (a * LS\ :sub:`b`\ + (256 - a) * LS\ :sub:`a`\) / 256
> >> + *
> >> + * Where a is the @mesh_alpha_r/g/b value, and LS\ :sub:`a`\ and LS\ :sub:`b`\
> >> + * are the node cofficients for the two tables selected by the
> >> + * @mesh_alpha_bank_r/g/b value.
> >> + *
> >> + * The scale of the applied correction may also be controlled by tuning the
> >> + * @mesh_strength member. This is a modifier to the final coefficients which can
> >> + * be used to globally reduce the gains applied.
> >> + *
> >> + * header.type should be set to MALI_C55_PARAM_MESH_SHADING_SELECTION from
> >> + * :c:type:`mali_c55_param_block_type` for this block.
> >> + *
> >> + * @header: The Mali-C55 parameters block header
> >> + * @mesh_alpha_bank_r: Red mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`)
> >> + * @mesh_alpha_bank_g: Green mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`)
> >> + * @mesh_alpha_bank_b: Blue mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`)
> >> + * @mesh_alpha_r: Blend coefficient for R [0..255]
> >> + * @mesh_alpha_g: Blend coefficient for G [0..255]
> >> + * @mesh_alpha_b: Blend coefficient for B [0..255]
> >> + * @mesh_strength: Mesh strength in Q4.12 format [0..4096]
> >> + */
> >> +struct mali_c55_params_mesh_shading_selection {
> >> + struct mali_c55_params_block_header header;
> >> + __u8 mesh_alpha_bank_r;
> >> + __u8 mesh_alpha_bank_g;
> >> + __u8 mesh_alpha_bank_b;
> >> + __u8 mesh_alpha_r;
> >> + __u8 mesh_alpha_g;
> >> + __u8 mesh_alpha_b;
> >> + __u16 mesh_strength;
> >> +};
> >> +
> >> +/**
> >> + * union mali_c55_params_block - Generalisation of a parameter block
> >> + *
> >> + * This union allows the driver to treat a block as a generic pointer to this
> >> + * union and safely access the header and block-specific struct without having
> >> + * to resort to casting. The header member is accessed first, and the type field
> >
> > If this is for driver usage only, I would move the structure to the
> > driver. Let's minimize the UAPI surface.
>
> I have actually used the union in libcamera too, to clean up the code there handling filling the
> blocks. We could redefine it within libcamera of course but given it's useful in userspace too, I
> thought it made sense to keep it in the header...what do you think?
I would prefer minimizing the area of the UAPI. Have you looked at the
parameters helper I implemented for the rkisp1 IPA module ? Maybe a
similar approach could be used for the C55, and that wouldn't require
the union.
> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >
> >> + * checked which allows the driver to determine which of the other members
> >> + * should be used. The data member at the end allows a pointer to an address
> >> + * within the data member of :c:type:`mali_c55_params_buffer` to initialise a
> >> + * union variable.
> >> + *
> >> + * @header: Pointer to the shared header struct embedded as the
> >> + * first member of all the possible other members (except
> >> + * @data). This member would be accessed first and the type
> >> + * field checked to determine which of the other members
> >> + * should be accessed.
> >> + * @sensor_offs: For header->type == MALI_C55_PARAM_BLOCK_SENSOR_OFFS
> >> + * @aexp_hist: For header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST and
> >> + * header->type == MALI_C55_PARAM_BLOCK_AEXP_IHIST
> >> + * @aexp_weights: For header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS
> >> + * and header->type = MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS
> >> + * @digital_gain: For header->type == MALI_C55_PARAM_BLOCK_DIGITAL_GAIN
> >> + * @awb_gains: For header->type == MALI_C55_PARAM_BLOCK_AWB_GAINS and
> >> + * header->type = MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP
> >> + * @awb_config: For header->type == MALI_C55_PARAM_MESH_SHADING_CONFIG
> >> + * @shading_config: For header->type == MALI_C55_PARAM_MESH_SHADING_SELECTION
> >> + * @shading_selection: For header->type == MALI_C55_PARAM_BLOCK_SENSOR_OFFS
> >> + * @data: Allows easy initialisation of a union variable with a
> >> + * pointer into a __u8 array.
> >> + */
> >> +union mali_c55_params_block {
> >> + struct mali_c55_params_block_header *header;
> >> + struct mali_c55_params_sensor_off_preshading *sensor_offs;
> >> + struct mali_c55_params_aexp_hist *aexp_hist;
> >> + struct mali_c55_params_aexp_weights *aexp_weights;
> >> + struct mali_c55_params_digital_gain *digital_gain;
> >> + struct mali_c55_params_awb_gains *awb_gains;
> >> + struct mali_c55_params_awb_config *awb_config;
> >> + struct mali_c55_params_mesh_shading_config *shading_config;
> >> + struct mali_c55_params_mesh_shading_selection *shading_selection;
> >> + __u8 *data;
> >> +};
> >> +
> >> +/**
> >> + * define MALI_C55_PARAMS_MAX_SIZE - Maximum size of all Mali C55 Parameters
> >> + *
> >> + * Though the parameters for the Mali-C55 are passed as optional blocks, the
> >> + * driver still needs to know the absolute maximum size so that it can allocate
> >> + * a buffer sized appropriately to accommodate userspace attempting to set all
> >> + * possible parameters in a single frame.
> >> + *
> >> + * Some structs are in this list multiple times. Where that's the case, it just
> >> + * reflects the fact that the same struct can be used with multiple different
> >> + * header types from :c:type:`mali_c55_param_block_type`.
> >> + */
> >> +#define MALI_C55_PARAMS_MAX_SIZE \
> >> + (sizeof(struct mali_c55_params_sensor_off_preshading) + \
> >> + sizeof(struct mali_c55_params_aexp_hist) + \
> >> + sizeof(struct mali_c55_params_aexp_weights) + \
> >> + sizeof(struct mali_c55_params_aexp_hist) + \
> >> + sizeof(struct mali_c55_params_aexp_weights) + \
> >> + sizeof(struct mali_c55_params_digital_gain) + \
> >> + sizeof(struct mali_c55_params_awb_gains) + \
> >> + sizeof(struct mali_c55_params_awb_config) + \
> >> + sizeof(struct mali_c55_params_awb_gains) + \
> >> + sizeof(struct mali_c55_params_mesh_shading_config) + \
> >> + sizeof(struct mali_c55_params_mesh_shading_selection))
> >> +
> >> +/**
> >> + * struct mali_c55_params_buffer - 3A configuration parameters
> >> + *
> >> + * This struct contains the configuration parameters of the Mali-C55 ISP
> >> + * algorithms, serialized by userspace into a data buffer. Each configuration
> >> + * parameter block is represented by a block-specific structure which contains a
> >> + * :c:type:`mali_c55_params_block_header` entry as first member. Userspace
> >> + * populates the @data buffer with configuration parameters for the blocks that
> >> + * it intends to configure. As a consequence, the data buffer effective size
> >> + * changes according to the number of ISP blocks that userspace intends to
> >> + * configure.
> >> + *
> >> + * The parameters buffer is versioned by the @version field to allow modifying
> >> + * and extending its definition. Userspace shall populate the @version field to
> >> + * inform the driver about the version it intends to use. The driver will parse
> >> + * and handle the @data buffer according to the data layout specific to the
> >> + * indicated version and return an error if the desired version is not
> >> + * supported.
> >> + *
> >> + * For each ISP block that userspace wants to configure, a block-specific
> >> + * structure is appended to the @data buffer, one after the other without gaps
> >> + * in between nor overlaps. Userspace shall populate the @total_size field with
> >> + * the effective size, in bytes, of the @data buffer.
> >> + *
> >> + * The expected memory layout of the parameters buffer is::
> >> + *
> >> + * +-------------------- struct mali_c55_params_buffer ------------------+
> >> + * | version = MALI_C55_PARAM_BUFFER_V1; |
> >> + * | total_size = sizeof(struct mali_c55_params_sensor_off_preshading) |
> >> + * | sizeof(struct mali_c55_params_aexp_hist); |
> >> + * | +------------------------- data ---------------------------------+ |
> >> + * | | +--------- struct mali_c55_params_sensor_off_preshading ------+ | |
> >> + * | | | +-------- struct mali_c55_params_block_header header -----+ | | |
> >> + * | | | | type = MALI_C55_PARAM_BLOCK_SENSOR_OFFS; | | | |
> >> + * | | | | enabled = 1; | | | |
> >> + * | | | | size = | | | |
> >> + * | | | | sizeof(struct mali_c55_params_sensor_off_preshading);| | | |
> >> + * | | | +---------------------------------------------------------+ | | |
> >> + * | | | chan00 = ...; | | |
> >> + * | | | chan01 = ...; | | |
> >> + * | | | chan10 = ...; | | |
> >> + * | | | chan11 = ...; | | |
> >> + * | | +------------ struct mali_c55_params_aexp_hist ---------------+ | |
> >> + * | | | +-------- struct mali_c55_params_block_header header -----+ | | |
> >> + * | | | | type = MALI_C55_PARAM_BLOCK_AEXP_HIST; | | | |
> >> + * | | | | enabled = 1; | | | |
> >> + * | | | | size = sizeof(struct mali_c55_params_aexp_hist); | | | |
> >> + * | | | +---------------------------------------------------------+ | | |
> >> + * | | | skip_x = ...; | | |
> >> + * | | | offset_x = ...; | | |
> >> + * | | | skip_y = ...; | | |
> >> + * | | | offset_y = ...; | | |
> >> + * | | | scale_bottom = ...; | | |
> >> + * | | | scale_top = ...; | | |
> >> + * | | | plane_mode = ...; | | |
> >> + * | | | tap_point = ...; | | |
> >> + * | | +-------------------------------------------------------------+ | |
> >> + * | +-----------------------------------------------------------------+ |
> >> + * +---------------------------------------------------------------------+
> >> + *
> >> + * @version: The version from :c:type:`mali_c55_param_buffer_version`
> >> + * @total_size: The Mali-C55 configuration data effective size, excluding this
> >> + * header
> >> + * @data: The Mali-C55 configuration blocks data
> >> + */
> >> +struct mali_c55_params_buffer {
> >> + __u8 version;
> >> + __u32 total_size;
> >> + __u8 data[MALI_C55_PARAMS_MAX_SIZE];
> >> +};
> >> +
> >> #endif /* __UAPI_MALI_C55_CONFIG_H */
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 01/18] media: mc-entity: Record number of video devices in a pipeline
2024-07-09 13:28 ` [PATCH v6 01/18] media: mc-entity: Record number of video devices in a pipeline Daniel Scally
@ 2024-07-30 15:09 ` Laurent Pinchart
2024-07-30 15:18 ` Dan Scally
0 siblings, 1 reply; 41+ messages in thread
From: Laurent Pinchart @ 2024-07-30 15:09 UTC (permalink / raw)
To: Daniel Scally
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Dan,
Thank you for the patch.
On Tue, Jul 09, 2024 at 02:28:49PM +0100, Daniel Scally wrote:
> Record the number of video devices in a pipeline so that we can track
> in a central location how many need to be started before drivers must
> actually begin streaming.
>
> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
> ---
> Changes in v6:
>
> - New patch. This is intended to support Sakari's requirement for the
> driver not to start streaming before all of the video devices have
> called streamon(). This was the cleanest way I could think to acheive
> the goal, and lets us just check for start_count == required_count
> before streaming.
This violates the abstraction layers a bit, the MC code isn't supposed
to handle video devices like that.
What you can instead do is to use media_pipeline_for_each_entity() in
your driver when starting the pipeline to iterate over all entities, and
count the video devices there.
>
> drivers/media/mc/mc-entity.c | 5 +++++
> include/media/media-entity.h | 2 ++
> 2 files changed, 7 insertions(+)
>
> diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
> index 96dd0f6ccd0d..1e8186b13b55 100644
> --- a/drivers/media/mc/mc-entity.c
> +++ b/drivers/media/mc/mc-entity.c
> @@ -596,6 +596,9 @@ static int media_pipeline_add_pad(struct media_pipeline *pipe,
>
> list_add_tail(&ppad->list, &pipe->pads);
>
> + if (pad->entity->obj_type == MEDIA_ENTITY_TYPE_VIDEO_DEVICE)
> + pipe->required_count++;
> +
> dev_dbg(pad->graph_obj.mdev->dev,
> "media pipeline: added pad '%s':%u\n",
> pad->entity->name, pad->index);
> @@ -713,6 +716,8 @@ static void media_pipeline_cleanup(struct media_pipeline *pipe)
> list_del(&ppad->list);
> kfree(ppad);
> }
> +
> + pipe->required_count = 0;
> }
>
> static int media_pipeline_populate(struct media_pipeline *pipe,
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index 0393b23129eb..ab84458b40dc 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -104,12 +104,14 @@ struct media_graph {
> * @mdev: The media device the pipeline is part of
> * @pads: List of media_pipeline_pad
> * @start_count: Media pipeline start - stop count
> + * @required_count: Number of starts required to be "running"
> */
> struct media_pipeline {
> bool allocated;
> struct media_device *mdev;
> struct list_head pads;
> int start_count;
> + int required_count;
> };
>
> /**
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 08/18] media: Documentation: Add Mali-C55 ISP Documentation
2024-07-09 13:28 ` [PATCH v6 08/18] media: Documentation: Add Mali-C55 ISP Documentation Daniel Scally
@ 2024-07-30 15:15 ` Laurent Pinchart
0 siblings, 0 replies; 41+ messages in thread
From: Laurent Pinchart @ 2024-07-30 15:15 UTC (permalink / raw)
To: Daniel Scally
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Dan,
Thank you for the patch.
On Tue, Jul 09, 2024 at 02:28:56PM +0100, Daniel Scally wrote:
> Add a documentation page for the mali-c55 driver, which gives a brief
> overview of the hardware and explains how to use the driver's capture
> devices and the crop/scaler functions.
>
> Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Changes in v6:
>
> - Minor formatting changes
>
> Changes in v5:
>
> - None
>
> Changes in v4:
> - None
>
> Changes in v3:
> - Documented the synchronised buffer sequence numbers (Sakari)
> - Clarified that the downscale pipe cannot output raw data, the ISP'S
> resolution limits and choice of media bus format code (Kieran)
>
> Changes in v2:
>
> - none
>
> .../admin-guide/media/mali-c55-graph.dot | 19 +
> Documentation/admin-guide/media/mali-c55.rst | 340 ++++++++++++++++++
> .../admin-guide/media/v4l-drivers.rst | 1 +
> 3 files changed, 360 insertions(+)
> create mode 100644 Documentation/admin-guide/media/mali-c55-graph.dot
> create mode 100644 Documentation/admin-guide/media/mali-c55.rst
>
> diff --git a/Documentation/admin-guide/media/mali-c55-graph.dot b/Documentation/admin-guide/media/mali-c55-graph.dot
> new file mode 100644
> index 000000000000..0775ba42bf4c
> --- /dev/null
> +++ b/Documentation/admin-guide/media/mali-c55-graph.dot
> @@ -0,0 +1,19 @@
> +digraph board {
> + rankdir=TB
> + n00000001 [label="{{} | mali-c55 tpg\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> + n00000001:port0 -> n00000003:port0 [style=dashed]
> + n00000003 [label="{{<port0> 0} | mali-c55 isp\n/dev/v4l-subdev1 | {<port1> 1 | <port2> 2}}", shape=Mrecord, style=filled, fillcolor=green]
> + n00000003:port1 -> n00000007:port0 [style=bold]
> + n00000003:port2 -> n00000007:port2 [style=bold]
> + n00000003:port1 -> n0000000b:port0 [style=bold]
> + n00000007 [label="{{<port0> 0 | <port2> 2} | mali-c55 resizer fr\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> + n00000007:port1 -> n0000000e [style=bold]
> + n0000000b [label="{{<port0> 0} | mali-c55 resizer ds\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> + n0000000b:port1 -> n00000012 [style=bold]
> + n0000000e [label="mali-c55 fr\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
> + n00000012 [label="mali-c55 ds\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
> + n00000022 [label="{{<port0> 0} | csi2-rx\n/dev/v4l-subdev4 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> + n00000022:port1 -> n00000003:port0
> + n00000027 [label="{{} | imx415 1-001a\n/dev/v4l-subdev5 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> + n00000027:port0 -> n00000022:port0 [style=bold]
> +}
> \ No newline at end of file
> diff --git a/Documentation/admin-guide/media/mali-c55.rst b/Documentation/admin-guide/media/mali-c55.rst
> new file mode 100644
> index 000000000000..72cdded507b3
> --- /dev/null
> +++ b/Documentation/admin-guide/media/mali-c55.rst
> @@ -0,0 +1,340 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +==========================================
> +ARM Mali-C55 Image Signal Processor driver
> +==========================================
> +
> +Introduction
> +============
> +
> +This file documents the driver for ARM's Mali-C55 Image Signal Processor. The
> +driver is located under drivers/media/platform/arm/mali-c55.
> +
> +The Mali-C55 ISP receives data in either raw Bayer format or RGB/YUV format from
> +sensors through either a parallel interface or a memory bus before processing it
> +and outputting it through an internal DMA engine. Two output pipelines are
> +possible (though one may not be fitted, depending on the implementation). These
> +are referred to as "Full resolution" and "Downscale", but the naming is historic
> +and both pipes are capable of cropping/scaling operations. The full resolution
> +pipe is also capable of outputting RAW data, bypassing much of the ISP's
> +processing. The downscale pipe cannot output RAW data. An integrated test
> +pattern generator can be used to drive the ISP and produce image data in the
> +absence of a connected camera sensor. The driver module is named mali_c55, and
> +is enabled through the CONFIG_VIDEO_MALI_C55 config option.
> +
> +The driver implements V4L2, Media Controller and V4L2 Subdevice interfaces and
> +expects camera sensors connected to the ISP to have V4L2 subdevice interfaces.
> +
> +Mali-C55 ISP hardware
> +=====================
> +
> +A high level functional view of the Mali-C55 ISP is presented below. The ISP
> +takes input from either a live source or through a DMA engine for memory input,
> +depending on the SoC integration.::
> +
> + +---------+ +----------+ +--------+
> + | Sensor |--->| CSI-2 Rx | "Full Resolution" | DMA |
> + +---------+ +----------+ |\ Output +--->| Writer |
> + | | \ | +--------+
> + | | \ +----------+ +------+---> Streaming I/O
> + +------------+ +------->| | | | |
> + | | | |-->| Mali-C55 |--+
> + | DMA Reader |--------------->| | | ISP | |
> + | | | / | | | +---> Streaming I/O
> + +------------+ | / +----------+ | |
> + |/ +------+
> + | +--------+
> + +--->| DMA |
> + "Downscaled" | Writer |
> + Output +--------+
> +
> +Media Controller Topology
> +=========================
> +
> +An example of the ISP's topology (as implemented in a system with an IMX415
> +camera sensor and generic CSI-2 receiver) is below:
> +
> +
> +.. kernel-figure:: mali-c55-graph.dot
> + :alt: mali-c55-graph.dot
> + :align: center
> +
> +The driver has 4 V4L2 subdevices:
> +
> +- `mali_c55 isp`: Responsible for configuring input crop and color space
> + conversion
> +- `mali_c55 tpg`: The test pattern generator, emulating a camera sensor.
> +- `mali_c55 resizer fr`: The Full-Resolution pipe resizer
> +- `mali_c55 resizer ds`: The Downscale pipe resizer
> +
> +The driver has 2 V4L2 video devices:
> +
> +- `mali-c55 fr`: The full-resolution pipe's capture device
> +- `mali-c55 ds`: The downscale pipe's capture device
> +
> +Frame sequences are synchronised across to two capture devices, meaning if one
> +pipe is started later than the other the sequence numbers returned in its
> +buffers will match those of the other pipe rather than starting from zero.
> +
> +Idiosyncrasies
> +--------------
> +
> +**mali-c55 isp**
> +The `mali-c55 isp` subdevice has a single sink pad to which all sources of data
> +should be connected. The active source is selected by enabling the appropriate
> +media link and disabling all others. The ISP has two source pads, reflecting the
> +different paths through which it can internally route data. Tap points within
> +the ISP allow users to divert data to avoid processing by some or all of the
> +hardware's processing steps. The diagram below is intended only to highlight how
> +the bypassing works and is not a true reflection of those processing steps; for
> +a high-level functional block diagram see ARM's developer page for the
> +ISP [3]_::
> +
> + +--------------------------------------------------------------+
> + | Possible Internal ISP Data Routes |
> + | +------------+ +----------+ +------------+ |
> + +---+ | | | | | Colour | +---+
> + | 0 |--+-->| Processing |->| Demosaic |->| Space |--->| 1 |
> + +---+ | | | | | | Conversion | +---+
> + | | +------------+ +----------+ +------------+ |
> + | | +---+
> + | +---------------------------------------------------| 2 |
> + | +---+
> + | |
> + +--------------------------------------------------------------+
> +
> +
> +.. flat-table::
> + :header-rows: 1
> +
> + * - Pad
> + - Direction
> + - Purpose
> +
> + * - 0
> + - sink
> + - Data input, connected to the TPG and camera sensors
> +
> + * - 1
> + - source
> + - RGB/YUV data, connected to the FR and DS V4L2 subdevices
> +
> + * - 2
> + - source
> + - RAW bayer data, connected to the FR V4L2 subdevices
> +
> +The ISP is limited to both input and output resolutions between 640x480 and
> +8192x8192, and this is reflected in the ISP and resizer subdevice's .set_fmt()
> +operations.
> +
> +**mali-c55 resizer fr**
> +The `mali-c55 resizer fr` subdevice has two _sink_ pads to reflect the different
> +insertion points in the hardware (either RAW or demosaiced data):
> +
> +.. flat-table::
> + :header-rows: 1
> +
> + * - Pad
> + - Direction
> + - Purpose
> +
> + * - 0
> + - sink
> + - Data input connected to the ISP's demosaiced stream.
> +
> + * - 1
> + - source
> + - Data output connected to the capture video device
> +
> + * - 2
> + - sink
> + - Data input connected to the ISP's raw data stream
> +
> +The data source in use is selected through the routing API; two routes each of a
> +single stream are available:
> +
> +.. flat-table::
> + :header-rows: 1
> +
> + * - Sink Pad
> + - Source Pad
> + - Purpose
> +
> + * - 0
> + - 1
> + - Demosaiced data route
> +
> + * - 2
> + - 1
> + - Raw data route
> +
> +
> +If the demosaiced route is active then the FR pipe is only capable of output
> +in RGB/YUV formats. If the raw route is active then the output reflects the
> +input (which may be either Bayer or RGB/YUV data).
> +
> +Using the driver to capture video
> +=================================
> +
> +Using the media controller APIs we can configure the input source and ISP to
> +capture images in a variety of formats. In the examples below, configuring the
> +media graph is done with the v4l-utils [1]_ package's media-ctl utility.
> +Capturing the images is done with yavta [2]_.
> +
> +Configuring the input source
> +----------------------------
> +
> +The first step is to set the input source that we wish by enabling the correct
> +media link. Using the example topology above, we can select the TPG as follows:
> +
> +.. code-block:: none
> +
> + media-ctl -l "'lte-csi2-rx':1->'mali-c55 isp':0[0]"
> + media-ctl -l "'mali-c55 tpg':0->'mali-c55 isp':0[1]"
> +
> +Configuring which video devices will stream data
> +------------------------------------------------
> +
> +The driver will wait for all video devices to have their VIDIOC_STREAMON ioctl
> +called before it tells the sensor to start streaming. To facilitate this we need
> +to enable links to the video devices that we want to use. In the example below
> +we enable the links to both of the image capture video devices
> +
> +.. code-block:: none
> +
> + media-ctl -l "'mali-c55 resizer fr':1->'mali-c55 fr':0[1]"
> + media-ctl -l "'mali-c55 resizer ds':1->'mali-c55 ds':0[1]"
> +
> +Capturing bayer data from the source and processing to RGB/YUV
> +--------------------------------------------------------------
> +
> +To capture 1920x1080 bayer data from the source and push it through the ISP's
> +full processing pipeline, we configure the data formats appropriately on the
> +source, ISP and resizer subdevices and set the FR resizer's routing to select
> +processed data. The media bus format on the resizer's source pad will be either
> +RGB121212_1X36 or YUV10_1X30, depending on whether you want to capture RGB or
> +YUV. The ISP's debayering block outputs RGB data natively, setting the source
> +pad format to YUV10_1X30 enables the colour space conversion block.
> +
> +In this example we target RGB565 output, so select RGB121212_1X36 as the resizer
> +source pad's format:
> +
> +.. code-block:: none
> +
> + # Set formats on the TPG and ISP
> + media-ctl -V "'mali-c55 tpg':0[fmt:SRGGB20_1X20/1920x1080]"
> + media-ctl -V "'mali-c55 isp':0[fmt:SRGGB20_1X20/1920x1080]"
> + media-ctl -V "'mali-c55 isp':1[fmt:SRGGB20_1X20/1920x1080]"
> +
> + # Set routing on the FR resizer
> + media-ctl -R "'mali-c55 resizer fr'[0/0->1/0[1],2/0->1/0[0]]"
> +
> + # Set format on the resizer, must be done AFTER the routing.
> + media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/1920x1080]"
> +
> +The downscale output can also be used to stream data at the same time. In this
> +case since only processed data can be captured through the downscale output no
> +routing need be set:
> +
> +.. code-block:: none
> +
> + # Set format on the resizer
> + media-ctl -V "'mali-c55 resizer ds':1[fmt:RGB121212_1X36/1920x1080]"
> +
> +Following which images can be captured from both the FR and DS output's video
> +devices (simultaneously, if desired):
> +
> +.. code-block:: none
> +
> + yavta -f RGB565 -s 1920x1080 -c10 /dev/video0
> + yavta -f RGB565 -s 1920x1080 -c10 /dev/video1
> +
> +Cropping the image
> +~~~~~~~~~~~~~~~~~~
> +
> +Both the full resolution and downscale pipes can crop to a minimum resolution of
> +640x480. To crop the image simply configure the resizer's sink pad's crop and
> +compose rectangles and set the format on the video device:
> +
> +.. code-block:: none
> +
> + media-ctl -V "'mali-c55 resizer fr':0[fmt:RGB121212_1X36/1920x1080 crop:(480,270)/640x480 compose:(0,0)/640x480]"
> + media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/640x480]"
> + yavta -f RGB565 -s 640x480 -c10 /dev/video0
> +
> +Downscaling the image
> +~~~~~~~~~~~~~~~~~~~~~
> +
> +Both the full resolution and downscale pipes can downscale the image by up to 8x
> +provided the minimum 640x480 output resolution is adhered to. For the best image
> +result the scaling ratio for each direction should be the same. To configure
> +scaling we use the compose rectangle on the resizer's sink pad:
> +
> +.. code-block:: none
> +
> + media-ctl -V "'mali-c55 resizer fr':0[fmt:RGB121212_1X36/1920x1080 crop:(0,0)/1920x1080 compose:(0,0)/640x480]"
> + media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/640x480]"
> + yavta -f RGB565 -s 640x480 -c10 /dev/video0
> +
> +Capturing images in YUV formats
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +If we need to output YUV data rather than RGB the color space conversion block
> +needs to be active, which is achieved by setting MEDIA_BUS_FMT_YUV10_1X30 on the
> +resizer's source pad. We can then configure a capture format like NV12 (here in
> +its multi-planar variant)
> +
> +.. code-block:: none
> +
> + media-ctl -V "'mali-c55 resizer fr':1[fmt:YUV10_1X30/1920x1080]"
> + yavta -f NV12M -s 1920x1080 -c10 /dev/video0
> +
> +Capturing RGB data from the source and processing it with the resizers
> +----------------------------------------------------------------------
> +
> +The Mali-C55 ISP can work with sensors capable of outputting RGB data. In this
> +case although none of the image quality blocks would be used it can still
> +crop/scale the data in the usual way. For this reason RGB data input to the ISP
> +still goes through the ISP subdevice's pad 1 to the resizer.
> +
> +To achieve this, the ISP's sink pad's format is set to
> +MEDIA_BUS_FMT_RGB202020_1X60 - this reflects the format that data must be in to
> +work with the ISP. Converting the camera sensor's output to that format is the
> +responsibility of external hardware.
> +
> +In this example we ask the test pattern generator to give us RGB data instead of
> +bayer.
> +
> +.. code-block:: none
> +
> + media-ctl -V "'mali-c55 tpg':0[fmt:RGB202020_1X60/1920x1080]"
> + media-ctl -V "'mali-c55 isp':0[fmt:RGB202020_1X60/1920x1080]"
> +
> +Cropping or scaling the data can be done in exactly the same way as outlined
> +earlier.
> +
> +Capturing raw data from the source and outputting it unmodified
> +-----------------------------------------------------------------
> +
> +The ISP can additionally capture raw data from the source and output it on the
> +full resolution pipe only, completely unmodified. In this case the downscale
> +pipe can still process the data normally and be used at the same time.
> +
> +To configure raw bypass the FR resizer's subdevice's routing table needs to be
> +configured, followed by formats in the appropriate places:
> +
> +.. code-block:: none
> +
> + media-ctl -R "'mali-c55 resizer fr'[0/0->1/0[0],2/0->1/0[1]]"
> + media-ctl -V "'mali-c55 isp':0[fmt:RGB202020_1X60/1920x1080]"
> + media-ctl -V "'mali-c55 resizer fr':2[fmt:RGB202020_1X60/1920x1080]"
> + media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB202020_1X60/1920x1080]"
> +
> + # Set format on the video device and stream
> + yavta -f RGB565 -s 1920x1080 -c10 /dev/video0
> +
> +References
> +==========
> +.. [1] https://git.linuxtv.org/v4l-utils.git/
> +.. [2] https://git.ideasonboard.org/yavta.git
> +.. [3] https://developer.arm.com/Processors/Mali-C55
> diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst
> index 4120eded9a13..1d9485860d93 100644
> --- a/Documentation/admin-guide/media/v4l-drivers.rst
> +++ b/Documentation/admin-guide/media/v4l-drivers.rst
> @@ -18,6 +18,7 @@ Video4Linux (V4L) driver-specific documentation
> ipu3
> ipu6-isys
> ivtv
> + mali-c55
> mgb4
> omap3isp
> omap4_camera
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 01/18] media: mc-entity: Record number of video devices in a pipeline
2024-07-30 15:09 ` Laurent Pinchart
@ 2024-07-30 15:18 ` Dan Scally
0 siblings, 0 replies; 41+ messages in thread
From: Dan Scally @ 2024-07-30 15:18 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Laurent
On 30/07/2024 16:09, Laurent Pinchart wrote:
> Hi Dan,
>
> Thank you for the patch.
>
> On Tue, Jul 09, 2024 at 02:28:49PM +0100, Daniel Scally wrote:
>> Record the number of video devices in a pipeline so that we can track
>> in a central location how many need to be started before drivers must
>> actually begin streaming.
>>
>> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
>> ---
>> Changes in v6:
>>
>> - New patch. This is intended to support Sakari's requirement for the
>> driver not to start streaming before all of the video devices have
>> called streamon(). This was the cleanest way I could think to acheive
>> the goal, and lets us just check for start_count == required_count
>> before streaming.
> This violates the abstraction layers a bit, the MC code isn't supposed
> to handle video devices like that.
>
> What you can instead do is to use media_pipeline_for_each_entity() in
> your driver when starting the pipeline to iterate over all entities, and
> count the video devices there.
That'll do! Thanks for the suggestion.
>
>> drivers/media/mc/mc-entity.c | 5 +++++
>> include/media/media-entity.h | 2 ++
>> 2 files changed, 7 insertions(+)
>>
>> diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
>> index 96dd0f6ccd0d..1e8186b13b55 100644
>> --- a/drivers/media/mc/mc-entity.c
>> +++ b/drivers/media/mc/mc-entity.c
>> @@ -596,6 +596,9 @@ static int media_pipeline_add_pad(struct media_pipeline *pipe,
>>
>> list_add_tail(&ppad->list, &pipe->pads);
>>
>> + if (pad->entity->obj_type == MEDIA_ENTITY_TYPE_VIDEO_DEVICE)
>> + pipe->required_count++;
>> +
>> dev_dbg(pad->graph_obj.mdev->dev,
>> "media pipeline: added pad '%s':%u\n",
>> pad->entity->name, pad->index);
>> @@ -713,6 +716,8 @@ static void media_pipeline_cleanup(struct media_pipeline *pipe)
>> list_del(&ppad->list);
>> kfree(ppad);
>> }
>> +
>> + pipe->required_count = 0;
>> }
>>
>> static int media_pipeline_populate(struct media_pipeline *pipe,
>> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
>> index 0393b23129eb..ab84458b40dc 100644
>> --- a/include/media/media-entity.h
>> +++ b/include/media/media-entity.h
>> @@ -104,12 +104,14 @@ struct media_graph {
>> * @mdev: The media device the pipeline is part of
>> * @pads: List of media_pipeline_pad
>> * @start_count: Media pipeline start - stop count
>> + * @required_count: Number of starts required to be "running"
>> */
>> struct media_pipeline {
>> bool allocated;
>> struct media_device *mdev;
>> struct list_head pads;
>> int start_count;
>> + int required_count;
>> };
>>
>> /**
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 07/18] media: mali-c55: Add Mali-C55 ISP driver
2024-07-09 13:28 ` [PATCH v6 07/18] media: mali-c55: Add Mali-C55 ISP driver Daniel Scally
@ 2024-07-30 21:23 ` Laurent Pinchart
2024-08-13 15:20 ` Dan Scally
0 siblings, 1 reply; 41+ messages in thread
From: Laurent Pinchart @ 2024-07-30 21:23 UTC (permalink / raw)
To: Daniel Scally
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Dan,
Thank you for the patch.
On Tue, Jul 09, 2024 at 02:28:55PM +0100, Daniel Scally wrote:
> Add a driver for Arm's Mali-C55 Image Signal Processor. The driver is
> V4L2 and Media Controller compliant and creates subdevices to manage
> the ISP itself, its internal test pattern generator as well as the
> crop, scaler and output format functionality for each of its two
> output devices.
>
> Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
> ---
> Changes in v6:
>
> - Reverted the rework of mali_c55_update_bits() so it matches how the
> regmap_update_bits() function works
> - Global rename of "rzr" abbreviation for "resizer" to "rsz"
> - Added separate functions to write to hardware and register buffer.
> - Honoured custom strides from userspace
> - Used .enable_streams(), .disable_streams() and
> v4l2_subdev_[en|dis]able_streams() throughout
> - Only call streamon for the sensor when all video devices included in
> the pipeline through enabled links are started.
> - Don't write registers until streamon() in the capture devices
> - Commented mutex/spinlocks
> - Moved v4l2 async notifier from isp.c to core.c
> - Reconfigured hardware in pm_runtime_resume() callback to make sure
> they are not lost on power-off
> - A lot of other changes, which are mostly stylistic in nature. THe full
> list is too large to include but I can send a diff between the version
> if needed.
>
> Changes in v5:
>
> - Reworked input formats - previously we allowed representing input data
> as any 8-16 bit format. Now we only allow input data to be represented
> by the new 20-bit bayer formats, which is corrected to the equivalent
> 16-bit format in RAW bypass mode.
> - Stopped bypassing blocks that we haven't added supporting parameters
> for yet.
> - Addressed most of Sakari's comments from the list
>
> Changes not yet made in v5:
>
> - The output pipelines can still be started and stopped independently of
> one another - I'd like to discuss that more.
> - the TPG subdev still uses .s_stream() - I need to rebase onto a tree
> with working .enable_streams() for a single-source-pad subdevice.
>
> Changes in v4:
>
> - Reworked mali_c55_update_bits() to internally perform the bit-shift
> - Reworked the resizer to allow cropping during streaming
> - Fixed a bug in NV12 output
>
> Changes in v3:
>
> - Mostly minor fixes suggested by Sakari
> - Fixed the sequencing of vb2 buffers to be synchronised across the two
> capture devices.
>
> Changes in v2:
>
> - Clock handling
> - Fixed the warnings raised by the kernel test robot
>
> drivers/media/platform/Kconfig | 1 +
> drivers/media/platform/Makefile | 1 +
> drivers/media/platform/arm/Kconfig | 5 +
> drivers/media/platform/arm/Makefile | 2 +
> drivers/media/platform/arm/mali-c55/Kconfig | 17 +
> drivers/media/platform/arm/mali-c55/Makefile | 9 +
> .../platform/arm/mali-c55/mali-c55-capture.c | 961 +++++++++++++++
> .../platform/arm/mali-c55/mali-c55-common.h | 247 ++++
> .../platform/arm/mali-c55/mali-c55-core.c | 893 ++++++++++++++
> .../platform/arm/mali-c55/mali-c55-isp.c | 531 ++++++++
> .../arm/mali-c55/mali-c55-registers.h | 313 +++++
> .../platform/arm/mali-c55/mali-c55-resizer.c | 1096 +++++++++++++++++
> .../platform/arm/mali-c55/mali-c55-tpg.c | 438 +++++++
> 13 files changed, 4514 insertions(+)
> create mode 100644 drivers/media/platform/arm/Kconfig
> create mode 100644 drivers/media/platform/arm/Makefile
> create mode 100644 drivers/media/platform/arm/mali-c55/Kconfig
> create mode 100644 drivers/media/platform/arm/mali-c55/Makefile
> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-capture.c
> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-common.h
> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-core.c
> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-registers.h
> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-tpg.c
>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index 2d79bfc68c15..c929169766aa 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -65,6 +65,7 @@ config VIDEO_MUX
> source "drivers/media/platform/allegro-dvt/Kconfig"
> source "drivers/media/platform/amlogic/Kconfig"
> source "drivers/media/platform/amphion/Kconfig"
> +source "drivers/media/platform/arm/Kconfig"
> source "drivers/media/platform/aspeed/Kconfig"
> source "drivers/media/platform/atmel/Kconfig"
> source "drivers/media/platform/broadcom/Kconfig"
> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> index da17301f7439..9a647abd5218 100644
> --- a/drivers/media/platform/Makefile
> +++ b/drivers/media/platform/Makefile
> @@ -8,6 +8,7 @@
> obj-y += allegro-dvt/
> obj-y += amlogic/
> obj-y += amphion/
> +obj-y += arm/
> obj-y += aspeed/
> obj-y += atmel/
> obj-y += broadcom/
> diff --git a/drivers/media/platform/arm/Kconfig b/drivers/media/platform/arm/Kconfig
> new file mode 100644
> index 000000000000..4f0764c329c7
> --- /dev/null
> +++ b/drivers/media/platform/arm/Kconfig
> @@ -0,0 +1,5 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +
> +comment "ARM media platform drivers"
> +
> +source "drivers/media/platform/arm/mali-c55/Kconfig"
> diff --git a/drivers/media/platform/arm/Makefile b/drivers/media/platform/arm/Makefile
> new file mode 100644
> index 000000000000..8cc4918725ef
> --- /dev/null
> +++ b/drivers/media/platform/arm/Makefile
> @@ -0,0 +1,2 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +obj-y += mali-c55/
> diff --git a/drivers/media/platform/arm/mali-c55/Kconfig b/drivers/media/platform/arm/mali-c55/Kconfig
> new file mode 100644
> index 000000000000..6ba70e765b8d
> --- /dev/null
> +++ b/drivers/media/platform/arm/mali-c55/Kconfig
> @@ -0,0 +1,17 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +config VIDEO_MALI_C55
> + tristate "ARM Mali-C55 Image Signal Processor driver"
> + depends on ARCH_VEXPRESS || COMPILE_TEST
> + depends on V4L_PLATFORM_DRIVERS
> + depends on VIDEO_DEV && OF
> + select GENERIC_PHY_MIPI_DPHY
> + select MEDIA_CONTROLLER
> + select V4L2_FWNODE
> + select VIDEO_V4L2_SUBDEV_API
> + select VIDEOBUF2_DMA_CONTIG
> + select VIDEOBUF2_VMALLOC
> + help
> + Enable this to support Arm's Mali-C55 Image Signal Processor.
> +
> + To compile this driver as a module, choose M here: the module
> + will be called mali-c55.
> diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile
> new file mode 100644
> index 000000000000..9178ac35e50e
> --- /dev/null
> +++ b/drivers/media/platform/arm/mali-c55/Makefile
> @@ -0,0 +1,9 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +mali-c55-y := mali-c55-capture.o \
> + mali-c55-core.o \
> + mali-c55-isp.o \
> + mali-c55-resizer.o \
> + mali-c55-tpg.o
> +
> +obj-$(CONFIG_VIDEO_MALI_C55) += mali-c55.o
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-capture.c b/drivers/media/platform/arm/mali-c55/mali-c55-capture.c
> new file mode 100644
> index 000000000000..508245f06a09
> --- /dev/null
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-capture.c
> @@ -0,0 +1,961 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * ARM Mali-C55 ISP Driver - Video capture devices
> + *
> + * Copyright (C) 2024 Ideas on Board Oy
> + */
> +
> +#include <linux/cleanup.h>
> +#include <linux/minmax.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/string.h>
> +#include <linux/videodev2.h>
> +
> +#include <media/v4l2-dev.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mali-c55-common.h"
> +#include "mali-c55-registers.h"
> +
> +static const struct mali_c55_format_info mali_c55_fmts[] = {
> + /*
> + * This table is missing some entries which need further work or
> + * investigation:
> + *
> + * Base mode 1 is a backwards V4L2_PIX_FMT_XRGB32 with no V4L2 equivalent
> + * Base mode 5 is "Generic Data"
> + * Base mode 8 is a backwards V4L2_PIX_FMT_XYUV32 - no V4L2 equivalent
> + * Base mode 9 seems to have no V4L2 equivalent
> + * Base mode 17, 19 and 20 describe formats which seem to have no V4L2
> + * equivalent
> + */
> + {
> + .fourcc = V4L2_PIX_FMT_ARGB2101010,
> + .mbus_codes = {
> + MEDIA_BUS_FMT_RGB121212_1X36,
> + MEDIA_BUS_FMT_RGB202020_1X60,
> + },
> + .is_raw = false,
> + .registers = {
> + .base_mode = MALI_C55_OUTPUT_A2R10G10B10,
> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
> + }
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_RGB565,
> + .mbus_codes = {
> + MEDIA_BUS_FMT_RGB121212_1X36,
> + MEDIA_BUS_FMT_RGB202020_1X60,
> + },
> + .is_raw = false,
> + .registers = {
> + .base_mode = MALI_C55_OUTPUT_RGB565,
> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
> + }
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_BGR24,
> + .mbus_codes = {
> + MEDIA_BUS_FMT_RGB121212_1X36,
> + MEDIA_BUS_FMT_RGB202020_1X60,
> + },
> + .is_raw = false,
> + .registers = {
> + .base_mode = MALI_C55_OUTPUT_RGB24,
> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
> + }
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_YUYV,
> + .mbus_codes = {
> + MEDIA_BUS_FMT_YUV10_1X30,
> + },
> + .is_raw = false,
> + .registers = {
> + .base_mode = MALI_C55_OUTPUT_YUY2,
> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
> + }
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_UYVY,
> + .mbus_codes = {
> + MEDIA_BUS_FMT_YUV10_1X30,
> + },
> + .is_raw = false,
> + .registers = {
> + .base_mode = MALI_C55_OUTPUT_UYVY,
> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
> + }
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_Y210,
> + .mbus_codes = {
> + MEDIA_BUS_FMT_YUV10_1X30,
> + },
> + .is_raw = false,
> + .registers = {
> + .base_mode = MALI_C55_OUTPUT_Y210,
> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
> + }
> + },
> + /*
> + * This is something of a hack, the ISP thinks it's running NV12M but
> + * by setting uv_plane = 0 we simply discard that planes and only output
> + * the Y-plane.
> + */
> + {
> + .fourcc = V4L2_PIX_FMT_GREY,
> + .mbus_codes = {
> + MEDIA_BUS_FMT_YUV10_1X30,
> + },
> + .is_raw = false,
> + .registers = {
> + .base_mode = MALI_C55_OUTPUT_NV12_21,
> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
> + }
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_NV12M,
> + .mbus_codes = {
> + MEDIA_BUS_FMT_YUV10_1X30,
> + },
> + .is_raw = false,
> + .registers = {
> + .base_mode = MALI_C55_OUTPUT_NV12_21,
> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT1
> + }
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_NV21M,
> + .mbus_codes = {
> + MEDIA_BUS_FMT_YUV10_1X30,
> + },
> + .is_raw = false,
> + .registers = {
> + .base_mode = MALI_C55_OUTPUT_NV12_21,
> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT2
> + }
> + },
> + /*
> + * RAW uncompressed formats are all packed in 16 bpp.
> + * TODO: Expand this list to encompass all possible RAW formats.
> + */
> + {
> + .fourcc = V4L2_PIX_FMT_SRGGB16,
> + .mbus_codes = {
> + MEDIA_BUS_FMT_SRGGB16_1X16,
> + },
> + .is_raw = true,
> + .registers = {
> + .base_mode = MALI_C55_OUTPUT_RAW16,
> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
> + }
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SBGGR16,
> + .mbus_codes = {
> + MEDIA_BUS_FMT_SBGGR16_1X16,
> + },
> + .is_raw = true,
> + .registers = {
> + .base_mode = MALI_C55_OUTPUT_RAW16,
> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
> + }
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGBRG16,
> + .mbus_codes = {
> + MEDIA_BUS_FMT_SGBRG16_1X16,
> + },
> + .is_raw = true,
> + .registers = {
> + .base_mode = MALI_C55_OUTPUT_RAW16,
> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
> + }
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGRBG16,
> + .mbus_codes = {
> + MEDIA_BUS_FMT_SGRBG16_1X16,
> + },
> + .is_raw = true,
> + .registers = {
> + .base_mode = MALI_C55_OUTPUT_RAW16,
> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
> + }
> + },
> +};
> +
> +void mali_c55_cap_dev_write(struct mali_c55_cap_dev *cap_dev, unsigned int addr,
> + u32 val)
> +{
> + mali_c55_ctx_write(cap_dev->mali_c55, addr + cap_dev->reg_offset, val);
> +}
> +
> +static u32 mali_c55_cap_dev_read(struct mali_c55_cap_dev *cap_dev, unsigned int addr)
> +{
> + return mali_c55_ctx_read(cap_dev->mali_c55, addr + cap_dev->reg_offset);
> +}
> +
> +static void mali_c55_cap_dev_update_bits(struct mali_c55_cap_dev *cap_dev,
> + unsigned int addr, u32 mask, u32 val)
> +{
> + u32 orig, tmp;
> +
> + orig = mali_c55_cap_dev_read(cap_dev, addr);
> +
> + tmp = orig & ~mask;
> + tmp |= val & mask;
> +
> + if (tmp != orig)
> + mali_c55_cap_dev_write(cap_dev, addr, tmp);
> +}
> +
> +static bool
> +mali_c55_mbus_code_can_produce_fmt(const struct mali_c55_format_info *fmt,
> + u32 code)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(fmt->mbus_codes); i++) {
> + if (fmt->mbus_codes[i] == code)
> + return true;
> + }
> +
> + return false;
> +}
> +
> +bool mali_c55_format_is_raw(unsigned int mbus_code)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(mali_c55_fmts); i++) {
> + if (mali_c55_fmts[i].mbus_codes[0] == mbus_code)
> + return mali_c55_fmts[i].is_raw;
> + }
> +
> + return false;
> +}
> +
> +static const struct mali_c55_format_info *
> +mali_c55_format_from_pix(const u32 pixelformat)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(mali_c55_fmts); i++) {
> + if (mali_c55_fmts[i].fourcc == pixelformat)
> + return &mali_c55_fmts[i];
> + }
> +
> + /*
> + * If we find no matching pixelformat, we'll just default to the first
> + * one for now.
> + */
> +
> + return &mali_c55_fmts[0];
> +}
> +
> +static const char * const capture_device_names[] = {
> + "mali-c55 fr",
> + "mali-c55 ds",
> +};
> +
> +static int mali_c55_link_validate(struct media_link *link)
> +{
> + struct video_device *vdev =
> + media_entity_to_video_device(link->sink->entity);
> + struct mali_c55_cap_dev *cap_dev = video_get_drvdata(vdev);
> + struct v4l2_subdev *sd =
> + media_entity_to_v4l2_subdev(link->source->entity);
> + const struct v4l2_pix_format_mplane *pix_mp;
> + const struct mali_c55_format_info *cap_fmt;
> + struct v4l2_subdev_format sd_fmt = {
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .pad = link->source->index,
> + };
> + int ret;
> +
> + ret = v4l2_subdev_call(sd, pad, get_fmt, NULL, &sd_fmt);
> + if (ret)
> + return ret;
> +
> + pix_mp = &cap_dev->format.format;
> + cap_fmt = cap_dev->format.info;
> +
> + if (sd_fmt.format.width != pix_mp->width ||
> + sd_fmt.format.height != pix_mp->height) {
> + dev_dbg(cap_dev->mali_c55->dev,
> + "link '%s':%u -> '%s':%u not valid: %ux%u != %ux%u\n",
> + link->source->entity->name, link->source->index,
> + link->sink->entity->name, link->sink->index,
> + sd_fmt.format.width, sd_fmt.format.height,
> + pix_mp->width, pix_mp->height);
> + return -EPIPE;
> + }
> +
> + if (!mali_c55_mbus_code_can_produce_fmt(cap_fmt, sd_fmt.format.code)) {
> + dev_dbg(cap_dev->mali_c55->dev,
> + "link '%s':%u -> '%s':%u not valid: mbus_code 0x%04x cannot produce pixel format %p4cc\n",
> + link->source->entity->name, link->source->index,
> + link->sink->entity->name, link->sink->index,
> + sd_fmt.format.code, &pix_mp->pixelformat);
> + return -EPIPE;
> + }
> +
> + return 0;
> +}
> +
> +static const struct media_entity_operations mali_c55_media_ops = {
> + .link_validate = mali_c55_link_validate,
> +};
> +
> +static int mali_c55_vb2_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
> + unsigned int *num_planes, unsigned int sizes[],
> + struct device *alloc_devs[])
> +{
> + struct mali_c55_cap_dev *cap_dev = q->drv_priv;
> + unsigned int i;
> +
> + if (*num_planes) {
> + if (*num_planes != cap_dev->format.format.num_planes)
> + return -EINVAL;
> +
> + for (i = 0; i < cap_dev->format.format.num_planes; i++)
> + if (sizes[i] < cap_dev->format.format.plane_fmt[i].sizeimage)
> + return -EINVAL;
> + } else {
> + *num_planes = cap_dev->format.format.num_planes;
> + for (i = 0; i < cap_dev->format.format.num_planes; i++)
> + sizes[i] = cap_dev->format.format.plane_fmt[i].sizeimage;
> + }
> +
> + return 0;
> +}
> +
> +static void mali_c55_buf_queue(struct vb2_buffer *vb)
> +{
> + struct mali_c55_cap_dev *cap_dev = vb2_get_drv_priv(vb->vb2_queue);
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct mali_c55_buffer *buf = container_of(vbuf,
> + struct mali_c55_buffer, vb);
> + unsigned int i;
> +
> + buf->planes_pending = cap_dev->format.format.num_planes;
> +
> + for (i = 0; i < cap_dev->format.format.num_planes; i++) {
> + unsigned long size = cap_dev->format.format.plane_fmt[i].sizeimage;
> +
> + vb2_set_plane_payload(vb, i, size);
> + }
> +
> + buf->vb.field = V4L2_FIELD_NONE;
> +
> + spin_lock(&cap_dev->buffers.lock);
> + list_add_tail(&buf->queue, &cap_dev->buffers.queue);
> + spin_unlock(&cap_dev->buffers.lock);
> +}
> +
> +static int mali_c55_buf_init(struct vb2_buffer *vb)
> +{
> + struct mali_c55_cap_dev *cap_dev = vb2_get_drv_priv(vb->vb2_queue);
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct mali_c55_buffer *buf = container_of(vbuf,
> + struct mali_c55_buffer, vb);
> + unsigned int i;
> +
> + for (i = 0; i < cap_dev->format.format.num_planes; i++)
> + buf->addrs[i] = vb2_dma_contig_plane_dma_addr(vb, i);
> +
> + return 0;
> +}
> +
> +void mali_c55_set_next_buffer(struct mali_c55_cap_dev *cap_dev)
> +{
> + guard(spinlock)(&cap_dev->buffers.lock);
> +
> + cap_dev->buffers.curr = cap_dev->buffers.next;
> + cap_dev->buffers.next = NULL;
> +
> + if (!list_empty(&cap_dev->buffers.queue)) {
> + struct v4l2_pix_format_mplane *pix_mp;
> + const struct v4l2_format_info *info;
> + dma_addr_t *addrs;
> +
> + pix_mp = &cap_dev->format.format;
> + info = v4l2_format_info(pix_mp->pixelformat);
> +
> + mali_c55_cap_dev_update_bits(cap_dev,
> + MALI_C55_REG_Y_WRITER_MODE,
> + MALI_C55_WRITER_FRAME_WRITE_MASK,
> + MALI_C55_WRITER_FRAME_WRITE_ENABLE);
> + if (cap_dev->format.info->registers.uv_plane)
> + mali_c55_cap_dev_update_bits(cap_dev,
> + MALI_C55_REG_UV_WRITER_MODE,
> + MALI_C55_WRITER_FRAME_WRITE_MASK,
> + MALI_C55_WRITER_FRAME_WRITE_ENABLE);
> +
> + cap_dev->buffers.next = list_first_entry(&cap_dev->buffers.queue,
> + struct mali_c55_buffer,
> + queue);
> + list_del(&cap_dev->buffers.next->queue);
> +
> + addrs = cap_dev->buffers.next->addrs;
> + mali_c55_cap_dev_write(cap_dev,
> + MALI_C55_REG_Y_WRITER_BANKS_BASE,
> + addrs[MALI_C55_PLANE_Y]);
> + mali_c55_cap_dev_write(cap_dev,
> + MALI_C55_REG_UV_WRITER_BANKS_BASE,
> + addrs[MALI_C55_PLANE_UV]);
> + mali_c55_cap_dev_write(cap_dev,
> + MALI_C55_REG_Y_WRITER_OFFSET,
> + pix_mp->width * info->bpp[MALI_C55_PLANE_Y]);
> + mali_c55_cap_dev_write(cap_dev,
> + MALI_C55_REG_UV_WRITER_OFFSET,
> + pix_mp->width * info->bpp[MALI_C55_PLANE_UV]
> + / info->hdiv);
> + } else {
> + /*
> + * If we underflow then we can tell the ISP that we don't want
> + * to write out the next frame.
> + */
> + mali_c55_cap_dev_update_bits(cap_dev,
> + MALI_C55_REG_Y_WRITER_MODE,
> + MALI_C55_WRITER_FRAME_WRITE_MASK,
> + 0x00);
> + mali_c55_cap_dev_update_bits(cap_dev,
> + MALI_C55_REG_UV_WRITER_MODE,
> + MALI_C55_WRITER_FRAME_WRITE_MASK,
> + 0x00);
> + }
> +}
> +
> +static void mali_c55_handle_buffer(struct mali_c55_buffer *curr_buf,
> + unsigned int framecount)
> +{
> + curr_buf->vb.vb2_buf.timestamp = ktime_get_boottime_ns();
> + curr_buf->vb.sequence = framecount;
> + vb2_buffer_done(&curr_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
> +}
> +
> +/**
> + * mali_c55_set_plane_done - mark the plane as written and process the buffer if
> + * both planes are finished.
> + * @cap_dev: pointer to the fr or ds pipe output
> + * @plane: the plane to mark as completed
> + *
> + * The Mali C55 ISP has muliplanar outputs for some formats that come with two
> + * separate "buffer write completed" interrupts - we need to flag each plane's
> + * completion and check whether both planes are done - if so, complete the buf
> + * in vb2.
> + */
> +void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
> + enum mali_c55_planes plane)
> +{
> + struct mali_c55_isp *isp = &cap_dev->mali_c55->isp;
> + struct mali_c55_buffer *curr_buf;
> +
> + guard(spinlock)(&cap_dev->buffers.lock);
> + curr_buf = cap_dev->buffers.curr;
> +
> + /*
> + * This _should_ never happen. If no buffer was available from vb2 then
> + * we tell the ISP not to bother writing the next frame, which means the
> + * interrupts that call this function should never trigger. If it does
> + * happen then one of our assumptions is horribly wrong - complain
> + * loudly and do nothing.
> + */
> + if (!curr_buf) {
> + dev_err(cap_dev->mali_c55->dev, "%s null buffer in %s()\n",
> + cap_dev->vdev.name, __func__);
> + return;
> + }
> +
> + /* If the other plane is also done... */
> + if (!--curr_buf->planes_pending) {
> + mali_c55_handle_buffer(curr_buf, isp->frame_sequence);
> + cap_dev->buffers.curr = NULL;
> + isp->frame_sequence++;
> + }
> +}
> +
> +static void mali_c55_cap_dev_stream_disable(struct mali_c55_cap_dev *cap_dev)
> +{
> + mali_c55_cap_dev_update_bits(cap_dev, MALI_C55_REG_Y_WRITER_MODE,
> + MALI_C55_WRITER_FRAME_WRITE_MASK, 0x00);
> + mali_c55_cap_dev_update_bits(cap_dev, MALI_C55_REG_UV_WRITER_MODE,
> + MALI_C55_WRITER_FRAME_WRITE_MASK, 0x00);
> +}
> +
> +static void mali_c55_cap_dev_stream_enable(struct mali_c55_cap_dev *cap_dev)
> +{
> + /*
> + * The Mali ISP can hold up to 5 buffer addresses and simply cycle
> + * through them, but it's not clear to me that the vb2 queue _guarantees_
> + * it will queue buffers to the driver in a fixed order, and ensuring
> + * we call vb2_buffer_done() for the right buffer seems to me to add
> + * pointless complexity given in multi-context mode we'd need to
> + * re-write those registers every frame anyway...so we tell the ISP to
> + * use a single register and update it for each frame.
> + */
> + mali_c55_cap_dev_update_bits(cap_dev,
> + MALI_C55_REG_Y_WRITER_BANKS_CONFIG,
> + MALI_C55_REG_Y_WRITER_MAX_BANKS_MASK, 0);
> + mali_c55_cap_dev_update_bits(cap_dev,
> + MALI_C55_REG_UV_WRITER_BANKS_CONFIG,
> + MALI_C55_REG_UV_WRITER_MAX_BANKS_MASK, 0);
> +
> + mali_c55_set_next_buffer(cap_dev);
> +}
> +
> +static void mali_c55_cap_dev_return_buffers(struct mali_c55_cap_dev *cap_dev,
> + enum vb2_buffer_state state)
> +{
> + struct mali_c55_buffer *buf, *tmp;
> +
> + guard(spinlock)(&cap_dev->buffers.lock);
> +
> + if (cap_dev->buffers.curr) {
> + vb2_buffer_done(&cap_dev->buffers.curr->vb.vb2_buf,
> + state);
> + cap_dev->buffers.curr = NULL;
> + }
> +
> + if (cap_dev->buffers.next) {
> + vb2_buffer_done(&cap_dev->buffers.next->vb.vb2_buf,
> + state);
> + cap_dev->buffers.next = NULL;
> + }
> +
> + list_for_each_entry_safe(buf, tmp, &cap_dev->buffers.queue, queue) {
> + list_del(&buf->queue);
> + vb2_buffer_done(&buf->vb.vb2_buf, state);
> + }
> +}
> +
> +static void mali_c55_cap_dev_format_configure(struct mali_c55_cap_dev *cap_dev)
> +{
> + const struct mali_c55_format_info *capture_format = cap_dev->format.info;
> + struct v4l2_pix_format_mplane *pix_mp = &cap_dev->format.format;
const
> + const struct v4l2_format_info *info;
> +
> + info = v4l2_format_info(pix_mp->pixelformat);
> + if (WARN_ON(!info))
> + return;
> +
> + mali_c55_cap_dev_write(cap_dev, MALI_C55_REG_Y_WRITER_MODE,
> + capture_format->registers.base_mode);
> + mali_c55_cap_dev_write(cap_dev, MALI_C55_REG_ACTIVE_OUT_Y_SIZE,
> + MALI_C55_REG_ACTIVE_OUT_SIZE_W(pix_mp->width) |
> + MALI_C55_REG_ACTIVE_OUT_SIZE_H(pix_mp->height));
> +
> + if (info->mem_planes > 1) {
> + mali_c55_cap_dev_write(cap_dev, MALI_C55_REG_UV_WRITER_MODE,
> + capture_format->registers.base_mode);
> + mali_c55_cap_dev_update_bits(cap_dev,
> + MALI_C55_REG_UV_WRITER_MODE,
> + MALI_C55_WRITER_SUBMODE_MASK,
> + MALI_C55_WRITER_SUBMODE(capture_format->registers.uv_plane));
> +
> + mali_c55_cap_dev_write(cap_dev, MALI_C55_REG_ACTIVE_OUT_UV_SIZE,
> + MALI_C55_REG_ACTIVE_OUT_SIZE_W(pix_mp->width) |
> + MALI_C55_REG_ACTIVE_OUT_SIZE_H(pix_mp->height));
> + }
> +
> + if (info->pixel_enc == V4L2_PIXEL_ENC_YUV) {
> + mali_c55_cap_dev_write(cap_dev, MALI_C55_REG_CS_CONV_CONFIG,
> + MALI_C55_CS_CONV_MATRIX_MASK);
> +
> + if (info->hdiv > 1)
> + mali_c55_cap_dev_update_bits(cap_dev,
> + MALI_C55_REG_CS_CONV_CONFIG,
> + MALI_C55_CS_CONV_HORZ_DOWNSAMPLE_MASK,
> + MALI_C55_CS_CONV_HORZ_DOWNSAMPLE_ENABLE);
> + if (info->vdiv > 1)
> + mali_c55_cap_dev_update_bits(cap_dev,
> + MALI_C55_REG_CS_CONV_CONFIG,
> + MALI_C55_CS_CONV_VERT_DOWNSAMPLE_MASK,
> + MALI_C55_CS_CONV_VERT_DOWNSAMPLE_ENABLE);
> + if (info->hdiv > 1 || info->vdiv > 1)
> + mali_c55_cap_dev_update_bits(cap_dev,
> + MALI_C55_REG_CS_CONV_CONFIG,
> + MALI_C55_CS_CONV_FILTER_MASK,
> + MALI_C55_CS_CONV_FILTER_ENABLE);
> + }
> +}
> +
> +static int mali_c55_vb2_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> + struct mali_c55_cap_dev *cap_dev = q->drv_priv;
> + struct mali_c55 *mali_c55 = cap_dev->mali_c55;
> + struct mali_c55_resizer *rsz = cap_dev->rsz;
> + struct mali_c55_isp *isp = &mali_c55->isp;
> + int ret;
> +
> + guard(mutex)(&isp->capture_lock);
> +
> + ret = pm_runtime_resume_and_get(mali_c55->dev);
> + if (ret)
> + return ret;
> +
> + mali_c55_cap_dev_format_configure(cap_dev);
I think this should go after video_device_pipeline_start(), to avoid
configuring the device needlessly if pipeline validation fails.
> +
> + ret = video_device_pipeline_start(&cap_dev->vdev,
> + &cap_dev->mali_c55->pipe);
> + if (ret) {
> + dev_dbg(mali_c55->dev, "%s failed to start media pipeline\n",
> + cap_dev->vdev.name);
> + goto err_pm_put;
> + }
> +
> + mali_c55_cap_dev_stream_enable(cap_dev);
> +
> + ret = v4l2_subdev_enable_streams(&rsz->sd, MALI_C55_RSZ_SOURCE_PAD,
> + BIT(0));
> + if (ret)
> + goto err_disable_cap_dev;
> +
> + if (mali_c55->pipe.start_count == mali_c55->pipe.required_count) {
> + /*
> + * The ISP has a "processed" and "bypass" source pad, but they
> + * don't have a separate start process so we'll just start the
> + * processed pad unconditionally.
> + */
> + ret = v4l2_subdev_enable_streams(&mali_c55->isp.sd,
> + MALI_C55_ISP_PAD_SOURCE_VIDEO,
> + BIT(0));
The resizer subdev is supposed to start its source, but I'll ignore that
:-)
> + if (ret)
> + goto err_disable_rsz;
> + }
> +
> + return 0;
> +
> +err_disable_rsz:
> + v4l2_subdev_disable_streams(&rsz->sd, MALI_C55_RSZ_SOURCE_PAD, BIT(0));
> +err_disable_cap_dev:
> + mali_c55_cap_dev_stream_disable(cap_dev);
> + video_device_pipeline_stop(&cap_dev->vdev);
> +err_pm_put:
> + pm_runtime_put(mali_c55->dev);
> + mali_c55_cap_dev_return_buffers(cap_dev, VB2_BUF_STATE_QUEUED);
> +
> + return ret;
> +}
> +
> +static void mali_c55_vb2_stop_streaming(struct vb2_queue *q)
> +{
> + struct mali_c55_cap_dev *cap_dev = q->drv_priv;
> + struct mali_c55 *mali_c55 = cap_dev->mali_c55;
> + struct mali_c55_resizer *rsz = cap_dev->rsz;
> + struct mali_c55_isp *isp = &mali_c55->isp;
> +
> + guard(mutex)(&isp->capture_lock);
> +
> + v4l2_subdev_disable_streams(&mali_c55->isp.sd,
> + MALI_C55_ISP_PAD_SOURCE_VIDEO, BIT(0));
Shouldn't this be done only when you stop the first video device, to
match how you start the ISP ?
> + v4l2_subdev_disable_streams(&rsz->sd, MALI_C55_RSZ_SOURCE_PAD, BIT(0));
> + mali_c55_cap_dev_stream_disable(cap_dev);
> + mali_c55_cap_dev_return_buffers(cap_dev, VB2_BUF_STATE_ERROR);
> + video_device_pipeline_stop(&cap_dev->vdev);
> + pm_runtime_put_autosuspend(mali_c55->dev);
pm_runtime_mark_last_busy(...);
> +}
> +
> +static const struct vb2_ops mali_c55_vb2_ops = {
> + .queue_setup = &mali_c55_vb2_queue_setup,
> + .buf_queue = &mali_c55_buf_queue,
> + .buf_init = &mali_c55_buf_init,
> + .wait_prepare = vb2_ops_wait_prepare,
> + .wait_finish = vb2_ops_wait_finish,
> + .start_streaming = &mali_c55_vb2_start_streaming,
> + .stop_streaming = &mali_c55_vb2_stop_streaming,
> +};
> +
> +static const struct v4l2_file_operations mali_c55_v4l2_fops = {
> + .owner = THIS_MODULE,
> + .unlocked_ioctl = video_ioctl2,
> + .open = v4l2_fh_open,
> + .release = vb2_fop_release,
> + .poll = vb2_fop_poll,
> + .mmap = vb2_fop_mmap,
> +};
> +
> +static void mali_c55_try_fmt(struct v4l2_pix_format_mplane *pix_mp)
> +{
> + const struct mali_c55_format_info *capture_format;
> + const struct v4l2_format_info *info;
> + struct v4l2_plane_pix_format *plane, *y_plane;
> + unsigned int padding;
> + unsigned int i;
> +
> + capture_format = mali_c55_format_from_pix(pix_mp->pixelformat);
> + pix_mp->pixelformat = capture_format->fourcc;
> +
> + pix_mp->width = clamp(pix_mp->width, MALI_C55_MIN_WIDTH,
> + MALI_C55_MAX_WIDTH);
> + pix_mp->height = clamp(pix_mp->height, MALI_C55_MIN_HEIGHT,
> + MALI_C55_MAX_HEIGHT);
> +
> + pix_mp->field = V4L2_FIELD_NONE;
> + pix_mp->colorspace = V4L2_COLORSPACE_DEFAULT;
> + pix_mp->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> + pix_mp->quantization = V4L2_QUANTIZATION_DEFAULT;
> +
> + info = v4l2_format_info(pix_mp->pixelformat);
> + pix_mp->num_planes = info->mem_planes;
> + memset(pix_mp->plane_fmt, 0, sizeof(pix_mp->plane_fmt));
> +
> + y_plane = &pix_mp->plane_fmt[0];
> + y_plane->bytesperline = clamp(y_plane->bytesperline,
> + info->bpp[0] * pix_mp->width, 65535U);
It's nice that you make the stride configurable, but unless I'm missing
something, the setting never makes it to the hardware.
> + y_plane->sizeimage = y_plane->bytesperline * pix_mp->height;
> +
> + padding = y_plane->bytesperline - (pix_mp->width * info->bpp[0]);
> +
> + for (i = 1; i < info->comp_planes; i++) {
> + plane = &pix_mp->plane_fmt[i];
> +
> + plane->bytesperline = DIV_ROUND_UP(info->bpp[i] * pix_mp->width,
> + info->hdiv) + padding;
What's the hardware constraint for the stride for multi-planar formats ?
Is there a single stride register, or one per plane ? If there's a
single one, I would be surprised if the stride for the second and
subsequent planes would be calculated by the hardware with the above
formula.
> + plane->sizeimage = DIV_ROUND_UP(
> + plane->bytesperline * pix_mp->height,
> + info->vdiv);
> + }
> +
> + if (info->mem_planes == 1) {
> + for (i = 1; i < info->comp_planes; i++) {
> + plane = &pix_mp->plane_fmt[i];
> + y_plane->sizeimage += plane->sizeimage;
> + }
> + }
> +}
> +
> +static int mali_c55_try_fmt_vid_cap_mplane(struct file *file, void *fh,
> + struct v4l2_format *f)
> +{
> + mali_c55_try_fmt(&f->fmt.pix_mp);
> +
> + return 0;
> +}
> +
> +static void mali_c55_set_format(struct mali_c55_cap_dev *cap_dev,
> + struct v4l2_pix_format_mplane *pix_mp)
> +{
> + mali_c55_try_fmt(pix_mp);
> +
> + cap_dev->format.format = *pix_mp;
> + cap_dev->format.info = mali_c55_format_from_pix(pix_mp->pixelformat);
You could avoid a double lookup by returning the info pointer from
mali_c55_try_fmt().
> +}
> +
> +static int mali_c55_s_fmt_vid_cap_mplane(struct file *file, void *fh,
> + struct v4l2_format *f)
> +{
> + struct mali_c55_cap_dev *cap_dev = video_drvdata(file);
> +
> + if (vb2_is_busy(&cap_dev->queue))
> + return -EBUSY;
> +
> + mali_c55_set_format(cap_dev, &f->fmt.pix_mp);
> +
> + return 0;
> +}
> +
> +static int mali_c55_g_fmt_vid_cap_mplane(struct file *file, void *fh,
> + struct v4l2_format *f)
> +{
> + struct mali_c55_cap_dev *cap_dev = video_drvdata(file);
> +
> + f->fmt.pix_mp = cap_dev->format.format;
> +
> + return 0;
> +}
> +
> +static int mali_c55_enum_fmt_vid_cap_mplane(struct file *file, void *fh,
> + struct v4l2_fmtdesc *f)
> +{
> + struct mali_c55_cap_dev *cap_dev = video_drvdata(file);
> + unsigned int j = 0;
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(mali_c55_fmts); i++) {
> + if (f->mbus_code &&
> + !mali_c55_mbus_code_can_produce_fmt(&mali_c55_fmts[i],
> + f->mbus_code))
> + continue;
> +
> + /* Downscale pipe can't output RAW formats */
> + if (mali_c55_fmts[i].is_raw &&
> + cap_dev->reg_offset == MALI_C55_CAP_DEV_DS_REG_OFFSET)
> + continue;
> +
> + if (j++ == f->index) {
> + f->pixelformat = mali_c55_fmts[i].fourcc;
> + return 0;
> + }
> + }
> +
> + return -EINVAL;
> +}
> +
> +static int mali_c55_querycap(struct file *file, void *fh,
> + struct v4l2_capability *cap)
> +{
> + strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver));
> + strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card));
> +
> + return 0;
> +}
> +
> +static const struct v4l2_ioctl_ops mali_c55_v4l2_ioctl_ops = {
> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> + .vidioc_querybuf = vb2_ioctl_querybuf,
> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> + .vidioc_qbuf = vb2_ioctl_qbuf,
> + .vidioc_expbuf = vb2_ioctl_expbuf,
> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> + .vidioc_streamon = vb2_ioctl_streamon,
> + .vidioc_streamoff = vb2_ioctl_streamoff,
> + .vidioc_try_fmt_vid_cap_mplane = mali_c55_try_fmt_vid_cap_mplane,
> + .vidioc_s_fmt_vid_cap_mplane = mali_c55_s_fmt_vid_cap_mplane,
> + .vidioc_g_fmt_vid_cap_mplane = mali_c55_g_fmt_vid_cap_mplane,
> + .vidioc_enum_fmt_vid_cap = mali_c55_enum_fmt_vid_cap_mplane,
> + .vidioc_querycap = mali_c55_querycap,
> + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static int mali_c55_register_cap_dev(struct mali_c55 *mali_c55,
> + enum mali_c55_cap_devs cap_dev_id)
> +{
> + struct mali_c55_cap_dev *cap_dev = &mali_c55->cap_devs[cap_dev_id];
> + struct v4l2_pix_format_mplane pix_mp;
> + struct video_device *vdev;
> + struct vb2_queue *vb2q;
> + int ret;
> +
> + vdev = &cap_dev->vdev;
> + vb2q = &cap_dev->queue;
> +
> + cap_dev->mali_c55 = mali_c55;
> + mutex_init(&cap_dev->lock);
> + INIT_LIST_HEAD(&cap_dev->buffers.queue);
> +
> + switch (cap_dev_id) {
> + case MALI_C55_CAP_DEV_FR:
> + cap_dev->rsz = &mali_c55->resizers[MALI_C55_RSZ_FR];
> + cap_dev->reg_offset = MALI_C55_CAP_DEV_FR_REG_OFFSET;
> + break;
> + case MALI_C55_CAP_DEV_DS:
> + cap_dev->rsz = &mali_c55->resizers[MALI_C55_RSZ_DS];
> + cap_dev->reg_offset = MALI_C55_CAP_DEV_DS_REG_OFFSET;
> + break;
> + default:
> + ret = -EINVAL;
> + goto err_destroy_mutex;
> + }
> +
> + cap_dev->pad.flags = MEDIA_PAD_FL_SINK;
> + ret = media_entity_pads_init(&cap_dev->vdev.entity, 1, &cap_dev->pad);
> + if (ret) {
> + mutex_destroy(&cap_dev->lock);
> + goto err_destroy_mutex;
> + }
> +
> + vb2q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> + vb2q->io_modes = VB2_MMAP | VB2_DMABUF;
> + vb2q->drv_priv = cap_dev;
> + vb2q->mem_ops = &vb2_dma_contig_memops;
> + vb2q->ops = &mali_c55_vb2_ops;
> + vb2q->buf_struct_size = sizeof(struct mali_c55_buffer);
> + vb2q->min_queued_buffers = 1;
> + vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> + vb2q->lock = &cap_dev->lock;
> + vb2q->dev = mali_c55->dev;
> +
> + ret = vb2_queue_init(vb2q);
> + if (ret) {
> + dev_err(mali_c55->dev, "%s vb2 queue init failed\n",
> + cap_dev->vdev.name);
> + goto err_cleanup_media_entity;
> + }
> +
> + strscpy(cap_dev->vdev.name, capture_device_names[cap_dev_id],
> + sizeof(cap_dev->vdev.name));
> + vdev->release = video_device_release_empty;
> + vdev->fops = &mali_c55_v4l2_fops;
> + vdev->ioctl_ops = &mali_c55_v4l2_ioctl_ops;
> + vdev->lock = &cap_dev->lock;
> + vdev->v4l2_dev = &mali_c55->v4l2_dev;
> + vdev->queue = &cap_dev->queue;
> + vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE_MPLANE |
> + V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
> + vdev->entity.ops = &mali_c55_media_ops;
> + video_set_drvdata(vdev, cap_dev);
> +
> + memset(&pix_mp, 0, sizeof(pix_mp));
> + pix_mp.pixelformat = V4L2_PIX_FMT_RGB565;
> + pix_mp.width = MALI_C55_DEFAULT_WIDTH;
> + pix_mp.height = MALI_C55_DEFAULT_HEIGHT;
> + mali_c55_set_format(cap_dev, &pix_mp);
> +
> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> + if (ret) {
> + dev_err(mali_c55->dev,
> + "%s failed to register video device\n",
> + cap_dev->vdev.name);
> + goto err_release_vb2q;
> + }
> +
> + return 0;
> +
> +err_release_vb2q:
> + vb2_queue_release(vb2q);
> +err_cleanup_media_entity:
> + media_entity_cleanup(&cap_dev->vdev.entity);
> +err_destroy_mutex:
> + mutex_destroy(&cap_dev->lock);
> +
> + return ret;
> +}
> +
> +int mali_c55_register_capture_devs(struct mali_c55 *mali_c55)
> +{
> + int ret;
> +
> + ret = mali_c55_register_cap_dev(mali_c55, MALI_C55_CAP_DEV_FR);
> + if (ret)
> + return ret;
> +
> + if (mali_c55->capabilities & MALI_C55_GPS_DS_PIPE_FITTED) {
> + ret = mali_c55_register_cap_dev(mali_c55, MALI_C55_CAP_DEV_DS);
> + if (ret) {
> + mali_c55_unregister_capture_devs(mali_c55);
> + return ret;
> + }
> + }
> +
> + return 0;
> +}
> +
> +static void mali_c55_unregister_cap_dev(struct mali_c55 *mali_c55,
> + enum mali_c55_cap_devs cap_dev_id)
> +{
> + struct mali_c55_cap_dev *cap_dev = &mali_c55->cap_devs[cap_dev_id];
> +
> + if (!video_is_registered(&cap_dev->vdev))
> + return;
> +
> + vb2_video_unregister_device(&cap_dev->vdev);
> + media_entity_cleanup(&cap_dev->vdev.entity);
> + mutex_destroy(&cap_dev->lock);
> +}
> +
> +void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55)
> +{
> + mali_c55_unregister_cap_dev(mali_c55, MALI_C55_CAP_DEV_FR);
> + if (mali_c55->capabilities & MALI_C55_GPS_DS_PIPE_FITTED)
> + mali_c55_unregister_cap_dev(mali_c55, MALI_C55_CAP_DEV_DS);
> +}
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> new file mode 100644
> index 000000000000..f7764a938e9f
> --- /dev/null
> @@ -0,0 +1,247 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * ARM Mali-C55 ISP Driver - Common definitions
> + *
> + * Copyright (C) 2024 Ideas on Board Oy
> + */
> +
> +#ifndef _MALI_C55_COMMON_H
> +#define _MALI_C55_COMMON_H
> +
> +#include <linux/clk.h>
> +#include <linux/io.h>
> +#include <linux/list.h>
> +#include <linux/mutex.h>
> +#include <linux/spinlock.h>
> +#include <linux/videodev2.h>
> +
> +#include <media/media-device.h>
> +#include <media/v4l2-async.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-dev.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#define MALI_C55_DRIVER_NAME "mali-c55"
> +
> +/* min and max values for the image sizes */
> +#define MALI_C55_MIN_WIDTH 640U
> +#define MALI_C55_MIN_HEIGHT 480U
> +#define MALI_C55_MAX_WIDTH 8192U
> +#define MALI_C55_MAX_HEIGHT 8192U
> +#define MALI_C55_DEFAULT_WIDTH 1920U
> +#define MALI_C55_DEFAULT_HEIGHT 1080U
> +
> +#define MALI_C55_DEFAULT_MEDIA_BUS_FMT MEDIA_BUS_FMT_RGB121212_1X36
> +
> +#define MALI_C55_NUM_CLKS 2
> +
> +struct device;
> +struct dma_chan;
> +struct mali_c55;
> +struct mali_c55_cap_dev;
> +struct platform_device;
> +struct resource;
> +
> +enum mali_c55_isp_pads {
> + MALI_C55_ISP_PAD_SINK_VIDEO,
> + MALI_C55_ISP_PAD_SOURCE_VIDEO,
> + MALI_C55_ISP_PAD_SOURCE_BYPASS,
> + MALI_C55_ISP_NUM_PADS,
> +};
> +
> +struct mali_c55_tpg {
> + struct mali_c55 *mali_c55;
> + struct v4l2_subdev sd;
> + struct media_pad pad;
> + struct mali_c55_tpg_ctrls {
> + struct v4l2_ctrl_handler handler;
> + struct v4l2_ctrl *vblank;
> + } ctrls;
> +};
> +
> +struct mali_c55_isp {
> + struct mali_c55 *mali_c55;
> + struct v4l2_subdev sd;
> + struct media_pad pads[MALI_C55_ISP_NUM_PADS];
> + struct media_pad *remote_src;
> + /* Mutex to guard vb2 start/stop streaming */
> + struct mutex capture_lock;
> + unsigned int frame_sequence;
> +};
> +
> +enum mali_c55_resizer_ids {
> + MALI_C55_RSZ_FR,
> + MALI_C55_RSZ_DS,
> + MALI_C55_NUM_RSZS,
> +};
> +
> +enum mali_c55_rsz_pads {
> + MALI_C55_RSZ_SINK_PAD,
> + MALI_C55_RSZ_SOURCE_PAD,
> + MALI_C55_RSZ_SINK_BYPASS_PAD,
> + MALI_C55_RSZ_NUM_PADS
> +};
> +
> +struct mali_c55_resizer {
> + struct mali_c55 *mali_c55;
> + struct mali_c55_cap_dev *cap_dev;
> + enum mali_c55_resizer_ids id;
> + struct v4l2_subdev sd;
> + struct media_pad pads[MALI_C55_RSZ_NUM_PADS];
> + unsigned int num_routes;
> + bool streaming;
> +};
> +
> +enum mali_c55_cap_devs {
> + MALI_C55_CAP_DEV_FR,
> + MALI_C55_CAP_DEV_DS,
> + MALI_C55_NUM_CAP_DEVS
> +};
> +
> +struct mali_c55_format_info {
> + u32 fourcc;
> + /*
> + * The output formats can be produced by a couple of different media bus
> + * formats, depending on how the ISP is configured.
> + */
> + unsigned int mbus_codes[2];
> + bool is_raw;
> + struct {
> + u32 base_mode;
> + u32 uv_plane;
> + } registers;
> +};
> +
> +struct mali_c55_isp_fmt {
struct mali_c55_isp_format_info
> + u32 code;
> + bool bypass;
> + u32 order;
> +};
> +
> +enum mali_c55_planes {
> + MALI_C55_PLANE_Y,
> + MALI_C55_PLANE_UV,
> + MALI_C55_NUM_PLANES
> +};
> +
> +struct mali_c55_buffer {
> + struct vb2_v4l2_buffer vb;
> + unsigned int planes_pending;
> + struct list_head queue;
> + dma_addr_t addrs[MALI_C55_NUM_PLANES];
> +};
> +
> +struct mali_c55_cap_dev {
> + struct mali_c55 *mali_c55;
> + struct mali_c55_resizer *rsz;
> + struct video_device vdev;
> + struct media_pad pad;
> + struct vb2_queue queue;
> + /* Mutex to provide to vb2 */
> + struct mutex lock;
> + unsigned int reg_offset;
> +
> + struct {
> + const struct mali_c55_format_info *info;
> + struct v4l2_pix_format_mplane format;
> + } format;
> +
> + struct {
> + /* Spinlock to guard buffer queue */
> + spinlock_t lock;
> + struct list_head queue;
> + struct mali_c55_buffer *curr;
> + struct mali_c55_buffer *next;
> + } buffers;
> +
> + bool streaming;
> +};
> +
> +enum mali_c55_config_spaces {
> + MALI_C55_CONFIG_PING,
> + MALI_C55_CONFIG_PONG,
> + MALI_C55_NUM_CONFIG_SPACES
The last enumerator isn't used.
> +};
> +
> +/**
> + * struct mali_c55_context - Fields relating to a single camera context
> + *
> + * @mali_c55: Pointer to the main struct mali_c55
> + * @registers: A pointer to some allocated memory holding register
> + * values to be written to the hardware at frame interrupt
> + * @base: Base address of the config space in the hardware
> + * @lock: A spinlock to protect against writes to @registers whilst that
> + * space is being copied to the hardware
> + * @list: A list head to facilitate a context queue
> + */
> +struct mali_c55_context {
> + struct mali_c55 *mali_c55;
> + void *registers;
> + phys_addr_t base;
> + /* Spinlock to prevent simultaneous access of register space */
> + spinlock_t lock;
> + struct list_head list;
> +};
> +
> +struct mali_c55 {
> + struct device *dev;
> + void __iomem *base;
> + struct dma_chan *channel;
> + struct clk_bulk_data clks[MALI_C55_NUM_CLKS];
> +
> + u16 capabilities;
> + struct media_device media_dev;
> + struct v4l2_device v4l2_dev;
> + struct v4l2_async_notifier notifier;
> + struct media_pipeline pipe;
> +
> + struct mali_c55_tpg tpg;
> + struct mali_c55_isp isp;
> + struct mali_c55_resizer resizers[MALI_C55_NUM_RSZS];
> + struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS];
> +
> + struct mali_c55_context context;
> + enum mali_c55_config_spaces next_config;
> +};
> +
> +void mali_c55_write(struct mali_c55 *mali_c55, unsigned int addr, u32 val);
> +void mali_c55_cap_dev_write(struct mali_c55_cap_dev *cap_dev, unsigned int addr,
> + u32 val);
> +void mali_c55_update_bits(struct mali_c55 *mali_c55, unsigned int addr,
> + u32 mask, u32 val);
> +u32 mali_c55_read(struct mali_c55 *mali_c55, unsigned int addr);
> +void mali_c55_ctx_write(struct mali_c55 *mali_c55, unsigned int addr, u32 val);
> +u32 mali_c55_ctx_read(struct mali_c55 *mali_c55, unsigned int addr);
> +void mali_c55_ctx_update_bits(struct mali_c55 *mali_c55, unsigned int addr,
> + u32 mask, u32 val);
> +
> +int mali_c55_config_write(struct mali_c55_context *ctx,
> + enum mali_c55_config_spaces cfg_space);
> +
> +int mali_c55_register_isp(struct mali_c55 *mali_c55);
> +int mali_c55_register_tpg(struct mali_c55 *mali_c55);
> +void mali_c55_unregister_tpg(struct mali_c55 *mali_c55);
> +void mali_c55_unregister_isp(struct mali_c55 *mali_c55);
> +int mali_c55_register_resizers(struct mali_c55 *mali_c55);
> +void mali_c55_unregister_resizers(struct mali_c55 *mali_c55);
> +int mali_c55_register_capture_devs(struct mali_c55 *mali_c55);
> +void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55);
> +struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55);
> +void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
> + enum mali_c55_planes plane);
> +void mali_c55_set_next_buffer(struct mali_c55_cap_dev *cap_dev);
> +void mali_c55_isp_queue_event_sof(struct mali_c55 *mali_c55);
> +
> +bool mali_c55_format_is_raw(unsigned int mbus_code);
> +
> +const struct mali_c55_isp_fmt *
> +mali_c55_isp_fmt_next(const struct mali_c55_isp_fmt *fmt);
> +const struct mali_c55_isp_fmt *
> +mali_c55_isp_get_mbus_config_by_code(u32 code);
> +const struct mali_c55_isp_fmt *
> +mali_c55_isp_get_mbus_config_by_index(u32 index);
> +
> +#endif /* _MALI_C55_COMMON_H */
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> new file mode 100644
> index 000000000000..dbc07d12d3a3
> --- /dev/null
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> @@ -0,0 +1,893 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * ARM Mali-C55 ISP Driver - Core driver code
> + *
> + * Copyright (C) 2024 Ideas on Board Oy
> + */
> +
> +#include <linux/bitops.h>
> +#include <linux/cleanup.h>
> +#include <linux/clk.h>
> +#include <linux/delay.h>
> +#include <linux/device.h>
> +#include <linux/dmaengine.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/interrupt.h>
> +#include <linux/iopoll.h>
> +#include <linux/ioport.h>
> +#include <linux/mod_devicetable.h>
> +#include <linux/of.h>
> +#include <linux/of_reserved_mem.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/slab.h>
> +#include <linux/string.h>
> +
> +#include <media/media-entity.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-mc.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mali-c55-common.h"
> +#include "mali-c55-registers.h"
> +
> +static const char * const mali_c55_interrupt_names[] = {
> + [MALI_C55_IRQ_ISP_START] = "ISP start",
> + [MALI_C55_IRQ_ISP_DONE] = "ISP done",
> + [MALI_C55_IRQ_MCM_ERROR] = "Multi-context management error",
> + [MALI_C55_IRQ_BROKEN_FRAME_ERROR] = "Broken frame error",
> + [MALI_C55_IRQ_MET_AF_DONE] = "AF metering done",
> + [MALI_C55_IRQ_MET_AEXP_DONE] = "AEXP metering done",
> + [MALI_C55_IRQ_MET_AWB_DONE] = "AWB metering done",
> + [MALI_C55_IRQ_AEXP_1024_DONE] = "AEXP 1024-bit histogram done",
> + [MALI_C55_IRQ_IRIDIX_MET_DONE] = "Iridix metering done",
> + [MALI_C55_IRQ_LUT_INIT_DONE] = "LUT memory init done",
> + [MALI_C55_IRQ_FR_Y_DONE] = "Full resolution Y plane DMA done",
> + [MALI_C55_IRQ_FR_UV_DONE] = "Full resolution U/V plane DMA done",
> + [MALI_C55_IRQ_DS_Y_DONE] = "Downscale Y plane DMA done",
> + [MALI_C55_IRQ_DS_UV_DONE] = "Downscale U/V plane DMA done",
> + [MALI_C55_IRQ_LINEARIZATION_DONE] = "Linearisation done",
> + [MALI_C55_IRQ_RAW_FRONTEND_DONE] = "Raw frontend processing done",
> + [MALI_C55_IRQ_NOISE_REDUCTION_DONE] = "Noise reduction done",
> + [MALI_C55_IRQ_IRIDIX_DONE] = "Iridix done",
> + [MALI_C55_IRQ_BAYER2RGB_DONE] = "Bayer to RGB conversion done",
> + [MALI_C55_IRQ_WATCHDOG_TIMER] = "Watchdog timer timed out",
> + [MALI_C55_IRQ_FRAME_COLLISION] = "Frame collision error",
> + [MALI_C55_IRQ_UNUSED] = "IRQ bit unused",
> + [MALI_C55_IRQ_DMA_ERROR] = "DMA error",
> + [MALI_C55_IRQ_INPUT_STOPPED] = "Input port safely stopped",
> + [MALI_C55_IRQ_MET_AWB_TARGET1_HIT] = "AWB metering target 1 address hit",
> + [MALI_C55_IRQ_MET_AWB_TARGET2_HIT] = "AWB metering target 2 address hit"
> +};
> +
> +static const unsigned int config_space_addrs[] = {
> + [MALI_C55_CONFIG_PING] = 0x0ab6c,
> + [MALI_C55_CONFIG_PONG] = 0x22b2c,
> +};
> +
> +static const char * const mali_c55_clk_names[] = {
static const char * const mali_c55_clk_names[MALI_C55_NUM_CLKS] = {
so the compiler will warn you if you have more entries in the array than
expected.
> + "aclk",
> + "hclk",
> +};
> +
> +/*
> + * System IO
> + *
> + * The Mali-C55 ISP has up to two configuration register spaces (called 'ping'
> + * and 'pong'), with the expectation that the 'active' space will be left
> + * untouched whilst a frame is being processed and the 'inactive' space
> + * configured ready to be switched to during the blanking period before the next
> + * frame processing starts. These spaces should ideally be set via DMA transfer
> + * from a buffer rather than through individual register set operations. There
> + * is also a shared global register space which should be set normally. Of
> + * course, the ISP might be included in a system which lacks a suitable DMA
> + * engine, and the second configuration space might not be fitted at all, which
> + * means we need to support four scenarios:
> + *
> + * 1. Multi config space, with DMA engine.
> + * 2. Multi config space, no DMA engine.
> + * 3. Single config space, with DMA engine.
> + * 4. Single config space, no DMA engine.
> + *
> + * The first case is very easy, but the rest present annoying problems. The best
> + * way to solve them seems to be simply to replicate the concept of DMAing over
> + * the configuration buffer even if there's no DMA engine on the board, for
> + * which we rely on memcpy. To facilitate this any read/write call that is made
> + * to an address within those config spaces should infact be directed to a
> + * buffer that was allocated to hold them rather than the IO memory itself. The
> + * actual copy of that buffer to IO mem will happen on interrupt.
> + */
> +
> +void mali_c55_write(struct mali_c55 *mali_c55, unsigned int addr, u32 val)
> +{
> + WARN_ON(addr >= MALI_C55_REG_CONFIG_SPACES_OFFSET);
> +
> + writel(val, mali_c55->base + addr);
> +}
> +
> +u32 mali_c55_read(struct mali_c55 *mali_c55, unsigned int addr)
> +{
> + WARN_ON(addr >= MALI_C55_REG_CONFIG_SPACES_OFFSET);
> +
> + return readl(mali_c55->base + addr);
> +}
> +
> +void mali_c55_update_bits(struct mali_c55 *mali_c55, unsigned int addr,
> + u32 mask, u32 val)
> +{
> + u32 orig, tmp;
s/tmp/new/
> +
> + orig = mali_c55_read(mali_c55, addr);
> +
> + tmp = orig & ~mask;
> + tmp |= val & mask;
> +
> + if (tmp != orig)
> + mali_c55_write(mali_c55, addr, tmp);
> +}
> +
> +static void __mali_c55_ctx_write(struct mali_c55_context *ctx,
> + unsigned int addr, u32 val)
> +{
> + spin_lock(&ctx->lock);
> + addr = (addr - MALI_C55_REG_CONFIG_SPACES_OFFSET) / 4;
> + ((u32 *)ctx->registers)[addr] = val;
ctx->registers could be a u32 *, it would simplify code here.
> + spin_unlock(&ctx->lock);
> +}
> +
> +void mali_c55_ctx_write(struct mali_c55 *mali_c55, unsigned int addr, u32 val)
> +{
> + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
That's OK for now, but I think the context will need to be passed to
this function. I'd say you could prepare for it already, but doing it
later is likely better.
> +
> + WARN_ON(addr < MALI_C55_REG_CONFIG_SPACES_OFFSET);
> + __mali_c55_ctx_write(ctx, addr, val);
> +}
> +
> +static u32 __mali_c55_ctx_read(struct mali_c55_context *ctx, unsigned int addr)
> +{
> + u32 val;
> +
> + spin_lock(&ctx->lock);
> + addr = (addr - MALI_C55_REG_CONFIG_SPACES_OFFSET) / 4;
This can be done before taking the spinlock.
> + val = ((u32 *)ctx->registers)[addr];
> + spin_unlock(&ctx->lock);
> +
> + return val;
> +}
> +
> +u32 mali_c55_ctx_read(struct mali_c55 *mali_c55, unsigned int addr)
> +{
> + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
> +
> + WARN_ON(addr < MALI_C55_REG_CONFIG_SPACES_OFFSET);
> + return __mali_c55_ctx_read(ctx, addr);
> +}
> +
> +void mali_c55_ctx_update_bits(struct mali_c55 *mali_c55, unsigned int addr,
> + u32 mask, u32 val)
> +{
> + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
> + u32 orig, tmp;
> +
> + WARN_ON(addr < MALI_C55_REG_CONFIG_SPACES_OFFSET);
> +
> + orig = __mali_c55_ctx_read(ctx, addr);
> +
> + tmp = orig & ~mask;
> + tmp |= val & mask;
> +
> + if (tmp != orig)
> + __mali_c55_ctx_write(ctx, addr, tmp);
Shouldn't you take the spinlock around the whole read/write ?
> +}
> +
> +static void mali_c55_dma_xfer_complete(void *param,
> + const struct dmaengine_result *result)
> +{
> + struct mali_c55 *mali_c55 = param;
> +
> + if (result->result != DMA_TRANS_NOERROR)
> + dev_err(mali_c55->dev, "Failed to DMA xfer ISP config\n");
> +}
> +
> +static int mali_c55_dma_xfer(struct mali_c55_context *ctx, dma_addr_t src,
> + dma_addr_t dst)
> +{
> + struct mali_c55 *mali_c55 = ctx->mali_c55;
> + struct dma_async_tx_descriptor *tx;
> + dma_cookie_t cookie;
> +
> + tx = dmaengine_prep_dma_memcpy(mali_c55->channel, dst, src,
> + MALI_C55_CONFIG_SPACE_SIZE, 0);
> + if (!tx) {
> + dev_err(mali_c55->dev, "failed to prep DMA memcpy\n");
> + return -EIO;
> + }
> +
> + tx->callback_result = mali_c55_dma_xfer_complete;
> + tx->callback_param = mali_c55;
> +
> + cookie = dmaengine_submit(tx);
> + if (dma_submit_error(cookie)) {
> + dev_err(mali_c55->dev, "error submitting dma transfer\n");
> + return -EIO;
> + }
> +
> + dma_async_issue_pending(mali_c55->channel);
> +
> + return 0;
> +}
> +
> +static int mali_c55_dma_write(struct mali_c55_context *ctx,
> + enum mali_c55_config_spaces cfg_space)
> +{
> + struct mali_c55 *mali_c55 = ctx->mali_c55;
> + struct device *dma_dev = mali_c55->channel->device->dev;
> + dma_addr_t dst = ctx->base + config_space_addrs[cfg_space];
> + dma_addr_t src;
> + int ret;
> +
> + guard(spinlock)(&ctx->lock);
> +
> + src = dma_map_single(dma_dev, ctx->registers,
> + MALI_C55_CONFIG_SPACE_SIZE, DMA_TO_DEVICE);
> + if (dma_mapping_error(dma_dev, src)) {
> + dev_err(mali_c55->dev, "failed to map DMA addr\n");
> + return -EIO;
> + }
> +
> + ret = mali_c55_dma_xfer(ctx, src, dst);
> + dma_unmap_single(dma_dev, src, MALI_C55_CONFIG_SPACE_SIZE,
> + DMA_TO_DEVICE);
> +
> + return ret;
> +}
> +
> +int mali_c55_config_write(struct mali_c55_context *ctx,
> + enum mali_c55_config_spaces cfg_space)
> +{
> + struct mali_c55 *mali_c55 = ctx->mali_c55;
> +
> + if (mali_c55->channel)
> + return mali_c55_dma_write(ctx, cfg_space);
> +
> + memcpy_toio(mali_c55->base + config_space_addrs[cfg_space],
> + ctx->registers, MALI_C55_CONFIG_SPACE_SIZE);
So, when using DMA, this function issues the memcpy and returns without
waiting for it to complete, while otherwise the copy is synchronous.
That's not a problem as such, but in the asynchronous case, the caller
needs to handle the fact that the copy will complete later. Looking at
mali_c55_isp_start() for instance, it seems you actually start the ISP
before the copy completes, which doesn't seem safe. Similarly, in
mali_c55_swap_next_config(), I think the ISP is instructed to start
using the other configuration space before the copy completes, which
seems equally dangerous. Does all this need some more careful
consideration ?
> +
> + return 0;
> +}
> +
> +struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55)
> +{
> + return &mali_c55->context;
> +}
> +
> +static int mali_c55_parse_endpoint(struct mali_c55 *mali_c55)
> +{
> + struct v4l2_async_connection *asc;
> + struct fwnode_handle *ep;
> +
> + /*
> + * The ISP should have a single endpoint pointing to some flavour of
> + * CSI-2 receiver...but for now at least we do want everything to work
> + * normally even with no sensors connected, as we have the TPG. If we
> + * don't find a sensor just warn and return success.
> + */
> + ep = fwnode_graph_get_endpoint_by_id(dev_fwnode(mali_c55->dev),
> + 0, 0, 0);
> + if (!ep) {
> + dev_warn(mali_c55->dev, "no local endpoint found\n");
> + return 0;
> + }
> +
> + asc = v4l2_async_nf_add_fwnode_remote(&mali_c55->notifier, ep,
> + struct v4l2_async_connection);
> + fwnode_handle_put(ep);
> + if (IS_ERR(asc)) {
> + dev_err(mali_c55->dev, "failed to add remote fwnode\n");
> + return PTR_ERR(asc);
> + }
> +
> + return 0;
> +}
I would move this right after mali_c55_notifier_ops, as it's more
related to the notifier and the following function than to the next two
functions right below. Up to you.
> +
> +static void mali_c55_remove_links(struct mali_c55 *mali_c55)
> +{
> + unsigned int i;
> +
> + media_entity_remove_links(&mali_c55->tpg.sd.entity);
> + media_entity_remove_links(&mali_c55->isp.sd.entity);
> +
> + for (i = 0; i < MALI_C55_NUM_RSZS; i++)
> + media_entity_remove_links(&mali_c55->resizers[i].sd.entity);
> +
> + for (i = 0; i < MALI_C55_NUM_CAP_DEVS; i++)
> + media_entity_remove_links(&mali_c55->cap_devs[i].vdev.entity);
> +}
> +
> +static int mali_c55_create_links(struct mali_c55 *mali_c55)
> +{
> + int ret;
> +
> + /* Test pattern generator to ISP */
> + ret = media_create_pad_link(&mali_c55->tpg.sd.entity, 0,
> + &mali_c55->isp.sd.entity,
> + MALI_C55_ISP_PAD_SINK_VIDEO, 0);
> + if (ret) {
> + dev_err(mali_c55->dev, "failed to link TPG and ISP\n");
> + goto err_remove_links;
> + }
> +
> + /* Full resolution resizer pipe. */
> + ret = media_create_pad_link(&mali_c55->isp.sd.entity,
> + MALI_C55_ISP_PAD_SOURCE_VIDEO,
> + &mali_c55->resizers[MALI_C55_RSZ_FR].sd.entity,
> + MALI_C55_RSZ_SINK_PAD,
> + MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE);
> + if (ret) {
> + dev_err(mali_c55->dev, "failed to link ISP and FR resizer\n");
> + goto err_remove_links;
> + }
> +
> + /* Full resolution bypass. */
> + ret = media_create_pad_link(&mali_c55->isp.sd.entity,
> + MALI_C55_ISP_PAD_SOURCE_BYPASS,
> + &mali_c55->resizers[MALI_C55_RSZ_FR].sd.entity,
> + MALI_C55_RSZ_SINK_BYPASS_PAD,
> + MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE);
> + if (ret) {
> + dev_err(mali_c55->dev, "failed to link ISP and FR resizer\n");
> + goto err_remove_links;
> + }
> +
> + /* Resizer pipe to video capture nodes. */
> + ret = media_create_pad_link(&mali_c55->resizers[0].sd.entity,
> + MALI_C55_RSZ_SOURCE_PAD,
> + &mali_c55->cap_devs[MALI_C55_CAP_DEV_FR].vdev.entity,
> + 0, MEDIA_LNK_FL_ENABLED);
> + if (ret) {
> + dev_err(mali_c55->dev,
> + "failed to link FR resizer and video device\n");
> + goto err_remove_links;
> + }
> +
> + /* The downscale pipe is an optional hardware block */
> + if (mali_c55->capabilities & MALI_C55_GPS_DS_PIPE_FITTED) {
> + ret = media_create_pad_link(&mali_c55->isp.sd.entity,
> + MALI_C55_ISP_PAD_SOURCE_VIDEO,
> + &mali_c55->resizers[MALI_C55_RSZ_DS].sd.entity,
> + MALI_C55_RSZ_SINK_PAD,
> + MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE);
> + if (ret) {
> + dev_err(mali_c55->dev,
> + "failed to link ISP and DS resizer\n");
> + goto err_remove_links;
> + }
> +
> + ret = media_create_pad_link(&mali_c55->resizers[1].sd.entity,
> + MALI_C55_RSZ_SOURCE_PAD,
> + &mali_c55->cap_devs[MALI_C55_CAP_DEV_DS].vdev.entity,
> + 0, MEDIA_LNK_FL_ENABLED);
> + if (ret) {
> + dev_err(mali_c55->dev,
> + "failed to link DS resizer and video device\n");
> + goto err_remove_links;
> + }
> + }
> +
> + return 0;
> +
> +err_remove_links:
> + mali_c55_remove_links(mali_c55);
> + return ret;
> +}
> +
> +static void mali_c55_unregister_entities(struct mali_c55 *mali_c55)
> +{
> + mali_c55_remove_links(mali_c55);
> + mali_c55_unregister_tpg(mali_c55);
> + mali_c55_unregister_isp(mali_c55);
> + mali_c55_unregister_resizers(mali_c55);
> + mali_c55_unregister_capture_devs(mali_c55);
> +}
> +
> +static int mali_c55_register_entities(struct mali_c55 *mali_c55)
> +{
> + int ret;
> +
> + ret = mali_c55_register_tpg(mali_c55);
> + if (ret)
> + return ret;
> +
> + ret = mali_c55_register_isp(mali_c55);
> + if (ret)
> + goto err_unregister_entities;
> +
> + ret = mali_c55_register_resizers(mali_c55);
> + if (ret)
> + goto err_unregister_entities;
> +
> + ret = mali_c55_register_capture_devs(mali_c55);
> + if (ret)
> + goto err_unregister_entities;
> +
> + ret = mali_c55_create_links(mali_c55);
> + if (ret)
> + goto err_unregister_entities;
> +
> + return 0;
> +
> +err_unregister_entities:
> + mali_c55_unregister_entities(mali_c55);
> +
> + return ret;
> +}
> +
> +static int mali_c55_notifier_bound(struct v4l2_async_notifier *notifier,
> + struct v4l2_subdev *subdev,
> + struct v4l2_async_connection *asc)
> +{
> + struct mali_c55 *mali_c55 = container_of(notifier,
> + struct mali_c55, notifier);
> + struct media_pad *pad = &mali_c55->isp.pads[MALI_C55_ISP_PAD_SINK_VIDEO];
> + int ret;
> +
> + /*
> + * By default we'll flag this link enabled and the TPG disabled, but
> + * no immutable flag because we need to be able to switch between the
> + * two.
> + */
> + ret = v4l2_create_fwnode_links_to_pad(subdev, pad,
> + MEDIA_LNK_FL_ENABLED);
> + if (ret)
> + dev_err(mali_c55->dev, "failed to create link for %s\n",
> + subdev->name);
> +
> + return ret;
> +}
> +
> +static int mali_c55_notifier_complete(struct v4l2_async_notifier *notifier)
> +{
> + struct mali_c55 *mali_c55 = container_of(notifier,
> + struct mali_c55, notifier);
> +
> + return v4l2_device_register_subdev_nodes(&mali_c55->v4l2_dev);
> +}
> +
> +static const struct v4l2_async_notifier_operations mali_c55_notifier_ops = {
> + .bound = mali_c55_notifier_bound,
> + .complete = mali_c55_notifier_complete,
> +};
> +
> +static int mali_c55_media_frameworks_init(struct mali_c55 *mali_c55)
> +{
> + int ret;
> +
> + media_device_init(&mali_c55->media_dev);
> + ret = media_device_register(&mali_c55->media_dev);
> + if (ret)
> + goto err_cleanup_media_device;
> +
> + mali_c55->v4l2_dev.mdev = &mali_c55->media_dev;
> + ret = v4l2_device_register(mali_c55->dev, &mali_c55->v4l2_dev);
> + if (ret) {
> + dev_err(mali_c55->dev, "failed to register V4L2 device\n");
> + goto err_unregister_media_device;
> + };
> +
> + mali_c55->notifier.ops = &mali_c55_notifier_ops;
> + v4l2_async_nf_init(&mali_c55->notifier, &mali_c55->v4l2_dev);
> +
> + ret = mali_c55_register_entities(mali_c55);
> + if (ret) {
> + dev_err(mali_c55->dev, "failed to register entities\n");
> + goto err_cleanup_nf;
> + }
> +
> + ret = mali_c55_parse_endpoint(mali_c55);
> + if (ret)
> + goto err_cleanup_nf;
> +
> + ret = v4l2_async_nf_register(&mali_c55->notifier);
> + if (ret) {
> + dev_err(mali_c55->dev, "failed to register notifier\n");
> + goto err_unregister_entities;
> + }
> +
> + return 0;
> +
> +err_unregister_entities:
> + mali_c55_unregister_entities(mali_c55);
> +err_cleanup_nf:
> + v4l2_async_nf_cleanup(&mali_c55->notifier);
> + v4l2_device_unregister(&mali_c55->v4l2_dev);
> +err_unregister_media_device:
> + media_device_unregister(&mali_c55->media_dev);
> +err_cleanup_media_device:
> + media_device_cleanup(&mali_c55->media_dev);
> +
> + return ret;
> +}
> +
> +static void mali_c55_media_frameworks_deinit(struct mali_c55 *mali_c55)
> +{
> + v4l2_async_nf_unregister(&mali_c55->notifier);
> + mali_c55_unregister_entities(mali_c55);
> + v4l2_async_nf_cleanup(&mali_c55->notifier);
> + v4l2_device_unregister(&mali_c55->v4l2_dev);
> + media_device_unregister(&mali_c55->media_dev);
> + media_device_cleanup(&mali_c55->media_dev);
> +}
> +
> +static int mali_c55_check_hwcfg(struct mali_c55 *mali_c55)
> +{
> + u32 product, version, revision, capabilities;
> +
> + product = mali_c55_read(mali_c55, MALI_C55_REG_PRODUCT);
> + version = mali_c55_read(mali_c55, MALI_C55_REG_VERSION);
> + revision = mali_c55_read(mali_c55, MALI_C55_REG_REVISION);
> +
> + mali_c55->media_dev.hw_revision = version;
> +
> + dev_info(mali_c55->dev, "Detected Mali-C55 ISP %u.%u.%u\n",
> + product, version, revision);
> +
> + capabilities = mali_c55_read(mali_c55,
> + MALI_C55_REG_GLOBAL_PARAMETER_STATUS);
> +
> + /*
> + * In its current iteration, the driver only supports inline mode. Given
> + * we cannot control input data timing in this mode, we cannot guarantee
> + * that the vertical blanking periods between frames will be long enough
> + * for us to write configuration data to the ISP during them. For that
> + * reason we can't really support single config space configuration
> + * until memory input mode is implemented.
> + */
> + if (!capabilities & MALI_C55_GPS_PONG_FITTED) {
> + dev_err(mali_c55->dev, "Pong config space not fitted.\n");
> + return -EINVAL;
> + }
> +
> + mali_c55->capabilities = (capabilities & 0xffff);
No need for parentheses.
> +
> + return 0;
> +}
> +
> +static void mali_c55_swap_next_config(struct mali_c55 *mali_c55)
> +{
> + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
> + u32 curr_config, next_config;
> +
> + curr_config = mali_c55_read(mali_c55, MALI_C55_REG_PING_PONG_READ);
> + curr_config = (curr_config & MALI_C55_REG_PING_PONG_READ_MASK)
> + >> (ffs(MALI_C55_REG_PING_PONG_READ_MASK) - 1);
> + next_config = curr_config ^ 1;
> +
> + mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG,
> + MALI_C55_REG_MCU_CONFIG_WRITE_MASK,
> + MALI_C55_MCU_CONFIG_WRITE(next_config));
How about
curr_config = mali_c55_read(mali_c55, MALI_C55_REG_PING_PONG_READ)
& MALI_C55_REG_PING_PONG_READ_MASK;
next_config = curr_config ^ MALI_C55_REG_PING_PONG_READ_MASK;
mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG,
MALI_C55_REG_MCU_CONFIG_WRITE_MASK,
next_config);
> + mali_c55_config_write(ctx, next_config ?
> + MALI_C55_CONFIG_PING : MALI_C55_CONFIG_PONG);
> +}
> +
> +static irqreturn_t mali_c55_isr(int irq, void *context)
> +{
> + struct device *dev = context;
> + struct mali_c55 *mali_c55 = dev_get_drvdata(dev);
> + u32 interrupt_status;
> + unsigned int i, j;
> +
> + interrupt_status = mali_c55_read(mali_c55,
> + MALI_C55_REG_INTERRUPT_STATUS_VECTOR);
> + if (!interrupt_status)
> + return IRQ_NONE;
> +
> + mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR_VECTOR,
> + interrupt_status);
> + mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR, 0);
> + mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR, 1);
> +
> + for (i = 0; i < MALI_C55_NUM_IRQ_BITS; i++) {
> + if (!(interrupt_status & (1 << i)))
> + continue;
> +
> + switch (i) {
> + case MALI_C55_IRQ_ISP_START:
> + mali_c55_isp_queue_event_sof(mali_c55);
> +
> + for (j = i; j < MALI_C55_NUM_CAP_DEVS; j++)
> + mali_c55_set_next_buffer(&mali_c55->cap_devs[j]);
> +
> + mali_c55_swap_next_config(mali_c55);
> +
> + break;
> + case MALI_C55_IRQ_ISP_DONE:
> + /*
> + * TODO: Where the ISP has no Pong config fitted, we'd
> + * have to do the mali_c55_swap_next_config() call here.
> + */
> + break;
> + case MALI_C55_IRQ_FR_Y_DONE:
> + mali_c55_set_plane_done(
> + &mali_c55->cap_devs[MALI_C55_CAP_DEV_FR],
> + MALI_C55_PLANE_Y);
> + break;
> + case MALI_C55_IRQ_FR_UV_DONE:
> + mali_c55_set_plane_done(
> + &mali_c55->cap_devs[MALI_C55_CAP_DEV_FR],
> + MALI_C55_PLANE_UV);
> + break;
> + case MALI_C55_IRQ_DS_Y_DONE:
> + mali_c55_set_plane_done(
> + &mali_c55->cap_devs[MALI_C55_CAP_DEV_DS],
> + MALI_C55_PLANE_Y);
> + break;
> + case MALI_C55_IRQ_DS_UV_DONE:
> + mali_c55_set_plane_done(
> + &mali_c55->cap_devs[MALI_C55_CAP_DEV_DS],
> + MALI_C55_PLANE_UV);
> + break;
> + default:
> + /*
> + * Only the above interrupts are currently unmasked. If
> + * we receive anything else here then something weird
> + * has gone on.
> + */
> + dev_err(dev, "masked interrupt %s triggered\n",
> + mali_c55_interrupt_names[i]);
> + }
> + }
> +
> + return IRQ_HANDLED;
> +}
> +
> +static int mali_c55_init_context(struct mali_c55 *mali_c55,
> + struct resource *res)
> +{
> + struct mali_c55_context *ctx = &mali_c55->context;
> +
> + ctx->base = res->start;
> + ctx->mali_c55 = mali_c55;
> +
> + ctx->registers = kzalloc(MALI_C55_CONFIG_SPACE_SIZE,
> + GFP_KERNEL | GFP_DMA);
> + if (!ctx->registers)
> + return -ENOMEM;
> +
> + /*
> + * The allocated memory is empty, we need to load the default
> + * register settings. We just read Ping; it's identical to Pong.
> + */
> + memcpy_fromio(ctx->registers,
> + mali_c55->base + config_space_addrs[MALI_C55_CONFIG_PING],
> + MALI_C55_CONFIG_SPACE_SIZE);
> +
> + /*
> + * Some features of the ISP need to be disabled by default and only
> + * enabled at the same time as they're configured by a parameters buffer
> + */
> +
> + /* Bypass the sqrt and square compression and expansion modules */
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BYPASS_1,
> + MALI_C55_REG_BYPASS_1_FE_SQRT,
> + MALI_C55_REG_BYPASS_1_FE_SQRT);
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BYPASS_3,
> + MALI_C55_REG_BYPASS_3_SQUARE_BE,
> + MALI_C55_REG_BYPASS_3_SQUARE_BE);
> +
> + /* Bypass the temper module */
> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_BYPASS_2,
> + MALI_C55_REG_BYPASS_2_TEMPER);
> +
> + /* Bypass the colour noise reduction */
> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_BYPASS_4,
> + MALI_C55_REG_BYPASS_4_CNR);
> +
> + /* Disable the sinter module */
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_SINTER_CONFIG,
> + MALI_C55_SINTER_ENABLE_MASK, 0x00);
> +
> + /* Disable the RGB Gamma module for each output */
> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_FR_GAMMA_RGB_ENABLE, 0x00);
As registers are 32-bit wide this would need to be 0x00000000, but I
think you can just write 0.
> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_DS_GAMMA_RGB_ENABLE, 0x00);
> +
> + /* Disable the colour correction matrix */
> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_CCM_ENABLE, 0x00);
> +
> + return 0;
> +}
> +
> +static int mali_c55_runtime_resume(struct device *dev)
> +{
> + struct mali_c55 *mali_c55 = dev_get_drvdata(dev);
> + int ret;
> +
> + ret = clk_bulk_prepare_enable(ARRAY_SIZE(mali_c55->clks),
> + mali_c55->clks);
> + if (ret) {
> + dev_err(mali_c55->dev, "failed to enable clocks\n");
> + return ret;
> + }
> +
> + /*
> + * Mask the interrupts and clear any that were set, then unmask the ones
> + * that we actually want to handle.
> + */
> + mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_MASK_VECTOR,
> + MALI_C55_INTERRUPT_MASK_ALL);
> + mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR_VECTOR,
> + MALI_C55_INTERRUPT_MASK_ALL);
> + mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR, 0x01);
> + mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR, 0x00);
> +
> + mali_c55_update_bits(mali_c55, MALI_C55_REG_INTERRUPT_MASK_VECTOR,
> + MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_ISP_START) |
> + MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_ISP_DONE) |
> + MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_FR_Y_DONE) |
> + MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_FR_UV_DONE) |
> + MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_DS_Y_DONE) |
> + MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_DS_UV_DONE),
> + 0x00);
> +
> + return 0;
> +}
> +
> +static int mali_c55_runtime_suspend(struct device *dev)
> +{
> + struct mali_c55 *mali_c55 = dev_get_drvdata(dev);
> +
> + clk_bulk_disable_unprepare(ARRAY_SIZE(mali_c55->clks), mali_c55->clks);
> + return 0;
> +}
> +
> +static const struct dev_pm_ops mali_c55_pm_ops = {
> + SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
> + pm_runtime_force_resume)
> + SET_RUNTIME_PM_OPS(mali_c55_runtime_suspend, mali_c55_runtime_resume,
> + NULL)
> +};
> +
> +static int mali_c55_probe(struct platform_device *pdev)
> +{
> + struct device *dev = &pdev->dev;
> + struct mali_c55 *mali_c55;
> + struct resource *res;
> + dma_cap_mask_t mask;
> + int ret;
> + u32 val;
> +
> + mali_c55 = devm_kzalloc(dev, sizeof(*mali_c55), GFP_KERNEL);
> + if (!mali_c55)
> + return -ENOMEM;
> +
> + mali_c55->dev = dev;
> + platform_set_drvdata(pdev, mali_c55);
> +
> + mali_c55->base = devm_platform_get_and_ioremap_resource(pdev, 0,
> + &res);
> + if (IS_ERR(mali_c55->base))
> + return dev_err_probe(dev, PTR_ERR(mali_c55->base),
> + "failed to map IO memory\n");
> +
> + for (unsigned int i = 0; i < ARRAY_SIZE(mali_c55_clk_names); i++)
> + mali_c55->clks[i].id = mali_c55_clk_names[i];
> +
> + ret = devm_clk_bulk_get(dev, ARRAY_SIZE(mali_c55->clks), mali_c55->clks);
> + if (ret)
> + return dev_err_probe(dev, ret, "failed to acquire clocks\n");
> +
> + of_reserved_mem_device_init(dev);
> + vb2_dma_contig_set_max_seg_size(dev, UINT_MAX);
> +
> + dma_cap_zero(mask);
> + dma_cap_set(DMA_MEMCPY, mask);
> +
> + pm_runtime_set_autosuspend_delay(&pdev->dev, 2000);
> + pm_runtime_use_autosuspend(&pdev->dev);
> + pm_runtime_enable(&pdev->dev);
> +
> + ret = pm_runtime_resume_and_get(&pdev->dev);
> + if (ret)
> + goto err_pm_runtime_disable;
> +
> + ret = mali_c55_check_hwcfg(mali_c55);
> + if (ret)
> + goto err_pm_runtime_put;
> +
> + /* Use "software only" context management. */
> + mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG,
> + MALI_C55_REG_MCU_CONFIG_OVERRIDE_MASK, 0x01);
Move this to the runtime PM resume handler, or to ISP start time, as
otherwise it may be lost when power is cut.
> +
> + /*
> + * No error check, because we will just fallback on memcpy if there is
> + * no usable DMA channel on the system.
> + */
> + mali_c55->channel = dma_request_channel(mask, NULL, NULL);
> +
> + ret = mali_c55_init_context(mali_c55, res);
> + if (ret)
> + goto err_release_dma_channel;
> +
> + mali_c55->media_dev.dev = dev;
> + strscpy(mali_c55->media_dev.model, "ARM Mali-C55 ISP",
> + sizeof(mali_c55->media_dev.model));
This can be moved to mali_c55_media_frameworks_init().
> +
> + ret = mali_c55_media_frameworks_init(mali_c55);
> + if (ret)
> + goto err_free_context_registers;
> +
> + /* Set safe stop to ensure we're in a non-streaming state */
> + mali_c55_write(mali_c55, MALI_C55_REG_INPUT_MODE_REQUEST,
> + MALI_C55_INPUT_SAFE_STOP);
> + readl_poll_timeout(mali_c55->base + MALI_C55_REG_MODE_STATUS,
> + val, !val, 10 * USEC_PER_MSEC, 250 * USEC_PER_MSEC);
Also a candidate for the runtime PM handler ?
> +
> + pm_runtime_put(&pdev->dev);
> +
> + ret = platform_get_irq(pdev, 0);
> + if (ret < 0) {
> + dev_err(dev, "failed to get interrupt\n");
> + goto err_deinit_media_frameworks;
> + }
> +
> + /*
> + * The driver needs to transfer large amounts of register settings to
> + * the ISP each frame, using either a DMA transfer or memcpy. We use a
> + * threaded IRQ to avoid disabling interrupts the entire time that's
> + * happening.
> + */
> + ret = devm_request_threaded_irq(&pdev->dev, ret, NULL,
> + mali_c55_isr, IRQF_ONESHOT,
> + dev_driver_string(&pdev->dev),
> + &pdev->dev);
> + if (ret) {
> + dev_err(dev, "failed to request irq\n");
> + goto err_deinit_media_frameworks;
> + }
> +
> + return 0;
> +
> +err_deinit_media_frameworks:
> + mali_c55_media_frameworks_deinit(mali_c55);
> +err_free_context_registers:
> + kfree(mali_c55->context.registers);
> +err_release_dma_channel:
> + dma_release_channel(mali_c55->channel);
> +err_pm_runtime_put:
> + pm_runtime_put(&pdev->dev);
> +err_pm_runtime_disable:
> + pm_runtime_disable(&pdev->dev);
> +
> + return ret;
> +}
> +
> +static void mali_c55_remove(struct platform_device *pdev)
> +{
> + struct mali_c55 *mali_c55 = platform_get_drvdata(pdev);
> +
> + kfree(mali_c55->context.registers);
> + mali_c55_media_frameworks_deinit(mali_c55);
> + dma_release_channel(mali_c55->channel);
> +}
> +
> +static const struct of_device_id mali_c55_of_match[] = {
> + { .compatible = "arm,mali-c55", },
> + { /* Sentinel */},
s/}/ }/
> +};
> +MODULE_DEVICE_TABLE(of, mali_c55_of_match);
> +
> +static struct platform_driver mali_c55_driver = {
> + .driver = {
> + .name = "mali-c55",
> + .of_match_table = mali_c55_of_match,
> + .pm = &mali_c55_pm_ops,
> + },
> + .probe = mali_c55_probe,
> + .remove_new = mali_c55_remove,
You can switch back to .remove().
> +};
> +
> +module_platform_driver(mali_c55_driver);
> +
> +MODULE_AUTHOR("Daniel Scally <dan.scally@ideasonboard.com>");
> +MODULE_AUTHOR("Jacopo Mondi <jacopo.mondi@ideasonboard.com>");
> +MODULE_DESCRIPTION("ARM Mali-C55 ISP platform driver");
> +MODULE_LICENSE("GPL");
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> new file mode 100644
> index 000000000000..f784983a4ccc
> --- /dev/null
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> @@ -0,0 +1,531 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * ARM Mali-C55 ISP Driver - Image signal processor
> + *
> + * Copyright (C) 2024 Ideas on Board Oy
> + */
> +
> +#include <linux/delay.h>
> +#include <linux/iopoll.h>
> +#include <linux/property.h>
> +#include <linux/string.h>
> +
> +#include <media/media-entity.h>
> +#include <media/v4l2-common.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-mc.h>
> +#include <media/v4l2-subdev.h>
> +
> +#include "mali-c55-common.h"
> +#include "mali-c55-registers.h"
> +
> +static const struct mali_c55_isp_fmt mali_c55_isp_fmts[] = {
> + {
> + .code = MEDIA_BUS_FMT_SRGGB20_1X20,
> + .order = MALI_C55_BAYER_ORDER_RGGB,
> + .bypass = false,
> + },
> + {
> + .code = MEDIA_BUS_FMT_SGRBG20_1X20,
> + .order = MALI_C55_BAYER_ORDER_GRBG,
> + .bypass = false,
> + },
> + {
> + .code = MEDIA_BUS_FMT_SGBRG20_1X20,
> + .order = MALI_C55_BAYER_ORDER_GBRG,
> + .bypass = false,
> + },
> + {
> + .code = MEDIA_BUS_FMT_SBGGR20_1X20,
> + .order = MALI_C55_BAYER_ORDER_BGGR,
> + .bypass = false,
> + },
> + {
> + .code = MEDIA_BUS_FMT_RGB202020_1X60,
> + .order = 0, /* Not relevant for this format */
> + .bypass = true,
> + }
> + /*
> + * TODO: Support MEDIA_BUS_FMT_YUV20_1X60 here. This is so that we can
> + * also support YUV input from a sensor passed-through to the output. At
> + * present we have no mechanism to test that though so it may have to
> + * wait a while...
> + */
> +};
> +
> +const struct mali_c55_isp_fmt *
> +mali_c55_isp_get_mbus_config_by_index(u32 index)
> +{
> + if (index < ARRAY_SIZE(mali_c55_isp_fmts))
> + return &mali_c55_isp_fmts[index];
> +
> + return NULL;
> +}
> +
> +const struct mali_c55_isp_fmt *
> +mali_c55_isp_get_mbus_config_by_code(u32 code)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(mali_c55_isp_fmts); i++) {
> + if (mali_c55_isp_fmts[i].code == code)
> + return &mali_c55_isp_fmts[i];
> + }
> +
> + return NULL;
> +}
> +
> +static void mali_c55_isp_stop(struct mali_c55 *mali_c55)
> +{
> + u32 val;
> +
> + mali_c55_write(mali_c55, MALI_C55_REG_INPUT_MODE_REQUEST,
> + MALI_C55_INPUT_SAFE_STOP);
> + readl_poll_timeout(mali_c55->base + MALI_C55_REG_MODE_STATUS,
> + val, !val, 10 * USEC_PER_MSEC, 250 * USEC_PER_MSEC);
> +}
> +
> +static int mali_c55_isp_start(struct mali_c55 *mali_c55)
> +{
> + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
> + struct v4l2_subdev *sd = &mali_c55->isp.sd;
> + const struct v4l2_mbus_framefmt *format;
> + const struct mali_c55_isp_fmt *cfg;
> + struct v4l2_subdev_state *state;
const, thanks to commit 85af84852f11 ("media: v4l2-subdev: Provide
const-aware subdev state accessors") that has landed in v6.11-rc1 :-)
> + const struct v4l2_rect *crop;
> + u32 val;
> + int ret;
> +
> + mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG,
> + MALI_C55_REG_MCU_CONFIG_WRITE_MASK,
> + MALI_C55_REG_MCU_CONFIG_WRITE_PING);
> +
> + /* Apply input windowing */
> + state = v4l2_subdev_get_locked_active_state(sd);
You have the state in the caller, you can pass it to this function.
> + crop = v4l2_subdev_state_get_crop(state, MALI_C55_ISP_PAD_SINK_VIDEO);
> + format = v4l2_subdev_state_get_format(state,
> + MALI_C55_ISP_PAD_SINK_VIDEO);
> + cfg = mali_c55_isp_get_mbus_config_by_code(format->code);
> +
> + mali_c55_write(mali_c55, MALI_C55_REG_HC_START,
> + MALI_C55_HC_START(crop->left));
> + mali_c55_write(mali_c55, MALI_C55_REG_HC_SIZE,
> + MALI_C55_HC_SIZE(crop->width));
> + mali_c55_write(mali_c55, MALI_C55_REG_VC_START_SIZE,
> + MALI_C55_VC_START(crop->top) |
> + MALI_C55_VC_SIZE(crop->height));
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BASE_ADDR,
> + MALI_C55_REG_ACTIVE_WIDTH_MASK, format->width);
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BASE_ADDR,
> + MALI_C55_REG_ACTIVE_HEIGHT_MASK,
> + format->height << 16);
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BAYER_ORDER,
> + MALI_C55_BAYER_ORDER_MASK, cfg->order);
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_INPUT_WIDTH,
> + MALI_C55_INPUT_WIDTH_MASK,
> + MALI_C55_INPUT_WIDTH_20BIT);
> +
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_ISP_RAW_BYPASS,
> + MALI_C55_ISP_RAW_BYPASS_BYPASS_MASK,
> + cfg->bypass ? MALI_C55_ISP_RAW_BYPASS_BYPASS_MASK :
> + 0x00);
> +
> + ret = mali_c55_config_write(ctx, MALI_C55_CONFIG_PING);
> + if (ret) {
> + dev_err(mali_c55->dev, "failed to write ISP config\n");
> + return ret;
> + }
> +
> + mali_c55_write(mali_c55, MALI_C55_REG_INPUT_MODE_REQUEST,
> + MALI_C55_INPUT_SAFE_START);
> + readl_poll_timeout(mali_c55->base + MALI_C55_REG_MODE_STATUS, val, !val,
> + 10 * USEC_PER_MSEC, 250 * USEC_PER_MSEC);
Should you return an error in case of timeout ?
> +
> + return 0;
> +}
> +
> +static int mali_c55_isp_enum_mbus_code(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + /*
> + * Only the internal RGB processed format is allowed on the regular
> + * processing source pad.
> + */
> + if (code->pad == MALI_C55_ISP_PAD_SOURCE_VIDEO) {
> + if (code->index)
> + return -EINVAL;
> +
> + code->code = MEDIA_BUS_FMT_RGB121212_1X36;
> + return 0;
> + }
> +
> + /* On the sink and bypass pads all the supported formats are allowed. */
> + if (code->index >= ARRAY_SIZE(mali_c55_isp_fmts))
> + return -EINVAL;
> +
> + code->code = mali_c55_isp_fmts[code->index].code;
> +
> + return 0;
> +}
> +
> +static int mali_c55_isp_enum_frame_size(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_frame_size_enum *fse)
> +{
> + const struct mali_c55_isp_fmt *cfg;
> +
> + if (fse->index > 0)
> + return -EINVAL;
> +
> + /*
> + * Only the internal RGB processed format is allowed on the regular
> + * processing source pad.
> + *
> + * On the sink and bypass pads all the supported formats are allowed.
> + */
> + if (fse->pad == MALI_C55_ISP_PAD_SOURCE_VIDEO) {
> + if (fse->code != MEDIA_BUS_FMT_RGB121212_1X36)
> + return -EINVAL;
> + } else {
> + cfg = mali_c55_isp_get_mbus_config_by_code(fse->code);
> + if (!cfg)
> + return -EINVAL;
> + }
> +
> + fse->min_width = MALI_C55_MIN_WIDTH;
> + fse->min_height = MALI_C55_MIN_HEIGHT;
> + fse->max_width = MALI_C55_MAX_WIDTH;
> + fse->max_height = MALI_C55_MAX_HEIGHT;
> +
> + return 0;
> +}
> +
> +static int mali_c55_isp_set_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *format)
> +{
> + struct v4l2_mbus_framefmt *fmt = &format->format;
> + struct v4l2_mbus_framefmt *src_fmt, *sink_fmt;
> + const struct mali_c55_isp_fmt *cfg;
> + struct v4l2_rect *crop;
> +
> + /*
> + * Disallow set_fmt on the source pads; format is fixed and the sizes
> + * are the result of applying the sink crop rectangle to the sink
> + * format.
> + */
> + if (format->pad != MALI_C55_ISP_PAD_SINK_VIDEO)
> + return v4l2_subdev_get_fmt(sd, state, format);
> +
> + sink_fmt = v4l2_subdev_state_get_format(state,
> + MALI_C55_ISP_PAD_SINK_VIDEO);
> +
> + cfg = mali_c55_isp_get_mbus_config_by_code(fmt->code);
> + sink_fmt->code = cfg ? fmt->code : MEDIA_BUS_FMT_SRGGB20_1X20;
> +
> + /*
> + * Clamp sizes in the accepted limits and clamp the crop rectangle in
> + * the new sizes.
> + */
> + sink_fmt->width = clamp(fmt->width, MALI_C55_MIN_WIDTH,
> + MALI_C55_MAX_WIDTH);
> + sink_fmt->height = clamp(fmt->height, MALI_C55_MIN_HEIGHT,
> + MALI_C55_MAX_HEIGHT);
> +
> + *fmt = *sink_fmt;
> +
> + crop = v4l2_subdev_state_get_crop(state, MALI_C55_ISP_PAD_SINK_VIDEO);
> + crop->left = 0;
> + crop->top = 0;
> + crop->width = sink_fmt->width;
> + crop->height = sink_fmt->height;
> +
> + /*
> + * Propagate format to source pads. On the 'regular' output pad use
> + * the internal RGB processed format, while on the bypass pad simply
> + * replicate the ISP sink format. The sizes on both pads are the same as
> + * the ISP sink crop rectangle. The "field" and "colorspace" fields are
> + * set in .init_state() and fixed for both source pads, as is the "code"
> + * field for the processed data source pad.
> + */
> + src_fmt = v4l2_subdev_state_get_format(state,
> + MALI_C55_ISP_PAD_SOURCE_VIDEO);
> + src_fmt->width = crop->width;
> + src_fmt->height = crop->height;
> +
> + src_fmt = v4l2_subdev_state_get_format(state,
> + MALI_C55_ISP_PAD_SOURCE_BYPASS);
> + src_fmt->code = sink_fmt->code;
> + src_fmt->width = crop->width;
> + src_fmt->height = crop->height;
> +
> + return 0;
> +}
> +
> +static int mali_c55_isp_get_selection(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_selection *sel)
> +{
> + if (sel->pad != MALI_C55_ISP_PAD_SINK_VIDEO ||
> + sel->target != V4L2_SEL_TGT_CROP)
> + return -EINVAL;
> +
> + sel->r = *v4l2_subdev_state_get_crop(state, MALI_C55_ISP_PAD_SINK_VIDEO);
> +
> + return 0;
> +}
> +
> +static int mali_c55_isp_set_selection(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_selection *sel)
> +{
> + struct v4l2_mbus_framefmt *src_fmt;
> + const struct v4l2_mbus_framefmt *fmt;
> + struct v4l2_rect *crop;
> +
> + if (sel->pad != MALI_C55_ISP_PAD_SINK_VIDEO ||
> + sel->target != V4L2_SEL_TGT_CROP)
> + return -EINVAL;
> +
> + fmt = v4l2_subdev_state_get_format(state, MALI_C55_ISP_PAD_SINK_VIDEO);
> +
> + sel->r.left = clamp_t(unsigned int, sel->r.left, 0, fmt->width);
> + sel->r.top = clamp_t(unsigned int, sel->r.top, 0, fmt->height);
> + sel->r.width = clamp_t(unsigned int, sel->r.width, MALI_C55_MIN_WIDTH,
> + fmt->width - sel->r.left);
> + sel->r.height = clamp_t(unsigned int, sel->r.height,
> + MALI_C55_MIN_HEIGHT,
> + fmt->height - sel->r.top);
> +
> + crop = v4l2_subdev_state_get_crop(state, MALI_C55_ISP_PAD_SINK_VIDEO);
> + *crop = sel->r;
> +
> + /*
> + * Propagate the crop rectangle sizes to the source pad format. The crop
> + * isn't propagated to the bypass source pad, because the bypassed data
> + * cannot be cropped.
> + */
> + src_fmt = v4l2_subdev_state_get_format(state,
> + MALI_C55_ISP_PAD_SOURCE_VIDEO);
> + src_fmt->width = crop->width;
> + src_fmt->height = crop->height;
> +
> + return 0;
> +}
> +
> +static int mali_c55_isp_enable_streams(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state, u32 pad,
> + u64 streams_mask)
> +{
> + struct mali_c55_isp *isp = container_of(sd, struct mali_c55_isp, sd);
> + struct mali_c55 *mali_c55 = isp->mali_c55;
> + struct v4l2_subdev *src_sd;
> + struct media_pad *sink_pad;
> + int ret;
> +
> + /*
> + * We have two source pads, both of which have only a single stream. The
> + * core v4l2 code already validated those parameters so we can just get
> + * on with starting the ISP.
> + */
> +
> + sink_pad = &isp->pads[MALI_C55_ISP_PAD_SINK_VIDEO];
> + isp->remote_src = media_pad_remote_pad_unique(sink_pad);
> + if (IS_ERR(isp->remote_src)) {
As you set the MUST_CONNECT flag for the MALI_C55_ISP_PAD_SINK_VIDEO pad
I think you can drop this check.
> + dev_err(mali_c55->dev, "Failed to get source for ISP\n");
> + return PTR_ERR(isp->remote_src);
> + }
> +
> + src_sd = media_entity_to_v4l2_subdev(isp->remote_src->entity);
> +
> + isp->frame_sequence = 0;
> + ret = mali_c55_isp_start(mali_c55);
> + if (ret) {
> + dev_err(mali_c55->dev, "Failed to start ISP\n");
> + isp->remote_src = NULL;
> + return ret;
> + }
> +
> + /*
> + * We only support a single input stream, so we can just enable the 1st
> + * entry in the streams mask.
> + */
> + ret = v4l2_subdev_enable_streams(src_sd, isp->remote_src->index, BIT(0));
> + if (ret) {
> + dev_err(mali_c55->dev, "Failed to start ISP source\n");
> + mali_c55_isp_stop(mali_c55);
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> +static int mali_c55_isp_disable_streams(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state, u32 pad,
> + u64 streams_mask)
> +{
> + struct mali_c55_isp *isp = container_of(sd, struct mali_c55_isp, sd);
> + struct mali_c55 *mali_c55 = isp->mali_c55;
> + struct v4l2_subdev *src_sd;
> +
> + if (isp->remote_src) {
> + src_sd = media_entity_to_v4l2_subdev(isp->remote_src->entity);
> + v4l2_subdev_disable_streams(src_sd, isp->remote_src->index,
> + BIT(0));
> + }
> + isp->remote_src = NULL;
> +
> + mali_c55_isp_stop(mali_c55);
> +
> + return 0;
> +}
> +
> +static const struct v4l2_subdev_pad_ops mali_c55_isp_pad_ops = {
> + .enum_mbus_code = mali_c55_isp_enum_mbus_code,
> + .enum_frame_size = mali_c55_isp_enum_frame_size,
> + .get_fmt = v4l2_subdev_get_fmt,
> + .set_fmt = mali_c55_isp_set_fmt,
> + .get_selection = mali_c55_isp_get_selection,
> + .set_selection = mali_c55_isp_set_selection,
> + .link_validate = v4l2_subdev_link_validate_default,
> + .enable_streams = mali_c55_isp_enable_streams,
> + .disable_streams = mali_c55_isp_disable_streams,
> +};
> +
> +void mali_c55_isp_queue_event_sof(struct mali_c55 *mali_c55)
> +{
> + struct v4l2_event event = {
> + .type = V4L2_EVENT_FRAME_SYNC,
> + };
> +
> + event.u.frame_sync.frame_sequence = mali_c55->isp.frame_sequence;
> + v4l2_event_queue(mali_c55->isp.sd.devnode, &event);
> +}
> +
> +static int
> +mali_c55_isp_subscribe_event(struct v4l2_subdev *sd, struct v4l2_fh *fh,
> + struct v4l2_event_subscription *sub)
> +{
> + if (sub->type != V4L2_EVENT_FRAME_SYNC)
> + return -EINVAL;
> +
> + /* V4L2_EVENT_FRAME_SYNC doesn't require an id, so zero should be set */
> + if (sub->id != 0)
> + return -EINVAL;
> +
> + return v4l2_event_subscribe(fh, sub, 0, NULL);
> +}
> +
> +static const struct v4l2_subdev_core_ops mali_c55_isp_core_ops = {
> + .subscribe_event = mali_c55_isp_subscribe_event,
> + .unsubscribe_event = v4l2_event_subdev_unsubscribe,
> +};
> +
> +static const struct v4l2_subdev_ops mali_c55_isp_ops = {
> + .pad = &mali_c55_isp_pad_ops,
> + .core = &mali_c55_isp_core_ops,
> +};
> +
> +static int mali_c55_isp_init_state(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state)
> +{
> + struct v4l2_mbus_framefmt *sink_fmt, *src_fmt;
> + struct v4l2_rect *in_crop;
> +
> + sink_fmt = v4l2_subdev_state_get_format(state,
> + MALI_C55_ISP_PAD_SINK_VIDEO);
> + src_fmt = v4l2_subdev_state_get_format(state,
> + MALI_C55_ISP_PAD_SOURCE_VIDEO);
> + in_crop = v4l2_subdev_state_get_crop(state,
> + MALI_C55_ISP_PAD_SINK_VIDEO);
> +
> + sink_fmt->width = MALI_C55_DEFAULT_WIDTH;
> + sink_fmt->height = MALI_C55_DEFAULT_HEIGHT;
> + sink_fmt->field = V4L2_FIELD_NONE;
> + sink_fmt->code = MEDIA_BUS_FMT_SRGGB20_1X20;
> + sink_fmt->colorspace = V4L2_COLORSPACE_RAW;
You should initialize the three other colorspace-related fields :-) Same
below.
> +
> + *v4l2_subdev_state_get_format(state,
> + MALI_C55_ISP_PAD_SOURCE_BYPASS) = *sink_fmt;
> +
> + src_fmt->width = MALI_C55_DEFAULT_WIDTH;
> + src_fmt->height = MALI_C55_DEFAULT_HEIGHT;
> + src_fmt->field = V4L2_FIELD_NONE;
> + src_fmt->code = MEDIA_BUS_FMT_RGB121212_1X36;
> + src_fmt->colorspace = V4L2_COLORSPACE_SRGB;
> +
> + in_crop->top = 0;
> + in_crop->left = 0;
> + in_crop->width = MALI_C55_DEFAULT_WIDTH;
> + in_crop->height = MALI_C55_DEFAULT_HEIGHT;
> +
> + return 0;
> +}
> +
> +static const struct v4l2_subdev_internal_ops mali_c55_isp_internal_ops = {
> + .init_state = mali_c55_isp_init_state,
> +};
> +
> +static const struct media_entity_operations mali_c55_isp_media_ops = {
> + .link_validate = v4l2_subdev_link_validate,
.link_validate = v4l2_subdev_link_validate,
to match mali_c55_isp_internal_ops.
> +};
> +
> +int mali_c55_register_isp(struct mali_c55 *mali_c55)
> +{
> + struct mali_c55_isp *isp = &mali_c55->isp;
> + struct v4l2_subdev *sd = &isp->sd;
> + int ret;
> +
> + isp->mali_c55 = mali_c55;
> +
> + v4l2_subdev_init(sd, &mali_c55_isp_ops);
> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
> + sd->entity.ops = &mali_c55_isp_media_ops;
> + sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_ISP;
> + sd->internal_ops = &mali_c55_isp_internal_ops;
> + strscpy(sd->name, MALI_C55_DRIVER_NAME " isp", sizeof(sd->name));
> +
> + isp->pads[MALI_C55_ISP_PAD_SINK_VIDEO].flags = MEDIA_PAD_FL_SINK |
> + MEDIA_PAD_FL_MUST_CONNECT;
> + isp->pads[MALI_C55_ISP_PAD_SOURCE_VIDEO].flags = MEDIA_PAD_FL_SOURCE;
> + isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE;
> +
> + ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS,
> + isp->pads);
> + if (ret)
> + return ret;
> +
> + ret = v4l2_subdev_init_finalize(sd);
> + if (ret)
> + goto err_cleanup_media_entity;
> +
> + ret = v4l2_device_register_subdev(&mali_c55->v4l2_dev, sd);
> + if (ret)
> + goto err_cleanup_subdev;
> +
> + mutex_init(&isp->capture_lock);
> +
> + return 0;
> +
> +err_cleanup_subdev:
> + v4l2_subdev_cleanup(sd);
> +err_cleanup_media_entity:
> + media_entity_cleanup(&sd->entity);
> + isp->mali_c55 = NULL;
> +
> + return ret;
> +}
> +
> +void mali_c55_unregister_isp(struct mali_c55 *mali_c55)
> +{
> + struct mali_c55_isp *isp = &mali_c55->isp;
> +
> + if (!isp->mali_c55)
> + return;
> +
> + mutex_destroy(&isp->capture_lock);
> + v4l2_device_unregister_subdev(&isp->sd);
> + v4l2_subdev_cleanup(&isp->sd);
> + media_entity_cleanup(&isp->sd.entity);
> +}
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
> new file mode 100644
> index 000000000000..0a391f8a043e
> --- /dev/null
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
> @@ -0,0 +1,313 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * ARM Mali-C55 ISP Driver - Register definitions
> + *
> + * Copyright (C) 2024 Ideas on Board Oy
> + */
> +
> +#ifndef _MALI_C55_REGISTERS_H
> +#define _MALI_C55_REGISTERS_H
> +
> +#include <linux/bits.h>
> +
> +/* ISP Common 0x00000 - 0x000ff */
> +
> +#define MALI_C55_REG_API 0x00000
> +#define MALI_C55_REG_PRODUCT 0x00004
> +#define MALI_C55_REG_VERSION 0x00008
> +#define MALI_C55_REG_REVISION 0x0000c
> +#define MALI_C55_REG_PULSE_MODE 0x0003c
> +#define MALI_C55_REG_INPUT_MODE_REQUEST 0x0009c
> +#define MALI_C55_INPUT_SAFE_STOP 0x00
> +#define MALI_C55_INPUT_SAFE_START 0x01
> +#define MALI_C55_REG_MODE_STATUS 0x000a0
> +#define MALI_C55_REG_INTERRUPT_MASK_VECTOR 0x00030
> +#define MALI_C55_INTERRUPT_MASK_ALL GENMASK(31, 0)
> +
> +#define MALI_C55_REG_GLOBAL_MONITOR 0x00050
> +
> +#define MALI_C55_REG_GEN_VIDEO 0x00080
> +#define MALI_C55_REG_GEN_VIDEO_ON_MASK BIT(0)
> +#define MALI_C55_REG_GEN_VIDEO_MULTI_MASK BIT(1)
> +#define MALI_C55_REG_GEN_PREFETCH_MASK GENMASK(31, 16)
> +
> +#define MALI_C55_REG_MCU_CONFIG 0x00020
> +#define MALI_C55_REG_MCU_CONFIG_OVERRIDE_MASK BIT(0)
For single bit fields like this one, you can rename the macro to
MALI_C55_REG_MCU_CONFIG_OVERRIDE and write
mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG,
MALI_C55_REG_MCU_CONFIG_OVERRIDE,
MALI_C55_REG_MCU_CONFIG_OVERRIDE);
> +#define MALI_C55_REG_MCU_CONFIG_WRITE_MASK BIT(1)
> +#define MALI_C55_MCU_CONFIG_WRITE(x) ((x) << 1)
> +#define MALI_C55_REG_MCU_CONFIG_WRITE_PING BIT(1)
> +#define MALI_C55_REG_MCU_CONFIG_WRITE_PONG 0x00
> +#define MALI_C55_REG_MULTI_CONTEXT_MODE_MASK BIT(8)
> +#define MALI_C55_REG_PING_PONG_READ 0x00024
> +#define MALI_C55_REG_PING_PONG_READ_MASK BIT(2)
> +
> +#define MALI_C55_REG_INTERRUPT_CLEAR_VECTOR 0x00034
> +#define MALI_C55_REG_INTERRUPT_CLEAR 0x00040
> +#define MALI_C55_REG_INTERRUPT_STATUS_VECTOR 0x00044
> +
> +enum mali_c55_interrupts {
> + MALI_C55_IRQ_ISP_START,
> + MALI_C55_IRQ_ISP_DONE,
> + MALI_C55_IRQ_MCM_ERROR,
> + MALI_C55_IRQ_BROKEN_FRAME_ERROR,
> + MALI_C55_IRQ_MET_AF_DONE,
> + MALI_C55_IRQ_MET_AEXP_DONE,
> + MALI_C55_IRQ_MET_AWB_DONE,
> + MALI_C55_IRQ_AEXP_1024_DONE,
> + MALI_C55_IRQ_IRIDIX_MET_DONE,
> + MALI_C55_IRQ_LUT_INIT_DONE,
> + MALI_C55_IRQ_FR_Y_DONE,
> + MALI_C55_IRQ_FR_UV_DONE,
> + MALI_C55_IRQ_DS_Y_DONE,
> + MALI_C55_IRQ_DS_UV_DONE,
> + MALI_C55_IRQ_LINEARIZATION_DONE,
> + MALI_C55_IRQ_RAW_FRONTEND_DONE,
> + MALI_C55_IRQ_NOISE_REDUCTION_DONE,
> + MALI_C55_IRQ_IRIDIX_DONE,
> + MALI_C55_IRQ_BAYER2RGB_DONE,
> + MALI_C55_IRQ_WATCHDOG_TIMER,
> + MALI_C55_IRQ_FRAME_COLLISION,
> + MALI_C55_IRQ_UNUSED,
> + MALI_C55_IRQ_DMA_ERROR,
> + MALI_C55_IRQ_INPUT_STOPPED,
> + MALI_C55_IRQ_MET_AWB_TARGET1_HIT,
> + MALI_C55_IRQ_MET_AWB_TARGET2_HIT,
> + MALI_C55_NUM_IRQ_BITS
> +};
> +
> +#define MALI_C55_INTERRUPT_BIT(x) (0x01 << (x))
#define MALI_C55_INTERRUPT_BIT(x) BIT(x)
> +
> +#define MALI_C55_REG_GLOBAL_PARAMETER_STATUS 0x00068
> +#define MALI_C55_GPS_PONG_FITTED BIT(0)
> +#define MALI_C55_GPS_WDR_FITTED BIT(1)
> +#define MALI_C55_GPS_COMPRESSION_FITTED BIT(2)
> +#define MALI_C55_GPS_TEMPER_FITTED BIT(3)
> +#define MALI_C55_GPS_SINTER_LITE_FITTED BIT(4)
> +#define MALI_C55_GPS_SINTER_FITTED BIT(5)
> +#define MALI_C55_GPS_IRIDIX_LTM_FITTED BIT(6)
> +#define MALI_C55_GPS_IRIDIX_GTM_FITTED BIT(7)
> +#define MALI_C55_GPS_CNR_FITTED BIT(8)
> +#define MALI_C55_GPS_FRSCALER_FITTED BIT(9)
> +#define MALI_C55_GPS_DS_PIPE_FITTED BIT(10)
> +
> +#define MALI_C55_REG_BLANKING 0x00084
> +#define MALI_C55_REG_HBLANK_MASK GENMASK(15, 0)
> +#define MALI_C55_REG_VBLANK_MASK GENMASK(31, 16)
> +#define MALI_C55_VBLANK(x) ((x) << 16)
> +
> +#define MALI_C55_REG_HC_START 0x00088
> +#define MALI_C55_HC_START(h) (((h) & 0xffff) << 16)
> +#define MALI_C55_REG_HC_SIZE 0x0008c
> +#define MALI_C55_HC_SIZE(h) ((h) & 0xffff)
> +#define MALI_C55_REG_VC_START_SIZE 0x00094
> +#define MALI_C55_VC_START(v) ((v) & 0xffff)
> +#define MALI_C55_VC_SIZE(v) (((v) & 0xffff) << 16)
> +
> +/* Ping/Pong Configuration Space */
> +#define MALI_C55_REG_BASE_ADDR 0x18e88
> +#define MALI_C55_REG_BYPASS_0 0x18eac
> +#define MALI_C55_REG_BYPASS_0_VIDEO_TEST BIT(0)
> +#define MALI_C55_REG_BYPASS_0_INPUT_FMT BIT(1)
> +#define MALI_C55_REG_BYPASS_0_DECOMPANDER BIT(2)
> +#define MALI_C55_REG_BYPASS_0_SENSOR_OFFSET_WDR BIT(3)
> +#define MALI_C55_REG_BYPASS_0_GAIN_WDR BIT(4)
> +#define MALI_C55_REG_BYPASS_0_FRAME_STITCH BIT(5)
> +#define MALI_C55_REG_BYPASS_1 0x18eb0
> +#define MALI_C55_REG_BYPASS_1_DIGI_GAIN BIT(0)
> +#define MALI_C55_REG_BYPASS_1_FE_SENSOR_OFFS BIT(1)
> +#define MALI_C55_REG_BYPASS_1_FE_SQRT BIT(2)
> +#define MALI_C55_REG_BYPASS_1_RAW_FE BIT(3)
> +#define MALI_C55_REG_BYPASS_2 0x18eb8
> +#define MALI_C55_REG_BYPASS_2_SINTER BIT(0)
> +#define MALI_C55_REG_BYPASS_2_TEMPER BIT(1)
> +#define MALI_C55_REG_BYPASS_3 0x18ebc
> +#define MALI_C55_REG_BYPASS_3_SQUARE_BE BIT(0)
> +#define MALI_C55_REG_BYPASS_3_SENSOR_OFFSET_PRE_SH BIT(1)
> +#define MALI_C55_REG_BYPASS_3_MESH_SHADING BIT(3)
> +#define MALI_C55_REG_BYPASS_3_WHITE_BALANCE BIT(4)
> +#define MALI_C55_REG_BYPASS_3_IRIDIX BIT(5)
> +#define MALI_C55_REG_BYPASS_3_IRIDIX_GAIN BIT(6)
> +#define MALI_C55_REG_BYPASS_4 0x18ec0
> +#define MALI_C55_REG_BYPASS_4_DEMOSAIC_RGB BIT(1)
> +#define MALI_C55_REG_BYPASS_4_PF_CORRECTION BIT(3)
> +#define MALI_C55_REG_BYPASS_4_CCM BIT(4)
> +#define MALI_C55_REG_BYPASS_4_CNR BIT(5)
> +#define MALI_C55_REG_FR_BYPASS 0x18ec4
> +#define MALI_C55_REG_DS_BYPASS 0x18ec8
> +#define MALI_C55_BYPASS_CROP BIT(0)
> +#define MALI_C55_BYPASS_SCALER BIT(1)
> +#define MALI_C55_BYPASS_GAMMA_RGB BIT(2)
> +#define MALI_C55_BYPASS_SHARPEN BIT(3)
> +#define MALI_C55_BYPASS_CS_CONV BIT(4)
> +#define MALI_C55_REG_ISP_RAW_BYPASS 0x18ecc
> +#define MALI_C55_ISP_RAW_BYPASS_BYPASS_MASK BIT(0)
> +#define MALI_C55_ISP_RAW_BYPASS_FR_BYPASS_MASK GENMASK(9, 8)
> +#define MALI_C55_ISP_RAW_BYPASS_RAW_FR_BYPASS (2 << 8)
> +#define MALI_C55_ISP_RAW_BYPASS_RGB_FR_BYPASS (1 << 8)
> +#define MALI_C55_ISP_RAW_BYPASS_DS_PIPE_DISABLE BIT(1)
> +#define MALI_C55_ISP_RAW_BYPASS_RAW_BYPASS BIT(0)
> +
> +#define MALI_C55_REG_ACTIVE_WIDTH_MASK 0xffff
> +#define MALI_C55_REG_ACTIVE_HEIGHT_MASK 0xffff0000
> +#define MALI_C55_REG_BAYER_ORDER 0x18e8c
> +#define MALI_C55_BAYER_ORDER_MASK GENMASK(1, 0)
> +#define MALI_C55_BAYER_ORDER_RGGB 0
> +#define MALI_C55_BAYER_ORDER_GRBG 1
> +#define MALI_C55_BAYER_ORDER_GBRG 2
> +#define MALI_C55_BAYER_ORDER_BGGR 3
> +
> +#define MALI_C55_REG_TPG_CH0 0x18ed8
> +#define MALI_C55_TEST_PATTERN_ON_OFF BIT(0)
> +#define MALI_C55_TEST_PATTERN_RGB_MASK BIT(1)
> +#define MALI_C55_TEST_PATTERN_RGB(x) ((x) << 1)
> +#define MALI_C55_REG_TPG_R_BACKGROUND 0x18ee0
> +#define MALI_C55_REG_TPG_G_BACKGROUND 0x18ee4
> +#define MALI_C55_REG_TPG_B_BACKGROUND 0x18ee8
> +#define MALI_C55_TPG_BACKGROUND_MAX 0xfffff
> +#define MALI_C55_REG_INPUT_WIDTH 0x18f98
> +#define MALI_C55_INPUT_WIDTH_MASK GENMASK(18, 16)
> +#define MALI_C55_INPUT_WIDTH_8BIT (0 << 16)
> +#define MALI_C55_INPUT_WIDTH_10BIT (1 << 16)
> +#define MALI_C55_INPUT_WIDTH_12BIT (2 << 16)
> +#define MALI_C55_INPUT_WIDTH_14BIT (3 << 16)
> +#define MALI_C55_INPUT_WIDTH_16BIT (4 << 16)
> +#define MALI_C55_INPUT_WIDTH_20BIT (5 << 16)
> +#define MALI_C55_REG_SPACE_SIZE 0x4000
> +#define MALI_C55_REG_CONFIG_SPACES_OFFSET 0x0ab6c
> +#define MALI_C55_CONFIG_SPACE_SIZE 0x1231c
> +
> +#define MALI_C55_REG_SINTER_CONFIG 0x19348
> +#define MALI_C55_SINTER_VIEW_FILTER_MASK GENMASK(1, 0)
> +#define MALI_C55_SINTER_SCALE_MODE_MASK GENMASK(3, 2)
> +#define MALI_C55_SINTER_ENABLE_MASK BIT(4)
> +#define MALI_C55_SINTER_FILTER_SELECT_MASK BIT(5)
> +#define MALI_C55_SINTER_INT_SELECT_MASK BIT(6)
> +#define MALI_C55_SINTER_RM_ENABLE_MASK BIT(7)
> +
> +/* Colour Correction Matrix Configuration */
> +#define MALI_C55_REG_CCM_ENABLE 0x1b07c
> +#define MALI_C55_CCM_ENABLE_MASK BIT(0)
> +#define MALI_C55_REG_CCM_COEF_R_R 0x1b080
> +#define MALI_C55_REG_CCM_COEF_R_G 0x1b084
> +#define MALI_C55_REG_CCM_COEF_R_B 0x1b088
> +#define MALI_C55_REG_CCM_COEF_G_R 0x1b090
> +#define MALI_C55_REG_CCM_COEF_G_G 0x1b094
> +#define MALI_C55_REG_CCM_COEF_G_B 0x1b098
> +#define MALI_C55_REG_CCM_COEF_B_R 0x1b0a0
> +#define MALI_C55_REG_CCM_COEF_B_G 0x1b0a4
> +#define MALI_C55_REG_CCM_COEF_B_B 0x1b0a8
> +#define MALI_C55_CCM_COEF_MASK GENMASK(12, 0)
> +#define MALI_C55_REG_CCM_ANTIFOG_GAIN_R 0x1b0b0
> +#define MALI_C55_REG_CCM_ANTIFOG_GAIN_G 0x1b0b4
> +#define MALI_C55_REG_CCM_ANTIFOG_GAIN_B 0x1b0b8
> +#define MALI_C55_CCM_ANTIFOG_GAIN_MASK GENMASK(11, 0)
> +#define MALI_C55_REG_CCM_ANTIFOG_OFFSET_R 0x1b0c0
> +#define MALI_C55_REG_CCM_ANTIFOG_OFFSET_G 0x1b0c4
> +#define MALI_C55_REG_CCM_ANTIFOG_OFFSET_B 0x1b0c8
> +#define MALI_C55_CCM_ANTIFOG_OFFSET_MASK GENMASK(11, 0)
> +
> +/*
> + * The Mali-C55 ISP has up to two output pipes; known as full resolution and
> + * down scaled. The register space for these is laid out identically, but offset
> + * by 372 bytes.
> + */
> +#define MALI_C55_CAP_DEV_FR_REG_OFFSET 0x0
> +#define MALI_C55_CAP_DEV_DS_REG_OFFSET 0x174
> +
> +#define MALI_C55_REG_CS_CONV_CONFIG 0x1c098
> +#define MALI_C55_CS_CONV_MATRIX_MASK BIT(0)
> +#define MALI_C55_CS_CONV_FILTER_MASK BIT(1)
> +#define MALI_C55_CS_CONV_HORZ_DOWNSAMPLE_MASK BIT(2)
> +#define MALI_C55_CS_CONV_VERT_DOWNSAMPLE_MASK BIT(3)
> +#define MALI_C55_CS_CONV_FILTER_ENABLE (0x01 << 1)
> +#define MALI_C55_CS_CONV_HORZ_DOWNSAMPLE_ENABLE (0x01 << 2)
> +#define MALI_C55_CS_CONV_VERT_DOWNSAMPLE_ENABLE (0x01 << 3)
> +#define MALI_C55_REG_Y_WRITER_MODE 0x1c0ec
> +#define MALI_C55_REG_UV_WRITER_MODE 0x1c144
> +#define MALI_C55_WRITER_MODE_MASK GENMASK(4, 0)
> +#define MALI_C55_OUTPUT_DISABLED 0
> +#define MALI_C55_OUTPUT_RGB32 1
> +#define MALI_C55_OUTPUT_A2R10G10B10 2
> +#define MALI_C55_OUTPUT_RGB565 3
> +#define MALI_C55_OUTPUT_RGB24 4
> +#define MALI_C55_OUTPUT_GEN32 5
> +#define MALI_C55_OUTPUT_RAW16 6
> +#define MALI_C55_OUTPUT_AYUV 8
> +#define MALI_C55_OUTPUT_Y410 9
> +#define MALI_C55_OUTPUT_YUY2 10
> +#define MALI_C55_OUTPUT_UYVY 11
> +#define MALI_C55_OUTPUT_Y210 12
> +#define MALI_C55_OUTPUT_NV12_21 13
> +#define MALI_C55_OUTPUT_YUV_420_422 17
> +#define MALI_C55_OUTPUT_P210_P010 19
> +#define MALI_C55_OUTPUT_YUV422 20
> +#define MALI_C55_WRITER_SUBMODE_MASK GENMASK(7, 6)
> +#define MALI_C55_WRITER_SUBMODE(x) ((x) << 6)
> +#define MALI_C55_OUTPUT_PLANE_ALT0 0
> +#define MALI_C55_OUTPUT_PLANE_ALT1 1
> +#define MALI_C55_OUTPUT_PLANE_ALT2 2
> +#define MALI_C55_WRITER_FRAME_WRITE_MASK BIT(9)
> +#define MALI_C55_WRITER_FRAME_WRITE_ENABLE (0x01 << 9)
> +#define MALI_C55_REG_ACTIVE_OUT_Y_SIZE 0x1c0f0
> +#define MALI_C55_REG_ACTIVE_OUT_UV_SIZE 0x1c148
> +#define MALI_C55_REG_ACTIVE_OUT_SIZE_W(w) ((w) << 0)
> +#define MALI_C55_REG_ACTIVE_OUT_SIZE_H(h) ((h) << 16)
> +#define MALI_C55_REG_Y_WRITER_BANKS_BASE 0x1c0f4
> +#define MALI_C55_REG_Y_WRITER_BANKS_CONFIG 0x1c108
> +#define MALI_C55_REG_Y_WRITER_MAX_BANKS_MASK GENMASK(2, 0)
> +#define MALI_C55_REG_Y_WRITER_BANKS_RESTART BIT(3)
> +#define MALI_C55_REG_Y_WRITER_OFFSET 0x1c10c
> +#define MALI_C55_REG_UV_WRITER_BANKS_BASE 0x1c14c
> +#define MALI_C55_REG_UV_WRITER_BANKS_CONFIG 0x1c160
> +#define MALI_C55_REG_UV_WRITER_MAX_BANKS_MASK GENMASK(2, 0)
> +#define MALI_C55_REG_UV_WRITER_BANKS_RESTART BIT(3)
> +#define MALI_C55_REG_UV_WRITER_OFFSET 0x1c164
> +
> +#define MALI_C55_REG_TEST_GEN_CH0_OFF_ON
> +#define MALI_C55_REG_TEST_GEN_CH0_PATTERN_TYPE 0x18edc
> +
> +#define MALI_C55_REG_CROP_EN 0x1c028
> +#define MALI_C55_CROP_ENABLE BIT(0)
> +#define MALI_C55_REG_CROP_X_START 0x1c02c
> +#define MALI_C55_REG_CROP_Y_START 0x1c030
> +#define MALI_C55_REG_CROP_X_SIZE 0x1c034
> +#define MALI_C55_REG_CROP_Y_SIZE 0x1c038
> +#define MALI_C55_REG_SCALER_TIMEOUT_EN 0x1c040
> +#define MALI_C55_SCALER_TIMEOUT_EN BIT(4)
> +#define MALI_C55_SCALER_TIMEOUT(t) ((t) << 16)
> +#define MALI_C55_REG_SCALER_IN_WIDTH 0x1c044
> +#define MALI_C55_REG_SCALER_IN_HEIGHT 0x1c048
> +#define MALI_C55_REG_SCALER_OUT_WIDTH 0x1c04c
> +#define MALI_C55_REG_SCALER_OUT_HEIGHT 0x1c050
> +#define MALI_C55_REG_SCALER_HFILT_TINC 0x1c054
> +#define MALI_C55_REG_SCALER_HFILT_COEF 0x1c058
> +#define MALI_C55_REG_SCALER_VFILT_TINC 0x1c05c
> +#define MALI_C55_REG_SCALER_VFILT_COEF 0x1c060
> +
> +#define MALI_C55_REG_GAMMA_RGB_ENABLE 0x1c064
> +#define MALI_C55_GAMMA_ENABLE_MASK BIT(0)
> +#define MALI_C55_REG_GAMMA_GAINS_1 0x1c068
> +#define MALI_C55_GAMMA_GAIN_R_MASK GENMASK(11, 0)
> +#define MALI_C55_GAMMA_GAIN_G_MASK GENMASK(27, 16)
> +#define MALI_C55_REG_GAMMA_GAINS_2 0x1c06c
> +#define MALI_C55_GAMMA_GAIN_B_MASK GENMASK(11, 0)
> +#define MALI_C55_REG_GAMMA_OFFSETS_1 0x1c070
> +#define MALI_C55_GAMMA_OFFSET_R_MASK GENMASK(11, 0)
> +#define MALI_C55_GAMMA_OFFSET_G_MASK GENMASK(27, 16)
> +#define MALI_C55_REG_GAMMA_OFFSETS_2 0x1c074
> +#define MALI_C55_GAMMA_OFFSET_B_MASK GENMASK(11, 0)
> +
> +/*
> + * A re-definition of an above register. These will usually be written on a per
> + * capture device basis and handled with mali_c55_cap_dev_write(), but on
> + * startup is written by core.c
> + */
> +#define MALI_C55_REG_FR_GAMMA_RGB_ENABLE 0x1c064
> +#define MALI_C55_REG_DS_GAMMA_RGB_ENABLE 0x1c1d8
> +
> +#define MALI_C55_REG_FR_SCALER_HFILT 0x34a8
> +#define MALI_C55_REG_FR_SCALER_VFILT 0x44a8
> +#define MALI_C55_REG_DS_SCALER_HFILT 0x14a8
> +#define MALI_C55_REG_DS_SCALER_VFILT 0x24a8
> +
> +#endif /* _MALI_C55_REGISTERS_H */
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c b/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
> new file mode 100644
> index 000000000000..b63a94cbe2fc
> --- /dev/null
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
> @@ -0,0 +1,1096 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * ARM Mali-C55 ISP Driver - Image signal processor
> + *
> + * Copyright (C) 2024 Ideas on Board Oy
> + */
> +
> +#include <linux/math.h>
> +#include <linux/minmax.h>
> +
> +#include <media/media-entity.h>
> +#include <media/v4l2-subdev.h>
> +
> +#include "mali-c55-common.h"
> +#include "mali-c55-registers.h"
> +
> +/* Scaling factor in Q4.20 format. */
> +#define MALI_C55_RSZ_SCALER_FACTOR (1U << 20)
> +
> +#define MALI_C55_RSZ_COEFS_BANKS 8
> +#define MALI_C55_RSZ_COEFS_ENTRIES 64
> +
> +static inline struct mali_c55_resizer *
> +sd_to_mali_c55_rsz(struct v4l2_subdev *sd)
> +{
> + return container_of(sd, struct mali_c55_resizer, sd);
> +}
> +
> +static const unsigned int
> +mali_c55_rsz_filter_coeffs_h[MALI_C55_RSZ_COEFS_BANKS]
> + [MALI_C55_RSZ_COEFS_ENTRIES] = {
> + { /* Bank 0 */
> + 0x24fc0000, 0x0000fc24, 0x27fc0000, 0x0000fc21,
> + 0x28fc0000, 0x0000fd1f, 0x2cfb0000, 0x0000fd1c,
> + 0x2efb0000, 0x0000fd1a, 0x30fb0000, 0x0000fe17,
> + 0x32fb0000, 0x0000fe15, 0x35fb0000, 0x0000fe12,
> + 0x35fc0000, 0x0000ff10, 0x37fc0000, 0x0000ff0e,
> + 0x39fc0000, 0x0000ff0c, 0x3afd0000, 0x0000ff0a,
> + 0x3afe0000, 0x00000008, 0x3cfe0000, 0x00000006,
> + 0x3dff0000, 0x00000004, 0x3d000000, 0x00000003,
> + 0x3c020000, 0x00000002, 0x3d030000, 0x00000000,
> + 0x3d040000, 0x000000ff, 0x3c060000, 0x000000fe,
> + 0x3a080000, 0x000000fe, 0x3a0aff00, 0x000000fd,
> + 0x390cff00, 0x000000fc, 0x370eff00, 0x000000fc,
> + 0x3510ff00, 0x000000fc, 0x3512fe00, 0x000000fb,
> + 0x3215fe00, 0x000000fb, 0x3017fe00, 0x000000fb,
> + 0x2e1afd00, 0x000000fb, 0x2c1cfd00, 0x000000fb,
> + 0x281ffd00, 0x000000fc, 0x2721fc00, 0x000000fc,
> + },
> + { /* Bank 1 */
> + 0x25fb0000, 0x0000fb25, 0x27fb0000, 0x0000fb23,
> + 0x29fb0000, 0x0000fb21, 0x2afc0000, 0x0000fb1f,
> + 0x2cfc0000, 0x0000fb1d, 0x2efc0000, 0x0000fb1b,
> + 0x2ffd0000, 0x0000fb19, 0x2ffe0000, 0x0000fc17,
> + 0x31fe0000, 0x0000fc15, 0x32ff0000, 0x0000fc13,
> + 0x3400ff00, 0x0000fc11, 0x3301ff00, 0x0000fd10,
> + 0x3402ff00, 0x0000fd0e, 0x3503ff00, 0x0000fd0c,
> + 0x3505ff00, 0x0000fd0a, 0x3506fe00, 0x0000fe09,
> + 0x3607fe00, 0x0000fe07, 0x3509fe00, 0x0000fe06,
> + 0x350afd00, 0x0000ff05, 0x350cfd00, 0x0000ff03,
> + 0x340efd00, 0x0000ff02, 0x3310fd00, 0x0000ff01,
> + 0x3411fc00, 0x0000ff00, 0x3213fc00, 0x000000ff,
> + 0x3115fc00, 0x000000fe, 0x2f17fc00, 0x000000fe,
> + 0x2f19fb00, 0x000000fd, 0x2e1bfb00, 0x000000fc,
> + 0x2c1dfb00, 0x000000fc, 0x2a1ffb00, 0x000000fc,
> + 0x2921fb00, 0x000000fb, 0x2723fb00, 0x000000fb,
> + },
> + { /* Bank 2 */
> + 0x1f010000, 0x0000011f, 0x21010000, 0x0000001e,
> + 0x21020000, 0x0000001d, 0x22020000, 0x0000001c,
> + 0x23030000, 0x0000ff1b, 0x2404ff00, 0x0000ff1a,
> + 0x2504ff00, 0x0000ff19, 0x2505ff00, 0x0000ff18,
> + 0x2606ff00, 0x0000fe17, 0x2607ff00, 0x0000fe16,
> + 0x2708ff00, 0x0000fe14, 0x2709ff00, 0x0000fe13,
> + 0x270aff00, 0x0000fe12, 0x280bfe00, 0x0000fe11,
> + 0x280cfe00, 0x0000fe10, 0x280dfe00, 0x0000fe0f,
> + 0x280efe00, 0x0000fe0e, 0x280ffe00, 0x0000fe0d,
> + 0x2810fe00, 0x0000fe0c, 0x2811fe00, 0x0000fe0b,
> + 0x2712fe00, 0x0000ff0a, 0x2713fe00, 0x0000ff09,
> + 0x2714fe00, 0x0000ff08, 0x2616fe00, 0x0000ff07,
> + 0x2617fe00, 0x0000ff06, 0x2518ff00, 0x0000ff05,
> + 0x2519ff00, 0x0000ff04, 0x241aff00, 0x0000ff04,
> + 0x231bff00, 0x00000003, 0x221c0000, 0x00000002,
> + 0x211d0000, 0x00000002, 0x211e0000, 0x00000001,
> + },
> + { /* Bank 3 */
> + 0x1b06ff00, 0x00ff061b, 0x1b07ff00, 0x00ff061a,
> + 0x1c07ff00, 0x00ff051a, 0x1c08ff00, 0x00ff0519,
> + 0x1c09ff00, 0x00ff0419, 0x1d09ff00, 0x00ff0418,
> + 0x1e0aff00, 0x00ff0317, 0x1e0aff00, 0x00ff0317,
> + 0x1e0bff00, 0x00ff0316, 0x1f0cff00, 0x00ff0215,
> + 0x1e0cff00, 0x00000215, 0x1e0dff00, 0x00000214,
> + 0x1e0e0000, 0x00000113, 0x1e0e0000, 0x00000113,
> + 0x1e0f0000, 0x00000112, 0x1f100000, 0x00000011,
> + 0x20100000, 0x00000010, 0x1f110000, 0x00000010,
> + 0x1e120100, 0x0000000f, 0x1e130100, 0x0000000e,
> + 0x1e130100, 0x0000000e, 0x1e140200, 0x0000ff0d,
> + 0x1e150200, 0x0000ff0c, 0x1f1502ff, 0x0000ff0c,
> + 0x1e1603ff, 0x0000ff0b, 0x1e1703ff, 0x0000ff0a,
> + 0x1e1703ff, 0x0000ff0a, 0x1d1804ff, 0x0000ff09,
> + 0x1c1904ff, 0x0000ff09, 0x1c1905ff, 0x0000ff08,
> + 0x1c1a05ff, 0x0000ff07, 0x1b1a06ff, 0x0000ff07,
> + },
> + { /* Bank 4 */
> + 0x17090000, 0x00000917, 0x18090000, 0x00000916,
> + 0x170a0100, 0x00000816, 0x170a0100, 0x00000816,
> + 0x180b0100, 0x00000715, 0x180b0100, 0x00000715,
> + 0x170c0100, 0x00000715, 0x190c0100, 0x00000614,
> + 0x180d0100, 0x00000614, 0x190d0200, 0x00000513,
> + 0x180e0200, 0x00000513, 0x180e0200, 0x00000513,
> + 0x1a0e0200, 0x00000412, 0x190f0200, 0x00000412,
> + 0x190f0300, 0x00000411, 0x18100300, 0x00000411,
> + 0x1a100300, 0x00000310, 0x18110400, 0x00000310,
> + 0x19110400, 0x0000030f, 0x19120400, 0x0000020f,
> + 0x1a120400, 0x0000020e, 0x18130500, 0x0000020e,
> + 0x18130500, 0x0000020e, 0x19130500, 0x0000020d,
> + 0x18140600, 0x0000010d, 0x19140600, 0x0000010c,
> + 0x17150700, 0x0000010c, 0x18150700, 0x0000010b,
> + 0x18150700, 0x0000010b, 0x17160800, 0x0000010a,
> + 0x17160800, 0x0000010a, 0x18160900, 0x00000009,
> + },
> + { /* Bank 5 */
> + 0x120b0300, 0x00030b12, 0x120c0300, 0x00030b11,
> + 0x110c0400, 0x00030b11, 0x110c0400, 0x00030b11,
> + 0x130c0400, 0x00020a11, 0x120d0400, 0x00020a11,
> + 0x110d0500, 0x00020a11, 0x110d0500, 0x00020a11,
> + 0x130d0500, 0x00010911, 0x130e0500, 0x00010910,
> + 0x120e0600, 0x00010910, 0x120e0600, 0x00010910,
> + 0x130e0600, 0x00010810, 0x120f0600, 0x00010810,
> + 0x120f0700, 0x00000810, 0x130f0700, 0x0000080f,
> + 0x140f0700, 0x0000070f, 0x130f0800, 0x0000070f,
> + 0x12100800, 0x0000070f, 0x12100801, 0x0000060f,
> + 0x13100801, 0x0000060e, 0x12100901, 0x0000060e,
> + 0x12100901, 0x0000060e, 0x13100901, 0x0000050e,
> + 0x13110901, 0x0000050d, 0x11110a02, 0x0000050d,
> + 0x11110a02, 0x0000050d, 0x12110a02, 0x0000040d,
> + 0x13110a02, 0x0000040c, 0x11110b03, 0x0000040c,
> + 0x11110b03, 0x0000040c, 0x12110b03, 0x0000030c,
> + },
> + { /* Bank 6 */
> + 0x0b0a0805, 0x00080a0c, 0x0b0a0805, 0x00080a0c,
> + 0x0c0a0805, 0x00080a0b, 0x0c0a0805, 0x00080a0b,
> + 0x0d0a0805, 0x00070a0b, 0x0d0a0805, 0x00070a0b,
> + 0x0d0a0805, 0x00070a0b, 0x0c0a0806, 0x00070a0b,
> + 0x0b0b0806, 0x00070a0b, 0x0c0b0806, 0x0007090b,
> + 0x0b0b0906, 0x0007090b, 0x0b0b0906, 0x0007090b,
> + 0x0b0b0906, 0x0007090b, 0x0b0b0906, 0x0007090b,
> + 0x0b0b0906, 0x0007090b, 0x0c0b0906, 0x0006090b,
> + 0x0c0b0906, 0x0006090b, 0x0c0b0906, 0x0006090b,
> + 0x0b0b0907, 0x0006090b, 0x0b0b0907, 0x0006090b,
> + 0x0b0b0907, 0x0006090b, 0x0b0b0907, 0x0006090b,
> + 0x0b0b0907, 0x0006090b, 0x0c0b0907, 0x0006080b,
> + 0x0b0b0a07, 0x0006080b, 0x0c0b0a07, 0x0006080a,
> + 0x0d0b0a07, 0x0005080a, 0x0d0b0a07, 0x0005080a,
> + 0x0d0b0a07, 0x0005080a, 0x0c0b0a08, 0x0005080a,
> + 0x0c0b0a08, 0x0005080a, 0x0c0b0a08, 0x0005080a,
> + },
> + { /* Bank 7 */
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
> + }
> +};
> +
> +static const unsigned int
> +mali_c55_rsz_filter_coeffs_v[MALI_C55_RSZ_COEFS_BANKS]
> + [MALI_C55_RSZ_COEFS_ENTRIES] = {
> + { /* Bank 0 */
> + 0x2424fc00, 0x000000fc, 0x2721fc00, 0x000000fc,
> + 0x281ffd00, 0x000000fc, 0x2c1cfd00, 0x000000fb,
> + 0x2e1afd00, 0x000000fb, 0x3017fe00, 0x000000fb,
> + 0x3215fe00, 0x000000fb, 0x3512fe00, 0x000000fb,
> + 0x3510ff00, 0x000000fc, 0x370eff00, 0x000000fc,
> + 0x390cff00, 0x000000fc, 0x3a0aff00, 0x000000fd,
> + 0x3a080000, 0x000000fe, 0x3c060000, 0x000000fe,
> + 0x3d040000, 0x000000ff, 0x3d030000, 0x00000000,
> + 0x3c020000, 0x00000002, 0x3d000000, 0x00000003,
> + 0x3dff0000, 0x00000004, 0x3cfe0000, 0x00000006,
> + 0x3afe0000, 0x00000008, 0x3afd0000, 0x0000ff0a,
> + 0x39fc0000, 0x0000ff0c, 0x37fc0000, 0x0000ff0e,
> + 0x35fc0000, 0x0000ff10, 0x35fb0000, 0x0000fe12,
> + 0x32fb0000, 0x0000fe15, 0x30fb0000, 0x0000fe17,
> + 0x2efb0000, 0x0000fd1a, 0x2cfb0000, 0x0000fd1c,
> + 0x28fc0000, 0x0000fd1f, 0x27fc0000, 0x0000fc21,
> + },
> + { /* Bank 1 */
> + 0x2525fb00, 0x000000fb, 0x2723fb00, 0x000000fb,
> + 0x2921fb00, 0x000000fb, 0x2a1ffb00, 0x000000fc,
> + 0x2c1dfb00, 0x000000fc, 0x2e1bfb00, 0x000000fc,
> + 0x2f19fb00, 0x000000fd, 0x2f17fc00, 0x000000fe,
> + 0x3115fc00, 0x000000fe, 0x3213fc00, 0x000000ff,
> + 0x3411fc00, 0x0000ff00, 0x3310fd00, 0x0000ff01,
> + 0x340efd00, 0x0000ff02, 0x350cfd00, 0x0000ff03,
> + 0x350afd00, 0x0000ff05, 0x3509fe00, 0x0000fe06,
> + 0x3607fe00, 0x0000fe07, 0x3506fe00, 0x0000fe09,
> + 0x3505ff00, 0x0000fd0a, 0x3503ff00, 0x0000fd0c,
> + 0x3402ff00, 0x0000fd0e, 0x3301ff00, 0x0000fd10,
> + 0x3400ff00, 0x0000fc11, 0x32ff0000, 0x0000fc13,
> + 0x31fe0000, 0x0000fc15, 0x2ffe0000, 0x0000fc17,
> + 0x2ffd0000, 0x0000fb19, 0x2efc0000, 0x0000fb1b,
> + 0x2cfc0000, 0x0000fb1d, 0x2afc0000, 0x0000fb1f,
> + 0x29fb0000, 0x0000fb21, 0x27fb0000, 0x0000fb23,
> + },
> + { /* Bank 2 */
> + 0x1f1f0100, 0x00000001, 0x211e0000, 0x00000001,
> + 0x211d0000, 0x00000002, 0x221c0000, 0x00000002,
> + 0x231bff00, 0x00000003, 0x241aff00, 0x0000ff04,
> + 0x2519ff00, 0x0000ff04, 0x2518ff00, 0x0000ff05,
> + 0x2617fe00, 0x0000ff06, 0x2616fe00, 0x0000ff07,
> + 0x2714fe00, 0x0000ff08, 0x2713fe00, 0x0000ff09,
> + 0x2712fe00, 0x0000ff0a, 0x2811fe00, 0x0000fe0b,
> + 0x2810fe00, 0x0000fe0c, 0x280ffe00, 0x0000fe0d,
> + 0x280efe00, 0x0000fe0e, 0x280dfe00, 0x0000fe0f,
> + 0x280cfe00, 0x0000fe10, 0x280bfe00, 0x0000fe11,
> + 0x270aff00, 0x0000fe12, 0x2709ff00, 0x0000fe13,
> + 0x2708ff00, 0x0000fe14, 0x2607ff00, 0x0000fe16,
> + 0x2606ff00, 0x0000fe17, 0x2505ff00, 0x0000ff18,
> + 0x2504ff00, 0x0000ff19, 0x2404ff00, 0x0000ff1a,
> + 0x23030000, 0x0000ff1b, 0x22020000, 0x0000001c,
> + 0x21020000, 0x0000001d, 0x21010000, 0x0000001e,
> + },
> + { /* Bank 3 */
> + 0x1b1b06ff, 0x0000ff06, 0x1b1a06ff, 0x0000ff07,
> + 0x1c1a05ff, 0x0000ff07, 0x1c1905ff, 0x0000ff08,
> + 0x1c1904ff, 0x0000ff09, 0x1d1804ff, 0x0000ff09,
> + 0x1e1703ff, 0x0000ff0a, 0x1e1703ff, 0x0000ff0a,
> + 0x1e1603ff, 0x0000ff0b, 0x1f1502ff, 0x0000ff0c,
> + 0x1e150200, 0x0000ff0c, 0x1e140200, 0x0000ff0d,
> + 0x1e130100, 0x0000000e, 0x1e130100, 0x0000000e,
> + 0x1e120100, 0x0000000f, 0x1f110000, 0x00000010,
> + 0x20100000, 0x00000010, 0x1f100000, 0x00000011,
> + 0x1e0f0000, 0x00000112, 0x1e0e0000, 0x00000113,
> + 0x1e0e0000, 0x00000113, 0x1e0dff00, 0x00000214,
> + 0x1e0cff00, 0x00000215, 0x1f0cff00, 0x00ff0215,
> + 0x1e0bff00, 0x00ff0316, 0x1e0aff00, 0x00ff0317,
> + 0x1e0aff00, 0x00ff0317, 0x1d09ff00, 0x00ff0418,
> + 0x1c09ff00, 0x00ff0419, 0x1c08ff00, 0x00ff0519,
> + 0x1c07ff00, 0x00ff051a, 0x1b07ff00, 0x00ff061a,
> + },
> + { /* Bank 4 */
> + 0x17170900, 0x00000009, 0x18160900, 0x00000009,
> + 0x17160800, 0x0000010a, 0x17160800, 0x0000010a,
> + 0x18150700, 0x0000010b, 0x18150700, 0x0000010b,
> + 0x17150700, 0x0000010c, 0x19140600, 0x0000010c,
> + 0x18140600, 0x0000010d, 0x19130500, 0x0000020d,
> + 0x18130500, 0x0000020e, 0x18130500, 0x0000020e,
> + 0x1a120400, 0x0000020e, 0x19120400, 0x0000020f,
> + 0x19110400, 0x0000030f, 0x18110400, 0x00000310,
> + 0x1a100300, 0x00000310, 0x18100300, 0x00000411,
> + 0x190f0300, 0x00000411, 0x190f0200, 0x00000412,
> + 0x1a0e0200, 0x00000412, 0x180e0200, 0x00000513,
> + 0x180e0200, 0x00000513, 0x190d0200, 0x00000513,
> + 0x180d0100, 0x00000614, 0x190c0100, 0x00000614,
> + 0x170c0100, 0x00000715, 0x180b0100, 0x00000715,
> + 0x180b0100, 0x00000715, 0x170a0100, 0x00000816,
> + 0x170a0100, 0x00000816, 0x18090000, 0x00000916,
> + },
> + { /* Bank 5 */
> + 0x12120b03, 0x0000030b, 0x12110b03, 0x0000030c,
> + 0x11110b03, 0x0000040c, 0x11110b03, 0x0000040c,
> + 0x13110a02, 0x0000040c, 0x12110a02, 0x0000040d,
> + 0x11110a02, 0x0000050d, 0x11110a02, 0x0000050d,
> + 0x13110901, 0x0000050d, 0x13100901, 0x0000050e,
> + 0x12100901, 0x0000060e, 0x12100901, 0x0000060e,
> + 0x13100801, 0x0000060e, 0x12100801, 0x0000060f,
> + 0x12100800, 0x0000070f, 0x130f0800, 0x0000070f,
> + 0x140f0700, 0x0000070f, 0x130f0700, 0x0000080f,
> + 0x120f0700, 0x00000810, 0x120f0600, 0x00010810,
> + 0x130e0600, 0x00010810, 0x120e0600, 0x00010910,
> + 0x120e0600, 0x00010910, 0x130e0500, 0x00010910,
> + 0x130d0500, 0x00010911, 0x110d0500, 0x00020a11,
> + 0x110d0500, 0x00020a11, 0x120d0400, 0x00020a11,
> + 0x130c0400, 0x00020a11, 0x110c0400, 0x00030b11,
> + 0x110c0400, 0x00030b11, 0x120c0300, 0x00030b11,
> + },
> + { /* Bank 6 */
> + 0x0b0c0a08, 0x0005080a, 0x0b0c0a08, 0x0005080a,
> + 0x0c0b0a08, 0x0005080a, 0x0c0b0a08, 0x0005080a,
> + 0x0d0b0a07, 0x0005080a, 0x0d0b0a07, 0x0005080a,
> + 0x0d0b0a07, 0x0005080a, 0x0c0b0a07, 0x0006080a,
> + 0x0b0b0a07, 0x0006080b, 0x0c0b0907, 0x0006080b,
> + 0x0b0b0907, 0x0006090b, 0x0b0b0907, 0x0006090b,
> + 0x0b0b0907, 0x0006090b, 0x0b0b0907, 0x0006090b,
> + 0x0b0b0907, 0x0006090b, 0x0c0b0906, 0x0006090b,
> + 0x0c0b0906, 0x0006090b, 0x0c0b0906, 0x0006090b,
> + 0x0b0b0906, 0x0007090b, 0x0b0b0906, 0x0007090b,
> + 0x0b0b0906, 0x0007090b, 0x0b0b0906, 0x0007090b,
> + 0x0b0b0906, 0x0007090b, 0x0c0b0806, 0x0007090b,
> + 0x0b0b0806, 0x00070a0b, 0x0c0a0806, 0x00070a0b,
> + 0x0d0a0805, 0x00070a0b, 0x0d0a0805, 0x00070a0b,
> + 0x0d0a0805, 0x00070a0b, 0x0c0a0805, 0x00080a0b,
> + 0x0c0a0805, 0x00080a0b, 0x0c0a0805, 0x00080a0b,
> + },
> + { /* Bank 7 */
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
> + }
> +};
> +
> +static const struct mali_c55_rsz_coef_bank {
> + unsigned int top;
> + unsigned int bottom;
The bottom value of bank N is always equal to the top value of bank N+1.
You can simplify this by storing a single value.
> +} mali_c55_rsz_coef_banks[] = {
> + {
> + .top = 1000,
> + .bottom = 770,
> + }, {
> + .top = 769,
> + .bottom = 600,
> + }, {
> + .top = 599,
> + .bottom = 460,
> + }, {
> + .top = 459,
> + .bottom = 354,
> + }, {
> + .top = 353,
> + .bottom = 273,
> + }, {
> + .top = 272,
> + .bottom = 210,
> + }, {
> + .top = 209,
> + .bottom = 162,
> + }, {
> + .top = 161,
> + .bottom = 125,
> + },
> +};
> +
> +/*
> + * Select the right filter coefficients bank based on the scaler input and the
> + * scaler output sizes ratio, set by the v4l2 crop and scale selection
> + * rectangles respectively.
> + */
> +static unsigned int mali_c55_rsz_calculate_bank(struct mali_c55 *mali_c55,
> + unsigned int rsz_in,
> + unsigned int rsz_out)
> +{
> + unsigned int rsz_ratio = (rsz_out * 1000U) / rsz_in;
> +
> + for (unsigned int i = 0; i < ARRAY_SIZE(mali_c55_rsz_coef_banks); i++) {
> + if (rsz_ratio >= mali_c55_rsz_coef_banks[i].bottom &&
> + rsz_ratio <= mali_c55_rsz_coef_banks[i].top)
> + return i;
> + }
> +
> + /*
> + * We shouldn't ever get here, in theory. As we have no good choices
> + * simply warn the user and use the first bank of coefficients.
> + */
> + dev_warn(mali_c55->dev, "scaling factor outside defined bounds\n");
> + return 0;
> +}
> +
> +static const u32 rsz_non_bypass_src_fmts[] = {
> + MEDIA_BUS_FMT_RGB121212_1X36,
> + MEDIA_BUS_FMT_YUV10_1X30
> +};
> +
> +static void mali_c55_resizer_program_coefficients(struct mali_c55_resizer *rsz)
> +{
> + struct mali_c55 *mali_c55 = rsz->mali_c55;
> + unsigned int haddr = rsz->id == MALI_C55_RSZ_FR ?
> + MALI_C55_REG_FR_SCALER_HFILT :
> + MALI_C55_REG_DS_SCALER_HFILT;
> + unsigned int vaddr = rsz->id == MALI_C55_RSZ_FR ?
> + MALI_C55_REG_FR_SCALER_VFILT :
> + MALI_C55_REG_DS_SCALER_VFILT;
> +
> + for (unsigned int i = 0; i < MALI_C55_RSZ_COEFS_BANKS; i++) {
> + for (unsigned int j = 0; j < MALI_C55_RSZ_COEFS_ENTRIES; j++) {
> + mali_c55_write(mali_c55, haddr,
> + mali_c55_rsz_filter_coeffs_h[i][j]);
> + mali_c55_write(mali_c55, vaddr,
> + mali_c55_rsz_filter_coeffs_v[i][j]);
> +
> + haddr += sizeof(u32);
> + vaddr += sizeof(u32);
> + }
> + }
> +}
> +
> +static int mali_c55_rsz_program_crop(struct mali_c55_resizer *rsz,
> + struct v4l2_subdev_state *state)
const
> +{
> + const struct v4l2_mbus_framefmt *fmt;
> + const struct v4l2_rect *crop;
> +
> + /* Verify if crop should be enabled. */
> + fmt = v4l2_subdev_state_get_format(state, MALI_C55_RSZ_SINK_PAD, 0);
> + crop = v4l2_subdev_state_get_crop(state, MALI_C55_RSZ_SINK_PAD, 0);
> +
> + if (fmt->width == crop->width && fmt->height == crop->height)
> + return MALI_C55_BYPASS_CROP;
> +
> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_CROP_X_START,
> + crop->left);
> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_CROP_Y_START,
> + crop->top);
> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_CROP_X_SIZE,
> + crop->width);
> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_CROP_Y_SIZE,
> + crop->height);
> +
> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_CROP_EN,
> + MALI_C55_CROP_ENABLE);
> +
> + return 0;
> +}
> +
> +static int mali_c55_rsz_program_resizer(struct mali_c55_resizer *rsz,
> + struct v4l2_subdev_state *state)
const
> +{
> + struct mali_c55 *mali_c55 = rsz->mali_c55;
> + const struct v4l2_rect *crop, *scale;
> + unsigned int h_bank, v_bank;
> + u64 h_scale, v_scale;
> +
> + /* Verify if scaling should be enabled. */
> + crop = v4l2_subdev_state_get_crop(state, MALI_C55_RSZ_SINK_PAD, 0);
> + scale = v4l2_subdev_state_get_compose(state, MALI_C55_RSZ_SINK_PAD, 0);
> +
> + if (crop->width == scale->width && crop->height == scale->height)
> + return MALI_C55_BYPASS_SCALER;
> +
> + /* Program the scaler coefficients if the scaler is in use. */
> + mali_c55_resizer_program_coefficients(rsz);
> +
> + /* Program the V/H scaling factor in Q4.20 format. */
> + h_scale = crop->width * MALI_C55_RSZ_SCALER_FACTOR;
> + v_scale = crop->height * MALI_C55_RSZ_SCALER_FACTOR;
> +
> + do_div(h_scale, scale->width);
> + do_div(v_scale, scale->height);
> +
> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_IN_WIDTH,
> + crop->width);
> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_IN_HEIGHT,
> + crop->height);
> +
> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_OUT_WIDTH,
> + scale->width);
> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_OUT_HEIGHT,
> + scale->height);
> +
> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_HFILT_TINC,
> + h_scale);
> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_VFILT_TINC,
> + v_scale);
> +
> + /* Select the scaler coefficients bank to use. */
> + h_bank = mali_c55_rsz_calculate_bank(mali_c55, crop->width,
> + scale->width);
> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_HFILT_COEF,
> + h_bank);
> +
> + v_bank = mali_c55_rsz_calculate_bank(mali_c55, crop->height,
> + scale->height);
> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_VFILT_COEF,
> + v_bank);
> +
> + return 0;
> +}
> +
> +static void mali_c55_rsz_program(struct mali_c55_resizer *rsz,
> + struct v4l2_subdev_state *state)
> +{
> + struct mali_c55 *mali_c55 = rsz->mali_c55;
> + u32 bypass = 0;
> +
> + /* Verify if cropping and scaling should be enabled. */
> + bypass |= mali_c55_rsz_program_crop(rsz, state);
> + bypass |= mali_c55_rsz_program_resizer(rsz, state);
> +
> + mali_c55_ctx_update_bits(mali_c55, rsz->id == MALI_C55_RSZ_FR ?
> + MALI_C55_REG_FR_BYPASS : MALI_C55_REG_DS_BYPASS,
> + MALI_C55_BYPASS_CROP | MALI_C55_BYPASS_SCALER,
> + bypass);
> +}
> +
> +/*
> + * Inspect the routing table to know which of the two (mutually exclusive)
> + * routes is enabled and return the sink pad id of the active route.
> + */
> +static unsigned int mali_c55_rsz_get_active_sink(struct v4l2_subdev_state *state)
> +{
> + struct v4l2_subdev_krouting *routing = &state->routing;
> + struct v4l2_subdev_route *route;
> +
> + /* A single route is enabled at a time. */
> + for_each_active_route(routing, route)
> + return route->sink_pad;
> +
> + return MALI_C55_RSZ_SINK_PAD;
> +}
> +
> +/*
> + * When operating in bypass mode, the ISP takes input in a 20-bit format, but
> + * can only output 16-bit RAW bayer data (with the 4 least significant bits from
> + * the input being lost). Return the 16-bit version of the 20-bit input formats.
> + */
> +static u32 mali_c55_rsz_shift_mbus_code(u32 mbus_code)
> +{
> + switch (mbus_code) {
> + case MEDIA_BUS_FMT_SBGGR20_1X20:
> + return MEDIA_BUS_FMT_SBGGR16_1X16;
> + case MEDIA_BUS_FMT_SGBRG20_1X20:
> + return MEDIA_BUS_FMT_SGBRG16_1X16;
> + case MEDIA_BUS_FMT_SGRBG20_1X20:
> + return MEDIA_BUS_FMT_SGRBG16_1X16;
> + case MEDIA_BUS_FMT_SRGGB20_1X20:
> + return MEDIA_BUS_FMT_SRGGB16_1X16;
> + }
Would it make sense to add the shifted code to mali_c55_isp_fmt ?
> +
> + return 0;
> +}
> +
> +static int __mali_c55_rsz_set_routing(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + const struct v4l2_subdev_krouting *routing)
> +{
> + struct mali_c55_resizer *rsz = sd_to_mali_c55_rsz(sd);
> + unsigned int active_sink = UINT_MAX;
> + struct v4l2_mbus_framefmt *src_fmt;
> + struct v4l2_rect *crop, *compose;
> + struct v4l2_subdev_route *route;
> + unsigned int active_routes = 0;
> + struct v4l2_mbus_framefmt *fmt;
> + int ret;
> +
> + ret = v4l2_subdev_routing_validate(sd, routing, 0);
> + if (ret)
> + return ret;
> +
> + /* Only a single route can be enabled at a time. */
> + for_each_active_route(routing, route) {
> + if (++active_routes > 1) {
> + dev_dbg(rsz->mali_c55->dev,
> + "Only one route can be active");
> + return -EINVAL;
> + }
> +
> + active_sink = route->sink_pad;
> + }
> + if (active_sink == UINT_MAX) {
> + dev_dbg(rsz->mali_c55->dev, "One route has to be active");
> + return -EINVAL;
> + }
> +
> + ret = v4l2_subdev_set_routing(sd, state, routing);
> + if (ret) {
> + dev_dbg(rsz->mali_c55->dev, "Failed to set routing\n");
> + return ret;
> + }
> +
> + fmt = v4l2_subdev_state_get_format(state, active_sink, 0);
> + crop = v4l2_subdev_state_get_crop(state, active_sink, 0);
> + compose = v4l2_subdev_state_get_compose(state, active_sink, 0);
These two lines, as well as the crop and compose variables, can move to
the first branch of the if() below.
> +
> + fmt->width = MALI_C55_DEFAULT_WIDTH;
> + fmt->height = MALI_C55_DEFAULT_HEIGHT;
> + fmt->colorspace = V4L2_COLORSPACE_SRGB;
There are tree other colorspace-related fields.
> + fmt->field = V4L2_FIELD_NONE;
> +
> + if (active_sink == MALI_C55_RSZ_SINK_PAD) {
> + fmt->code = MEDIA_BUS_FMT_RGB121212_1X36;
> +
> + crop->left = 0;
> + crop->top = 0;
> + crop->width = MALI_C55_DEFAULT_WIDTH;
> + crop->height = MALI_C55_DEFAULT_HEIGHT;
> +
> + *compose = *crop;
> + } else {
> + fmt->code = MEDIA_BUS_FMT_SRGGB20_1X20;
> + }
> +
> + /* Propagate the format to the source pad */
> + src_fmt = v4l2_subdev_state_get_format(state, MALI_C55_RSZ_SOURCE_PAD,
> + 0);
> + *src_fmt = *fmt;
> +
> + /* In the event this is the bypass pad the mbus code needs correcting */
> + if (active_sink == MALI_C55_RSZ_SINK_BYPASS_PAD)
> + src_fmt->code = mali_c55_rsz_shift_mbus_code(src_fmt->code);
> +
> + return 0;
> +}
> +
> +static int mali_c55_rsz_enum_mbus_code(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + struct v4l2_mbus_framefmt *sink_fmt;
> + const struct mali_c55_isp_fmt *fmt;
> + u32 sink_pad;
> +
> + switch (code->pad) {
> + case MALI_C55_RSZ_SINK_PAD:
> + if (code->index)
> + return -EINVAL;
> +
> + code->code = MEDIA_BUS_FMT_RGB121212_1X36;
> +
> + return 0;
> + case MALI_C55_RSZ_SOURCE_PAD:
> + sink_pad = mali_c55_rsz_get_active_sink(state);
> + sink_fmt = v4l2_subdev_state_get_format(state, sink_pad, 0);
> +
> + /*
> + * If the active route is from the Bypass sink pad, then the
> + * source pad is a simple passthrough of the sink format,
> + * downshifted to 16-bits.
> + */
> +
> + if (sink_pad == MALI_C55_RSZ_SINK_BYPASS_PAD) {
> + if (code->index)
> + return -EINVAL;
> +
> + code->code = mali_c55_rsz_shift_mbus_code(sink_fmt->code);
> + if (!code->code)
> + return -EINVAL;
> +
> + return 0;
> + }
> +
> + /*
> + * If the active route is from the non-bypass sink then we can
> + * select either RGB or conversion to YUV.
> + */
> +
> + if (code->index >= ARRAY_SIZE(rsz_non_bypass_src_fmts))
> + return -EINVAL;
> +
> + code->code = rsz_non_bypass_src_fmts[code->index];
> +
> + return 0;
> + case MALI_C55_RSZ_SINK_BYPASS_PAD:
> + fmt = mali_c55_isp_get_mbus_config_by_index(code->index);
> + if (fmt) {
> + code->code = fmt->code;
> + return 0;
> + }
> +
> + break;
> + }
> +
> + return -EINVAL;
> +}
> +
> +static int mali_c55_rsz_enum_frame_size(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_frame_size_enum *fse)
> +{
> + if (fse->index)
> + return -EINVAL;
You need to verify that fse->code is supported.
> +
> + fse->max_width = MALI_C55_MAX_WIDTH;
> + fse->max_height = MALI_C55_MAX_HEIGHT;
> + fse->min_width = MALI_C55_MIN_WIDTH;
> + fse->min_height = MALI_C55_MIN_HEIGHT;
> +
> + return 0;
> +}
> +
> +static int mali_c55_rsz_set_sink_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *format)
> +{
> + struct v4l2_mbus_framefmt *fmt = &format->format;
> + unsigned int active_sink;
> + struct v4l2_rect *rect;
> +
> + /*
> + * Clamp to min/max and then reset crop and compose rectangles to the
> + * newly applied size.
> + */
> + fmt->width = clamp_t(unsigned int, fmt->width, MALI_C55_MIN_WIDTH,
> + MALI_C55_MAX_WIDTH);
> + fmt->height = clamp_t(unsigned int, fmt->height, MALI_C55_MIN_HEIGHT,
> + MALI_C55_MAX_HEIGHT);
You're accepting all values for fmt->field and all the fmt
colorspace-related fields. Is that on purpose ? You've updated some of
the set format handlers in this version to change this, by adjusting and
copying selected fields of fmt into the format stored in the state, and
then copying the whole state format to fmt, instead of doing it the
other way around. That could be a better option here too.
> +
> + rect = v4l2_subdev_state_get_crop(state, format->pad);
> + rect->left = 0;
> + rect->top = 0;
> + rect->width = fmt->width;
> + rect->height = fmt->height;
> +
> + rect = v4l2_subdev_state_get_compose(state, format->pad);
> + rect->left = 0;
> + rect->top = 0;
> + rect->width = fmt->width;
> + rect->height = fmt->height;
The crop and compose rectangles don't exist on the sink bypass pad.
> +
> + if (format->pad == MALI_C55_RSZ_SINK_BYPASS_PAD) {
> + /*
> + * Make sure the media bus code is one of the supported
> + * ISP input media bus codes. Default it to SRGGB otherwise.
> + */
> + if (!mali_c55_isp_get_mbus_config_by_code(fmt->code))
> + fmt->code = MEDIA_BUS_FMT_SRGGB20_1X20;
> + } else {
> + fmt->code = MEDIA_BUS_FMT_RGB121212_1X36;
> + }
> +
> + *v4l2_subdev_state_get_format(state, format->pad, 0) = *fmt;
> +
> + /* If format->pad is routed to the source pad, propagate the format. */
> + active_sink = mali_c55_rsz_get_active_sink(state);
> + if (active_sink == format->pad) {
> + /* If the bypass route is used, downshift the code to 16bpp. */
> + if (active_sink == MALI_C55_RSZ_SINK_BYPASS_PAD)
> + fmt->code = mali_c55_rsz_shift_mbus_code(fmt->code);
> +
> + *v4l2_subdev_state_get_format(state,
> + MALI_C55_RSZ_SOURCE_PAD, 0) = *fmt;
> + }
> +
> + return 0;
> +}
> +
> +static int mali_c55_rsz_set_source_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *format)
> +{
> + struct v4l2_mbus_framefmt *fmt = &format->format;
> + struct v4l2_mbus_framefmt *sink_fmt;
> + struct v4l2_rect *sink_compose;
> + unsigned int active_sink;
> +
> + active_sink = mali_c55_rsz_get_active_sink(state);
> + sink_fmt = v4l2_subdev_state_get_format(state, active_sink, 0);
> + sink_compose = v4l2_subdev_state_get_compose(state, active_sink, 0);
> +
> + /*
> + * The source pad format sizes come directly from the active sink pad
> + * compose rectangle.
> + */
> + fmt->width = sink_compose->width;
> + fmt->height = sink_compose->height;
> +
> + if (active_sink == MALI_C55_RSZ_SINK_PAD) {
> + /*
> + * Regular processing pipe: RGB121212 can be color-space
> + * converted to YUV101010.
> + */
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(rsz_non_bypass_src_fmts); i++) {
> + if (fmt->code == rsz_non_bypass_src_fmts[i])
> + break;
> + }
> +
> + if (i == ARRAY_SIZE(rsz_non_bypass_src_fmts))
> + fmt->code = MEDIA_BUS_FMT_RGB121212_1X36;
> + } else {
> + /*
> + * Bypass pipe: the source format is the same as the bypass
> + * sink pad downshifted to 16bpp.
> + */
> + fmt->code = mali_c55_rsz_shift_mbus_code(sink_fmt->code);
> + }
> +
> + *v4l2_subdev_state_get_format(state, MALI_C55_RSZ_SOURCE_PAD) = *fmt;
Do it the other way around here too. Set selected fields in the state,
then copy the whole format to format->format. I proposed a possible
implementation in the review of the previous version.
> +
> + return 0;
> +}
> +
> +static int mali_c55_rsz_set_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *format)
> +{
> + /*
> + * On sink pads fmt is either fixed for the 'regular' processing
> + * pad or a RAW format or 20-bit wide RGB/YUV format for the FR bypass
> + * pad.
> + *
> + * On source pad sizes are the result of crop+compose on the sink
> + * pad sizes, while the format depends on the active route.
> + */
> +
> + if (format->pad == MALI_C55_RSZ_SINK_PAD ||
> + format->pad == MALI_C55_RSZ_SINK_BYPASS_PAD)
> + return mali_c55_rsz_set_sink_fmt(sd, state, format);
> +
> + return mali_c55_rsz_set_source_fmt(sd, state, format);
> +}
> +
> +static int mali_c55_rsz_get_selection(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_selection *sel)
> +{
> + if (sel->pad != MALI_C55_RSZ_SINK_PAD)
> + return -EINVAL;
> +
> + if (sel->target != V4L2_SEL_TGT_CROP &&
> + sel->target != V4L2_SEL_TGT_COMPOSE)
> + return -EINVAL;
> +
> + sel->r = sel->target == V4L2_SEL_TGT_CROP
> + ? *v4l2_subdev_state_get_crop(state, MALI_C55_RSZ_SINK_PAD)
> + : *v4l2_subdev_state_get_compose(state, MALI_C55_RSZ_SINK_PAD);
> +
> + return 0;
> +}
> +
> +static int mali_c55_rsz_set_selection(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_selection *sel)
> +{
> + struct mali_c55_resizer *rsz = sd_to_mali_c55_rsz(sd);
> + struct v4l2_mbus_framefmt *sink_fmt;
> + struct v4l2_rect *crop, *compose;
> +
> + if (sel->pad != MALI_C55_RSZ_SINK_PAD)
> + return -EINVAL;
> +
> + if (sel->target != V4L2_SEL_TGT_CROP &&
> + sel->target != V4L2_SEL_TGT_COMPOSE)
> + return -EINVAL;
> +
> + sink_fmt = v4l2_subdev_state_get_format(state, MALI_C55_RSZ_SINK_PAD);
> + crop = v4l2_subdev_state_get_crop(state, MALI_C55_RSZ_SINK_PAD);
> + compose = v4l2_subdev_state_get_compose(state, MALI_C55_RSZ_SINK_PAD);
> +
> + /* During streaming, it is allowed to only change the crop rectangle. */
> + if (rsz->streaming && sel->target != V4L2_SEL_TGT_CROP)
> + return -EBUSY;
> +
> + /*
> + * Update the desired target and then clamp the crop rectangle to the
> + * sink format sizes and the compose size to the crop rectangle.
> + */
> + if (sel->target == V4L2_SEL_TGT_CROP)
> + *crop = sel->r;
> + else
> + *compose = sel->r;
> +
> + crop->left = clamp_t(unsigned int, crop->left, 0, sink_fmt->width);
Double space.
> + crop->top = clamp_t(unsigned int, crop->top, 0, sink_fmt->height);
> + crop->width = clamp_t(unsigned int, crop->width, MALI_C55_MIN_WIDTH,
> + sink_fmt->width - crop->left);
> + crop->height = clamp_t(unsigned int, crop->height, MALI_C55_MIN_HEIGHT,
> + sink_fmt->height - crop->top);
You don't need to do this when setting the compose rectangle, as crop
and sink_fmt haven't changed.
I think this function could be rewritten to make it easier to read and
understand.
> +
> + if (rsz->streaming) {
> + /*
> + * It is not valid to apply at runtime a crop rectangle smaller
> + * than the compose one, as it requires re-programming the
> + * scaler output sizes and the output buffer sizes.
> + *
> + * Adjust the crop rectangle to be at least as large as the
> + * compose one if we're streaming and apply it immediately.
> + */
> + if (crop->width < compose->width)
> + crop->width = compose->width;
> + if (crop->height < compose->height)
> + crop->height = compose->height;
> +
> + sel->r = *crop;
> +
> + mali_c55_rsz_program(rsz, state);
> +
> + return 0;
> + }
> +
> + compose->left = 0;
> + compose->top = 0;
> + compose->width = clamp_t(unsigned int, compose->width, crop->width / 8,
> + crop->width);
> + compose->height = clamp_t(unsigned int, compose->height,
> + crop->height / 8, crop->height);
> +
> + sel->r = sel->target == V4L2_SEL_TGT_CROP ? *crop : *compose;
> +
> + return 0;
> +}
> +
> +static int mali_c55_rsz_set_routing(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + enum v4l2_subdev_format_whence which,
> + struct v4l2_subdev_krouting *routing)
> +{
> + if (which == V4L2_SUBDEV_FORMAT_ACTIVE &&
> + media_entity_is_streaming(&sd->entity))
> + return -EBUSY;
> +
> + return __mali_c55_rsz_set_routing(sd, state, routing);
> +}
> +
> +static int mali_c55_rsz_enable_streams(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state, u32 pad,
> + u64 streams_mask)
> +{
> + struct mali_c55_resizer *rsz = sd_to_mali_c55_rsz(sd);
> + struct mali_c55 *mali_c55 = rsz->mali_c55;
> + unsigned int sink_pad;
> +
> + sink_pad = mali_c55_rsz_get_active_sink(state);
> + if (sink_pad == MALI_C55_RSZ_SINK_BYPASS_PAD) {
> + /* Bypass FR pipe processing if the bypass route is active. */
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_ISP_RAW_BYPASS,
> + MALI_C55_ISP_RAW_BYPASS_FR_BYPASS_MASK,
> + MALI_C55_ISP_RAW_BYPASS_RAW_FR_BYPASS);
> + rsz->streaming = true;
> + return 0;
> + }
> +
> + /* Disable bypass and use regular processing. */
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_ISP_RAW_BYPASS,
> + MALI_C55_ISP_RAW_BYPASS_FR_BYPASS_MASK, 0);
> + mali_c55_rsz_program(rsz, state);
> +
> + rsz->streaming = true;
Can rsz->streaming be replaced with v4l2_subdev_is_streaming() ?
> +
> + return 0;
> +}
> +
> +static int mali_c55_rsz_disable_streams(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state, u32 pad,
> + u64 streams_mask)
> +{
> + struct mali_c55_resizer *rsz = sd_to_mali_c55_rsz(sd);
> +
> + rsz->streaming = false;
> + return 0;
> +}
> +
> +static const struct v4l2_subdev_pad_ops mali_c55_resizer_pad_ops = {
> + .enum_mbus_code = mali_c55_rsz_enum_mbus_code,
> + .enum_frame_size = mali_c55_rsz_enum_frame_size,
> + .get_fmt = v4l2_subdev_get_fmt,
> + .set_fmt = mali_c55_rsz_set_fmt,
> + .get_selection = mali_c55_rsz_get_selection,
> + .set_selection = mali_c55_rsz_set_selection,
> + .set_routing = mali_c55_rsz_set_routing,
> + .enable_streams = mali_c55_rsz_enable_streams,
> + .disable_streams = mali_c55_rsz_disable_streams,
> +};
> +
> +static const struct v4l2_subdev_ops mali_c55_resizer_ops = {
> + .pad = &mali_c55_resizer_pad_ops,
> +};
> +
> +static int mali_c55_rsz_init_state(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state)
> +{
> + struct mali_c55_resizer *rsz = sd_to_mali_c55_rsz(sd);
> + struct v4l2_subdev_route routes[2] = {
> + {
> + .sink_pad = MALI_C55_RSZ_SINK_PAD,
> + .source_pad = MALI_C55_RSZ_SOURCE_PAD,
> + .flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> + }, {
> + .sink_pad = MALI_C55_RSZ_SINK_BYPASS_PAD,
> + .source_pad = MALI_C55_RSZ_SOURCE_PAD,
> + },
> + };
> + struct v4l2_subdev_krouting routing = {
> + .num_routes = rsz->num_routes,
> + .routes = routes,
> + };
> +
> + return __mali_c55_rsz_set_routing(sd, state, &routing);
> +}
> +
> +static const struct v4l2_subdev_internal_ops mali_c55_resizer_internal_ops = {
> + .init_state = mali_c55_rsz_init_state,
> +};
> +
> +static int mali_c55_register_resizer(struct mali_c55 *mali_c55,
> + struct mali_c55_resizer *rsz)
> +{
> + struct v4l2_subdev *sd = &rsz->sd;
> + unsigned int num_pads;
> + int ret;
> +
> + rsz->streaming = false;
> +
> + v4l2_subdev_init(sd, &mali_c55_resizer_ops);
> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_STREAMS;
> + sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
> + sd->internal_ops = &mali_c55_resizer_internal_ops;
> +
> + rsz->pads[MALI_C55_RSZ_SINK_PAD].flags = MEDIA_PAD_FL_SINK;
> + rsz->pads[MALI_C55_RSZ_SOURCE_PAD].flags = MEDIA_PAD_FL_SOURCE;
> +
> + if (rsz->id == MALI_C55_RSZ_FR) {
> + num_pads = MALI_C55_RSZ_NUM_PADS;
> + rsz->num_routes = 2;
> +
> + rsz->pads[MALI_C55_RSZ_SINK_BYPASS_PAD].flags =
> + MEDIA_PAD_FL_SINK;
> +
> + snprintf(sd->name, ARRAY_SIZE(sd->name), "%s resizer fr",
> + MALI_C55_DRIVER_NAME);
snprintf(sd->name, sizeof(sd->name), "%s resizer fr",
MALI_C55_DRIVER_NAME);
> +
> + } else {
> + num_pads = MALI_C55_RSZ_NUM_PADS - 1;
> + rsz->num_routes = 1;
> +
> + snprintf(sd->name, ARRAY_SIZE(sd->name), "%s resizer ds",
> + MALI_C55_DRIVER_NAME);
Same here.
> + }
> +
> + ret = media_entity_pads_init(&sd->entity, num_pads, rsz->pads);
> + if (ret)
> + return ret;
> +
> + ret = v4l2_subdev_init_finalize(sd);
> + if (ret)
> + goto err_media_cleanup;
> +
> + ret = v4l2_device_register_subdev(&mali_c55->v4l2_dev, sd);
> + if (ret)
> + goto err_subdev_cleanup;
> +
> + return 0;
> +
> +err_subdev_cleanup:
> + v4l2_subdev_cleanup(sd);
> +err_media_cleanup:
> + media_entity_cleanup(&sd->entity);
> +
> + return ret;
> +}
> +
> +static void mali_c55_unregister_resizer(struct mali_c55_resizer *rsz)
> +{
> + if (!rsz->mali_c55)
> + return;
> +
> + v4l2_device_unregister_subdev(&rsz->sd);
> + v4l2_subdev_cleanup(&rsz->sd);
> + media_entity_cleanup(&rsz->sd.entity);
> +}
> +
> +int mali_c55_register_resizers(struct mali_c55 *mali_c55)
> +{
> + unsigned int i;
> + int ret;
> +
> + for (i = 0; i < MALI_C55_NUM_RSZS; ++i) {
> + struct mali_c55_resizer *rsz = &mali_c55->resizers[i];
> +
> + rsz->id = i;
> + ret = mali_c55_register_resizer(mali_c55, rsz);
> + if (ret)
> + goto err_cleanup;
> +
> + rsz->cap_dev = &mali_c55->cap_devs[i];
> + rsz->mali_c55 = mali_c55;
You can move the rsz->id assignment and the last two lines to
mali_c55_register_resizer() by passing the index as an argument. This
would allow unrolling the loop. Up to you. Even if you don't want to
unroll the loop, you could still move the assignments to the function.
> + }
> +
> + return 0;
> +
> +err_cleanup:
> + for (; i >= 0; --i)
> + mali_c55_unregister_resizer(&mali_c55->resizers[i]);
Infinite loop and wrong index handling.
> +
> + return ret;
> +}
> +
> +void mali_c55_unregister_resizers(struct mali_c55 *mali_c55)
> +{
> + for (unsigned int i = 0; i < MALI_C55_NUM_RSZS; i++)
> + mali_c55_unregister_resizer(&mali_c55->resizers[i]);
> +}
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-tpg.c b/drivers/media/platform/arm/mali-c55/mali-c55-tpg.c
> new file mode 100644
> index 000000000000..56bb617eb5a4
> --- /dev/null
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-tpg.c
> @@ -0,0 +1,438 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * ARM Mali-C55 ISP Driver - Test pattern generator
> + *
> + * Copyright (C) 2024 Ideas on Board Oy
> + */
> +
> +#include <linux/minmax.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/string.h>
> +
> +#include <media/media-entity.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-subdev.h>
> +
> +#include "mali-c55-common.h"
> +#include "mali-c55-registers.h"
> +
> +#define MALI_C55_TPG_SRC_PAD 0
> +#define MALI_C55_TPG_FIXED_HBLANK 0x20
> +#define MALI_C55_TPG_DEFAULT_MIN_VBLANK 66
> +#define MALI_C55_TPG_DEFAULT_DEF_VBLANK 626
> +#define MALI_C55_TPG_MAX_VBLANK 0xffff
> +#define MALI_C55_TPG_PIXEL_RATE 100000000
> +
> +static const char * const mali_c55_tpg_test_pattern_menu[] = {
> + "Flat field",
> + "Horizontal gradient",
> + "Vertical gradient",
> + "Vertical bars",
> + "Arbitrary rectangle",
> + "White frame on black field"
> +};
> +
> +static const u32 mali_c55_tpg_mbus_codes[] = {
> + MEDIA_BUS_FMT_SRGGB20_1X20,
> + MEDIA_BUS_FMT_RGB202020_1X60,
> +};
> +
> +static void mali_c55_tpg_update_vblank(struct mali_c55_tpg *tpg,
> + struct v4l2_mbus_framefmt *format)
> +{
> + unsigned int def_vblank;
> + unsigned int min_vblank;
> + unsigned int hts;
> + int tgt_fps;
> +
> + hts = format->width + MALI_C55_TPG_FIXED_HBLANK;
> +
> + /*
> + * The ISP has minimum vertical blanking requirements that must be
> + * adhered to by the TPG. The minimum is a function of the Iridix blocks
> + * clocking requirements and the width of the image and horizontal
> + * blanking, but if we assume the worst case iVariance and sVariance
> + * values then it boils down to the below (plus one to the numerator to
> + * ensure the answer is rounded up).
> + */
> + min_vblank = 15 + (120501 / hts);
> +
> + /*
> + * We need to set a sensible default vblank for whatever format height
> + * we happen to be given from set_fmt(). This function just targets
> + * an even multiple of 15fps. If we can't get 15fps, let's target 5fps.
> + * If we can't get 5fps we'll take whatever the minimum vblank gives us.
> + */
> + tgt_fps = MALI_C55_TPG_PIXEL_RATE / hts / (format->height + min_vblank);
> +
> + if (tgt_fps < 5)
> + def_vblank = min_vblank;
> + else
> + def_vblank = (MALI_C55_TPG_PIXEL_RATE / hts
> + / max(rounddown(tgt_fps, 15), 5)) - format->height;
> +
> + def_vblank = ALIGN_DOWN(def_vblank, 2);
> +
> + __v4l2_ctrl_modify_range(tpg->ctrls.vblank, min_vblank,
> + MALI_C55_TPG_MAX_VBLANK, 1, def_vblank);
> + __v4l2_ctrl_s_ctrl(tpg->ctrls.vblank, def_vblank);
> +}
> +
> +static int mali_c55_tpg_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> + struct mali_c55_tpg *tpg = container_of(ctrl->handler,
> + struct mali_c55_tpg,
> + ctrls.handler);
> + struct mali_c55 *mali_c55 = container_of(tpg, struct mali_c55, tpg);
> +
> + if (!pm_runtime_get_if_in_use(mali_c55->dev))
> + return 0;
> +
> + switch (ctrl->id) {
> + case V4L2_CID_TEST_PATTERN:
> + mali_c55_ctx_write(mali_c55,
> + MALI_C55_REG_TEST_GEN_CH0_PATTERN_TYPE,
> + ctrl->val);
> + break;
> + case V4L2_CID_VBLANK:
> + mali_c55_update_bits(mali_c55, MALI_C55_REG_BLANKING,
> + MALI_C55_REG_VBLANK_MASK,
> + MALI_C55_VBLANK(ctrl->val));
> + break;
> + default:
> + return -EINVAL;
You're skipping the pm_runtime_put().
default:
ret = -EINVAL;
break;
and return ret balow.
> + }
> +
> + pm_runtime_put(mali_c55->dev);
pm_runtime_put_autosuspend(mali_c55->dev);
pm_runtime_mark_last_busy(...);
> +
> + return 0;
> +}
> +
> +static const struct v4l2_ctrl_ops mali_c55_tpg_ctrl_ops = {
> + .s_ctrl = &mali_c55_tpg_s_ctrl,
> +};
> +
> +static void mali_c55_tpg_configure(struct mali_c55_tpg *tpg)
> +{
> + struct v4l2_subdev *sd = &tpg->sd;
> + struct v4l2_subdev_state *state;
> + struct v4l2_mbus_framefmt *fmt;
> + u32 test_pattern_format;
> +
> + /*
> + * hblank needs setting, but is a read-only control and thus won't be
> + * called during __v4l2_ctrl_handler_setup(). Do it here instead.
> + */
> + mali_c55_update_bits(tpg->mali_c55, MALI_C55_REG_BLANKING,
> + MALI_C55_REG_HBLANK_MASK,
> + MALI_C55_TPG_FIXED_HBLANK);
> + mali_c55_update_bits(tpg->mali_c55, MALI_C55_REG_GEN_VIDEO,
> + MALI_C55_REG_GEN_VIDEO_MULTI_MASK,
> + MALI_C55_REG_GEN_VIDEO_MULTI_MASK);
> +
> + state = v4l2_subdev_get_locked_active_state(sd);
You can pass the state from the caller.
> + fmt = v4l2_subdev_state_get_format(state, MALI_C55_TPG_SRC_PAD);
> +
> + test_pattern_format = fmt->code == MEDIA_BUS_FMT_RGB202020_1X60 ?
> + 0x01 : 0x0;
test_pattern_format = fmt->code == MEDIA_BUS_FMT_RGB202020_1X60 ?
0x01 : 0x0;
or
test_pattern_format = fmt->code == MEDIA_BUS_FMT_RGB202020_1X60
? 0x01 : 0x0;
I prefer the latter, but I know that's not a widely approved choice.
> +
> + mali_c55_ctx_update_bits(tpg->mali_c55, MALI_C55_REG_TPG_CH0,
> + MALI_C55_TEST_PATTERN_RGB_MASK,
> + MALI_C55_TEST_PATTERN_RGB(test_pattern_format));
> +
> + v4l2_subdev_unlock_state(state);
And this is wrong, you're unlocking a state you haven't locked. Drop it.
> +}
> +
> +static int mali_c55_tpg_enum_mbus_code(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + if (code->index >= ARRAY_SIZE(mali_c55_tpg_mbus_codes))
> + return -EINVAL;
> +
> + code->code = mali_c55_tpg_mbus_codes[code->index];
> +
> + return 0;
> +}
> +
> +static int mali_c55_tpg_enum_frame_size(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_frame_size_enum *fse)
> +{
> + unsigned int i;
> +
> + if (fse->index > 0)
> + return -EINVAL;
> +
> + for (i = 0; i < ARRAY_SIZE(mali_c55_tpg_mbus_codes); i++) {
> + if (fse->code == mali_c55_tpg_mbus_codes[i])
> + break;
> + }
> +
> + if (i == ARRAY_SIZE(mali_c55_tpg_mbus_codes))
> + return -EINVAL;
> +
> + fse->min_width = MALI_C55_MIN_WIDTH;
> + fse->max_width = MALI_C55_MAX_WIDTH;
> + fse->min_height = MALI_C55_MIN_HEIGHT;
> + fse->max_height = MALI_C55_MAX_HEIGHT;
> +
> + return 0;
> +}
> +
> +static int mali_c55_tpg_set_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *format)
> +{
> + struct mali_c55_tpg *tpg = container_of(sd, struct mali_c55_tpg, sd);
> + struct v4l2_mbus_framefmt *fmt;
> + unsigned int i;
> +
> + fmt = v4l2_subdev_state_get_format(state, MALI_C55_TPG_SRC_PAD);
> + fmt->code = format->format.code;
> +
> + for (i = 0; i < ARRAY_SIZE(mali_c55_tpg_mbus_codes); i++) {
> + if (fmt->code == mali_c55_tpg_mbus_codes[i])
> + break;
> + }
> +
> + if (i == ARRAY_SIZE(mali_c55_tpg_mbus_codes))
> + fmt->code = MEDIA_BUS_FMT_SRGGB20_1X20;
> +
> + /*
> + * The TPG says that the test frame timing generation logic expects a
> + * minimum framesize of 4x4 pixels, but given the rest of the ISP can't
> + * handle anything smaller than 128x128 it seems pointless to allow a
> + * smaller frame.
> + */
> + fmt->width = clamp(fmt->width, MALI_C55_MIN_WIDTH, MALI_C55_MAX_WIDTH);
> + fmt->height = clamp(fmt->height, MALI_C55_MIN_HEIGHT,
> + MALI_C55_MAX_HEIGHT);
> +
> + format->format = *fmt;
> +
> + if (format->which == V4L2_SUBDEV_FORMAT_TRY)
> + return 0;
> +
> + mali_c55_tpg_update_vblank(tpg, fmt);
> +
> + return 0;
> +}
> +
> +static int mali_c55_tpg_enable_streams(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state, u32 pad,
> + u64 streams_mask)
> +{
> + struct mali_c55_tpg *tpg = container_of(sd, struct mali_c55_tpg, sd);
> + struct mali_c55 *mali_c55 = container_of(tpg, struct mali_c55, tpg);
> +
> + /*
> + * We only have a source pad and a single stream, and v4l2-core already
> + * validated both so we don't need to do that. One might reasonably
> + * expect the framesize to be set here given it's configurable in
> + * .set_fmt(), but it's done in the ISP subdevice's .enable_streams()
> + * instead, as the same register is also used to indicate the size of
> + * the data coming from the sensor.
> + */
> + mali_c55_tpg_configure(tpg);
> + __v4l2_ctrl_handler_setup(sd->ctrl_handler);
> +
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_TPG_CH0,
> + MALI_C55_TEST_PATTERN_ON_OFF,
> + MALI_C55_TEST_PATTERN_ON_OFF);
> + mali_c55_update_bits(mali_c55, MALI_C55_REG_GEN_VIDEO,
> + MALI_C55_REG_GEN_VIDEO_ON_MASK,
> + MALI_C55_REG_GEN_VIDEO_ON_MASK);
> +
> + return 0;
> +}
> +
> +static int mali_c55_tpg_disable_streams(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state, u32 pad,
> + u64 streams_mask)
> +{
> + struct mali_c55_tpg *tpg = container_of(sd, struct mali_c55_tpg, sd);
> + struct mali_c55 *mali_c55 = container_of(tpg, struct mali_c55, tpg);
> +
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_TPG_CH0,
> + MALI_C55_TEST_PATTERN_ON_OFF, 0x00);
> + mali_c55_update_bits(mali_c55, MALI_C55_REG_GEN_VIDEO,
> + MALI_C55_REG_GEN_VIDEO_ON_MASK, 0x00);
> +
> + return 0;
> +}
> +
> +static const struct v4l2_subdev_pad_ops mali_c55_tpg_pad_ops = {
> + .enum_mbus_code = mali_c55_tpg_enum_mbus_code,
> + .enum_frame_size = mali_c55_tpg_enum_frame_size,
> + .get_fmt = v4l2_subdev_get_fmt,
> + .set_fmt = mali_c55_tpg_set_fmt,
> + .enable_streams = mali_c55_tpg_enable_streams,
> + .disable_streams = mali_c55_tpg_disable_streams,
> +};
> +
> +static const struct v4l2_subdev_core_ops mali_c55_isp_core_ops = {
> + .subscribe_event = v4l2_ctrl_subdev_subscribe_event,
> + .unsubscribe_event = v4l2_event_subdev_unsubscribe,
> +};
> +
> +static const struct v4l2_subdev_ops mali_c55_tpg_ops = {
> + .core = &mali_c55_isp_core_ops,
> + .pad = &mali_c55_tpg_pad_ops,
> +};
> +
> +static int mali_c55_tpg_init_state(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state)
> +{
> + struct v4l2_mbus_framefmt *fmt =
> + v4l2_subdev_state_get_format(state, MALI_C55_TPG_SRC_PAD);
> +
> + fmt->width = MALI_C55_DEFAULT_WIDTH;
> + fmt->height = MALI_C55_DEFAULT_HEIGHT;
> + fmt->field = V4L2_FIELD_NONE;
> + fmt->code = MEDIA_BUS_FMT_SRGGB20_1X20;
> + fmt->colorspace = V4L2_COLORSPACE_RAW;
Initialize the other three colorspace-related fields.
> +
> + return 0;
> +}
> +
> +static const struct v4l2_subdev_internal_ops mali_c55_tpg_internal_ops = {
> + .init_state = mali_c55_tpg_init_state,
> +};
> +
> +static int mali_c55_tpg_init_controls(struct mali_c55 *mali_c55)
> +{
> + struct mali_c55_tpg_ctrls *ctrls = &mali_c55->tpg.ctrls;
> + struct v4l2_ctrl *pixel_rate;
> + struct v4l2_ctrl *hblank;
> + int ret;
> +
> + ret = v4l2_ctrl_handler_init(&ctrls->handler, 4);
> + if (ret)
> + return ret;
> +
> + v4l2_ctrl_new_std_menu_items(&ctrls->handler,
> + &mali_c55_tpg_ctrl_ops, V4L2_CID_TEST_PATTERN,
> + ARRAY_SIZE(mali_c55_tpg_test_pattern_menu) - 1,
> + 0, 3, mali_c55_tpg_test_pattern_menu);
> +
> + /*
> + * We fix hblank at the minimum allowed value and control framerate
> + * solely through the vblank control.
> + */
> + hblank = v4l2_ctrl_new_std(&ctrls->handler, &mali_c55_tpg_ctrl_ops,
> + V4L2_CID_HBLANK, MALI_C55_TPG_FIXED_HBLANK,
> + MALI_C55_TPG_FIXED_HBLANK, 1,
> + MALI_C55_TPG_FIXED_HBLANK);
> + if (hblank)
> + hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
> +
> + ctrls->vblank = v4l2_ctrl_new_std(&ctrls->handler,
> + &mali_c55_tpg_ctrl_ops,
> + V4L2_CID_VBLANK,
> + MALI_C55_TPG_DEFAULT_MIN_VBLANK,
> + MALI_C55_TPG_MAX_VBLANK, 1,
> + MALI_C55_TPG_DEFAULT_DEF_VBLANK);
> +
> + pixel_rate = v4l2_ctrl_new_std(&ctrls->handler, &mali_c55_tpg_ctrl_ops,
> + V4L2_CID_PIXEL_RATE,
> + MALI_C55_TPG_PIXEL_RATE,
> + MALI_C55_TPG_PIXEL_RATE, 1,
> + MALI_C55_TPG_PIXEL_RATE);
> + if (pixel_rate)
> + pixel_rate->flags |= V4L2_CTRL_FLAG_READ_ONLY;
> +
> + if (ctrls->handler.error) {
> + dev_err(mali_c55->dev, "Error during v4l2 controls init\n");
> + ret = ctrls->handler.error;
> + goto err_free_handler;
Replace the goto with
v4l2_ctrl_handler_free(&ctrls->handler);
return ret;
as the label is used here only.
> + }
> +
> + mali_c55->tpg.sd.ctrl_handler = &ctrls->handler;
> + mali_c55->tpg.sd.state_lock = ctrls->handler.lock;
I would move the last line to mali_c55_register_tpg(). Up to you.
> +
> + return 0;
> +
> +err_free_handler:
> + v4l2_ctrl_handler_free(&ctrls->handler);
> +
> + return ret;
> +}
> +
> +int mali_c55_register_tpg(struct mali_c55 *mali_c55)
> +{
> + struct mali_c55_tpg *tpg = &mali_c55->tpg;
> + struct v4l2_subdev *sd = &tpg->sd;
> + struct media_pad *pad = &tpg->pad;
> + int ret;
> +
> + v4l2_subdev_init(sd, &mali_c55_tpg_ops);
> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
> + sd->entity.function = MEDIA_ENT_F_CAM_SENSOR;
> + sd->internal_ops = &mali_c55_tpg_internal_ops;
> + strscpy(sd->name, MALI_C55_DRIVER_NAME " tpg", sizeof(sd->name));
> +
> + pad->flags = MEDIA_PAD_FL_SOURCE;
> + ret = media_entity_pads_init(&sd->entity, 1, pad);
> + if (ret) {
> + dev_err(mali_c55->dev,
> + "Failed to initialize media entity pads\n");
> + return ret;
> + }
> +
> + ret = mali_c55_tpg_init_controls(mali_c55);
> + if (ret) {
> + dev_err(mali_c55->dev,
> + "Error initialising controls\n");
> + goto err_cleanup_media_entity;
> + }
> +
> + ret = v4l2_subdev_init_finalize(sd);
> + if (ret)
> + goto err_free_ctrl_handler;
> +
> + ret = v4l2_device_register_subdev(&mali_c55->v4l2_dev, sd);
> + if (ret) {
> + dev_err(mali_c55->dev, "Failed to register tpg subdev\n");
> + goto err_cleanup_subdev;
> + }
> +
> + /*
> + * By default the colour settings lead to a very dim image that is
> + * nearly indistinguishable from black on some monitor settings. Ramp
> + * them up a bit so the image is brighter.
> + */
> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_TPG_R_BACKGROUND,
> + MALI_C55_TPG_BACKGROUND_MAX);
> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_TPG_G_BACKGROUND,
> + MALI_C55_TPG_BACKGROUND_MAX);
> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_TPG_B_BACKGROUND,
> + MALI_C55_TPG_BACKGROUND_MAX);
> +
> + tpg->mali_c55 = mali_c55;
> +
> + return 0;
> +
> +err_cleanup_subdev:
> + v4l2_subdev_cleanup(sd);
> +err_free_ctrl_handler:
> + v4l2_ctrl_handler_free(&tpg->ctrls.handler);
> +err_cleanup_media_entity:
> + media_entity_cleanup(&sd->entity);
> +
> + return ret;
> +}
> +
> +void mali_c55_unregister_tpg(struct mali_c55 *mali_c55)
> +{
> + struct mali_c55_tpg *tpg = &mali_c55->tpg;
> +
> + if (!tpg->mali_c55)
> + return;
> +
> + v4l2_device_unregister_subdev(&tpg->sd);
> + v4l2_ctrl_handler_free(&tpg->ctrls.handler);
> + v4l2_subdev_cleanup(&tpg->sd);
> + media_entity_cleanup(&tpg->sd.entity);
> +}
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 13/18] media: platform: Fill stats buffer on ISP_START
2024-07-09 13:29 ` [PATCH v6 13/18] media: platform: Fill stats buffer on ISP_START Daniel Scally
@ 2024-07-30 21:46 ` Laurent Pinchart
0 siblings, 0 replies; 41+ messages in thread
From: Laurent Pinchart @ 2024-07-30 21:46 UTC (permalink / raw)
To: Daniel Scally
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Dan,
Thank you for the patch.
On Tue, Jul 09, 2024 at 02:29:01PM +0100, Daniel Scally wrote:
> On ISP_START, fill the stats buffer by reading out the metering space
> in the ISP's memory. This is done for the non-active config just as
> the dma transfer of the registers is. To acheive that, move the
s/acheive/achieve/
> checking of the current config outside of mali_c55_swap_next_config()
> so we can use it for both functions.
>
> Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
> ---
> Changes in v6:
>
> - None
>
> Changes in v5:
>
> - New patch
>
> .../platform/arm/mali-c55/mali-c55-core.c | 34 ++++++++++++++-----
> 1 file changed, 26 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> index eedc8f450184..ed0db34767a4 100644
> --- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> @@ -567,15 +567,9 @@ static int mali_c55_check_hwcfg(struct mali_c55 *mali_c55)
> return 0;
> }
>
> -static void mali_c55_swap_next_config(struct mali_c55 *mali_c55)
> +static void mali_c55_swap_next_config(struct mali_c55 *mali_c55, u32 next_config)
> {
> struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
> - u32 curr_config, next_config;
> -
> - curr_config = mali_c55_read(mali_c55, MALI_C55_REG_PING_PONG_READ);
> - curr_config = (curr_config & MALI_C55_REG_PING_PONG_READ_MASK)
> - >> (ffs(MALI_C55_REG_PING_PONG_READ_MASK) - 1);
> - next_config = curr_config ^ 1;
>
> mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG,
> MALI_C55_REG_MCU_CONFIG_WRITE_MASK,
> @@ -588,6 +582,7 @@ static irqreturn_t mali_c55_isr(int irq, void *context)
> {
> struct device *dev = context;
> struct mali_c55 *mali_c55 = dev_get_drvdata(dev);
> + u32 curr_config, next_config;
> u32 interrupt_status;
> unsigned int i, j;
>
> @@ -612,7 +607,30 @@ static irqreturn_t mali_c55_isr(int irq, void *context)
> for (j = i; j < MALI_C55_NUM_CAP_DEVS; j++)
> mali_c55_set_next_buffer(&mali_c55->cap_devs[j]);
>
> - mali_c55_swap_next_config(mali_c55);
> + /*
> + * When the ISP starts a frame we have some work to do:
> + *
> + * 1. Copy over the config for the **next** frame
> + * 2. Read out the metering stats for the **last** frame
> + */
> +
> + curr_config = mali_c55_read(mali_c55,
> + MALI_C55_REG_PING_PONG_READ);
> + curr_config &= MALI_C55_REG_PING_PONG_READ_MASK;
> + curr_config >>= ffs(MALI_C55_REG_PING_PONG_READ_MASK) - 1;
> + next_config = curr_config ^ 1;
> +
> + /*
> + * The ordering of these two is currently important as
> + * mali_c55_stats_fill_buffer() is asynchronous whereas
> + * mali_c55_swap_next_config() is not.
> + *
> + * TODO: Should mali_c55_swap_next_config() be async?
Isn't it in this version of the series, at least when using DMA ?
As I wrote in the review of 07/18, I think the reconfiguration probably
needs more careful consideration.
> + */
> + mali_c55_stats_fill_buffer(mali_c55,
> + next_config ? MALI_C55_CONFIG_PING :
> + MALI_C55_CONFIG_PONG);
> + mali_c55_swap_next_config(mali_c55, next_config);
>
> break;
> case MALI_C55_IRQ_ISP_DONE:
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 17/18] media: platform: Add mali-c55 parameters video node
2024-07-09 13:29 ` [PATCH v6 17/18] media: platform: Add mali-c55 parameters video node Daniel Scally
@ 2024-07-30 22:16 ` Laurent Pinchart
2024-07-31 6:53 ` Jacopo Mondi
2024-07-31 15:12 ` Dan Scally
0 siblings, 2 replies; 41+ messages in thread
From: Laurent Pinchart @ 2024-07-30 22:16 UTC (permalink / raw)
To: Daniel Scally
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Dan,
Thank you for the patch.
On Tue, Jul 09, 2024 at 02:29:05PM +0100, Daniel Scally wrote:
> Add a new code file to the mali-c55 driver that registers an output
> video node for userspace to queue buffers of parameters to. Handlers
> are included to program the statistics generation plus the white
> balance, black level correction and mesh shading correction blocks.
>
> Update the rest of the driver to register and link the new video node
>
> Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
> ---
> Changes in v6:
>
> - Used a union to generalise the block pointer rather than resorting to
> casting everywhere - fantastic idea Sakari, this made it much cleaner.
> - Reworked the loop in mali_c55_params_write_config() so that we can be
> sure there's remaining space for the next block header.
>
> Changes in v5:
>
> - New patch
>
> drivers/media/platform/arm/mali-c55/Makefile | 1 +
> .../platform/arm/mali-c55/mali-c55-common.h | 20 +
> .../platform/arm/mali-c55/mali-c55-core.c | 23 +
> .../platform/arm/mali-c55/mali-c55-isp.c | 21 +-
> .../platform/arm/mali-c55/mali-c55-params.c | 671 ++++++++++++++++++
> .../arm/mali-c55/mali-c55-registers.h | 128 ++++
> 6 files changed, 863 insertions(+), 1 deletion(-)
> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-params.c
>
> diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile
> index b5a22d414479..d5718b0b23e0 100644
> --- a/drivers/media/platform/arm/mali-c55/Makefile
> +++ b/drivers/media/platform/arm/mali-c55/Makefile
> @@ -3,6 +3,7 @@
> mali-c55-y := mali-c55-capture.o \
> mali-c55-core.o \
> mali-c55-isp.o \
> + mali-c55-params.o \
> mali-c55-resizer.o \
> mali-c55-stats.o \
> mali-c55-tpg.o
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> index 136c785c68ba..66a46a7c0547 100644
> --- a/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> @@ -50,6 +50,7 @@ enum mali_c55_isp_pads {
> MALI_C55_ISP_PAD_SOURCE_VIDEO,
> MALI_C55_ISP_PAD_SOURCE_BYPASS,
> MALI_C55_ISP_PAD_SOURCE_STATS,
> + MALI_C55_ISP_PAD_SINK_PARAMS,
> MALI_C55_ISP_NUM_PADS,
> };
>
> @@ -184,6 +185,21 @@ struct mali_c55_stats {
> } buffers;
> };
>
> +struct mali_c55_params {
> + struct mali_c55 *mali_c55;
> + struct video_device vdev;
> + struct vb2_queue queue;
> + struct media_pad pad;
> + /* Mutex to provide to vb2 */
> + struct mutex lock;
> +
> + struct {
> + /* Spinlock to guard buffer queue */
> + spinlock_t lock;
> + struct list_head queue;
> + } buffers;
> +};
> +
> enum mali_c55_config_spaces {
> MALI_C55_CONFIG_PING,
> MALI_C55_CONFIG_PONG,
> @@ -226,6 +242,7 @@ struct mali_c55 {
> struct mali_c55_isp isp;
> struct mali_c55_resizer resizers[MALI_C55_NUM_RSZS];
> struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS];
> + struct mali_c55_params params;
> struct mali_c55_stats stats;
>
> struct mali_c55_context context;
> @@ -256,6 +273,8 @@ int mali_c55_register_capture_devs(struct mali_c55 *mali_c55);
> void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55);
> int mali_c55_register_stats(struct mali_c55 *mali_c55);
> void mali_c55_unregister_stats(struct mali_c55 *mali_c55);
> +int mali_c55_register_params(struct mali_c55 *mali_c55);
> +void mali_c55_unregister_params(struct mali_c55 *mali_c55);
> struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55);
> void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
> enum mali_c55_planes plane);
> @@ -272,5 +291,6 @@ const struct mali_c55_isp_fmt *
> mali_c55_isp_get_mbus_config_by_index(u32 index);
> void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
> enum mali_c55_config_spaces cfg_space);
> +void mali_c55_params_write_config(struct mali_c55 *mali_c55);
>
> #endif /* _MALI_C55_COMMON_H */
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> index ed0db34767a4..55b3cbf53791 100644
> --- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> @@ -384,6 +384,16 @@ static int mali_c55_create_links(struct mali_c55 *mali_c55)
> goto err_remove_links;
> }
>
> + ret = media_create_pad_link(&mali_c55->params.vdev.entity, 0,
> + &mali_c55->isp.sd.entity,
> + MALI_C55_ISP_PAD_SINK_PARAMS,
> + MEDIA_LNK_FL_ENABLED);
Should this be immutable, or do you think it makes sense to support
operating the ISP without parameters ? I know we did so when developing
the driver to test the initial code, but are there real use cases now ?
> + if (ret) {
> + dev_err(mali_c55->dev,
> + "failed to link ISP and parameters video node\n");
> + goto err_remove_links;
> + }
> +
> return 0;
>
> err_remove_links:
> @@ -398,6 +408,7 @@ static void mali_c55_unregister_entities(struct mali_c55 *mali_c55)
> mali_c55_unregister_isp(mali_c55);
> mali_c55_unregister_resizers(mali_c55);
> mali_c55_unregister_capture_devs(mali_c55);
> + mali_c55_unregister_params(mali_c55);
> mali_c55_unregister_stats(mali_c55);
> }
>
> @@ -421,6 +432,10 @@ static int mali_c55_register_entities(struct mali_c55 *mali_c55)
> if (ret)
> goto err_unregister_entities;
>
> + ret = mali_c55_register_params(mali_c55);
> + if (ret)
> + goto err_unregister_entities;
> +
> ret = mali_c55_register_stats(mali_c55);
> if (ret)
> goto err_unregister_entities;
> @@ -620,6 +635,14 @@ static irqreturn_t mali_c55_isr(int irq, void *context)
> curr_config >>= ffs(MALI_C55_REG_PING_PONG_READ_MASK) - 1;
> next_config = curr_config ^ 1;
>
> + /*
> + * Write the configuration parameters received from
> + * userspace into the configuration buffer, which will
> + * be transferred to the 'next' active config space at
> + * by mali_c55_swap_next_config().
> + */
> + mali_c55_params_write_config(mali_c55);
> +
> /*
> * The ordering of these two is currently important as
> * mali_c55_stats_fill_buffer() is asynchronous whereas
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> index 2f450c00300a..40d7ef6eda22 100644
> --- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> @@ -132,6 +132,7 @@ static int mali_c55_isp_start(struct mali_c55 *mali_c55)
> cfg->bypass ? MALI_C55_ISP_RAW_BYPASS_BYPASS_MASK :
> 0x00);
>
> + mali_c55_params_write_config(mali_c55);
> ret = mali_c55_config_write(ctx, MALI_C55_CONFIG_PING);
> if (ret) {
> dev_err(mali_c55->dev, "failed to write ISP config\n");
> @@ -464,12 +465,17 @@ static int mali_c55_isp_init_state(struct v4l2_subdev *sd,
>
> src_fmt = v4l2_subdev_state_get_format(state,
> MALI_C55_ISP_PAD_SOURCE_STATS);
> + sink_fmt = v4l2_subdev_state_get_format(state,
> + MALI_C55_ISP_PAD_SINK_PARAMS);
>
> src_fmt->width = sizeof(struct mali_c55_stats_buffer);
> src_fmt->height = 1;
> src_fmt->field = V4L2_FIELD_NONE;
> src_fmt->code = MEDIA_BUS_FMT_METADATA_FIXED;
>
> + *sink_fmt = *src_fmt;
I would initialize the fields individually, I think the code would be
clearer.
> + sink_fmt->width = sizeof(struct mali_c55_params_buffer);
> +
> return 0;
> }
>
> @@ -477,8 +483,20 @@ static const struct v4l2_subdev_internal_ops mali_c55_isp_internal_ops = {
> .init_state = mali_c55_isp_init_state,
> };
>
> +static int mali_c55_subdev_link_validate(struct media_link *link)
> +{
> + /*
> + * Skip validation for the parameters sink pad, as the source is not
> + * a subdevice.
> + */
> + if (link->sink->index == MALI_C55_ISP_PAD_SINK_PARAMS)
> + return 0;
> +
> + return v4l2_subdev_link_validate(link);
> +}
> +
> static const struct media_entity_operations mali_c55_isp_media_ops = {
> - .link_validate = v4l2_subdev_link_validate,
> + .link_validate = mali_c55_subdev_link_validate,
> };
>
> int mali_c55_register_isp(struct mali_c55 *mali_c55)
> @@ -501,6 +519,7 @@ int mali_c55_register_isp(struct mali_c55 *mali_c55)
> isp->pads[MALI_C55_ISP_PAD_SOURCE_VIDEO].flags = MEDIA_PAD_FL_SOURCE;
> isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE;
> isp->pads[MALI_C55_ISP_PAD_SOURCE_STATS].flags = MEDIA_PAD_FL_SOURCE;
> + isp->pads[MALI_C55_ISP_PAD_SINK_PARAMS].flags = MEDIA_PAD_FL_SINK;
>
> ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS,
> isp->pads);
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-params.c b/drivers/media/platform/arm/mali-c55/mali-c55-params.c
> new file mode 100644
> index 000000000000..c0ca4a759653
> --- /dev/null
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-params.c
> @@ -0,0 +1,671 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * ARM Mali-C55 ISP Driver - Configuration parameters output device
> + *
> + * Copyright (C) 2024 Ideas on Board Oy
> + */
> +#include <linux/media/arm/mali-c55-config.h>
> +
> +#include <media/media-entity.h>
> +#include <media/v4l2-dev.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-fh.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mali-c55-common.h"
> +#include "mali-c55-registers.h"
> +
> +typedef void (*mali_c55_block_handler)(struct mali_c55 *mali_c55,
> + union mali_c55_params_block block);
> +
> +struct mali_c55_block_handler {
> + size_t size;
> + mali_c55_block_handler handler;
> +};
> +
> +static void mali_c55_params_sensor_offs(struct mali_c55 *mali_c55,
> + union mali_c55_params_block block)
> +{
> + struct mali_c55_params_sensor_off_preshading *p = block.sensor_offs;
> + __u32 global_offset;
> +
> + if (!block.header->enabled)
> + return;
> +
> + if (!(p->chan00 || p->chan01 || p->chan10 || p->chan11))
> + return;
> +
> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_00,
> + p->chan00 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_01,
> + p->chan01 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_10,
> + p->chan10 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_11,
> + p->chan11 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
> +
> + /*
> + * The average offset is applied as a global offset for the digital
> + * gain block
> + */
> + global_offset = (p->chan00 + p->chan01 + p->chan10 + p->chan11) >> 2;
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_DIGITAL_GAIN_OFFSET,
> + MALI_C55_DIGITAL_GAIN_OFFSET_MASK,
> + global_offset);
> +
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BYPASS_3,
> + MALI_C55_REG_BYPASS_3_SENSOR_OFFSET_PRE_SH,
> + 0x00);
> +}
> +
> +static void mali_c55_params_aexp_hist(struct mali_c55 *mali_c55,
> + union mali_c55_params_block block)
> +{
> + u32 disable_mask;
> + u32 disable_val;
> + u32 base;
> +
> + if (block.header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST) {
> + disable_mask = MALI_C55_AEXP_HIST_DISABLE_MASK;
> + disable_val = MALI_C55_AEXP_HIST_DISABLE;
> + base = MALI_C55_REG_AEXP_HIST_BASE;
> + } else {
> + disable_mask = MALI_C55_AEXP_IHIST_DISABLE_MASK;
> + disable_val = MALI_C55_AEXP_IHIST_DISABLE;
> + base = MALI_C55_REG_AEXP_IHIST_BASE;
> + }
> + struct mali_c55_params_aexp_hist *params = block.aexp_hist;
> +
> + if (!block.header->enabled) {
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> + disable_mask, disable_val);
> + return;
> + }
> +
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> + disable_mask, false);
> +
> + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
> + MALI_C55_AEXP_HIST_SKIP_X_MASK, params->skip_x);
> + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
> + MALI_C55_AEXP_HIST_OFFSET_X_MASK,
> + MALI_C55_AEXP_HIST_OFFSET_X(params->offset_x));
> + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
> + MALI_C55_AEXP_HIST_SKIP_Y_MASK,
> + MALI_C55_AEXP_HIST_SKIP_Y(params->skip_y));
> + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
> + MALI_C55_AEXP_HIST_OFFSET_Y_MASK,
> + MALI_C55_AEXP_HIST_OFFSET_Y(params->offset_y));
> +
> + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SCALE_OFFSET,
> + MALI_C55_AEXP_HIST_SCALE_BOTTOM_MASK,
> + params->scale_bottom);
> + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SCALE_OFFSET,
> + MALI_C55_AEXP_HIST_SCALE_TOP_MASK,
> + MALI_C55_AEXP_HIST_SCALE_TOP(params->scale_top));
> +
> + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_PLANE_MODE_OFFSET,
> + MALI_C55_AEXP_HIST_PLANE_MODE_MASK,
> + params->plane_mode);
> +
> + if (block.header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST)
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> + MALI_C55_AEXP_HIST_SWITCH_MASK,
> + MALI_C55_AEXP_HIST_SWITCH(params->tap_point));
> +}
> +
> +static void
> +mali_c55_params_aexp_hist_weights(struct mali_c55 *mali_c55,
> + union mali_c55_params_block block)
> +{
> + struct mali_c55_params_aexp_weights *params = block.aexp_weights;
> + u32 base;
> +
> + if (!block.header->enabled)
> + return;
> +
> + base = block.header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS ?
> + MALI_C55_REG_AEXP_HIST_BASE :
> + MALI_C55_REG_AEXP_IHIST_BASE;
> +
> + mali_c55_ctx_update_bits(mali_c55,
> + base + MALI_C55_AEXP_HIST_NODES_USED_OFFSET,
> + MALI_C55_AEXP_HIST_NODES_USED_HORIZ_MASK,
> + params->nodes_used_horiz);
> + mali_c55_ctx_update_bits(mali_c55,
> + base + MALI_C55_AEXP_HIST_NODES_USED_OFFSET,
> + MALI_C55_AEXP_HIST_NODES_USED_VERT_MASK,
> + MALI_C55_AEXP_HIST_NODES_USED_VERT(params->nodes_used_vert));
> +
> + /*
> + * The zone weights array is a 225-element array of u8 values, but that
> + * is a bit annoying to handle given the ISP expects 32-bit writes. We
> + * just reinterpret it as a 57-element array of 32-bit values for the
> + * purposes of this transaction (the 3 bytes of additional space at the
> + * end of the write is just padding for the array of weights in the ISP
> + * memory space anyway, so there's no risk of overwriting other
> + * registers).
> + */
> + for (unsigned int i = 0; i < 57; i++) {
> + u32 val = ((u32 *)params->zone_weights)[i]
> + & MALI_C55_AEXP_HIST_ZONE_WEIGHT_MASK;
> + u32 addr = base + MALI_C55_AEXP_HIST_ZONE_WEIGHTS_OFFSET + (4 * i);
> +
> + mali_c55_ctx_write(mali_c55, addr, val);
> + }
> +}
> +
> +static void mali_c55_params_digital_gain(struct mali_c55 *mali_c55,
> + union mali_c55_params_block block)
> +{
> + struct mali_c55_params_digital_gain *dgain = block.digital_gain;
> +
> + /*
> + * If the block is flagged as disabled we write a gain of 1.0, which in
> + * Q5.8 format is 256.
> + */
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_DIGITAL_GAIN,
> + MALI_C55_DIGITAL_GAIN_MASK,
> + block.header->enabled ? dgain->gain : 256);
> +}
> +
> +static void mali_c55_params_awb_gains(struct mali_c55 *mali_c55,
> + union mali_c55_params_block block)
> +{
> + struct mali_c55_params_awb_gains *gains = block.awb_gains;
> +
> + /*
> + * There are two places AWB gains can be set in the ISP; one affects the
> + * image output data and the other affects the statistics for the
> + * AEXP-0 tap point.
> + */
> + u32 addr1 = block.header->type == MALI_C55_PARAM_BLOCK_AWB_GAINS ?
> + MALI_C55_REG_AWB_GAINS1 :
> + MALI_C55_REG_AWB_GAINS1_AEXP;
> + u32 addr2 = block.header->type == MALI_C55_PARAM_BLOCK_AWB_GAINS ?
> + MALI_C55_REG_AWB_GAINS2 :
> + MALI_C55_REG_AWB_GAINS2_AEXP;
> +
> + mali_c55_ctx_update_bits(mali_c55, addr1, MALI_C55_AWB_GAIN00_MASK,
> + gains->gain00);
> + mali_c55_ctx_update_bits(mali_c55, addr1, MALI_C55_AWB_GAIN01_MASK,
> + MALI_C55_AWB_GAIN01(gains->gain01));
> + mali_c55_ctx_update_bits(mali_c55, addr2, MALI_C55_AWB_GAIN10_MASK,
> + gains->gain10);
> + mali_c55_ctx_update_bits(mali_c55, addr2, MALI_C55_AWB_GAIN11_MASK,
> + MALI_C55_AWB_GAIN11(gains->gain11));
> +}
> +
> +static void mali_c55_params_awb_config(struct mali_c55 *mali_c55,
> + union mali_c55_params_block block)
> +{
> + struct mali_c55_params_awb_config *params = block.awb_config;
> +
> + if (!block.header->enabled) {
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> + MALI_C55_AWB_DISABLE_MASK,
> + MALI_C55_AWB_DISABLE_MASK);
> + return;
> + }
> +
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> + MALI_C55_AWB_DISABLE_MASK, false);
> +
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_STATS_MODE,
> + MALI_C55_AWB_STATS_MODE_MASK, params->stats_mode);
> +
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_WHITE_LEVEL,
> + MALI_C55_AWB_WHITE_LEVEL_MASK, params->white_level);
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_BLACK_LEVEL,
> + MALI_C55_AWB_BLACK_LEVEL_MASK, params->black_level);
> +
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_MAX,
> + MALI_C55_AWB_CR_MAX_MASK, params->cr_max);
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_MIN,
> + MALI_C55_AWB_CR_MIN_MASK, params->cr_min);
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_MAX,
> + MALI_C55_AWB_CB_MAX_MASK, params->cb_max);
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_MIN,
> + MALI_C55_AWB_CB_MIN_MASK, params->cb_min);
> +
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_NODES_USED,
> + MALI_C55_AWB_NODES_USED_HORIZ_MASK,
> + params->nodes_used_horiz);
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_NODES_USED,
> + MALI_C55_AWB_NODES_USED_VERT_MASK,
> + MALI_C55_AWB_NODES_USED_VERT(params->nodes_used_vert));
> +
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_HIGH,
> + MALI_C55_AWB_CR_HIGH_MASK, params->cr_high);
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_LOW,
> + MALI_C55_AWB_CR_LOW_MASK, params->cr_low);
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_HIGH,
> + MALI_C55_AWB_CB_HIGH_MASK, params->cb_high);
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_LOW,
> + MALI_C55_AWB_CB_LOW_MASK, params->cb_low);
> +
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> + MALI_C55_AWB_SWITCH_MASK,
> + MALI_C55_AWB_SWITCH(params->tap_point));
> +}
> +
> +static void mali_c55_params_lsc_config(struct mali_c55 *mali_c55,
> + union mali_c55_params_block block)
> +{
> + struct mali_c55_params_mesh_shading_config *params = block.shading_config;
> + unsigned int i;
> + u32 addr;
> +
> + if (!block.header->enabled) {
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> + MALI_C55_MESH_SHADING_ENABLE_MASK,
> + false);
> + return;
> + }
> +
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> + MALI_C55_MESH_SHADING_ENABLE_MASK, true);
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> + MALI_C55_MESH_SHADING_MESH_SHOW_MASK,
> + MALI_C55_MESH_SHADING_MESH_SHOW(params->mesh_show));
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> + MALI_C55_MESH_SHADING_SCALE_MASK,
> + MALI_C55_MESH_SHADING_SCALE(params->mesh_scale));
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> + MALI_C55_MESH_SHADING_PAGE_R_MASK,
> + MALI_C55_MESH_SHADING_PAGE_R(params->mesh_page_r));
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> + MALI_C55_MESH_SHADING_PAGE_G_MASK,
> + MALI_C55_MESH_SHADING_PAGE_G(params->mesh_page_g));
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> + MALI_C55_MESH_SHADING_PAGE_B_MASK,
> + MALI_C55_MESH_SHADING_PAGE_B(params->mesh_page_b));
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> + MALI_C55_MESH_SHADING_MESH_WIDTH_MASK,
> + MALI_C55_MESH_SHADING_MESH_WIDTH(params->mesh_width));
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> + MALI_C55_MESH_SHADING_MESH_HEIGHT_MASK,
> + MALI_C55_MESH_SHADING_MESH_HEIGHT(params->mesh_height));
> +
> + for (i = 0; i < MALI_C55_NUM_MESH_SHADING_ELEMENTS; i++) {
> + addr = MALI_C55_REG_MESH_SHADING_TABLES + (i * 4);
> + mali_c55_ctx_write(mali_c55, addr, params->mesh[i]);
> + }
> +}
> +
> +static void mali_c55_params_lsc_selection(struct mali_c55 *mali_c55,
> + union mali_c55_params_block block)
> +{
> + struct mali_c55_params_mesh_shading_selection *params =
> + block.shading_selection;
> +
> + if (!block.header->enabled)
> + return;
> +
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK,
> + MALI_C55_MESH_SHADING_ALPHA_BANK_R_MASK,
> + params->mesh_alpha_bank_r);
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK,
> + MALI_C55_MESH_SHADING_ALPHA_BANK_G_MASK,
> + MALI_C55_MESH_SHADING_ALPHA_BANK_G(params->mesh_alpha_bank_g));
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK,
> + MALI_C55_MESH_SHADING_ALPHA_BANK_B_MASK,
> + MALI_C55_MESH_SHADING_ALPHA_BANK_B(params->mesh_alpha_bank_b));
> +
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA,
> + MALI_C55_MESH_SHADING_ALPHA_R_MASK,
> + params->mesh_alpha_r);
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA,
> + MALI_C55_MESH_SHADING_ALPHA_G_MASK,
> + MALI_C55_MESH_SHADING_ALPHA_G(params->mesh_alpha_g));
> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA,
> + MALI_C55_MESH_SHADING_ALPHA_B_MASK,
> + MALI_C55_MESH_SHADING_ALPHA_B(params->mesh_alpha_b));
> +
> + mali_c55_ctx_update_bits(mali_c55,
> + MALI_C55_REG_MESH_SHADING_MESH_STRENGTH,
> + MALI_c55_MESH_STRENGTH_MASK,
> + params->mesh_strength);
> +}
> +
> +static const struct mali_c55_block_handler mali_c55_block_handlers[] = {
> + [MALI_C55_PARAM_BLOCK_SENSOR_OFFS] = {
> + .size = sizeof(struct mali_c55_params_sensor_off_preshading),
> + .handler = &mali_c55_params_sensor_offs,
> + },
> + [MALI_C55_PARAM_BLOCK_AEXP_HIST] = {
> + .size = sizeof(struct mali_c55_params_aexp_hist),
> + .handler = &mali_c55_params_aexp_hist,
> + },
> + [MALI_C55_PARAM_BLOCK_AEXP_IHIST] = {
> + .size = sizeof(struct mali_c55_params_aexp_hist),
> + .handler = &mali_c55_params_aexp_hist,
> + },
> + [MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS] = {
> + .size = sizeof(struct mali_c55_params_aexp_weights),
> + .handler = &mali_c55_params_aexp_hist_weights,
> + },
> + [MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS] = {
> + .size = sizeof(struct mali_c55_params_aexp_weights),
> + .handler = &mali_c55_params_aexp_hist_weights,
> + },
> + [MALI_C55_PARAM_BLOCK_DIGITAL_GAIN] = {
> + .size = sizeof(struct mali_c55_params_digital_gain),
> + .handler = &mali_c55_params_digital_gain,
> + },
> + [MALI_C55_PARAM_BLOCK_AWB_GAINS] = {
> + .size = sizeof(struct mali_c55_params_awb_gains),
> + .handler = &mali_c55_params_awb_gains,
> + },
> + [MALI_C55_PARAM_BLOCK_AWB_CONFIG] = {
> + .size = sizeof(struct mali_c55_params_awb_config),
> + .handler = &mali_c55_params_awb_config,
> + },
> + [MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP] = {
> + .size = sizeof(struct mali_c55_params_awb_gains),
> + .handler = &mali_c55_params_awb_gains,
> + },
> + [MALI_C55_PARAM_MESH_SHADING_CONFIG] = {
> + .size = sizeof(struct mali_c55_params_mesh_shading_config),
> + .handler = &mali_c55_params_lsc_config,
> + },
> + [MALI_C55_PARAM_MESH_SHADING_SELECTION] = {
> + .size = sizeof(struct mali_c55_params_mesh_shading_selection),
> + .handler = &mali_c55_params_lsc_selection,
> + },
> +};
> +
> +static int mali_c55_params_enum_fmt_meta_out(struct file *file, void *fh,
> + struct v4l2_fmtdesc *f)
> +{
> + if (f->index)
> + return -EINVAL;
> +
> + f->pixelformat = V4L2_META_FMT_MALI_C55_PARAMS;
> +
> + return 0;
> +}
> +
> +static int mali_c55_params_g_fmt_meta_out(struct file *file, void *fh,
> + struct v4l2_format *f)
> +{
> + static const struct v4l2_meta_format mfmt = {
> + .dataformat = V4L2_META_FMT_MALI_C55_PARAMS,
> + .buffersize = sizeof(struct mali_c55_params_buffer),
> + };
> +
> + f->fmt.meta = mfmt;
> +
> + return 0;
> +}
> +
> +static int mali_c55_params_querycap(struct file *file,
> + void *priv, struct v4l2_capability *cap)
> +{
> + strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver));
> + strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card));
> +
> + return 0;
> +}
> +
> +static const struct v4l2_ioctl_ops mali_c55_params_v4l2_ioctl_ops = {
> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> + .vidioc_querybuf = vb2_ioctl_querybuf,
> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> + .vidioc_qbuf = vb2_ioctl_qbuf,
> + .vidioc_expbuf = vb2_ioctl_expbuf,
> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> + .vidioc_streamon = vb2_ioctl_streamon,
> + .vidioc_streamoff = vb2_ioctl_streamoff,
> + .vidioc_enum_fmt_meta_out = mali_c55_params_enum_fmt_meta_out,
> + .vidioc_g_fmt_meta_out = mali_c55_params_g_fmt_meta_out,
> + .vidioc_s_fmt_meta_out = mali_c55_params_g_fmt_meta_out,
> + .vidioc_try_fmt_meta_out = mali_c55_params_g_fmt_meta_out,
> + .vidioc_querycap = mali_c55_params_querycap,
> + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static const struct v4l2_file_operations mali_c55_params_v4l2_fops = {
> + .owner = THIS_MODULE,
> + .unlocked_ioctl = video_ioctl2,
> + .open = v4l2_fh_open,
> + .release = vb2_fop_release,
> + .poll = vb2_fop_poll,
> + .mmap = vb2_fop_mmap,
> +};
> +
> +static int
> +mali_c55_params_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
> + unsigned int *num_planes, unsigned int sizes[],
> + struct device *alloc_devs[])
> +{
> + if (*num_planes && *num_planes > 1)
> + return -EINVAL;
> +
> + if (sizes[0] && sizes[0] < sizeof(struct mali_c55_params_buffer))
> + return -EINVAL;
> +
> + *num_planes = 1;
> +
> + if (!sizes[0])
> + sizes[0] = sizeof(struct mali_c55_params_buffer);
> +
> + return 0;
> +}
> +
> +static void mali_c55_params_buf_queue(struct vb2_buffer *vb)
> +{
> + struct mali_c55_params *params = vb2_get_drv_priv(vb->vb2_queue);
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct mali_c55_buffer *buf = container_of(vbuf,
> + struct mali_c55_buffer, vb);
> +
> + vb2_set_plane_payload(vb, 0, sizeof(struct mali_c55_params_buffer));
> +
> + spin_lock(¶ms->buffers.lock);
> + list_add_tail(&buf->queue, ¶ms->buffers.queue);
> + spin_unlock(¶ms->buffers.lock);
> +}
> +
> +static int mali_c55_params_start_streaming(struct vb2_queue *q,
> + unsigned int count)
> +{
> + struct mali_c55_params *params = vb2_get_drv_priv(q);
> + struct mali_c55 *mali_c55 = params->mali_c55;
> + int ret;
> +
> + ret = video_device_pipeline_start(¶ms->vdev,
> + ¶ms->mali_c55->pipe);
> + if (ret)
> + return ret;
> +
> + if (mali_c55->pipe.start_count == mali_c55->pipe.required_count) {
> + ret = v4l2_subdev_enable_streams(&mali_c55->isp.sd,
> + MALI_C55_ISP_PAD_SOURCE_VIDEO,
> + BIT(0));
> + if (ret)
> + goto err_stop_pipeline;
> + }
> +
> + return 0;
> +
> +err_stop_pipeline:
> + video_device_pipeline_stop(¶ms->vdev);
When a failure happens you need to return all queued buffers to vb2 in
the QUEUED state.
> +
> + return ret;
> +}
> +
> +static void mali_c55_params_stop_streaming(struct vb2_queue *q)
> +{
> + struct mali_c55_params *params = vb2_get_drv_priv(q);
> + struct mali_c55_buffer *buf, *tmp;
> +
> + spin_lock(¶ms->buffers.lock);
> +
> + list_for_each_entry_safe(buf, tmp, ¶ms->buffers.queue, queue) {
> + list_del(&buf->queue);
> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> + }
This can be turned into a helper function that takes the state as an
argument, to be used in the start_streaming error path. It may even be
possible to share the helper with all video devices in this driver.
> +
> + spin_unlock(¶ms->buffers.lock);
> +
> + video_device_pipeline_stop(¶ms->vdev);
I think you need to stop the ISP too, if this is the first video device
being stopped. You should return buffers to vb2 only after stopping the
ISP.
This may be fixed by stopping the ISP, but currently I think you're
racing with mali_c55_params_write_config()
> +}
> +
> +static const struct vb2_ops mali_c55_params_vb2_ops = {
> + .queue_setup = mali_c55_params_queue_setup,
> + .buf_queue = mali_c55_params_buf_queue,
> + .wait_prepare = vb2_ops_wait_prepare,
> + .wait_finish = vb2_ops_wait_finish,
> + .start_streaming = mali_c55_params_start_streaming,
> + .stop_streaming = mali_c55_params_stop_streaming,
> +};
> +
> +void mali_c55_params_write_config(struct mali_c55 *mali_c55)
> +{
> + struct mali_c55_params *params = &mali_c55->params;
> + enum vb2_buffer_state state = VB2_BUF_STATE_DONE;
> + struct mali_c55_params_buffer *config;
> + struct mali_c55_buffer *buf;
> + size_t block_offset = 0;
> + size_t max_offset;
> +
> + spin_lock(¶ms->buffers.lock);
> +
> + buf = list_first_entry_or_null(¶ms->buffers.queue,
> + struct mali_c55_buffer, queue);
> + if (buf)
> + list_del(&buf->queue);
> + spin_unlock(¶ms->buffers.lock);
> +
> + if (!buf)
> + return;
If this happens we'll lose synchronization. I suppose we can't do much
about this, we really need a request-based API.
> +
> + buf->vb.sequence = mali_c55->isp.frame_sequence;
> + config = vb2_plane_vaddr(&buf->vb.vb2_buf, 0);
> +
> + if (config->total_size > MALI_C55_PARAMS_MAX_SIZE) {
> + dev_dbg(mali_c55->dev, "Invalid parameters buffer size %u\n",
> + config->total_size);
> + state = VB2_BUF_STATE_ERROR;
> + goto err_buffer_done;
> + }
> +
> + max_offset = config->total_size - sizeof(struct mali_c55_params_block_header);
> +
> + /* Walk the list of parameter blocks and process them. */
> + while (block_offset < max_offset) {
> + const struct mali_c55_block_handler *block_handler;
> + union mali_c55_params_block block;
> +
> + block = (union mali_c55_params_block)
> + &config->data[block_offset];
> +
> + if (block.header->type >= MALI_C55_PARAM_BLOCK_SENTINEL) {
> + dev_dbg(mali_c55->dev, "Invalid parameters block type\n");
> + state = VB2_BUF_STATE_ERROR;
> + goto err_buffer_done;
> + }
> +
> + if (block_offset + block.header->size >= config->total_size) {
> + dev_dbg(mali_c55->dev, "Parameters block too large\n");
> + state = VB2_BUF_STATE_ERROR;
> + goto err_buffer_done;
> + }
> +
> + block_handler = &mali_c55_block_handlers[block.header->type];
> + if (block.header->size != block_handler->size) {
> + dev_dbg(mali_c55->dev, "Invalid parameters block size\n");
> + state = VB2_BUF_STATE_ERROR;
> + goto err_buffer_done;
> + }
Most of the validation should be done with the buffer is queued.
Furthermore, you need to make a copy of the data. See the latest version
of the rkisp1 extensible parameters format series. There may be other
recent improvements in that series worth copying here.
> +
> + block_handler->handler(mali_c55, block);
> +
> + block_offset += block.header->size;
> + }
> +
> +err_buffer_done:
> + vb2_buffer_done(&buf->vb.vb2_buf, state);
> +}
> +
> +void mali_c55_unregister_params(struct mali_c55 *mali_c55)
> +{
> + struct mali_c55_params *params = &mali_c55->params;
> +
> + if (!video_is_registered(¶ms->vdev))
> + return;
> +
> + vb2_video_unregister_device(¶ms->vdev);
> + media_entity_cleanup(¶ms->vdev.entity);
> + mutex_destroy(¶ms->lock);
> +}
> +
> +int mali_c55_register_params(struct mali_c55 *mali_c55)
> +{
> + struct mali_c55_params *params = &mali_c55->params;
> + struct video_device *vdev = ¶ms->vdev;
> + struct vb2_queue *vb2q = ¶ms->queue;
> + int ret;
> +
> + mutex_init(¶ms->lock);
> + INIT_LIST_HEAD(¶ms->buffers.queue);
> +
> + params->pad.flags = MEDIA_PAD_FL_SOURCE;
> + ret = media_entity_pads_init(¶ms->vdev.entity, 1, ¶ms->pad);
> + if (ret)
> + goto err_destroy_mutex;
> +
> + vb2q->type = V4L2_BUF_TYPE_META_OUTPUT;
> + vb2q->io_modes = VB2_MMAP | VB2_DMABUF;
> + vb2q->drv_priv = params;
> + vb2q->mem_ops = &vb2_dma_contig_memops;
> + vb2q->ops = &mali_c55_params_vb2_ops;
> + vb2q->buf_struct_size = sizeof(struct mali_c55_buffer);
> + vb2q->min_queued_buffers = 1;
> + vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> + vb2q->lock = ¶ms->lock;
> + vb2q->dev = mali_c55->dev;
> +
> + ret = vb2_queue_init(vb2q);
> + if (ret) {
> + dev_err(mali_c55->dev, "params vb2 queue init failed\n");
> + goto err_cleanup_entity;
> + }
> +
> + strscpy(params->vdev.name, "mali-c55 3a params",
> + sizeof(params->vdev.name));
> + vdev->release = video_device_release_empty;
> + vdev->fops = &mali_c55_params_v4l2_fops;
> + vdev->ioctl_ops = &mali_c55_params_v4l2_ioctl_ops;
> + vdev->lock = ¶ms->lock;
> + vdev->v4l2_dev = &mali_c55->v4l2_dev;
> + vdev->queue = ¶ms->queue;
> + vdev->device_caps = V4L2_CAP_META_OUTPUT | V4L2_CAP_STREAMING;
Shouldn't you set V4L2_CAP_IO_MC and implement media bus code-based
format enumeration ?
> + vdev->vfl_dir = VFL_DIR_TX;
> + video_set_drvdata(vdev, params);
> +
> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> + if (ret) {
> + dev_err(mali_c55->dev,
> + "failed to register params video device\n");
> + goto err_release_vb2q;
> + }
> +
> + params->mali_c55 = mali_c55;
> +
> + return 0;
> +
> +err_release_vb2q:
> + vb2_queue_release(vb2q);
> +err_cleanup_entity:
> + media_entity_cleanup(¶ms->vdev.entity);
> +err_destroy_mutex:
> + mutex_destroy(¶ms->lock);
> +
> + return ret;
> +}
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
> index e72e749b81d5..f2cad402492c 100644
> --- a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
> @@ -159,6 +159,23 @@ enum mali_c55_interrupts {
> #define MALI_C55_BAYER_ORDER_GBRG 2
> #define MALI_C55_BAYER_ORDER_BGGR 3
>
> +#define MALI_C55_REG_METERING_CONFIG 0x18ed0
> +#define MALI_C55_5BIN_HIST_DISABLE_MASK BIT(0)
> +#define MALI_C55_5BIN_HIST_SWITCH_MASK GENMASK(2, 1)
> +#define MALI_C55_5BIN_HIST_SWITCH(x) 1
> +#define MALI_C55_AF_DISABLE_MASK BIT(4)
> +#define MALI_C55_AF_SWITCH_MASK BIT(5)
> +#define MALI_C55_AWB_DISABLE_MASK BIT(8)
> +#define MALI_C55_AWB_SWITCH_MASK BIT(9)
> +#define MALI_C55_AWB_SWITCH(x) ((x) << 9)
> +#define MALI_C55_AEXP_HIST_DISABLE_MASK BIT(12)
> +#define MALI_C55_AEXP_HIST_DISABLE (0x01 << 12)
> +#define MALI_C55_AEXP_HIST_SWITCH_MASK GENMASK(14, 13)
> +#define MALI_C55_AEXP_HIST_SWITCH(x) ((x) << 13)
> +#define MALI_C55_AEXP_IHIST_DISABLE_MASK BIT(16)
> +#define MALI_C55_AEXP_IHIST_DISABLE (0x01 << 12)
> +#define MALI_C55_AEXP_SRC_MASK BIT(24)
> +
> #define MALI_C55_REG_TPG_CH0 0x18ed8
> #define MALI_C55_TEST_PATTERN_ON_OFF BIT(0)
> #define MALI_C55_TEST_PATTERN_RGB_MASK BIT(1)
> @@ -179,6 +196,11 @@ enum mali_c55_interrupts {
> #define MALI_C55_REG_CONFIG_SPACES_OFFSET 0x0ab6c
> #define MALI_C55_CONFIG_SPACE_SIZE 0x1231c
>
> +#define MALI_C55_REG_DIGITAL_GAIN 0x1926c
> +#define MALI_C55_DIGITAL_GAIN_MASK GENMASK(12, 0)
> +#define MALI_C55_REG_DIGITAL_GAIN_OFFSET 0x19270
> +#define MALI_C55_DIGITAL_GAIN_OFFSET_MASK GENMASK(19, 0)
> +
> #define MALI_C55_REG_SINTER_CONFIG 0x19348
> #define MALI_C55_SINTER_VIEW_FILTER_MASK GENMASK(1, 0)
> #define MALI_C55_SINTER_SCALE_MODE_MASK GENMASK(3, 2)
> @@ -187,6 +209,59 @@ enum mali_c55_interrupts {
> #define MALI_C55_SINTER_INT_SELECT_MASK BIT(6)
> #define MALI_C55_SINTER_RM_ENABLE_MASK BIT(7)
>
> +/* Black Level Correction Configuration */
> +#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_00 0x1abcc
> +#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_01 0x1abd0
> +#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_10 0x1abd4
> +#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_11 0x1abd8
> +#define MALI_C55_SENSOR_OFF_PRE_SHA_MASK 0xfffff
> +
> +/* Lens Mesh Shading Configuration */
> +#define MALI_C55_REG_MESH_SHADING_TABLES 0x13074
> +#define MALI_C55_REG_MESH_SHADING_CONFIG 0x1abfc
> +#define MALI_C55_MESH_SHADING_ENABLE_MASK BIT(0)
> +#define MALI_C55_MESH_SHADING_MESH_SHOW_MASK BIT(1)
> +#define MALI_C55_MESH_SHADING_MESH_SHOW(x) ((x) << 1)
> +#define MALI_C55_MESH_SHADING_SCALE_MASK GENMASK(4, 2)
> +#define MALI_C55_MESH_SHADING_SCALE(x) ((x) << 2)
> +#define MALI_C55_MESH_SHADING_PAGE_R_MASK GENMASK(9, 8)
> +#define MALI_C55_MESH_SHADING_PAGE_R(x) ((x) << 8)
> +#define MALI_C55_MESH_SHADING_PAGE_G_MASK GENMASK(11, 10)
> +#define MALI_C55_MESH_SHADING_PAGE_G(x) ((x) << 10)
> +#define MALI_C55_MESH_SHADING_PAGE_B_MASK GENMASK(13, 12)
> +#define MALI_C55_MESH_SHADING_PAGE_B(x) ((x) << 12)
> +#define MALI_C55_MESH_SHADING_MESH_WIDTH_MASK GENMASK(21, 16)
> +#define MALI_C55_MESH_SHADING_MESH_WIDTH(x) ((x) << 16)
> +#define MALI_C55_MESH_SHADING_MESH_HEIGHT_MASK GENMASK(29, 24)
> +#define MALI_C55_MESH_SHADING_MESH_HEIGHT(x) ((x) << 24)
> +
> +#define MALI_C55_REG_MESH_SHADING_ALPHA_BANK 0x1ac04
> +#define MALI_C55_MESH_SHADING_ALPHA_BANK_R_MASK GENMASK(2, 0)
> +#define MALI_C55_MESH_SHADING_ALPHA_BANK_G_MASK GENMASK(5, 3)
> +#define MALI_C55_MESH_SHADING_ALPHA_BANK_G(x) ((x) << 3)
> +#define MALI_C55_MESH_SHADING_ALPHA_BANK_B_MASK GENMASK(8, 6)
> +#define MALI_C55_MESH_SHADING_ALPHA_BANK_B(x) ((x) << 6)
> +#define MALI_C55_REG_MESH_SHADING_ALPHA 0x1ac08
> +#define MALI_C55_MESH_SHADING_ALPHA_R_MASK GENMASK(7, 0)
> +#define MALI_C55_MESH_SHADING_ALPHA_G_MASK GENMASK(15, 8)
> +#define MALI_C55_MESH_SHADING_ALPHA_G(x) ((x) << 8)
> +#define MALI_C55_MESH_SHADING_ALPHA_B_MASK GENMASK(23, 16)
> +#define MALI_C55_MESH_SHADING_ALPHA_B(x) ((x) << 16)
> +#define MALI_C55_REG_MESH_SHADING_MESH_STRENGTH 0x1ac0c
> +#define MALI_c55_MESH_STRENGTH_MASK GENMASK(15, 0)
> +
> +/* AWB Gains Configuration */
> +#define MALI_C55_REG_AWB_GAINS1 0x1ac10
> +#define MALI_C55_AWB_GAIN00_MASK GENMASK(11, 0)
> +#define MALI_C55_AWB_GAIN01_MASK GENMASK(27, 16)
> +#define MALI_C55_AWB_GAIN01(x) ((x) << 16)
> +#define MALI_C55_REG_AWB_GAINS2 0x1ac14
> +#define MALI_C55_AWB_GAIN10_MASK GENMASK(11, 0)
> +#define MALI_C55_AWB_GAIN11_MASK GENMASK(27, 16)
> +#define MALI_C55_AWB_GAIN11(x) ((x) << 16)
> +#define MALI_C55_REG_AWB_GAINS1_AEXP 0x1ac18
> +#define MALI_C55_REG_AWB_GAINS2_AEXP 0x1ac1c
> +
> /* Colour Correction Matrix Configuration */
> #define MALI_C55_REG_CCM_ENABLE 0x1b07c
> #define MALI_C55_CCM_ENABLE_MASK BIT(0)
> @@ -209,6 +284,59 @@ enum mali_c55_interrupts {
> #define MALI_C55_REG_CCM_ANTIFOG_OFFSET_B 0x1b0c8
> #define MALI_C55_CCM_ANTIFOG_OFFSET_MASK GENMASK(11, 0)
>
> +/* AWB Statistics Configuration */
> +#define MALI_C55_REG_AWB_STATS_MODE 0x1b29c
> +#define MALI_C55_AWB_STATS_MODE_MASK BIT(0)
> +#define MALI_C55_REG_AWB_WHITE_LEVEL 0x1b2a0
> +#define MALI_C55_AWB_WHITE_LEVEL_MASK GENMASK(9, 0)
> +#define MALI_C55_REG_AWB_BLACK_LEVEL 0x1b2a4
> +#define MALI_C55_AWB_BLACK_LEVEL_MASK GENMASK(9, 0)
> +#define MALI_C55_REG_AWB_CR_MAX 0x1b2a8
> +#define MALI_C55_AWB_CR_MAX_MASK GENMASK(11, 0)
> +#define MALI_C55_REG_AWB_CR_MIN 0x1b2ac
> +#define MALI_C55_AWB_CR_MIN_MASK GENMASK(11, 0)
> +#define MALI_C55_REG_AWB_CB_MAX 0x1b2b0
> +#define MALI_C55_AWB_CB_MAX_MASK GENMASK(11, 0)
> +#define MALI_C55_REG_AWB_CB_MIN 0x1b2b4
> +#define MALI_C55_AWB_CB_MIN_MASK GENMASK(11, 0)
> +#define MALI_C55_REG_AWB_NODES_USED 0x1b2c4
> +#define MALI_C55_AWB_NODES_USED_HORIZ_MASK GENMASK(7, 0)
> +#define MALI_C55_AWB_NODES_USED_VERT_MASK GENMASK(15, 8)
> +#define MALI_C55_AWB_NODES_USED_VERT(x) ((x) << 8)
> +#define MALI_C55_REG_AWB_CR_HIGH 0x1b2c8
> +#define MALI_C55_AWB_CR_HIGH_MASK GENMASK(11, 0)
> +#define MALI_C55_REG_AWB_CR_LOW 0x1b2cc
> +#define MALI_C55_AWB_CR_LOW_MASK GENMASK(11, 0)
> +#define MALI_C55_REG_AWB_CB_HIGH 0x1b2d0
> +#define MALI_C55_AWB_CB_HIGH_MASK GENMASK(11, 0)
> +#define MALI_C55_REG_AWB_CB_LOW 0x1b2d4
> +#define MALI_C55_AWB_CB_LOW_MASK GENMASK(11, 0)
> +
> +/* AEXP Metering Histogram Configuration */
> +#define MALI_C55_REG_AEXP_HIST_BASE 0x1b730
> +#define MALI_C55_REG_AEXP_IHIST_BASE 0x1bbac
> +#define MALI_C55_AEXP_HIST_SKIP_OFFSET 0
> +#define MALI_C55_AEXP_HIST_SKIP_X_MASK GENMASK(2, 0)
> +#define MALI_c55_AEXP_HIST_SKIP_X(x) 0
> +#define MALI_C55_AEXP_HIST_OFFSET_X_MASK BIT(3)
> +#define MALI_C55_AEXP_HIST_OFFSET_X(x) ((x) << 3)
> +#define MALI_C55_AEXP_HIST_SKIP_Y_MASK GENMASK(6, 4)
> +#define MALI_C55_AEXP_HIST_SKIP_Y(x) ((x) << 4)
> +#define MALI_C55_AEXP_HIST_OFFSET_Y_MASK BIT(7)
> +#define MALI_C55_AEXP_HIST_OFFSET_Y(x) ((x) << 7)
> +#define MALI_C55_AEXP_HIST_SCALE_OFFSET 4
> +#define MALI_C55_AEXP_HIST_SCALE_BOTTOM_MASK GENMASK(3, 0)
> +#define MALI_C55_AEXP_HIST_SCALE_TOP_MASK GENMASK(7, 4)
> +#define MALI_C55_AEXP_HIST_SCALE_TOP(x) ((x) << 4)
> +#define MALI_C55_AEXP_HIST_PLANE_MODE_OFFSET 16
> +#define MALI_C55_AEXP_HIST_PLANE_MODE_MASK GENMASK(2, 0)
> +#define MALI_C55_AEXP_HIST_NODES_USED_OFFSET 52
> +#define MALI_C55_AEXP_HIST_NODES_USED_HORIZ_MASK GENMASK(7, 0)
> +#define MALI_C55_AEXP_HIST_NODES_USED_VERT_MASK GENMASK(15, 8)
> +#define MALI_C55_AEXP_HIST_NODES_USED_VERT(x) ((x) << 8)
> +#define MALI_C55_AEXP_HIST_ZONE_WEIGHTS_OFFSET 56
> +#define MALI_C55_AEXP_HIST_ZONE_WEIGHT_MASK 0x0f0f0f0f
> +
> /*
> * The Mali-C55 ISP has up to two output pipes; known as full resolution and
> * down scaled. The register space for these is laid out identically, but offset
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 17/18] media: platform: Add mali-c55 parameters video node
2024-07-30 22:16 ` Laurent Pinchart
@ 2024-07-31 6:53 ` Jacopo Mondi
2024-07-31 7:02 ` Laurent Pinchart
2024-07-31 15:12 ` Dan Scally
1 sibling, 1 reply; 41+ messages in thread
From: Jacopo Mondi @ 2024-07-31 6:53 UTC (permalink / raw)
To: Laurent Pinchart
Cc: Daniel Scally, linux-media, devicetree, linux-arm-kernel,
jacopo.mondi, nayden.kanchev, robh+dt, mchehab,
krzysztof.kozlowski+dt, conor+dt, jerome.forissier,
kieran.bingham, sakari.ailus
Hi Laurent
On Wed, Jul 31, 2024 at 01:16:16AM GMT, Laurent Pinchart wrote:
> Hi Dan,
>
> Thank you for the patch.
>
> On Tue, Jul 09, 2024 at 02:29:05PM +0100, Daniel Scally wrote:
> > Add a new code file to the mali-c55 driver that registers an output
> > video node for userspace to queue buffers of parameters to. Handlers
> > are included to program the statistics generation plus the white
> > balance, black level correction and mesh shading correction blocks.
> >
> > Update the rest of the driver to register and link the new video node
> >
> > Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
> > Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
> > ---
> > Changes in v6:
> >
> > - Used a union to generalise the block pointer rather than resorting to
> > casting everywhere - fantastic idea Sakari, this made it much cleaner.
> > - Reworked the loop in mali_c55_params_write_config() so that we can be
> > sure there's remaining space for the next block header.
> >
> > Changes in v5:
> >
> > - New patch
> >
> > drivers/media/platform/arm/mali-c55/Makefile | 1 +
> > .../platform/arm/mali-c55/mali-c55-common.h | 20 +
> > .../platform/arm/mali-c55/mali-c55-core.c | 23 +
> > .../platform/arm/mali-c55/mali-c55-isp.c | 21 +-
> > .../platform/arm/mali-c55/mali-c55-params.c | 671 ++++++++++++++++++
> > .../arm/mali-c55/mali-c55-registers.h | 128 ++++
> > 6 files changed, 863 insertions(+), 1 deletion(-)
> > create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-params.c
> >
> > diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile
> > index b5a22d414479..d5718b0b23e0 100644
> > --- a/drivers/media/platform/arm/mali-c55/Makefile
> > +++ b/drivers/media/platform/arm/mali-c55/Makefile
> > @@ -3,6 +3,7 @@
> > mali-c55-y := mali-c55-capture.o \
> > mali-c55-core.o \
> > mali-c55-isp.o \
> > + mali-c55-params.o \
> > mali-c55-resizer.o \
> > mali-c55-stats.o \
> > mali-c55-tpg.o
> > diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> > index 136c785c68ba..66a46a7c0547 100644
> > --- a/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> > +++ b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> > @@ -50,6 +50,7 @@ enum mali_c55_isp_pads {
> > MALI_C55_ISP_PAD_SOURCE_VIDEO,
> > MALI_C55_ISP_PAD_SOURCE_BYPASS,
> > MALI_C55_ISP_PAD_SOURCE_STATS,
> > + MALI_C55_ISP_PAD_SINK_PARAMS,
> > MALI_C55_ISP_NUM_PADS,
> > };
> >
> > @@ -184,6 +185,21 @@ struct mali_c55_stats {
> > } buffers;
> > };
> >
> > +struct mali_c55_params {
> > + struct mali_c55 *mali_c55;
> > + struct video_device vdev;
> > + struct vb2_queue queue;
> > + struct media_pad pad;
> > + /* Mutex to provide to vb2 */
> > + struct mutex lock;
> > +
> > + struct {
> > + /* Spinlock to guard buffer queue */
> > + spinlock_t lock;
> > + struct list_head queue;
> > + } buffers;
> > +};
> > +
> > enum mali_c55_config_spaces {
> > MALI_C55_CONFIG_PING,
> > MALI_C55_CONFIG_PONG,
> > @@ -226,6 +242,7 @@ struct mali_c55 {
> > struct mali_c55_isp isp;
> > struct mali_c55_resizer resizers[MALI_C55_NUM_RSZS];
> > struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS];
> > + struct mali_c55_params params;
> > struct mali_c55_stats stats;
> >
> > struct mali_c55_context context;
> > @@ -256,6 +273,8 @@ int mali_c55_register_capture_devs(struct mali_c55 *mali_c55);
> > void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55);
> > int mali_c55_register_stats(struct mali_c55 *mali_c55);
> > void mali_c55_unregister_stats(struct mali_c55 *mali_c55);
> > +int mali_c55_register_params(struct mali_c55 *mali_c55);
> > +void mali_c55_unregister_params(struct mali_c55 *mali_c55);
> > struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55);
> > void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
> > enum mali_c55_planes plane);
> > @@ -272,5 +291,6 @@ const struct mali_c55_isp_fmt *
> > mali_c55_isp_get_mbus_config_by_index(u32 index);
> > void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
> > enum mali_c55_config_spaces cfg_space);
> > +void mali_c55_params_write_config(struct mali_c55 *mali_c55);
> >
> > #endif /* _MALI_C55_COMMON_H */
> > diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> > index ed0db34767a4..55b3cbf53791 100644
> > --- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> > +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> > @@ -384,6 +384,16 @@ static int mali_c55_create_links(struct mali_c55 *mali_c55)
> > goto err_remove_links;
> > }
> >
> > + ret = media_create_pad_link(&mali_c55->params.vdev.entity, 0,
> > + &mali_c55->isp.sd.entity,
> > + MALI_C55_ISP_PAD_SINK_PARAMS,
> > + MEDIA_LNK_FL_ENABLED);
>
> Should this be immutable, or do you think it makes sense to support
> operating the ISP without parameters ? I know we did so when developing
> the driver to test the initial code, but are there real use cases now ?
>
> > + if (ret) {
> > + dev_err(mali_c55->dev,
> > + "failed to link ISP and parameters video node\n");
> > + goto err_remove_links;
> > + }
> > +
> > return 0;
> >
> > err_remove_links:
> > @@ -398,6 +408,7 @@ static void mali_c55_unregister_entities(struct mali_c55 *mali_c55)
> > mali_c55_unregister_isp(mali_c55);
> > mali_c55_unregister_resizers(mali_c55);
> > mali_c55_unregister_capture_devs(mali_c55);
> > + mali_c55_unregister_params(mali_c55);
> > mali_c55_unregister_stats(mali_c55);
> > }
> >
> > @@ -421,6 +432,10 @@ static int mali_c55_register_entities(struct mali_c55 *mali_c55)
> > if (ret)
> > goto err_unregister_entities;
> >
> > + ret = mali_c55_register_params(mali_c55);
> > + if (ret)
> > + goto err_unregister_entities;
> > +
> > ret = mali_c55_register_stats(mali_c55);
> > if (ret)
> > goto err_unregister_entities;
> > @@ -620,6 +635,14 @@ static irqreturn_t mali_c55_isr(int irq, void *context)
> > curr_config >>= ffs(MALI_C55_REG_PING_PONG_READ_MASK) - 1;
> > next_config = curr_config ^ 1;
> >
> > + /*
> > + * Write the configuration parameters received from
> > + * userspace into the configuration buffer, which will
> > + * be transferred to the 'next' active config space at
> > + * by mali_c55_swap_next_config().
> > + */
> > + mali_c55_params_write_config(mali_c55);
> > +
> > /*
> > * The ordering of these two is currently important as
> > * mali_c55_stats_fill_buffer() is asynchronous whereas
> > diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> > index 2f450c00300a..40d7ef6eda22 100644
> > --- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> > +++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> > @@ -132,6 +132,7 @@ static int mali_c55_isp_start(struct mali_c55 *mali_c55)
> > cfg->bypass ? MALI_C55_ISP_RAW_BYPASS_BYPASS_MASK :
> > 0x00);
> >
> > + mali_c55_params_write_config(mali_c55);
> > ret = mali_c55_config_write(ctx, MALI_C55_CONFIG_PING);
> > if (ret) {
> > dev_err(mali_c55->dev, "failed to write ISP config\n");
> > @@ -464,12 +465,17 @@ static int mali_c55_isp_init_state(struct v4l2_subdev *sd,
> >
> > src_fmt = v4l2_subdev_state_get_format(state,
> > MALI_C55_ISP_PAD_SOURCE_STATS);
> > + sink_fmt = v4l2_subdev_state_get_format(state,
> > + MALI_C55_ISP_PAD_SINK_PARAMS);
> >
> > src_fmt->width = sizeof(struct mali_c55_stats_buffer);
> > src_fmt->height = 1;
> > src_fmt->field = V4L2_FIELD_NONE;
> > src_fmt->code = MEDIA_BUS_FMT_METADATA_FIXED;
> >
> > + *sink_fmt = *src_fmt;
>
> I would initialize the fields individually, I think the code would be
> clearer.
>
> > + sink_fmt->width = sizeof(struct mali_c55_params_buffer);
> > +
> > return 0;
> > }
> >
> > @@ -477,8 +483,20 @@ static const struct v4l2_subdev_internal_ops mali_c55_isp_internal_ops = {
> > .init_state = mali_c55_isp_init_state,
> > };
> >
> > +static int mali_c55_subdev_link_validate(struct media_link *link)
> > +{
> > + /*
> > + * Skip validation for the parameters sink pad, as the source is not
> > + * a subdevice.
> > + */
> > + if (link->sink->index == MALI_C55_ISP_PAD_SINK_PARAMS)
> > + return 0;
> > +
> > + return v4l2_subdev_link_validate(link);
> > +}
> > +
> > static const struct media_entity_operations mali_c55_isp_media_ops = {
> > - .link_validate = v4l2_subdev_link_validate,
> > + .link_validate = mali_c55_subdev_link_validate,
> > };
> >
> > int mali_c55_register_isp(struct mali_c55 *mali_c55)
> > @@ -501,6 +519,7 @@ int mali_c55_register_isp(struct mali_c55 *mali_c55)
> > isp->pads[MALI_C55_ISP_PAD_SOURCE_VIDEO].flags = MEDIA_PAD_FL_SOURCE;
> > isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE;
> > isp->pads[MALI_C55_ISP_PAD_SOURCE_STATS].flags = MEDIA_PAD_FL_SOURCE;
> > + isp->pads[MALI_C55_ISP_PAD_SINK_PARAMS].flags = MEDIA_PAD_FL_SINK;
> >
> > ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS,
> > isp->pads);
> > diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-params.c b/drivers/media/platform/arm/mali-c55/mali-c55-params.c
> > new file mode 100644
> > index 000000000000..c0ca4a759653
> > --- /dev/null
> > +++ b/drivers/media/platform/arm/mali-c55/mali-c55-params.c
> > @@ -0,0 +1,671 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * ARM Mali-C55 ISP Driver - Configuration parameters output device
> > + *
> > + * Copyright (C) 2024 Ideas on Board Oy
> > + */
> > +#include <linux/media/arm/mali-c55-config.h>
> > +
> > +#include <media/media-entity.h>
> > +#include <media/v4l2-dev.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-fh.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mali-c55-common.h"
> > +#include "mali-c55-registers.h"
> > +
> > +typedef void (*mali_c55_block_handler)(struct mali_c55 *mali_c55,
> > + union mali_c55_params_block block);
> > +
> > +struct mali_c55_block_handler {
> > + size_t size;
> > + mali_c55_block_handler handler;
> > +};
> > +
> > +static void mali_c55_params_sensor_offs(struct mali_c55 *mali_c55,
> > + union mali_c55_params_block block)
> > +{
> > + struct mali_c55_params_sensor_off_preshading *p = block.sensor_offs;
> > + __u32 global_offset;
> > +
> > + if (!block.header->enabled)
> > + return;
> > +
> > + if (!(p->chan00 || p->chan01 || p->chan10 || p->chan11))
> > + return;
> > +
> > + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_00,
> > + p->chan00 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
> > + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_01,
> > + p->chan01 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
> > + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_10,
> > + p->chan10 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
> > + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_11,
> > + p->chan11 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
> > +
> > + /*
> > + * The average offset is applied as a global offset for the digital
> > + * gain block
> > + */
> > + global_offset = (p->chan00 + p->chan01 + p->chan10 + p->chan11) >> 2;
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_DIGITAL_GAIN_OFFSET,
> > + MALI_C55_DIGITAL_GAIN_OFFSET_MASK,
> > + global_offset);
> > +
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BYPASS_3,
> > + MALI_C55_REG_BYPASS_3_SENSOR_OFFSET_PRE_SH,
> > + 0x00);
> > +}
> > +
> > +static void mali_c55_params_aexp_hist(struct mali_c55 *mali_c55,
> > + union mali_c55_params_block block)
> > +{
> > + u32 disable_mask;
> > + u32 disable_val;
> > + u32 base;
> > +
> > + if (block.header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST) {
> > + disable_mask = MALI_C55_AEXP_HIST_DISABLE_MASK;
> > + disable_val = MALI_C55_AEXP_HIST_DISABLE;
> > + base = MALI_C55_REG_AEXP_HIST_BASE;
> > + } else {
> > + disable_mask = MALI_C55_AEXP_IHIST_DISABLE_MASK;
> > + disable_val = MALI_C55_AEXP_IHIST_DISABLE;
> > + base = MALI_C55_REG_AEXP_IHIST_BASE;
> > + }
> > + struct mali_c55_params_aexp_hist *params = block.aexp_hist;
> > +
> > + if (!block.header->enabled) {
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> > + disable_mask, disable_val);
> > + return;
> > + }
> > +
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> > + disable_mask, false);
> > +
> > + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
> > + MALI_C55_AEXP_HIST_SKIP_X_MASK, params->skip_x);
> > + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
> > + MALI_C55_AEXP_HIST_OFFSET_X_MASK,
> > + MALI_C55_AEXP_HIST_OFFSET_X(params->offset_x));
> > + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
> > + MALI_C55_AEXP_HIST_SKIP_Y_MASK,
> > + MALI_C55_AEXP_HIST_SKIP_Y(params->skip_y));
> > + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
> > + MALI_C55_AEXP_HIST_OFFSET_Y_MASK,
> > + MALI_C55_AEXP_HIST_OFFSET_Y(params->offset_y));
> > +
> > + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SCALE_OFFSET,
> > + MALI_C55_AEXP_HIST_SCALE_BOTTOM_MASK,
> > + params->scale_bottom);
> > + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SCALE_OFFSET,
> > + MALI_C55_AEXP_HIST_SCALE_TOP_MASK,
> > + MALI_C55_AEXP_HIST_SCALE_TOP(params->scale_top));
> > +
> > + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_PLANE_MODE_OFFSET,
> > + MALI_C55_AEXP_HIST_PLANE_MODE_MASK,
> > + params->plane_mode);
> > +
> > + if (block.header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST)
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> > + MALI_C55_AEXP_HIST_SWITCH_MASK,
> > + MALI_C55_AEXP_HIST_SWITCH(params->tap_point));
> > +}
> > +
> > +static void
> > +mali_c55_params_aexp_hist_weights(struct mali_c55 *mali_c55,
> > + union mali_c55_params_block block)
> > +{
> > + struct mali_c55_params_aexp_weights *params = block.aexp_weights;
> > + u32 base;
> > +
> > + if (!block.header->enabled)
> > + return;
> > +
> > + base = block.header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS ?
> > + MALI_C55_REG_AEXP_HIST_BASE :
> > + MALI_C55_REG_AEXP_IHIST_BASE;
> > +
> > + mali_c55_ctx_update_bits(mali_c55,
> > + base + MALI_C55_AEXP_HIST_NODES_USED_OFFSET,
> > + MALI_C55_AEXP_HIST_NODES_USED_HORIZ_MASK,
> > + params->nodes_used_horiz);
> > + mali_c55_ctx_update_bits(mali_c55,
> > + base + MALI_C55_AEXP_HIST_NODES_USED_OFFSET,
> > + MALI_C55_AEXP_HIST_NODES_USED_VERT_MASK,
> > + MALI_C55_AEXP_HIST_NODES_USED_VERT(params->nodes_used_vert));
> > +
> > + /*
> > + * The zone weights array is a 225-element array of u8 values, but that
> > + * is a bit annoying to handle given the ISP expects 32-bit writes. We
> > + * just reinterpret it as a 57-element array of 32-bit values for the
> > + * purposes of this transaction (the 3 bytes of additional space at the
> > + * end of the write is just padding for the array of weights in the ISP
> > + * memory space anyway, so there's no risk of overwriting other
> > + * registers).
> > + */
> > + for (unsigned int i = 0; i < 57; i++) {
> > + u32 val = ((u32 *)params->zone_weights)[i]
> > + & MALI_C55_AEXP_HIST_ZONE_WEIGHT_MASK;
> > + u32 addr = base + MALI_C55_AEXP_HIST_ZONE_WEIGHTS_OFFSET + (4 * i);
> > +
> > + mali_c55_ctx_write(mali_c55, addr, val);
> > + }
> > +}
> > +
> > +static void mali_c55_params_digital_gain(struct mali_c55 *mali_c55,
> > + union mali_c55_params_block block)
> > +{
> > + struct mali_c55_params_digital_gain *dgain = block.digital_gain;
> > +
> > + /*
> > + * If the block is flagged as disabled we write a gain of 1.0, which in
> > + * Q5.8 format is 256.
> > + */
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_DIGITAL_GAIN,
> > + MALI_C55_DIGITAL_GAIN_MASK,
> > + block.header->enabled ? dgain->gain : 256);
> > +}
> > +
> > +static void mali_c55_params_awb_gains(struct mali_c55 *mali_c55,
> > + union mali_c55_params_block block)
> > +{
> > + struct mali_c55_params_awb_gains *gains = block.awb_gains;
> > +
> > + /*
> > + * There are two places AWB gains can be set in the ISP; one affects the
> > + * image output data and the other affects the statistics for the
> > + * AEXP-0 tap point.
> > + */
> > + u32 addr1 = block.header->type == MALI_C55_PARAM_BLOCK_AWB_GAINS ?
> > + MALI_C55_REG_AWB_GAINS1 :
> > + MALI_C55_REG_AWB_GAINS1_AEXP;
> > + u32 addr2 = block.header->type == MALI_C55_PARAM_BLOCK_AWB_GAINS ?
> > + MALI_C55_REG_AWB_GAINS2 :
> > + MALI_C55_REG_AWB_GAINS2_AEXP;
> > +
> > + mali_c55_ctx_update_bits(mali_c55, addr1, MALI_C55_AWB_GAIN00_MASK,
> > + gains->gain00);
> > + mali_c55_ctx_update_bits(mali_c55, addr1, MALI_C55_AWB_GAIN01_MASK,
> > + MALI_C55_AWB_GAIN01(gains->gain01));
> > + mali_c55_ctx_update_bits(mali_c55, addr2, MALI_C55_AWB_GAIN10_MASK,
> > + gains->gain10);
> > + mali_c55_ctx_update_bits(mali_c55, addr2, MALI_C55_AWB_GAIN11_MASK,
> > + MALI_C55_AWB_GAIN11(gains->gain11));
> > +}
> > +
> > +static void mali_c55_params_awb_config(struct mali_c55 *mali_c55,
> > + union mali_c55_params_block block)
> > +{
> > + struct mali_c55_params_awb_config *params = block.awb_config;
> > +
> > + if (!block.header->enabled) {
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> > + MALI_C55_AWB_DISABLE_MASK,
> > + MALI_C55_AWB_DISABLE_MASK);
> > + return;
> > + }
> > +
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> > + MALI_C55_AWB_DISABLE_MASK, false);
> > +
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_STATS_MODE,
> > + MALI_C55_AWB_STATS_MODE_MASK, params->stats_mode);
> > +
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_WHITE_LEVEL,
> > + MALI_C55_AWB_WHITE_LEVEL_MASK, params->white_level);
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_BLACK_LEVEL,
> > + MALI_C55_AWB_BLACK_LEVEL_MASK, params->black_level);
> > +
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_MAX,
> > + MALI_C55_AWB_CR_MAX_MASK, params->cr_max);
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_MIN,
> > + MALI_C55_AWB_CR_MIN_MASK, params->cr_min);
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_MAX,
> > + MALI_C55_AWB_CB_MAX_MASK, params->cb_max);
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_MIN,
> > + MALI_C55_AWB_CB_MIN_MASK, params->cb_min);
> > +
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_NODES_USED,
> > + MALI_C55_AWB_NODES_USED_HORIZ_MASK,
> > + params->nodes_used_horiz);
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_NODES_USED,
> > + MALI_C55_AWB_NODES_USED_VERT_MASK,
> > + MALI_C55_AWB_NODES_USED_VERT(params->nodes_used_vert));
> > +
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_HIGH,
> > + MALI_C55_AWB_CR_HIGH_MASK, params->cr_high);
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_LOW,
> > + MALI_C55_AWB_CR_LOW_MASK, params->cr_low);
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_HIGH,
> > + MALI_C55_AWB_CB_HIGH_MASK, params->cb_high);
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_LOW,
> > + MALI_C55_AWB_CB_LOW_MASK, params->cb_low);
> > +
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> > + MALI_C55_AWB_SWITCH_MASK,
> > + MALI_C55_AWB_SWITCH(params->tap_point));
> > +}
> > +
> > +static void mali_c55_params_lsc_config(struct mali_c55 *mali_c55,
> > + union mali_c55_params_block block)
> > +{
> > + struct mali_c55_params_mesh_shading_config *params = block.shading_config;
> > + unsigned int i;
> > + u32 addr;
> > +
> > + if (!block.header->enabled) {
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > + MALI_C55_MESH_SHADING_ENABLE_MASK,
> > + false);
> > + return;
> > + }
> > +
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > + MALI_C55_MESH_SHADING_ENABLE_MASK, true);
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > + MALI_C55_MESH_SHADING_MESH_SHOW_MASK,
> > + MALI_C55_MESH_SHADING_MESH_SHOW(params->mesh_show));
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > + MALI_C55_MESH_SHADING_SCALE_MASK,
> > + MALI_C55_MESH_SHADING_SCALE(params->mesh_scale));
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > + MALI_C55_MESH_SHADING_PAGE_R_MASK,
> > + MALI_C55_MESH_SHADING_PAGE_R(params->mesh_page_r));
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > + MALI_C55_MESH_SHADING_PAGE_G_MASK,
> > + MALI_C55_MESH_SHADING_PAGE_G(params->mesh_page_g));
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > + MALI_C55_MESH_SHADING_PAGE_B_MASK,
> > + MALI_C55_MESH_SHADING_PAGE_B(params->mesh_page_b));
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > + MALI_C55_MESH_SHADING_MESH_WIDTH_MASK,
> > + MALI_C55_MESH_SHADING_MESH_WIDTH(params->mesh_width));
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > + MALI_C55_MESH_SHADING_MESH_HEIGHT_MASK,
> > + MALI_C55_MESH_SHADING_MESH_HEIGHT(params->mesh_height));
> > +
> > + for (i = 0; i < MALI_C55_NUM_MESH_SHADING_ELEMENTS; i++) {
> > + addr = MALI_C55_REG_MESH_SHADING_TABLES + (i * 4);
> > + mali_c55_ctx_write(mali_c55, addr, params->mesh[i]);
> > + }
> > +}
> > +
> > +static void mali_c55_params_lsc_selection(struct mali_c55 *mali_c55,
> > + union mali_c55_params_block block)
> > +{
> > + struct mali_c55_params_mesh_shading_selection *params =
> > + block.shading_selection;
> > +
> > + if (!block.header->enabled)
> > + return;
> > +
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK,
> > + MALI_C55_MESH_SHADING_ALPHA_BANK_R_MASK,
> > + params->mesh_alpha_bank_r);
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK,
> > + MALI_C55_MESH_SHADING_ALPHA_BANK_G_MASK,
> > + MALI_C55_MESH_SHADING_ALPHA_BANK_G(params->mesh_alpha_bank_g));
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK,
> > + MALI_C55_MESH_SHADING_ALPHA_BANK_B_MASK,
> > + MALI_C55_MESH_SHADING_ALPHA_BANK_B(params->mesh_alpha_bank_b));
> > +
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA,
> > + MALI_C55_MESH_SHADING_ALPHA_R_MASK,
> > + params->mesh_alpha_r);
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA,
> > + MALI_C55_MESH_SHADING_ALPHA_G_MASK,
> > + MALI_C55_MESH_SHADING_ALPHA_G(params->mesh_alpha_g));
> > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA,
> > + MALI_C55_MESH_SHADING_ALPHA_B_MASK,
> > + MALI_C55_MESH_SHADING_ALPHA_B(params->mesh_alpha_b));
> > +
> > + mali_c55_ctx_update_bits(mali_c55,
> > + MALI_C55_REG_MESH_SHADING_MESH_STRENGTH,
> > + MALI_c55_MESH_STRENGTH_MASK,
> > + params->mesh_strength);
> > +}
> > +
> > +static const struct mali_c55_block_handler mali_c55_block_handlers[] = {
> > + [MALI_C55_PARAM_BLOCK_SENSOR_OFFS] = {
> > + .size = sizeof(struct mali_c55_params_sensor_off_preshading),
> > + .handler = &mali_c55_params_sensor_offs,
> > + },
> > + [MALI_C55_PARAM_BLOCK_AEXP_HIST] = {
> > + .size = sizeof(struct mali_c55_params_aexp_hist),
> > + .handler = &mali_c55_params_aexp_hist,
> > + },
> > + [MALI_C55_PARAM_BLOCK_AEXP_IHIST] = {
> > + .size = sizeof(struct mali_c55_params_aexp_hist),
> > + .handler = &mali_c55_params_aexp_hist,
> > + },
> > + [MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS] = {
> > + .size = sizeof(struct mali_c55_params_aexp_weights),
> > + .handler = &mali_c55_params_aexp_hist_weights,
> > + },
> > + [MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS] = {
> > + .size = sizeof(struct mali_c55_params_aexp_weights),
> > + .handler = &mali_c55_params_aexp_hist_weights,
> > + },
> > + [MALI_C55_PARAM_BLOCK_DIGITAL_GAIN] = {
> > + .size = sizeof(struct mali_c55_params_digital_gain),
> > + .handler = &mali_c55_params_digital_gain,
> > + },
> > + [MALI_C55_PARAM_BLOCK_AWB_GAINS] = {
> > + .size = sizeof(struct mali_c55_params_awb_gains),
> > + .handler = &mali_c55_params_awb_gains,
> > + },
> > + [MALI_C55_PARAM_BLOCK_AWB_CONFIG] = {
> > + .size = sizeof(struct mali_c55_params_awb_config),
> > + .handler = &mali_c55_params_awb_config,
> > + },
> > + [MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP] = {
> > + .size = sizeof(struct mali_c55_params_awb_gains),
> > + .handler = &mali_c55_params_awb_gains,
> > + },
> > + [MALI_C55_PARAM_MESH_SHADING_CONFIG] = {
> > + .size = sizeof(struct mali_c55_params_mesh_shading_config),
> > + .handler = &mali_c55_params_lsc_config,
> > + },
> > + [MALI_C55_PARAM_MESH_SHADING_SELECTION] = {
> > + .size = sizeof(struct mali_c55_params_mesh_shading_selection),
> > + .handler = &mali_c55_params_lsc_selection,
> > + },
> > +};
> > +
> > +static int mali_c55_params_enum_fmt_meta_out(struct file *file, void *fh,
> > + struct v4l2_fmtdesc *f)
> > +{
> > + if (f->index)
> > + return -EINVAL;
> > +
> > + f->pixelformat = V4L2_META_FMT_MALI_C55_PARAMS;
> > +
> > + return 0;
> > +}
> > +
> > +static int mali_c55_params_g_fmt_meta_out(struct file *file, void *fh,
> > + struct v4l2_format *f)
> > +{
> > + static const struct v4l2_meta_format mfmt = {
> > + .dataformat = V4L2_META_FMT_MALI_C55_PARAMS,
> > + .buffersize = sizeof(struct mali_c55_params_buffer),
> > + };
> > +
> > + f->fmt.meta = mfmt;
> > +
> > + return 0;
> > +}
> > +
> > +static int mali_c55_params_querycap(struct file *file,
> > + void *priv, struct v4l2_capability *cap)
> > +{
> > + strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver));
> > + strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card));
> > +
> > + return 0;
> > +}
> > +
> > +static const struct v4l2_ioctl_ops mali_c55_params_v4l2_ioctl_ops = {
> > + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> > + .vidioc_querybuf = vb2_ioctl_querybuf,
> > + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> > + .vidioc_qbuf = vb2_ioctl_qbuf,
> > + .vidioc_expbuf = vb2_ioctl_expbuf,
> > + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> > + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > + .vidioc_streamon = vb2_ioctl_streamon,
> > + .vidioc_streamoff = vb2_ioctl_streamoff,
> > + .vidioc_enum_fmt_meta_out = mali_c55_params_enum_fmt_meta_out,
> > + .vidioc_g_fmt_meta_out = mali_c55_params_g_fmt_meta_out,
> > + .vidioc_s_fmt_meta_out = mali_c55_params_g_fmt_meta_out,
> > + .vidioc_try_fmt_meta_out = mali_c55_params_g_fmt_meta_out,
> > + .vidioc_querycap = mali_c55_params_querycap,
> > + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> > + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_file_operations mali_c55_params_v4l2_fops = {
> > + .owner = THIS_MODULE,
> > + .unlocked_ioctl = video_ioctl2,
> > + .open = v4l2_fh_open,
> > + .release = vb2_fop_release,
> > + .poll = vb2_fop_poll,
> > + .mmap = vb2_fop_mmap,
> > +};
> > +
> > +static int
> > +mali_c55_params_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
> > + unsigned int *num_planes, unsigned int sizes[],
> > + struct device *alloc_devs[])
> > +{
> > + if (*num_planes && *num_planes > 1)
> > + return -EINVAL;
> > +
> > + if (sizes[0] && sizes[0] < sizeof(struct mali_c55_params_buffer))
> > + return -EINVAL;
> > +
> > + *num_planes = 1;
> > +
> > + if (!sizes[0])
> > + sizes[0] = sizeof(struct mali_c55_params_buffer);
> > +
> > + return 0;
> > +}
> > +
> > +static void mali_c55_params_buf_queue(struct vb2_buffer *vb)
> > +{
> > + struct mali_c55_params *params = vb2_get_drv_priv(vb->vb2_queue);
> > + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > + struct mali_c55_buffer *buf = container_of(vbuf,
> > + struct mali_c55_buffer, vb);
> > +
> > + vb2_set_plane_payload(vb, 0, sizeof(struct mali_c55_params_buffer));
> > +
> > + spin_lock(¶ms->buffers.lock);
> > + list_add_tail(&buf->queue, ¶ms->buffers.queue);
> > + spin_unlock(¶ms->buffers.lock);
> > +}
> > +
> > +static int mali_c55_params_start_streaming(struct vb2_queue *q,
> > + unsigned int count)
> > +{
> > + struct mali_c55_params *params = vb2_get_drv_priv(q);
> > + struct mali_c55 *mali_c55 = params->mali_c55;
> > + int ret;
> > +
> > + ret = video_device_pipeline_start(¶ms->vdev,
> > + ¶ms->mali_c55->pipe);
> > + if (ret)
> > + return ret;
> > +
> > + if (mali_c55->pipe.start_count == mali_c55->pipe.required_count) {
> > + ret = v4l2_subdev_enable_streams(&mali_c55->isp.sd,
> > + MALI_C55_ISP_PAD_SOURCE_VIDEO,
> > + BIT(0));
> > + if (ret)
> > + goto err_stop_pipeline;
> > + }
> > +
> > + return 0;
> > +
> > +err_stop_pipeline:
> > + video_device_pipeline_stop(¶ms->vdev);
>
> When a failure happens you need to return all queued buffers to vb2 in
> the QUEUED state.
>
> > +
> > + return ret;
> > +}
> > +
> > +static void mali_c55_params_stop_streaming(struct vb2_queue *q)
> > +{
> > + struct mali_c55_params *params = vb2_get_drv_priv(q);
> > + struct mali_c55_buffer *buf, *tmp;
> > +
> > + spin_lock(¶ms->buffers.lock);
> > +
> > + list_for_each_entry_safe(buf, tmp, ¶ms->buffers.queue, queue) {
> > + list_del(&buf->queue);
> > + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> > + }
>
> This can be turned into a helper function that takes the state as an
> argument, to be used in the start_streaming error path. It may even be
> possible to share the helper with all video devices in this driver.
>
> > +
> > + spin_unlock(¶ms->buffers.lock);
> > +
> > + video_device_pipeline_stop(¶ms->vdev);
>
> I think you need to stop the ISP too, if this is the first video device
> being stopped. You should return buffers to vb2 only after stopping the
> ISP.
>
> This may be fixed by stopping the ISP, but currently I think you're
> racing with mali_c55_params_write_config()
>
> > +}
> > +
> > +static const struct vb2_ops mali_c55_params_vb2_ops = {
> > + .queue_setup = mali_c55_params_queue_setup,
> > + .buf_queue = mali_c55_params_buf_queue,
> > + .wait_prepare = vb2_ops_wait_prepare,
> > + .wait_finish = vb2_ops_wait_finish,
> > + .start_streaming = mali_c55_params_start_streaming,
> > + .stop_streaming = mali_c55_params_stop_streaming,
> > +};
> > +
> > +void mali_c55_params_write_config(struct mali_c55 *mali_c55)
> > +{
> > + struct mali_c55_params *params = &mali_c55->params;
> > + enum vb2_buffer_state state = VB2_BUF_STATE_DONE;
> > + struct mali_c55_params_buffer *config;
> > + struct mali_c55_buffer *buf;
> > + size_t block_offset = 0;
> > + size_t max_offset;
> > +
> > + spin_lock(¶ms->buffers.lock);
> > +
> > + buf = list_first_entry_or_null(¶ms->buffers.queue,
> > + struct mali_c55_buffer, queue);
> > + if (buf)
> > + list_del(&buf->queue);
> > + spin_unlock(¶ms->buffers.lock);
> > +
> > + if (!buf)
> > + return;
>
> If this happens we'll lose synchronization. I suppose we can't do much
> about this, we really need a request-based API.
>
> > +
> > + buf->vb.sequence = mali_c55->isp.frame_sequence;
> > + config = vb2_plane_vaddr(&buf->vb.vb2_buf, 0);
> > +
> > + if (config->total_size > MALI_C55_PARAMS_MAX_SIZE) {
> > + dev_dbg(mali_c55->dev, "Invalid parameters buffer size %u\n",
> > + config->total_size);
> > + state = VB2_BUF_STATE_ERROR;
> > + goto err_buffer_done;
> > + }
> > +
> > + max_offset = config->total_size - sizeof(struct mali_c55_params_block_header);
> > +
> > + /* Walk the list of parameter blocks and process them. */
> > + while (block_offset < max_offset) {
> > + const struct mali_c55_block_handler *block_handler;
> > + union mali_c55_params_block block;
> > +
> > + block = (union mali_c55_params_block)
> > + &config->data[block_offset];
> > +
> > + if (block.header->type >= MALI_C55_PARAM_BLOCK_SENTINEL) {
> > + dev_dbg(mali_c55->dev, "Invalid parameters block type\n");
> > + state = VB2_BUF_STATE_ERROR;
> > + goto err_buffer_done;
> > + }
> > +
> > + if (block_offset + block.header->size >= config->total_size) {
> > + dev_dbg(mali_c55->dev, "Parameters block too large\n");
> > + state = VB2_BUF_STATE_ERROR;
> > + goto err_buffer_done;
> > + }
> > +
> > + block_handler = &mali_c55_block_handlers[block.header->type];
> > + if (block.header->size != block_handler->size) {
> > + dev_dbg(mali_c55->dev, "Invalid parameters block size\n");
> > + state = VB2_BUF_STATE_ERROR;
> > + goto err_buffer_done;
> > + }
>
> Most of the validation should be done with the buffer is queued.
> Furthermore, you need to make a copy of the data. See the latest version
> of the rkisp1 extensible parameters format series. There may be other
> recent improvements in that series worth copying here.
>
> > +
> > + block_handler->handler(mali_c55, block);
> > +
> > + block_offset += block.header->size;
> > + }
> > +
> > +err_buffer_done:
> > + vb2_buffer_done(&buf->vb.vb2_buf, state);
> > +}
> > +
> > +void mali_c55_unregister_params(struct mali_c55 *mali_c55)
> > +{
> > + struct mali_c55_params *params = &mali_c55->params;
> > +
> > + if (!video_is_registered(¶ms->vdev))
> > + return;
> > +
> > + vb2_video_unregister_device(¶ms->vdev);
> > + media_entity_cleanup(¶ms->vdev.entity);
> > + mutex_destroy(¶ms->lock);
> > +}
> > +
> > +int mali_c55_register_params(struct mali_c55 *mali_c55)
> > +{
> > + struct mali_c55_params *params = &mali_c55->params;
> > + struct video_device *vdev = ¶ms->vdev;
> > + struct vb2_queue *vb2q = ¶ms->queue;
> > + int ret;
> > +
> > + mutex_init(¶ms->lock);
> > + INIT_LIST_HEAD(¶ms->buffers.queue);
> > +
> > + params->pad.flags = MEDIA_PAD_FL_SOURCE;
> > + ret = media_entity_pads_init(¶ms->vdev.entity, 1, ¶ms->pad);
> > + if (ret)
> > + goto err_destroy_mutex;
> > +
> > + vb2q->type = V4L2_BUF_TYPE_META_OUTPUT;
> > + vb2q->io_modes = VB2_MMAP | VB2_DMABUF;
> > + vb2q->drv_priv = params;
> > + vb2q->mem_ops = &vb2_dma_contig_memops;
> > + vb2q->ops = &mali_c55_params_vb2_ops;
> > + vb2q->buf_struct_size = sizeof(struct mali_c55_buffer);
> > + vb2q->min_queued_buffers = 1;
> > + vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > + vb2q->lock = ¶ms->lock;
> > + vb2q->dev = mali_c55->dev;
> > +
> > + ret = vb2_queue_init(vb2q);
> > + if (ret) {
> > + dev_err(mali_c55->dev, "params vb2 queue init failed\n");
> > + goto err_cleanup_entity;
> > + }
> > +
> > + strscpy(params->vdev.name, "mali-c55 3a params",
> > + sizeof(params->vdev.name));
> > + vdev->release = video_device_release_empty;
> > + vdev->fops = &mali_c55_params_v4l2_fops;
> > + vdev->ioctl_ops = &mali_c55_params_v4l2_ioctl_ops;
> > + vdev->lock = ¶ms->lock;
> > + vdev->v4l2_dev = &mali_c55->v4l2_dev;
> > + vdev->queue = ¶ms->queue;
> > + vdev->device_caps = V4L2_CAP_META_OUTPUT | V4L2_CAP_STREAMING;
>
> Shouldn't you set V4L2_CAP_IO_MC and implement media bus code-based
> format enumeration ?
The params (and stats) nodes support a single format, what's the point
of performing media bus code based filtering ?
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 17/18] media: platform: Add mali-c55 parameters video node
2024-07-31 6:53 ` Jacopo Mondi
@ 2024-07-31 7:02 ` Laurent Pinchart
0 siblings, 0 replies; 41+ messages in thread
From: Laurent Pinchart @ 2024-07-31 7:02 UTC (permalink / raw)
To: Jacopo Mondi
Cc: Daniel Scally, linux-media, devicetree, linux-arm-kernel,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
On Wed, Jul 31, 2024 at 08:53:00AM +0200, Jacopo Mondi wrote:
> Hi Laurent
>
> On Wed, Jul 31, 2024 at 01:16:16AM GMT, Laurent Pinchart wrote:
> > Hi Dan,
> >
> > Thank you for the patch.
> >
> > On Tue, Jul 09, 2024 at 02:29:05PM +0100, Daniel Scally wrote:
> > > Add a new code file to the mali-c55 driver that registers an output
> > > video node for userspace to queue buffers of parameters to. Handlers
> > > are included to program the statistics generation plus the white
> > > balance, black level correction and mesh shading correction blocks.
> > >
> > > Update the rest of the driver to register and link the new video node
> > >
> > > Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
> > > Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
> > > ---
> > > Changes in v6:
> > >
> > > - Used a union to generalise the block pointer rather than resorting to
> > > casting everywhere - fantastic idea Sakari, this made it much cleaner.
> > > - Reworked the loop in mali_c55_params_write_config() so that we can be
> > > sure there's remaining space for the next block header.
> > >
> > > Changes in v5:
> > >
> > > - New patch
> > >
> > > drivers/media/platform/arm/mali-c55/Makefile | 1 +
> > > .../platform/arm/mali-c55/mali-c55-common.h | 20 +
> > > .../platform/arm/mali-c55/mali-c55-core.c | 23 +
> > > .../platform/arm/mali-c55/mali-c55-isp.c | 21 +-
> > > .../platform/arm/mali-c55/mali-c55-params.c | 671 ++++++++++++++++++
> > > .../arm/mali-c55/mali-c55-registers.h | 128 ++++
> > > 6 files changed, 863 insertions(+), 1 deletion(-)
> > > create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-params.c
> > >
> > > diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile
> > > index b5a22d414479..d5718b0b23e0 100644
> > > --- a/drivers/media/platform/arm/mali-c55/Makefile
> > > +++ b/drivers/media/platform/arm/mali-c55/Makefile
> > > @@ -3,6 +3,7 @@
> > > mali-c55-y := mali-c55-capture.o \
> > > mali-c55-core.o \
> > > mali-c55-isp.o \
> > > + mali-c55-params.o \
> > > mali-c55-resizer.o \
> > > mali-c55-stats.o \
> > > mali-c55-tpg.o
> > > diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> > > index 136c785c68ba..66a46a7c0547 100644
> > > --- a/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> > > +++ b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> > > @@ -50,6 +50,7 @@ enum mali_c55_isp_pads {
> > > MALI_C55_ISP_PAD_SOURCE_VIDEO,
> > > MALI_C55_ISP_PAD_SOURCE_BYPASS,
> > > MALI_C55_ISP_PAD_SOURCE_STATS,
> > > + MALI_C55_ISP_PAD_SINK_PARAMS,
> > > MALI_C55_ISP_NUM_PADS,
> > > };
> > >
> > > @@ -184,6 +185,21 @@ struct mali_c55_stats {
> > > } buffers;
> > > };
> > >
> > > +struct mali_c55_params {
> > > + struct mali_c55 *mali_c55;
> > > + struct video_device vdev;
> > > + struct vb2_queue queue;
> > > + struct media_pad pad;
> > > + /* Mutex to provide to vb2 */
> > > + struct mutex lock;
> > > +
> > > + struct {
> > > + /* Spinlock to guard buffer queue */
> > > + spinlock_t lock;
> > > + struct list_head queue;
> > > + } buffers;
> > > +};
> > > +
> > > enum mali_c55_config_spaces {
> > > MALI_C55_CONFIG_PING,
> > > MALI_C55_CONFIG_PONG,
> > > @@ -226,6 +242,7 @@ struct mali_c55 {
> > > struct mali_c55_isp isp;
> > > struct mali_c55_resizer resizers[MALI_C55_NUM_RSZS];
> > > struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS];
> > > + struct mali_c55_params params;
> > > struct mali_c55_stats stats;
> > >
> > > struct mali_c55_context context;
> > > @@ -256,6 +273,8 @@ int mali_c55_register_capture_devs(struct mali_c55 *mali_c55);
> > > void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55);
> > > int mali_c55_register_stats(struct mali_c55 *mali_c55);
> > > void mali_c55_unregister_stats(struct mali_c55 *mali_c55);
> > > +int mali_c55_register_params(struct mali_c55 *mali_c55);
> > > +void mali_c55_unregister_params(struct mali_c55 *mali_c55);
> > > struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55);
> > > void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
> > > enum mali_c55_planes plane);
> > > @@ -272,5 +291,6 @@ const struct mali_c55_isp_fmt *
> > > mali_c55_isp_get_mbus_config_by_index(u32 index);
> > > void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
> > > enum mali_c55_config_spaces cfg_space);
> > > +void mali_c55_params_write_config(struct mali_c55 *mali_c55);
> > >
> > > #endif /* _MALI_C55_COMMON_H */
> > > diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> > > index ed0db34767a4..55b3cbf53791 100644
> > > --- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> > > +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> > > @@ -384,6 +384,16 @@ static int mali_c55_create_links(struct mali_c55 *mali_c55)
> > > goto err_remove_links;
> > > }
> > >
> > > + ret = media_create_pad_link(&mali_c55->params.vdev.entity, 0,
> > > + &mali_c55->isp.sd.entity,
> > > + MALI_C55_ISP_PAD_SINK_PARAMS,
> > > + MEDIA_LNK_FL_ENABLED);
> >
> > Should this be immutable, or do you think it makes sense to support
> > operating the ISP without parameters ? I know we did so when developing
> > the driver to test the initial code, but are there real use cases now ?
> >
> > > + if (ret) {
> > > + dev_err(mali_c55->dev,
> > > + "failed to link ISP and parameters video node\n");
> > > + goto err_remove_links;
> > > + }
> > > +
> > > return 0;
> > >
> > > err_remove_links:
> > > @@ -398,6 +408,7 @@ static void mali_c55_unregister_entities(struct mali_c55 *mali_c55)
> > > mali_c55_unregister_isp(mali_c55);
> > > mali_c55_unregister_resizers(mali_c55);
> > > mali_c55_unregister_capture_devs(mali_c55);
> > > + mali_c55_unregister_params(mali_c55);
> > > mali_c55_unregister_stats(mali_c55);
> > > }
> > >
> > > @@ -421,6 +432,10 @@ static int mali_c55_register_entities(struct mali_c55 *mali_c55)
> > > if (ret)
> > > goto err_unregister_entities;
> > >
> > > + ret = mali_c55_register_params(mali_c55);
> > > + if (ret)
> > > + goto err_unregister_entities;
> > > +
> > > ret = mali_c55_register_stats(mali_c55);
> > > if (ret)
> > > goto err_unregister_entities;
> > > @@ -620,6 +635,14 @@ static irqreturn_t mali_c55_isr(int irq, void *context)
> > > curr_config >>= ffs(MALI_C55_REG_PING_PONG_READ_MASK) - 1;
> > > next_config = curr_config ^ 1;
> > >
> > > + /*
> > > + * Write the configuration parameters received from
> > > + * userspace into the configuration buffer, which will
> > > + * be transferred to the 'next' active config space at
> > > + * by mali_c55_swap_next_config().
> > > + */
> > > + mali_c55_params_write_config(mali_c55);
> > > +
> > > /*
> > > * The ordering of these two is currently important as
> > > * mali_c55_stats_fill_buffer() is asynchronous whereas
> > > diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> > > index 2f450c00300a..40d7ef6eda22 100644
> > > --- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> > > +++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> > > @@ -132,6 +132,7 @@ static int mali_c55_isp_start(struct mali_c55 *mali_c55)
> > > cfg->bypass ? MALI_C55_ISP_RAW_BYPASS_BYPASS_MASK :
> > > 0x00);
> > >
> > > + mali_c55_params_write_config(mali_c55);
> > > ret = mali_c55_config_write(ctx, MALI_C55_CONFIG_PING);
> > > if (ret) {
> > > dev_err(mali_c55->dev, "failed to write ISP config\n");
> > > @@ -464,12 +465,17 @@ static int mali_c55_isp_init_state(struct v4l2_subdev *sd,
> > >
> > > src_fmt = v4l2_subdev_state_get_format(state,
> > > MALI_C55_ISP_PAD_SOURCE_STATS);
> > > + sink_fmt = v4l2_subdev_state_get_format(state,
> > > + MALI_C55_ISP_PAD_SINK_PARAMS);
> > >
> > > src_fmt->width = sizeof(struct mali_c55_stats_buffer);
> > > src_fmt->height = 1;
> > > src_fmt->field = V4L2_FIELD_NONE;
> > > src_fmt->code = MEDIA_BUS_FMT_METADATA_FIXED;
> > >
> > > + *sink_fmt = *src_fmt;
> >
> > I would initialize the fields individually, I think the code would be
> > clearer.
> >
> > > + sink_fmt->width = sizeof(struct mali_c55_params_buffer);
> > > +
> > > return 0;
> > > }
> > >
> > > @@ -477,8 +483,20 @@ static const struct v4l2_subdev_internal_ops mali_c55_isp_internal_ops = {
> > > .init_state = mali_c55_isp_init_state,
> > > };
> > >
> > > +static int mali_c55_subdev_link_validate(struct media_link *link)
> > > +{
> > > + /*
> > > + * Skip validation for the parameters sink pad, as the source is not
> > > + * a subdevice.
> > > + */
> > > + if (link->sink->index == MALI_C55_ISP_PAD_SINK_PARAMS)
> > > + return 0;
> > > +
> > > + return v4l2_subdev_link_validate(link);
> > > +}
> > > +
> > > static const struct media_entity_operations mali_c55_isp_media_ops = {
> > > - .link_validate = v4l2_subdev_link_validate,
> > > + .link_validate = mali_c55_subdev_link_validate,
> > > };
> > >
> > > int mali_c55_register_isp(struct mali_c55 *mali_c55)
> > > @@ -501,6 +519,7 @@ int mali_c55_register_isp(struct mali_c55 *mali_c55)
> > > isp->pads[MALI_C55_ISP_PAD_SOURCE_VIDEO].flags = MEDIA_PAD_FL_SOURCE;
> > > isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE;
> > > isp->pads[MALI_C55_ISP_PAD_SOURCE_STATS].flags = MEDIA_PAD_FL_SOURCE;
> > > + isp->pads[MALI_C55_ISP_PAD_SINK_PARAMS].flags = MEDIA_PAD_FL_SINK;
> > >
> > > ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS,
> > > isp->pads);
> > > diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-params.c b/drivers/media/platform/arm/mali-c55/mali-c55-params.c
> > > new file mode 100644
> > > index 000000000000..c0ca4a759653
> > > --- /dev/null
> > > +++ b/drivers/media/platform/arm/mali-c55/mali-c55-params.c
> > > @@ -0,0 +1,671 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +/*
> > > + * ARM Mali-C55 ISP Driver - Configuration parameters output device
> > > + *
> > > + * Copyright (C) 2024 Ideas on Board Oy
> > > + */
> > > +#include <linux/media/arm/mali-c55-config.h>
> > > +
> > > +#include <media/media-entity.h>
> > > +#include <media/v4l2-dev.h>
> > > +#include <media/v4l2-event.h>
> > > +#include <media/v4l2-fh.h>
> > > +#include <media/v4l2-ioctl.h>
> > > +#include <media/videobuf2-core.h>
> > > +#include <media/videobuf2-dma-contig.h>
> > > +
> > > +#include "mali-c55-common.h"
> > > +#include "mali-c55-registers.h"
> > > +
> > > +typedef void (*mali_c55_block_handler)(struct mali_c55 *mali_c55,
> > > + union mali_c55_params_block block);
> > > +
> > > +struct mali_c55_block_handler {
> > > + size_t size;
> > > + mali_c55_block_handler handler;
> > > +};
> > > +
> > > +static void mali_c55_params_sensor_offs(struct mali_c55 *mali_c55,
> > > + union mali_c55_params_block block)
> > > +{
> > > + struct mali_c55_params_sensor_off_preshading *p = block.sensor_offs;
> > > + __u32 global_offset;
> > > +
> > > + if (!block.header->enabled)
> > > + return;
> > > +
> > > + if (!(p->chan00 || p->chan01 || p->chan10 || p->chan11))
> > > + return;
> > > +
> > > + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_00,
> > > + p->chan00 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
> > > + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_01,
> > > + p->chan01 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
> > > + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_10,
> > > + p->chan10 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
> > > + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_11,
> > > + p->chan11 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
> > > +
> > > + /*
> > > + * The average offset is applied as a global offset for the digital
> > > + * gain block
> > > + */
> > > + global_offset = (p->chan00 + p->chan01 + p->chan10 + p->chan11) >> 2;
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_DIGITAL_GAIN_OFFSET,
> > > + MALI_C55_DIGITAL_GAIN_OFFSET_MASK,
> > > + global_offset);
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BYPASS_3,
> > > + MALI_C55_REG_BYPASS_3_SENSOR_OFFSET_PRE_SH,
> > > + 0x00);
> > > +}
> > > +
> > > +static void mali_c55_params_aexp_hist(struct mali_c55 *mali_c55,
> > > + union mali_c55_params_block block)
> > > +{
> > > + u32 disable_mask;
> > > + u32 disable_val;
> > > + u32 base;
> > > +
> > > + if (block.header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST) {
> > > + disable_mask = MALI_C55_AEXP_HIST_DISABLE_MASK;
> > > + disable_val = MALI_C55_AEXP_HIST_DISABLE;
> > > + base = MALI_C55_REG_AEXP_HIST_BASE;
> > > + } else {
> > > + disable_mask = MALI_C55_AEXP_IHIST_DISABLE_MASK;
> > > + disable_val = MALI_C55_AEXP_IHIST_DISABLE;
> > > + base = MALI_C55_REG_AEXP_IHIST_BASE;
> > > + }
> > > + struct mali_c55_params_aexp_hist *params = block.aexp_hist;
> > > +
> > > + if (!block.header->enabled) {
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> > > + disable_mask, disable_val);
> > > + return;
> > > + }
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> > > + disable_mask, false);
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
> > > + MALI_C55_AEXP_HIST_SKIP_X_MASK, params->skip_x);
> > > + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
> > > + MALI_C55_AEXP_HIST_OFFSET_X_MASK,
> > > + MALI_C55_AEXP_HIST_OFFSET_X(params->offset_x));
> > > + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
> > > + MALI_C55_AEXP_HIST_SKIP_Y_MASK,
> > > + MALI_C55_AEXP_HIST_SKIP_Y(params->skip_y));
> > > + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
> > > + MALI_C55_AEXP_HIST_OFFSET_Y_MASK,
> > > + MALI_C55_AEXP_HIST_OFFSET_Y(params->offset_y));
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SCALE_OFFSET,
> > > + MALI_C55_AEXP_HIST_SCALE_BOTTOM_MASK,
> > > + params->scale_bottom);
> > > + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SCALE_OFFSET,
> > > + MALI_C55_AEXP_HIST_SCALE_TOP_MASK,
> > > + MALI_C55_AEXP_HIST_SCALE_TOP(params->scale_top));
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_PLANE_MODE_OFFSET,
> > > + MALI_C55_AEXP_HIST_PLANE_MODE_MASK,
> > > + params->plane_mode);
> > > +
> > > + if (block.header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST)
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> > > + MALI_C55_AEXP_HIST_SWITCH_MASK,
> > > + MALI_C55_AEXP_HIST_SWITCH(params->tap_point));
> > > +}
> > > +
> > > +static void
> > > +mali_c55_params_aexp_hist_weights(struct mali_c55 *mali_c55,
> > > + union mali_c55_params_block block)
> > > +{
> > > + struct mali_c55_params_aexp_weights *params = block.aexp_weights;
> > > + u32 base;
> > > +
> > > + if (!block.header->enabled)
> > > + return;
> > > +
> > > + base = block.header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS ?
> > > + MALI_C55_REG_AEXP_HIST_BASE :
> > > + MALI_C55_REG_AEXP_IHIST_BASE;
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55,
> > > + base + MALI_C55_AEXP_HIST_NODES_USED_OFFSET,
> > > + MALI_C55_AEXP_HIST_NODES_USED_HORIZ_MASK,
> > > + params->nodes_used_horiz);
> > > + mali_c55_ctx_update_bits(mali_c55,
> > > + base + MALI_C55_AEXP_HIST_NODES_USED_OFFSET,
> > > + MALI_C55_AEXP_HIST_NODES_USED_VERT_MASK,
> > > + MALI_C55_AEXP_HIST_NODES_USED_VERT(params->nodes_used_vert));
> > > +
> > > + /*
> > > + * The zone weights array is a 225-element array of u8 values, but that
> > > + * is a bit annoying to handle given the ISP expects 32-bit writes. We
> > > + * just reinterpret it as a 57-element array of 32-bit values for the
> > > + * purposes of this transaction (the 3 bytes of additional space at the
> > > + * end of the write is just padding for the array of weights in the ISP
> > > + * memory space anyway, so there's no risk of overwriting other
> > > + * registers).
> > > + */
> > > + for (unsigned int i = 0; i < 57; i++) {
> > > + u32 val = ((u32 *)params->zone_weights)[i]
> > > + & MALI_C55_AEXP_HIST_ZONE_WEIGHT_MASK;
> > > + u32 addr = base + MALI_C55_AEXP_HIST_ZONE_WEIGHTS_OFFSET + (4 * i);
> > > +
> > > + mali_c55_ctx_write(mali_c55, addr, val);
> > > + }
> > > +}
> > > +
> > > +static void mali_c55_params_digital_gain(struct mali_c55 *mali_c55,
> > > + union mali_c55_params_block block)
> > > +{
> > > + struct mali_c55_params_digital_gain *dgain = block.digital_gain;
> > > +
> > > + /*
> > > + * If the block is flagged as disabled we write a gain of 1.0, which in
> > > + * Q5.8 format is 256.
> > > + */
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_DIGITAL_GAIN,
> > > + MALI_C55_DIGITAL_GAIN_MASK,
> > > + block.header->enabled ? dgain->gain : 256);
> > > +}
> > > +
> > > +static void mali_c55_params_awb_gains(struct mali_c55 *mali_c55,
> > > + union mali_c55_params_block block)
> > > +{
> > > + struct mali_c55_params_awb_gains *gains = block.awb_gains;
> > > +
> > > + /*
> > > + * There are two places AWB gains can be set in the ISP; one affects the
> > > + * image output data and the other affects the statistics for the
> > > + * AEXP-0 tap point.
> > > + */
> > > + u32 addr1 = block.header->type == MALI_C55_PARAM_BLOCK_AWB_GAINS ?
> > > + MALI_C55_REG_AWB_GAINS1 :
> > > + MALI_C55_REG_AWB_GAINS1_AEXP;
> > > + u32 addr2 = block.header->type == MALI_C55_PARAM_BLOCK_AWB_GAINS ?
> > > + MALI_C55_REG_AWB_GAINS2 :
> > > + MALI_C55_REG_AWB_GAINS2_AEXP;
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, addr1, MALI_C55_AWB_GAIN00_MASK,
> > > + gains->gain00);
> > > + mali_c55_ctx_update_bits(mali_c55, addr1, MALI_C55_AWB_GAIN01_MASK,
> > > + MALI_C55_AWB_GAIN01(gains->gain01));
> > > + mali_c55_ctx_update_bits(mali_c55, addr2, MALI_C55_AWB_GAIN10_MASK,
> > > + gains->gain10);
> > > + mali_c55_ctx_update_bits(mali_c55, addr2, MALI_C55_AWB_GAIN11_MASK,
> > > + MALI_C55_AWB_GAIN11(gains->gain11));
> > > +}
> > > +
> > > +static void mali_c55_params_awb_config(struct mali_c55 *mali_c55,
> > > + union mali_c55_params_block block)
> > > +{
> > > + struct mali_c55_params_awb_config *params = block.awb_config;
> > > +
> > > + if (!block.header->enabled) {
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> > > + MALI_C55_AWB_DISABLE_MASK,
> > > + MALI_C55_AWB_DISABLE_MASK);
> > > + return;
> > > + }
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> > > + MALI_C55_AWB_DISABLE_MASK, false);
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_STATS_MODE,
> > > + MALI_C55_AWB_STATS_MODE_MASK, params->stats_mode);
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_WHITE_LEVEL,
> > > + MALI_C55_AWB_WHITE_LEVEL_MASK, params->white_level);
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_BLACK_LEVEL,
> > > + MALI_C55_AWB_BLACK_LEVEL_MASK, params->black_level);
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_MAX,
> > > + MALI_C55_AWB_CR_MAX_MASK, params->cr_max);
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_MIN,
> > > + MALI_C55_AWB_CR_MIN_MASK, params->cr_min);
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_MAX,
> > > + MALI_C55_AWB_CB_MAX_MASK, params->cb_max);
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_MIN,
> > > + MALI_C55_AWB_CB_MIN_MASK, params->cb_min);
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_NODES_USED,
> > > + MALI_C55_AWB_NODES_USED_HORIZ_MASK,
> > > + params->nodes_used_horiz);
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_NODES_USED,
> > > + MALI_C55_AWB_NODES_USED_VERT_MASK,
> > > + MALI_C55_AWB_NODES_USED_VERT(params->nodes_used_vert));
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_HIGH,
> > > + MALI_C55_AWB_CR_HIGH_MASK, params->cr_high);
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_LOW,
> > > + MALI_C55_AWB_CR_LOW_MASK, params->cr_low);
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_HIGH,
> > > + MALI_C55_AWB_CB_HIGH_MASK, params->cb_high);
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_LOW,
> > > + MALI_C55_AWB_CB_LOW_MASK, params->cb_low);
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
> > > + MALI_C55_AWB_SWITCH_MASK,
> > > + MALI_C55_AWB_SWITCH(params->tap_point));
> > > +}
> > > +
> > > +static void mali_c55_params_lsc_config(struct mali_c55 *mali_c55,
> > > + union mali_c55_params_block block)
> > > +{
> > > + struct mali_c55_params_mesh_shading_config *params = block.shading_config;
> > > + unsigned int i;
> > > + u32 addr;
> > > +
> > > + if (!block.header->enabled) {
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > > + MALI_C55_MESH_SHADING_ENABLE_MASK,
> > > + false);
> > > + return;
> > > + }
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > > + MALI_C55_MESH_SHADING_ENABLE_MASK, true);
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > > + MALI_C55_MESH_SHADING_MESH_SHOW_MASK,
> > > + MALI_C55_MESH_SHADING_MESH_SHOW(params->mesh_show));
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > > + MALI_C55_MESH_SHADING_SCALE_MASK,
> > > + MALI_C55_MESH_SHADING_SCALE(params->mesh_scale));
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > > + MALI_C55_MESH_SHADING_PAGE_R_MASK,
> > > + MALI_C55_MESH_SHADING_PAGE_R(params->mesh_page_r));
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > > + MALI_C55_MESH_SHADING_PAGE_G_MASK,
> > > + MALI_C55_MESH_SHADING_PAGE_G(params->mesh_page_g));
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > > + MALI_C55_MESH_SHADING_PAGE_B_MASK,
> > > + MALI_C55_MESH_SHADING_PAGE_B(params->mesh_page_b));
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > > + MALI_C55_MESH_SHADING_MESH_WIDTH_MASK,
> > > + MALI_C55_MESH_SHADING_MESH_WIDTH(params->mesh_width));
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
> > > + MALI_C55_MESH_SHADING_MESH_HEIGHT_MASK,
> > > + MALI_C55_MESH_SHADING_MESH_HEIGHT(params->mesh_height));
> > > +
> > > + for (i = 0; i < MALI_C55_NUM_MESH_SHADING_ELEMENTS; i++) {
> > > + addr = MALI_C55_REG_MESH_SHADING_TABLES + (i * 4);
> > > + mali_c55_ctx_write(mali_c55, addr, params->mesh[i]);
> > > + }
> > > +}
> > > +
> > > +static void mali_c55_params_lsc_selection(struct mali_c55 *mali_c55,
> > > + union mali_c55_params_block block)
> > > +{
> > > + struct mali_c55_params_mesh_shading_selection *params =
> > > + block.shading_selection;
> > > +
> > > + if (!block.header->enabled)
> > > + return;
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK,
> > > + MALI_C55_MESH_SHADING_ALPHA_BANK_R_MASK,
> > > + params->mesh_alpha_bank_r);
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK,
> > > + MALI_C55_MESH_SHADING_ALPHA_BANK_G_MASK,
> > > + MALI_C55_MESH_SHADING_ALPHA_BANK_G(params->mesh_alpha_bank_g));
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK,
> > > + MALI_C55_MESH_SHADING_ALPHA_BANK_B_MASK,
> > > + MALI_C55_MESH_SHADING_ALPHA_BANK_B(params->mesh_alpha_bank_b));
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA,
> > > + MALI_C55_MESH_SHADING_ALPHA_R_MASK,
> > > + params->mesh_alpha_r);
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA,
> > > + MALI_C55_MESH_SHADING_ALPHA_G_MASK,
> > > + MALI_C55_MESH_SHADING_ALPHA_G(params->mesh_alpha_g));
> > > + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA,
> > > + MALI_C55_MESH_SHADING_ALPHA_B_MASK,
> > > + MALI_C55_MESH_SHADING_ALPHA_B(params->mesh_alpha_b));
> > > +
> > > + mali_c55_ctx_update_bits(mali_c55,
> > > + MALI_C55_REG_MESH_SHADING_MESH_STRENGTH,
> > > + MALI_c55_MESH_STRENGTH_MASK,
> > > + params->mesh_strength);
> > > +}
> > > +
> > > +static const struct mali_c55_block_handler mali_c55_block_handlers[] = {
> > > + [MALI_C55_PARAM_BLOCK_SENSOR_OFFS] = {
> > > + .size = sizeof(struct mali_c55_params_sensor_off_preshading),
> > > + .handler = &mali_c55_params_sensor_offs,
> > > + },
> > > + [MALI_C55_PARAM_BLOCK_AEXP_HIST] = {
> > > + .size = sizeof(struct mali_c55_params_aexp_hist),
> > > + .handler = &mali_c55_params_aexp_hist,
> > > + },
> > > + [MALI_C55_PARAM_BLOCK_AEXP_IHIST] = {
> > > + .size = sizeof(struct mali_c55_params_aexp_hist),
> > > + .handler = &mali_c55_params_aexp_hist,
> > > + },
> > > + [MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS] = {
> > > + .size = sizeof(struct mali_c55_params_aexp_weights),
> > > + .handler = &mali_c55_params_aexp_hist_weights,
> > > + },
> > > + [MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS] = {
> > > + .size = sizeof(struct mali_c55_params_aexp_weights),
> > > + .handler = &mali_c55_params_aexp_hist_weights,
> > > + },
> > > + [MALI_C55_PARAM_BLOCK_DIGITAL_GAIN] = {
> > > + .size = sizeof(struct mali_c55_params_digital_gain),
> > > + .handler = &mali_c55_params_digital_gain,
> > > + },
> > > + [MALI_C55_PARAM_BLOCK_AWB_GAINS] = {
> > > + .size = sizeof(struct mali_c55_params_awb_gains),
> > > + .handler = &mali_c55_params_awb_gains,
> > > + },
> > > + [MALI_C55_PARAM_BLOCK_AWB_CONFIG] = {
> > > + .size = sizeof(struct mali_c55_params_awb_config),
> > > + .handler = &mali_c55_params_awb_config,
> > > + },
> > > + [MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP] = {
> > > + .size = sizeof(struct mali_c55_params_awb_gains),
> > > + .handler = &mali_c55_params_awb_gains,
> > > + },
> > > + [MALI_C55_PARAM_MESH_SHADING_CONFIG] = {
> > > + .size = sizeof(struct mali_c55_params_mesh_shading_config),
> > > + .handler = &mali_c55_params_lsc_config,
> > > + },
> > > + [MALI_C55_PARAM_MESH_SHADING_SELECTION] = {
> > > + .size = sizeof(struct mali_c55_params_mesh_shading_selection),
> > > + .handler = &mali_c55_params_lsc_selection,
> > > + },
> > > +};
> > > +
> > > +static int mali_c55_params_enum_fmt_meta_out(struct file *file, void *fh,
> > > + struct v4l2_fmtdesc *f)
> > > +{
> > > + if (f->index)
> > > + return -EINVAL;
> > > +
> > > + f->pixelformat = V4L2_META_FMT_MALI_C55_PARAMS;
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static int mali_c55_params_g_fmt_meta_out(struct file *file, void *fh,
> > > + struct v4l2_format *f)
> > > +{
> > > + static const struct v4l2_meta_format mfmt = {
> > > + .dataformat = V4L2_META_FMT_MALI_C55_PARAMS,
> > > + .buffersize = sizeof(struct mali_c55_params_buffer),
> > > + };
> > > +
> > > + f->fmt.meta = mfmt;
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static int mali_c55_params_querycap(struct file *file,
> > > + void *priv, struct v4l2_capability *cap)
> > > +{
> > > + strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver));
> > > + strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card));
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static const struct v4l2_ioctl_ops mali_c55_params_v4l2_ioctl_ops = {
> > > + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> > > + .vidioc_querybuf = vb2_ioctl_querybuf,
> > > + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> > > + .vidioc_qbuf = vb2_ioctl_qbuf,
> > > + .vidioc_expbuf = vb2_ioctl_expbuf,
> > > + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> > > + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > > + .vidioc_streamon = vb2_ioctl_streamon,
> > > + .vidioc_streamoff = vb2_ioctl_streamoff,
> > > + .vidioc_enum_fmt_meta_out = mali_c55_params_enum_fmt_meta_out,
> > > + .vidioc_g_fmt_meta_out = mali_c55_params_g_fmt_meta_out,
> > > + .vidioc_s_fmt_meta_out = mali_c55_params_g_fmt_meta_out,
> > > + .vidioc_try_fmt_meta_out = mali_c55_params_g_fmt_meta_out,
> > > + .vidioc_querycap = mali_c55_params_querycap,
> > > + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> > > + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > > +};
> > > +
> > > +static const struct v4l2_file_operations mali_c55_params_v4l2_fops = {
> > > + .owner = THIS_MODULE,
> > > + .unlocked_ioctl = video_ioctl2,
> > > + .open = v4l2_fh_open,
> > > + .release = vb2_fop_release,
> > > + .poll = vb2_fop_poll,
> > > + .mmap = vb2_fop_mmap,
> > > +};
> > > +
> > > +static int
> > > +mali_c55_params_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
> > > + unsigned int *num_planes, unsigned int sizes[],
> > > + struct device *alloc_devs[])
> > > +{
> > > + if (*num_planes && *num_planes > 1)
> > > + return -EINVAL;
> > > +
> > > + if (sizes[0] && sizes[0] < sizeof(struct mali_c55_params_buffer))
> > > + return -EINVAL;
> > > +
> > > + *num_planes = 1;
> > > +
> > > + if (!sizes[0])
> > > + sizes[0] = sizeof(struct mali_c55_params_buffer);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static void mali_c55_params_buf_queue(struct vb2_buffer *vb)
> > > +{
> > > + struct mali_c55_params *params = vb2_get_drv_priv(vb->vb2_queue);
> > > + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > + struct mali_c55_buffer *buf = container_of(vbuf,
> > > + struct mali_c55_buffer, vb);
> > > +
> > > + vb2_set_plane_payload(vb, 0, sizeof(struct mali_c55_params_buffer));
> > > +
> > > + spin_lock(¶ms->buffers.lock);
> > > + list_add_tail(&buf->queue, ¶ms->buffers.queue);
> > > + spin_unlock(¶ms->buffers.lock);
> > > +}
> > > +
> > > +static int mali_c55_params_start_streaming(struct vb2_queue *q,
> > > + unsigned int count)
> > > +{
> > > + struct mali_c55_params *params = vb2_get_drv_priv(q);
> > > + struct mali_c55 *mali_c55 = params->mali_c55;
> > > + int ret;
> > > +
> > > + ret = video_device_pipeline_start(¶ms->vdev,
> > > + ¶ms->mali_c55->pipe);
> > > + if (ret)
> > > + return ret;
> > > +
> > > + if (mali_c55->pipe.start_count == mali_c55->pipe.required_count) {
> > > + ret = v4l2_subdev_enable_streams(&mali_c55->isp.sd,
> > > + MALI_C55_ISP_PAD_SOURCE_VIDEO,
> > > + BIT(0));
> > > + if (ret)
> > > + goto err_stop_pipeline;
> > > + }
> > > +
> > > + return 0;
> > > +
> > > +err_stop_pipeline:
> > > + video_device_pipeline_stop(¶ms->vdev);
> >
> > When a failure happens you need to return all queued buffers to vb2 in
> > the QUEUED state.
> >
> > > +
> > > + return ret;
> > > +}
> > > +
> > > +static void mali_c55_params_stop_streaming(struct vb2_queue *q)
> > > +{
> > > + struct mali_c55_params *params = vb2_get_drv_priv(q);
> > > + struct mali_c55_buffer *buf, *tmp;
> > > +
> > > + spin_lock(¶ms->buffers.lock);
> > > +
> > > + list_for_each_entry_safe(buf, tmp, ¶ms->buffers.queue, queue) {
> > > + list_del(&buf->queue);
> > > + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> > > + }
> >
> > This can be turned into a helper function that takes the state as an
> > argument, to be used in the start_streaming error path. It may even be
> > possible to share the helper with all video devices in this driver.
> >
> > > +
> > > + spin_unlock(¶ms->buffers.lock);
> > > +
> > > + video_device_pipeline_stop(¶ms->vdev);
> >
> > I think you need to stop the ISP too, if this is the first video device
> > being stopped. You should return buffers to vb2 only after stopping the
> > ISP.
> >
> > This may be fixed by stopping the ISP, but currently I think you're
> > racing with mali_c55_params_write_config()
> >
> > > +}
> > > +
> > > +static const struct vb2_ops mali_c55_params_vb2_ops = {
> > > + .queue_setup = mali_c55_params_queue_setup,
> > > + .buf_queue = mali_c55_params_buf_queue,
> > > + .wait_prepare = vb2_ops_wait_prepare,
> > > + .wait_finish = vb2_ops_wait_finish,
> > > + .start_streaming = mali_c55_params_start_streaming,
> > > + .stop_streaming = mali_c55_params_stop_streaming,
> > > +};
> > > +
> > > +void mali_c55_params_write_config(struct mali_c55 *mali_c55)
> > > +{
> > > + struct mali_c55_params *params = &mali_c55->params;
> > > + enum vb2_buffer_state state = VB2_BUF_STATE_DONE;
> > > + struct mali_c55_params_buffer *config;
> > > + struct mali_c55_buffer *buf;
> > > + size_t block_offset = 0;
> > > + size_t max_offset;
> > > +
> > > + spin_lock(¶ms->buffers.lock);
> > > +
> > > + buf = list_first_entry_or_null(¶ms->buffers.queue,
> > > + struct mali_c55_buffer, queue);
> > > + if (buf)
> > > + list_del(&buf->queue);
> > > + spin_unlock(¶ms->buffers.lock);
> > > +
> > > + if (!buf)
> > > + return;
> >
> > If this happens we'll lose synchronization. I suppose we can't do much
> > about this, we really need a request-based API.
> >
> > > +
> > > + buf->vb.sequence = mali_c55->isp.frame_sequence;
> > > + config = vb2_plane_vaddr(&buf->vb.vb2_buf, 0);
> > > +
> > > + if (config->total_size > MALI_C55_PARAMS_MAX_SIZE) {
> > > + dev_dbg(mali_c55->dev, "Invalid parameters buffer size %u\n",
> > > + config->total_size);
> > > + state = VB2_BUF_STATE_ERROR;
> > > + goto err_buffer_done;
> > > + }
> > > +
> > > + max_offset = config->total_size - sizeof(struct mali_c55_params_block_header);
> > > +
> > > + /* Walk the list of parameter blocks and process them. */
> > > + while (block_offset < max_offset) {
> > > + const struct mali_c55_block_handler *block_handler;
> > > + union mali_c55_params_block block;
> > > +
> > > + block = (union mali_c55_params_block)
> > > + &config->data[block_offset];
> > > +
> > > + if (block.header->type >= MALI_C55_PARAM_BLOCK_SENTINEL) {
> > > + dev_dbg(mali_c55->dev, "Invalid parameters block type\n");
> > > + state = VB2_BUF_STATE_ERROR;
> > > + goto err_buffer_done;
> > > + }
> > > +
> > > + if (block_offset + block.header->size >= config->total_size) {
> > > + dev_dbg(mali_c55->dev, "Parameters block too large\n");
> > > + state = VB2_BUF_STATE_ERROR;
> > > + goto err_buffer_done;
> > > + }
> > > +
> > > + block_handler = &mali_c55_block_handlers[block.header->type];
> > > + if (block.header->size != block_handler->size) {
> > > + dev_dbg(mali_c55->dev, "Invalid parameters block size\n");
> > > + state = VB2_BUF_STATE_ERROR;
> > > + goto err_buffer_done;
> > > + }
> >
> > Most of the validation should be done with the buffer is queued.
> > Furthermore, you need to make a copy of the data. See the latest version
> > of the rkisp1 extensible parameters format series. There may be other
> > recent improvements in that series worth copying here.
> >
> > > +
> > > + block_handler->handler(mali_c55, block);
> > > +
> > > + block_offset += block.header->size;
> > > + }
> > > +
> > > +err_buffer_done:
> > > + vb2_buffer_done(&buf->vb.vb2_buf, state);
> > > +}
> > > +
> > > +void mali_c55_unregister_params(struct mali_c55 *mali_c55)
> > > +{
> > > + struct mali_c55_params *params = &mali_c55->params;
> > > +
> > > + if (!video_is_registered(¶ms->vdev))
> > > + return;
> > > +
> > > + vb2_video_unregister_device(¶ms->vdev);
> > > + media_entity_cleanup(¶ms->vdev.entity);
> > > + mutex_destroy(¶ms->lock);
> > > +}
> > > +
> > > +int mali_c55_register_params(struct mali_c55 *mali_c55)
> > > +{
> > > + struct mali_c55_params *params = &mali_c55->params;
> > > + struct video_device *vdev = ¶ms->vdev;
> > > + struct vb2_queue *vb2q = ¶ms->queue;
> > > + int ret;
> > > +
> > > + mutex_init(¶ms->lock);
> > > + INIT_LIST_HEAD(¶ms->buffers.queue);
> > > +
> > > + params->pad.flags = MEDIA_PAD_FL_SOURCE;
> > > + ret = media_entity_pads_init(¶ms->vdev.entity, 1, ¶ms->pad);
> > > + if (ret)
> > > + goto err_destroy_mutex;
> > > +
> > > + vb2q->type = V4L2_BUF_TYPE_META_OUTPUT;
> > > + vb2q->io_modes = VB2_MMAP | VB2_DMABUF;
> > > + vb2q->drv_priv = params;
> > > + vb2q->mem_ops = &vb2_dma_contig_memops;
> > > + vb2q->ops = &mali_c55_params_vb2_ops;
> > > + vb2q->buf_struct_size = sizeof(struct mali_c55_buffer);
> > > + vb2q->min_queued_buffers = 1;
> > > + vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > > + vb2q->lock = ¶ms->lock;
> > > + vb2q->dev = mali_c55->dev;
> > > +
> > > + ret = vb2_queue_init(vb2q);
> > > + if (ret) {
> > > + dev_err(mali_c55->dev, "params vb2 queue init failed\n");
> > > + goto err_cleanup_entity;
> > > + }
> > > +
> > > + strscpy(params->vdev.name, "mali-c55 3a params",
> > > + sizeof(params->vdev.name));
> > > + vdev->release = video_device_release_empty;
> > > + vdev->fops = &mali_c55_params_v4l2_fops;
> > > + vdev->ioctl_ops = &mali_c55_params_v4l2_ioctl_ops;
> > > + vdev->lock = ¶ms->lock;
> > > + vdev->v4l2_dev = &mali_c55->v4l2_dev;
> > > + vdev->queue = ¶ms->queue;
> > > + vdev->device_caps = V4L2_CAP_META_OUTPUT | V4L2_CAP_STREAMING;
> >
> > Shouldn't you set V4L2_CAP_IO_MC and implement media bus code-based
> > format enumeration ?
>
> The params (and stats) nodes support a single format, what's the point
> of performing media bus code based filtering ?
Just consistency, enabling generic userspace code.
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 17/18] media: platform: Add mali-c55 parameters video node
2024-07-30 22:16 ` Laurent Pinchart
2024-07-31 6:53 ` Jacopo Mondi
@ 2024-07-31 15:12 ` Dan Scally
2024-08-02 15:03 ` Laurent Pinchart
1 sibling, 1 reply; 41+ messages in thread
From: Dan Scally @ 2024-07-31 15:12 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Laurent
On 30/07/2024 23:16, Laurent Pinchart wrote:
> Hi Dan,
>
> Thank you for the patch.
>
> On Tue, Jul 09, 2024 at 02:29:05PM +0100, Daniel Scally wrote:
>> Add a new code file to the mali-c55 driver that registers an output
>> video node for userspace to queue buffers of parameters to. Handlers
>> are included to program the statistics generation plus the white
>> balance, black level correction and mesh shading correction blocks.
>>
>> Update the rest of the driver to register and link the new video node
>>
>> Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
>> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
>> ---
>> Changes in v6:
>>
>> - Used a union to generalise the block pointer rather than resorting to
>> casting everywhere - fantastic idea Sakari, this made it much cleaner.
>> - Reworked the loop in mali_c55_params_write_config() so that we can be
>> sure there's remaining space for the next block header.
>>
>> Changes in v5:
>>
>> - New patch
>>
>> drivers/media/platform/arm/mali-c55/Makefile | 1 +
>> .../platform/arm/mali-c55/mali-c55-common.h | 20 +
>> .../platform/arm/mali-c55/mali-c55-core.c | 23 +
>> .../platform/arm/mali-c55/mali-c55-isp.c | 21 +-
>> .../platform/arm/mali-c55/mali-c55-params.c | 671 ++++++++++++++++++
>> .../arm/mali-c55/mali-c55-registers.h | 128 ++++
>> 6 files changed, 863 insertions(+), 1 deletion(-)
>> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-params.c
>>
>> diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile
>> index b5a22d414479..d5718b0b23e0 100644
>> --- a/drivers/media/platform/arm/mali-c55/Makefile
>> +++ b/drivers/media/platform/arm/mali-c55/Makefile
>> @@ -3,6 +3,7 @@
>> mali-c55-y := mali-c55-capture.o \
>> mali-c55-core.o \
>> mali-c55-isp.o \
>> + mali-c55-params.o \
>> mali-c55-resizer.o \
>> mali-c55-stats.o \
>> mali-c55-tpg.o
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
>> index 136c785c68ba..66a46a7c0547 100644
>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-common.h
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
>> @@ -50,6 +50,7 @@ enum mali_c55_isp_pads {
>> MALI_C55_ISP_PAD_SOURCE_VIDEO,
>> MALI_C55_ISP_PAD_SOURCE_BYPASS,
>> MALI_C55_ISP_PAD_SOURCE_STATS,
>> + MALI_C55_ISP_PAD_SINK_PARAMS,
>> MALI_C55_ISP_NUM_PADS,
>> };
>>
>> @@ -184,6 +185,21 @@ struct mali_c55_stats {
>> } buffers;
>> };
>>
>> +struct mali_c55_params {
>> + struct mali_c55 *mali_c55;
>> + struct video_device vdev;
>> + struct vb2_queue queue;
>> + struct media_pad pad;
>> + /* Mutex to provide to vb2 */
>> + struct mutex lock;
>> +
>> + struct {
>> + /* Spinlock to guard buffer queue */
>> + spinlock_t lock;
>> + struct list_head queue;
>> + } buffers;
>> +};
>> +
>> enum mali_c55_config_spaces {
>> MALI_C55_CONFIG_PING,
>> MALI_C55_CONFIG_PONG,
>> @@ -226,6 +242,7 @@ struct mali_c55 {
>> struct mali_c55_isp isp;
>> struct mali_c55_resizer resizers[MALI_C55_NUM_RSZS];
>> struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS];
>> + struct mali_c55_params params;
>> struct mali_c55_stats stats;
>>
>> struct mali_c55_context context;
>> @@ -256,6 +273,8 @@ int mali_c55_register_capture_devs(struct mali_c55 *mali_c55);
>> void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55);
>> int mali_c55_register_stats(struct mali_c55 *mali_c55);
>> void mali_c55_unregister_stats(struct mali_c55 *mali_c55);
>> +int mali_c55_register_params(struct mali_c55 *mali_c55);
>> +void mali_c55_unregister_params(struct mali_c55 *mali_c55);
>> struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55);
>> void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
>> enum mali_c55_planes plane);
>> @@ -272,5 +291,6 @@ const struct mali_c55_isp_fmt *
>> mali_c55_isp_get_mbus_config_by_index(u32 index);
>> void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
>> enum mali_c55_config_spaces cfg_space);
>> +void mali_c55_params_write_config(struct mali_c55 *mali_c55);
>>
>> #endif /* _MALI_C55_COMMON_H */
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
>> index ed0db34767a4..55b3cbf53791 100644
>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
>> @@ -384,6 +384,16 @@ static int mali_c55_create_links(struct mali_c55 *mali_c55)
>> goto err_remove_links;
>> }
>>
>> + ret = media_create_pad_link(&mali_c55->params.vdev.entity, 0,
>> + &mali_c55->isp.sd.entity,
>> + MALI_C55_ISP_PAD_SINK_PARAMS,
>> + MEDIA_LNK_FL_ENABLED);
> Should this be immutable, or do you think it makes sense to support
> operating the ISP without parameters ? I know we did so when developing
> the driver to test the initial code, but are there real use cases now ?
If all you're interested in is RAW data bypassed through the ISP then you could skip the parameters
and statistics - which would be the case for RGB/YUV cameras feeding the ISP too. Is that sufficient
use-case to leave them mutable?
>
>> + if (ret) {
>> + dev_err(mali_c55->dev,
>> + "failed to link ISP and parameters video node\n");
>> + goto err_remove_links;
>> + }
>> +
>> return 0;
>>
>> err_remove_links:
>> @@ -398,6 +408,7 @@ static void mali_c55_unregister_entities(struct mali_c55 *mali_c55)
>> mali_c55_unregister_isp(mali_c55);
>> mali_c55_unregister_resizers(mali_c55);
>> mali_c55_unregister_capture_devs(mali_c55);
>> + mali_c55_unregister_params(mali_c55);
>> mali_c55_unregister_stats(mali_c55);
>> }
>>
>> @@ -421,6 +432,10 @@ static int mali_c55_register_entities(struct mali_c55 *mali_c55)
>> if (ret)
>> goto err_unregister_entities;
>>
>> + ret = mali_c55_register_params(mali_c55);
>> + if (ret)
>> + goto err_unregister_entities;
>> +
>> ret = mali_c55_register_stats(mali_c55);
>> if (ret)
>> goto err_unregister_entities;
>> @@ -620,6 +635,14 @@ static irqreturn_t mali_c55_isr(int irq, void *context)
>> curr_config >>= ffs(MALI_C55_REG_PING_PONG_READ_MASK) - 1;
>> next_config = curr_config ^ 1;
>>
>> + /*
>> + * Write the configuration parameters received from
>> + * userspace into the configuration buffer, which will
>> + * be transferred to the 'next' active config space at
>> + * by mali_c55_swap_next_config().
>> + */
>> + mali_c55_params_write_config(mali_c55);
>> +
>> /*
>> * The ordering of these two is currently important as
>> * mali_c55_stats_fill_buffer() is asynchronous whereas
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>> index 2f450c00300a..40d7ef6eda22 100644
>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>> @@ -132,6 +132,7 @@ static int mali_c55_isp_start(struct mali_c55 *mali_c55)
>> cfg->bypass ? MALI_C55_ISP_RAW_BYPASS_BYPASS_MASK :
>> 0x00);
>>
>> + mali_c55_params_write_config(mali_c55);
>> ret = mali_c55_config_write(ctx, MALI_C55_CONFIG_PING);
>> if (ret) {
>> dev_err(mali_c55->dev, "failed to write ISP config\n");
>> @@ -464,12 +465,17 @@ static int mali_c55_isp_init_state(struct v4l2_subdev *sd,
>>
>> src_fmt = v4l2_subdev_state_get_format(state,
>> MALI_C55_ISP_PAD_SOURCE_STATS);
>> + sink_fmt = v4l2_subdev_state_get_format(state,
>> + MALI_C55_ISP_PAD_SINK_PARAMS);
>>
>> src_fmt->width = sizeof(struct mali_c55_stats_buffer);
>> src_fmt->height = 1;
>> src_fmt->field = V4L2_FIELD_NONE;
>> src_fmt->code = MEDIA_BUS_FMT_METADATA_FIXED;
>>
>> + *sink_fmt = *src_fmt;
> I would initialize the fields individually, I think the code would be
> clearer.
>
>> + sink_fmt->width = sizeof(struct mali_c55_params_buffer);
>> +
>> return 0;
>> }
>>
>> @@ -477,8 +483,20 @@ static const struct v4l2_subdev_internal_ops mali_c55_isp_internal_ops = {
>> .init_state = mali_c55_isp_init_state,
>> };
>>
>> +static int mali_c55_subdev_link_validate(struct media_link *link)
>> +{
>> + /*
>> + * Skip validation for the parameters sink pad, as the source is not
>> + * a subdevice.
>> + */
>> + if (link->sink->index == MALI_C55_ISP_PAD_SINK_PARAMS)
>> + return 0;
>> +
>> + return v4l2_subdev_link_validate(link);
>> +}
>> +
>> static const struct media_entity_operations mali_c55_isp_media_ops = {
>> - .link_validate = v4l2_subdev_link_validate,
>> + .link_validate = mali_c55_subdev_link_validate,
>> };
>>
>> int mali_c55_register_isp(struct mali_c55 *mali_c55)
>> @@ -501,6 +519,7 @@ int mali_c55_register_isp(struct mali_c55 *mali_c55)
>> isp->pads[MALI_C55_ISP_PAD_SOURCE_VIDEO].flags = MEDIA_PAD_FL_SOURCE;
>> isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE;
>> isp->pads[MALI_C55_ISP_PAD_SOURCE_STATS].flags = MEDIA_PAD_FL_SOURCE;
>> + isp->pads[MALI_C55_ISP_PAD_SINK_PARAMS].flags = MEDIA_PAD_FL_SINK;
>>
>> ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS,
>> isp->pads);
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-params.c b/drivers/media/platform/arm/mali-c55/mali-c55-params.c
>> new file mode 100644
>> index 000000000000..c0ca4a759653
>> --- /dev/null
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-params.c
>> @@ -0,0 +1,671 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * ARM Mali-C55 ISP Driver - Configuration parameters output device
>> + *
>> + * Copyright (C) 2024 Ideas on Board Oy
>> + */
>> +#include <linux/media/arm/mali-c55-config.h>
>> +
>> +#include <media/media-entity.h>
>> +#include <media/v4l2-dev.h>
>> +#include <media/v4l2-event.h>
>> +#include <media/v4l2-fh.h>
>> +#include <media/v4l2-ioctl.h>
>> +#include <media/videobuf2-core.h>
>> +#include <media/videobuf2-dma-contig.h>
>> +
>> +#include "mali-c55-common.h"
>> +#include "mali-c55-registers.h"
>> +
>> +typedef void (*mali_c55_block_handler)(struct mali_c55 *mali_c55,
>> + union mali_c55_params_block block);
>> +
>> +struct mali_c55_block_handler {
>> + size_t size;
>> + mali_c55_block_handler handler;
>> +};
>> +
>> +static void mali_c55_params_sensor_offs(struct mali_c55 *mali_c55,
>> + union mali_c55_params_block block)
>> +{
>> + struct mali_c55_params_sensor_off_preshading *p = block.sensor_offs;
>> + __u32 global_offset;
>> +
>> + if (!block.header->enabled)
>> + return;
>> +
>> + if (!(p->chan00 || p->chan01 || p->chan10 || p->chan11))
>> + return;
>> +
>> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_00,
>> + p->chan00 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
>> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_01,
>> + p->chan01 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
>> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_10,
>> + p->chan10 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
>> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_11,
>> + p->chan11 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK);
>> +
>> + /*
>> + * The average offset is applied as a global offset for the digital
>> + * gain block
>> + */
>> + global_offset = (p->chan00 + p->chan01 + p->chan10 + p->chan11) >> 2;
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_DIGITAL_GAIN_OFFSET,
>> + MALI_C55_DIGITAL_GAIN_OFFSET_MASK,
>> + global_offset);
>> +
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BYPASS_3,
>> + MALI_C55_REG_BYPASS_3_SENSOR_OFFSET_PRE_SH,
>> + 0x00);
>> +}
>> +
>> +static void mali_c55_params_aexp_hist(struct mali_c55 *mali_c55,
>> + union mali_c55_params_block block)
>> +{
>> + u32 disable_mask;
>> + u32 disable_val;
>> + u32 base;
>> +
>> + if (block.header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST) {
>> + disable_mask = MALI_C55_AEXP_HIST_DISABLE_MASK;
>> + disable_val = MALI_C55_AEXP_HIST_DISABLE;
>> + base = MALI_C55_REG_AEXP_HIST_BASE;
>> + } else {
>> + disable_mask = MALI_C55_AEXP_IHIST_DISABLE_MASK;
>> + disable_val = MALI_C55_AEXP_IHIST_DISABLE;
>> + base = MALI_C55_REG_AEXP_IHIST_BASE;
>> + }
>> + struct mali_c55_params_aexp_hist *params = block.aexp_hist;
>> +
>> + if (!block.header->enabled) {
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
>> + disable_mask, disable_val);
>> + return;
>> + }
>> +
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
>> + disable_mask, false);
>> +
>> + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
>> + MALI_C55_AEXP_HIST_SKIP_X_MASK, params->skip_x);
>> + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
>> + MALI_C55_AEXP_HIST_OFFSET_X_MASK,
>> + MALI_C55_AEXP_HIST_OFFSET_X(params->offset_x));
>> + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
>> + MALI_C55_AEXP_HIST_SKIP_Y_MASK,
>> + MALI_C55_AEXP_HIST_SKIP_Y(params->skip_y));
>> + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET,
>> + MALI_C55_AEXP_HIST_OFFSET_Y_MASK,
>> + MALI_C55_AEXP_HIST_OFFSET_Y(params->offset_y));
>> +
>> + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SCALE_OFFSET,
>> + MALI_C55_AEXP_HIST_SCALE_BOTTOM_MASK,
>> + params->scale_bottom);
>> + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SCALE_OFFSET,
>> + MALI_C55_AEXP_HIST_SCALE_TOP_MASK,
>> + MALI_C55_AEXP_HIST_SCALE_TOP(params->scale_top));
>> +
>> + mali_c55_ctx_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_PLANE_MODE_OFFSET,
>> + MALI_C55_AEXP_HIST_PLANE_MODE_MASK,
>> + params->plane_mode);
>> +
>> + if (block.header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST)
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
>> + MALI_C55_AEXP_HIST_SWITCH_MASK,
>> + MALI_C55_AEXP_HIST_SWITCH(params->tap_point));
>> +}
>> +
>> +static void
>> +mali_c55_params_aexp_hist_weights(struct mali_c55 *mali_c55,
>> + union mali_c55_params_block block)
>> +{
>> + struct mali_c55_params_aexp_weights *params = block.aexp_weights;
>> + u32 base;
>> +
>> + if (!block.header->enabled)
>> + return;
>> +
>> + base = block.header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS ?
>> + MALI_C55_REG_AEXP_HIST_BASE :
>> + MALI_C55_REG_AEXP_IHIST_BASE;
>> +
>> + mali_c55_ctx_update_bits(mali_c55,
>> + base + MALI_C55_AEXP_HIST_NODES_USED_OFFSET,
>> + MALI_C55_AEXP_HIST_NODES_USED_HORIZ_MASK,
>> + params->nodes_used_horiz);
>> + mali_c55_ctx_update_bits(mali_c55,
>> + base + MALI_C55_AEXP_HIST_NODES_USED_OFFSET,
>> + MALI_C55_AEXP_HIST_NODES_USED_VERT_MASK,
>> + MALI_C55_AEXP_HIST_NODES_USED_VERT(params->nodes_used_vert));
>> +
>> + /*
>> + * The zone weights array is a 225-element array of u8 values, but that
>> + * is a bit annoying to handle given the ISP expects 32-bit writes. We
>> + * just reinterpret it as a 57-element array of 32-bit values for the
>> + * purposes of this transaction (the 3 bytes of additional space at the
>> + * end of the write is just padding for the array of weights in the ISP
>> + * memory space anyway, so there's no risk of overwriting other
>> + * registers).
>> + */
>> + for (unsigned int i = 0; i < 57; i++) {
>> + u32 val = ((u32 *)params->zone_weights)[i]
>> + & MALI_C55_AEXP_HIST_ZONE_WEIGHT_MASK;
>> + u32 addr = base + MALI_C55_AEXP_HIST_ZONE_WEIGHTS_OFFSET + (4 * i);
>> +
>> + mali_c55_ctx_write(mali_c55, addr, val);
>> + }
>> +}
>> +
>> +static void mali_c55_params_digital_gain(struct mali_c55 *mali_c55,
>> + union mali_c55_params_block block)
>> +{
>> + struct mali_c55_params_digital_gain *dgain = block.digital_gain;
>> +
>> + /*
>> + * If the block is flagged as disabled we write a gain of 1.0, which in
>> + * Q5.8 format is 256.
>> + */
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_DIGITAL_GAIN,
>> + MALI_C55_DIGITAL_GAIN_MASK,
>> + block.header->enabled ? dgain->gain : 256);
>> +}
>> +
>> +static void mali_c55_params_awb_gains(struct mali_c55 *mali_c55,
>> + union mali_c55_params_block block)
>> +{
>> + struct mali_c55_params_awb_gains *gains = block.awb_gains;
>> +
>> + /*
>> + * There are two places AWB gains can be set in the ISP; one affects the
>> + * image output data and the other affects the statistics for the
>> + * AEXP-0 tap point.
>> + */
>> + u32 addr1 = block.header->type == MALI_C55_PARAM_BLOCK_AWB_GAINS ?
>> + MALI_C55_REG_AWB_GAINS1 :
>> + MALI_C55_REG_AWB_GAINS1_AEXP;
>> + u32 addr2 = block.header->type == MALI_C55_PARAM_BLOCK_AWB_GAINS ?
>> + MALI_C55_REG_AWB_GAINS2 :
>> + MALI_C55_REG_AWB_GAINS2_AEXP;
>> +
>> + mali_c55_ctx_update_bits(mali_c55, addr1, MALI_C55_AWB_GAIN00_MASK,
>> + gains->gain00);
>> + mali_c55_ctx_update_bits(mali_c55, addr1, MALI_C55_AWB_GAIN01_MASK,
>> + MALI_C55_AWB_GAIN01(gains->gain01));
>> + mali_c55_ctx_update_bits(mali_c55, addr2, MALI_C55_AWB_GAIN10_MASK,
>> + gains->gain10);
>> + mali_c55_ctx_update_bits(mali_c55, addr2, MALI_C55_AWB_GAIN11_MASK,
>> + MALI_C55_AWB_GAIN11(gains->gain11));
>> +}
>> +
>> +static void mali_c55_params_awb_config(struct mali_c55 *mali_c55,
>> + union mali_c55_params_block block)
>> +{
>> + struct mali_c55_params_awb_config *params = block.awb_config;
>> +
>> + if (!block.header->enabled) {
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
>> + MALI_C55_AWB_DISABLE_MASK,
>> + MALI_C55_AWB_DISABLE_MASK);
>> + return;
>> + }
>> +
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
>> + MALI_C55_AWB_DISABLE_MASK, false);
>> +
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_STATS_MODE,
>> + MALI_C55_AWB_STATS_MODE_MASK, params->stats_mode);
>> +
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_WHITE_LEVEL,
>> + MALI_C55_AWB_WHITE_LEVEL_MASK, params->white_level);
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_BLACK_LEVEL,
>> + MALI_C55_AWB_BLACK_LEVEL_MASK, params->black_level);
>> +
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_MAX,
>> + MALI_C55_AWB_CR_MAX_MASK, params->cr_max);
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_MIN,
>> + MALI_C55_AWB_CR_MIN_MASK, params->cr_min);
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_MAX,
>> + MALI_C55_AWB_CB_MAX_MASK, params->cb_max);
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_MIN,
>> + MALI_C55_AWB_CB_MIN_MASK, params->cb_min);
>> +
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_NODES_USED,
>> + MALI_C55_AWB_NODES_USED_HORIZ_MASK,
>> + params->nodes_used_horiz);
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_NODES_USED,
>> + MALI_C55_AWB_NODES_USED_VERT_MASK,
>> + MALI_C55_AWB_NODES_USED_VERT(params->nodes_used_vert));
>> +
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_HIGH,
>> + MALI_C55_AWB_CR_HIGH_MASK, params->cr_high);
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CR_LOW,
>> + MALI_C55_AWB_CR_LOW_MASK, params->cr_low);
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_HIGH,
>> + MALI_C55_AWB_CB_HIGH_MASK, params->cb_high);
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_AWB_CB_LOW,
>> + MALI_C55_AWB_CB_LOW_MASK, params->cb_low);
>> +
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG,
>> + MALI_C55_AWB_SWITCH_MASK,
>> + MALI_C55_AWB_SWITCH(params->tap_point));
>> +}
>> +
>> +static void mali_c55_params_lsc_config(struct mali_c55 *mali_c55,
>> + union mali_c55_params_block block)
>> +{
>> + struct mali_c55_params_mesh_shading_config *params = block.shading_config;
>> + unsigned int i;
>> + u32 addr;
>> +
>> + if (!block.header->enabled) {
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
>> + MALI_C55_MESH_SHADING_ENABLE_MASK,
>> + false);
>> + return;
>> + }
>> +
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
>> + MALI_C55_MESH_SHADING_ENABLE_MASK, true);
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
>> + MALI_C55_MESH_SHADING_MESH_SHOW_MASK,
>> + MALI_C55_MESH_SHADING_MESH_SHOW(params->mesh_show));
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
>> + MALI_C55_MESH_SHADING_SCALE_MASK,
>> + MALI_C55_MESH_SHADING_SCALE(params->mesh_scale));
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
>> + MALI_C55_MESH_SHADING_PAGE_R_MASK,
>> + MALI_C55_MESH_SHADING_PAGE_R(params->mesh_page_r));
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
>> + MALI_C55_MESH_SHADING_PAGE_G_MASK,
>> + MALI_C55_MESH_SHADING_PAGE_G(params->mesh_page_g));
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
>> + MALI_C55_MESH_SHADING_PAGE_B_MASK,
>> + MALI_C55_MESH_SHADING_PAGE_B(params->mesh_page_b));
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
>> + MALI_C55_MESH_SHADING_MESH_WIDTH_MASK,
>> + MALI_C55_MESH_SHADING_MESH_WIDTH(params->mesh_width));
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG,
>> + MALI_C55_MESH_SHADING_MESH_HEIGHT_MASK,
>> + MALI_C55_MESH_SHADING_MESH_HEIGHT(params->mesh_height));
>> +
>> + for (i = 0; i < MALI_C55_NUM_MESH_SHADING_ELEMENTS; i++) {
>> + addr = MALI_C55_REG_MESH_SHADING_TABLES + (i * 4);
>> + mali_c55_ctx_write(mali_c55, addr, params->mesh[i]);
>> + }
>> +}
>> +
>> +static void mali_c55_params_lsc_selection(struct mali_c55 *mali_c55,
>> + union mali_c55_params_block block)
>> +{
>> + struct mali_c55_params_mesh_shading_selection *params =
>> + block.shading_selection;
>> +
>> + if (!block.header->enabled)
>> + return;
>> +
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK,
>> + MALI_C55_MESH_SHADING_ALPHA_BANK_R_MASK,
>> + params->mesh_alpha_bank_r);
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK,
>> + MALI_C55_MESH_SHADING_ALPHA_BANK_G_MASK,
>> + MALI_C55_MESH_SHADING_ALPHA_BANK_G(params->mesh_alpha_bank_g));
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK,
>> + MALI_C55_MESH_SHADING_ALPHA_BANK_B_MASK,
>> + MALI_C55_MESH_SHADING_ALPHA_BANK_B(params->mesh_alpha_bank_b));
>> +
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA,
>> + MALI_C55_MESH_SHADING_ALPHA_R_MASK,
>> + params->mesh_alpha_r);
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA,
>> + MALI_C55_MESH_SHADING_ALPHA_G_MASK,
>> + MALI_C55_MESH_SHADING_ALPHA_G(params->mesh_alpha_g));
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA,
>> + MALI_C55_MESH_SHADING_ALPHA_B_MASK,
>> + MALI_C55_MESH_SHADING_ALPHA_B(params->mesh_alpha_b));
>> +
>> + mali_c55_ctx_update_bits(mali_c55,
>> + MALI_C55_REG_MESH_SHADING_MESH_STRENGTH,
>> + MALI_c55_MESH_STRENGTH_MASK,
>> + params->mesh_strength);
>> +}
>> +
>> +static const struct mali_c55_block_handler mali_c55_block_handlers[] = {
>> + [MALI_C55_PARAM_BLOCK_SENSOR_OFFS] = {
>> + .size = sizeof(struct mali_c55_params_sensor_off_preshading),
>> + .handler = &mali_c55_params_sensor_offs,
>> + },
>> + [MALI_C55_PARAM_BLOCK_AEXP_HIST] = {
>> + .size = sizeof(struct mali_c55_params_aexp_hist),
>> + .handler = &mali_c55_params_aexp_hist,
>> + },
>> + [MALI_C55_PARAM_BLOCK_AEXP_IHIST] = {
>> + .size = sizeof(struct mali_c55_params_aexp_hist),
>> + .handler = &mali_c55_params_aexp_hist,
>> + },
>> + [MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS] = {
>> + .size = sizeof(struct mali_c55_params_aexp_weights),
>> + .handler = &mali_c55_params_aexp_hist_weights,
>> + },
>> + [MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS] = {
>> + .size = sizeof(struct mali_c55_params_aexp_weights),
>> + .handler = &mali_c55_params_aexp_hist_weights,
>> + },
>> + [MALI_C55_PARAM_BLOCK_DIGITAL_GAIN] = {
>> + .size = sizeof(struct mali_c55_params_digital_gain),
>> + .handler = &mali_c55_params_digital_gain,
>> + },
>> + [MALI_C55_PARAM_BLOCK_AWB_GAINS] = {
>> + .size = sizeof(struct mali_c55_params_awb_gains),
>> + .handler = &mali_c55_params_awb_gains,
>> + },
>> + [MALI_C55_PARAM_BLOCK_AWB_CONFIG] = {
>> + .size = sizeof(struct mali_c55_params_awb_config),
>> + .handler = &mali_c55_params_awb_config,
>> + },
>> + [MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP] = {
>> + .size = sizeof(struct mali_c55_params_awb_gains),
>> + .handler = &mali_c55_params_awb_gains,
>> + },
>> + [MALI_C55_PARAM_MESH_SHADING_CONFIG] = {
>> + .size = sizeof(struct mali_c55_params_mesh_shading_config),
>> + .handler = &mali_c55_params_lsc_config,
>> + },
>> + [MALI_C55_PARAM_MESH_SHADING_SELECTION] = {
>> + .size = sizeof(struct mali_c55_params_mesh_shading_selection),
>> + .handler = &mali_c55_params_lsc_selection,
>> + },
>> +};
>> +
>> +static int mali_c55_params_enum_fmt_meta_out(struct file *file, void *fh,
>> + struct v4l2_fmtdesc *f)
>> +{
>> + if (f->index)
>> + return -EINVAL;
>> +
>> + f->pixelformat = V4L2_META_FMT_MALI_C55_PARAMS;
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_params_g_fmt_meta_out(struct file *file, void *fh,
>> + struct v4l2_format *f)
>> +{
>> + static const struct v4l2_meta_format mfmt = {
>> + .dataformat = V4L2_META_FMT_MALI_C55_PARAMS,
>> + .buffersize = sizeof(struct mali_c55_params_buffer),
>> + };
>> +
>> + f->fmt.meta = mfmt;
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_params_querycap(struct file *file,
>> + void *priv, struct v4l2_capability *cap)
>> +{
>> + strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver));
>> + strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card));
>> +
>> + return 0;
>> +}
>> +
>> +static const struct v4l2_ioctl_ops mali_c55_params_v4l2_ioctl_ops = {
>> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
>> + .vidioc_querybuf = vb2_ioctl_querybuf,
>> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
>> + .vidioc_qbuf = vb2_ioctl_qbuf,
>> + .vidioc_expbuf = vb2_ioctl_expbuf,
>> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
>> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>> + .vidioc_streamon = vb2_ioctl_streamon,
>> + .vidioc_streamoff = vb2_ioctl_streamoff,
>> + .vidioc_enum_fmt_meta_out = mali_c55_params_enum_fmt_meta_out,
>> + .vidioc_g_fmt_meta_out = mali_c55_params_g_fmt_meta_out,
>> + .vidioc_s_fmt_meta_out = mali_c55_params_g_fmt_meta_out,
>> + .vidioc_try_fmt_meta_out = mali_c55_params_g_fmt_meta_out,
>> + .vidioc_querycap = mali_c55_params_querycap,
>> + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
>> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
>> +};
>> +
>> +static const struct v4l2_file_operations mali_c55_params_v4l2_fops = {
>> + .owner = THIS_MODULE,
>> + .unlocked_ioctl = video_ioctl2,
>> + .open = v4l2_fh_open,
>> + .release = vb2_fop_release,
>> + .poll = vb2_fop_poll,
>> + .mmap = vb2_fop_mmap,
>> +};
>> +
>> +static int
>> +mali_c55_params_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
>> + unsigned int *num_planes, unsigned int sizes[],
>> + struct device *alloc_devs[])
>> +{
>> + if (*num_planes && *num_planes > 1)
>> + return -EINVAL;
>> +
>> + if (sizes[0] && sizes[0] < sizeof(struct mali_c55_params_buffer))
>> + return -EINVAL;
>> +
>> + *num_planes = 1;
>> +
>> + if (!sizes[0])
>> + sizes[0] = sizeof(struct mali_c55_params_buffer);
>> +
>> + return 0;
>> +}
>> +
>> +static void mali_c55_params_buf_queue(struct vb2_buffer *vb)
>> +{
>> + struct mali_c55_params *params = vb2_get_drv_priv(vb->vb2_queue);
>> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>> + struct mali_c55_buffer *buf = container_of(vbuf,
>> + struct mali_c55_buffer, vb);
>> +
>> + vb2_set_plane_payload(vb, 0, sizeof(struct mali_c55_params_buffer));
>> +
>> + spin_lock(¶ms->buffers.lock);
>> + list_add_tail(&buf->queue, ¶ms->buffers.queue);
>> + spin_unlock(¶ms->buffers.lock);
>> +}
>> +
>> +static int mali_c55_params_start_streaming(struct vb2_queue *q,
>> + unsigned int count)
>> +{
>> + struct mali_c55_params *params = vb2_get_drv_priv(q);
>> + struct mali_c55 *mali_c55 = params->mali_c55;
>> + int ret;
>> +
>> + ret = video_device_pipeline_start(¶ms->vdev,
>> + ¶ms->mali_c55->pipe);
>> + if (ret)
>> + return ret;
>> +
>> + if (mali_c55->pipe.start_count == mali_c55->pipe.required_count) {
>> + ret = v4l2_subdev_enable_streams(&mali_c55->isp.sd,
>> + MALI_C55_ISP_PAD_SOURCE_VIDEO,
>> + BIT(0));
>> + if (ret)
>> + goto err_stop_pipeline;
>> + }
>> +
>> + return 0;
>> +
>> +err_stop_pipeline:
>> + video_device_pipeline_stop(¶ms->vdev);
> When a failure happens you need to return all queued buffers to vb2 in
> the QUEUED state.
>
>> +
>> + return ret;
>> +}
>> +
>> +static void mali_c55_params_stop_streaming(struct vb2_queue *q)
>> +{
>> + struct mali_c55_params *params = vb2_get_drv_priv(q);
>> + struct mali_c55_buffer *buf, *tmp;
>> +
>> + spin_lock(¶ms->buffers.lock);
>> +
>> + list_for_each_entry_safe(buf, tmp, ¶ms->buffers.queue, queue) {
>> + list_del(&buf->queue);
>> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
>> + }
> This can be turned into a helper function that takes the state as an
> argument, to be used in the start_streaming error path. It may even be
> possible to share the helper with all video devices in this driver.
>
>> +
>> + spin_unlock(¶ms->buffers.lock);
>> +
>> + video_device_pipeline_stop(¶ms->vdev);
> I think you need to stop the ISP too, if this is the first video device
> being stopped. You should return buffers to vb2 only after stopping the
> ISP.
>
> This may be fixed by stopping the ISP, but currently I think you're
> racing with mali_c55_params_write_config()
>
>> +}
>> +
>> +static const struct vb2_ops mali_c55_params_vb2_ops = {
>> + .queue_setup = mali_c55_params_queue_setup,
>> + .buf_queue = mali_c55_params_buf_queue,
>> + .wait_prepare = vb2_ops_wait_prepare,
>> + .wait_finish = vb2_ops_wait_finish,
>> + .start_streaming = mali_c55_params_start_streaming,
>> + .stop_streaming = mali_c55_params_stop_streaming,
>> +};
>> +
>> +void mali_c55_params_write_config(struct mali_c55 *mali_c55)
>> +{
>> + struct mali_c55_params *params = &mali_c55->params;
>> + enum vb2_buffer_state state = VB2_BUF_STATE_DONE;
>> + struct mali_c55_params_buffer *config;
>> + struct mali_c55_buffer *buf;
>> + size_t block_offset = 0;
>> + size_t max_offset;
>> +
>> + spin_lock(¶ms->buffers.lock);
>> +
>> + buf = list_first_entry_or_null(¶ms->buffers.queue,
>> + struct mali_c55_buffer, queue);
>> + if (buf)
>> + list_del(&buf->queue);
>> + spin_unlock(¶ms->buffers.lock);
>> +
>> + if (!buf)
>> + return;
> If this happens we'll lose synchronization. I suppose we can't do much
> about this, we really need a request-based API.
>
>> +
>> + buf->vb.sequence = mali_c55->isp.frame_sequence;
>> + config = vb2_plane_vaddr(&buf->vb.vb2_buf, 0);
>> +
>> + if (config->total_size > MALI_C55_PARAMS_MAX_SIZE) {
>> + dev_dbg(mali_c55->dev, "Invalid parameters buffer size %u\n",
>> + config->total_size);
>> + state = VB2_BUF_STATE_ERROR;
>> + goto err_buffer_done;
>> + }
>> +
>> + max_offset = config->total_size - sizeof(struct mali_c55_params_block_header);
>> +
>> + /* Walk the list of parameter blocks and process them. */
>> + while (block_offset < max_offset) {
>> + const struct mali_c55_block_handler *block_handler;
>> + union mali_c55_params_block block;
>> +
>> + block = (union mali_c55_params_block)
>> + &config->data[block_offset];
>> +
>> + if (block.header->type >= MALI_C55_PARAM_BLOCK_SENTINEL) {
>> + dev_dbg(mali_c55->dev, "Invalid parameters block type\n");
>> + state = VB2_BUF_STATE_ERROR;
>> + goto err_buffer_done;
>> + }
>> +
>> + if (block_offset + block.header->size >= config->total_size) {
>> + dev_dbg(mali_c55->dev, "Parameters block too large\n");
>> + state = VB2_BUF_STATE_ERROR;
>> + goto err_buffer_done;
>> + }
>> +
>> + block_handler = &mali_c55_block_handlers[block.header->type];
>> + if (block.header->size != block_handler->size) {
>> + dev_dbg(mali_c55->dev, "Invalid parameters block size\n");
>> + state = VB2_BUF_STATE_ERROR;
>> + goto err_buffer_done;
>> + }
> Most of the validation should be done with the buffer is queued.
> Furthermore, you need to make a copy of the data. See the latest version
> of the rkisp1 extensible parameters format series. There may be other
> recent improvements in that series worth copying here.
>
>> +
>> + block_handler->handler(mali_c55, block);
>> +
>> + block_offset += block.header->size;
>> + }
>> +
>> +err_buffer_done:
>> + vb2_buffer_done(&buf->vb.vb2_buf, state);
>> +}
>> +
>> +void mali_c55_unregister_params(struct mali_c55 *mali_c55)
>> +{
>> + struct mali_c55_params *params = &mali_c55->params;
>> +
>> + if (!video_is_registered(¶ms->vdev))
>> + return;
>> +
>> + vb2_video_unregister_device(¶ms->vdev);
>> + media_entity_cleanup(¶ms->vdev.entity);
>> + mutex_destroy(¶ms->lock);
>> +}
>> +
>> +int mali_c55_register_params(struct mali_c55 *mali_c55)
>> +{
>> + struct mali_c55_params *params = &mali_c55->params;
>> + struct video_device *vdev = ¶ms->vdev;
>> + struct vb2_queue *vb2q = ¶ms->queue;
>> + int ret;
>> +
>> + mutex_init(¶ms->lock);
>> + INIT_LIST_HEAD(¶ms->buffers.queue);
>> +
>> + params->pad.flags = MEDIA_PAD_FL_SOURCE;
>> + ret = media_entity_pads_init(¶ms->vdev.entity, 1, ¶ms->pad);
>> + if (ret)
>> + goto err_destroy_mutex;
>> +
>> + vb2q->type = V4L2_BUF_TYPE_META_OUTPUT;
>> + vb2q->io_modes = VB2_MMAP | VB2_DMABUF;
>> + vb2q->drv_priv = params;
>> + vb2q->mem_ops = &vb2_dma_contig_memops;
>> + vb2q->ops = &mali_c55_params_vb2_ops;
>> + vb2q->buf_struct_size = sizeof(struct mali_c55_buffer);
>> + vb2q->min_queued_buffers = 1;
>> + vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
>> + vb2q->lock = ¶ms->lock;
>> + vb2q->dev = mali_c55->dev;
>> +
>> + ret = vb2_queue_init(vb2q);
>> + if (ret) {
>> + dev_err(mali_c55->dev, "params vb2 queue init failed\n");
>> + goto err_cleanup_entity;
>> + }
>> +
>> + strscpy(params->vdev.name, "mali-c55 3a params",
>> + sizeof(params->vdev.name));
>> + vdev->release = video_device_release_empty;
>> + vdev->fops = &mali_c55_params_v4l2_fops;
>> + vdev->ioctl_ops = &mali_c55_params_v4l2_ioctl_ops;
>> + vdev->lock = ¶ms->lock;
>> + vdev->v4l2_dev = &mali_c55->v4l2_dev;
>> + vdev->queue = ¶ms->queue;
>> + vdev->device_caps = V4L2_CAP_META_OUTPUT | V4L2_CAP_STREAMING;
> Shouldn't you set V4L2_CAP_IO_MC and implement media bus code-based
> format enumeration ?
>
>> + vdev->vfl_dir = VFL_DIR_TX;
>> + video_set_drvdata(vdev, params);
>> +
>> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>> + if (ret) {
>> + dev_err(mali_c55->dev,
>> + "failed to register params video device\n");
>> + goto err_release_vb2q;
>> + }
>> +
>> + params->mali_c55 = mali_c55;
>> +
>> + return 0;
>> +
>> +err_release_vb2q:
>> + vb2_queue_release(vb2q);
>> +err_cleanup_entity:
>> + media_entity_cleanup(¶ms->vdev.entity);
>> +err_destroy_mutex:
>> + mutex_destroy(¶ms->lock);
>> +
>> + return ret;
>> +}
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>> index e72e749b81d5..f2cad402492c 100644
>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>> @@ -159,6 +159,23 @@ enum mali_c55_interrupts {
>> #define MALI_C55_BAYER_ORDER_GBRG 2
>> #define MALI_C55_BAYER_ORDER_BGGR 3
>>
>> +#define MALI_C55_REG_METERING_CONFIG 0x18ed0
>> +#define MALI_C55_5BIN_HIST_DISABLE_MASK BIT(0)
>> +#define MALI_C55_5BIN_HIST_SWITCH_MASK GENMASK(2, 1)
>> +#define MALI_C55_5BIN_HIST_SWITCH(x) 1
>> +#define MALI_C55_AF_DISABLE_MASK BIT(4)
>> +#define MALI_C55_AF_SWITCH_MASK BIT(5)
>> +#define MALI_C55_AWB_DISABLE_MASK BIT(8)
>> +#define MALI_C55_AWB_SWITCH_MASK BIT(9)
>> +#define MALI_C55_AWB_SWITCH(x) ((x) << 9)
>> +#define MALI_C55_AEXP_HIST_DISABLE_MASK BIT(12)
>> +#define MALI_C55_AEXP_HIST_DISABLE (0x01 << 12)
>> +#define MALI_C55_AEXP_HIST_SWITCH_MASK GENMASK(14, 13)
>> +#define MALI_C55_AEXP_HIST_SWITCH(x) ((x) << 13)
>> +#define MALI_C55_AEXP_IHIST_DISABLE_MASK BIT(16)
>> +#define MALI_C55_AEXP_IHIST_DISABLE (0x01 << 12)
>> +#define MALI_C55_AEXP_SRC_MASK BIT(24)
>> +
>> #define MALI_C55_REG_TPG_CH0 0x18ed8
>> #define MALI_C55_TEST_PATTERN_ON_OFF BIT(0)
>> #define MALI_C55_TEST_PATTERN_RGB_MASK BIT(1)
>> @@ -179,6 +196,11 @@ enum mali_c55_interrupts {
>> #define MALI_C55_REG_CONFIG_SPACES_OFFSET 0x0ab6c
>> #define MALI_C55_CONFIG_SPACE_SIZE 0x1231c
>>
>> +#define MALI_C55_REG_DIGITAL_GAIN 0x1926c
>> +#define MALI_C55_DIGITAL_GAIN_MASK GENMASK(12, 0)
>> +#define MALI_C55_REG_DIGITAL_GAIN_OFFSET 0x19270
>> +#define MALI_C55_DIGITAL_GAIN_OFFSET_MASK GENMASK(19, 0)
>> +
>> #define MALI_C55_REG_SINTER_CONFIG 0x19348
>> #define MALI_C55_SINTER_VIEW_FILTER_MASK GENMASK(1, 0)
>> #define MALI_C55_SINTER_SCALE_MODE_MASK GENMASK(3, 2)
>> @@ -187,6 +209,59 @@ enum mali_c55_interrupts {
>> #define MALI_C55_SINTER_INT_SELECT_MASK BIT(6)
>> #define MALI_C55_SINTER_RM_ENABLE_MASK BIT(7)
>>
>> +/* Black Level Correction Configuration */
>> +#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_00 0x1abcc
>> +#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_01 0x1abd0
>> +#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_10 0x1abd4
>> +#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_11 0x1abd8
>> +#define MALI_C55_SENSOR_OFF_PRE_SHA_MASK 0xfffff
>> +
>> +/* Lens Mesh Shading Configuration */
>> +#define MALI_C55_REG_MESH_SHADING_TABLES 0x13074
>> +#define MALI_C55_REG_MESH_SHADING_CONFIG 0x1abfc
>> +#define MALI_C55_MESH_SHADING_ENABLE_MASK BIT(0)
>> +#define MALI_C55_MESH_SHADING_MESH_SHOW_MASK BIT(1)
>> +#define MALI_C55_MESH_SHADING_MESH_SHOW(x) ((x) << 1)
>> +#define MALI_C55_MESH_SHADING_SCALE_MASK GENMASK(4, 2)
>> +#define MALI_C55_MESH_SHADING_SCALE(x) ((x) << 2)
>> +#define MALI_C55_MESH_SHADING_PAGE_R_MASK GENMASK(9, 8)
>> +#define MALI_C55_MESH_SHADING_PAGE_R(x) ((x) << 8)
>> +#define MALI_C55_MESH_SHADING_PAGE_G_MASK GENMASK(11, 10)
>> +#define MALI_C55_MESH_SHADING_PAGE_G(x) ((x) << 10)
>> +#define MALI_C55_MESH_SHADING_PAGE_B_MASK GENMASK(13, 12)
>> +#define MALI_C55_MESH_SHADING_PAGE_B(x) ((x) << 12)
>> +#define MALI_C55_MESH_SHADING_MESH_WIDTH_MASK GENMASK(21, 16)
>> +#define MALI_C55_MESH_SHADING_MESH_WIDTH(x) ((x) << 16)
>> +#define MALI_C55_MESH_SHADING_MESH_HEIGHT_MASK GENMASK(29, 24)
>> +#define MALI_C55_MESH_SHADING_MESH_HEIGHT(x) ((x) << 24)
>> +
>> +#define MALI_C55_REG_MESH_SHADING_ALPHA_BANK 0x1ac04
>> +#define MALI_C55_MESH_SHADING_ALPHA_BANK_R_MASK GENMASK(2, 0)
>> +#define MALI_C55_MESH_SHADING_ALPHA_BANK_G_MASK GENMASK(5, 3)
>> +#define MALI_C55_MESH_SHADING_ALPHA_BANK_G(x) ((x) << 3)
>> +#define MALI_C55_MESH_SHADING_ALPHA_BANK_B_MASK GENMASK(8, 6)
>> +#define MALI_C55_MESH_SHADING_ALPHA_BANK_B(x) ((x) << 6)
>> +#define MALI_C55_REG_MESH_SHADING_ALPHA 0x1ac08
>> +#define MALI_C55_MESH_SHADING_ALPHA_R_MASK GENMASK(7, 0)
>> +#define MALI_C55_MESH_SHADING_ALPHA_G_MASK GENMASK(15, 8)
>> +#define MALI_C55_MESH_SHADING_ALPHA_G(x) ((x) << 8)
>> +#define MALI_C55_MESH_SHADING_ALPHA_B_MASK GENMASK(23, 16)
>> +#define MALI_C55_MESH_SHADING_ALPHA_B(x) ((x) << 16)
>> +#define MALI_C55_REG_MESH_SHADING_MESH_STRENGTH 0x1ac0c
>> +#define MALI_c55_MESH_STRENGTH_MASK GENMASK(15, 0)
>> +
>> +/* AWB Gains Configuration */
>> +#define MALI_C55_REG_AWB_GAINS1 0x1ac10
>> +#define MALI_C55_AWB_GAIN00_MASK GENMASK(11, 0)
>> +#define MALI_C55_AWB_GAIN01_MASK GENMASK(27, 16)
>> +#define MALI_C55_AWB_GAIN01(x) ((x) << 16)
>> +#define MALI_C55_REG_AWB_GAINS2 0x1ac14
>> +#define MALI_C55_AWB_GAIN10_MASK GENMASK(11, 0)
>> +#define MALI_C55_AWB_GAIN11_MASK GENMASK(27, 16)
>> +#define MALI_C55_AWB_GAIN11(x) ((x) << 16)
>> +#define MALI_C55_REG_AWB_GAINS1_AEXP 0x1ac18
>> +#define MALI_C55_REG_AWB_GAINS2_AEXP 0x1ac1c
>> +
>> /* Colour Correction Matrix Configuration */
>> #define MALI_C55_REG_CCM_ENABLE 0x1b07c
>> #define MALI_C55_CCM_ENABLE_MASK BIT(0)
>> @@ -209,6 +284,59 @@ enum mali_c55_interrupts {
>> #define MALI_C55_REG_CCM_ANTIFOG_OFFSET_B 0x1b0c8
>> #define MALI_C55_CCM_ANTIFOG_OFFSET_MASK GENMASK(11, 0)
>>
>> +/* AWB Statistics Configuration */
>> +#define MALI_C55_REG_AWB_STATS_MODE 0x1b29c
>> +#define MALI_C55_AWB_STATS_MODE_MASK BIT(0)
>> +#define MALI_C55_REG_AWB_WHITE_LEVEL 0x1b2a0
>> +#define MALI_C55_AWB_WHITE_LEVEL_MASK GENMASK(9, 0)
>> +#define MALI_C55_REG_AWB_BLACK_LEVEL 0x1b2a4
>> +#define MALI_C55_AWB_BLACK_LEVEL_MASK GENMASK(9, 0)
>> +#define MALI_C55_REG_AWB_CR_MAX 0x1b2a8
>> +#define MALI_C55_AWB_CR_MAX_MASK GENMASK(11, 0)
>> +#define MALI_C55_REG_AWB_CR_MIN 0x1b2ac
>> +#define MALI_C55_AWB_CR_MIN_MASK GENMASK(11, 0)
>> +#define MALI_C55_REG_AWB_CB_MAX 0x1b2b0
>> +#define MALI_C55_AWB_CB_MAX_MASK GENMASK(11, 0)
>> +#define MALI_C55_REG_AWB_CB_MIN 0x1b2b4
>> +#define MALI_C55_AWB_CB_MIN_MASK GENMASK(11, 0)
>> +#define MALI_C55_REG_AWB_NODES_USED 0x1b2c4
>> +#define MALI_C55_AWB_NODES_USED_HORIZ_MASK GENMASK(7, 0)
>> +#define MALI_C55_AWB_NODES_USED_VERT_MASK GENMASK(15, 8)
>> +#define MALI_C55_AWB_NODES_USED_VERT(x) ((x) << 8)
>> +#define MALI_C55_REG_AWB_CR_HIGH 0x1b2c8
>> +#define MALI_C55_AWB_CR_HIGH_MASK GENMASK(11, 0)
>> +#define MALI_C55_REG_AWB_CR_LOW 0x1b2cc
>> +#define MALI_C55_AWB_CR_LOW_MASK GENMASK(11, 0)
>> +#define MALI_C55_REG_AWB_CB_HIGH 0x1b2d0
>> +#define MALI_C55_AWB_CB_HIGH_MASK GENMASK(11, 0)
>> +#define MALI_C55_REG_AWB_CB_LOW 0x1b2d4
>> +#define MALI_C55_AWB_CB_LOW_MASK GENMASK(11, 0)
>> +
>> +/* AEXP Metering Histogram Configuration */
>> +#define MALI_C55_REG_AEXP_HIST_BASE 0x1b730
>> +#define MALI_C55_REG_AEXP_IHIST_BASE 0x1bbac
>> +#define MALI_C55_AEXP_HIST_SKIP_OFFSET 0
>> +#define MALI_C55_AEXP_HIST_SKIP_X_MASK GENMASK(2, 0)
>> +#define MALI_c55_AEXP_HIST_SKIP_X(x) 0
>> +#define MALI_C55_AEXP_HIST_OFFSET_X_MASK BIT(3)
>> +#define MALI_C55_AEXP_HIST_OFFSET_X(x) ((x) << 3)
>> +#define MALI_C55_AEXP_HIST_SKIP_Y_MASK GENMASK(6, 4)
>> +#define MALI_C55_AEXP_HIST_SKIP_Y(x) ((x) << 4)
>> +#define MALI_C55_AEXP_HIST_OFFSET_Y_MASK BIT(7)
>> +#define MALI_C55_AEXP_HIST_OFFSET_Y(x) ((x) << 7)
>> +#define MALI_C55_AEXP_HIST_SCALE_OFFSET 4
>> +#define MALI_C55_AEXP_HIST_SCALE_BOTTOM_MASK GENMASK(3, 0)
>> +#define MALI_C55_AEXP_HIST_SCALE_TOP_MASK GENMASK(7, 4)
>> +#define MALI_C55_AEXP_HIST_SCALE_TOP(x) ((x) << 4)
>> +#define MALI_C55_AEXP_HIST_PLANE_MODE_OFFSET 16
>> +#define MALI_C55_AEXP_HIST_PLANE_MODE_MASK GENMASK(2, 0)
>> +#define MALI_C55_AEXP_HIST_NODES_USED_OFFSET 52
>> +#define MALI_C55_AEXP_HIST_NODES_USED_HORIZ_MASK GENMASK(7, 0)
>> +#define MALI_C55_AEXP_HIST_NODES_USED_VERT_MASK GENMASK(15, 8)
>> +#define MALI_C55_AEXP_HIST_NODES_USED_VERT(x) ((x) << 8)
>> +#define MALI_C55_AEXP_HIST_ZONE_WEIGHTS_OFFSET 56
>> +#define MALI_C55_AEXP_HIST_ZONE_WEIGHT_MASK 0x0f0f0f0f
>> +
>> /*
>> * The Mali-C55 ISP has up to two output pipes; known as full resolution and
>> * down scaled. The register space for these is laid out identically, but offset
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 17/18] media: platform: Add mali-c55 parameters video node
2024-07-31 15:12 ` Dan Scally
@ 2024-08-02 15:03 ` Laurent Pinchart
0 siblings, 0 replies; 41+ messages in thread
From: Laurent Pinchart @ 2024-08-02 15:03 UTC (permalink / raw)
To: Dan Scally
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Dan,
On Wed, Jul 31, 2024 at 04:12:41PM +0100, Daniel Scally wrote:
> On 30/07/2024 23:16, Laurent Pinchart wrote:
> > On Tue, Jul 09, 2024 at 02:29:05PM +0100, Daniel Scally wrote:
> >> Add a new code file to the mali-c55 driver that registers an output
> >> video node for userspace to queue buffers of parameters to. Handlers
> >> are included to program the statistics generation plus the white
> >> balance, black level correction and mesh shading correction blocks.
> >>
> >> Update the rest of the driver to register and link the new video node
> >>
> >> Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
> >> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> >> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> >> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
> >> ---
> >> Changes in v6:
> >>
> >> - Used a union to generalise the block pointer rather than resorting to
> >> casting everywhere - fantastic idea Sakari, this made it much cleaner.
> >> - Reworked the loop in mali_c55_params_write_config() so that we can be
> >> sure there's remaining space for the next block header.
> >>
> >> Changes in v5:
> >>
> >> - New patch
> >>
> >> drivers/media/platform/arm/mali-c55/Makefile | 1 +
> >> .../platform/arm/mali-c55/mali-c55-common.h | 20 +
> >> .../platform/arm/mali-c55/mali-c55-core.c | 23 +
> >> .../platform/arm/mali-c55/mali-c55-isp.c | 21 +-
> >> .../platform/arm/mali-c55/mali-c55-params.c | 671 ++++++++++++++++++
> >> .../arm/mali-c55/mali-c55-registers.h | 128 ++++
> >> 6 files changed, 863 insertions(+), 1 deletion(-)
> >> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-params.c
[snip]
> >> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> >> index ed0db34767a4..55b3cbf53791 100644
> >> --- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> >> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> >> @@ -384,6 +384,16 @@ static int mali_c55_create_links(struct mali_c55 *mali_c55)
> >> goto err_remove_links;
> >> }
> >>
> >> + ret = media_create_pad_link(&mali_c55->params.vdev.entity, 0,
> >> + &mali_c55->isp.sd.entity,
> >> + MALI_C55_ISP_PAD_SINK_PARAMS,
> >> + MEDIA_LNK_FL_ENABLED);
> > Should this be immutable, or do you think it makes sense to support
> > operating the ISP without parameters ? I know we did so when developing
> > the driver to test the initial code, but are there real use cases now ?
>
> If all you're interested in is RAW data bypassed through the ISP then
> you could skip the parameters and statistics - which would be the case
> for RGB/YUV cameras feeding the ISP too. Is that sufficient use-case
> to leave them mutable?
I suppose that's a good point, yes. Fine with me.
> >> + if (ret) {
> >> + dev_err(mali_c55->dev,
> >> + "failed to link ISP and parameters video node\n");
> >> + goto err_remove_links;
> >> + }
> >> +
> >> return 0;
> >>
> >> err_remove_links:
[snip]
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 12/18] media: platform: Add mali-c55 3a stats devnode
2024-07-22 22:55 ` Laurent Pinchart
@ 2024-08-13 13:13 ` Dan Scally
2024-08-25 20:20 ` Laurent Pinchart
0 siblings, 1 reply; 41+ messages in thread
From: Dan Scally @ 2024-08-13 13:13 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Laurent, thanks for the review
On 22/07/2024 23:55, Laurent Pinchart wrote:
> On Mon, Jul 22, 2024 at 05:51:00PM +0300, Laurent Pinchart wrote:
>> Hi Dan,
>>
>> Thank you for the patch.
>>
>> On Tue, Jul 09, 2024 at 02:29:00PM +0100, Daniel Scally wrote:
>>> Add a new code file to govern the 3a statistics capture node.
>>>
>>> Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
>>> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>>> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>>> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
>>> ---
>>> Changes in v6:
>>>
>>> - Fixed mising includes
>>> - Minor renames and formatting
>>> - Reworked mali_c55_stats_metering_complete() so it could only return
>>> buffers when both halves of the DMA read were done
>>> - Terminate dma transfers on streamoff
>>>
>>> Changes in v5:
>>>
>>> - New patch
>>>
>>> drivers/media/platform/arm/mali-c55/Makefile | 1 +
>>> .../platform/arm/mali-c55/mali-c55-common.h | 29 ++
>>> .../platform/arm/mali-c55/mali-c55-core.c | 15 +
>>> .../platform/arm/mali-c55/mali-c55-isp.c | 11 +
>>> .../arm/mali-c55/mali-c55-registers.h | 3 +
>>> .../platform/arm/mali-c55/mali-c55-stats.c | 373 ++++++++++++++++++
>>> 6 files changed, 432 insertions(+)
>>> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-stats.c
>>>
>>> diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile
>>> index 9178ac35e50e..b5a22d414479 100644
>>> --- a/drivers/media/platform/arm/mali-c55/Makefile
>>> +++ b/drivers/media/platform/arm/mali-c55/Makefile
>>> @@ -4,6 +4,7 @@ mali-c55-y := mali-c55-capture.o \
>>> mali-c55-core.o \
>>> mali-c55-isp.o \
>>> mali-c55-resizer.o \
>>> + mali-c55-stats.o \
>>> mali-c55-tpg.o
>>>
>>> obj-$(CONFIG_VIDEO_MALI_C55) += mali-c55.o
>>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
>>> index f7764a938e9f..136c785c68ba 100644
>>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-common.h
>>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
>>> @@ -49,6 +49,7 @@ enum mali_c55_isp_pads {
>>> MALI_C55_ISP_PAD_SINK_VIDEO,
>>> MALI_C55_ISP_PAD_SOURCE_VIDEO,
>>> MALI_C55_ISP_PAD_SOURCE_BYPASS,
>>> + MALI_C55_ISP_PAD_SOURCE_STATS,
>>> MALI_C55_ISP_NUM_PADS,
>>> };
>>>
>>> @@ -160,6 +161,29 @@ struct mali_c55_cap_dev {
>>> bool streaming;
>>> };
>>>
>>> +struct mali_c55_stats_buf {
>>> + struct vb2_v4l2_buffer vb;
>>> + unsigned int segments_remaining;
>>> + struct list_head queue;
>>> + bool failed;
>>> +};
>>> +
>>> +struct mali_c55_stats {
>>> + struct mali_c55 *mali_c55;
>>> + struct video_device vdev;
>>> + struct dma_chan *channel;
>>> + struct vb2_queue queue;
>>> + struct media_pad pad;
>>> + /* Mutex to provide to vb2 */
>>> + struct mutex lock;
>>> +
>>> + struct {
>>> + /* Spinlock to guard buffer queue */
>>> + spinlock_t lock;
>>> + struct list_head queue;
>>> + } buffers;
>>> +};
>>> +
>>> enum mali_c55_config_spaces {
>>> MALI_C55_CONFIG_PING,
>>> MALI_C55_CONFIG_PONG,
>>> @@ -202,6 +226,7 @@ struct mali_c55 {
>>> struct mali_c55_isp isp;
>>> struct mali_c55_resizer resizers[MALI_C55_NUM_RSZS];
>>> struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS];
>>> + struct mali_c55_stats stats;
>>>
>>> struct mali_c55_context context;
>>> enum mali_c55_config_spaces next_config;
>>> @@ -229,6 +254,8 @@ int mali_c55_register_resizers(struct mali_c55 *mali_c55);
>>> void mali_c55_unregister_resizers(struct mali_c55 *mali_c55);
>>> int mali_c55_register_capture_devs(struct mali_c55 *mali_c55);
>>> void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55);
>>> +int mali_c55_register_stats(struct mali_c55 *mali_c55);
>>> +void mali_c55_unregister_stats(struct mali_c55 *mali_c55);
>>> struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55);
>>> void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
>>> enum mali_c55_planes plane);
>>> @@ -243,5 +270,7 @@ const struct mali_c55_isp_fmt *
>>> mali_c55_isp_get_mbus_config_by_code(u32 code);
>>> const struct mali_c55_isp_fmt *
>>> mali_c55_isp_get_mbus_config_by_index(u32 index);
>>> +void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
>>> + enum mali_c55_config_spaces cfg_space);
>>>
>>> #endif /* _MALI_C55_COMMON_H */
>>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
>>> index dbc07d12d3a3..eedc8f450184 100644
>>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
>>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
>>> @@ -374,6 +374,16 @@ static int mali_c55_create_links(struct mali_c55 *mali_c55)
>>> }
>>> }
>>>
>>> + ret = media_create_pad_link(&mali_c55->isp.sd.entity,
>>> + MALI_C55_ISP_PAD_SOURCE_STATS,
>>> + &mali_c55->stats.vdev.entity, 0,
>>> + MEDIA_LNK_FL_ENABLED);
>>> + if (ret) {
>>> + dev_err(mali_c55->dev,
>>> + "failed to link ISP and 3a stats node\n");
>>> + goto err_remove_links;
>>> + }
>>> +
>>> return 0;
>>>
>>> err_remove_links:
>>> @@ -388,6 +398,7 @@ static void mali_c55_unregister_entities(struct mali_c55 *mali_c55)
>>> mali_c55_unregister_isp(mali_c55);
>>> mali_c55_unregister_resizers(mali_c55);
>>> mali_c55_unregister_capture_devs(mali_c55);
>>> + mali_c55_unregister_stats(mali_c55);
>>> }
>>>
>>> static int mali_c55_register_entities(struct mali_c55 *mali_c55)
>>> @@ -410,6 +421,10 @@ static int mali_c55_register_entities(struct mali_c55 *mali_c55)
>>> if (ret)
>>> goto err_unregister_entities;
>>>
>>> + ret = mali_c55_register_stats(mali_c55);
>>> + if (ret)
>>> + goto err_unregister_entities;
>>> +
>>> ret = mali_c55_create_links(mali_c55);
>>> if (ret)
>>> goto err_unregister_entities;
>>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>>> index f784983a4ccc..2f450c00300a 100644
>>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>>> @@ -5,6 +5,8 @@
>>> * Copyright (C) 2024 Ideas on Board Oy
>>> */
>>>
>>> +#include <linux/media/arm/mali-c55-config.h>
>>> +
>>> #include <linux/delay.h>
>>> #include <linux/iopoll.h>
>>> #include <linux/property.h>
>>> @@ -460,6 +462,14 @@ static int mali_c55_isp_init_state(struct v4l2_subdev *sd,
>>> in_crop->width = MALI_C55_DEFAULT_WIDTH;
>>> in_crop->height = MALI_C55_DEFAULT_HEIGHT;
>>>
>>> + src_fmt = v4l2_subdev_state_get_format(state,
>>> + MALI_C55_ISP_PAD_SOURCE_STATS);
>>> +
>>> + src_fmt->width = sizeof(struct mali_c55_stats_buffer);
>>> + src_fmt->height = 1;
>> According to
>> https://docs.kernel.org/userspace-api/media/v4l/subdev-formats.html#metadata-formats,
>> width and height should be set to 0 for MEDIA_BUS_FMT_METADATA_FIXED. I
>> haven't checked if v4l2-compliance expects this, we may have
>> discrepancies between the API documentation and the existing
>> implementations in the kernel and userspace. In any case, this should be
>> sorted out, either by fixing the kernel code and enforcing the
>> requirement in v4l2-compliance, or fixing the documentation.
>>
>>> + src_fmt->field = V4L2_FIELD_NONE;
>>> + src_fmt->code = MEDIA_BUS_FMT_METADATA_FIXED;
>>> +
>>> return 0;
>>> }
>>>
>>> @@ -490,6 +500,7 @@ int mali_c55_register_isp(struct mali_c55 *mali_c55)
>>> MEDIA_PAD_FL_MUST_CONNECT;
>>> isp->pads[MALI_C55_ISP_PAD_SOURCE_VIDEO].flags = MEDIA_PAD_FL_SOURCE;
>>> isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE;
>>> + isp->pads[MALI_C55_ISP_PAD_SOURCE_STATS].flags = MEDIA_PAD_FL_SOURCE;
>>>
>>> ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS,
>>> isp->pads);
>>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>>> index 0a391f8a043e..e72e749b81d5 100644
>>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>>> @@ -103,6 +103,9 @@ enum mali_c55_interrupts {
>>> #define MALI_C55_VC_START(v) ((v) & 0xffff)
>>> #define MALI_C55_VC_SIZE(v) (((v) & 0xffff) << 16)
>>>
>>> +#define MALI_C55_REG_1024BIN_HIST 0x054a8
>>> +#define MALI_C55_1024BIN_HIST_SIZE 4096
>>> +
>>> /* Ping/Pong Configuration Space */
>>> #define MALI_C55_REG_BASE_ADDR 0x18e88
>>> #define MALI_C55_REG_BYPASS_0 0x18eac
>>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-stats.c b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
>>> new file mode 100644
>>> index 000000000000..38a17fb5d052
>>> --- /dev/null
>>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
>>> @@ -0,0 +1,373 @@
>>> +// SPDX-License-Identifier: GPL-2.0
>>> +/*
>>> + * ARM Mali-C55 ISP Driver - 3A Statistics capture device
>>> + *
>>> + * Copyright (C) 2023 Ideas on Board Oy
>>> + */
>>> +
>>> +#include <linux/container_of.h>
>>> +#include <linux/dev_printk.h>
>>> +#include <linux/dmaengine.h>
>>> +#include <linux/list.h>
>>> +#include <linux/media/arm/mali-c55-config.h>
>>> +#include <linux/mutex.h>
>>> +#include <linux/spinlock.h>
>>> +#include <linux/string.h>
>>> +
>>> +#include <media/media-entity.h>
>>> +#include <media/v4l2-dev.h>
>>> +#include <media/v4l2-event.h>
>>> +#include <media/v4l2-fh.h>
>>> +#include <media/v4l2-ioctl.h>
>>> +#include <media/videobuf2-core.h>
>>> +#include <media/videobuf2-dma-contig.h>
>>> +
>>> +#include "mali-c55-common.h"
>>> +#include "mali-c55-registers.h"
>>> +
>>> +static const unsigned int metering_space_addrs[] = {
>>> + [MALI_C55_CONFIG_PING] = 0x095ac,
>>> + [MALI_C55_CONFIG_PONG] = 0x2156c,
>>> +};
>>> +
>>> +static int mali_c55_stats_enum_fmt_meta_cap(struct file *file, void *fh,
>>> + struct v4l2_fmtdesc *f)
>>> +{
>>> + if (f->index)
>>> + return -EINVAL;
>>> +
>>> + f->pixelformat = V4L2_META_FMT_MALI_C55_STATS;
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static int mali_c55_stats_g_fmt_meta_cap(struct file *file, void *fh,
>>> + struct v4l2_format *f)
>>> +{
>>> + static const struct v4l2_meta_format mfmt = {
>>> + .dataformat = V4L2_META_FMT_MALI_C55_STATS,
>>> + .buffersize = sizeof(struct mali_c55_stats_buffer)
>>> + };
>>> +
>>> + f->fmt.meta = mfmt;
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static int mali_c55_stats_querycap(struct file *file,
>>> + void *priv, struct v4l2_capability *cap)
>>> +{
>>> + strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver));
>>> + strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card));
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static const struct v4l2_ioctl_ops mali_c55_stats_v4l2_ioctl_ops = {
>>> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
>>> + .vidioc_querybuf = vb2_ioctl_querybuf,
>>> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
>>> + .vidioc_qbuf = vb2_ioctl_qbuf,
>>> + .vidioc_expbuf = vb2_ioctl_expbuf,
>>> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
>>> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>> + .vidioc_streamon = vb2_ioctl_streamon,
>>> + .vidioc_streamoff = vb2_ioctl_streamoff,
>>> + .vidioc_enum_fmt_meta_cap = mali_c55_stats_enum_fmt_meta_cap,
>>> + .vidioc_g_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
>>> + .vidioc_s_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
>>> + .vidioc_try_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
>>> + .vidioc_querycap = mali_c55_stats_querycap,
>>> + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
>>> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
>>> +};
>>> +
>>> +static const struct v4l2_file_operations mali_c55_stats_v4l2_fops = {
>>> + .owner = THIS_MODULE,
>>> + .unlocked_ioctl = video_ioctl2,
>>> + .open = v4l2_fh_open,
>>> + .release = vb2_fop_release,
>>> + .poll = vb2_fop_poll,
>>> + .mmap = vb2_fop_mmap,
>>> +};
>>> +
>>> +static int
>>> +mali_c55_stats_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
>>> + unsigned int *num_planes, unsigned int sizes[],
>>> + struct device *alloc_devs[])
>>> +{
>>> + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
>>> +
>>> + if (*num_planes && *num_planes > 1)
>>> + return -EINVAL;
>>> +
>>> + if (sizes[0] && sizes[0] < sizeof(struct mali_c55_stats_buffer))
>>> + return -EINVAL;
>>> +
>>> + *num_planes = 1;
>>> +
>>> + if (!sizes[0])
>>> + sizes[0] = sizeof(struct mali_c55_stats_buffer);
>>> +
>>> + if (stats->channel)
>>> + alloc_devs[0] = stats->channel->device->dev;
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static void mali_c55_stats_buf_queue(struct vb2_buffer *vb)
>>> +{
>>> + struct mali_c55_stats *stats = vb2_get_drv_priv(vb->vb2_queue);
>>> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>>> + struct mali_c55_stats_buf *buf = container_of(vbuf,
>>> + struct mali_c55_stats_buf, vb);
>>> +
>>> + vb2_set_plane_payload(vb, 0, sizeof(struct mali_c55_stats_buffer));
>>> + buf->segments_remaining = 2;
>>> + buf->failed = false;
>>> +
>>> + spin_lock(&stats->buffers.lock);
>>> + list_add_tail(&buf->queue, &stats->buffers.queue);
>>> + spin_unlock(&stats->buffers.lock);
>>> +}
>>> +
>>> +static int mali_c55_stats_start_streaming(struct vb2_queue *q,
>>> + unsigned int count)
>>> +{
>>> + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
>>> + struct mali_c55 *mali_c55 = stats->mali_c55;
>>> + int ret;
>>> +
>>> + ret = video_device_pipeline_start(&stats->vdev,
>>> + &stats->mali_c55->pipe);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + if (mali_c55->pipe.start_count == mali_c55->pipe.required_count) {
>>> + ret = v4l2_subdev_enable_streams(&mali_c55->isp.sd,
>>> + MALI_C55_ISP_PAD_SOURCE_VIDEO,
>>> + BIT(0));
>>> + if (ret)
>>> + goto err_stop_pipeline;
>>> + }
>>> +
>>> + return 0;
>>> +
>>> +err_stop_pipeline:
>>> + video_device_pipeline_stop(&stats->vdev);
>>> +
>>> + return ret;
>>> +}
>>> +
>>> +static void mali_c55_stats_stop_streaming(struct vb2_queue *q)
>>> +{
>>> + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
>>> + struct mali_c55_stats_buf *buf, *tmp;
>>> +
>>> + dmaengine_terminate_sync(stats->channel);
>>> +
>>> + spin_lock(&stats->buffers.lock);
>>> +
>>> + list_for_each_entry_safe(buf, tmp, &stats->buffers.queue, queue) {
>>> + list_del(&buf->queue);
>>> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
>>> + }
>>> +
>>> + spin_unlock(&stats->buffers.lock);
>>> +
>>> + video_device_pipeline_stop(&stats->vdev);
> There's a lack of symmetry here, you call v4l2_subdev_enable_streams()
> in mali_c55_stats_start_streaming() but you don't call
> v4l2_subdev_disable_streams() anywhere. Is that intentional ?
It's conditionally called, provided the start_count of the media pipeline has reached a value that
matches the number of video devices...this fulfils the requirement to only start streaming when all
of the video devices have started. The v4l2_subdev_disable_streams() call I left the responsibility
of the image capture video devices...but perhaps it should behave similarly? Should stopping stream
on any of the video devices cause the stream to stop on all of them?
>
>>> +}
>>> +
>>> +static const struct vb2_ops mali_c55_stats_vb2_ops = {
>>> + .queue_setup = mali_c55_stats_queue_setup,
>>> + .buf_queue = mali_c55_stats_buf_queue,
>>> + .wait_prepare = vb2_ops_wait_prepare,
>>> + .wait_finish = vb2_ops_wait_finish,
>>> + .start_streaming = mali_c55_stats_start_streaming,
>>> + .stop_streaming = mali_c55_stats_stop_streaming,
>>> +};
>>> +
>>> +static void
>>> +mali_c55_stats_metering_complete(void *param,
>>> + const struct dmaengine_result *result)
>>> +{
>>> + struct mali_c55_stats_buf *buf = param;
>>> +
>>> + if (result->result != DMA_TRANS_NOERROR)
>>> + buf->failed = true;
>>> +
>>> + if (!--buf->segments_remaining)
>>> + vb2_buffer_done(&buf->vb.vb2_buf, buf->failed ?
>>> + VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE);
>>> +}
>>> +
>>> +static int mali_c55_stats_dma_xfer(struct mali_c55_stats *stats, dma_addr_t src,
>>> + dma_addr_t dst,
>>> + struct mali_c55_stats_buf *buf,
>>> + size_t length)
>>> +{
>>> + struct dma_async_tx_descriptor *tx;
>>> + dma_cookie_t cookie;
>>> +
>>> + tx = dmaengine_prep_dma_memcpy(stats->channel, dst, src, length, 0);
>>> + if (!tx) {
>>> + dev_err(stats->mali_c55->dev, "failed to prep stats DMA\n");
>>> + return -EIO;
>>> + }
>>> +
>>> + tx->callback_result = mali_c55_stats_metering_complete;
>>> + tx->callback_param = buf;
>>> +
>>> + cookie = dmaengine_submit(tx);
>>> + if (dma_submit_error(cookie)) {
>>> + dev_err(stats->mali_c55->dev, "failed to submit stats DMA\n");
>>> + return -EIO;
>>> + }
>>> +
>>> + dma_async_issue_pending(stats->channel);
> You could possibly lower the overhead a bit by calling
> dma_async_issue_pending() only once after submitting the two transfers.
> It may not make any real difference though, I don't recall the details.
I can test if we think it's worthwhile...I was working under the assumption it would be better to
kick start the first transfer so it's running earlier, but I didn't test it.
>
>>> + return 0;
>>> +}
>>> +
>>> +void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
>>> + enum mali_c55_config_spaces cfg_space)
>>> +{
>>> + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
>>> + struct mali_c55_stats *stats = &mali_c55->stats;
>>> + struct mali_c55_stats_buf *buf = NULL;
>>> + dma_addr_t src, dst;
>>> + size_t length;
>>> + int ret;
>>> +
>>> + spin_lock(&stats->buffers.lock);
>>> + if (!list_empty(&stats->buffers.queue)) {
>>> + buf = list_first_entry(&stats->buffers.queue,
>>> + struct mali_c55_stats_buf, queue);
>>> + list_del(&buf->queue);
>>> + }
>>> + spin_unlock(&stats->buffers.lock);
>>> +
>>> + if (!buf)
>>> + return;
>>> +
>>> + buf->vb.sequence = mali_c55->isp.frame_sequence;
>>> + buf->vb.vb2_buf.timestamp = ktime_get_boottime_ns();
>>> +
>>> + /*
>>> + * There are in fact two noncontiguous sections of the ISP's
>>> + * memory space that hold statistics for 3a algorithms to use: A
>>> + * section in each config space and a global section holding
>>> + * histograms which is double buffered and so holds data for the
>>> + * last frame. We need to read both.
>>> + */
>>> + src = ctx->base + MALI_C55_REG_1024BIN_HIST;
>>> + dst = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
>>> +
>>> + ret = mali_c55_stats_dma_xfer(stats, src, dst, buf,
>>> + MALI_C55_1024BIN_HIST_SIZE);
>>> + if (ret)
>>> + goto err_fail_buffer;
>>> +
>>> + src = ctx->base + metering_space_addrs[cfg_space];
>>> + dst += MALI_C55_1024BIN_HIST_SIZE;
>>> +
>>> + length = sizeof(struct mali_c55_stats_buffer) - MALI_C55_1024BIN_HIST_SIZE;
>>> + ret = mali_c55_stats_dma_xfer(stats, src, dst, buf, length);
>>> + if (ret) {
>>> + dmaengine_terminate_sync(stats->channel);
>>> + goto err_fail_buffer;
>>> + }
>>> +
>>> + return;
>>> +
>>> +err_fail_buffer:
>>> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
>>> +}
>>> +
>>> +void mali_c55_unregister_stats(struct mali_c55 *mali_c55)
>>> +{
>>> + struct mali_c55_stats *stats = &mali_c55->stats;
>>> +
>>> + if (!video_is_registered(&stats->vdev))
>>> + return;
>>> +
>>> + vb2_video_unregister_device(&stats->vdev);
>>> + media_entity_cleanup(&stats->vdev.entity);
>>> + dma_release_channel(stats->channel);
>>> + mutex_destroy(&stats->lock);
>>> +}
>>> +
>>> +int mali_c55_register_stats(struct mali_c55 *mali_c55)
>>> +{
>>> + struct mali_c55_stats *stats = &mali_c55->stats;
>>> + struct video_device *vdev = &stats->vdev;
>>> + struct vb2_queue *vb2q = &stats->queue;
>>> + dma_cap_mask_t mask;
>>> + int ret;
>>> +
>>> + mutex_init(&stats->lock);
>>> + INIT_LIST_HEAD(&stats->buffers.queue);
>>> +
>>> + dma_cap_zero(mask);
>>> + dma_cap_set(DMA_MEMCPY, mask);
>>> +
>>> + stats->channel = dma_request_channel(mask, 0, NULL);
>>> + if (!stats->channel) {
>>> + ret = -ENODEV;
>>> + goto err_destroy_mutex;
>>> + }
>>> +
>>> + stats->pad.flags = MEDIA_PAD_FL_SINK;
>>> + ret = media_entity_pads_init(&stats->vdev.entity, 1, &stats->pad);
>>> + if (ret)
>>> + goto err_release_dma_channel;
>>> +
>>> + vb2q->type = V4L2_BUF_TYPE_META_CAPTURE;
>>> + vb2q->io_modes = VB2_MMAP | VB2_DMABUF;
>>> + vb2q->drv_priv = stats;
>>> + vb2q->mem_ops = &vb2_dma_contig_memops;
>>> + vb2q->ops = &mali_c55_stats_vb2_ops;
>>> + vb2q->buf_struct_size = sizeof(struct mali_c55_stats_buf);
>>> + vb2q->min_queued_buffers = 1;
>>> + vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
>>> + vb2q->lock = &stats->lock;
>>> + vb2q->dev = stats->channel->device->dev;
>>> +
>>> + ret = vb2_queue_init(vb2q);
>>> + if (ret) {
>>> + dev_err(mali_c55->dev, "stats vb2 queue init failed\n");
>>> + goto err_cleanup_entity;
>>> + }
>>> +
>>> + strscpy(stats->vdev.name, "mali-c55 3a stats", sizeof(stats->vdev.name));
>>> + vdev->release = video_device_release_empty;
> Not a good idea at all, but I won't ask you to fix object lifetime
> management in V4L2 as a prerequisite to merging this driver :-)
>
> With those issues addressed,
>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>
>>> + vdev->fops = &mali_c55_stats_v4l2_fops;
>>> + vdev->ioctl_ops = &mali_c55_stats_v4l2_ioctl_ops;
>>> + vdev->lock = &stats->lock;
>>> + vdev->v4l2_dev = &mali_c55->v4l2_dev;
>>> + vdev->queue = &stats->queue;
>>> + vdev->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
>>> + vdev->vfl_dir = VFL_DIR_RX;
>>> + video_set_drvdata(vdev, stats);
>>> +
>>> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>>> + if (ret) {
>>> + dev_err(mali_c55->dev,
>>> + "failed to register stats video device\n");
>>> + goto err_release_vb2q;
>>> + }
>>> +
>>> + stats->mali_c55 = mali_c55;
>>> +
>>> + return 0;
>>> +
>>> +err_release_vb2q:
>>> + vb2_queue_release(vb2q);
>>> +err_cleanup_entity:
>>> + media_entity_cleanup(&stats->vdev.entity);
>>> +err_release_dma_channel:
>>> + dma_release_channel(stats->channel);
>>> +err_destroy_mutex:
>>> + mutex_destroy(&stats->lock);
>>> +
>>> + return ret;
>>> +}
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 12/18] media: platform: Add mali-c55 3a stats devnode
2024-07-22 14:50 ` Laurent Pinchart
2024-07-22 22:55 ` Laurent Pinchart
2024-07-29 10:53 ` Dan Scally
@ 2024-08-13 13:51 ` Dan Scally
2 siblings, 0 replies; 41+ messages in thread
From: Dan Scally @ 2024-08-13 13:51 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Laurent
On 22/07/2024 15:50, Laurent Pinchart wrote:
> Hi Dan,
>
> Thank you for the patch.
>
> On Tue, Jul 09, 2024 at 02:29:00PM +0100, Daniel Scally wrote:
>> Add a new code file to govern the 3a statistics capture node.
>>
>> Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
>> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
>> ---
>> Changes in v6:
>>
>> - Fixed mising includes
>> - Minor renames and formatting
>> - Reworked mali_c55_stats_metering_complete() so it could only return
>> buffers when both halves of the DMA read were done
>> - Terminate dma transfers on streamoff
>>
>> Changes in v5:
>>
>> - New patch
>>
>> drivers/media/platform/arm/mali-c55/Makefile | 1 +
>> .../platform/arm/mali-c55/mali-c55-common.h | 29 ++
>> .../platform/arm/mali-c55/mali-c55-core.c | 15 +
>> .../platform/arm/mali-c55/mali-c55-isp.c | 11 +
>> .../arm/mali-c55/mali-c55-registers.h | 3 +
>> .../platform/arm/mali-c55/mali-c55-stats.c | 373 ++++++++++++++++++
>> 6 files changed, 432 insertions(+)
>> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-stats.c
>>
>> diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile
>> index 9178ac35e50e..b5a22d414479 100644
>> --- a/drivers/media/platform/arm/mali-c55/Makefile
>> +++ b/drivers/media/platform/arm/mali-c55/Makefile
>> @@ -4,6 +4,7 @@ mali-c55-y := mali-c55-capture.o \
>> mali-c55-core.o \
>> mali-c55-isp.o \
>> mali-c55-resizer.o \
>> + mali-c55-stats.o \
>> mali-c55-tpg.o
>>
>> obj-$(CONFIG_VIDEO_MALI_C55) += mali-c55.o
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
>> index f7764a938e9f..136c785c68ba 100644
>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-common.h
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
>> @@ -49,6 +49,7 @@ enum mali_c55_isp_pads {
>> MALI_C55_ISP_PAD_SINK_VIDEO,
>> MALI_C55_ISP_PAD_SOURCE_VIDEO,
>> MALI_C55_ISP_PAD_SOURCE_BYPASS,
>> + MALI_C55_ISP_PAD_SOURCE_STATS,
>> MALI_C55_ISP_NUM_PADS,
>> };
>>
>> @@ -160,6 +161,29 @@ struct mali_c55_cap_dev {
>> bool streaming;
>> };
>>
>> +struct mali_c55_stats_buf {
>> + struct vb2_v4l2_buffer vb;
>> + unsigned int segments_remaining;
>> + struct list_head queue;
>> + bool failed;
>> +};
>> +
>> +struct mali_c55_stats {
>> + struct mali_c55 *mali_c55;
>> + struct video_device vdev;
>> + struct dma_chan *channel;
>> + struct vb2_queue queue;
>> + struct media_pad pad;
>> + /* Mutex to provide to vb2 */
>> + struct mutex lock;
>> +
>> + struct {
>> + /* Spinlock to guard buffer queue */
>> + spinlock_t lock;
>> + struct list_head queue;
>> + } buffers;
>> +};
>> +
>> enum mali_c55_config_spaces {
>> MALI_C55_CONFIG_PING,
>> MALI_C55_CONFIG_PONG,
>> @@ -202,6 +226,7 @@ struct mali_c55 {
>> struct mali_c55_isp isp;
>> struct mali_c55_resizer resizers[MALI_C55_NUM_RSZS];
>> struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS];
>> + struct mali_c55_stats stats;
>>
>> struct mali_c55_context context;
>> enum mali_c55_config_spaces next_config;
>> @@ -229,6 +254,8 @@ int mali_c55_register_resizers(struct mali_c55 *mali_c55);
>> void mali_c55_unregister_resizers(struct mali_c55 *mali_c55);
>> int mali_c55_register_capture_devs(struct mali_c55 *mali_c55);
>> void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55);
>> +int mali_c55_register_stats(struct mali_c55 *mali_c55);
>> +void mali_c55_unregister_stats(struct mali_c55 *mali_c55);
>> struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55);
>> void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
>> enum mali_c55_planes plane);
>> @@ -243,5 +270,7 @@ const struct mali_c55_isp_fmt *
>> mali_c55_isp_get_mbus_config_by_code(u32 code);
>> const struct mali_c55_isp_fmt *
>> mali_c55_isp_get_mbus_config_by_index(u32 index);
>> +void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
>> + enum mali_c55_config_spaces cfg_space);
>>
>> #endif /* _MALI_C55_COMMON_H */
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
>> index dbc07d12d3a3..eedc8f450184 100644
>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
>> @@ -374,6 +374,16 @@ static int mali_c55_create_links(struct mali_c55 *mali_c55)
>> }
>> }
>>
>> + ret = media_create_pad_link(&mali_c55->isp.sd.entity,
>> + MALI_C55_ISP_PAD_SOURCE_STATS,
>> + &mali_c55->stats.vdev.entity, 0,
>> + MEDIA_LNK_FL_ENABLED);
>> + if (ret) {
>> + dev_err(mali_c55->dev,
>> + "failed to link ISP and 3a stats node\n");
>> + goto err_remove_links;
>> + }
>> +
>> return 0;
>>
>> err_remove_links:
>> @@ -388,6 +398,7 @@ static void mali_c55_unregister_entities(struct mali_c55 *mali_c55)
>> mali_c55_unregister_isp(mali_c55);
>> mali_c55_unregister_resizers(mali_c55);
>> mali_c55_unregister_capture_devs(mali_c55);
>> + mali_c55_unregister_stats(mali_c55);
>> }
>>
>> static int mali_c55_register_entities(struct mali_c55 *mali_c55)
>> @@ -410,6 +421,10 @@ static int mali_c55_register_entities(struct mali_c55 *mali_c55)
>> if (ret)
>> goto err_unregister_entities;
>>
>> + ret = mali_c55_register_stats(mali_c55);
>> + if (ret)
>> + goto err_unregister_entities;
>> +
>> ret = mali_c55_create_links(mali_c55);
>> if (ret)
>> goto err_unregister_entities;
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>> index f784983a4ccc..2f450c00300a 100644
>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>> @@ -5,6 +5,8 @@
>> * Copyright (C) 2024 Ideas on Board Oy
>> */
>>
>> +#include <linux/media/arm/mali-c55-config.h>
>> +
>> #include <linux/delay.h>
>> #include <linux/iopoll.h>
>> #include <linux/property.h>
>> @@ -460,6 +462,14 @@ static int mali_c55_isp_init_state(struct v4l2_subdev *sd,
>> in_crop->width = MALI_C55_DEFAULT_WIDTH;
>> in_crop->height = MALI_C55_DEFAULT_HEIGHT;
>>
>> + src_fmt = v4l2_subdev_state_get_format(state,
>> + MALI_C55_ISP_PAD_SOURCE_STATS);
>> +
>> + src_fmt->width = sizeof(struct mali_c55_stats_buffer);
>> + src_fmt->height = 1;
> According to
> https://docs.kernel.org/userspace-api/media/v4l/subdev-formats.html#metadata-formats,
> width and height should be set to 0 for MEDIA_BUS_FMT_METADATA_FIXED. I
> haven't checked if v4l2-compliance expects this, we may have
> discrepancies between the API documentation and the existing
> implementations in the kernel and userspace. In any case, this should be
> sorted out, either by fixing the kernel code and enforcing the
> requirement in v4l2-compliance, or fixing the documentation.
So I originally set it this way to fix v4l2-compliance warnings, which complain about width being
zero (but not height). I think that the format code is right, and so (assuming the documentation is
right) v4l2-compliance needs to be patched for that somehow...I'll add that to my todo list
>
>> + src_fmt->field = V4L2_FIELD_NONE;
>> + src_fmt->code = MEDIA_BUS_FMT_METADATA_FIXED;
>> +
>> return 0;
>> }
>>
>> @@ -490,6 +500,7 @@ int mali_c55_register_isp(struct mali_c55 *mali_c55)
>> MEDIA_PAD_FL_MUST_CONNECT;
>> isp->pads[MALI_C55_ISP_PAD_SOURCE_VIDEO].flags = MEDIA_PAD_FL_SOURCE;
>> isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE;
>> + isp->pads[MALI_C55_ISP_PAD_SOURCE_STATS].flags = MEDIA_PAD_FL_SOURCE;
>>
>> ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS,
>> isp->pads);
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>> index 0a391f8a043e..e72e749b81d5 100644
>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>> @@ -103,6 +103,9 @@ enum mali_c55_interrupts {
>> #define MALI_C55_VC_START(v) ((v) & 0xffff)
>> #define MALI_C55_VC_SIZE(v) (((v) & 0xffff) << 16)
>>
>> +#define MALI_C55_REG_1024BIN_HIST 0x054a8
>> +#define MALI_C55_1024BIN_HIST_SIZE 4096
>> +
>> /* Ping/Pong Configuration Space */
>> #define MALI_C55_REG_BASE_ADDR 0x18e88
>> #define MALI_C55_REG_BYPASS_0 0x18eac
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-stats.c b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
>> new file mode 100644
>> index 000000000000..38a17fb5d052
>> --- /dev/null
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
>> @@ -0,0 +1,373 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * ARM Mali-C55 ISP Driver - 3A Statistics capture device
>> + *
>> + * Copyright (C) 2023 Ideas on Board Oy
>> + */
>> +
>> +#include <linux/container_of.h>
>> +#include <linux/dev_printk.h>
>> +#include <linux/dmaengine.h>
>> +#include <linux/list.h>
>> +#include <linux/media/arm/mali-c55-config.h>
>> +#include <linux/mutex.h>
>> +#include <linux/spinlock.h>
>> +#include <linux/string.h>
>> +
>> +#include <media/media-entity.h>
>> +#include <media/v4l2-dev.h>
>> +#include <media/v4l2-event.h>
>> +#include <media/v4l2-fh.h>
>> +#include <media/v4l2-ioctl.h>
>> +#include <media/videobuf2-core.h>
>> +#include <media/videobuf2-dma-contig.h>
>> +
>> +#include "mali-c55-common.h"
>> +#include "mali-c55-registers.h"
>> +
>> +static const unsigned int metering_space_addrs[] = {
>> + [MALI_C55_CONFIG_PING] = 0x095ac,
>> + [MALI_C55_CONFIG_PONG] = 0x2156c,
>> +};
>> +
>> +static int mali_c55_stats_enum_fmt_meta_cap(struct file *file, void *fh,
>> + struct v4l2_fmtdesc *f)
>> +{
>> + if (f->index)
>> + return -EINVAL;
>> +
>> + f->pixelformat = V4L2_META_FMT_MALI_C55_STATS;
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_stats_g_fmt_meta_cap(struct file *file, void *fh,
>> + struct v4l2_format *f)
>> +{
>> + static const struct v4l2_meta_format mfmt = {
>> + .dataformat = V4L2_META_FMT_MALI_C55_STATS,
>> + .buffersize = sizeof(struct mali_c55_stats_buffer)
>> + };
>> +
>> + f->fmt.meta = mfmt;
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_stats_querycap(struct file *file,
>> + void *priv, struct v4l2_capability *cap)
>> +{
>> + strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver));
>> + strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card));
>> +
>> + return 0;
>> +}
>> +
>> +static const struct v4l2_ioctl_ops mali_c55_stats_v4l2_ioctl_ops = {
>> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
>> + .vidioc_querybuf = vb2_ioctl_querybuf,
>> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
>> + .vidioc_qbuf = vb2_ioctl_qbuf,
>> + .vidioc_expbuf = vb2_ioctl_expbuf,
>> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
>> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>> + .vidioc_streamon = vb2_ioctl_streamon,
>> + .vidioc_streamoff = vb2_ioctl_streamoff,
>> + .vidioc_enum_fmt_meta_cap = mali_c55_stats_enum_fmt_meta_cap,
>> + .vidioc_g_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
>> + .vidioc_s_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
>> + .vidioc_try_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
>> + .vidioc_querycap = mali_c55_stats_querycap,
>> + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
>> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
>> +};
>> +
>> +static const struct v4l2_file_operations mali_c55_stats_v4l2_fops = {
>> + .owner = THIS_MODULE,
>> + .unlocked_ioctl = video_ioctl2,
>> + .open = v4l2_fh_open,
>> + .release = vb2_fop_release,
>> + .poll = vb2_fop_poll,
>> + .mmap = vb2_fop_mmap,
>> +};
>> +
>> +static int
>> +mali_c55_stats_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
>> + unsigned int *num_planes, unsigned int sizes[],
>> + struct device *alloc_devs[])
>> +{
>> + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
>> +
>> + if (*num_planes && *num_planes > 1)
>> + return -EINVAL;
>> +
>> + if (sizes[0] && sizes[0] < sizeof(struct mali_c55_stats_buffer))
>> + return -EINVAL;
>> +
>> + *num_planes = 1;
>> +
>> + if (!sizes[0])
>> + sizes[0] = sizeof(struct mali_c55_stats_buffer);
>> +
>> + if (stats->channel)
>> + alloc_devs[0] = stats->channel->device->dev;
>> +
>> + return 0;
>> +}
>> +
>> +static void mali_c55_stats_buf_queue(struct vb2_buffer *vb)
>> +{
>> + struct mali_c55_stats *stats = vb2_get_drv_priv(vb->vb2_queue);
>> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>> + struct mali_c55_stats_buf *buf = container_of(vbuf,
>> + struct mali_c55_stats_buf, vb);
>> +
>> + vb2_set_plane_payload(vb, 0, sizeof(struct mali_c55_stats_buffer));
>> + buf->segments_remaining = 2;
>> + buf->failed = false;
>> +
>> + spin_lock(&stats->buffers.lock);
>> + list_add_tail(&buf->queue, &stats->buffers.queue);
>> + spin_unlock(&stats->buffers.lock);
>> +}
>> +
>> +static int mali_c55_stats_start_streaming(struct vb2_queue *q,
>> + unsigned int count)
>> +{
>> + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
>> + struct mali_c55 *mali_c55 = stats->mali_c55;
>> + int ret;
>> +
>> + ret = video_device_pipeline_start(&stats->vdev,
>> + &stats->mali_c55->pipe);
>> + if (ret)
>> + return ret;
>> +
>> + if (mali_c55->pipe.start_count == mali_c55->pipe.required_count) {
>> + ret = v4l2_subdev_enable_streams(&mali_c55->isp.sd,
>> + MALI_C55_ISP_PAD_SOURCE_VIDEO,
>> + BIT(0));
>> + if (ret)
>> + goto err_stop_pipeline;
>> + }
>> +
>> + return 0;
>> +
>> +err_stop_pipeline:
>> + video_device_pipeline_stop(&stats->vdev);
>> +
>> + return ret;
>> +}
>> +
>> +static void mali_c55_stats_stop_streaming(struct vb2_queue *q)
>> +{
>> + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
>> + struct mali_c55_stats_buf *buf, *tmp;
>> +
>> + dmaengine_terminate_sync(stats->channel);
>> +
>> + spin_lock(&stats->buffers.lock);
>> +
>> + list_for_each_entry_safe(buf, tmp, &stats->buffers.queue, queue) {
>> + list_del(&buf->queue);
>> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
>> + }
>> +
>> + spin_unlock(&stats->buffers.lock);
>> +
>> + video_device_pipeline_stop(&stats->vdev);
>> +}
>> +
>> +static const struct vb2_ops mali_c55_stats_vb2_ops = {
>> + .queue_setup = mali_c55_stats_queue_setup,
>> + .buf_queue = mali_c55_stats_buf_queue,
>> + .wait_prepare = vb2_ops_wait_prepare,
>> + .wait_finish = vb2_ops_wait_finish,
>> + .start_streaming = mali_c55_stats_start_streaming,
>> + .stop_streaming = mali_c55_stats_stop_streaming,
>> +};
>> +
>> +static void
>> +mali_c55_stats_metering_complete(void *param,
>> + const struct dmaengine_result *result)
>> +{
>> + struct mali_c55_stats_buf *buf = param;
>> +
>> + if (result->result != DMA_TRANS_NOERROR)
>> + buf->failed = true;
>> +
>> + if (!--buf->segments_remaining)
>> + vb2_buffer_done(&buf->vb.vb2_buf, buf->failed ?
>> + VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE);
>> +}
>> +
>> +static int mali_c55_stats_dma_xfer(struct mali_c55_stats *stats, dma_addr_t src,
>> + dma_addr_t dst,
>> + struct mali_c55_stats_buf *buf,
>> + size_t length)
>> +{
>> + struct dma_async_tx_descriptor *tx;
>> + dma_cookie_t cookie;
>> +
>> + tx = dmaengine_prep_dma_memcpy(stats->channel, dst, src, length, 0);
>> + if (!tx) {
>> + dev_err(stats->mali_c55->dev, "failed to prep stats DMA\n");
>> + return -EIO;
>> + }
>> +
>> + tx->callback_result = mali_c55_stats_metering_complete;
>> + tx->callback_param = buf;
>> +
>> + cookie = dmaengine_submit(tx);
>> + if (dma_submit_error(cookie)) {
>> + dev_err(stats->mali_c55->dev, "failed to submit stats DMA\n");
>> + return -EIO;
>> + }
>> +
>> + dma_async_issue_pending(stats->channel);
>> + return 0;
>> +}
>> +
>> +void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
>> + enum mali_c55_config_spaces cfg_space)
>> +{
>> + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
>> + struct mali_c55_stats *stats = &mali_c55->stats;
>> + struct mali_c55_stats_buf *buf = NULL;
>> + dma_addr_t src, dst;
>> + size_t length;
>> + int ret;
>> +
>> + spin_lock(&stats->buffers.lock);
>> + if (!list_empty(&stats->buffers.queue)) {
>> + buf = list_first_entry(&stats->buffers.queue,
>> + struct mali_c55_stats_buf, queue);
>> + list_del(&buf->queue);
>> + }
>> + spin_unlock(&stats->buffers.lock);
>> +
>> + if (!buf)
>> + return;
>> +
>> + buf->vb.sequence = mali_c55->isp.frame_sequence;
>> + buf->vb.vb2_buf.timestamp = ktime_get_boottime_ns();
>> +
>> + /*
>> + * There are in fact two noncontiguous sections of the ISP's
>> + * memory space that hold statistics for 3a algorithms to use: A
>> + * section in each config space and a global section holding
>> + * histograms which is double buffered and so holds data for the
>> + * last frame. We need to read both.
>> + */
>> + src = ctx->base + MALI_C55_REG_1024BIN_HIST;
>> + dst = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
>> +
>> + ret = mali_c55_stats_dma_xfer(stats, src, dst, buf,
>> + MALI_C55_1024BIN_HIST_SIZE);
>> + if (ret)
>> + goto err_fail_buffer;
>> +
>> + src = ctx->base + metering_space_addrs[cfg_space];
>> + dst += MALI_C55_1024BIN_HIST_SIZE;
>> +
>> + length = sizeof(struct mali_c55_stats_buffer) - MALI_C55_1024BIN_HIST_SIZE;
>> + ret = mali_c55_stats_dma_xfer(stats, src, dst, buf, length);
>> + if (ret) {
>> + dmaengine_terminate_sync(stats->channel);
>> + goto err_fail_buffer;
>> + }
>> +
>> + return;
>> +
>> +err_fail_buffer:
>> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
>> +}
>> +
>> +void mali_c55_unregister_stats(struct mali_c55 *mali_c55)
>> +{
>> + struct mali_c55_stats *stats = &mali_c55->stats;
>> +
>> + if (!video_is_registered(&stats->vdev))
>> + return;
>> +
>> + vb2_video_unregister_device(&stats->vdev);
>> + media_entity_cleanup(&stats->vdev.entity);
>> + dma_release_channel(stats->channel);
>> + mutex_destroy(&stats->lock);
>> +}
>> +
>> +int mali_c55_register_stats(struct mali_c55 *mali_c55)
>> +{
>> + struct mali_c55_stats *stats = &mali_c55->stats;
>> + struct video_device *vdev = &stats->vdev;
>> + struct vb2_queue *vb2q = &stats->queue;
>> + dma_cap_mask_t mask;
>> + int ret;
>> +
>> + mutex_init(&stats->lock);
>> + INIT_LIST_HEAD(&stats->buffers.queue);
>> +
>> + dma_cap_zero(mask);
>> + dma_cap_set(DMA_MEMCPY, mask);
>> +
>> + stats->channel = dma_request_channel(mask, 0, NULL);
>> + if (!stats->channel) {
>> + ret = -ENODEV;
>> + goto err_destroy_mutex;
>> + }
>> +
>> + stats->pad.flags = MEDIA_PAD_FL_SINK;
>> + ret = media_entity_pads_init(&stats->vdev.entity, 1, &stats->pad);
>> + if (ret)
>> + goto err_release_dma_channel;
>> +
>> + vb2q->type = V4L2_BUF_TYPE_META_CAPTURE;
>> + vb2q->io_modes = VB2_MMAP | VB2_DMABUF;
>> + vb2q->drv_priv = stats;
>> + vb2q->mem_ops = &vb2_dma_contig_memops;
>> + vb2q->ops = &mali_c55_stats_vb2_ops;
>> + vb2q->buf_struct_size = sizeof(struct mali_c55_stats_buf);
>> + vb2q->min_queued_buffers = 1;
>> + vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
>> + vb2q->lock = &stats->lock;
>> + vb2q->dev = stats->channel->device->dev;
>> +
>> + ret = vb2_queue_init(vb2q);
>> + if (ret) {
>> + dev_err(mali_c55->dev, "stats vb2 queue init failed\n");
>> + goto err_cleanup_entity;
>> + }
>> +
>> + strscpy(stats->vdev.name, "mali-c55 3a stats", sizeof(stats->vdev.name));
>> + vdev->release = video_device_release_empty;
>> + vdev->fops = &mali_c55_stats_v4l2_fops;
>> + vdev->ioctl_ops = &mali_c55_stats_v4l2_ioctl_ops;
>> + vdev->lock = &stats->lock;
>> + vdev->v4l2_dev = &mali_c55->v4l2_dev;
>> + vdev->queue = &stats->queue;
>> + vdev->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
>> + vdev->vfl_dir = VFL_DIR_RX;
>> + video_set_drvdata(vdev, stats);
>> +
>> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>> + if (ret) {
>> + dev_err(mali_c55->dev,
>> + "failed to register stats video device\n");
>> + goto err_release_vb2q;
>> + }
>> +
>> + stats->mali_c55 = mali_c55;
>> +
>> + return 0;
>> +
>> +err_release_vb2q:
>> + vb2_queue_release(vb2q);
>> +err_cleanup_entity:
>> + media_entity_cleanup(&stats->vdev.entity);
>> +err_release_dma_channel:
>> + dma_release_channel(stats->channel);
>> +err_destroy_mutex:
>> + mutex_destroy(&stats->lock);
>> +
>> + return ret;
>> +}
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 07/18] media: mali-c55: Add Mali-C55 ISP driver
2024-07-30 21:23 ` Laurent Pinchart
@ 2024-08-13 15:20 ` Dan Scally
0 siblings, 0 replies; 41+ messages in thread
From: Dan Scally @ 2024-08-13 15:20 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Laurent - thanks for the detailed review
On 30/07/2024 22:23, Laurent Pinchart wrote:
> Hi Dan,
>
> Thank you for the patch.
>
> On Tue, Jul 09, 2024 at 02:28:55PM +0100, Daniel Scally wrote:
>> Add a driver for Arm's Mali-C55 Image Signal Processor. The driver is
>> V4L2 and Media Controller compliant and creates subdevices to manage
>> the ISP itself, its internal test pattern generator as well as the
>> crop, scaler and output format functionality for each of its two
>> output devices.
>>
>> Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
>> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
>> ---
>> Changes in v6:
>>
>> - Reverted the rework of mali_c55_update_bits() so it matches how the
>> regmap_update_bits() function works
>> - Global rename of "rzr" abbreviation for "resizer" to "rsz"
>> - Added separate functions to write to hardware and register buffer.
>> - Honoured custom strides from userspace
>> - Used .enable_streams(), .disable_streams() and
>> v4l2_subdev_[en|dis]able_streams() throughout
>> - Only call streamon for the sensor when all video devices included in
>> the pipeline through enabled links are started.
>> - Don't write registers until streamon() in the capture devices
>> - Commented mutex/spinlocks
>> - Moved v4l2 async notifier from isp.c to core.c
>> - Reconfigured hardware in pm_runtime_resume() callback to make sure
>> they are not lost on power-off
>> - A lot of other changes, which are mostly stylistic in nature. THe full
>> list is too large to include but I can send a diff between the version
>> if needed.
>>
>> Changes in v5:
>>
>> - Reworked input formats - previously we allowed representing input data
>> as any 8-16 bit format. Now we only allow input data to be represented
>> by the new 20-bit bayer formats, which is corrected to the equivalent
>> 16-bit format in RAW bypass mode.
>> - Stopped bypassing blocks that we haven't added supporting parameters
>> for yet.
>> - Addressed most of Sakari's comments from the list
>>
>> Changes not yet made in v5:
>>
>> - The output pipelines can still be started and stopped independently of
>> one another - I'd like to discuss that more.
>> - the TPG subdev still uses .s_stream() - I need to rebase onto a tree
>> with working .enable_streams() for a single-source-pad subdevice.
>>
>> Changes in v4:
>>
>> - Reworked mali_c55_update_bits() to internally perform the bit-shift
>> - Reworked the resizer to allow cropping during streaming
>> - Fixed a bug in NV12 output
>>
>> Changes in v3:
>>
>> - Mostly minor fixes suggested by Sakari
>> - Fixed the sequencing of vb2 buffers to be synchronised across the two
>> capture devices.
>>
>> Changes in v2:
>>
>> - Clock handling
>> - Fixed the warnings raised by the kernel test robot
>>
>> drivers/media/platform/Kconfig | 1 +
>> drivers/media/platform/Makefile | 1 +
>> drivers/media/platform/arm/Kconfig | 5 +
>> drivers/media/platform/arm/Makefile | 2 +
>> drivers/media/platform/arm/mali-c55/Kconfig | 17 +
>> drivers/media/platform/arm/mali-c55/Makefile | 9 +
>> .../platform/arm/mali-c55/mali-c55-capture.c | 961 +++++++++++++++
>> .../platform/arm/mali-c55/mali-c55-common.h | 247 ++++
>> .../platform/arm/mali-c55/mali-c55-core.c | 893 ++++++++++++++
>> .../platform/arm/mali-c55/mali-c55-isp.c | 531 ++++++++
>> .../arm/mali-c55/mali-c55-registers.h | 313 +++++
>> .../platform/arm/mali-c55/mali-c55-resizer.c | 1096 +++++++++++++++++
>> .../platform/arm/mali-c55/mali-c55-tpg.c | 438 +++++++
>> 13 files changed, 4514 insertions(+)
>> create mode 100644 drivers/media/platform/arm/Kconfig
>> create mode 100644 drivers/media/platform/arm/Makefile
>> create mode 100644 drivers/media/platform/arm/mali-c55/Kconfig
>> create mode 100644 drivers/media/platform/arm/mali-c55/Makefile
>> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-capture.c
>> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-common.h
>> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-core.c
>> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
>> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-tpg.c
>>
>> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
>> index 2d79bfc68c15..c929169766aa 100644
>> --- a/drivers/media/platform/Kconfig
>> +++ b/drivers/media/platform/Kconfig
>> @@ -65,6 +65,7 @@ config VIDEO_MUX
>> source "drivers/media/platform/allegro-dvt/Kconfig"
>> source "drivers/media/platform/amlogic/Kconfig"
>> source "drivers/media/platform/amphion/Kconfig"
>> +source "drivers/media/platform/arm/Kconfig"
>> source "drivers/media/platform/aspeed/Kconfig"
>> source "drivers/media/platform/atmel/Kconfig"
>> source "drivers/media/platform/broadcom/Kconfig"
>> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
>> index da17301f7439..9a647abd5218 100644
>> --- a/drivers/media/platform/Makefile
>> +++ b/drivers/media/platform/Makefile
>> @@ -8,6 +8,7 @@
>> obj-y += allegro-dvt/
>> obj-y += amlogic/
>> obj-y += amphion/
>> +obj-y += arm/
>> obj-y += aspeed/
>> obj-y += atmel/
>> obj-y += broadcom/
>> diff --git a/drivers/media/platform/arm/Kconfig b/drivers/media/platform/arm/Kconfig
>> new file mode 100644
>> index 000000000000..4f0764c329c7
>> --- /dev/null
>> +++ b/drivers/media/platform/arm/Kconfig
>> @@ -0,0 +1,5 @@
>> +# SPDX-License-Identifier: GPL-2.0-only
>> +
>> +comment "ARM media platform drivers"
>> +
>> +source "drivers/media/platform/arm/mali-c55/Kconfig"
>> diff --git a/drivers/media/platform/arm/Makefile b/drivers/media/platform/arm/Makefile
>> new file mode 100644
>> index 000000000000..8cc4918725ef
>> --- /dev/null
>> +++ b/drivers/media/platform/arm/Makefile
>> @@ -0,0 +1,2 @@
>> +# SPDX-License-Identifier: GPL-2.0-only
>> +obj-y += mali-c55/
>> diff --git a/drivers/media/platform/arm/mali-c55/Kconfig b/drivers/media/platform/arm/mali-c55/Kconfig
>> new file mode 100644
>> index 000000000000..6ba70e765b8d
>> --- /dev/null
>> +++ b/drivers/media/platform/arm/mali-c55/Kconfig
>> @@ -0,0 +1,17 @@
>> +# SPDX-License-Identifier: GPL-2.0-only
>> +config VIDEO_MALI_C55
>> + tristate "ARM Mali-C55 Image Signal Processor driver"
>> + depends on ARCH_VEXPRESS || COMPILE_TEST
>> + depends on V4L_PLATFORM_DRIVERS
>> + depends on VIDEO_DEV && OF
>> + select GENERIC_PHY_MIPI_DPHY
>> + select MEDIA_CONTROLLER
>> + select V4L2_FWNODE
>> + select VIDEO_V4L2_SUBDEV_API
>> + select VIDEOBUF2_DMA_CONTIG
>> + select VIDEOBUF2_VMALLOC
>> + help
>> + Enable this to support Arm's Mali-C55 Image Signal Processor.
>> +
>> + To compile this driver as a module, choose M here: the module
>> + will be called mali-c55.
>> diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile
>> new file mode 100644
>> index 000000000000..9178ac35e50e
>> --- /dev/null
>> +++ b/drivers/media/platform/arm/mali-c55/Makefile
>> @@ -0,0 +1,9 @@
>> +# SPDX-License-Identifier: GPL-2.0
>> +
>> +mali-c55-y := mali-c55-capture.o \
>> + mali-c55-core.o \
>> + mali-c55-isp.o \
>> + mali-c55-resizer.o \
>> + mali-c55-tpg.o
>> +
>> +obj-$(CONFIG_VIDEO_MALI_C55) += mali-c55.o
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-capture.c b/drivers/media/platform/arm/mali-c55/mali-c55-capture.c
>> new file mode 100644
>> index 000000000000..508245f06a09
>> --- /dev/null
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-capture.c
>> @@ -0,0 +1,961 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * ARM Mali-C55 ISP Driver - Video capture devices
>> + *
>> + * Copyright (C) 2024 Ideas on Board Oy
>> + */
>> +
>> +#include <linux/cleanup.h>
>> +#include <linux/minmax.h>
>> +#include <linux/pm_runtime.h>
>> +#include <linux/string.h>
>> +#include <linux/videodev2.h>
>> +
>> +#include <media/v4l2-dev.h>
>> +#include <media/v4l2-event.h>
>> +#include <media/v4l2-ioctl.h>
>> +#include <media/v4l2-subdev.h>
>> +#include <media/videobuf2-core.h>
>> +#include <media/videobuf2-dma-contig.h>
>> +
>> +#include "mali-c55-common.h"
>> +#include "mali-c55-registers.h"
>> +
>> +static const struct mali_c55_format_info mali_c55_fmts[] = {
>> + /*
>> + * This table is missing some entries which need further work or
>> + * investigation:
>> + *
>> + * Base mode 1 is a backwards V4L2_PIX_FMT_XRGB32 with no V4L2 equivalent
>> + * Base mode 5 is "Generic Data"
>> + * Base mode 8 is a backwards V4L2_PIX_FMT_XYUV32 - no V4L2 equivalent
>> + * Base mode 9 seems to have no V4L2 equivalent
>> + * Base mode 17, 19 and 20 describe formats which seem to have no V4L2
>> + * equivalent
>> + */
>> + {
>> + .fourcc = V4L2_PIX_FMT_ARGB2101010,
>> + .mbus_codes = {
>> + MEDIA_BUS_FMT_RGB121212_1X36,
>> + MEDIA_BUS_FMT_RGB202020_1X60,
>> + },
>> + .is_raw = false,
>> + .registers = {
>> + .base_mode = MALI_C55_OUTPUT_A2R10G10B10,
>> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
>> + }
>> + },
>> + {
>> + .fourcc = V4L2_PIX_FMT_RGB565,
>> + .mbus_codes = {
>> + MEDIA_BUS_FMT_RGB121212_1X36,
>> + MEDIA_BUS_FMT_RGB202020_1X60,
>> + },
>> + .is_raw = false,
>> + .registers = {
>> + .base_mode = MALI_C55_OUTPUT_RGB565,
>> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
>> + }
>> + },
>> + {
>> + .fourcc = V4L2_PIX_FMT_BGR24,
>> + .mbus_codes = {
>> + MEDIA_BUS_FMT_RGB121212_1X36,
>> + MEDIA_BUS_FMT_RGB202020_1X60,
>> + },
>> + .is_raw = false,
>> + .registers = {
>> + .base_mode = MALI_C55_OUTPUT_RGB24,
>> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
>> + }
>> + },
>> + {
>> + .fourcc = V4L2_PIX_FMT_YUYV,
>> + .mbus_codes = {
>> + MEDIA_BUS_FMT_YUV10_1X30,
>> + },
>> + .is_raw = false,
>> + .registers = {
>> + .base_mode = MALI_C55_OUTPUT_YUY2,
>> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
>> + }
>> + },
>> + {
>> + .fourcc = V4L2_PIX_FMT_UYVY,
>> + .mbus_codes = {
>> + MEDIA_BUS_FMT_YUV10_1X30,
>> + },
>> + .is_raw = false,
>> + .registers = {
>> + .base_mode = MALI_C55_OUTPUT_UYVY,
>> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
>> + }
>> + },
>> + {
>> + .fourcc = V4L2_PIX_FMT_Y210,
>> + .mbus_codes = {
>> + MEDIA_BUS_FMT_YUV10_1X30,
>> + },
>> + .is_raw = false,
>> + .registers = {
>> + .base_mode = MALI_C55_OUTPUT_Y210,
>> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
>> + }
>> + },
>> + /*
>> + * This is something of a hack, the ISP thinks it's running NV12M but
>> + * by setting uv_plane = 0 we simply discard that planes and only output
>> + * the Y-plane.
>> + */
>> + {
>> + .fourcc = V4L2_PIX_FMT_GREY,
>> + .mbus_codes = {
>> + MEDIA_BUS_FMT_YUV10_1X30,
>> + },
>> + .is_raw = false,
>> + .registers = {
>> + .base_mode = MALI_C55_OUTPUT_NV12_21,
>> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
>> + }
>> + },
>> + {
>> + .fourcc = V4L2_PIX_FMT_NV12M,
>> + .mbus_codes = {
>> + MEDIA_BUS_FMT_YUV10_1X30,
>> + },
>> + .is_raw = false,
>> + .registers = {
>> + .base_mode = MALI_C55_OUTPUT_NV12_21,
>> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT1
>> + }
>> + },
>> + {
>> + .fourcc = V4L2_PIX_FMT_NV21M,
>> + .mbus_codes = {
>> + MEDIA_BUS_FMT_YUV10_1X30,
>> + },
>> + .is_raw = false,
>> + .registers = {
>> + .base_mode = MALI_C55_OUTPUT_NV12_21,
>> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT2
>> + }
>> + },
>> + /*
>> + * RAW uncompressed formats are all packed in 16 bpp.
>> + * TODO: Expand this list to encompass all possible RAW formats.
>> + */
>> + {
>> + .fourcc = V4L2_PIX_FMT_SRGGB16,
>> + .mbus_codes = {
>> + MEDIA_BUS_FMT_SRGGB16_1X16,
>> + },
>> + .is_raw = true,
>> + .registers = {
>> + .base_mode = MALI_C55_OUTPUT_RAW16,
>> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
>> + }
>> + },
>> + {
>> + .fourcc = V4L2_PIX_FMT_SBGGR16,
>> + .mbus_codes = {
>> + MEDIA_BUS_FMT_SBGGR16_1X16,
>> + },
>> + .is_raw = true,
>> + .registers = {
>> + .base_mode = MALI_C55_OUTPUT_RAW16,
>> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
>> + }
>> + },
>> + {
>> + .fourcc = V4L2_PIX_FMT_SGBRG16,
>> + .mbus_codes = {
>> + MEDIA_BUS_FMT_SGBRG16_1X16,
>> + },
>> + .is_raw = true,
>> + .registers = {
>> + .base_mode = MALI_C55_OUTPUT_RAW16,
>> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
>> + }
>> + },
>> + {
>> + .fourcc = V4L2_PIX_FMT_SGRBG16,
>> + .mbus_codes = {
>> + MEDIA_BUS_FMT_SGRBG16_1X16,
>> + },
>> + .is_raw = true,
>> + .registers = {
>> + .base_mode = MALI_C55_OUTPUT_RAW16,
>> + .uv_plane = MALI_C55_OUTPUT_PLANE_ALT0
>> + }
>> + },
>> +};
>> +
>> +void mali_c55_cap_dev_write(struct mali_c55_cap_dev *cap_dev, unsigned int addr,
>> + u32 val)
>> +{
>> + mali_c55_ctx_write(cap_dev->mali_c55, addr + cap_dev->reg_offset, val);
>> +}
>> +
>> +static u32 mali_c55_cap_dev_read(struct mali_c55_cap_dev *cap_dev, unsigned int addr)
>> +{
>> + return mali_c55_ctx_read(cap_dev->mali_c55, addr + cap_dev->reg_offset);
>> +}
>> +
>> +static void mali_c55_cap_dev_update_bits(struct mali_c55_cap_dev *cap_dev,
>> + unsigned int addr, u32 mask, u32 val)
>> +{
>> + u32 orig, tmp;
>> +
>> + orig = mali_c55_cap_dev_read(cap_dev, addr);
>> +
>> + tmp = orig & ~mask;
>> + tmp |= val & mask;
>> +
>> + if (tmp != orig)
>> + mali_c55_cap_dev_write(cap_dev, addr, tmp);
>> +}
>> +
>> +static bool
>> +mali_c55_mbus_code_can_produce_fmt(const struct mali_c55_format_info *fmt,
>> + u32 code)
>> +{
>> + unsigned int i;
>> +
>> + for (i = 0; i < ARRAY_SIZE(fmt->mbus_codes); i++) {
>> + if (fmt->mbus_codes[i] == code)
>> + return true;
>> + }
>> +
>> + return false;
>> +}
>> +
>> +bool mali_c55_format_is_raw(unsigned int mbus_code)
>> +{
>> + unsigned int i;
>> +
>> + for (i = 0; i < ARRAY_SIZE(mali_c55_fmts); i++) {
>> + if (mali_c55_fmts[i].mbus_codes[0] == mbus_code)
>> + return mali_c55_fmts[i].is_raw;
>> + }
>> +
>> + return false;
>> +}
>> +
>> +static const struct mali_c55_format_info *
>> +mali_c55_format_from_pix(const u32 pixelformat)
>> +{
>> + unsigned int i;
>> +
>> + for (i = 0; i < ARRAY_SIZE(mali_c55_fmts); i++) {
>> + if (mali_c55_fmts[i].fourcc == pixelformat)
>> + return &mali_c55_fmts[i];
>> + }
>> +
>> + /*
>> + * If we find no matching pixelformat, we'll just default to the first
>> + * one for now.
>> + */
>> +
>> + return &mali_c55_fmts[0];
>> +}
>> +
>> +static const char * const capture_device_names[] = {
>> + "mali-c55 fr",
>> + "mali-c55 ds",
>> +};
>> +
>> +static int mali_c55_link_validate(struct media_link *link)
>> +{
>> + struct video_device *vdev =
>> + media_entity_to_video_device(link->sink->entity);
>> + struct mali_c55_cap_dev *cap_dev = video_get_drvdata(vdev);
>> + struct v4l2_subdev *sd =
>> + media_entity_to_v4l2_subdev(link->source->entity);
>> + const struct v4l2_pix_format_mplane *pix_mp;
>> + const struct mali_c55_format_info *cap_fmt;
>> + struct v4l2_subdev_format sd_fmt = {
>> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
>> + .pad = link->source->index,
>> + };
>> + int ret;
>> +
>> + ret = v4l2_subdev_call(sd, pad, get_fmt, NULL, &sd_fmt);
>> + if (ret)
>> + return ret;
>> +
>> + pix_mp = &cap_dev->format.format;
>> + cap_fmt = cap_dev->format.info;
>> +
>> + if (sd_fmt.format.width != pix_mp->width ||
>> + sd_fmt.format.height != pix_mp->height) {
>> + dev_dbg(cap_dev->mali_c55->dev,
>> + "link '%s':%u -> '%s':%u not valid: %ux%u != %ux%u\n",
>> + link->source->entity->name, link->source->index,
>> + link->sink->entity->name, link->sink->index,
>> + sd_fmt.format.width, sd_fmt.format.height,
>> + pix_mp->width, pix_mp->height);
>> + return -EPIPE;
>> + }
>> +
>> + if (!mali_c55_mbus_code_can_produce_fmt(cap_fmt, sd_fmt.format.code)) {
>> + dev_dbg(cap_dev->mali_c55->dev,
>> + "link '%s':%u -> '%s':%u not valid: mbus_code 0x%04x cannot produce pixel format %p4cc\n",
>> + link->source->entity->name, link->source->index,
>> + link->sink->entity->name, link->sink->index,
>> + sd_fmt.format.code, &pix_mp->pixelformat);
>> + return -EPIPE;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static const struct media_entity_operations mali_c55_media_ops = {
>> + .link_validate = mali_c55_link_validate,
>> +};
>> +
>> +static int mali_c55_vb2_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
>> + unsigned int *num_planes, unsigned int sizes[],
>> + struct device *alloc_devs[])
>> +{
>> + struct mali_c55_cap_dev *cap_dev = q->drv_priv;
>> + unsigned int i;
>> +
>> + if (*num_planes) {
>> + if (*num_planes != cap_dev->format.format.num_planes)
>> + return -EINVAL;
>> +
>> + for (i = 0; i < cap_dev->format.format.num_planes; i++)
>> + if (sizes[i] < cap_dev->format.format.plane_fmt[i].sizeimage)
>> + return -EINVAL;
>> + } else {
>> + *num_planes = cap_dev->format.format.num_planes;
>> + for (i = 0; i < cap_dev->format.format.num_planes; i++)
>> + sizes[i] = cap_dev->format.format.plane_fmt[i].sizeimage;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static void mali_c55_buf_queue(struct vb2_buffer *vb)
>> +{
>> + struct mali_c55_cap_dev *cap_dev = vb2_get_drv_priv(vb->vb2_queue);
>> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>> + struct mali_c55_buffer *buf = container_of(vbuf,
>> + struct mali_c55_buffer, vb);
>> + unsigned int i;
>> +
>> + buf->planes_pending = cap_dev->format.format.num_planes;
>> +
>> + for (i = 0; i < cap_dev->format.format.num_planes; i++) {
>> + unsigned long size = cap_dev->format.format.plane_fmt[i].sizeimage;
>> +
>> + vb2_set_plane_payload(vb, i, size);
>> + }
>> +
>> + buf->vb.field = V4L2_FIELD_NONE;
>> +
>> + spin_lock(&cap_dev->buffers.lock);
>> + list_add_tail(&buf->queue, &cap_dev->buffers.queue);
>> + spin_unlock(&cap_dev->buffers.lock);
>> +}
>> +
>> +static int mali_c55_buf_init(struct vb2_buffer *vb)
>> +{
>> + struct mali_c55_cap_dev *cap_dev = vb2_get_drv_priv(vb->vb2_queue);
>> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>> + struct mali_c55_buffer *buf = container_of(vbuf,
>> + struct mali_c55_buffer, vb);
>> + unsigned int i;
>> +
>> + for (i = 0; i < cap_dev->format.format.num_planes; i++)
>> + buf->addrs[i] = vb2_dma_contig_plane_dma_addr(vb, i);
>> +
>> + return 0;
>> +}
>> +
>> +void mali_c55_set_next_buffer(struct mali_c55_cap_dev *cap_dev)
>> +{
>> + guard(spinlock)(&cap_dev->buffers.lock);
>> +
>> + cap_dev->buffers.curr = cap_dev->buffers.next;
>> + cap_dev->buffers.next = NULL;
>> +
>> + if (!list_empty(&cap_dev->buffers.queue)) {
>> + struct v4l2_pix_format_mplane *pix_mp;
>> + const struct v4l2_format_info *info;
>> + dma_addr_t *addrs;
>> +
>> + pix_mp = &cap_dev->format.format;
>> + info = v4l2_format_info(pix_mp->pixelformat);
>> +
>> + mali_c55_cap_dev_update_bits(cap_dev,
>> + MALI_C55_REG_Y_WRITER_MODE,
>> + MALI_C55_WRITER_FRAME_WRITE_MASK,
>> + MALI_C55_WRITER_FRAME_WRITE_ENABLE);
>> + if (cap_dev->format.info->registers.uv_plane)
>> + mali_c55_cap_dev_update_bits(cap_dev,
>> + MALI_C55_REG_UV_WRITER_MODE,
>> + MALI_C55_WRITER_FRAME_WRITE_MASK,
>> + MALI_C55_WRITER_FRAME_WRITE_ENABLE);
>> +
>> + cap_dev->buffers.next = list_first_entry(&cap_dev->buffers.queue,
>> + struct mali_c55_buffer,
>> + queue);
>> + list_del(&cap_dev->buffers.next->queue);
>> +
>> + addrs = cap_dev->buffers.next->addrs;
>> + mali_c55_cap_dev_write(cap_dev,
>> + MALI_C55_REG_Y_WRITER_BANKS_BASE,
>> + addrs[MALI_C55_PLANE_Y]);
>> + mali_c55_cap_dev_write(cap_dev,
>> + MALI_C55_REG_UV_WRITER_BANKS_BASE,
>> + addrs[MALI_C55_PLANE_UV]);
>> + mali_c55_cap_dev_write(cap_dev,
>> + MALI_C55_REG_Y_WRITER_OFFSET,
>> + pix_mp->width * info->bpp[MALI_C55_PLANE_Y]);
>> + mali_c55_cap_dev_write(cap_dev,
>> + MALI_C55_REG_UV_WRITER_OFFSET,
>> + pix_mp->width * info->bpp[MALI_C55_PLANE_UV]
>> + / info->hdiv);
>> + } else {
>> + /*
>> + * If we underflow then we can tell the ISP that we don't want
>> + * to write out the next frame.
>> + */
>> + mali_c55_cap_dev_update_bits(cap_dev,
>> + MALI_C55_REG_Y_WRITER_MODE,
>> + MALI_C55_WRITER_FRAME_WRITE_MASK,
>> + 0x00);
>> + mali_c55_cap_dev_update_bits(cap_dev,
>> + MALI_C55_REG_UV_WRITER_MODE,
>> + MALI_C55_WRITER_FRAME_WRITE_MASK,
>> + 0x00);
>> + }
>> +}
>> +
>> +static void mali_c55_handle_buffer(struct mali_c55_buffer *curr_buf,
>> + unsigned int framecount)
>> +{
>> + curr_buf->vb.vb2_buf.timestamp = ktime_get_boottime_ns();
>> + curr_buf->vb.sequence = framecount;
>> + vb2_buffer_done(&curr_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
>> +}
>> +
>> +/**
>> + * mali_c55_set_plane_done - mark the plane as written and process the buffer if
>> + * both planes are finished.
>> + * @cap_dev: pointer to the fr or ds pipe output
>> + * @plane: the plane to mark as completed
>> + *
>> + * The Mali C55 ISP has muliplanar outputs for some formats that come with two
>> + * separate "buffer write completed" interrupts - we need to flag each plane's
>> + * completion and check whether both planes are done - if so, complete the buf
>> + * in vb2.
>> + */
>> +void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
>> + enum mali_c55_planes plane)
>> +{
>> + struct mali_c55_isp *isp = &cap_dev->mali_c55->isp;
>> + struct mali_c55_buffer *curr_buf;
>> +
>> + guard(spinlock)(&cap_dev->buffers.lock);
>> + curr_buf = cap_dev->buffers.curr;
>> +
>> + /*
>> + * This _should_ never happen. If no buffer was available from vb2 then
>> + * we tell the ISP not to bother writing the next frame, which means the
>> + * interrupts that call this function should never trigger. If it does
>> + * happen then one of our assumptions is horribly wrong - complain
>> + * loudly and do nothing.
>> + */
>> + if (!curr_buf) {
>> + dev_err(cap_dev->mali_c55->dev, "%s null buffer in %s()\n",
>> + cap_dev->vdev.name, __func__);
>> + return;
>> + }
>> +
>> + /* If the other plane is also done... */
>> + if (!--curr_buf->planes_pending) {
>> + mali_c55_handle_buffer(curr_buf, isp->frame_sequence);
>> + cap_dev->buffers.curr = NULL;
>> + isp->frame_sequence++;
>> + }
>> +}
>> +
>> +static void mali_c55_cap_dev_stream_disable(struct mali_c55_cap_dev *cap_dev)
>> +{
>> + mali_c55_cap_dev_update_bits(cap_dev, MALI_C55_REG_Y_WRITER_MODE,
>> + MALI_C55_WRITER_FRAME_WRITE_MASK, 0x00);
>> + mali_c55_cap_dev_update_bits(cap_dev, MALI_C55_REG_UV_WRITER_MODE,
>> + MALI_C55_WRITER_FRAME_WRITE_MASK, 0x00);
>> +}
>> +
>> +static void mali_c55_cap_dev_stream_enable(struct mali_c55_cap_dev *cap_dev)
>> +{
>> + /*
>> + * The Mali ISP can hold up to 5 buffer addresses and simply cycle
>> + * through them, but it's not clear to me that the vb2 queue _guarantees_
>> + * it will queue buffers to the driver in a fixed order, and ensuring
>> + * we call vb2_buffer_done() for the right buffer seems to me to add
>> + * pointless complexity given in multi-context mode we'd need to
>> + * re-write those registers every frame anyway...so we tell the ISP to
>> + * use a single register and update it for each frame.
>> + */
>> + mali_c55_cap_dev_update_bits(cap_dev,
>> + MALI_C55_REG_Y_WRITER_BANKS_CONFIG,
>> + MALI_C55_REG_Y_WRITER_MAX_BANKS_MASK, 0);
>> + mali_c55_cap_dev_update_bits(cap_dev,
>> + MALI_C55_REG_UV_WRITER_BANKS_CONFIG,
>> + MALI_C55_REG_UV_WRITER_MAX_BANKS_MASK, 0);
>> +
>> + mali_c55_set_next_buffer(cap_dev);
>> +}
>> +
>> +static void mali_c55_cap_dev_return_buffers(struct mali_c55_cap_dev *cap_dev,
>> + enum vb2_buffer_state state)
>> +{
>> + struct mali_c55_buffer *buf, *tmp;
>> +
>> + guard(spinlock)(&cap_dev->buffers.lock);
>> +
>> + if (cap_dev->buffers.curr) {
>> + vb2_buffer_done(&cap_dev->buffers.curr->vb.vb2_buf,
>> + state);
>> + cap_dev->buffers.curr = NULL;
>> + }
>> +
>> + if (cap_dev->buffers.next) {
>> + vb2_buffer_done(&cap_dev->buffers.next->vb.vb2_buf,
>> + state);
>> + cap_dev->buffers.next = NULL;
>> + }
>> +
>> + list_for_each_entry_safe(buf, tmp, &cap_dev->buffers.queue, queue) {
>> + list_del(&buf->queue);
>> + vb2_buffer_done(&buf->vb.vb2_buf, state);
>> + }
>> +}
>> +
>> +static void mali_c55_cap_dev_format_configure(struct mali_c55_cap_dev *cap_dev)
>> +{
>> + const struct mali_c55_format_info *capture_format = cap_dev->format.info;
>> + struct v4l2_pix_format_mplane *pix_mp = &cap_dev->format.format;
> const
>
>> + const struct v4l2_format_info *info;
>> +
>> + info = v4l2_format_info(pix_mp->pixelformat);
>> + if (WARN_ON(!info))
>> + return;
>> +
>> + mali_c55_cap_dev_write(cap_dev, MALI_C55_REG_Y_WRITER_MODE,
>> + capture_format->registers.base_mode);
>> + mali_c55_cap_dev_write(cap_dev, MALI_C55_REG_ACTIVE_OUT_Y_SIZE,
>> + MALI_C55_REG_ACTIVE_OUT_SIZE_W(pix_mp->width) |
>> + MALI_C55_REG_ACTIVE_OUT_SIZE_H(pix_mp->height));
>> +
>> + if (info->mem_planes > 1) {
>> + mali_c55_cap_dev_write(cap_dev, MALI_C55_REG_UV_WRITER_MODE,
>> + capture_format->registers.base_mode);
>> + mali_c55_cap_dev_update_bits(cap_dev,
>> + MALI_C55_REG_UV_WRITER_MODE,
>> + MALI_C55_WRITER_SUBMODE_MASK,
>> + MALI_C55_WRITER_SUBMODE(capture_format->registers.uv_plane));
>> +
>> + mali_c55_cap_dev_write(cap_dev, MALI_C55_REG_ACTIVE_OUT_UV_SIZE,
>> + MALI_C55_REG_ACTIVE_OUT_SIZE_W(pix_mp->width) |
>> + MALI_C55_REG_ACTIVE_OUT_SIZE_H(pix_mp->height));
>> + }
>> +
>> + if (info->pixel_enc == V4L2_PIXEL_ENC_YUV) {
>> + mali_c55_cap_dev_write(cap_dev, MALI_C55_REG_CS_CONV_CONFIG,
>> + MALI_C55_CS_CONV_MATRIX_MASK);
>> +
>> + if (info->hdiv > 1)
>> + mali_c55_cap_dev_update_bits(cap_dev,
>> + MALI_C55_REG_CS_CONV_CONFIG,
>> + MALI_C55_CS_CONV_HORZ_DOWNSAMPLE_MASK,
>> + MALI_C55_CS_CONV_HORZ_DOWNSAMPLE_ENABLE);
>> + if (info->vdiv > 1)
>> + mali_c55_cap_dev_update_bits(cap_dev,
>> + MALI_C55_REG_CS_CONV_CONFIG,
>> + MALI_C55_CS_CONV_VERT_DOWNSAMPLE_MASK,
>> + MALI_C55_CS_CONV_VERT_DOWNSAMPLE_ENABLE);
>> + if (info->hdiv > 1 || info->vdiv > 1)
>> + mali_c55_cap_dev_update_bits(cap_dev,
>> + MALI_C55_REG_CS_CONV_CONFIG,
>> + MALI_C55_CS_CONV_FILTER_MASK,
>> + MALI_C55_CS_CONV_FILTER_ENABLE);
>> + }
>> +}
>> +
>> +static int mali_c55_vb2_start_streaming(struct vb2_queue *q, unsigned int count)
>> +{
>> + struct mali_c55_cap_dev *cap_dev = q->drv_priv;
>> + struct mali_c55 *mali_c55 = cap_dev->mali_c55;
>> + struct mali_c55_resizer *rsz = cap_dev->rsz;
>> + struct mali_c55_isp *isp = &mali_c55->isp;
>> + int ret;
>> +
>> + guard(mutex)(&isp->capture_lock);
>> +
>> + ret = pm_runtime_resume_and_get(mali_c55->dev);
>> + if (ret)
>> + return ret;
>> +
>> + mali_c55_cap_dev_format_configure(cap_dev);
> I think this should go after video_device_pipeline_start(), to avoid
> configuring the device needlessly if pipeline validation fails.
>
>> +
>> + ret = video_device_pipeline_start(&cap_dev->vdev,
>> + &cap_dev->mali_c55->pipe);
>> + if (ret) {
>> + dev_dbg(mali_c55->dev, "%s failed to start media pipeline\n",
>> + cap_dev->vdev.name);
>> + goto err_pm_put;
>> + }
>> +
>> + mali_c55_cap_dev_stream_enable(cap_dev);
>> +
>> + ret = v4l2_subdev_enable_streams(&rsz->sd, MALI_C55_RSZ_SOURCE_PAD,
>> + BIT(0));
>> + if (ret)
>> + goto err_disable_cap_dev;
>> +
>> + if (mali_c55->pipe.start_count == mali_c55->pipe.required_count) {
>> + /*
>> + * The ISP has a "processed" and "bypass" source pad, but they
>> + * don't have a separate start process so we'll just start the
>> + * processed pad unconditionally.
>> + */
>> + ret = v4l2_subdev_enable_streams(&mali_c55->isp.sd,
>> + MALI_C55_ISP_PAD_SOURCE_VIDEO,
>> + BIT(0));
> The resizer subdev is supposed to start its source, but I'll ignore that
> :-)
No need! It's an easy change...done.
>> + if (ret)
>> + goto err_disable_rsz;
>> + }
>> +
>> + return 0;
>> +
>> +err_disable_rsz:
>> + v4l2_subdev_disable_streams(&rsz->sd, MALI_C55_RSZ_SOURCE_PAD, BIT(0));
>> +err_disable_cap_dev:
>> + mali_c55_cap_dev_stream_disable(cap_dev);
>> + video_device_pipeline_stop(&cap_dev->vdev);
>> +err_pm_put:
>> + pm_runtime_put(mali_c55->dev);
>> + mali_c55_cap_dev_return_buffers(cap_dev, VB2_BUF_STATE_QUEUED);
>> +
>> + return ret;
>> +}
>> +
>> +static void mali_c55_vb2_stop_streaming(struct vb2_queue *q)
>> +{
>> + struct mali_c55_cap_dev *cap_dev = q->drv_priv;
>> + struct mali_c55 *mali_c55 = cap_dev->mali_c55;
>> + struct mali_c55_resizer *rsz = cap_dev->rsz;
>> + struct mali_c55_isp *isp = &mali_c55->isp;
>> +
>> + guard(mutex)(&isp->capture_lock);
>> +
>> + v4l2_subdev_disable_streams(&mali_c55->isp.sd,
>> + MALI_C55_ISP_PAD_SOURCE_VIDEO, BIT(0));
> Shouldn't this be done only when you stop the first video device, to
> match how you start the ISP ?
Ah - I guess that answers that question from before...
>
>> + v4l2_subdev_disable_streams(&rsz->sd, MALI_C55_RSZ_SOURCE_PAD, BIT(0));
>> + mali_c55_cap_dev_stream_disable(cap_dev);
>> + mali_c55_cap_dev_return_buffers(cap_dev, VB2_BUF_STATE_ERROR);
>> + video_device_pipeline_stop(&cap_dev->vdev);
>> + pm_runtime_put_autosuspend(mali_c55->dev);
> pm_runtime_mark_last_busy(...);
>
>> +}
>> +
>> +static const struct vb2_ops mali_c55_vb2_ops = {
>> + .queue_setup = &mali_c55_vb2_queue_setup,
>> + .buf_queue = &mali_c55_buf_queue,
>> + .buf_init = &mali_c55_buf_init,
>> + .wait_prepare = vb2_ops_wait_prepare,
>> + .wait_finish = vb2_ops_wait_finish,
>> + .start_streaming = &mali_c55_vb2_start_streaming,
>> + .stop_streaming = &mali_c55_vb2_stop_streaming,
>> +};
>> +
>> +static const struct v4l2_file_operations mali_c55_v4l2_fops = {
>> + .owner = THIS_MODULE,
>> + .unlocked_ioctl = video_ioctl2,
>> + .open = v4l2_fh_open,
>> + .release = vb2_fop_release,
>> + .poll = vb2_fop_poll,
>> + .mmap = vb2_fop_mmap,
>> +};
>> +
>> +static void mali_c55_try_fmt(struct v4l2_pix_format_mplane *pix_mp)
>> +{
>> + const struct mali_c55_format_info *capture_format;
>> + const struct v4l2_format_info *info;
>> + struct v4l2_plane_pix_format *plane, *y_plane;
>> + unsigned int padding;
>> + unsigned int i;
>> +
>> + capture_format = mali_c55_format_from_pix(pix_mp->pixelformat);
>> + pix_mp->pixelformat = capture_format->fourcc;
>> +
>> + pix_mp->width = clamp(pix_mp->width, MALI_C55_MIN_WIDTH,
>> + MALI_C55_MAX_WIDTH);
>> + pix_mp->height = clamp(pix_mp->height, MALI_C55_MIN_HEIGHT,
>> + MALI_C55_MAX_HEIGHT);
>> +
>> + pix_mp->field = V4L2_FIELD_NONE;
>> + pix_mp->colorspace = V4L2_COLORSPACE_DEFAULT;
>> + pix_mp->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>> + pix_mp->quantization = V4L2_QUANTIZATION_DEFAULT;
>> +
>> + info = v4l2_format_info(pix_mp->pixelformat);
>> + pix_mp->num_planes = info->mem_planes;
>> + memset(pix_mp->plane_fmt, 0, sizeof(pix_mp->plane_fmt));
>> +
>> + y_plane = &pix_mp->plane_fmt[0];
>> + y_plane->bytesperline = clamp(y_plane->bytesperline,
>> + info->bpp[0] * pix_mp->width, 65535U);
> It's nice that you make the stride configurable, but unless I'm missing
> something, the setting never makes it to the hardware.
Oops!
>
>> + y_plane->sizeimage = y_plane->bytesperline * pix_mp->height;
>> +
>> + padding = y_plane->bytesperline - (pix_mp->width * info->bpp[0]);
>> +
>> + for (i = 1; i < info->comp_planes; i++) {
>> + plane = &pix_mp->plane_fmt[i];
>> +
>> + plane->bytesperline = DIV_ROUND_UP(info->bpp[i] * pix_mp->width,
>> + info->hdiv) + padding;
> What's the hardware constraint for the stride for multi-planar formats ?
> Is there a single stride register, or one per plane ? If there's a
> single one, I would be surprised if the stride for the second and
> subsequent planes would be calculated by the hardware with the above
> formula.
One per plane...let me revisit this then.
>
>> + plane->sizeimage = DIV_ROUND_UP(
>> + plane->bytesperline * pix_mp->height,
>> + info->vdiv);
>> + }
>> +
>> + if (info->mem_planes == 1) {
>> + for (i = 1; i < info->comp_planes; i++) {
>> + plane = &pix_mp->plane_fmt[i];
>> + y_plane->sizeimage += plane->sizeimage;
>> + }
>> + }
>> +}
>> +
>> +static int mali_c55_try_fmt_vid_cap_mplane(struct file *file, void *fh,
>> + struct v4l2_format *f)
>> +{
>> + mali_c55_try_fmt(&f->fmt.pix_mp);
>> +
>> + return 0;
>> +}
>> +
>> +static void mali_c55_set_format(struct mali_c55_cap_dev *cap_dev,
>> + struct v4l2_pix_format_mplane *pix_mp)
>> +{
>> + mali_c55_try_fmt(pix_mp);
>> +
>> + cap_dev->format.format = *pix_mp;
>> + cap_dev->format.info = mali_c55_format_from_pix(pix_mp->pixelformat);
> You could avoid a double lookup by returning the info pointer from
> mali_c55_try_fmt().
>
>> +}
>> +
>> +static int mali_c55_s_fmt_vid_cap_mplane(struct file *file, void *fh,
>> + struct v4l2_format *f)
>> +{
>> + struct mali_c55_cap_dev *cap_dev = video_drvdata(file);
>> +
>> + if (vb2_is_busy(&cap_dev->queue))
>> + return -EBUSY;
>> +
>> + mali_c55_set_format(cap_dev, &f->fmt.pix_mp);
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_g_fmt_vid_cap_mplane(struct file *file, void *fh,
>> + struct v4l2_format *f)
>> +{
>> + struct mali_c55_cap_dev *cap_dev = video_drvdata(file);
>> +
>> + f->fmt.pix_mp = cap_dev->format.format;
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_enum_fmt_vid_cap_mplane(struct file *file, void *fh,
>> + struct v4l2_fmtdesc *f)
>> +{
>> + struct mali_c55_cap_dev *cap_dev = video_drvdata(file);
>> + unsigned int j = 0;
>> + unsigned int i;
>> +
>> + for (i = 0; i < ARRAY_SIZE(mali_c55_fmts); i++) {
>> + if (f->mbus_code &&
>> + !mali_c55_mbus_code_can_produce_fmt(&mali_c55_fmts[i],
>> + f->mbus_code))
>> + continue;
>> +
>> + /* Downscale pipe can't output RAW formats */
>> + if (mali_c55_fmts[i].is_raw &&
>> + cap_dev->reg_offset == MALI_C55_CAP_DEV_DS_REG_OFFSET)
>> + continue;
>> +
>> + if (j++ == f->index) {
>> + f->pixelformat = mali_c55_fmts[i].fourcc;
>> + return 0;
>> + }
>> + }
>> +
>> + return -EINVAL;
>> +}
>> +
>> +static int mali_c55_querycap(struct file *file, void *fh,
>> + struct v4l2_capability *cap)
>> +{
>> + strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver));
>> + strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card));
>> +
>> + return 0;
>> +}
>> +
>> +static const struct v4l2_ioctl_ops mali_c55_v4l2_ioctl_ops = {
>> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
>> + .vidioc_querybuf = vb2_ioctl_querybuf,
>> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
>> + .vidioc_qbuf = vb2_ioctl_qbuf,
>> + .vidioc_expbuf = vb2_ioctl_expbuf,
>> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
>> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>> + .vidioc_streamon = vb2_ioctl_streamon,
>> + .vidioc_streamoff = vb2_ioctl_streamoff,
>> + .vidioc_try_fmt_vid_cap_mplane = mali_c55_try_fmt_vid_cap_mplane,
>> + .vidioc_s_fmt_vid_cap_mplane = mali_c55_s_fmt_vid_cap_mplane,
>> + .vidioc_g_fmt_vid_cap_mplane = mali_c55_g_fmt_vid_cap_mplane,
>> + .vidioc_enum_fmt_vid_cap = mali_c55_enum_fmt_vid_cap_mplane,
>> + .vidioc_querycap = mali_c55_querycap,
>> + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
>> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
>> +};
>> +
>> +static int mali_c55_register_cap_dev(struct mali_c55 *mali_c55,
>> + enum mali_c55_cap_devs cap_dev_id)
>> +{
>> + struct mali_c55_cap_dev *cap_dev = &mali_c55->cap_devs[cap_dev_id];
>> + struct v4l2_pix_format_mplane pix_mp;
>> + struct video_device *vdev;
>> + struct vb2_queue *vb2q;
>> + int ret;
>> +
>> + vdev = &cap_dev->vdev;
>> + vb2q = &cap_dev->queue;
>> +
>> + cap_dev->mali_c55 = mali_c55;
>> + mutex_init(&cap_dev->lock);
>> + INIT_LIST_HEAD(&cap_dev->buffers.queue);
>> +
>> + switch (cap_dev_id) {
>> + case MALI_C55_CAP_DEV_FR:
>> + cap_dev->rsz = &mali_c55->resizers[MALI_C55_RSZ_FR];
>> + cap_dev->reg_offset = MALI_C55_CAP_DEV_FR_REG_OFFSET;
>> + break;
>> + case MALI_C55_CAP_DEV_DS:
>> + cap_dev->rsz = &mali_c55->resizers[MALI_C55_RSZ_DS];
>> + cap_dev->reg_offset = MALI_C55_CAP_DEV_DS_REG_OFFSET;
>> + break;
>> + default:
>> + ret = -EINVAL;
>> + goto err_destroy_mutex;
>> + }
>> +
>> + cap_dev->pad.flags = MEDIA_PAD_FL_SINK;
>> + ret = media_entity_pads_init(&cap_dev->vdev.entity, 1, &cap_dev->pad);
>> + if (ret) {
>> + mutex_destroy(&cap_dev->lock);
>> + goto err_destroy_mutex;
>> + }
>> +
>> + vb2q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
>> + vb2q->io_modes = VB2_MMAP | VB2_DMABUF;
>> + vb2q->drv_priv = cap_dev;
>> + vb2q->mem_ops = &vb2_dma_contig_memops;
>> + vb2q->ops = &mali_c55_vb2_ops;
>> + vb2q->buf_struct_size = sizeof(struct mali_c55_buffer);
>> + vb2q->min_queued_buffers = 1;
>> + vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
>> + vb2q->lock = &cap_dev->lock;
>> + vb2q->dev = mali_c55->dev;
>> +
>> + ret = vb2_queue_init(vb2q);
>> + if (ret) {
>> + dev_err(mali_c55->dev, "%s vb2 queue init failed\n",
>> + cap_dev->vdev.name);
>> + goto err_cleanup_media_entity;
>> + }
>> +
>> + strscpy(cap_dev->vdev.name, capture_device_names[cap_dev_id],
>> + sizeof(cap_dev->vdev.name));
>> + vdev->release = video_device_release_empty;
>> + vdev->fops = &mali_c55_v4l2_fops;
>> + vdev->ioctl_ops = &mali_c55_v4l2_ioctl_ops;
>> + vdev->lock = &cap_dev->lock;
>> + vdev->v4l2_dev = &mali_c55->v4l2_dev;
>> + vdev->queue = &cap_dev->queue;
>> + vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE_MPLANE |
>> + V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
>> + vdev->entity.ops = &mali_c55_media_ops;
>> + video_set_drvdata(vdev, cap_dev);
>> +
>> + memset(&pix_mp, 0, sizeof(pix_mp));
>> + pix_mp.pixelformat = V4L2_PIX_FMT_RGB565;
>> + pix_mp.width = MALI_C55_DEFAULT_WIDTH;
>> + pix_mp.height = MALI_C55_DEFAULT_HEIGHT;
>> + mali_c55_set_format(cap_dev, &pix_mp);
>> +
>> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>> + if (ret) {
>> + dev_err(mali_c55->dev,
>> + "%s failed to register video device\n",
>> + cap_dev->vdev.name);
>> + goto err_release_vb2q;
>> + }
>> +
>> + return 0;
>> +
>> +err_release_vb2q:
>> + vb2_queue_release(vb2q);
>> +err_cleanup_media_entity:
>> + media_entity_cleanup(&cap_dev->vdev.entity);
>> +err_destroy_mutex:
>> + mutex_destroy(&cap_dev->lock);
>> +
>> + return ret;
>> +}
>> +
>> +int mali_c55_register_capture_devs(struct mali_c55 *mali_c55)
>> +{
>> + int ret;
>> +
>> + ret = mali_c55_register_cap_dev(mali_c55, MALI_C55_CAP_DEV_FR);
>> + if (ret)
>> + return ret;
>> +
>> + if (mali_c55->capabilities & MALI_C55_GPS_DS_PIPE_FITTED) {
>> + ret = mali_c55_register_cap_dev(mali_c55, MALI_C55_CAP_DEV_DS);
>> + if (ret) {
>> + mali_c55_unregister_capture_devs(mali_c55);
>> + return ret;
>> + }
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static void mali_c55_unregister_cap_dev(struct mali_c55 *mali_c55,
>> + enum mali_c55_cap_devs cap_dev_id)
>> +{
>> + struct mali_c55_cap_dev *cap_dev = &mali_c55->cap_devs[cap_dev_id];
>> +
>> + if (!video_is_registered(&cap_dev->vdev))
>> + return;
>> +
>> + vb2_video_unregister_device(&cap_dev->vdev);
>> + media_entity_cleanup(&cap_dev->vdev.entity);
>> + mutex_destroy(&cap_dev->lock);
>> +}
>> +
>> +void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55)
>> +{
>> + mali_c55_unregister_cap_dev(mali_c55, MALI_C55_CAP_DEV_FR);
>> + if (mali_c55->capabilities & MALI_C55_GPS_DS_PIPE_FITTED)
>> + mali_c55_unregister_cap_dev(mali_c55, MALI_C55_CAP_DEV_DS);
>> +}
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
>> new file mode 100644
>> index 000000000000..f7764a938e9f
>> --- /dev/null
>> @@ -0,0 +1,247 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + * ARM Mali-C55 ISP Driver - Common definitions
>> + *
>> + * Copyright (C) 2024 Ideas on Board Oy
>> + */
>> +
>> +#ifndef _MALI_C55_COMMON_H
>> +#define _MALI_C55_COMMON_H
>> +
>> +#include <linux/clk.h>
>> +#include <linux/io.h>
>> +#include <linux/list.h>
>> +#include <linux/mutex.h>
>> +#include <linux/spinlock.h>
>> +#include <linux/videodev2.h>
>> +
>> +#include <media/media-device.h>
>> +#include <media/v4l2-async.h>
>> +#include <media/v4l2-ctrls.h>
>> +#include <media/v4l2-dev.h>
>> +#include <media/v4l2-device.h>
>> +#include <media/v4l2-subdev.h>
>> +#include <media/videobuf2-core.h>
>> +#include <media/videobuf2-v4l2.h>
>> +
>> +#define MALI_C55_DRIVER_NAME "mali-c55"
>> +
>> +/* min and max values for the image sizes */
>> +#define MALI_C55_MIN_WIDTH 640U
>> +#define MALI_C55_MIN_HEIGHT 480U
>> +#define MALI_C55_MAX_WIDTH 8192U
>> +#define MALI_C55_MAX_HEIGHT 8192U
>> +#define MALI_C55_DEFAULT_WIDTH 1920U
>> +#define MALI_C55_DEFAULT_HEIGHT 1080U
>> +
>> +#define MALI_C55_DEFAULT_MEDIA_BUS_FMT MEDIA_BUS_FMT_RGB121212_1X36
>> +
>> +#define MALI_C55_NUM_CLKS 2
>> +
>> +struct device;
>> +struct dma_chan;
>> +struct mali_c55;
>> +struct mali_c55_cap_dev;
>> +struct platform_device;
>> +struct resource;
>> +
>> +enum mali_c55_isp_pads {
>> + MALI_C55_ISP_PAD_SINK_VIDEO,
>> + MALI_C55_ISP_PAD_SOURCE_VIDEO,
>> + MALI_C55_ISP_PAD_SOURCE_BYPASS,
>> + MALI_C55_ISP_NUM_PADS,
>> +};
>> +
>> +struct mali_c55_tpg {
>> + struct mali_c55 *mali_c55;
>> + struct v4l2_subdev sd;
>> + struct media_pad pad;
>> + struct mali_c55_tpg_ctrls {
>> + struct v4l2_ctrl_handler handler;
>> + struct v4l2_ctrl *vblank;
>> + } ctrls;
>> +};
>> +
>> +struct mali_c55_isp {
>> + struct mali_c55 *mali_c55;
>> + struct v4l2_subdev sd;
>> + struct media_pad pads[MALI_C55_ISP_NUM_PADS];
>> + struct media_pad *remote_src;
>> + /* Mutex to guard vb2 start/stop streaming */
>> + struct mutex capture_lock;
>> + unsigned int frame_sequence;
>> +};
>> +
>> +enum mali_c55_resizer_ids {
>> + MALI_C55_RSZ_FR,
>> + MALI_C55_RSZ_DS,
>> + MALI_C55_NUM_RSZS,
>> +};
>> +
>> +enum mali_c55_rsz_pads {
>> + MALI_C55_RSZ_SINK_PAD,
>> + MALI_C55_RSZ_SOURCE_PAD,
>> + MALI_C55_RSZ_SINK_BYPASS_PAD,
>> + MALI_C55_RSZ_NUM_PADS
>> +};
>> +
>> +struct mali_c55_resizer {
>> + struct mali_c55 *mali_c55;
>> + struct mali_c55_cap_dev *cap_dev;
>> + enum mali_c55_resizer_ids id;
>> + struct v4l2_subdev sd;
>> + struct media_pad pads[MALI_C55_RSZ_NUM_PADS];
>> + unsigned int num_routes;
>> + bool streaming;
>> +};
>> +
>> +enum mali_c55_cap_devs {
>> + MALI_C55_CAP_DEV_FR,
>> + MALI_C55_CAP_DEV_DS,
>> + MALI_C55_NUM_CAP_DEVS
>> +};
>> +
>> +struct mali_c55_format_info {
>> + u32 fourcc;
>> + /*
>> + * The output formats can be produced by a couple of different media bus
>> + * formats, depending on how the ISP is configured.
>> + */
>> + unsigned int mbus_codes[2];
>> + bool is_raw;
>> + struct {
>> + u32 base_mode;
>> + u32 uv_plane;
>> + } registers;
>> +};
>> +
>> +struct mali_c55_isp_fmt {
> struct mali_c55_isp_format_info
>
>> + u32 code;
>> + bool bypass;
>> + u32 order;
>> +};
>> +
>> +enum mali_c55_planes {
>> + MALI_C55_PLANE_Y,
>> + MALI_C55_PLANE_UV,
>> + MALI_C55_NUM_PLANES
>> +};
>> +
>> +struct mali_c55_buffer {
>> + struct vb2_v4l2_buffer vb;
>> + unsigned int planes_pending;
>> + struct list_head queue;
>> + dma_addr_t addrs[MALI_C55_NUM_PLANES];
>> +};
>> +
>> +struct mali_c55_cap_dev {
>> + struct mali_c55 *mali_c55;
>> + struct mali_c55_resizer *rsz;
>> + struct video_device vdev;
>> + struct media_pad pad;
>> + struct vb2_queue queue;
>> + /* Mutex to provide to vb2 */
>> + struct mutex lock;
>> + unsigned int reg_offset;
>> +
>> + struct {
>> + const struct mali_c55_format_info *info;
>> + struct v4l2_pix_format_mplane format;
>> + } format;
>> +
>> + struct {
>> + /* Spinlock to guard buffer queue */
>> + spinlock_t lock;
>> + struct list_head queue;
>> + struct mali_c55_buffer *curr;
>> + struct mali_c55_buffer *next;
>> + } buffers;
>> +
>> + bool streaming;
>> +};
>> +
>> +enum mali_c55_config_spaces {
>> + MALI_C55_CONFIG_PING,
>> + MALI_C55_CONFIG_PONG,
>> + MALI_C55_NUM_CONFIG_SPACES
> The last enumerator isn't used.
>
>> +};
>> +
>> +/**
>> + * struct mali_c55_context - Fields relating to a single camera context
>> + *
>> + * @mali_c55: Pointer to the main struct mali_c55
>> + * @registers: A pointer to some allocated memory holding register
>> + * values to be written to the hardware at frame interrupt
>> + * @base: Base address of the config space in the hardware
>> + * @lock: A spinlock to protect against writes to @registers whilst that
>> + * space is being copied to the hardware
>> + * @list: A list head to facilitate a context queue
>> + */
>> +struct mali_c55_context {
>> + struct mali_c55 *mali_c55;
>> + void *registers;
>> + phys_addr_t base;
>> + /* Spinlock to prevent simultaneous access of register space */
>> + spinlock_t lock;
>> + struct list_head list;
>> +};
>> +
>> +struct mali_c55 {
>> + struct device *dev;
>> + void __iomem *base;
>> + struct dma_chan *channel;
>> + struct clk_bulk_data clks[MALI_C55_NUM_CLKS];
>> +
>> + u16 capabilities;
>> + struct media_device media_dev;
>> + struct v4l2_device v4l2_dev;
>> + struct v4l2_async_notifier notifier;
>> + struct media_pipeline pipe;
>> +
>> + struct mali_c55_tpg tpg;
>> + struct mali_c55_isp isp;
>> + struct mali_c55_resizer resizers[MALI_C55_NUM_RSZS];
>> + struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS];
>> +
>> + struct mali_c55_context context;
>> + enum mali_c55_config_spaces next_config;
>> +};
>> +
>> +void mali_c55_write(struct mali_c55 *mali_c55, unsigned int addr, u32 val);
>> +void mali_c55_cap_dev_write(struct mali_c55_cap_dev *cap_dev, unsigned int addr,
>> + u32 val);
>> +void mali_c55_update_bits(struct mali_c55 *mali_c55, unsigned int addr,
>> + u32 mask, u32 val);
>> +u32 mali_c55_read(struct mali_c55 *mali_c55, unsigned int addr);
>> +void mali_c55_ctx_write(struct mali_c55 *mali_c55, unsigned int addr, u32 val);
>> +u32 mali_c55_ctx_read(struct mali_c55 *mali_c55, unsigned int addr);
>> +void mali_c55_ctx_update_bits(struct mali_c55 *mali_c55, unsigned int addr,
>> + u32 mask, u32 val);
>> +
>> +int mali_c55_config_write(struct mali_c55_context *ctx,
>> + enum mali_c55_config_spaces cfg_space);
>> +
>> +int mali_c55_register_isp(struct mali_c55 *mali_c55);
>> +int mali_c55_register_tpg(struct mali_c55 *mali_c55);
>> +void mali_c55_unregister_tpg(struct mali_c55 *mali_c55);
>> +void mali_c55_unregister_isp(struct mali_c55 *mali_c55);
>> +int mali_c55_register_resizers(struct mali_c55 *mali_c55);
>> +void mali_c55_unregister_resizers(struct mali_c55 *mali_c55);
>> +int mali_c55_register_capture_devs(struct mali_c55 *mali_c55);
>> +void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55);
>> +struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55);
>> +void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
>> + enum mali_c55_planes plane);
>> +void mali_c55_set_next_buffer(struct mali_c55_cap_dev *cap_dev);
>> +void mali_c55_isp_queue_event_sof(struct mali_c55 *mali_c55);
>> +
>> +bool mali_c55_format_is_raw(unsigned int mbus_code);
>> +
>> +const struct mali_c55_isp_fmt *
>> +mali_c55_isp_fmt_next(const struct mali_c55_isp_fmt *fmt);
>> +const struct mali_c55_isp_fmt *
>> +mali_c55_isp_get_mbus_config_by_code(u32 code);
>> +const struct mali_c55_isp_fmt *
>> +mali_c55_isp_get_mbus_config_by_index(u32 index);
>> +
>> +#endif /* _MALI_C55_COMMON_H */
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
>> new file mode 100644
>> index 000000000000..dbc07d12d3a3
>> --- /dev/null
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
>> @@ -0,0 +1,893 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * ARM Mali-C55 ISP Driver - Core driver code
>> + *
>> + * Copyright (C) 2024 Ideas on Board Oy
>> + */
>> +
>> +#include <linux/bitops.h>
>> +#include <linux/cleanup.h>
>> +#include <linux/clk.h>
>> +#include <linux/delay.h>
>> +#include <linux/device.h>
>> +#include <linux/dmaengine.h>
>> +#include <linux/dma-mapping.h>
>> +#include <linux/interrupt.h>
>> +#include <linux/iopoll.h>
>> +#include <linux/ioport.h>
>> +#include <linux/mod_devicetable.h>
>> +#include <linux/of.h>
>> +#include <linux/of_reserved_mem.h>
>> +#include <linux/platform_device.h>
>> +#include <linux/pm_runtime.h>
>> +#include <linux/slab.h>
>> +#include <linux/string.h>
>> +
>> +#include <media/media-entity.h>
>> +#include <media/v4l2-device.h>
>> +#include <media/v4l2-mc.h>
>> +#include <media/videobuf2-dma-contig.h>
>> +
>> +#include "mali-c55-common.h"
>> +#include "mali-c55-registers.h"
>> +
>> +static const char * const mali_c55_interrupt_names[] = {
>> + [MALI_C55_IRQ_ISP_START] = "ISP start",
>> + [MALI_C55_IRQ_ISP_DONE] = "ISP done",
>> + [MALI_C55_IRQ_MCM_ERROR] = "Multi-context management error",
>> + [MALI_C55_IRQ_BROKEN_FRAME_ERROR] = "Broken frame error",
>> + [MALI_C55_IRQ_MET_AF_DONE] = "AF metering done",
>> + [MALI_C55_IRQ_MET_AEXP_DONE] = "AEXP metering done",
>> + [MALI_C55_IRQ_MET_AWB_DONE] = "AWB metering done",
>> + [MALI_C55_IRQ_AEXP_1024_DONE] = "AEXP 1024-bit histogram done",
>> + [MALI_C55_IRQ_IRIDIX_MET_DONE] = "Iridix metering done",
>> + [MALI_C55_IRQ_LUT_INIT_DONE] = "LUT memory init done",
>> + [MALI_C55_IRQ_FR_Y_DONE] = "Full resolution Y plane DMA done",
>> + [MALI_C55_IRQ_FR_UV_DONE] = "Full resolution U/V plane DMA done",
>> + [MALI_C55_IRQ_DS_Y_DONE] = "Downscale Y plane DMA done",
>> + [MALI_C55_IRQ_DS_UV_DONE] = "Downscale U/V plane DMA done",
>> + [MALI_C55_IRQ_LINEARIZATION_DONE] = "Linearisation done",
>> + [MALI_C55_IRQ_RAW_FRONTEND_DONE] = "Raw frontend processing done",
>> + [MALI_C55_IRQ_NOISE_REDUCTION_DONE] = "Noise reduction done",
>> + [MALI_C55_IRQ_IRIDIX_DONE] = "Iridix done",
>> + [MALI_C55_IRQ_BAYER2RGB_DONE] = "Bayer to RGB conversion done",
>> + [MALI_C55_IRQ_WATCHDOG_TIMER] = "Watchdog timer timed out",
>> + [MALI_C55_IRQ_FRAME_COLLISION] = "Frame collision error",
>> + [MALI_C55_IRQ_UNUSED] = "IRQ bit unused",
>> + [MALI_C55_IRQ_DMA_ERROR] = "DMA error",
>> + [MALI_C55_IRQ_INPUT_STOPPED] = "Input port safely stopped",
>> + [MALI_C55_IRQ_MET_AWB_TARGET1_HIT] = "AWB metering target 1 address hit",
>> + [MALI_C55_IRQ_MET_AWB_TARGET2_HIT] = "AWB metering target 2 address hit"
>> +};
>> +
>> +static const unsigned int config_space_addrs[] = {
>> + [MALI_C55_CONFIG_PING] = 0x0ab6c,
>> + [MALI_C55_CONFIG_PONG] = 0x22b2c,
>> +};
>> +
>> +static const char * const mali_c55_clk_names[] = {
> static const char * const mali_c55_clk_names[MALI_C55_NUM_CLKS] = {
>
> so the compiler will warn you if you have more entries in the array than
> expected.
>
>> + "aclk",
>> + "hclk",
>> +};
>> +
>> +/*
>> + * System IO
>> + *
>> + * The Mali-C55 ISP has up to two configuration register spaces (called 'ping'
>> + * and 'pong'), with the expectation that the 'active' space will be left
>> + * untouched whilst a frame is being processed and the 'inactive' space
>> + * configured ready to be switched to during the blanking period before the next
>> + * frame processing starts. These spaces should ideally be set via DMA transfer
>> + * from a buffer rather than through individual register set operations. There
>> + * is also a shared global register space which should be set normally. Of
>> + * course, the ISP might be included in a system which lacks a suitable DMA
>> + * engine, and the second configuration space might not be fitted at all, which
>> + * means we need to support four scenarios:
>> + *
>> + * 1. Multi config space, with DMA engine.
>> + * 2. Multi config space, no DMA engine.
>> + * 3. Single config space, with DMA engine.
>> + * 4. Single config space, no DMA engine.
>> + *
>> + * The first case is very easy, but the rest present annoying problems. The best
>> + * way to solve them seems to be simply to replicate the concept of DMAing over
>> + * the configuration buffer even if there's no DMA engine on the board, for
>> + * which we rely on memcpy. To facilitate this any read/write call that is made
>> + * to an address within those config spaces should infact be directed to a
>> + * buffer that was allocated to hold them rather than the IO memory itself. The
>> + * actual copy of that buffer to IO mem will happen on interrupt.
>> + */
>> +
>> +void mali_c55_write(struct mali_c55 *mali_c55, unsigned int addr, u32 val)
>> +{
>> + WARN_ON(addr >= MALI_C55_REG_CONFIG_SPACES_OFFSET);
>> +
>> + writel(val, mali_c55->base + addr);
>> +}
>> +
>> +u32 mali_c55_read(struct mali_c55 *mali_c55, unsigned int addr)
>> +{
>> + WARN_ON(addr >= MALI_C55_REG_CONFIG_SPACES_OFFSET);
>> +
>> + return readl(mali_c55->base + addr);
>> +}
>> +
>> +void mali_c55_update_bits(struct mali_c55 *mali_c55, unsigned int addr,
>> + u32 mask, u32 val)
>> +{
>> + u32 orig, tmp;
> s/tmp/new/
>
>> +
>> + orig = mali_c55_read(mali_c55, addr);
>> +
>> + tmp = orig & ~mask;
>> + tmp |= val & mask;
>> +
>> + if (tmp != orig)
>> + mali_c55_write(mali_c55, addr, tmp);
>> +}
>> +
>> +static void __mali_c55_ctx_write(struct mali_c55_context *ctx,
>> + unsigned int addr, u32 val)
>> +{
>> + spin_lock(&ctx->lock);
>> + addr = (addr - MALI_C55_REG_CONFIG_SPACES_OFFSET) / 4;
>> + ((u32 *)ctx->registers)[addr] = val;
> ctx->registers could be a u32 *, it would simplify code here.
>
>> + spin_unlock(&ctx->lock);
>> +}
>> +
>> +void mali_c55_ctx_write(struct mali_c55 *mali_c55, unsigned int addr, u32 val)
>> +{
>> + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
> That's OK for now, but I think the context will need to be passed to
> this function. I'd say you could prepare for it already, but doing it
> later is likely better.
>
>> +
>> + WARN_ON(addr < MALI_C55_REG_CONFIG_SPACES_OFFSET);
>> + __mali_c55_ctx_write(ctx, addr, val);
>> +}
>> +
>> +static u32 __mali_c55_ctx_read(struct mali_c55_context *ctx, unsigned int addr)
>> +{
>> + u32 val;
>> +
>> + spin_lock(&ctx->lock);
>> + addr = (addr - MALI_C55_REG_CONFIG_SPACES_OFFSET) / 4;
> This can be done before taking the spinlock.
>
>> + val = ((u32 *)ctx->registers)[addr];
>> + spin_unlock(&ctx->lock);
>> +
>> + return val;
>> +}
>> +
>> +u32 mali_c55_ctx_read(struct mali_c55 *mali_c55, unsigned int addr)
>> +{
>> + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
>> +
>> + WARN_ON(addr < MALI_C55_REG_CONFIG_SPACES_OFFSET);
>> + return __mali_c55_ctx_read(ctx, addr);
>> +}
>> +
>> +void mali_c55_ctx_update_bits(struct mali_c55 *mali_c55, unsigned int addr,
>> + u32 mask, u32 val)
>> +{
>> + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
>> + u32 orig, tmp;
>> +
>> + WARN_ON(addr < MALI_C55_REG_CONFIG_SPACES_OFFSET);
>> +
>> + orig = __mali_c55_ctx_read(ctx, addr);
>> +
>> + tmp = orig & ~mask;
>> + tmp |= val & mask;
>> +
>> + if (tmp != orig)
>> + __mali_c55_ctx_write(ctx, addr, tmp);
> Shouldn't you take the spinlock around the whole read/write ?
Yes that would probably make much more sense wouldn't it
>> +}
>> +
>> +static void mali_c55_dma_xfer_complete(void *param,
>> + const struct dmaengine_result *result)
>> +{
>> + struct mali_c55 *mali_c55 = param;
>> +
>> + if (result->result != DMA_TRANS_NOERROR)
>> + dev_err(mali_c55->dev, "Failed to DMA xfer ISP config\n");
>
>
>> +}
>> +
>> +static int mali_c55_dma_xfer(struct mali_c55_context *ctx, dma_addr_t src,
>> + dma_addr_t dst)
>> +{
>> + struct mali_c55 *mali_c55 = ctx->mali_c55;
>> + struct dma_async_tx_descriptor *tx;
>> + dma_cookie_t cookie;
>> +
>> + tx = dmaengine_prep_dma_memcpy(mali_c55->channel, dst, src,
>> + MALI_C55_CONFIG_SPACE_SIZE, 0);
>> + if (!tx) {
>> + dev_err(mali_c55->dev, "failed to prep DMA memcpy\n");
>> + return -EIO;
>> + }
>> +
>> + tx->callback_result = mali_c55_dma_xfer_complete;
>> + tx->callback_param = mali_c55;
>> +
>> + cookie = dmaengine_submit(tx);
>> + if (dma_submit_error(cookie)) {
>> + dev_err(mali_c55->dev, "error submitting dma transfer\n");
>> + return -EIO;
>> + }
>> +
>> + dma_async_issue_pending(mali_c55->channel);
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_dma_write(struct mali_c55_context *ctx,
>> + enum mali_c55_config_spaces cfg_space)
>> +{
>> + struct mali_c55 *mali_c55 = ctx->mali_c55;
>> + struct device *dma_dev = mali_c55->channel->device->dev;
>> + dma_addr_t dst = ctx->base + config_space_addrs[cfg_space];
>> + dma_addr_t src;
>> + int ret;
>> +
>> + guard(spinlock)(&ctx->lock);
>> +
>> + src = dma_map_single(dma_dev, ctx->registers,
>> + MALI_C55_CONFIG_SPACE_SIZE, DMA_TO_DEVICE);
>> + if (dma_mapping_error(dma_dev, src)) {
>> + dev_err(mali_c55->dev, "failed to map DMA addr\n");
>> + return -EIO;
>> + }
>> +
>> + ret = mali_c55_dma_xfer(ctx, src, dst);
>> + dma_unmap_single(dma_dev, src, MALI_C55_CONFIG_SPACE_SIZE,
>> + DMA_TO_DEVICE);
>> +
>> + return ret;
>> +}
>> +
>> +int mali_c55_config_write(struct mali_c55_context *ctx,
>> + enum mali_c55_config_spaces cfg_space)
>> +{
>> + struct mali_c55 *mali_c55 = ctx->mali_c55;
>> +
>> + if (mali_c55->channel)
>> + return mali_c55_dma_write(ctx, cfg_space);
>> +
>> + memcpy_toio(mali_c55->base + config_space_addrs[cfg_space],
>> + ctx->registers, MALI_C55_CONFIG_SPACE_SIZE);
> So, when using DMA, this function issues the memcpy and returns without
> waiting for it to complete, while otherwise the copy is synchronous.
> That's not a problem as such, but in the asynchronous case, the caller
> needs to handle the fact that the copy will complete later. Looking at
> mali_c55_isp_start() for instance, it seems you actually start the ISP
> before the copy completes, which doesn't seem safe. Similarly, in
> mali_c55_swap_next_config(), I think the ISP is instructed to start
> using the other configuration space before the copy completes, which
> seems equally dangerous. Does all this need some more careful
> consideration ?
Hm, you're right...perhaps the function needs a flag to set it to await completion with
dma_sync_wait() for the mali_c55_isp_start() path and then have the setting for MCU_CONFIG in the
completion handler of the config write instead for the interrupt.
I wonder if it's also worthwhile having a warning in case the configuration misses the boat, so to
speak.
>
>> +
>> + return 0;
>> +}
>> +
>> +struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55)
>> +{
>> + return &mali_c55->context;
>> +}
>> +
>> +static int mali_c55_parse_endpoint(struct mali_c55 *mali_c55)
>> +{
>> + struct v4l2_async_connection *asc;
>> + struct fwnode_handle *ep;
>> +
>> + /*
>> + * The ISP should have a single endpoint pointing to some flavour of
>> + * CSI-2 receiver...but for now at least we do want everything to work
>> + * normally even with no sensors connected, as we have the TPG. If we
>> + * don't find a sensor just warn and return success.
>> + */
>> + ep = fwnode_graph_get_endpoint_by_id(dev_fwnode(mali_c55->dev),
>> + 0, 0, 0);
>> + if (!ep) {
>> + dev_warn(mali_c55->dev, "no local endpoint found\n");
>> + return 0;
>> + }
>> +
>> + asc = v4l2_async_nf_add_fwnode_remote(&mali_c55->notifier, ep,
>> + struct v4l2_async_connection);
>> + fwnode_handle_put(ep);
>> + if (IS_ERR(asc)) {
>> + dev_err(mali_c55->dev, "failed to add remote fwnode\n");
>> + return PTR_ERR(asc);
>> + }
>> +
>> + return 0;
>> +}
> I would move this right after mali_c55_notifier_ops, as it's more
> related to the notifier and the following function than to the next two
> functions right below. Up to you.
>
>> +
>> +static void mali_c55_remove_links(struct mali_c55 *mali_c55)
>> +{
>> + unsigned int i;
>> +
>> + media_entity_remove_links(&mali_c55->tpg.sd.entity);
>> + media_entity_remove_links(&mali_c55->isp.sd.entity);
>> +
>> + for (i = 0; i < MALI_C55_NUM_RSZS; i++)
>> + media_entity_remove_links(&mali_c55->resizers[i].sd.entity);
>> +
>> + for (i = 0; i < MALI_C55_NUM_CAP_DEVS; i++)
>> + media_entity_remove_links(&mali_c55->cap_devs[i].vdev.entity);
>> +}
>> +
>> +static int mali_c55_create_links(struct mali_c55 *mali_c55)
>> +{
>> + int ret;
>> +
>> + /* Test pattern generator to ISP */
>> + ret = media_create_pad_link(&mali_c55->tpg.sd.entity, 0,
>> + &mali_c55->isp.sd.entity,
>> + MALI_C55_ISP_PAD_SINK_VIDEO, 0);
>> + if (ret) {
>> + dev_err(mali_c55->dev, "failed to link TPG and ISP\n");
>> + goto err_remove_links;
>> + }
>> +
>> + /* Full resolution resizer pipe. */
>> + ret = media_create_pad_link(&mali_c55->isp.sd.entity,
>> + MALI_C55_ISP_PAD_SOURCE_VIDEO,
>> + &mali_c55->resizers[MALI_C55_RSZ_FR].sd.entity,
>> + MALI_C55_RSZ_SINK_PAD,
>> + MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE);
>> + if (ret) {
>> + dev_err(mali_c55->dev, "failed to link ISP and FR resizer\n");
>> + goto err_remove_links;
>> + }
>> +
>> + /* Full resolution bypass. */
>> + ret = media_create_pad_link(&mali_c55->isp.sd.entity,
>> + MALI_C55_ISP_PAD_SOURCE_BYPASS,
>> + &mali_c55->resizers[MALI_C55_RSZ_FR].sd.entity,
>> + MALI_C55_RSZ_SINK_BYPASS_PAD,
>> + MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE);
>> + if (ret) {
>> + dev_err(mali_c55->dev, "failed to link ISP and FR resizer\n");
>> + goto err_remove_links;
>> + }
>> +
>> + /* Resizer pipe to video capture nodes. */
>> + ret = media_create_pad_link(&mali_c55->resizers[0].sd.entity,
>> + MALI_C55_RSZ_SOURCE_PAD,
>> + &mali_c55->cap_devs[MALI_C55_CAP_DEV_FR].vdev.entity,
>> + 0, MEDIA_LNK_FL_ENABLED);
>> + if (ret) {
>> + dev_err(mali_c55->dev,
>> + "failed to link FR resizer and video device\n");
>> + goto err_remove_links;
>> + }
>> +
>> + /* The downscale pipe is an optional hardware block */
>> + if (mali_c55->capabilities & MALI_C55_GPS_DS_PIPE_FITTED) {
>> + ret = media_create_pad_link(&mali_c55->isp.sd.entity,
>> + MALI_C55_ISP_PAD_SOURCE_VIDEO,
>> + &mali_c55->resizers[MALI_C55_RSZ_DS].sd.entity,
>> + MALI_C55_RSZ_SINK_PAD,
>> + MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE);
>> + if (ret) {
>> + dev_err(mali_c55->dev,
>> + "failed to link ISP and DS resizer\n");
>> + goto err_remove_links;
>> + }
>> +
>> + ret = media_create_pad_link(&mali_c55->resizers[1].sd.entity,
>> + MALI_C55_RSZ_SOURCE_PAD,
>> + &mali_c55->cap_devs[MALI_C55_CAP_DEV_DS].vdev.entity,
>> + 0, MEDIA_LNK_FL_ENABLED);
>> + if (ret) {
>> + dev_err(mali_c55->dev,
>> + "failed to link DS resizer and video device\n");
>> + goto err_remove_links;
>> + }
>> + }
>> +
>> + return 0;
>> +
>> +err_remove_links:
>> + mali_c55_remove_links(mali_c55);
>> + return ret;
>> +}
>> +
>> +static void mali_c55_unregister_entities(struct mali_c55 *mali_c55)
>> +{
>> + mali_c55_remove_links(mali_c55);
>> + mali_c55_unregister_tpg(mali_c55);
>> + mali_c55_unregister_isp(mali_c55);
>> + mali_c55_unregister_resizers(mali_c55);
>> + mali_c55_unregister_capture_devs(mali_c55);
>> +}
>> +
>> +static int mali_c55_register_entities(struct mali_c55 *mali_c55)
>> +{
>> + int ret;
>> +
>> + ret = mali_c55_register_tpg(mali_c55);
>> + if (ret)
>> + return ret;
>> +
>> + ret = mali_c55_register_isp(mali_c55);
>> + if (ret)
>> + goto err_unregister_entities;
>> +
>> + ret = mali_c55_register_resizers(mali_c55);
>> + if (ret)
>> + goto err_unregister_entities;
>> +
>> + ret = mali_c55_register_capture_devs(mali_c55);
>> + if (ret)
>> + goto err_unregister_entities;
>> +
>> + ret = mali_c55_create_links(mali_c55);
>> + if (ret)
>> + goto err_unregister_entities;
>> +
>> + return 0;
>> +
>> +err_unregister_entities:
>> + mali_c55_unregister_entities(mali_c55);
>> +
>> + return ret;
>> +}
>> +
>> +static int mali_c55_notifier_bound(struct v4l2_async_notifier *notifier,
>> + struct v4l2_subdev *subdev,
>> + struct v4l2_async_connection *asc)
>> +{
>> + struct mali_c55 *mali_c55 = container_of(notifier,
>> + struct mali_c55, notifier);
>> + struct media_pad *pad = &mali_c55->isp.pads[MALI_C55_ISP_PAD_SINK_VIDEO];
>> + int ret;
>> +
>> + /*
>> + * By default we'll flag this link enabled and the TPG disabled, but
>> + * no immutable flag because we need to be able to switch between the
>> + * two.
>> + */
>> + ret = v4l2_create_fwnode_links_to_pad(subdev, pad,
>> + MEDIA_LNK_FL_ENABLED);
>> + if (ret)
>> + dev_err(mali_c55->dev, "failed to create link for %s\n",
>> + subdev->name);
>> +
>> + return ret;
>> +}
>> +
>> +static int mali_c55_notifier_complete(struct v4l2_async_notifier *notifier)
>> +{
>> + struct mali_c55 *mali_c55 = container_of(notifier,
>> + struct mali_c55, notifier);
>> +
>> + return v4l2_device_register_subdev_nodes(&mali_c55->v4l2_dev);
>> +}
>> +
>> +static const struct v4l2_async_notifier_operations mali_c55_notifier_ops = {
>> + .bound = mali_c55_notifier_bound,
>> + .complete = mali_c55_notifier_complete,
>> +};
>> +
>> +static int mali_c55_media_frameworks_init(struct mali_c55 *mali_c55)
>> +{
>> + int ret;
>> +
>> + media_device_init(&mali_c55->media_dev);
>> + ret = media_device_register(&mali_c55->media_dev);
>> + if (ret)
>> + goto err_cleanup_media_device;
>> +
>> + mali_c55->v4l2_dev.mdev = &mali_c55->media_dev;
>> + ret = v4l2_device_register(mali_c55->dev, &mali_c55->v4l2_dev);
>> + if (ret) {
>> + dev_err(mali_c55->dev, "failed to register V4L2 device\n");
>> + goto err_unregister_media_device;
>> + };
>> +
>> + mali_c55->notifier.ops = &mali_c55_notifier_ops;
>> + v4l2_async_nf_init(&mali_c55->notifier, &mali_c55->v4l2_dev);
>> +
>> + ret = mali_c55_register_entities(mali_c55);
>> + if (ret) {
>> + dev_err(mali_c55->dev, "failed to register entities\n");
>> + goto err_cleanup_nf;
>> + }
>> +
>> + ret = mali_c55_parse_endpoint(mali_c55);
>> + if (ret)
>> + goto err_cleanup_nf;
>> +
>> + ret = v4l2_async_nf_register(&mali_c55->notifier);
>> + if (ret) {
>> + dev_err(mali_c55->dev, "failed to register notifier\n");
>> + goto err_unregister_entities;
>> + }
>> +
>> + return 0;
>> +
>> +err_unregister_entities:
>> + mali_c55_unregister_entities(mali_c55);
>> +err_cleanup_nf:
>> + v4l2_async_nf_cleanup(&mali_c55->notifier);
>> + v4l2_device_unregister(&mali_c55->v4l2_dev);
>> +err_unregister_media_device:
>> + media_device_unregister(&mali_c55->media_dev);
>> +err_cleanup_media_device:
>> + media_device_cleanup(&mali_c55->media_dev);
>> +
>> + return ret;
>> +}
>> +
>> +static void mali_c55_media_frameworks_deinit(struct mali_c55 *mali_c55)
>> +{
>> + v4l2_async_nf_unregister(&mali_c55->notifier);
>> + mali_c55_unregister_entities(mali_c55);
>> + v4l2_async_nf_cleanup(&mali_c55->notifier);
>> + v4l2_device_unregister(&mali_c55->v4l2_dev);
>> + media_device_unregister(&mali_c55->media_dev);
>> + media_device_cleanup(&mali_c55->media_dev);
>> +}
>> +
>> +static int mali_c55_check_hwcfg(struct mali_c55 *mali_c55)
>> +{
>> + u32 product, version, revision, capabilities;
>> +
>> + product = mali_c55_read(mali_c55, MALI_C55_REG_PRODUCT);
>> + version = mali_c55_read(mali_c55, MALI_C55_REG_VERSION);
>> + revision = mali_c55_read(mali_c55, MALI_C55_REG_REVISION);
>> +
>> + mali_c55->media_dev.hw_revision = version;
>> +
>> + dev_info(mali_c55->dev, "Detected Mali-C55 ISP %u.%u.%u\n",
>> + product, version, revision);
>> +
>> + capabilities = mali_c55_read(mali_c55,
>> + MALI_C55_REG_GLOBAL_PARAMETER_STATUS);
>> +
>> + /*
>> + * In its current iteration, the driver only supports inline mode. Given
>> + * we cannot control input data timing in this mode, we cannot guarantee
>> + * that the vertical blanking periods between frames will be long enough
>> + * for us to write configuration data to the ISP during them. For that
>> + * reason we can't really support single config space configuration
>> + * until memory input mode is implemented.
>> + */
>> + if (!capabilities & MALI_C55_GPS_PONG_FITTED) {
>> + dev_err(mali_c55->dev, "Pong config space not fitted.\n");
>> + return -EINVAL;
>> + }
>> +
>> + mali_c55->capabilities = (capabilities & 0xffff);
> No need for parentheses.
>
>> +
>> + return 0;
>> +}
>> +
>> +static void mali_c55_swap_next_config(struct mali_c55 *mali_c55)
>> +{
>> + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
>> + u32 curr_config, next_config;
>> +
>> + curr_config = mali_c55_read(mali_c55, MALI_C55_REG_PING_PONG_READ);
>> + curr_config = (curr_config & MALI_C55_REG_PING_PONG_READ_MASK)
>> + >> (ffs(MALI_C55_REG_PING_PONG_READ_MASK) - 1);
>> + next_config = curr_config ^ 1;
>> +
>> + mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG,
>> + MALI_C55_REG_MCU_CONFIG_WRITE_MASK,
>> + MALI_C55_MCU_CONFIG_WRITE(next_config));
> How about
>
> curr_config = mali_c55_read(mali_c55, MALI_C55_REG_PING_PONG_READ)
> & MALI_C55_REG_PING_PONG_READ_MASK;
> next_config = curr_config ^ MALI_C55_REG_PING_PONG_READ_MASK;
>
> mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG,
> MALI_C55_REG_MCU_CONFIG_WRITE_MASK,
> next_config);
I don't think that quite works; MALI_C55_REG_PING_PONG_READ_MASK is BIT(2) but
MALI_C55_REG_MCU_CONFIG_WRITE_MASK is BIT(1)
>
>> + mali_c55_config_write(ctx, next_config ?
>> + MALI_C55_CONFIG_PING : MALI_C55_CONFIG_PONG);
>> +}
>> +
>> +static irqreturn_t mali_c55_isr(int irq, void *context)
>> +{
>> + struct device *dev = context;
>> + struct mali_c55 *mali_c55 = dev_get_drvdata(dev);
>> + u32 interrupt_status;
>> + unsigned int i, j;
>> +
>> + interrupt_status = mali_c55_read(mali_c55,
>> + MALI_C55_REG_INTERRUPT_STATUS_VECTOR);
>> + if (!interrupt_status)
>> + return IRQ_NONE;
>> +
>> + mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR_VECTOR,
>> + interrupt_status);
>> + mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR, 0);
>> + mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR, 1);
>> +
>> + for (i = 0; i < MALI_C55_NUM_IRQ_BITS; i++) {
>> + if (!(interrupt_status & (1 << i)))
>> + continue;
>> +
>> + switch (i) {
>> + case MALI_C55_IRQ_ISP_START:
>> + mali_c55_isp_queue_event_sof(mali_c55);
>> +
>> + for (j = i; j < MALI_C55_NUM_CAP_DEVS; j++)
>> + mali_c55_set_next_buffer(&mali_c55->cap_devs[j]);
>> +
>> + mali_c55_swap_next_config(mali_c55);
>> +
>> + break;
>> + case MALI_C55_IRQ_ISP_DONE:
>> + /*
>> + * TODO: Where the ISP has no Pong config fitted, we'd
>> + * have to do the mali_c55_swap_next_config() call here.
>> + */
>> + break;
>> + case MALI_C55_IRQ_FR_Y_DONE:
>> + mali_c55_set_plane_done(
>> + &mali_c55->cap_devs[MALI_C55_CAP_DEV_FR],
>> + MALI_C55_PLANE_Y);
>> + break;
>> + case MALI_C55_IRQ_FR_UV_DONE:
>> + mali_c55_set_plane_done(
>> + &mali_c55->cap_devs[MALI_C55_CAP_DEV_FR],
>> + MALI_C55_PLANE_UV);
>> + break;
>> + case MALI_C55_IRQ_DS_Y_DONE:
>> + mali_c55_set_plane_done(
>> + &mali_c55->cap_devs[MALI_C55_CAP_DEV_DS],
>> + MALI_C55_PLANE_Y);
>> + break;
>> + case MALI_C55_IRQ_DS_UV_DONE:
>> + mali_c55_set_plane_done(
>> + &mali_c55->cap_devs[MALI_C55_CAP_DEV_DS],
>> + MALI_C55_PLANE_UV);
>> + break;
>> + default:
>> + /*
>> + * Only the above interrupts are currently unmasked. If
>> + * we receive anything else here then something weird
>> + * has gone on.
>> + */
>> + dev_err(dev, "masked interrupt %s triggered\n",
>> + mali_c55_interrupt_names[i]);
>> + }
>> + }
>> +
>> + return IRQ_HANDLED;
>> +}
>> +
>> +static int mali_c55_init_context(struct mali_c55 *mali_c55,
>> + struct resource *res)
>> +{
>> + struct mali_c55_context *ctx = &mali_c55->context;
>> +
>> + ctx->base = res->start;
>> + ctx->mali_c55 = mali_c55;
>> +
>> + ctx->registers = kzalloc(MALI_C55_CONFIG_SPACE_SIZE,
>> + GFP_KERNEL | GFP_DMA);
>> + if (!ctx->registers)
>> + return -ENOMEM;
>> +
>> + /*
>> + * The allocated memory is empty, we need to load the default
>> + * register settings. We just read Ping; it's identical to Pong.
>> + */
>> + memcpy_fromio(ctx->registers,
>> + mali_c55->base + config_space_addrs[MALI_C55_CONFIG_PING],
>> + MALI_C55_CONFIG_SPACE_SIZE);
>> +
>> + /*
>> + * Some features of the ISP need to be disabled by default and only
>> + * enabled at the same time as they're configured by a parameters buffer
>> + */
>> +
>> + /* Bypass the sqrt and square compression and expansion modules */
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BYPASS_1,
>> + MALI_C55_REG_BYPASS_1_FE_SQRT,
>> + MALI_C55_REG_BYPASS_1_FE_SQRT);
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BYPASS_3,
>> + MALI_C55_REG_BYPASS_3_SQUARE_BE,
>> + MALI_C55_REG_BYPASS_3_SQUARE_BE);
>> +
>> + /* Bypass the temper module */
>> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_BYPASS_2,
>> + MALI_C55_REG_BYPASS_2_TEMPER);
>> +
>> + /* Bypass the colour noise reduction */
>> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_BYPASS_4,
>> + MALI_C55_REG_BYPASS_4_CNR);
>> +
>> + /* Disable the sinter module */
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_SINTER_CONFIG,
>> + MALI_C55_SINTER_ENABLE_MASK, 0x00);
>> +
>> + /* Disable the RGB Gamma module for each output */
>> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_FR_GAMMA_RGB_ENABLE, 0x00);
> As registers are 32-bit wide this would need to be 0x00000000, but I
> think you can just write 0.
>
>> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_DS_GAMMA_RGB_ENABLE, 0x00);
>> +
>> + /* Disable the colour correction matrix */
>> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_CCM_ENABLE, 0x00);
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_runtime_resume(struct device *dev)
>> +{
>> + struct mali_c55 *mali_c55 = dev_get_drvdata(dev);
>> + int ret;
>> +
>> + ret = clk_bulk_prepare_enable(ARRAY_SIZE(mali_c55->clks),
>> + mali_c55->clks);
>> + if (ret) {
>> + dev_err(mali_c55->dev, "failed to enable clocks\n");
>> + return ret;
>> + }
>> +
>> + /*
>> + * Mask the interrupts and clear any that were set, then unmask the ones
>> + * that we actually want to handle.
>> + */
>> + mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_MASK_VECTOR,
>> + MALI_C55_INTERRUPT_MASK_ALL);
>> + mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR_VECTOR,
>> + MALI_C55_INTERRUPT_MASK_ALL);
>> + mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR, 0x01);
>> + mali_c55_write(mali_c55, MALI_C55_REG_INTERRUPT_CLEAR, 0x00);
>> +
>> + mali_c55_update_bits(mali_c55, MALI_C55_REG_INTERRUPT_MASK_VECTOR,
>> + MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_ISP_START) |
>> + MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_ISP_DONE) |
>> + MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_FR_Y_DONE) |
>> + MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_FR_UV_DONE) |
>> + MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_DS_Y_DONE) |
>> + MALI_C55_INTERRUPT_BIT(MALI_C55_IRQ_DS_UV_DONE),
>> + 0x00);
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_runtime_suspend(struct device *dev)
>> +{
>> + struct mali_c55 *mali_c55 = dev_get_drvdata(dev);
>> +
>> + clk_bulk_disable_unprepare(ARRAY_SIZE(mali_c55->clks), mali_c55->clks);
>> + return 0;
>> +}
>> +
>> +static const struct dev_pm_ops mali_c55_pm_ops = {
>> + SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
>> + pm_runtime_force_resume)
>> + SET_RUNTIME_PM_OPS(mali_c55_runtime_suspend, mali_c55_runtime_resume,
>> + NULL)
>> +};
>> +
>> +static int mali_c55_probe(struct platform_device *pdev)
>> +{
>> + struct device *dev = &pdev->dev;
>> + struct mali_c55 *mali_c55;
>> + struct resource *res;
>> + dma_cap_mask_t mask;
>> + int ret;
>> + u32 val;
>> +
>> + mali_c55 = devm_kzalloc(dev, sizeof(*mali_c55), GFP_KERNEL);
>> + if (!mali_c55)
>> + return -ENOMEM;
>> +
>> + mali_c55->dev = dev;
>> + platform_set_drvdata(pdev, mali_c55);
>> +
>> + mali_c55->base = devm_platform_get_and_ioremap_resource(pdev, 0,
>> + &res);
>> + if (IS_ERR(mali_c55->base))
>> + return dev_err_probe(dev, PTR_ERR(mali_c55->base),
>> + "failed to map IO memory\n");
>> +
>> + for (unsigned int i = 0; i < ARRAY_SIZE(mali_c55_clk_names); i++)
>> + mali_c55->clks[i].id = mali_c55_clk_names[i];
>> +
>> + ret = devm_clk_bulk_get(dev, ARRAY_SIZE(mali_c55->clks), mali_c55->clks);
>> + if (ret)
>> + return dev_err_probe(dev, ret, "failed to acquire clocks\n");
>> +
>> + of_reserved_mem_device_init(dev);
>> + vb2_dma_contig_set_max_seg_size(dev, UINT_MAX);
>> +
>> + dma_cap_zero(mask);
>> + dma_cap_set(DMA_MEMCPY, mask);
>> +
>> + pm_runtime_set_autosuspend_delay(&pdev->dev, 2000);
>> + pm_runtime_use_autosuspend(&pdev->dev);
>> + pm_runtime_enable(&pdev->dev);
>> +
>> + ret = pm_runtime_resume_and_get(&pdev->dev);
>> + if (ret)
>> + goto err_pm_runtime_disable;
>> +
>> + ret = mali_c55_check_hwcfg(mali_c55);
>> + if (ret)
>> + goto err_pm_runtime_put;
>> +
>> + /* Use "software only" context management. */
>> + mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG,
>> + MALI_C55_REG_MCU_CONFIG_OVERRIDE_MASK, 0x01);
> Move this to the runtime PM resume handler, or to ISP start time, as
> otherwise it may be lost when power is cut.
>
>> +
>> + /*
>> + * No error check, because we will just fallback on memcpy if there is
>> + * no usable DMA channel on the system.
>> + */
>> + mali_c55->channel = dma_request_channel(mask, NULL, NULL);
>> +
>> + ret = mali_c55_init_context(mali_c55, res);
>> + if (ret)
>> + goto err_release_dma_channel;
>> +
>> + mali_c55->media_dev.dev = dev;
>> + strscpy(mali_c55->media_dev.model, "ARM Mali-C55 ISP",
>> + sizeof(mali_c55->media_dev.model));
> This can be moved to mali_c55_media_frameworks_init().
>
>> +
>> + ret = mali_c55_media_frameworks_init(mali_c55);
>> + if (ret)
>> + goto err_free_context_registers;
>> +
>> + /* Set safe stop to ensure we're in a non-streaming state */
>> + mali_c55_write(mali_c55, MALI_C55_REG_INPUT_MODE_REQUEST,
>> + MALI_C55_INPUT_SAFE_STOP);
>> + readl_poll_timeout(mali_c55->base + MALI_C55_REG_MODE_STATUS,
>> + val, !val, 10 * USEC_PER_MSEC, 250 * USEC_PER_MSEC);
> Also a candidate for the runtime PM handler ?
>
>> +
>> + pm_runtime_put(&pdev->dev);
>> +
>> + ret = platform_get_irq(pdev, 0);
>> + if (ret < 0) {
>> + dev_err(dev, "failed to get interrupt\n");
>> + goto err_deinit_media_frameworks;
>> + }
>> +
>> + /*
>> + * The driver needs to transfer large amounts of register settings to
>> + * the ISP each frame, using either a DMA transfer or memcpy. We use a
>> + * threaded IRQ to avoid disabling interrupts the entire time that's
>> + * happening.
>> + */
>> + ret = devm_request_threaded_irq(&pdev->dev, ret, NULL,
>> + mali_c55_isr, IRQF_ONESHOT,
>> + dev_driver_string(&pdev->dev),
>> + &pdev->dev);
>> + if (ret) {
>> + dev_err(dev, "failed to request irq\n");
>> + goto err_deinit_media_frameworks;
>> + }
>> +
>> + return 0;
>> +
>> +err_deinit_media_frameworks:
>> + mali_c55_media_frameworks_deinit(mali_c55);
>> +err_free_context_registers:
>> + kfree(mali_c55->context.registers);
>> +err_release_dma_channel:
>> + dma_release_channel(mali_c55->channel);
>> +err_pm_runtime_put:
>> + pm_runtime_put(&pdev->dev);
>> +err_pm_runtime_disable:
>> + pm_runtime_disable(&pdev->dev);
>> +
>> + return ret;
>> +}
>> +
>> +static void mali_c55_remove(struct platform_device *pdev)
>> +{
>> + struct mali_c55 *mali_c55 = platform_get_drvdata(pdev);
>> +
>> + kfree(mali_c55->context.registers);
>> + mali_c55_media_frameworks_deinit(mali_c55);
>> + dma_release_channel(mali_c55->channel);
>> +}
>> +
>> +static const struct of_device_id mali_c55_of_match[] = {
>> + { .compatible = "arm,mali-c55", },
>> + { /* Sentinel */},
> s/}/ }/
>
>> +};
>> +MODULE_DEVICE_TABLE(of, mali_c55_of_match);
>> +
>> +static struct platform_driver mali_c55_driver = {
>> + .driver = {
>> + .name = "mali-c55",
>> + .of_match_table = mali_c55_of_match,
>> + .pm = &mali_c55_pm_ops,
>> + },
>> + .probe = mali_c55_probe,
>> + .remove_new = mali_c55_remove,
> You can switch back to .remove().
>
>> +};
>> +
>> +module_platform_driver(mali_c55_driver);
>> +
>> +MODULE_AUTHOR("Daniel Scally <dan.scally@ideasonboard.com>");
>> +MODULE_AUTHOR("Jacopo Mondi <jacopo.mondi@ideasonboard.com>");
>> +MODULE_DESCRIPTION("ARM Mali-C55 ISP platform driver");
>> +MODULE_LICENSE("GPL");
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>> new file mode 100644
>> index 000000000000..f784983a4ccc
>> --- /dev/null
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>> @@ -0,0 +1,531 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * ARM Mali-C55 ISP Driver - Image signal processor
>> + *
>> + * Copyright (C) 2024 Ideas on Board Oy
>> + */
>> +
>> +#include <linux/delay.h>
>> +#include <linux/iopoll.h>
>> +#include <linux/property.h>
>> +#include <linux/string.h>
>> +
>> +#include <media/media-entity.h>
>> +#include <media/v4l2-common.h>
>> +#include <media/v4l2-event.h>
>> +#include <media/v4l2-mc.h>
>> +#include <media/v4l2-subdev.h>
>> +
>> +#include "mali-c55-common.h"
>> +#include "mali-c55-registers.h"
>> +
>> +static const struct mali_c55_isp_fmt mali_c55_isp_fmts[] = {
>> + {
>> + .code = MEDIA_BUS_FMT_SRGGB20_1X20,
>> + .order = MALI_C55_BAYER_ORDER_RGGB,
>> + .bypass = false,
>> + },
>> + {
>> + .code = MEDIA_BUS_FMT_SGRBG20_1X20,
>> + .order = MALI_C55_BAYER_ORDER_GRBG,
>> + .bypass = false,
>> + },
>> + {
>> + .code = MEDIA_BUS_FMT_SGBRG20_1X20,
>> + .order = MALI_C55_BAYER_ORDER_GBRG,
>> + .bypass = false,
>> + },
>> + {
>> + .code = MEDIA_BUS_FMT_SBGGR20_1X20,
>> + .order = MALI_C55_BAYER_ORDER_BGGR,
>> + .bypass = false,
>> + },
>> + {
>> + .code = MEDIA_BUS_FMT_RGB202020_1X60,
>> + .order = 0, /* Not relevant for this format */
>> + .bypass = true,
>> + }
>> + /*
>> + * TODO: Support MEDIA_BUS_FMT_YUV20_1X60 here. This is so that we can
>> + * also support YUV input from a sensor passed-through to the output. At
>> + * present we have no mechanism to test that though so it may have to
>> + * wait a while...
>> + */
>> +};
>> +
>> +const struct mali_c55_isp_fmt *
>> +mali_c55_isp_get_mbus_config_by_index(u32 index)
>> +{
>> + if (index < ARRAY_SIZE(mali_c55_isp_fmts))
>> + return &mali_c55_isp_fmts[index];
>> +
>> + return NULL;
>> +}
>> +
>> +const struct mali_c55_isp_fmt *
>> +mali_c55_isp_get_mbus_config_by_code(u32 code)
>> +{
>> + unsigned int i;
>> +
>> + for (i = 0; i < ARRAY_SIZE(mali_c55_isp_fmts); i++) {
>> + if (mali_c55_isp_fmts[i].code == code)
>> + return &mali_c55_isp_fmts[i];
>> + }
>> +
>> + return NULL;
>> +}
>> +
>> +static void mali_c55_isp_stop(struct mali_c55 *mali_c55)
>> +{
>> + u32 val;
>> +
>> + mali_c55_write(mali_c55, MALI_C55_REG_INPUT_MODE_REQUEST,
>> + MALI_C55_INPUT_SAFE_STOP);
>> + readl_poll_timeout(mali_c55->base + MALI_C55_REG_MODE_STATUS,
>> + val, !val, 10 * USEC_PER_MSEC, 250 * USEC_PER_MSEC);
>> +}
>> +
>> +static int mali_c55_isp_start(struct mali_c55 *mali_c55)
>> +{
>> + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
>> + struct v4l2_subdev *sd = &mali_c55->isp.sd;
>> + const struct v4l2_mbus_framefmt *format;
>> + const struct mali_c55_isp_fmt *cfg;
>> + struct v4l2_subdev_state *state;
> const, thanks to commit 85af84852f11 ("media: v4l2-subdev: Provide
> const-aware subdev state accessors") that has landed in v6.11-rc1 :-)
>
>> + const struct v4l2_rect *crop;
>> + u32 val;
>> + int ret;
>> +
>> + mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG,
>> + MALI_C55_REG_MCU_CONFIG_WRITE_MASK,
>> + MALI_C55_REG_MCU_CONFIG_WRITE_PING);
>> +
>> + /* Apply input windowing */
>> + state = v4l2_subdev_get_locked_active_state(sd);
> You have the state in the caller, you can pass it to this function.
>
>> + crop = v4l2_subdev_state_get_crop(state, MALI_C55_ISP_PAD_SINK_VIDEO);
>> + format = v4l2_subdev_state_get_format(state,
>> + MALI_C55_ISP_PAD_SINK_VIDEO);
>> + cfg = mali_c55_isp_get_mbus_config_by_code(format->code);
>> +
>> + mali_c55_write(mali_c55, MALI_C55_REG_HC_START,
>> + MALI_C55_HC_START(crop->left));
>> + mali_c55_write(mali_c55, MALI_C55_REG_HC_SIZE,
>> + MALI_C55_HC_SIZE(crop->width));
>> + mali_c55_write(mali_c55, MALI_C55_REG_VC_START_SIZE,
>> + MALI_C55_VC_START(crop->top) |
>> + MALI_C55_VC_SIZE(crop->height));
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BASE_ADDR,
>> + MALI_C55_REG_ACTIVE_WIDTH_MASK, format->width);
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BASE_ADDR,
>> + MALI_C55_REG_ACTIVE_HEIGHT_MASK,
>> + format->height << 16);
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_BAYER_ORDER,
>> + MALI_C55_BAYER_ORDER_MASK, cfg->order);
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_INPUT_WIDTH,
>> + MALI_C55_INPUT_WIDTH_MASK,
>> + MALI_C55_INPUT_WIDTH_20BIT);
>> +
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_ISP_RAW_BYPASS,
>> + MALI_C55_ISP_RAW_BYPASS_BYPASS_MASK,
>> + cfg->bypass ? MALI_C55_ISP_RAW_BYPASS_BYPASS_MASK :
>> + 0x00);
>> +
>> + ret = mali_c55_config_write(ctx, MALI_C55_CONFIG_PING);
>> + if (ret) {
>> + dev_err(mali_c55->dev, "failed to write ISP config\n");
>> + return ret;
>> + }
>> +
>> + mali_c55_write(mali_c55, MALI_C55_REG_INPUT_MODE_REQUEST,
>> + MALI_C55_INPUT_SAFE_START);
>> + readl_poll_timeout(mali_c55->base + MALI_C55_REG_MODE_STATUS, val, !val,
>> + 10 * USEC_PER_MSEC, 250 * USEC_PER_MSEC);
> Should you return an error in case of timeout ?
>
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_isp_enum_mbus_code(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_mbus_code_enum *code)
>> +{
>> + /*
>> + * Only the internal RGB processed format is allowed on the regular
>> + * processing source pad.
>> + */
>> + if (code->pad == MALI_C55_ISP_PAD_SOURCE_VIDEO) {
>> + if (code->index)
>> + return -EINVAL;
>> +
>> + code->code = MEDIA_BUS_FMT_RGB121212_1X36;
>> + return 0;
>> + }
>> +
>> + /* On the sink and bypass pads all the supported formats are allowed. */
>> + if (code->index >= ARRAY_SIZE(mali_c55_isp_fmts))
>> + return -EINVAL;
>> +
>> + code->code = mali_c55_isp_fmts[code->index].code;
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_isp_enum_frame_size(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_frame_size_enum *fse)
>> +{
>> + const struct mali_c55_isp_fmt *cfg;
>> +
>> + if (fse->index > 0)
>> + return -EINVAL;
>> +
>> + /*
>> + * Only the internal RGB processed format is allowed on the regular
>> + * processing source pad.
>> + *
>> + * On the sink and bypass pads all the supported formats are allowed.
>> + */
>> + if (fse->pad == MALI_C55_ISP_PAD_SOURCE_VIDEO) {
>> + if (fse->code != MEDIA_BUS_FMT_RGB121212_1X36)
>> + return -EINVAL;
>> + } else {
>> + cfg = mali_c55_isp_get_mbus_config_by_code(fse->code);
>> + if (!cfg)
>> + return -EINVAL;
>> + }
>> +
>> + fse->min_width = MALI_C55_MIN_WIDTH;
>> + fse->min_height = MALI_C55_MIN_HEIGHT;
>> + fse->max_width = MALI_C55_MAX_WIDTH;
>> + fse->max_height = MALI_C55_MAX_HEIGHT;
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_isp_set_fmt(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_format *format)
>> +{
>> + struct v4l2_mbus_framefmt *fmt = &format->format;
>> + struct v4l2_mbus_framefmt *src_fmt, *sink_fmt;
>> + const struct mali_c55_isp_fmt *cfg;
>> + struct v4l2_rect *crop;
>> +
>> + /*
>> + * Disallow set_fmt on the source pads; format is fixed and the sizes
>> + * are the result of applying the sink crop rectangle to the sink
>> + * format.
>> + */
>> + if (format->pad != MALI_C55_ISP_PAD_SINK_VIDEO)
>> + return v4l2_subdev_get_fmt(sd, state, format);
>> +
>> + sink_fmt = v4l2_subdev_state_get_format(state,
>> + MALI_C55_ISP_PAD_SINK_VIDEO);
>> +
>> + cfg = mali_c55_isp_get_mbus_config_by_code(fmt->code);
>> + sink_fmt->code = cfg ? fmt->code : MEDIA_BUS_FMT_SRGGB20_1X20;
>> +
>> + /*
>> + * Clamp sizes in the accepted limits and clamp the crop rectangle in
>> + * the new sizes.
>> + */
>> + sink_fmt->width = clamp(fmt->width, MALI_C55_MIN_WIDTH,
>> + MALI_C55_MAX_WIDTH);
>> + sink_fmt->height = clamp(fmt->height, MALI_C55_MIN_HEIGHT,
>> + MALI_C55_MAX_HEIGHT);
>> +
>> + *fmt = *sink_fmt;
>> +
>> + crop = v4l2_subdev_state_get_crop(state, MALI_C55_ISP_PAD_SINK_VIDEO);
>> + crop->left = 0;
>> + crop->top = 0;
>> + crop->width = sink_fmt->width;
>> + crop->height = sink_fmt->height;
>> +
>> + /*
>> + * Propagate format to source pads. On the 'regular' output pad use
>> + * the internal RGB processed format, while on the bypass pad simply
>> + * replicate the ISP sink format. The sizes on both pads are the same as
>> + * the ISP sink crop rectangle. The "field" and "colorspace" fields are
>> + * set in .init_state() and fixed for both source pads, as is the "code"
>> + * field for the processed data source pad.
>> + */
>> + src_fmt = v4l2_subdev_state_get_format(state,
>> + MALI_C55_ISP_PAD_SOURCE_VIDEO);
>> + src_fmt->width = crop->width;
>> + src_fmt->height = crop->height;
>> +
>> + src_fmt = v4l2_subdev_state_get_format(state,
>> + MALI_C55_ISP_PAD_SOURCE_BYPASS);
>> + src_fmt->code = sink_fmt->code;
>> + src_fmt->width = crop->width;
>> + src_fmt->height = crop->height;
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_isp_get_selection(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_selection *sel)
>> +{
>> + if (sel->pad != MALI_C55_ISP_PAD_SINK_VIDEO ||
>> + sel->target != V4L2_SEL_TGT_CROP)
>> + return -EINVAL;
>> +
>> + sel->r = *v4l2_subdev_state_get_crop(state, MALI_C55_ISP_PAD_SINK_VIDEO);
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_isp_set_selection(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_selection *sel)
>> +{
>> + struct v4l2_mbus_framefmt *src_fmt;
>> + const struct v4l2_mbus_framefmt *fmt;
>> + struct v4l2_rect *crop;
>> +
>> + if (sel->pad != MALI_C55_ISP_PAD_SINK_VIDEO ||
>> + sel->target != V4L2_SEL_TGT_CROP)
>> + return -EINVAL;
>> +
>> + fmt = v4l2_subdev_state_get_format(state, MALI_C55_ISP_PAD_SINK_VIDEO);
>> +
>> + sel->r.left = clamp_t(unsigned int, sel->r.left, 0, fmt->width);
>> + sel->r.top = clamp_t(unsigned int, sel->r.top, 0, fmt->height);
>> + sel->r.width = clamp_t(unsigned int, sel->r.width, MALI_C55_MIN_WIDTH,
>> + fmt->width - sel->r.left);
>> + sel->r.height = clamp_t(unsigned int, sel->r.height,
>> + MALI_C55_MIN_HEIGHT,
>> + fmt->height - sel->r.top);
>> +
>> + crop = v4l2_subdev_state_get_crop(state, MALI_C55_ISP_PAD_SINK_VIDEO);
>> + *crop = sel->r;
>> +
>> + /*
>> + * Propagate the crop rectangle sizes to the source pad format. The crop
>> + * isn't propagated to the bypass source pad, because the bypassed data
>> + * cannot be cropped.
>> + */
>> + src_fmt = v4l2_subdev_state_get_format(state,
>> + MALI_C55_ISP_PAD_SOURCE_VIDEO);
>> + src_fmt->width = crop->width;
>> + src_fmt->height = crop->height;
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_isp_enable_streams(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state, u32 pad,
>> + u64 streams_mask)
>> +{
>> + struct mali_c55_isp *isp = container_of(sd, struct mali_c55_isp, sd);
>> + struct mali_c55 *mali_c55 = isp->mali_c55;
>> + struct v4l2_subdev *src_sd;
>> + struct media_pad *sink_pad;
>> + int ret;
>> +
>> + /*
>> + * We have two source pads, both of which have only a single stream. The
>> + * core v4l2 code already validated those parameters so we can just get
>> + * on with starting the ISP.
>> + */
>> +
>> + sink_pad = &isp->pads[MALI_C55_ISP_PAD_SINK_VIDEO];
>> + isp->remote_src = media_pad_remote_pad_unique(sink_pad);
>> + if (IS_ERR(isp->remote_src)) {
> As you set the MUST_CONNECT flag for the MALI_C55_ISP_PAD_SINK_VIDEO pad
> I think you can drop this check.
>
>> + dev_err(mali_c55->dev, "Failed to get source for ISP\n");
>> + return PTR_ERR(isp->remote_src);
>> + }
>> +
>> + src_sd = media_entity_to_v4l2_subdev(isp->remote_src->entity);
>> +
>> + isp->frame_sequence = 0;
>> + ret = mali_c55_isp_start(mali_c55);
>> + if (ret) {
>> + dev_err(mali_c55->dev, "Failed to start ISP\n");
>> + isp->remote_src = NULL;
>> + return ret;
>> + }
>> +
>> + /*
>> + * We only support a single input stream, so we can just enable the 1st
>> + * entry in the streams mask.
>> + */
>> + ret = v4l2_subdev_enable_streams(src_sd, isp->remote_src->index, BIT(0));
>> + if (ret) {
>> + dev_err(mali_c55->dev, "Failed to start ISP source\n");
>> + mali_c55_isp_stop(mali_c55);
>> + return ret;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_isp_disable_streams(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state, u32 pad,
>> + u64 streams_mask)
>> +{
>> + struct mali_c55_isp *isp = container_of(sd, struct mali_c55_isp, sd);
>> + struct mali_c55 *mali_c55 = isp->mali_c55;
>> + struct v4l2_subdev *src_sd;
>> +
>> + if (isp->remote_src) {
>> + src_sd = media_entity_to_v4l2_subdev(isp->remote_src->entity);
>> + v4l2_subdev_disable_streams(src_sd, isp->remote_src->index,
>> + BIT(0));
>> + }
>> + isp->remote_src = NULL;
>> +
>> + mali_c55_isp_stop(mali_c55);
>> +
>> + return 0;
>> +}
>> +
>> +static const struct v4l2_subdev_pad_ops mali_c55_isp_pad_ops = {
>> + .enum_mbus_code = mali_c55_isp_enum_mbus_code,
>> + .enum_frame_size = mali_c55_isp_enum_frame_size,
>> + .get_fmt = v4l2_subdev_get_fmt,
>> + .set_fmt = mali_c55_isp_set_fmt,
>> + .get_selection = mali_c55_isp_get_selection,
>> + .set_selection = mali_c55_isp_set_selection,
>> + .link_validate = v4l2_subdev_link_validate_default,
>> + .enable_streams = mali_c55_isp_enable_streams,
>> + .disable_streams = mali_c55_isp_disable_streams,
>> +};
>> +
>> +void mali_c55_isp_queue_event_sof(struct mali_c55 *mali_c55)
>> +{
>> + struct v4l2_event event = {
>> + .type = V4L2_EVENT_FRAME_SYNC,
>> + };
>> +
>> + event.u.frame_sync.frame_sequence = mali_c55->isp.frame_sequence;
>> + v4l2_event_queue(mali_c55->isp.sd.devnode, &event);
>> +}
>> +
>> +static int
>> +mali_c55_isp_subscribe_event(struct v4l2_subdev *sd, struct v4l2_fh *fh,
>> + struct v4l2_event_subscription *sub)
>> +{
>> + if (sub->type != V4L2_EVENT_FRAME_SYNC)
>> + return -EINVAL;
>> +
>> + /* V4L2_EVENT_FRAME_SYNC doesn't require an id, so zero should be set */
>> + if (sub->id != 0)
>> + return -EINVAL;
>> +
>> + return v4l2_event_subscribe(fh, sub, 0, NULL);
>> +}
>> +
>> +static const struct v4l2_subdev_core_ops mali_c55_isp_core_ops = {
>> + .subscribe_event = mali_c55_isp_subscribe_event,
>> + .unsubscribe_event = v4l2_event_subdev_unsubscribe,
>> +};
>> +
>> +static const struct v4l2_subdev_ops mali_c55_isp_ops = {
>> + .pad = &mali_c55_isp_pad_ops,
>> + .core = &mali_c55_isp_core_ops,
>> +};
>> +
>> +static int mali_c55_isp_init_state(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state)
>> +{
>> + struct v4l2_mbus_framefmt *sink_fmt, *src_fmt;
>> + struct v4l2_rect *in_crop;
>> +
>> + sink_fmt = v4l2_subdev_state_get_format(state,
>> + MALI_C55_ISP_PAD_SINK_VIDEO);
>> + src_fmt = v4l2_subdev_state_get_format(state,
>> + MALI_C55_ISP_PAD_SOURCE_VIDEO);
>> + in_crop = v4l2_subdev_state_get_crop(state,
>> + MALI_C55_ISP_PAD_SINK_VIDEO);
>> +
>> + sink_fmt->width = MALI_C55_DEFAULT_WIDTH;
>> + sink_fmt->height = MALI_C55_DEFAULT_HEIGHT;
>> + sink_fmt->field = V4L2_FIELD_NONE;
>> + sink_fmt->code = MEDIA_BUS_FMT_SRGGB20_1X20;
>> + sink_fmt->colorspace = V4L2_COLORSPACE_RAW;
> You should initialize the three other colorspace-related fields :-) Same
> below.
>
>> +
>> + *v4l2_subdev_state_get_format(state,
>> + MALI_C55_ISP_PAD_SOURCE_BYPASS) = *sink_fmt;
>> +
>> + src_fmt->width = MALI_C55_DEFAULT_WIDTH;
>> + src_fmt->height = MALI_C55_DEFAULT_HEIGHT;
>> + src_fmt->field = V4L2_FIELD_NONE;
>> + src_fmt->code = MEDIA_BUS_FMT_RGB121212_1X36;
>> + src_fmt->colorspace = V4L2_COLORSPACE_SRGB;
>> +
>> + in_crop->top = 0;
>> + in_crop->left = 0;
>> + in_crop->width = MALI_C55_DEFAULT_WIDTH;
>> + in_crop->height = MALI_C55_DEFAULT_HEIGHT;
>> +
>> + return 0;
>> +}
>> +
>> +static const struct v4l2_subdev_internal_ops mali_c55_isp_internal_ops = {
>> + .init_state = mali_c55_isp_init_state,
>> +};
>> +
>> +static const struct media_entity_operations mali_c55_isp_media_ops = {
>> + .link_validate = v4l2_subdev_link_validate,
> .link_validate = v4l2_subdev_link_validate,
>
> to match mali_c55_isp_internal_ops.
>
>> +};
>> +
>> +int mali_c55_register_isp(struct mali_c55 *mali_c55)
>> +{
>> + struct mali_c55_isp *isp = &mali_c55->isp;
>> + struct v4l2_subdev *sd = &isp->sd;
>> + int ret;
>> +
>> + isp->mali_c55 = mali_c55;
>> +
>> + v4l2_subdev_init(sd, &mali_c55_isp_ops);
>> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
>> + sd->entity.ops = &mali_c55_isp_media_ops;
>> + sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_ISP;
>> + sd->internal_ops = &mali_c55_isp_internal_ops;
>> + strscpy(sd->name, MALI_C55_DRIVER_NAME " isp", sizeof(sd->name));
>> +
>> + isp->pads[MALI_C55_ISP_PAD_SINK_VIDEO].flags = MEDIA_PAD_FL_SINK |
>> + MEDIA_PAD_FL_MUST_CONNECT;
>> + isp->pads[MALI_C55_ISP_PAD_SOURCE_VIDEO].flags = MEDIA_PAD_FL_SOURCE;
>> + isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE;
>> +
>> + ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS,
>> + isp->pads);
>> + if (ret)
>> + return ret;
>> +
>> + ret = v4l2_subdev_init_finalize(sd);
>> + if (ret)
>> + goto err_cleanup_media_entity;
>> +
>> + ret = v4l2_device_register_subdev(&mali_c55->v4l2_dev, sd);
>> + if (ret)
>> + goto err_cleanup_subdev;
>> +
>> + mutex_init(&isp->capture_lock);
>> +
>> + return 0;
>> +
>> +err_cleanup_subdev:
>> + v4l2_subdev_cleanup(sd);
>> +err_cleanup_media_entity:
>> + media_entity_cleanup(&sd->entity);
>> + isp->mali_c55 = NULL;
>> +
>> + return ret;
>> +}
>> +
>> +void mali_c55_unregister_isp(struct mali_c55 *mali_c55)
>> +{
>> + struct mali_c55_isp *isp = &mali_c55->isp;
>> +
>> + if (!isp->mali_c55)
>> + return;
>> +
>> + mutex_destroy(&isp->capture_lock);
>> + v4l2_device_unregister_subdev(&isp->sd);
>> + v4l2_subdev_cleanup(&isp->sd);
>> + media_entity_cleanup(&isp->sd.entity);
>> +}
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>> new file mode 100644
>> index 000000000000..0a391f8a043e
>> --- /dev/null
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>> @@ -0,0 +1,313 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + * ARM Mali-C55 ISP Driver - Register definitions
>> + *
>> + * Copyright (C) 2024 Ideas on Board Oy
>> + */
>> +
>> +#ifndef _MALI_C55_REGISTERS_H
>> +#define _MALI_C55_REGISTERS_H
>> +
>> +#include <linux/bits.h>
>> +
>> +/* ISP Common 0x00000 - 0x000ff */
>> +
>> +#define MALI_C55_REG_API 0x00000
>> +#define MALI_C55_REG_PRODUCT 0x00004
>> +#define MALI_C55_REG_VERSION 0x00008
>> +#define MALI_C55_REG_REVISION 0x0000c
>> +#define MALI_C55_REG_PULSE_MODE 0x0003c
>> +#define MALI_C55_REG_INPUT_MODE_REQUEST 0x0009c
>> +#define MALI_C55_INPUT_SAFE_STOP 0x00
>> +#define MALI_C55_INPUT_SAFE_START 0x01
>> +#define MALI_C55_REG_MODE_STATUS 0x000a0
>> +#define MALI_C55_REG_INTERRUPT_MASK_VECTOR 0x00030
>> +#define MALI_C55_INTERRUPT_MASK_ALL GENMASK(31, 0)
>> +
>> +#define MALI_C55_REG_GLOBAL_MONITOR 0x00050
>> +
>> +#define MALI_C55_REG_GEN_VIDEO 0x00080
>> +#define MALI_C55_REG_GEN_VIDEO_ON_MASK BIT(0)
>> +#define MALI_C55_REG_GEN_VIDEO_MULTI_MASK BIT(1)
>> +#define MALI_C55_REG_GEN_PREFETCH_MASK GENMASK(31, 16)
>> +
>> +#define MALI_C55_REG_MCU_CONFIG 0x00020
>> +#define MALI_C55_REG_MCU_CONFIG_OVERRIDE_MASK BIT(0)
> For single bit fields like this one, you can rename the macro to
> MALI_C55_REG_MCU_CONFIG_OVERRIDE and write
>
> mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG,
> MALI_C55_REG_MCU_CONFIG_OVERRIDE,
> MALI_C55_REG_MCU_CONFIG_OVERRIDE);
>
>> +#define MALI_C55_REG_MCU_CONFIG_WRITE_MASK BIT(1)
>> +#define MALI_C55_MCU_CONFIG_WRITE(x) ((x) << 1)
>> +#define MALI_C55_REG_MCU_CONFIG_WRITE_PING BIT(1)
>> +#define MALI_C55_REG_MCU_CONFIG_WRITE_PONG 0x00
>> +#define MALI_C55_REG_MULTI_CONTEXT_MODE_MASK BIT(8)
>> +#define MALI_C55_REG_PING_PONG_READ 0x00024
>> +#define MALI_C55_REG_PING_PONG_READ_MASK BIT(2)
>> +
>> +#define MALI_C55_REG_INTERRUPT_CLEAR_VECTOR 0x00034
>> +#define MALI_C55_REG_INTERRUPT_CLEAR 0x00040
>> +#define MALI_C55_REG_INTERRUPT_STATUS_VECTOR 0x00044
>> +
>> +enum mali_c55_interrupts {
>> + MALI_C55_IRQ_ISP_START,
>> + MALI_C55_IRQ_ISP_DONE,
>> + MALI_C55_IRQ_MCM_ERROR,
>> + MALI_C55_IRQ_BROKEN_FRAME_ERROR,
>> + MALI_C55_IRQ_MET_AF_DONE,
>> + MALI_C55_IRQ_MET_AEXP_DONE,
>> + MALI_C55_IRQ_MET_AWB_DONE,
>> + MALI_C55_IRQ_AEXP_1024_DONE,
>> + MALI_C55_IRQ_IRIDIX_MET_DONE,
>> + MALI_C55_IRQ_LUT_INIT_DONE,
>> + MALI_C55_IRQ_FR_Y_DONE,
>> + MALI_C55_IRQ_FR_UV_DONE,
>> + MALI_C55_IRQ_DS_Y_DONE,
>> + MALI_C55_IRQ_DS_UV_DONE,
>> + MALI_C55_IRQ_LINEARIZATION_DONE,
>> + MALI_C55_IRQ_RAW_FRONTEND_DONE,
>> + MALI_C55_IRQ_NOISE_REDUCTION_DONE,
>> + MALI_C55_IRQ_IRIDIX_DONE,
>> + MALI_C55_IRQ_BAYER2RGB_DONE,
>> + MALI_C55_IRQ_WATCHDOG_TIMER,
>> + MALI_C55_IRQ_FRAME_COLLISION,
>> + MALI_C55_IRQ_UNUSED,
>> + MALI_C55_IRQ_DMA_ERROR,
>> + MALI_C55_IRQ_INPUT_STOPPED,
>> + MALI_C55_IRQ_MET_AWB_TARGET1_HIT,
>> + MALI_C55_IRQ_MET_AWB_TARGET2_HIT,
>> + MALI_C55_NUM_IRQ_BITS
>> +};
>> +
>> +#define MALI_C55_INTERRUPT_BIT(x) (0x01 << (x))
> #define MALI_C55_INTERRUPT_BIT(x) BIT(x)
>
>> +
>> +#define MALI_C55_REG_GLOBAL_PARAMETER_STATUS 0x00068
>> +#define MALI_C55_GPS_PONG_FITTED BIT(0)
>> +#define MALI_C55_GPS_WDR_FITTED BIT(1)
>> +#define MALI_C55_GPS_COMPRESSION_FITTED BIT(2)
>> +#define MALI_C55_GPS_TEMPER_FITTED BIT(3)
>> +#define MALI_C55_GPS_SINTER_LITE_FITTED BIT(4)
>> +#define MALI_C55_GPS_SINTER_FITTED BIT(5)
>> +#define MALI_C55_GPS_IRIDIX_LTM_FITTED BIT(6)
>> +#define MALI_C55_GPS_IRIDIX_GTM_FITTED BIT(7)
>> +#define MALI_C55_GPS_CNR_FITTED BIT(8)
>> +#define MALI_C55_GPS_FRSCALER_FITTED BIT(9)
>> +#define MALI_C55_GPS_DS_PIPE_FITTED BIT(10)
>> +
>> +#define MALI_C55_REG_BLANKING 0x00084
>> +#define MALI_C55_REG_HBLANK_MASK GENMASK(15, 0)
>> +#define MALI_C55_REG_VBLANK_MASK GENMASK(31, 16)
>> +#define MALI_C55_VBLANK(x) ((x) << 16)
>> +
>> +#define MALI_C55_REG_HC_START 0x00088
>> +#define MALI_C55_HC_START(h) (((h) & 0xffff) << 16)
>> +#define MALI_C55_REG_HC_SIZE 0x0008c
>> +#define MALI_C55_HC_SIZE(h) ((h) & 0xffff)
>> +#define MALI_C55_REG_VC_START_SIZE 0x00094
>> +#define MALI_C55_VC_START(v) ((v) & 0xffff)
>> +#define MALI_C55_VC_SIZE(v) (((v) & 0xffff) << 16)
>> +
>> +/* Ping/Pong Configuration Space */
>> +#define MALI_C55_REG_BASE_ADDR 0x18e88
>> +#define MALI_C55_REG_BYPASS_0 0x18eac
>> +#define MALI_C55_REG_BYPASS_0_VIDEO_TEST BIT(0)
>> +#define MALI_C55_REG_BYPASS_0_INPUT_FMT BIT(1)
>> +#define MALI_C55_REG_BYPASS_0_DECOMPANDER BIT(2)
>> +#define MALI_C55_REG_BYPASS_0_SENSOR_OFFSET_WDR BIT(3)
>> +#define MALI_C55_REG_BYPASS_0_GAIN_WDR BIT(4)
>> +#define MALI_C55_REG_BYPASS_0_FRAME_STITCH BIT(5)
>> +#define MALI_C55_REG_BYPASS_1 0x18eb0
>> +#define MALI_C55_REG_BYPASS_1_DIGI_GAIN BIT(0)
>> +#define MALI_C55_REG_BYPASS_1_FE_SENSOR_OFFS BIT(1)
>> +#define MALI_C55_REG_BYPASS_1_FE_SQRT BIT(2)
>> +#define MALI_C55_REG_BYPASS_1_RAW_FE BIT(3)
>> +#define MALI_C55_REG_BYPASS_2 0x18eb8
>> +#define MALI_C55_REG_BYPASS_2_SINTER BIT(0)
>> +#define MALI_C55_REG_BYPASS_2_TEMPER BIT(1)
>> +#define MALI_C55_REG_BYPASS_3 0x18ebc
>> +#define MALI_C55_REG_BYPASS_3_SQUARE_BE BIT(0)
>> +#define MALI_C55_REG_BYPASS_3_SENSOR_OFFSET_PRE_SH BIT(1)
>> +#define MALI_C55_REG_BYPASS_3_MESH_SHADING BIT(3)
>> +#define MALI_C55_REG_BYPASS_3_WHITE_BALANCE BIT(4)
>> +#define MALI_C55_REG_BYPASS_3_IRIDIX BIT(5)
>> +#define MALI_C55_REG_BYPASS_3_IRIDIX_GAIN BIT(6)
>> +#define MALI_C55_REG_BYPASS_4 0x18ec0
>> +#define MALI_C55_REG_BYPASS_4_DEMOSAIC_RGB BIT(1)
>> +#define MALI_C55_REG_BYPASS_4_PF_CORRECTION BIT(3)
>> +#define MALI_C55_REG_BYPASS_4_CCM BIT(4)
>> +#define MALI_C55_REG_BYPASS_4_CNR BIT(5)
>> +#define MALI_C55_REG_FR_BYPASS 0x18ec4
>> +#define MALI_C55_REG_DS_BYPASS 0x18ec8
>> +#define MALI_C55_BYPASS_CROP BIT(0)
>> +#define MALI_C55_BYPASS_SCALER BIT(1)
>> +#define MALI_C55_BYPASS_GAMMA_RGB BIT(2)
>> +#define MALI_C55_BYPASS_SHARPEN BIT(3)
>> +#define MALI_C55_BYPASS_CS_CONV BIT(4)
>> +#define MALI_C55_REG_ISP_RAW_BYPASS 0x18ecc
>> +#define MALI_C55_ISP_RAW_BYPASS_BYPASS_MASK BIT(0)
>> +#define MALI_C55_ISP_RAW_BYPASS_FR_BYPASS_MASK GENMASK(9, 8)
>> +#define MALI_C55_ISP_RAW_BYPASS_RAW_FR_BYPASS (2 << 8)
>> +#define MALI_C55_ISP_RAW_BYPASS_RGB_FR_BYPASS (1 << 8)
>> +#define MALI_C55_ISP_RAW_BYPASS_DS_PIPE_DISABLE BIT(1)
>> +#define MALI_C55_ISP_RAW_BYPASS_RAW_BYPASS BIT(0)
>> +
>> +#define MALI_C55_REG_ACTIVE_WIDTH_MASK 0xffff
>> +#define MALI_C55_REG_ACTIVE_HEIGHT_MASK 0xffff0000
>> +#define MALI_C55_REG_BAYER_ORDER 0x18e8c
>> +#define MALI_C55_BAYER_ORDER_MASK GENMASK(1, 0)
>> +#define MALI_C55_BAYER_ORDER_RGGB 0
>> +#define MALI_C55_BAYER_ORDER_GRBG 1
>> +#define MALI_C55_BAYER_ORDER_GBRG 2
>> +#define MALI_C55_BAYER_ORDER_BGGR 3
>> +
>> +#define MALI_C55_REG_TPG_CH0 0x18ed8
>> +#define MALI_C55_TEST_PATTERN_ON_OFF BIT(0)
>> +#define MALI_C55_TEST_PATTERN_RGB_MASK BIT(1)
>> +#define MALI_C55_TEST_PATTERN_RGB(x) ((x) << 1)
>> +#define MALI_C55_REG_TPG_R_BACKGROUND 0x18ee0
>> +#define MALI_C55_REG_TPG_G_BACKGROUND 0x18ee4
>> +#define MALI_C55_REG_TPG_B_BACKGROUND 0x18ee8
>> +#define MALI_C55_TPG_BACKGROUND_MAX 0xfffff
>> +#define MALI_C55_REG_INPUT_WIDTH 0x18f98
>> +#define MALI_C55_INPUT_WIDTH_MASK GENMASK(18, 16)
>> +#define MALI_C55_INPUT_WIDTH_8BIT (0 << 16)
>> +#define MALI_C55_INPUT_WIDTH_10BIT (1 << 16)
>> +#define MALI_C55_INPUT_WIDTH_12BIT (2 << 16)
>> +#define MALI_C55_INPUT_WIDTH_14BIT (3 << 16)
>> +#define MALI_C55_INPUT_WIDTH_16BIT (4 << 16)
>> +#define MALI_C55_INPUT_WIDTH_20BIT (5 << 16)
>> +#define MALI_C55_REG_SPACE_SIZE 0x4000
>> +#define MALI_C55_REG_CONFIG_SPACES_OFFSET 0x0ab6c
>> +#define MALI_C55_CONFIG_SPACE_SIZE 0x1231c
>> +
>> +#define MALI_C55_REG_SINTER_CONFIG 0x19348
>> +#define MALI_C55_SINTER_VIEW_FILTER_MASK GENMASK(1, 0)
>> +#define MALI_C55_SINTER_SCALE_MODE_MASK GENMASK(3, 2)
>> +#define MALI_C55_SINTER_ENABLE_MASK BIT(4)
>> +#define MALI_C55_SINTER_FILTER_SELECT_MASK BIT(5)
>> +#define MALI_C55_SINTER_INT_SELECT_MASK BIT(6)
>> +#define MALI_C55_SINTER_RM_ENABLE_MASK BIT(7)
>> +
>> +/* Colour Correction Matrix Configuration */
>> +#define MALI_C55_REG_CCM_ENABLE 0x1b07c
>> +#define MALI_C55_CCM_ENABLE_MASK BIT(0)
>> +#define MALI_C55_REG_CCM_COEF_R_R 0x1b080
>> +#define MALI_C55_REG_CCM_COEF_R_G 0x1b084
>> +#define MALI_C55_REG_CCM_COEF_R_B 0x1b088
>> +#define MALI_C55_REG_CCM_COEF_G_R 0x1b090
>> +#define MALI_C55_REG_CCM_COEF_G_G 0x1b094
>> +#define MALI_C55_REG_CCM_COEF_G_B 0x1b098
>> +#define MALI_C55_REG_CCM_COEF_B_R 0x1b0a0
>> +#define MALI_C55_REG_CCM_COEF_B_G 0x1b0a4
>> +#define MALI_C55_REG_CCM_COEF_B_B 0x1b0a8
>> +#define MALI_C55_CCM_COEF_MASK GENMASK(12, 0)
>> +#define MALI_C55_REG_CCM_ANTIFOG_GAIN_R 0x1b0b0
>> +#define MALI_C55_REG_CCM_ANTIFOG_GAIN_G 0x1b0b4
>> +#define MALI_C55_REG_CCM_ANTIFOG_GAIN_B 0x1b0b8
>> +#define MALI_C55_CCM_ANTIFOG_GAIN_MASK GENMASK(11, 0)
>> +#define MALI_C55_REG_CCM_ANTIFOG_OFFSET_R 0x1b0c0
>> +#define MALI_C55_REG_CCM_ANTIFOG_OFFSET_G 0x1b0c4
>> +#define MALI_C55_REG_CCM_ANTIFOG_OFFSET_B 0x1b0c8
>> +#define MALI_C55_CCM_ANTIFOG_OFFSET_MASK GENMASK(11, 0)
>> +
>> +/*
>> + * The Mali-C55 ISP has up to two output pipes; known as full resolution and
>> + * down scaled. The register space for these is laid out identically, but offset
>> + * by 372 bytes.
>> + */
>> +#define MALI_C55_CAP_DEV_FR_REG_OFFSET 0x0
>> +#define MALI_C55_CAP_DEV_DS_REG_OFFSET 0x174
>> +
>> +#define MALI_C55_REG_CS_CONV_CONFIG 0x1c098
>> +#define MALI_C55_CS_CONV_MATRIX_MASK BIT(0)
>> +#define MALI_C55_CS_CONV_FILTER_MASK BIT(1)
>> +#define MALI_C55_CS_CONV_HORZ_DOWNSAMPLE_MASK BIT(2)
>> +#define MALI_C55_CS_CONV_VERT_DOWNSAMPLE_MASK BIT(3)
>> +#define MALI_C55_CS_CONV_FILTER_ENABLE (0x01 << 1)
>> +#define MALI_C55_CS_CONV_HORZ_DOWNSAMPLE_ENABLE (0x01 << 2)
>> +#define MALI_C55_CS_CONV_VERT_DOWNSAMPLE_ENABLE (0x01 << 3)
>> +#define MALI_C55_REG_Y_WRITER_MODE 0x1c0ec
>> +#define MALI_C55_REG_UV_WRITER_MODE 0x1c144
>> +#define MALI_C55_WRITER_MODE_MASK GENMASK(4, 0)
>> +#define MALI_C55_OUTPUT_DISABLED 0
>> +#define MALI_C55_OUTPUT_RGB32 1
>> +#define MALI_C55_OUTPUT_A2R10G10B10 2
>> +#define MALI_C55_OUTPUT_RGB565 3
>> +#define MALI_C55_OUTPUT_RGB24 4
>> +#define MALI_C55_OUTPUT_GEN32 5
>> +#define MALI_C55_OUTPUT_RAW16 6
>> +#define MALI_C55_OUTPUT_AYUV 8
>> +#define MALI_C55_OUTPUT_Y410 9
>> +#define MALI_C55_OUTPUT_YUY2 10
>> +#define MALI_C55_OUTPUT_UYVY 11
>> +#define MALI_C55_OUTPUT_Y210 12
>> +#define MALI_C55_OUTPUT_NV12_21 13
>> +#define MALI_C55_OUTPUT_YUV_420_422 17
>> +#define MALI_C55_OUTPUT_P210_P010 19
>> +#define MALI_C55_OUTPUT_YUV422 20
>> +#define MALI_C55_WRITER_SUBMODE_MASK GENMASK(7, 6)
>> +#define MALI_C55_WRITER_SUBMODE(x) ((x) << 6)
>> +#define MALI_C55_OUTPUT_PLANE_ALT0 0
>> +#define MALI_C55_OUTPUT_PLANE_ALT1 1
>> +#define MALI_C55_OUTPUT_PLANE_ALT2 2
>> +#define MALI_C55_WRITER_FRAME_WRITE_MASK BIT(9)
>> +#define MALI_C55_WRITER_FRAME_WRITE_ENABLE (0x01 << 9)
>> +#define MALI_C55_REG_ACTIVE_OUT_Y_SIZE 0x1c0f0
>> +#define MALI_C55_REG_ACTIVE_OUT_UV_SIZE 0x1c148
>> +#define MALI_C55_REG_ACTIVE_OUT_SIZE_W(w) ((w) << 0)
>> +#define MALI_C55_REG_ACTIVE_OUT_SIZE_H(h) ((h) << 16)
>> +#define MALI_C55_REG_Y_WRITER_BANKS_BASE 0x1c0f4
>> +#define MALI_C55_REG_Y_WRITER_BANKS_CONFIG 0x1c108
>> +#define MALI_C55_REG_Y_WRITER_MAX_BANKS_MASK GENMASK(2, 0)
>> +#define MALI_C55_REG_Y_WRITER_BANKS_RESTART BIT(3)
>> +#define MALI_C55_REG_Y_WRITER_OFFSET 0x1c10c
>> +#define MALI_C55_REG_UV_WRITER_BANKS_BASE 0x1c14c
>> +#define MALI_C55_REG_UV_WRITER_BANKS_CONFIG 0x1c160
>> +#define MALI_C55_REG_UV_WRITER_MAX_BANKS_MASK GENMASK(2, 0)
>> +#define MALI_C55_REG_UV_WRITER_BANKS_RESTART BIT(3)
>> +#define MALI_C55_REG_UV_WRITER_OFFSET 0x1c164
>> +
>> +#define MALI_C55_REG_TEST_GEN_CH0_OFF_ON
>> +#define MALI_C55_REG_TEST_GEN_CH0_PATTERN_TYPE 0x18edc
>> +
>> +#define MALI_C55_REG_CROP_EN 0x1c028
>> +#define MALI_C55_CROP_ENABLE BIT(0)
>> +#define MALI_C55_REG_CROP_X_START 0x1c02c
>> +#define MALI_C55_REG_CROP_Y_START 0x1c030
>> +#define MALI_C55_REG_CROP_X_SIZE 0x1c034
>> +#define MALI_C55_REG_CROP_Y_SIZE 0x1c038
>> +#define MALI_C55_REG_SCALER_TIMEOUT_EN 0x1c040
>> +#define MALI_C55_SCALER_TIMEOUT_EN BIT(4)
>> +#define MALI_C55_SCALER_TIMEOUT(t) ((t) << 16)
>> +#define MALI_C55_REG_SCALER_IN_WIDTH 0x1c044
>> +#define MALI_C55_REG_SCALER_IN_HEIGHT 0x1c048
>> +#define MALI_C55_REG_SCALER_OUT_WIDTH 0x1c04c
>> +#define MALI_C55_REG_SCALER_OUT_HEIGHT 0x1c050
>> +#define MALI_C55_REG_SCALER_HFILT_TINC 0x1c054
>> +#define MALI_C55_REG_SCALER_HFILT_COEF 0x1c058
>> +#define MALI_C55_REG_SCALER_VFILT_TINC 0x1c05c
>> +#define MALI_C55_REG_SCALER_VFILT_COEF 0x1c060
>> +
>> +#define MALI_C55_REG_GAMMA_RGB_ENABLE 0x1c064
>> +#define MALI_C55_GAMMA_ENABLE_MASK BIT(0)
>> +#define MALI_C55_REG_GAMMA_GAINS_1 0x1c068
>> +#define MALI_C55_GAMMA_GAIN_R_MASK GENMASK(11, 0)
>> +#define MALI_C55_GAMMA_GAIN_G_MASK GENMASK(27, 16)
>> +#define MALI_C55_REG_GAMMA_GAINS_2 0x1c06c
>> +#define MALI_C55_GAMMA_GAIN_B_MASK GENMASK(11, 0)
>> +#define MALI_C55_REG_GAMMA_OFFSETS_1 0x1c070
>> +#define MALI_C55_GAMMA_OFFSET_R_MASK GENMASK(11, 0)
>> +#define MALI_C55_GAMMA_OFFSET_G_MASK GENMASK(27, 16)
>> +#define MALI_C55_REG_GAMMA_OFFSETS_2 0x1c074
>> +#define MALI_C55_GAMMA_OFFSET_B_MASK GENMASK(11, 0)
>> +
>> +/*
>> + * A re-definition of an above register. These will usually be written on a per
>> + * capture device basis and handled with mali_c55_cap_dev_write(), but on
>> + * startup is written by core.c
>> + */
>> +#define MALI_C55_REG_FR_GAMMA_RGB_ENABLE 0x1c064
>> +#define MALI_C55_REG_DS_GAMMA_RGB_ENABLE 0x1c1d8
>> +
>> +#define MALI_C55_REG_FR_SCALER_HFILT 0x34a8
>> +#define MALI_C55_REG_FR_SCALER_VFILT 0x44a8
>> +#define MALI_C55_REG_DS_SCALER_HFILT 0x14a8
>> +#define MALI_C55_REG_DS_SCALER_VFILT 0x24a8
>> +
>> +#endif /* _MALI_C55_REGISTERS_H */
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c b/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
>> new file mode 100644
>> index 000000000000..b63a94cbe2fc
>> --- /dev/null
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
>> @@ -0,0 +1,1096 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * ARM Mali-C55 ISP Driver - Image signal processor
>> + *
>> + * Copyright (C) 2024 Ideas on Board Oy
>> + */
>> +
>> +#include <linux/math.h>
>> +#include <linux/minmax.h>
>> +
>> +#include <media/media-entity.h>
>> +#include <media/v4l2-subdev.h>
>> +
>> +#include "mali-c55-common.h"
>> +#include "mali-c55-registers.h"
>> +
>> +/* Scaling factor in Q4.20 format. */
>> +#define MALI_C55_RSZ_SCALER_FACTOR (1U << 20)
>> +
>> +#define MALI_C55_RSZ_COEFS_BANKS 8
>> +#define MALI_C55_RSZ_COEFS_ENTRIES 64
>> +
>> +static inline struct mali_c55_resizer *
>> +sd_to_mali_c55_rsz(struct v4l2_subdev *sd)
>> +{
>> + return container_of(sd, struct mali_c55_resizer, sd);
>> +}
>> +
>> +static const unsigned int
>> +mali_c55_rsz_filter_coeffs_h[MALI_C55_RSZ_COEFS_BANKS]
>> + [MALI_C55_RSZ_COEFS_ENTRIES] = {
>> + { /* Bank 0 */
>> + 0x24fc0000, 0x0000fc24, 0x27fc0000, 0x0000fc21,
>> + 0x28fc0000, 0x0000fd1f, 0x2cfb0000, 0x0000fd1c,
>> + 0x2efb0000, 0x0000fd1a, 0x30fb0000, 0x0000fe17,
>> + 0x32fb0000, 0x0000fe15, 0x35fb0000, 0x0000fe12,
>> + 0x35fc0000, 0x0000ff10, 0x37fc0000, 0x0000ff0e,
>> + 0x39fc0000, 0x0000ff0c, 0x3afd0000, 0x0000ff0a,
>> + 0x3afe0000, 0x00000008, 0x3cfe0000, 0x00000006,
>> + 0x3dff0000, 0x00000004, 0x3d000000, 0x00000003,
>> + 0x3c020000, 0x00000002, 0x3d030000, 0x00000000,
>> + 0x3d040000, 0x000000ff, 0x3c060000, 0x000000fe,
>> + 0x3a080000, 0x000000fe, 0x3a0aff00, 0x000000fd,
>> + 0x390cff00, 0x000000fc, 0x370eff00, 0x000000fc,
>> + 0x3510ff00, 0x000000fc, 0x3512fe00, 0x000000fb,
>> + 0x3215fe00, 0x000000fb, 0x3017fe00, 0x000000fb,
>> + 0x2e1afd00, 0x000000fb, 0x2c1cfd00, 0x000000fb,
>> + 0x281ffd00, 0x000000fc, 0x2721fc00, 0x000000fc,
>> + },
>> + { /* Bank 1 */
>> + 0x25fb0000, 0x0000fb25, 0x27fb0000, 0x0000fb23,
>> + 0x29fb0000, 0x0000fb21, 0x2afc0000, 0x0000fb1f,
>> + 0x2cfc0000, 0x0000fb1d, 0x2efc0000, 0x0000fb1b,
>> + 0x2ffd0000, 0x0000fb19, 0x2ffe0000, 0x0000fc17,
>> + 0x31fe0000, 0x0000fc15, 0x32ff0000, 0x0000fc13,
>> + 0x3400ff00, 0x0000fc11, 0x3301ff00, 0x0000fd10,
>> + 0x3402ff00, 0x0000fd0e, 0x3503ff00, 0x0000fd0c,
>> + 0x3505ff00, 0x0000fd0a, 0x3506fe00, 0x0000fe09,
>> + 0x3607fe00, 0x0000fe07, 0x3509fe00, 0x0000fe06,
>> + 0x350afd00, 0x0000ff05, 0x350cfd00, 0x0000ff03,
>> + 0x340efd00, 0x0000ff02, 0x3310fd00, 0x0000ff01,
>> + 0x3411fc00, 0x0000ff00, 0x3213fc00, 0x000000ff,
>> + 0x3115fc00, 0x000000fe, 0x2f17fc00, 0x000000fe,
>> + 0x2f19fb00, 0x000000fd, 0x2e1bfb00, 0x000000fc,
>> + 0x2c1dfb00, 0x000000fc, 0x2a1ffb00, 0x000000fc,
>> + 0x2921fb00, 0x000000fb, 0x2723fb00, 0x000000fb,
>> + },
>> + { /* Bank 2 */
>> + 0x1f010000, 0x0000011f, 0x21010000, 0x0000001e,
>> + 0x21020000, 0x0000001d, 0x22020000, 0x0000001c,
>> + 0x23030000, 0x0000ff1b, 0x2404ff00, 0x0000ff1a,
>> + 0x2504ff00, 0x0000ff19, 0x2505ff00, 0x0000ff18,
>> + 0x2606ff00, 0x0000fe17, 0x2607ff00, 0x0000fe16,
>> + 0x2708ff00, 0x0000fe14, 0x2709ff00, 0x0000fe13,
>> + 0x270aff00, 0x0000fe12, 0x280bfe00, 0x0000fe11,
>> + 0x280cfe00, 0x0000fe10, 0x280dfe00, 0x0000fe0f,
>> + 0x280efe00, 0x0000fe0e, 0x280ffe00, 0x0000fe0d,
>> + 0x2810fe00, 0x0000fe0c, 0x2811fe00, 0x0000fe0b,
>> + 0x2712fe00, 0x0000ff0a, 0x2713fe00, 0x0000ff09,
>> + 0x2714fe00, 0x0000ff08, 0x2616fe00, 0x0000ff07,
>> + 0x2617fe00, 0x0000ff06, 0x2518ff00, 0x0000ff05,
>> + 0x2519ff00, 0x0000ff04, 0x241aff00, 0x0000ff04,
>> + 0x231bff00, 0x00000003, 0x221c0000, 0x00000002,
>> + 0x211d0000, 0x00000002, 0x211e0000, 0x00000001,
>> + },
>> + { /* Bank 3 */
>> + 0x1b06ff00, 0x00ff061b, 0x1b07ff00, 0x00ff061a,
>> + 0x1c07ff00, 0x00ff051a, 0x1c08ff00, 0x00ff0519,
>> + 0x1c09ff00, 0x00ff0419, 0x1d09ff00, 0x00ff0418,
>> + 0x1e0aff00, 0x00ff0317, 0x1e0aff00, 0x00ff0317,
>> + 0x1e0bff00, 0x00ff0316, 0x1f0cff00, 0x00ff0215,
>> + 0x1e0cff00, 0x00000215, 0x1e0dff00, 0x00000214,
>> + 0x1e0e0000, 0x00000113, 0x1e0e0000, 0x00000113,
>> + 0x1e0f0000, 0x00000112, 0x1f100000, 0x00000011,
>> + 0x20100000, 0x00000010, 0x1f110000, 0x00000010,
>> + 0x1e120100, 0x0000000f, 0x1e130100, 0x0000000e,
>> + 0x1e130100, 0x0000000e, 0x1e140200, 0x0000ff0d,
>> + 0x1e150200, 0x0000ff0c, 0x1f1502ff, 0x0000ff0c,
>> + 0x1e1603ff, 0x0000ff0b, 0x1e1703ff, 0x0000ff0a,
>> + 0x1e1703ff, 0x0000ff0a, 0x1d1804ff, 0x0000ff09,
>> + 0x1c1904ff, 0x0000ff09, 0x1c1905ff, 0x0000ff08,
>> + 0x1c1a05ff, 0x0000ff07, 0x1b1a06ff, 0x0000ff07,
>> + },
>> + { /* Bank 4 */
>> + 0x17090000, 0x00000917, 0x18090000, 0x00000916,
>> + 0x170a0100, 0x00000816, 0x170a0100, 0x00000816,
>> + 0x180b0100, 0x00000715, 0x180b0100, 0x00000715,
>> + 0x170c0100, 0x00000715, 0x190c0100, 0x00000614,
>> + 0x180d0100, 0x00000614, 0x190d0200, 0x00000513,
>> + 0x180e0200, 0x00000513, 0x180e0200, 0x00000513,
>> + 0x1a0e0200, 0x00000412, 0x190f0200, 0x00000412,
>> + 0x190f0300, 0x00000411, 0x18100300, 0x00000411,
>> + 0x1a100300, 0x00000310, 0x18110400, 0x00000310,
>> + 0x19110400, 0x0000030f, 0x19120400, 0x0000020f,
>> + 0x1a120400, 0x0000020e, 0x18130500, 0x0000020e,
>> + 0x18130500, 0x0000020e, 0x19130500, 0x0000020d,
>> + 0x18140600, 0x0000010d, 0x19140600, 0x0000010c,
>> + 0x17150700, 0x0000010c, 0x18150700, 0x0000010b,
>> + 0x18150700, 0x0000010b, 0x17160800, 0x0000010a,
>> + 0x17160800, 0x0000010a, 0x18160900, 0x00000009,
>> + },
>> + { /* Bank 5 */
>> + 0x120b0300, 0x00030b12, 0x120c0300, 0x00030b11,
>> + 0x110c0400, 0x00030b11, 0x110c0400, 0x00030b11,
>> + 0x130c0400, 0x00020a11, 0x120d0400, 0x00020a11,
>> + 0x110d0500, 0x00020a11, 0x110d0500, 0x00020a11,
>> + 0x130d0500, 0x00010911, 0x130e0500, 0x00010910,
>> + 0x120e0600, 0x00010910, 0x120e0600, 0x00010910,
>> + 0x130e0600, 0x00010810, 0x120f0600, 0x00010810,
>> + 0x120f0700, 0x00000810, 0x130f0700, 0x0000080f,
>> + 0x140f0700, 0x0000070f, 0x130f0800, 0x0000070f,
>> + 0x12100800, 0x0000070f, 0x12100801, 0x0000060f,
>> + 0x13100801, 0x0000060e, 0x12100901, 0x0000060e,
>> + 0x12100901, 0x0000060e, 0x13100901, 0x0000050e,
>> + 0x13110901, 0x0000050d, 0x11110a02, 0x0000050d,
>> + 0x11110a02, 0x0000050d, 0x12110a02, 0x0000040d,
>> + 0x13110a02, 0x0000040c, 0x11110b03, 0x0000040c,
>> + 0x11110b03, 0x0000040c, 0x12110b03, 0x0000030c,
>> + },
>> + { /* Bank 6 */
>> + 0x0b0a0805, 0x00080a0c, 0x0b0a0805, 0x00080a0c,
>> + 0x0c0a0805, 0x00080a0b, 0x0c0a0805, 0x00080a0b,
>> + 0x0d0a0805, 0x00070a0b, 0x0d0a0805, 0x00070a0b,
>> + 0x0d0a0805, 0x00070a0b, 0x0c0a0806, 0x00070a0b,
>> + 0x0b0b0806, 0x00070a0b, 0x0c0b0806, 0x0007090b,
>> + 0x0b0b0906, 0x0007090b, 0x0b0b0906, 0x0007090b,
>> + 0x0b0b0906, 0x0007090b, 0x0b0b0906, 0x0007090b,
>> + 0x0b0b0906, 0x0007090b, 0x0c0b0906, 0x0006090b,
>> + 0x0c0b0906, 0x0006090b, 0x0c0b0906, 0x0006090b,
>> + 0x0b0b0907, 0x0006090b, 0x0b0b0907, 0x0006090b,
>> + 0x0b0b0907, 0x0006090b, 0x0b0b0907, 0x0006090b,
>> + 0x0b0b0907, 0x0006090b, 0x0c0b0907, 0x0006080b,
>> + 0x0b0b0a07, 0x0006080b, 0x0c0b0a07, 0x0006080a,
>> + 0x0d0b0a07, 0x0005080a, 0x0d0b0a07, 0x0005080a,
>> + 0x0d0b0a07, 0x0005080a, 0x0c0b0a08, 0x0005080a,
>> + 0x0c0b0a08, 0x0005080a, 0x0c0b0a08, 0x0005080a,
>> + },
>> + { /* Bank 7 */
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + 0x0909090a, 0x00090909, 0x0909090a, 0x00090909,
>> + }
>> +};
>> +
>> +static const unsigned int
>> +mali_c55_rsz_filter_coeffs_v[MALI_C55_RSZ_COEFS_BANKS]
>> + [MALI_C55_RSZ_COEFS_ENTRIES] = {
>> + { /* Bank 0 */
>> + 0x2424fc00, 0x000000fc, 0x2721fc00, 0x000000fc,
>> + 0x281ffd00, 0x000000fc, 0x2c1cfd00, 0x000000fb,
>> + 0x2e1afd00, 0x000000fb, 0x3017fe00, 0x000000fb,
>> + 0x3215fe00, 0x000000fb, 0x3512fe00, 0x000000fb,
>> + 0x3510ff00, 0x000000fc, 0x370eff00, 0x000000fc,
>> + 0x390cff00, 0x000000fc, 0x3a0aff00, 0x000000fd,
>> + 0x3a080000, 0x000000fe, 0x3c060000, 0x000000fe,
>> + 0x3d040000, 0x000000ff, 0x3d030000, 0x00000000,
>> + 0x3c020000, 0x00000002, 0x3d000000, 0x00000003,
>> + 0x3dff0000, 0x00000004, 0x3cfe0000, 0x00000006,
>> + 0x3afe0000, 0x00000008, 0x3afd0000, 0x0000ff0a,
>> + 0x39fc0000, 0x0000ff0c, 0x37fc0000, 0x0000ff0e,
>> + 0x35fc0000, 0x0000ff10, 0x35fb0000, 0x0000fe12,
>> + 0x32fb0000, 0x0000fe15, 0x30fb0000, 0x0000fe17,
>> + 0x2efb0000, 0x0000fd1a, 0x2cfb0000, 0x0000fd1c,
>> + 0x28fc0000, 0x0000fd1f, 0x27fc0000, 0x0000fc21,
>> + },
>> + { /* Bank 1 */
>> + 0x2525fb00, 0x000000fb, 0x2723fb00, 0x000000fb,
>> + 0x2921fb00, 0x000000fb, 0x2a1ffb00, 0x000000fc,
>> + 0x2c1dfb00, 0x000000fc, 0x2e1bfb00, 0x000000fc,
>> + 0x2f19fb00, 0x000000fd, 0x2f17fc00, 0x000000fe,
>> + 0x3115fc00, 0x000000fe, 0x3213fc00, 0x000000ff,
>> + 0x3411fc00, 0x0000ff00, 0x3310fd00, 0x0000ff01,
>> + 0x340efd00, 0x0000ff02, 0x350cfd00, 0x0000ff03,
>> + 0x350afd00, 0x0000ff05, 0x3509fe00, 0x0000fe06,
>> + 0x3607fe00, 0x0000fe07, 0x3506fe00, 0x0000fe09,
>> + 0x3505ff00, 0x0000fd0a, 0x3503ff00, 0x0000fd0c,
>> + 0x3402ff00, 0x0000fd0e, 0x3301ff00, 0x0000fd10,
>> + 0x3400ff00, 0x0000fc11, 0x32ff0000, 0x0000fc13,
>> + 0x31fe0000, 0x0000fc15, 0x2ffe0000, 0x0000fc17,
>> + 0x2ffd0000, 0x0000fb19, 0x2efc0000, 0x0000fb1b,
>> + 0x2cfc0000, 0x0000fb1d, 0x2afc0000, 0x0000fb1f,
>> + 0x29fb0000, 0x0000fb21, 0x27fb0000, 0x0000fb23,
>> + },
>> + { /* Bank 2 */
>> + 0x1f1f0100, 0x00000001, 0x211e0000, 0x00000001,
>> + 0x211d0000, 0x00000002, 0x221c0000, 0x00000002,
>> + 0x231bff00, 0x00000003, 0x241aff00, 0x0000ff04,
>> + 0x2519ff00, 0x0000ff04, 0x2518ff00, 0x0000ff05,
>> + 0x2617fe00, 0x0000ff06, 0x2616fe00, 0x0000ff07,
>> + 0x2714fe00, 0x0000ff08, 0x2713fe00, 0x0000ff09,
>> + 0x2712fe00, 0x0000ff0a, 0x2811fe00, 0x0000fe0b,
>> + 0x2810fe00, 0x0000fe0c, 0x280ffe00, 0x0000fe0d,
>> + 0x280efe00, 0x0000fe0e, 0x280dfe00, 0x0000fe0f,
>> + 0x280cfe00, 0x0000fe10, 0x280bfe00, 0x0000fe11,
>> + 0x270aff00, 0x0000fe12, 0x2709ff00, 0x0000fe13,
>> + 0x2708ff00, 0x0000fe14, 0x2607ff00, 0x0000fe16,
>> + 0x2606ff00, 0x0000fe17, 0x2505ff00, 0x0000ff18,
>> + 0x2504ff00, 0x0000ff19, 0x2404ff00, 0x0000ff1a,
>> + 0x23030000, 0x0000ff1b, 0x22020000, 0x0000001c,
>> + 0x21020000, 0x0000001d, 0x21010000, 0x0000001e,
>> + },
>> + { /* Bank 3 */
>> + 0x1b1b06ff, 0x0000ff06, 0x1b1a06ff, 0x0000ff07,
>> + 0x1c1a05ff, 0x0000ff07, 0x1c1905ff, 0x0000ff08,
>> + 0x1c1904ff, 0x0000ff09, 0x1d1804ff, 0x0000ff09,
>> + 0x1e1703ff, 0x0000ff0a, 0x1e1703ff, 0x0000ff0a,
>> + 0x1e1603ff, 0x0000ff0b, 0x1f1502ff, 0x0000ff0c,
>> + 0x1e150200, 0x0000ff0c, 0x1e140200, 0x0000ff0d,
>> + 0x1e130100, 0x0000000e, 0x1e130100, 0x0000000e,
>> + 0x1e120100, 0x0000000f, 0x1f110000, 0x00000010,
>> + 0x20100000, 0x00000010, 0x1f100000, 0x00000011,
>> + 0x1e0f0000, 0x00000112, 0x1e0e0000, 0x00000113,
>> + 0x1e0e0000, 0x00000113, 0x1e0dff00, 0x00000214,
>> + 0x1e0cff00, 0x00000215, 0x1f0cff00, 0x00ff0215,
>> + 0x1e0bff00, 0x00ff0316, 0x1e0aff00, 0x00ff0317,
>> + 0x1e0aff00, 0x00ff0317, 0x1d09ff00, 0x00ff0418,
>> + 0x1c09ff00, 0x00ff0419, 0x1c08ff00, 0x00ff0519,
>> + 0x1c07ff00, 0x00ff051a, 0x1b07ff00, 0x00ff061a,
>> + },
>> + { /* Bank 4 */
>> + 0x17170900, 0x00000009, 0x18160900, 0x00000009,
>> + 0x17160800, 0x0000010a, 0x17160800, 0x0000010a,
>> + 0x18150700, 0x0000010b, 0x18150700, 0x0000010b,
>> + 0x17150700, 0x0000010c, 0x19140600, 0x0000010c,
>> + 0x18140600, 0x0000010d, 0x19130500, 0x0000020d,
>> + 0x18130500, 0x0000020e, 0x18130500, 0x0000020e,
>> + 0x1a120400, 0x0000020e, 0x19120400, 0x0000020f,
>> + 0x19110400, 0x0000030f, 0x18110400, 0x00000310,
>> + 0x1a100300, 0x00000310, 0x18100300, 0x00000411,
>> + 0x190f0300, 0x00000411, 0x190f0200, 0x00000412,
>> + 0x1a0e0200, 0x00000412, 0x180e0200, 0x00000513,
>> + 0x180e0200, 0x00000513, 0x190d0200, 0x00000513,
>> + 0x180d0100, 0x00000614, 0x190c0100, 0x00000614,
>> + 0x170c0100, 0x00000715, 0x180b0100, 0x00000715,
>> + 0x180b0100, 0x00000715, 0x170a0100, 0x00000816,
>> + 0x170a0100, 0x00000816, 0x18090000, 0x00000916,
>> + },
>> + { /* Bank 5 */
>> + 0x12120b03, 0x0000030b, 0x12110b03, 0x0000030c,
>> + 0x11110b03, 0x0000040c, 0x11110b03, 0x0000040c,
>> + 0x13110a02, 0x0000040c, 0x12110a02, 0x0000040d,
>> + 0x11110a02, 0x0000050d, 0x11110a02, 0x0000050d,
>> + 0x13110901, 0x0000050d, 0x13100901, 0x0000050e,
>> + 0x12100901, 0x0000060e, 0x12100901, 0x0000060e,
>> + 0x13100801, 0x0000060e, 0x12100801, 0x0000060f,
>> + 0x12100800, 0x0000070f, 0x130f0800, 0x0000070f,
>> + 0x140f0700, 0x0000070f, 0x130f0700, 0x0000080f,
>> + 0x120f0700, 0x00000810, 0x120f0600, 0x00010810,
>> + 0x130e0600, 0x00010810, 0x120e0600, 0x00010910,
>> + 0x120e0600, 0x00010910, 0x130e0500, 0x00010910,
>> + 0x130d0500, 0x00010911, 0x110d0500, 0x00020a11,
>> + 0x110d0500, 0x00020a11, 0x120d0400, 0x00020a11,
>> + 0x130c0400, 0x00020a11, 0x110c0400, 0x00030b11,
>> + 0x110c0400, 0x00030b11, 0x120c0300, 0x00030b11,
>> + },
>> + { /* Bank 6 */
>> + 0x0b0c0a08, 0x0005080a, 0x0b0c0a08, 0x0005080a,
>> + 0x0c0b0a08, 0x0005080a, 0x0c0b0a08, 0x0005080a,
>> + 0x0d0b0a07, 0x0005080a, 0x0d0b0a07, 0x0005080a,
>> + 0x0d0b0a07, 0x0005080a, 0x0c0b0a07, 0x0006080a,
>> + 0x0b0b0a07, 0x0006080b, 0x0c0b0907, 0x0006080b,
>> + 0x0b0b0907, 0x0006090b, 0x0b0b0907, 0x0006090b,
>> + 0x0b0b0907, 0x0006090b, 0x0b0b0907, 0x0006090b,
>> + 0x0b0b0907, 0x0006090b, 0x0c0b0906, 0x0006090b,
>> + 0x0c0b0906, 0x0006090b, 0x0c0b0906, 0x0006090b,
>> + 0x0b0b0906, 0x0007090b, 0x0b0b0906, 0x0007090b,
>> + 0x0b0b0906, 0x0007090b, 0x0b0b0906, 0x0007090b,
>> + 0x0b0b0906, 0x0007090b, 0x0c0b0806, 0x0007090b,
>> + 0x0b0b0806, 0x00070a0b, 0x0c0a0806, 0x00070a0b,
>> + 0x0d0a0805, 0x00070a0b, 0x0d0a0805, 0x00070a0b,
>> + 0x0d0a0805, 0x00070a0b, 0x0c0a0805, 0x00080a0b,
>> + 0x0c0a0805, 0x00080a0b, 0x0c0a0805, 0x00080a0b,
>> + },
>> + { /* Bank 7 */
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + 0x09090909, 0x000a0909, 0x09090909, 0x000a0909,
>> + }
>> +};
>> +
>> +static const struct mali_c55_rsz_coef_bank {
>> + unsigned int top;
>> + unsigned int bottom;
> The bottom value of bank N is always equal to the top value of bank N+1.
> You can simplify this by storing a single value.
>
>> +} mali_c55_rsz_coef_banks[] = {
>> + {
>> + .top = 1000,
>> + .bottom = 770,
>> + }, {
>> + .top = 769,
>> + .bottom = 600,
>> + }, {
>> + .top = 599,
>> + .bottom = 460,
>> + }, {
>> + .top = 459,
>> + .bottom = 354,
>> + }, {
>> + .top = 353,
>> + .bottom = 273,
>> + }, {
>> + .top = 272,
>> + .bottom = 210,
>> + }, {
>> + .top = 209,
>> + .bottom = 162,
>> + }, {
>> + .top = 161,
>> + .bottom = 125,
>> + },
>> +};
>> +
>> +/*
>> + * Select the right filter coefficients bank based on the scaler input and the
>> + * scaler output sizes ratio, set by the v4l2 crop and scale selection
>> + * rectangles respectively.
>> + */
>> +static unsigned int mali_c55_rsz_calculate_bank(struct mali_c55 *mali_c55,
>> + unsigned int rsz_in,
>> + unsigned int rsz_out)
>> +{
>> + unsigned int rsz_ratio = (rsz_out * 1000U) / rsz_in;
>> +
>> + for (unsigned int i = 0; i < ARRAY_SIZE(mali_c55_rsz_coef_banks); i++) {
>> + if (rsz_ratio >= mali_c55_rsz_coef_banks[i].bottom &&
>> + rsz_ratio <= mali_c55_rsz_coef_banks[i].top)
>> + return i;
>> + }
>> +
>> + /*
>> + * We shouldn't ever get here, in theory. As we have no good choices
>> + * simply warn the user and use the first bank of coefficients.
>> + */
>> + dev_warn(mali_c55->dev, "scaling factor outside defined bounds\n");
>> + return 0;
>> +}
>> +
>> +static const u32 rsz_non_bypass_src_fmts[] = {
>> + MEDIA_BUS_FMT_RGB121212_1X36,
>> + MEDIA_BUS_FMT_YUV10_1X30
>> +};
>> +
>> +static void mali_c55_resizer_program_coefficients(struct mali_c55_resizer *rsz)
>> +{
>> + struct mali_c55 *mali_c55 = rsz->mali_c55;
>> + unsigned int haddr = rsz->id == MALI_C55_RSZ_FR ?
>> + MALI_C55_REG_FR_SCALER_HFILT :
>> + MALI_C55_REG_DS_SCALER_HFILT;
>> + unsigned int vaddr = rsz->id == MALI_C55_RSZ_FR ?
>> + MALI_C55_REG_FR_SCALER_VFILT :
>> + MALI_C55_REG_DS_SCALER_VFILT;
>> +
>> + for (unsigned int i = 0; i < MALI_C55_RSZ_COEFS_BANKS; i++) {
>> + for (unsigned int j = 0; j < MALI_C55_RSZ_COEFS_ENTRIES; j++) {
>> + mali_c55_write(mali_c55, haddr,
>> + mali_c55_rsz_filter_coeffs_h[i][j]);
>> + mali_c55_write(mali_c55, vaddr,
>> + mali_c55_rsz_filter_coeffs_v[i][j]);
>> +
>> + haddr += sizeof(u32);
>> + vaddr += sizeof(u32);
>> + }
>> + }
>> +}
>> +
>> +static int mali_c55_rsz_program_crop(struct mali_c55_resizer *rsz,
>> + struct v4l2_subdev_state *state)
> const
>
>> +{
>> + const struct v4l2_mbus_framefmt *fmt;
>> + const struct v4l2_rect *crop;
>> +
>> + /* Verify if crop should be enabled. */
>> + fmt = v4l2_subdev_state_get_format(state, MALI_C55_RSZ_SINK_PAD, 0);
>> + crop = v4l2_subdev_state_get_crop(state, MALI_C55_RSZ_SINK_PAD, 0);
>> +
>> + if (fmt->width == crop->width && fmt->height == crop->height)
>> + return MALI_C55_BYPASS_CROP;
>> +
>> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_CROP_X_START,
>> + crop->left);
>> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_CROP_Y_START,
>> + crop->top);
>> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_CROP_X_SIZE,
>> + crop->width);
>> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_CROP_Y_SIZE,
>> + crop->height);
>> +
>> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_CROP_EN,
>> + MALI_C55_CROP_ENABLE);
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_rsz_program_resizer(struct mali_c55_resizer *rsz,
>> + struct v4l2_subdev_state *state)
> const
>
>> +{
>> + struct mali_c55 *mali_c55 = rsz->mali_c55;
>> + const struct v4l2_rect *crop, *scale;
>> + unsigned int h_bank, v_bank;
>> + u64 h_scale, v_scale;
>> +
>> + /* Verify if scaling should be enabled. */
>> + crop = v4l2_subdev_state_get_crop(state, MALI_C55_RSZ_SINK_PAD, 0);
>> + scale = v4l2_subdev_state_get_compose(state, MALI_C55_RSZ_SINK_PAD, 0);
>> +
>> + if (crop->width == scale->width && crop->height == scale->height)
>> + return MALI_C55_BYPASS_SCALER;
>> +
>> + /* Program the scaler coefficients if the scaler is in use. */
>> + mali_c55_resizer_program_coefficients(rsz);
>> +
>> + /* Program the V/H scaling factor in Q4.20 format. */
>> + h_scale = crop->width * MALI_C55_RSZ_SCALER_FACTOR;
>> + v_scale = crop->height * MALI_C55_RSZ_SCALER_FACTOR;
>> +
>> + do_div(h_scale, scale->width);
>> + do_div(v_scale, scale->height);
>> +
>> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_IN_WIDTH,
>> + crop->width);
>> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_IN_HEIGHT,
>> + crop->height);
>> +
>> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_OUT_WIDTH,
>> + scale->width);
>> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_OUT_HEIGHT,
>> + scale->height);
>> +
>> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_HFILT_TINC,
>> + h_scale);
>> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_VFILT_TINC,
>> + v_scale);
>> +
>> + /* Select the scaler coefficients bank to use. */
>> + h_bank = mali_c55_rsz_calculate_bank(mali_c55, crop->width,
>> + scale->width);
>> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_HFILT_COEF,
>> + h_bank);
>> +
>> + v_bank = mali_c55_rsz_calculate_bank(mali_c55, crop->height,
>> + scale->height);
>> + mali_c55_cap_dev_write(rsz->cap_dev, MALI_C55_REG_SCALER_VFILT_COEF,
>> + v_bank);
>> +
>> + return 0;
>> +}
>> +
>> +static void mali_c55_rsz_program(struct mali_c55_resizer *rsz,
>> + struct v4l2_subdev_state *state)
>> +{
>> + struct mali_c55 *mali_c55 = rsz->mali_c55;
>> + u32 bypass = 0;
>> +
>> + /* Verify if cropping and scaling should be enabled. */
>> + bypass |= mali_c55_rsz_program_crop(rsz, state);
>> + bypass |= mali_c55_rsz_program_resizer(rsz, state);
>> +
>> + mali_c55_ctx_update_bits(mali_c55, rsz->id == MALI_C55_RSZ_FR ?
>> + MALI_C55_REG_FR_BYPASS : MALI_C55_REG_DS_BYPASS,
>> + MALI_C55_BYPASS_CROP | MALI_C55_BYPASS_SCALER,
>> + bypass);
>> +}
>> +
>> +/*
>> + * Inspect the routing table to know which of the two (mutually exclusive)
>> + * routes is enabled and return the sink pad id of the active route.
>> + */
>> +static unsigned int mali_c55_rsz_get_active_sink(struct v4l2_subdev_state *state)
>> +{
>> + struct v4l2_subdev_krouting *routing = &state->routing;
>> + struct v4l2_subdev_route *route;
>> +
>> + /* A single route is enabled at a time. */
>> + for_each_active_route(routing, route)
>> + return route->sink_pad;
>> +
>> + return MALI_C55_RSZ_SINK_PAD;
>> +}
>> +
>> +/*
>> + * When operating in bypass mode, the ISP takes input in a 20-bit format, but
>> + * can only output 16-bit RAW bayer data (with the 4 least significant bits from
>> + * the input being lost). Return the 16-bit version of the 20-bit input formats.
>> + */
>> +static u32 mali_c55_rsz_shift_mbus_code(u32 mbus_code)
>> +{
>> + switch (mbus_code) {
>> + case MEDIA_BUS_FMT_SBGGR20_1X20:
>> + return MEDIA_BUS_FMT_SBGGR16_1X16;
>> + case MEDIA_BUS_FMT_SGBRG20_1X20:
>> + return MEDIA_BUS_FMT_SGBRG16_1X16;
>> + case MEDIA_BUS_FMT_SGRBG20_1X20:
>> + return MEDIA_BUS_FMT_SGRBG16_1X16;
>> + case MEDIA_BUS_FMT_SRGGB20_1X20:
>> + return MEDIA_BUS_FMT_SRGGB16_1X16;
>> + }
> Would it make sense to add the shifted code to mali_c55_isp_fmt ?
>
>> +
>> + return 0;
>> +}
>> +
>> +static int __mali_c55_rsz_set_routing(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + const struct v4l2_subdev_krouting *routing)
>> +{
>> + struct mali_c55_resizer *rsz = sd_to_mali_c55_rsz(sd);
>> + unsigned int active_sink = UINT_MAX;
>> + struct v4l2_mbus_framefmt *src_fmt;
>> + struct v4l2_rect *crop, *compose;
>> + struct v4l2_subdev_route *route;
>> + unsigned int active_routes = 0;
>> + struct v4l2_mbus_framefmt *fmt;
>> + int ret;
>> +
>> + ret = v4l2_subdev_routing_validate(sd, routing, 0);
>> + if (ret)
>> + return ret;
>> +
>> + /* Only a single route can be enabled at a time. */
>> + for_each_active_route(routing, route) {
>> + if (++active_routes > 1) {
>> + dev_dbg(rsz->mali_c55->dev,
>> + "Only one route can be active");
>> + return -EINVAL;
>> + }
>> +
>> + active_sink = route->sink_pad;
>> + }
>> + if (active_sink == UINT_MAX) {
>> + dev_dbg(rsz->mali_c55->dev, "One route has to be active");
>> + return -EINVAL;
>> + }
>> +
>> + ret = v4l2_subdev_set_routing(sd, state, routing);
>> + if (ret) {
>> + dev_dbg(rsz->mali_c55->dev, "Failed to set routing\n");
>> + return ret;
>> + }
>> +
>> + fmt = v4l2_subdev_state_get_format(state, active_sink, 0);
>> + crop = v4l2_subdev_state_get_crop(state, active_sink, 0);
>> + compose = v4l2_subdev_state_get_compose(state, active_sink, 0);
> These two lines, as well as the crop and compose variables, can move to
> the first branch of the if() below.
>
>> +
>> + fmt->width = MALI_C55_DEFAULT_WIDTH;
>> + fmt->height = MALI_C55_DEFAULT_HEIGHT;
>> + fmt->colorspace = V4L2_COLORSPACE_SRGB;
> There are tree other colorspace-related fields.
>
>> + fmt->field = V4L2_FIELD_NONE;
>> +
>> + if (active_sink == MALI_C55_RSZ_SINK_PAD) {
>> + fmt->code = MEDIA_BUS_FMT_RGB121212_1X36;
>> +
>> + crop->left = 0;
>> + crop->top = 0;
>> + crop->width = MALI_C55_DEFAULT_WIDTH;
>> + crop->height = MALI_C55_DEFAULT_HEIGHT;
>> +
>> + *compose = *crop;
>> + } else {
>> + fmt->code = MEDIA_BUS_FMT_SRGGB20_1X20;
>> + }
>> +
>> + /* Propagate the format to the source pad */
>> + src_fmt = v4l2_subdev_state_get_format(state, MALI_C55_RSZ_SOURCE_PAD,
>> + 0);
>> + *src_fmt = *fmt;
>> +
>> + /* In the event this is the bypass pad the mbus code needs correcting */
>> + if (active_sink == MALI_C55_RSZ_SINK_BYPASS_PAD)
>> + src_fmt->code = mali_c55_rsz_shift_mbus_code(src_fmt->code);
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_rsz_enum_mbus_code(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_mbus_code_enum *code)
>> +{
>> + struct v4l2_mbus_framefmt *sink_fmt;
>> + const struct mali_c55_isp_fmt *fmt;
>> + u32 sink_pad;
>> +
>> + switch (code->pad) {
>> + case MALI_C55_RSZ_SINK_PAD:
>> + if (code->index)
>> + return -EINVAL;
>> +
>> + code->code = MEDIA_BUS_FMT_RGB121212_1X36;
>> +
>> + return 0;
>> + case MALI_C55_RSZ_SOURCE_PAD:
>> + sink_pad = mali_c55_rsz_get_active_sink(state);
>> + sink_fmt = v4l2_subdev_state_get_format(state, sink_pad, 0);
>> +
>> + /*
>> + * If the active route is from the Bypass sink pad, then the
>> + * source pad is a simple passthrough of the sink format,
>> + * downshifted to 16-bits.
>> + */
>> +
>> + if (sink_pad == MALI_C55_RSZ_SINK_BYPASS_PAD) {
>> + if (code->index)
>> + return -EINVAL;
>> +
>> + code->code = mali_c55_rsz_shift_mbus_code(sink_fmt->code);
>> + if (!code->code)
>> + return -EINVAL;
>> +
>> + return 0;
>> + }
>> +
>> + /*
>> + * If the active route is from the non-bypass sink then we can
>> + * select either RGB or conversion to YUV.
>> + */
>> +
>> + if (code->index >= ARRAY_SIZE(rsz_non_bypass_src_fmts))
>> + return -EINVAL;
>> +
>> + code->code = rsz_non_bypass_src_fmts[code->index];
>> +
>> + return 0;
>> + case MALI_C55_RSZ_SINK_BYPASS_PAD:
>> + fmt = mali_c55_isp_get_mbus_config_by_index(code->index);
>> + if (fmt) {
>> + code->code = fmt->code;
>> + return 0;
>> + }
>> +
>> + break;
>> + }
>> +
>> + return -EINVAL;
>> +}
>> +
>> +static int mali_c55_rsz_enum_frame_size(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_frame_size_enum *fse)
>> +{
>> + if (fse->index)
>> + return -EINVAL;
> You need to verify that fse->code is supported.
>
>> +
>> + fse->max_width = MALI_C55_MAX_WIDTH;
>> + fse->max_height = MALI_C55_MAX_HEIGHT;
>> + fse->min_width = MALI_C55_MIN_WIDTH;
>> + fse->min_height = MALI_C55_MIN_HEIGHT;
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_rsz_set_sink_fmt(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_format *format)
>> +{
>> + struct v4l2_mbus_framefmt *fmt = &format->format;
>> + unsigned int active_sink;
>> + struct v4l2_rect *rect;
>> +
>> + /*
>> + * Clamp to min/max and then reset crop and compose rectangles to the
>> + * newly applied size.
>> + */
>> + fmt->width = clamp_t(unsigned int, fmt->width, MALI_C55_MIN_WIDTH,
>> + MALI_C55_MAX_WIDTH);
>> + fmt->height = clamp_t(unsigned int, fmt->height, MALI_C55_MIN_HEIGHT,
>> + MALI_C55_MAX_HEIGHT);
> You're accepting all values for fmt->field and all the fmt
> colorspace-related fields. Is that on purpose ?
It is not; I should have changed this in line with the others, thanks.
> You've updated some of
> the set format handlers in this version to change this, by adjusting and
> copying selected fields of fmt into the format stored in the state, and
> then copying the whole state format to fmt, instead of doing it the
> other way around. That could be a better option here too.
>
>> +
>> + rect = v4l2_subdev_state_get_crop(state, format->pad);
>> + rect->left = 0;
>> + rect->top = 0;
>> + rect->width = fmt->width;
>> + rect->height = fmt->height;
>> +
>> + rect = v4l2_subdev_state_get_compose(state, format->pad);
>> + rect->left = 0;
>> + rect->top = 0;
>> + rect->width = fmt->width;
>> + rect->height = fmt->height;
> The crop and compose rectangles don't exist on the sink bypass pad.
>
>> +
>> + if (format->pad == MALI_C55_RSZ_SINK_BYPASS_PAD) {
>> + /*
>> + * Make sure the media bus code is one of the supported
>> + * ISP input media bus codes. Default it to SRGGB otherwise.
>> + */
>> + if (!mali_c55_isp_get_mbus_config_by_code(fmt->code))
>> + fmt->code = MEDIA_BUS_FMT_SRGGB20_1X20;
>> + } else {
>> + fmt->code = MEDIA_BUS_FMT_RGB121212_1X36;
>> + }
>> +
>> + *v4l2_subdev_state_get_format(state, format->pad, 0) = *fmt;
>> +
>> + /* If format->pad is routed to the source pad, propagate the format. */
>> + active_sink = mali_c55_rsz_get_active_sink(state);
>> + if (active_sink == format->pad) {
>> + /* If the bypass route is used, downshift the code to 16bpp. */
>> + if (active_sink == MALI_C55_RSZ_SINK_BYPASS_PAD)
>> + fmt->code = mali_c55_rsz_shift_mbus_code(fmt->code);
>> +
>> + *v4l2_subdev_state_get_format(state,
>> + MALI_C55_RSZ_SOURCE_PAD, 0) = *fmt;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_rsz_set_source_fmt(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_format *format)
>> +{
>> + struct v4l2_mbus_framefmt *fmt = &format->format;
>> + struct v4l2_mbus_framefmt *sink_fmt;
>> + struct v4l2_rect *sink_compose;
>> + unsigned int active_sink;
>> +
>> + active_sink = mali_c55_rsz_get_active_sink(state);
>> + sink_fmt = v4l2_subdev_state_get_format(state, active_sink, 0);
>> + sink_compose = v4l2_subdev_state_get_compose(state, active_sink, 0);
>> +
>> + /*
>> + * The source pad format sizes come directly from the active sink pad
>> + * compose rectangle.
>> + */
>> + fmt->width = sink_compose->width;
>> + fmt->height = sink_compose->height;
>> +
>> + if (active_sink == MALI_C55_RSZ_SINK_PAD) {
>> + /*
>> + * Regular processing pipe: RGB121212 can be color-space
>> + * converted to YUV101010.
>> + */
>> + unsigned int i;
>> +
>> + for (i = 0; i < ARRAY_SIZE(rsz_non_bypass_src_fmts); i++) {
>> + if (fmt->code == rsz_non_bypass_src_fmts[i])
>> + break;
>> + }
>> +
>> + if (i == ARRAY_SIZE(rsz_non_bypass_src_fmts))
>> + fmt->code = MEDIA_BUS_FMT_RGB121212_1X36;
>> + } else {
>> + /*
>> + * Bypass pipe: the source format is the same as the bypass
>> + * sink pad downshifted to 16bpp.
>> + */
>> + fmt->code = mali_c55_rsz_shift_mbus_code(sink_fmt->code);
>> + }
>> +
>> + *v4l2_subdev_state_get_format(state, MALI_C55_RSZ_SOURCE_PAD) = *fmt;
> Do it the other way around here too. Set selected fields in the state,
> then copy the whole format to format->format. I proposed a possible
> implementation in the review of the previous version.
>
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_rsz_set_fmt(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_format *format)
>> +{
>> + /*
>> + * On sink pads fmt is either fixed for the 'regular' processing
>> + * pad or a RAW format or 20-bit wide RGB/YUV format for the FR bypass
>> + * pad.
>> + *
>> + * On source pad sizes are the result of crop+compose on the sink
>> + * pad sizes, while the format depends on the active route.
>> + */
>> +
>> + if (format->pad == MALI_C55_RSZ_SINK_PAD ||
>> + format->pad == MALI_C55_RSZ_SINK_BYPASS_PAD)
>> + return mali_c55_rsz_set_sink_fmt(sd, state, format);
>> +
>> + return mali_c55_rsz_set_source_fmt(sd, state, format);
>> +}
>> +
>> +static int mali_c55_rsz_get_selection(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_selection *sel)
>> +{
>> + if (sel->pad != MALI_C55_RSZ_SINK_PAD)
>> + return -EINVAL;
>> +
>> + if (sel->target != V4L2_SEL_TGT_CROP &&
>> + sel->target != V4L2_SEL_TGT_COMPOSE)
>> + return -EINVAL;
>> +
>> + sel->r = sel->target == V4L2_SEL_TGT_CROP
>> + ? *v4l2_subdev_state_get_crop(state, MALI_C55_RSZ_SINK_PAD)
>> + : *v4l2_subdev_state_get_compose(state, MALI_C55_RSZ_SINK_PAD);
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_rsz_set_selection(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_selection *sel)
>> +{
>> + struct mali_c55_resizer *rsz = sd_to_mali_c55_rsz(sd);
>> + struct v4l2_mbus_framefmt *sink_fmt;
>> + struct v4l2_rect *crop, *compose;
>> +
>> + if (sel->pad != MALI_C55_RSZ_SINK_PAD)
>> + return -EINVAL;
>> +
>> + if (sel->target != V4L2_SEL_TGT_CROP &&
>> + sel->target != V4L2_SEL_TGT_COMPOSE)
>> + return -EINVAL;
>> +
>> + sink_fmt = v4l2_subdev_state_get_format(state, MALI_C55_RSZ_SINK_PAD);
>> + crop = v4l2_subdev_state_get_crop(state, MALI_C55_RSZ_SINK_PAD);
>> + compose = v4l2_subdev_state_get_compose(state, MALI_C55_RSZ_SINK_PAD);
>> +
>> + /* During streaming, it is allowed to only change the crop rectangle. */
>> + if (rsz->streaming && sel->target != V4L2_SEL_TGT_CROP)
>> + return -EBUSY;
>> +
>> + /*
>> + * Update the desired target and then clamp the crop rectangle to the
>> + * sink format sizes and the compose size to the crop rectangle.
>> + */
>> + if (sel->target == V4L2_SEL_TGT_CROP)
>> + *crop = sel->r;
>> + else
>> + *compose = sel->r;
>> +
>> + crop->left = clamp_t(unsigned int, crop->left, 0, sink_fmt->width);
> Double space.
>
>> + crop->top = clamp_t(unsigned int, crop->top, 0, sink_fmt->height);
>> + crop->width = clamp_t(unsigned int, crop->width, MALI_C55_MIN_WIDTH,
>> + sink_fmt->width - crop->left);
>> + crop->height = clamp_t(unsigned int, crop->height, MALI_C55_MIN_HEIGHT,
>> + sink_fmt->height - crop->top);
> You don't need to do this when setting the compose rectangle, as crop
> and sink_fmt haven't changed.
>
> I think this function could be rewritten to make it easier to read and
> understand.
>
>> +
>> + if (rsz->streaming) {
>> + /*
>> + * It is not valid to apply at runtime a crop rectangle smaller
>> + * than the compose one, as it requires re-programming the
>> + * scaler output sizes and the output buffer sizes.
>> + *
>> + * Adjust the crop rectangle to be at least as large as the
>> + * compose one if we're streaming and apply it immediately.
>> + */
>> + if (crop->width < compose->width)
>> + crop->width = compose->width;
>> + if (crop->height < compose->height)
>> + crop->height = compose->height;
>> +
>> + sel->r = *crop;
>> +
>> + mali_c55_rsz_program(rsz, state);
>> +
>> + return 0;
>> + }
>> +
>> + compose->left = 0;
>> + compose->top = 0;
>> + compose->width = clamp_t(unsigned int, compose->width, crop->width / 8,
>> + crop->width);
>> + compose->height = clamp_t(unsigned int, compose->height,
>> + crop->height / 8, crop->height);
>> +
>> + sel->r = sel->target == V4L2_SEL_TGT_CROP ? *crop : *compose;
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_rsz_set_routing(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + enum v4l2_subdev_format_whence which,
>> + struct v4l2_subdev_krouting *routing)
>> +{
>> + if (which == V4L2_SUBDEV_FORMAT_ACTIVE &&
>> + media_entity_is_streaming(&sd->entity))
>> + return -EBUSY;
>> +
>> + return __mali_c55_rsz_set_routing(sd, state, routing);
>> +}
>> +
>> +static int mali_c55_rsz_enable_streams(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state, u32 pad,
>> + u64 streams_mask)
>> +{
>> + struct mali_c55_resizer *rsz = sd_to_mali_c55_rsz(sd);
>> + struct mali_c55 *mali_c55 = rsz->mali_c55;
>> + unsigned int sink_pad;
>> +
>> + sink_pad = mali_c55_rsz_get_active_sink(state);
>> + if (sink_pad == MALI_C55_RSZ_SINK_BYPASS_PAD) {
>> + /* Bypass FR pipe processing if the bypass route is active. */
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_ISP_RAW_BYPASS,
>> + MALI_C55_ISP_RAW_BYPASS_FR_BYPASS_MASK,
>> + MALI_C55_ISP_RAW_BYPASS_RAW_FR_BYPASS);
>> + rsz->streaming = true;
>> + return 0;
>> + }
>> +
>> + /* Disable bypass and use regular processing. */
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_ISP_RAW_BYPASS,
>> + MALI_C55_ISP_RAW_BYPASS_FR_BYPASS_MASK, 0);
>> + mali_c55_rsz_program(rsz, state);
>> +
>> + rsz->streaming = true;
> Can rsz->streaming be replaced with v4l2_subdev_is_streaming() ?
Yes I think so.
>
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_rsz_disable_streams(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state, u32 pad,
>> + u64 streams_mask)
>> +{
>> + struct mali_c55_resizer *rsz = sd_to_mali_c55_rsz(sd);
>> +
>> + rsz->streaming = false;
>> + return 0;
>> +}
>> +
>> +static const struct v4l2_subdev_pad_ops mali_c55_resizer_pad_ops = {
>> + .enum_mbus_code = mali_c55_rsz_enum_mbus_code,
>> + .enum_frame_size = mali_c55_rsz_enum_frame_size,
>> + .get_fmt = v4l2_subdev_get_fmt,
>> + .set_fmt = mali_c55_rsz_set_fmt,
>> + .get_selection = mali_c55_rsz_get_selection,
>> + .set_selection = mali_c55_rsz_set_selection,
>> + .set_routing = mali_c55_rsz_set_routing,
>> + .enable_streams = mali_c55_rsz_enable_streams,
>> + .disable_streams = mali_c55_rsz_disable_streams,
>> +};
>> +
>> +static const struct v4l2_subdev_ops mali_c55_resizer_ops = {
>> + .pad = &mali_c55_resizer_pad_ops,
>> +};
>> +
>> +static int mali_c55_rsz_init_state(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state)
>> +{
>> + struct mali_c55_resizer *rsz = sd_to_mali_c55_rsz(sd);
>> + struct v4l2_subdev_route routes[2] = {
>> + {
>> + .sink_pad = MALI_C55_RSZ_SINK_PAD,
>> + .source_pad = MALI_C55_RSZ_SOURCE_PAD,
>> + .flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
>> + }, {
>> + .sink_pad = MALI_C55_RSZ_SINK_BYPASS_PAD,
>> + .source_pad = MALI_C55_RSZ_SOURCE_PAD,
>> + },
>> + };
>> + struct v4l2_subdev_krouting routing = {
>> + .num_routes = rsz->num_routes,
>> + .routes = routes,
>> + };
>> +
>> + return __mali_c55_rsz_set_routing(sd, state, &routing);
>> +}
>> +
>> +static const struct v4l2_subdev_internal_ops mali_c55_resizer_internal_ops = {
>> + .init_state = mali_c55_rsz_init_state,
>> +};
>> +
>> +static int mali_c55_register_resizer(struct mali_c55 *mali_c55,
>> + struct mali_c55_resizer *rsz)
>> +{
>> + struct v4l2_subdev *sd = &rsz->sd;
>> + unsigned int num_pads;
>> + int ret;
>> +
>> + rsz->streaming = false;
>> +
>> + v4l2_subdev_init(sd, &mali_c55_resizer_ops);
>> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_STREAMS;
>> + sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
>> + sd->internal_ops = &mali_c55_resizer_internal_ops;
>> +
>> + rsz->pads[MALI_C55_RSZ_SINK_PAD].flags = MEDIA_PAD_FL_SINK;
>> + rsz->pads[MALI_C55_RSZ_SOURCE_PAD].flags = MEDIA_PAD_FL_SOURCE;
>> +
>> + if (rsz->id == MALI_C55_RSZ_FR) {
>> + num_pads = MALI_C55_RSZ_NUM_PADS;
>> + rsz->num_routes = 2;
>> +
>> + rsz->pads[MALI_C55_RSZ_SINK_BYPASS_PAD].flags =
>> + MEDIA_PAD_FL_SINK;
>> +
>> + snprintf(sd->name, ARRAY_SIZE(sd->name), "%s resizer fr",
>> + MALI_C55_DRIVER_NAME);
> snprintf(sd->name, sizeof(sd->name), "%s resizer fr",
> MALI_C55_DRIVER_NAME);
>
>> +
>> + } else {
>> + num_pads = MALI_C55_RSZ_NUM_PADS - 1;
>> + rsz->num_routes = 1;
>> +
>> + snprintf(sd->name, ARRAY_SIZE(sd->name), "%s resizer ds",
>> + MALI_C55_DRIVER_NAME);
> Same here.
>
>> + }
>> +
>> + ret = media_entity_pads_init(&sd->entity, num_pads, rsz->pads);
>> + if (ret)
>> + return ret;
>> +
>> + ret = v4l2_subdev_init_finalize(sd);
>> + if (ret)
>> + goto err_media_cleanup;
>> +
>> + ret = v4l2_device_register_subdev(&mali_c55->v4l2_dev, sd);
>> + if (ret)
>> + goto err_subdev_cleanup;
>> +
>> + return 0;
>> +
>> +err_subdev_cleanup:
>> + v4l2_subdev_cleanup(sd);
>> +err_media_cleanup:
>> + media_entity_cleanup(&sd->entity);
>> +
>> + return ret;
>> +}
>> +
>> +static void mali_c55_unregister_resizer(struct mali_c55_resizer *rsz)
>> +{
>> + if (!rsz->mali_c55)
>> + return;
>> +
>> + v4l2_device_unregister_subdev(&rsz->sd);
>> + v4l2_subdev_cleanup(&rsz->sd);
>> + media_entity_cleanup(&rsz->sd.entity);
>> +}
>> +
>> +int mali_c55_register_resizers(struct mali_c55 *mali_c55)
>> +{
>> + unsigned int i;
>> + int ret;
>> +
>> + for (i = 0; i < MALI_C55_NUM_RSZS; ++i) {
>> + struct mali_c55_resizer *rsz = &mali_c55->resizers[i];
>> +
>> + rsz->id = i;
>> + ret = mali_c55_register_resizer(mali_c55, rsz);
>> + if (ret)
>> + goto err_cleanup;
>> +
>> + rsz->cap_dev = &mali_c55->cap_devs[i];
>> + rsz->mali_c55 = mali_c55;
> You can move the rsz->id assignment and the last two lines to
> mali_c55_register_resizer() by passing the index as an argument. This
> would allow unrolling the loop. Up to you. Even if you don't want to
> unroll the loop, you could still move the assignments to the function.
>
>> + }
>> +
>> + return 0;
>> +
>> +err_cleanup:
>> + for (; i >= 0; --i)
>> + mali_c55_unregister_resizer(&mali_c55->resizers[i]);
> Infinite loop and wrong index handling.
>
>> +
>> + return ret;
>> +}
>> +
>> +void mali_c55_unregister_resizers(struct mali_c55 *mali_c55)
>> +{
>> + for (unsigned int i = 0; i < MALI_C55_NUM_RSZS; i++)
>> + mali_c55_unregister_resizer(&mali_c55->resizers[i]);
>> +}
>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-tpg.c b/drivers/media/platform/arm/mali-c55/mali-c55-tpg.c
>> new file mode 100644
>> index 000000000000..56bb617eb5a4
>> --- /dev/null
>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-tpg.c
>> @@ -0,0 +1,438 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * ARM Mali-C55 ISP Driver - Test pattern generator
>> + *
>> + * Copyright (C) 2024 Ideas on Board Oy
>> + */
>> +
>> +#include <linux/minmax.h>
>> +#include <linux/pm_runtime.h>
>> +#include <linux/string.h>
>> +
>> +#include <media/media-entity.h>
>> +#include <media/v4l2-ctrls.h>
>> +#include <media/v4l2-event.h>
>> +#include <media/v4l2-subdev.h>
>> +
>> +#include "mali-c55-common.h"
>> +#include "mali-c55-registers.h"
>> +
>> +#define MALI_C55_TPG_SRC_PAD 0
>> +#define MALI_C55_TPG_FIXED_HBLANK 0x20
>> +#define MALI_C55_TPG_DEFAULT_MIN_VBLANK 66
>> +#define MALI_C55_TPG_DEFAULT_DEF_VBLANK 626
>> +#define MALI_C55_TPG_MAX_VBLANK 0xffff
>> +#define MALI_C55_TPG_PIXEL_RATE 100000000
>> +
>> +static const char * const mali_c55_tpg_test_pattern_menu[] = {
>> + "Flat field",
>> + "Horizontal gradient",
>> + "Vertical gradient",
>> + "Vertical bars",
>> + "Arbitrary rectangle",
>> + "White frame on black field"
>> +};
>> +
>> +static const u32 mali_c55_tpg_mbus_codes[] = {
>> + MEDIA_BUS_FMT_SRGGB20_1X20,
>> + MEDIA_BUS_FMT_RGB202020_1X60,
>> +};
>> +
>> +static void mali_c55_tpg_update_vblank(struct mali_c55_tpg *tpg,
>> + struct v4l2_mbus_framefmt *format)
>> +{
>> + unsigned int def_vblank;
>> + unsigned int min_vblank;
>> + unsigned int hts;
>> + int tgt_fps;
>> +
>> + hts = format->width + MALI_C55_TPG_FIXED_HBLANK;
>> +
>> + /*
>> + * The ISP has minimum vertical blanking requirements that must be
>> + * adhered to by the TPG. The minimum is a function of the Iridix blocks
>> + * clocking requirements and the width of the image and horizontal
>> + * blanking, but if we assume the worst case iVariance and sVariance
>> + * values then it boils down to the below (plus one to the numerator to
>> + * ensure the answer is rounded up).
>> + */
>> + min_vblank = 15 + (120501 / hts);
>> +
>> + /*
>> + * We need to set a sensible default vblank for whatever format height
>> + * we happen to be given from set_fmt(). This function just targets
>> + * an even multiple of 15fps. If we can't get 15fps, let's target 5fps.
>> + * If we can't get 5fps we'll take whatever the minimum vblank gives us.
>> + */
>> + tgt_fps = MALI_C55_TPG_PIXEL_RATE / hts / (format->height + min_vblank);
>> +
>> + if (tgt_fps < 5)
>> + def_vblank = min_vblank;
>> + else
>> + def_vblank = (MALI_C55_TPG_PIXEL_RATE / hts
>> + / max(rounddown(tgt_fps, 15), 5)) - format->height;
>> +
>> + def_vblank = ALIGN_DOWN(def_vblank, 2);
>> +
>> + __v4l2_ctrl_modify_range(tpg->ctrls.vblank, min_vblank,
>> + MALI_C55_TPG_MAX_VBLANK, 1, def_vblank);
>> + __v4l2_ctrl_s_ctrl(tpg->ctrls.vblank, def_vblank);
>> +}
>> +
>> +static int mali_c55_tpg_s_ctrl(struct v4l2_ctrl *ctrl)
>> +{
>> + struct mali_c55_tpg *tpg = container_of(ctrl->handler,
>> + struct mali_c55_tpg,
>> + ctrls.handler);
>> + struct mali_c55 *mali_c55 = container_of(tpg, struct mali_c55, tpg);
>> +
>> + if (!pm_runtime_get_if_in_use(mali_c55->dev))
>> + return 0;
>> +
>> + switch (ctrl->id) {
>> + case V4L2_CID_TEST_PATTERN:
>> + mali_c55_ctx_write(mali_c55,
>> + MALI_C55_REG_TEST_GEN_CH0_PATTERN_TYPE,
>> + ctrl->val);
>> + break;
>> + case V4L2_CID_VBLANK:
>> + mali_c55_update_bits(mali_c55, MALI_C55_REG_BLANKING,
>> + MALI_C55_REG_VBLANK_MASK,
>> + MALI_C55_VBLANK(ctrl->val));
>> + break;
>> + default:
>> + return -EINVAL;
> You're skipping the pm_runtime_put().
>
> default:
> ret = -EINVAL;
> break;
>
> and return ret balow.
>
>> + }
>> +
>> + pm_runtime_put(mali_c55->dev);
> pm_runtime_put_autosuspend(mali_c55->dev);
> pm_runtime_mark_last_busy(...);
>
>> +
>> + return 0;
>> +}
>> +
>> +static const struct v4l2_ctrl_ops mali_c55_tpg_ctrl_ops = {
>> + .s_ctrl = &mali_c55_tpg_s_ctrl,
>> +};
>> +
>> +static void mali_c55_tpg_configure(struct mali_c55_tpg *tpg)
>> +{
>> + struct v4l2_subdev *sd = &tpg->sd;
>> + struct v4l2_subdev_state *state;
>> + struct v4l2_mbus_framefmt *fmt;
>> + u32 test_pattern_format;
>> +
>> + /*
>> + * hblank needs setting, but is a read-only control and thus won't be
>> + * called during __v4l2_ctrl_handler_setup(). Do it here instead.
>> + */
>> + mali_c55_update_bits(tpg->mali_c55, MALI_C55_REG_BLANKING,
>> + MALI_C55_REG_HBLANK_MASK,
>> + MALI_C55_TPG_FIXED_HBLANK);
>> + mali_c55_update_bits(tpg->mali_c55, MALI_C55_REG_GEN_VIDEO,
>> + MALI_C55_REG_GEN_VIDEO_MULTI_MASK,
>> + MALI_C55_REG_GEN_VIDEO_MULTI_MASK);
>> +
>> + state = v4l2_subdev_get_locked_active_state(sd);
> You can pass the state from the caller.
>
>> + fmt = v4l2_subdev_state_get_format(state, MALI_C55_TPG_SRC_PAD);
>> +
>> + test_pattern_format = fmt->code == MEDIA_BUS_FMT_RGB202020_1X60 ?
>> + 0x01 : 0x0;
> test_pattern_format = fmt->code == MEDIA_BUS_FMT_RGB202020_1X60 ?
> 0x01 : 0x0;
>
> or
>
> test_pattern_format = fmt->code == MEDIA_BUS_FMT_RGB202020_1X60
> ? 0x01 : 0x0;
>
> I prefer the latter, but I know that's not a widely approved choice.
>
>> +
>> + mali_c55_ctx_update_bits(tpg->mali_c55, MALI_C55_REG_TPG_CH0,
>> + MALI_C55_TEST_PATTERN_RGB_MASK,
>> + MALI_C55_TEST_PATTERN_RGB(test_pattern_format));
>> +
>> + v4l2_subdev_unlock_state(state);
> And this is wrong, you're unlocking a state you haven't locked. Drop it.
>
>> +}
>> +
>> +static int mali_c55_tpg_enum_mbus_code(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_mbus_code_enum *code)
>> +{
>> + if (code->index >= ARRAY_SIZE(mali_c55_tpg_mbus_codes))
>> + return -EINVAL;
>> +
>> + code->code = mali_c55_tpg_mbus_codes[code->index];
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_tpg_enum_frame_size(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_frame_size_enum *fse)
>> +{
>> + unsigned int i;
>> +
>> + if (fse->index > 0)
>> + return -EINVAL;
>> +
>> + for (i = 0; i < ARRAY_SIZE(mali_c55_tpg_mbus_codes); i++) {
>> + if (fse->code == mali_c55_tpg_mbus_codes[i])
>> + break;
>> + }
>> +
>> + if (i == ARRAY_SIZE(mali_c55_tpg_mbus_codes))
>> + return -EINVAL;
>> +
>> + fse->min_width = MALI_C55_MIN_WIDTH;
>> + fse->max_width = MALI_C55_MAX_WIDTH;
>> + fse->min_height = MALI_C55_MIN_HEIGHT;
>> + fse->max_height = MALI_C55_MAX_HEIGHT;
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_tpg_set_fmt(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_format *format)
>> +{
>> + struct mali_c55_tpg *tpg = container_of(sd, struct mali_c55_tpg, sd);
>> + struct v4l2_mbus_framefmt *fmt;
>> + unsigned int i;
>> +
>> + fmt = v4l2_subdev_state_get_format(state, MALI_C55_TPG_SRC_PAD);
>> + fmt->code = format->format.code;
>> +
>> + for (i = 0; i < ARRAY_SIZE(mali_c55_tpg_mbus_codes); i++) {
>> + if (fmt->code == mali_c55_tpg_mbus_codes[i])
>> + break;
>> + }
>> +
>> + if (i == ARRAY_SIZE(mali_c55_tpg_mbus_codes))
>> + fmt->code = MEDIA_BUS_FMT_SRGGB20_1X20;
>> +
>> + /*
>> + * The TPG says that the test frame timing generation logic expects a
>> + * minimum framesize of 4x4 pixels, but given the rest of the ISP can't
>> + * handle anything smaller than 128x128 it seems pointless to allow a
>> + * smaller frame.
>> + */
>> + fmt->width = clamp(fmt->width, MALI_C55_MIN_WIDTH, MALI_C55_MAX_WIDTH);
>> + fmt->height = clamp(fmt->height, MALI_C55_MIN_HEIGHT,
>> + MALI_C55_MAX_HEIGHT);
>> +
>> + format->format = *fmt;
>> +
>> + if (format->which == V4L2_SUBDEV_FORMAT_TRY)
>> + return 0;
>> +
>> + mali_c55_tpg_update_vblank(tpg, fmt);
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_tpg_enable_streams(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state, u32 pad,
>> + u64 streams_mask)
>> +{
>> + struct mali_c55_tpg *tpg = container_of(sd, struct mali_c55_tpg, sd);
>> + struct mali_c55 *mali_c55 = container_of(tpg, struct mali_c55, tpg);
>> +
>> + /*
>> + * We only have a source pad and a single stream, and v4l2-core already
>> + * validated both so we don't need to do that. One might reasonably
>> + * expect the framesize to be set here given it's configurable in
>> + * .set_fmt(), but it's done in the ISP subdevice's .enable_streams()
>> + * instead, as the same register is also used to indicate the size of
>> + * the data coming from the sensor.
>> + */
>> + mali_c55_tpg_configure(tpg);
>> + __v4l2_ctrl_handler_setup(sd->ctrl_handler);
>> +
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_TPG_CH0,
>> + MALI_C55_TEST_PATTERN_ON_OFF,
>> + MALI_C55_TEST_PATTERN_ON_OFF);
>> + mali_c55_update_bits(mali_c55, MALI_C55_REG_GEN_VIDEO,
>> + MALI_C55_REG_GEN_VIDEO_ON_MASK,
>> + MALI_C55_REG_GEN_VIDEO_ON_MASK);
>> +
>> + return 0;
>> +}
>> +
>> +static int mali_c55_tpg_disable_streams(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state, u32 pad,
>> + u64 streams_mask)
>> +{
>> + struct mali_c55_tpg *tpg = container_of(sd, struct mali_c55_tpg, sd);
>> + struct mali_c55 *mali_c55 = container_of(tpg, struct mali_c55, tpg);
>> +
>> + mali_c55_ctx_update_bits(mali_c55, MALI_C55_REG_TPG_CH0,
>> + MALI_C55_TEST_PATTERN_ON_OFF, 0x00);
>> + mali_c55_update_bits(mali_c55, MALI_C55_REG_GEN_VIDEO,
>> + MALI_C55_REG_GEN_VIDEO_ON_MASK, 0x00);
>> +
>> + return 0;
>> +}
>> +
>> +static const struct v4l2_subdev_pad_ops mali_c55_tpg_pad_ops = {
>> + .enum_mbus_code = mali_c55_tpg_enum_mbus_code,
>> + .enum_frame_size = mali_c55_tpg_enum_frame_size,
>> + .get_fmt = v4l2_subdev_get_fmt,
>> + .set_fmt = mali_c55_tpg_set_fmt,
>> + .enable_streams = mali_c55_tpg_enable_streams,
>> + .disable_streams = mali_c55_tpg_disable_streams,
>> +};
>> +
>> +static const struct v4l2_subdev_core_ops mali_c55_isp_core_ops = {
>> + .subscribe_event = v4l2_ctrl_subdev_subscribe_event,
>> + .unsubscribe_event = v4l2_event_subdev_unsubscribe,
>> +};
>> +
>> +static const struct v4l2_subdev_ops mali_c55_tpg_ops = {
>> + .core = &mali_c55_isp_core_ops,
>> + .pad = &mali_c55_tpg_pad_ops,
>> +};
>> +
>> +static int mali_c55_tpg_init_state(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state)
>> +{
>> + struct v4l2_mbus_framefmt *fmt =
>> + v4l2_subdev_state_get_format(state, MALI_C55_TPG_SRC_PAD);
>> +
>> + fmt->width = MALI_C55_DEFAULT_WIDTH;
>> + fmt->height = MALI_C55_DEFAULT_HEIGHT;
>> + fmt->field = V4L2_FIELD_NONE;
>> + fmt->code = MEDIA_BUS_FMT_SRGGB20_1X20;
>> + fmt->colorspace = V4L2_COLORSPACE_RAW;
> Initialize the other three colorspace-related fields.
>
>> +
>> + return 0;
>> +}
>> +
>> +static const struct v4l2_subdev_internal_ops mali_c55_tpg_internal_ops = {
>> + .init_state = mali_c55_tpg_init_state,
>> +};
>> +
>> +static int mali_c55_tpg_init_controls(struct mali_c55 *mali_c55)
>> +{
>> + struct mali_c55_tpg_ctrls *ctrls = &mali_c55->tpg.ctrls;
>> + struct v4l2_ctrl *pixel_rate;
>> + struct v4l2_ctrl *hblank;
>> + int ret;
>> +
>> + ret = v4l2_ctrl_handler_init(&ctrls->handler, 4);
>> + if (ret)
>> + return ret;
>> +
>> + v4l2_ctrl_new_std_menu_items(&ctrls->handler,
>> + &mali_c55_tpg_ctrl_ops, V4L2_CID_TEST_PATTERN,
>> + ARRAY_SIZE(mali_c55_tpg_test_pattern_menu) - 1,
>> + 0, 3, mali_c55_tpg_test_pattern_menu);
>> +
>> + /*
>> + * We fix hblank at the minimum allowed value and control framerate
>> + * solely through the vblank control.
>> + */
>> + hblank = v4l2_ctrl_new_std(&ctrls->handler, &mali_c55_tpg_ctrl_ops,
>> + V4L2_CID_HBLANK, MALI_C55_TPG_FIXED_HBLANK,
>> + MALI_C55_TPG_FIXED_HBLANK, 1,
>> + MALI_C55_TPG_FIXED_HBLANK);
>> + if (hblank)
>> + hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
>> +
>> + ctrls->vblank = v4l2_ctrl_new_std(&ctrls->handler,
>> + &mali_c55_tpg_ctrl_ops,
>> + V4L2_CID_VBLANK,
>> + MALI_C55_TPG_DEFAULT_MIN_VBLANK,
>> + MALI_C55_TPG_MAX_VBLANK, 1,
>> + MALI_C55_TPG_DEFAULT_DEF_VBLANK);
>> +
>> + pixel_rate = v4l2_ctrl_new_std(&ctrls->handler, &mali_c55_tpg_ctrl_ops,
>> + V4L2_CID_PIXEL_RATE,
>> + MALI_C55_TPG_PIXEL_RATE,
>> + MALI_C55_TPG_PIXEL_RATE, 1,
>> + MALI_C55_TPG_PIXEL_RATE);
>> + if (pixel_rate)
>> + pixel_rate->flags |= V4L2_CTRL_FLAG_READ_ONLY;
>> +
>> + if (ctrls->handler.error) {
>> + dev_err(mali_c55->dev, "Error during v4l2 controls init\n");
>> + ret = ctrls->handler.error;
>> + goto err_free_handler;
> Replace the goto with
>
> v4l2_ctrl_handler_free(&ctrls->handler);
> return ret;
>
> as the label is used here only.
>
>> + }
>> +
>> + mali_c55->tpg.sd.ctrl_handler = &ctrls->handler;
>> + mali_c55->tpg.sd.state_lock = ctrls->handler.lock;
> I would move the last line to mali_c55_register_tpg(). Up to you.
>
>> +
>> + return 0;
>> +
>> +err_free_handler:
>> + v4l2_ctrl_handler_free(&ctrls->handler);
>> +
>> + return ret;
>> +}
>> +
>> +int mali_c55_register_tpg(struct mali_c55 *mali_c55)
>> +{
>> + struct mali_c55_tpg *tpg = &mali_c55->tpg;
>> + struct v4l2_subdev *sd = &tpg->sd;
>> + struct media_pad *pad = &tpg->pad;
>> + int ret;
>> +
>> + v4l2_subdev_init(sd, &mali_c55_tpg_ops);
>> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
>> + sd->entity.function = MEDIA_ENT_F_CAM_SENSOR;
>> + sd->internal_ops = &mali_c55_tpg_internal_ops;
>> + strscpy(sd->name, MALI_C55_DRIVER_NAME " tpg", sizeof(sd->name));
>> +
>> + pad->flags = MEDIA_PAD_FL_SOURCE;
>> + ret = media_entity_pads_init(&sd->entity, 1, pad);
>> + if (ret) {
>> + dev_err(mali_c55->dev,
>> + "Failed to initialize media entity pads\n");
>> + return ret;
>> + }
>> +
>> + ret = mali_c55_tpg_init_controls(mali_c55);
>> + if (ret) {
>> + dev_err(mali_c55->dev,
>> + "Error initialising controls\n");
>> + goto err_cleanup_media_entity;
>> + }
>> +
>> + ret = v4l2_subdev_init_finalize(sd);
>> + if (ret)
>> + goto err_free_ctrl_handler;
>> +
>> + ret = v4l2_device_register_subdev(&mali_c55->v4l2_dev, sd);
>> + if (ret) {
>> + dev_err(mali_c55->dev, "Failed to register tpg subdev\n");
>> + goto err_cleanup_subdev;
>> + }
>> +
>> + /*
>> + * By default the colour settings lead to a very dim image that is
>> + * nearly indistinguishable from black on some monitor settings. Ramp
>> + * them up a bit so the image is brighter.
>> + */
>> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_TPG_R_BACKGROUND,
>> + MALI_C55_TPG_BACKGROUND_MAX);
>> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_TPG_G_BACKGROUND,
>> + MALI_C55_TPG_BACKGROUND_MAX);
>> + mali_c55_ctx_write(mali_c55, MALI_C55_REG_TPG_B_BACKGROUND,
>> + MALI_C55_TPG_BACKGROUND_MAX);
>> +
>> + tpg->mali_c55 = mali_c55;
>> +
>> + return 0;
>> +
>> +err_cleanup_subdev:
>> + v4l2_subdev_cleanup(sd);
>> +err_free_ctrl_handler:
>> + v4l2_ctrl_handler_free(&tpg->ctrls.handler);
>> +err_cleanup_media_entity:
>> + media_entity_cleanup(&sd->entity);
>> +
>> + return ret;
>> +}
>> +
>> +void mali_c55_unregister_tpg(struct mali_c55 *mali_c55)
>> +{
>> + struct mali_c55_tpg *tpg = &mali_c55->tpg;
>> +
>> + if (!tpg->mali_c55)
>> + return;
>> +
>> + v4l2_device_unregister_subdev(&tpg->sd);
>> + v4l2_ctrl_handler_free(&tpg->ctrls.handler);
>> + v4l2_subdev_cleanup(&tpg->sd);
>> + media_entity_cleanup(&tpg->sd.entity);
>> +}
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v6 12/18] media: platform: Add mali-c55 3a stats devnode
2024-08-13 13:13 ` Dan Scally
@ 2024-08-25 20:20 ` Laurent Pinchart
0 siblings, 0 replies; 41+ messages in thread
From: Laurent Pinchart @ 2024-08-25 20:20 UTC (permalink / raw)
To: Dan Scally
Cc: linux-media, devicetree, linux-arm-kernel, jacopo.mondi,
nayden.kanchev, robh+dt, mchehab, krzysztof.kozlowski+dt,
conor+dt, jerome.forissier, kieran.bingham, sakari.ailus
Hi Dan,
On Tue, Aug 13, 2024 at 02:13:49PM +0100, Daniel Scally wrote:
> On 22/07/2024 23:55, Laurent Pinchart wrote:
> > On Mon, Jul 22, 2024 at 05:51:00PM +0300, Laurent Pinchart wrote:
> >> On Tue, Jul 09, 2024 at 02:29:00PM +0100, Daniel Scally wrote:
> >>> Add a new code file to govern the 3a statistics capture node.
> >>>
> >>> Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
> >>> Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> >>> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> >>> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
> >>> ---
> >>> Changes in v6:
> >>>
> >>> - Fixed mising includes
> >>> - Minor renames and formatting
> >>> - Reworked mali_c55_stats_metering_complete() so it could only return
> >>> buffers when both halves of the DMA read were done
> >>> - Terminate dma transfers on streamoff
> >>>
> >>> Changes in v5:
> >>>
> >>> - New patch
> >>>
> >>> drivers/media/platform/arm/mali-c55/Makefile | 1 +
> >>> .../platform/arm/mali-c55/mali-c55-common.h | 29 ++
> >>> .../platform/arm/mali-c55/mali-c55-core.c | 15 +
> >>> .../platform/arm/mali-c55/mali-c55-isp.c | 11 +
> >>> .../arm/mali-c55/mali-c55-registers.h | 3 +
> >>> .../platform/arm/mali-c55/mali-c55-stats.c | 373 ++++++++++++++++++
> >>> 6 files changed, 432 insertions(+)
> >>> create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-stats.c
> >>>
> >>> diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile
> >>> index 9178ac35e50e..b5a22d414479 100644
> >>> --- a/drivers/media/platform/arm/mali-c55/Makefile
> >>> +++ b/drivers/media/platform/arm/mali-c55/Makefile
> >>> @@ -4,6 +4,7 @@ mali-c55-y := mali-c55-capture.o \
> >>> mali-c55-core.o \
> >>> mali-c55-isp.o \
> >>> mali-c55-resizer.o \
> >>> + mali-c55-stats.o \
> >>> mali-c55-tpg.o
> >>>
> >>> obj-$(CONFIG_VIDEO_MALI_C55) += mali-c55.o
> >>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> >>> index f7764a938e9f..136c785c68ba 100644
> >>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> >>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-common.h
> >>> @@ -49,6 +49,7 @@ enum mali_c55_isp_pads {
> >>> MALI_C55_ISP_PAD_SINK_VIDEO,
> >>> MALI_C55_ISP_PAD_SOURCE_VIDEO,
> >>> MALI_C55_ISP_PAD_SOURCE_BYPASS,
> >>> + MALI_C55_ISP_PAD_SOURCE_STATS,
> >>> MALI_C55_ISP_NUM_PADS,
> >>> };
> >>>
> >>> @@ -160,6 +161,29 @@ struct mali_c55_cap_dev {
> >>> bool streaming;
> >>> };
> >>>
> >>> +struct mali_c55_stats_buf {
> >>> + struct vb2_v4l2_buffer vb;
> >>> + unsigned int segments_remaining;
> >>> + struct list_head queue;
> >>> + bool failed;
> >>> +};
> >>> +
> >>> +struct mali_c55_stats {
> >>> + struct mali_c55 *mali_c55;
> >>> + struct video_device vdev;
> >>> + struct dma_chan *channel;
> >>> + struct vb2_queue queue;
> >>> + struct media_pad pad;
> >>> + /* Mutex to provide to vb2 */
> >>> + struct mutex lock;
> >>> +
> >>> + struct {
> >>> + /* Spinlock to guard buffer queue */
> >>> + spinlock_t lock;
> >>> + struct list_head queue;
> >>> + } buffers;
> >>> +};
> >>> +
> >>> enum mali_c55_config_spaces {
> >>> MALI_C55_CONFIG_PING,
> >>> MALI_C55_CONFIG_PONG,
> >>> @@ -202,6 +226,7 @@ struct mali_c55 {
> >>> struct mali_c55_isp isp;
> >>> struct mali_c55_resizer resizers[MALI_C55_NUM_RSZS];
> >>> struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS];
> >>> + struct mali_c55_stats stats;
> >>>
> >>> struct mali_c55_context context;
> >>> enum mali_c55_config_spaces next_config;
> >>> @@ -229,6 +254,8 @@ int mali_c55_register_resizers(struct mali_c55 *mali_c55);
> >>> void mali_c55_unregister_resizers(struct mali_c55 *mali_c55);
> >>> int mali_c55_register_capture_devs(struct mali_c55 *mali_c55);
> >>> void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55);
> >>> +int mali_c55_register_stats(struct mali_c55 *mali_c55);
> >>> +void mali_c55_unregister_stats(struct mali_c55 *mali_c55);
> >>> struct mali_c55_context *mali_c55_get_active_context(struct mali_c55 *mali_c55);
> >>> void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev,
> >>> enum mali_c55_planes plane);
> >>> @@ -243,5 +270,7 @@ const struct mali_c55_isp_fmt *
> >>> mali_c55_isp_get_mbus_config_by_code(u32 code);
> >>> const struct mali_c55_isp_fmt *
> >>> mali_c55_isp_get_mbus_config_by_index(u32 index);
> >>> +void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
> >>> + enum mali_c55_config_spaces cfg_space);
> >>>
> >>> #endif /* _MALI_C55_COMMON_H */
> >>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> >>> index dbc07d12d3a3..eedc8f450184 100644
> >>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> >>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
> >>> @@ -374,6 +374,16 @@ static int mali_c55_create_links(struct mali_c55 *mali_c55)
> >>> }
> >>> }
> >>>
> >>> + ret = media_create_pad_link(&mali_c55->isp.sd.entity,
> >>> + MALI_C55_ISP_PAD_SOURCE_STATS,
> >>> + &mali_c55->stats.vdev.entity, 0,
> >>> + MEDIA_LNK_FL_ENABLED);
> >>> + if (ret) {
> >>> + dev_err(mali_c55->dev,
> >>> + "failed to link ISP and 3a stats node\n");
> >>> + goto err_remove_links;
> >>> + }
> >>> +
> >>> return 0;
> >>>
> >>> err_remove_links:
> >>> @@ -388,6 +398,7 @@ static void mali_c55_unregister_entities(struct mali_c55 *mali_c55)
> >>> mali_c55_unregister_isp(mali_c55);
> >>> mali_c55_unregister_resizers(mali_c55);
> >>> mali_c55_unregister_capture_devs(mali_c55);
> >>> + mali_c55_unregister_stats(mali_c55);
> >>> }
> >>>
> >>> static int mali_c55_register_entities(struct mali_c55 *mali_c55)
> >>> @@ -410,6 +421,10 @@ static int mali_c55_register_entities(struct mali_c55 *mali_c55)
> >>> if (ret)
> >>> goto err_unregister_entities;
> >>>
> >>> + ret = mali_c55_register_stats(mali_c55);
> >>> + if (ret)
> >>> + goto err_unregister_entities;
> >>> +
> >>> ret = mali_c55_create_links(mali_c55);
> >>> if (ret)
> >>> goto err_unregister_entities;
> >>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> >>> index f784983a4ccc..2f450c00300a 100644
> >>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> >>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> >>> @@ -5,6 +5,8 @@
> >>> * Copyright (C) 2024 Ideas on Board Oy
> >>> */
> >>>
> >>> +#include <linux/media/arm/mali-c55-config.h>
> >>> +
> >>> #include <linux/delay.h>
> >>> #include <linux/iopoll.h>
> >>> #include <linux/property.h>
> >>> @@ -460,6 +462,14 @@ static int mali_c55_isp_init_state(struct v4l2_subdev *sd,
> >>> in_crop->width = MALI_C55_DEFAULT_WIDTH;
> >>> in_crop->height = MALI_C55_DEFAULT_HEIGHT;
> >>>
> >>> + src_fmt = v4l2_subdev_state_get_format(state,
> >>> + MALI_C55_ISP_PAD_SOURCE_STATS);
> >>> +
> >>> + src_fmt->width = sizeof(struct mali_c55_stats_buffer);
> >>> + src_fmt->height = 1;
> >> According to
> >> https://docs.kernel.org/userspace-api/media/v4l/subdev-formats.html#metadata-formats,
> >> width and height should be set to 0 for MEDIA_BUS_FMT_METADATA_FIXED. I
> >> haven't checked if v4l2-compliance expects this, we may have
> >> discrepancies between the API documentation and the existing
> >> implementations in the kernel and userspace. In any case, this should be
> >> sorted out, either by fixing the kernel code and enforcing the
> >> requirement in v4l2-compliance, or fixing the documentation.
> >>
> >>> + src_fmt->field = V4L2_FIELD_NONE;
> >>> + src_fmt->code = MEDIA_BUS_FMT_METADATA_FIXED;
> >>> +
> >>> return 0;
> >>> }
> >>>
> >>> @@ -490,6 +500,7 @@ int mali_c55_register_isp(struct mali_c55 *mali_c55)
> >>> MEDIA_PAD_FL_MUST_CONNECT;
> >>> isp->pads[MALI_C55_ISP_PAD_SOURCE_VIDEO].flags = MEDIA_PAD_FL_SOURCE;
> >>> isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE;
> >>> + isp->pads[MALI_C55_ISP_PAD_SOURCE_STATS].flags = MEDIA_PAD_FL_SOURCE;
> >>>
> >>> ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS,
> >>> isp->pads);
> >>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
> >>> index 0a391f8a043e..e72e749b81d5 100644
> >>> --- a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
> >>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h
> >>> @@ -103,6 +103,9 @@ enum mali_c55_interrupts {
> >>> #define MALI_C55_VC_START(v) ((v) & 0xffff)
> >>> #define MALI_C55_VC_SIZE(v) (((v) & 0xffff) << 16)
> >>>
> >>> +#define MALI_C55_REG_1024BIN_HIST 0x054a8
> >>> +#define MALI_C55_1024BIN_HIST_SIZE 4096
> >>> +
> >>> /* Ping/Pong Configuration Space */
> >>> #define MALI_C55_REG_BASE_ADDR 0x18e88
> >>> #define MALI_C55_REG_BYPASS_0 0x18eac
> >>> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-stats.c b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
> >>> new file mode 100644
> >>> index 000000000000..38a17fb5d052
> >>> --- /dev/null
> >>> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
> >>> @@ -0,0 +1,373 @@
> >>> +// SPDX-License-Identifier: GPL-2.0
> >>> +/*
> >>> + * ARM Mali-C55 ISP Driver - 3A Statistics capture device
> >>> + *
> >>> + * Copyright (C) 2023 Ideas on Board Oy
> >>> + */
> >>> +
> >>> +#include <linux/container_of.h>
> >>> +#include <linux/dev_printk.h>
> >>> +#include <linux/dmaengine.h>
> >>> +#include <linux/list.h>
> >>> +#include <linux/media/arm/mali-c55-config.h>
> >>> +#include <linux/mutex.h>
> >>> +#include <linux/spinlock.h>
> >>> +#include <linux/string.h>
> >>> +
> >>> +#include <media/media-entity.h>
> >>> +#include <media/v4l2-dev.h>
> >>> +#include <media/v4l2-event.h>
> >>> +#include <media/v4l2-fh.h>
> >>> +#include <media/v4l2-ioctl.h>
> >>> +#include <media/videobuf2-core.h>
> >>> +#include <media/videobuf2-dma-contig.h>
> >>> +
> >>> +#include "mali-c55-common.h"
> >>> +#include "mali-c55-registers.h"
> >>> +
> >>> +static const unsigned int metering_space_addrs[] = {
> >>> + [MALI_C55_CONFIG_PING] = 0x095ac,
> >>> + [MALI_C55_CONFIG_PONG] = 0x2156c,
> >>> +};
> >>> +
> >>> +static int mali_c55_stats_enum_fmt_meta_cap(struct file *file, void *fh,
> >>> + struct v4l2_fmtdesc *f)
> >>> +{
> >>> + if (f->index)
> >>> + return -EINVAL;
> >>> +
> >>> + f->pixelformat = V4L2_META_FMT_MALI_C55_STATS;
> >>> +
> >>> + return 0;
> >>> +}
> >>> +
> >>> +static int mali_c55_stats_g_fmt_meta_cap(struct file *file, void *fh,
> >>> + struct v4l2_format *f)
> >>> +{
> >>> + static const struct v4l2_meta_format mfmt = {
> >>> + .dataformat = V4L2_META_FMT_MALI_C55_STATS,
> >>> + .buffersize = sizeof(struct mali_c55_stats_buffer)
> >>> + };
> >>> +
> >>> + f->fmt.meta = mfmt;
> >>> +
> >>> + return 0;
> >>> +}
> >>> +
> >>> +static int mali_c55_stats_querycap(struct file *file,
> >>> + void *priv, struct v4l2_capability *cap)
> >>> +{
> >>> + strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver));
> >>> + strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card));
> >>> +
> >>> + return 0;
> >>> +}
> >>> +
> >>> +static const struct v4l2_ioctl_ops mali_c55_stats_v4l2_ioctl_ops = {
> >>> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> >>> + .vidioc_querybuf = vb2_ioctl_querybuf,
> >>> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> >>> + .vidioc_qbuf = vb2_ioctl_qbuf,
> >>> + .vidioc_expbuf = vb2_ioctl_expbuf,
> >>> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> >>> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> >>> + .vidioc_streamon = vb2_ioctl_streamon,
> >>> + .vidioc_streamoff = vb2_ioctl_streamoff,
> >>> + .vidioc_enum_fmt_meta_cap = mali_c55_stats_enum_fmt_meta_cap,
> >>> + .vidioc_g_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
> >>> + .vidioc_s_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
> >>> + .vidioc_try_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap,
> >>> + .vidioc_querycap = mali_c55_stats_querycap,
> >>> + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> >>> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> >>> +};
> >>> +
> >>> +static const struct v4l2_file_operations mali_c55_stats_v4l2_fops = {
> >>> + .owner = THIS_MODULE,
> >>> + .unlocked_ioctl = video_ioctl2,
> >>> + .open = v4l2_fh_open,
> >>> + .release = vb2_fop_release,
> >>> + .poll = vb2_fop_poll,
> >>> + .mmap = vb2_fop_mmap,
> >>> +};
> >>> +
> >>> +static int
> >>> +mali_c55_stats_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
> >>> + unsigned int *num_planes, unsigned int sizes[],
> >>> + struct device *alloc_devs[])
> >>> +{
> >>> + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
> >>> +
> >>> + if (*num_planes && *num_planes > 1)
> >>> + return -EINVAL;
> >>> +
> >>> + if (sizes[0] && sizes[0] < sizeof(struct mali_c55_stats_buffer))
> >>> + return -EINVAL;
> >>> +
> >>> + *num_planes = 1;
> >>> +
> >>> + if (!sizes[0])
> >>> + sizes[0] = sizeof(struct mali_c55_stats_buffer);
> >>> +
> >>> + if (stats->channel)
> >>> + alloc_devs[0] = stats->channel->device->dev;
> >>> +
> >>> + return 0;
> >>> +}
> >>> +
> >>> +static void mali_c55_stats_buf_queue(struct vb2_buffer *vb)
> >>> +{
> >>> + struct mali_c55_stats *stats = vb2_get_drv_priv(vb->vb2_queue);
> >>> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> >>> + struct mali_c55_stats_buf *buf = container_of(vbuf,
> >>> + struct mali_c55_stats_buf, vb);
> >>> +
> >>> + vb2_set_plane_payload(vb, 0, sizeof(struct mali_c55_stats_buffer));
> >>> + buf->segments_remaining = 2;
> >>> + buf->failed = false;
> >>> +
> >>> + spin_lock(&stats->buffers.lock);
> >>> + list_add_tail(&buf->queue, &stats->buffers.queue);
> >>> + spin_unlock(&stats->buffers.lock);
> >>> +}
> >>> +
> >>> +static int mali_c55_stats_start_streaming(struct vb2_queue *q,
> >>> + unsigned int count)
> >>> +{
> >>> + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
> >>> + struct mali_c55 *mali_c55 = stats->mali_c55;
> >>> + int ret;
> >>> +
> >>> + ret = video_device_pipeline_start(&stats->vdev,
> >>> + &stats->mali_c55->pipe);
> >>> + if (ret)
> >>> + return ret;
> >>> +
> >>> + if (mali_c55->pipe.start_count == mali_c55->pipe.required_count) {
> >>> + ret = v4l2_subdev_enable_streams(&mali_c55->isp.sd,
> >>> + MALI_C55_ISP_PAD_SOURCE_VIDEO,
> >>> + BIT(0));
> >>> + if (ret)
> >>> + goto err_stop_pipeline;
> >>> + }
> >>> +
> >>> + return 0;
> >>> +
> >>> +err_stop_pipeline:
> >>> + video_device_pipeline_stop(&stats->vdev);
> >>> +
> >>> + return ret;
> >>> +}
> >>> +
> >>> +static void mali_c55_stats_stop_streaming(struct vb2_queue *q)
> >>> +{
> >>> + struct mali_c55_stats *stats = vb2_get_drv_priv(q);
> >>> + struct mali_c55_stats_buf *buf, *tmp;
> >>> +
> >>> + dmaengine_terminate_sync(stats->channel);
> >>> +
> >>> + spin_lock(&stats->buffers.lock);
> >>> +
> >>> + list_for_each_entry_safe(buf, tmp, &stats->buffers.queue, queue) {
> >>> + list_del(&buf->queue);
> >>> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> >>> + }
> >>> +
> >>> + spin_unlock(&stats->buffers.lock);
> >>> +
> >>> + video_device_pipeline_stop(&stats->vdev);
> >
> > There's a lack of symmetry here, you call v4l2_subdev_enable_streams()
> > in mali_c55_stats_start_streaming() but you don't call
> > v4l2_subdev_disable_streams() anywhere. Is that intentional ?
>
> It's conditionally called, provided the start_count of the media pipeline has reached a value that
> matches the number of video devices...this fulfils the requirement to only start streaming when all
> of the video devices have started. The v4l2_subdev_disable_streams() call I left the responsibility
> of the image capture video devices...but perhaps it should behave similarly? Should stopping stream
> on any of the video devices cause the stream to stop on all of them?
In general I like symmetry. In general too, I think APIs should be
designed first and foremost from the point of view of the user with
clear behaviours and ease of use. With those two principles in mind,
would it be simpler for userspace if the symmetry was broken as done
here, or if the start and stop behaviours were symmetrical ? Would
symmetry preclude some important use cases ?
> >>> +}
> >>> +
> >>> +static const struct vb2_ops mali_c55_stats_vb2_ops = {
> >>> + .queue_setup = mali_c55_stats_queue_setup,
> >>> + .buf_queue = mali_c55_stats_buf_queue,
> >>> + .wait_prepare = vb2_ops_wait_prepare,
> >>> + .wait_finish = vb2_ops_wait_finish,
> >>> + .start_streaming = mali_c55_stats_start_streaming,
> >>> + .stop_streaming = mali_c55_stats_stop_streaming,
> >>> +};
> >>> +
> >>> +static void
> >>> +mali_c55_stats_metering_complete(void *param,
> >>> + const struct dmaengine_result *result)
> >>> +{
> >>> + struct mali_c55_stats_buf *buf = param;
> >>> +
> >>> + if (result->result != DMA_TRANS_NOERROR)
> >>> + buf->failed = true;
> >>> +
> >>> + if (!--buf->segments_remaining)
> >>> + vb2_buffer_done(&buf->vb.vb2_buf, buf->failed ?
> >>> + VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE);
> >>> +}
> >>> +
> >>> +static int mali_c55_stats_dma_xfer(struct mali_c55_stats *stats, dma_addr_t src,
> >>> + dma_addr_t dst,
> >>> + struct mali_c55_stats_buf *buf,
> >>> + size_t length)
> >>> +{
> >>> + struct dma_async_tx_descriptor *tx;
> >>> + dma_cookie_t cookie;
> >>> +
> >>> + tx = dmaengine_prep_dma_memcpy(stats->channel, dst, src, length, 0);
> >>> + if (!tx) {
> >>> + dev_err(stats->mali_c55->dev, "failed to prep stats DMA\n");
> >>> + return -EIO;
> >>> + }
> >>> +
> >>> + tx->callback_result = mali_c55_stats_metering_complete;
> >>> + tx->callback_param = buf;
> >>> +
> >>> + cookie = dmaengine_submit(tx);
> >>> + if (dma_submit_error(cookie)) {
> >>> + dev_err(stats->mali_c55->dev, "failed to submit stats DMA\n");
> >>> + return -EIO;
> >>> + }
> >>> +
> >>> + dma_async_issue_pending(stats->channel);
> >
> > You could possibly lower the overhead a bit by calling
> > dma_async_issue_pending() only once after submitting the two transfers.
> > It may not make any real difference though, I don't recall the details.
>
> I can test if we think it's worthwhile...I was working under the assumption it would be better to
> kick start the first transfer so it's running earlier, but I didn't test it.
>
> >>> + return 0;
> >>> +}
> >>> +
> >>> +void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
> >>> + enum mali_c55_config_spaces cfg_space)
> >>> +{
> >>> + struct mali_c55_context *ctx = mali_c55_get_active_context(mali_c55);
> >>> + struct mali_c55_stats *stats = &mali_c55->stats;
> >>> + struct mali_c55_stats_buf *buf = NULL;
> >>> + dma_addr_t src, dst;
> >>> + size_t length;
> >>> + int ret;
> >>> +
> >>> + spin_lock(&stats->buffers.lock);
> >>> + if (!list_empty(&stats->buffers.queue)) {
> >>> + buf = list_first_entry(&stats->buffers.queue,
> >>> + struct mali_c55_stats_buf, queue);
> >>> + list_del(&buf->queue);
> >>> + }
> >>> + spin_unlock(&stats->buffers.lock);
> >>> +
> >>> + if (!buf)
> >>> + return;
> >>> +
> >>> + buf->vb.sequence = mali_c55->isp.frame_sequence;
> >>> + buf->vb.vb2_buf.timestamp = ktime_get_boottime_ns();
> >>> +
> >>> + /*
> >>> + * There are in fact two noncontiguous sections of the ISP's
> >>> + * memory space that hold statistics for 3a algorithms to use: A
> >>> + * section in each config space and a global section holding
> >>> + * histograms which is double buffered and so holds data for the
> >>> + * last frame. We need to read both.
> >>> + */
> >>> + src = ctx->base + MALI_C55_REG_1024BIN_HIST;
> >>> + dst = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> >>> +
> >>> + ret = mali_c55_stats_dma_xfer(stats, src, dst, buf,
> >>> + MALI_C55_1024BIN_HIST_SIZE);
> >>> + if (ret)
> >>> + goto err_fail_buffer;
> >>> +
> >>> + src = ctx->base + metering_space_addrs[cfg_space];
> >>> + dst += MALI_C55_1024BIN_HIST_SIZE;
> >>> +
> >>> + length = sizeof(struct mali_c55_stats_buffer) - MALI_C55_1024BIN_HIST_SIZE;
> >>> + ret = mali_c55_stats_dma_xfer(stats, src, dst, buf, length);
> >>> + if (ret) {
> >>> + dmaengine_terminate_sync(stats->channel);
> >>> + goto err_fail_buffer;
> >>> + }
> >>> +
> >>> + return;
> >>> +
> >>> +err_fail_buffer:
> >>> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> >>> +}
> >>> +
> >>> +void mali_c55_unregister_stats(struct mali_c55 *mali_c55)
> >>> +{
> >>> + struct mali_c55_stats *stats = &mali_c55->stats;
> >>> +
> >>> + if (!video_is_registered(&stats->vdev))
> >>> + return;
> >>> +
> >>> + vb2_video_unregister_device(&stats->vdev);
> >>> + media_entity_cleanup(&stats->vdev.entity);
> >>> + dma_release_channel(stats->channel);
> >>> + mutex_destroy(&stats->lock);
> >>> +}
> >>> +
> >>> +int mali_c55_register_stats(struct mali_c55 *mali_c55)
> >>> +{
> >>> + struct mali_c55_stats *stats = &mali_c55->stats;
> >>> + struct video_device *vdev = &stats->vdev;
> >>> + struct vb2_queue *vb2q = &stats->queue;
> >>> + dma_cap_mask_t mask;
> >>> + int ret;
> >>> +
> >>> + mutex_init(&stats->lock);
> >>> + INIT_LIST_HEAD(&stats->buffers.queue);
> >>> +
> >>> + dma_cap_zero(mask);
> >>> + dma_cap_set(DMA_MEMCPY, mask);
> >>> +
> >>> + stats->channel = dma_request_channel(mask, 0, NULL);
> >>> + if (!stats->channel) {
> >>> + ret = -ENODEV;
> >>> + goto err_destroy_mutex;
> >>> + }
> >>> +
> >>> + stats->pad.flags = MEDIA_PAD_FL_SINK;
> >>> + ret = media_entity_pads_init(&stats->vdev.entity, 1, &stats->pad);
> >>> + if (ret)
> >>> + goto err_release_dma_channel;
> >>> +
> >>> + vb2q->type = V4L2_BUF_TYPE_META_CAPTURE;
> >>> + vb2q->io_modes = VB2_MMAP | VB2_DMABUF;
> >>> + vb2q->drv_priv = stats;
> >>> + vb2q->mem_ops = &vb2_dma_contig_memops;
> >>> + vb2q->ops = &mali_c55_stats_vb2_ops;
> >>> + vb2q->buf_struct_size = sizeof(struct mali_c55_stats_buf);
> >>> + vb2q->min_queued_buffers = 1;
> >>> + vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> >>> + vb2q->lock = &stats->lock;
> >>> + vb2q->dev = stats->channel->device->dev;
> >>> +
> >>> + ret = vb2_queue_init(vb2q);
> >>> + if (ret) {
> >>> + dev_err(mali_c55->dev, "stats vb2 queue init failed\n");
> >>> + goto err_cleanup_entity;
> >>> + }
> >>> +
> >>> + strscpy(stats->vdev.name, "mali-c55 3a stats", sizeof(stats->vdev.name));
> >>> + vdev->release = video_device_release_empty;
> >
> > Not a good idea at all, but I won't ask you to fix object lifetime
> > management in V4L2 as a prerequisite to merging this driver :-)
> >
> > With those issues addressed,
> >
> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >
> >>> + vdev->fops = &mali_c55_stats_v4l2_fops;
> >>> + vdev->ioctl_ops = &mali_c55_stats_v4l2_ioctl_ops;
> >>> + vdev->lock = &stats->lock;
> >>> + vdev->v4l2_dev = &mali_c55->v4l2_dev;
> >>> + vdev->queue = &stats->queue;
> >>> + vdev->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
> >>> + vdev->vfl_dir = VFL_DIR_RX;
> >>> + video_set_drvdata(vdev, stats);
> >>> +
> >>> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> >>> + if (ret) {
> >>> + dev_err(mali_c55->dev,
> >>> + "failed to register stats video device\n");
> >>> + goto err_release_vb2q;
> >>> + }
> >>> +
> >>> + stats->mali_c55 = mali_c55;
> >>> +
> >>> + return 0;
> >>> +
> >>> +err_release_vb2q:
> >>> + vb2_queue_release(vb2q);
> >>> +err_cleanup_entity:
> >>> + media_entity_cleanup(&stats->vdev.entity);
> >>> +err_release_dma_channel:
> >>> + dma_release_channel(stats->channel);
> >>> +err_destroy_mutex:
> >>> + mutex_destroy(&stats->lock);
> >>> +
> >>> + return ret;
> >>> +}
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 41+ messages in thread
end of thread, other threads:[~2024-08-25 20:20 UTC | newest]
Thread overview: 41+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-07-09 13:28 [PATCH v6 00/18] Add Arm Mali-C55 Image Signal Processor Driver Daniel Scally
2024-07-09 13:28 ` [PATCH v6 01/18] media: mc-entity: Record number of video devices in a pipeline Daniel Scally
2024-07-30 15:09 ` Laurent Pinchart
2024-07-30 15:18 ` Dan Scally
2024-07-09 13:28 ` [PATCH v6 02/18] media: uapi: Add MEDIA_BUS_FMT_RGB202020_1X60 format code Daniel Scally
2024-07-09 13:28 ` [PATCH v6 03/18] media: uapi: Add 20-bit bayer formats Daniel Scally
2024-07-22 22:13 ` Laurent Pinchart
2024-07-09 13:28 ` [PATCH v6 04/18] media: v4l2-common: Add RAW16 format info Daniel Scally
2024-07-09 13:28 ` [PATCH v6 05/18] media: v4l2-common: Add RAW14 " Daniel Scally
2024-07-22 22:14 ` Laurent Pinchart
2024-07-09 13:28 ` [PATCH v6 06/18] dt-bindings: media: Add bindings for ARM mali-c55 Daniel Scally
2024-07-09 13:28 ` [PATCH v6 07/18] media: mali-c55: Add Mali-C55 ISP driver Daniel Scally
2024-07-30 21:23 ` Laurent Pinchart
2024-08-13 15:20 ` Dan Scally
2024-07-09 13:28 ` [PATCH v6 08/18] media: Documentation: Add Mali-C55 ISP Documentation Daniel Scally
2024-07-30 15:15 ` Laurent Pinchart
2024-07-09 13:28 ` [PATCH v6 09/18] MAINTAINERS: Add entry for mali-c55 driver Daniel Scally
2024-07-09 13:28 ` [PATCH v6 10/18] media: Add MALI_C55_3A_STATS meta format Daniel Scally
2024-07-09 13:28 ` [PATCH v6 11/18] media: uapi: Add 3a stats buffer for mali-c55 Daniel Scally
2024-07-09 13:29 ` [PATCH v6 12/18] media: platform: Add mali-c55 3a stats devnode Daniel Scally
2024-07-22 14:50 ` Laurent Pinchart
2024-07-22 22:55 ` Laurent Pinchart
2024-08-13 13:13 ` Dan Scally
2024-08-25 20:20 ` Laurent Pinchart
2024-07-29 10:53 ` Dan Scally
2024-08-13 13:51 ` Dan Scally
2024-07-09 13:29 ` [PATCH v6 13/18] media: platform: Fill stats buffer on ISP_START Daniel Scally
2024-07-30 21:46 ` Laurent Pinchart
2024-07-09 13:29 ` [PATCH v6 14/18] Documentation: mali-c55: Add Statistics documentation Daniel Scally
2024-07-09 13:29 ` [PATCH v6 15/18] media: mali-c55: Add image formats for Mali-C55 parameters buffer Daniel Scally
2024-07-09 13:29 ` [PATCH v6 16/18] media: uapi: Add parameters structs to mali-c55-config.h Daniel Scally
2024-07-22 23:48 ` Laurent Pinchart
2024-07-29 10:51 ` Dan Scally
2024-07-29 11:03 ` Laurent Pinchart
2024-07-09 13:29 ` [PATCH v6 17/18] media: platform: Add mali-c55 parameters video node Daniel Scally
2024-07-30 22:16 ` Laurent Pinchart
2024-07-31 6:53 ` Jacopo Mondi
2024-07-31 7:02 ` Laurent Pinchart
2024-07-31 15:12 ` Dan Scally
2024-08-02 15:03 ` Laurent Pinchart
2024-07-09 13:29 ` [PATCH v6 18/18] Documentation: mali-c55: Document the mali-c55 parameter setting Daniel Scally
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).