* [PATCH] [media] v4l2-core: fix use-after-free error
From: Sumit Gupta @ 2019-04-10 14:28 UTC (permalink / raw)
To: mchehab, hverkuil-cisco, sakari.ailus, paul.kocialkowski, tfiga,
keiichiw, sumitg
Cc: linux-media, linux-kernel
From: sumitg <sumitg@nvidia.com>
Fixing use-after-free within __v4l2_ctrl_handler_setup().
Memory is being freed with kfree(new_ref) for duplicate
control reference entry but cluster is still referring
to the duplicate entry. Change done to point cluster to
original reference instead of duplicate which is freed.
==================================================================
BUG: KASAN: use-after-free in __v4l2_ctrl_handler_setup+0x388/0x428
Read of size 8 at addr ffffffc324e78618 by task systemd-udevd/312
Allocated by task 312:
Freed by task 312:
The buggy address belongs to the object at ffffffc324e78600
which belongs to the cache kmalloc-64 of size 64
The buggy address is located 24 bytes inside of
64-byte region [ffffffc324e78600, ffffffc324e78640)
The buggy address belongs to the page:
page:ffffffbf0c939e00 count:1 mapcount:0 mapping:
(null) index:0xffffffc324e78f80
flags: 0x4000000000000100(slab)
raw: 4000000000000100 0000000000000000 ffffffc324e78f80 000000018020001a
raw: 0000000000000000 0000000100000001 ffffffc37040fb80 0000000000000000
page dumped because: kasan: bad access detected
Memory state around the buggy address:
ffffffc324e78500: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
ffffffc324e78580: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
>ffffffc324e78600: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
^
ffffffc324e78680: 00 00 00 00 00 00 00 00 fc fc fc fc fc fc fc fc
ffffffc324e78700: 00 00 00 00 00 fc fc fc fc fc fc fc fc fc fc fc
==================================================================
Signed-off-by: sumitg <sumitg@nvidia.com>
---
drivers/media/v4l2-core/v4l2-ctrls.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index 5e3806f..e971bab 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -2182,6 +2182,7 @@ static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
continue;
/* Don't add duplicates */
if (ref->ctrl->id == id) {
+ ctrl->cluster = &ref->ctrl;
kfree(new_ref);
goto unlock;
}
--
2.7.4
^ permalink raw reply related
* [PATCH v3] rcar-csi2: Move setting of Field Detection Control Register
From: Niklas Söderlund @ 2019-04-10 14:27 UTC (permalink / raw)
To: Hans Verkuil, Laurent Pinchart, linux-media
Cc: linux-renesas-soc, Niklas Söderlund, Ulrich Hecht,
Kieran Bingham
Later datasheet versions (rev 1.00) clarifies that the FLD register
should be set after LINKCNT.
Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Ulrich Hecht <uli+renesas@fpond.eu>
Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
drivers/media/platform/rcar-vin/rcar-csi2.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/rcar-vin/rcar-csi2.c b/drivers/media/platform/rcar-vin/rcar-csi2.c
index 73c51e9d4a1adcba..799e526fd3df5554 100644
--- a/drivers/media/platform/rcar-vin/rcar-csi2.c
+++ b/drivers/media/platform/rcar-vin/rcar-csi2.c
@@ -519,8 +519,6 @@ static int rcsi2_start_receiver(struct rcar_csi2 *priv)
rcsi2_write(priv, PHTC_REG, 0);
/* Configure */
- rcsi2_write(priv, FLD_REG, FLD_FLD_NUM(2) | FLD_FLD_EN4 |
- FLD_FLD_EN3 | FLD_FLD_EN2 | FLD_FLD_EN);
rcsi2_write(priv, VCDT_REG, vcdt);
if (vcdt2)
rcsi2_write(priv, VCDT2_REG, vcdt2);
@@ -551,6 +549,8 @@ static int rcsi2_start_receiver(struct rcar_csi2 *priv)
rcsi2_write(priv, PHYCNT_REG, phycnt);
rcsi2_write(priv, LINKCNT_REG, LINKCNT_MONITOR_EN |
LINKCNT_REG_MONI_PACT_EN | LINKCNT_ICLK_NONSTOP);
+ rcsi2_write(priv, FLD_REG, FLD_FLD_NUM(2) | FLD_FLD_EN4 |
+ FLD_FLD_EN3 | FLD_FLD_EN2 | FLD_FLD_EN);
rcsi2_write(priv, PHYCNT_REG, phycnt | PHYCNT_SHUTDOWNZ);
rcsi2_write(priv, PHYCNT_REG, phycnt | PHYCNT_SHUTDOWNZ | PHYCNT_RSTZ);
--
2.21.0
^ permalink raw reply related
* Re: [PATCH 4/7] media: atmel: atmel-isc: add support for DO_WHITE_BALANCE
From: Hans Verkuil @ 2019-04-10 14:26 UTC (permalink / raw)
To: Eugen.Hristev, linux-media, Nicolas.Ferre, linux-arm-kernel,
linux-kernel, mchehab
Cc: ksloat
In-Reply-To: <1554807715-2353-5-git-send-email-eugen.hristev@microchip.com>
On 4/9/19 1:07 PM, Eugen.Hristev@microchip.com wrote:
> From: Eugen Hristev <eugen.hristev@microchip.com>
>
> This adds support for the 'button' control DO_WHITE_BALANCE
> This feature will enable the ISC to compute the white balance coefficients
> in a one time shot, at the user discretion.
> This can be used if a color chart/grey chart is present in front of the camera.
> The ISC will adjust the coefficients and have them fixed until next balance
> or until sensor mode is changed.
> This is particularly useful for white balance adjustment in different
> lighting scenarios, and then taking photos to similar scenery.
> The old auto white balance stays in place, where the ISC will adjust every
> 4 frames to the current scenery lighting, if the scenery is approximately
> grey in average, otherwise grey world algorithm fails.
> One time white balance adjustments needs streaming to be enabled, such that
> capture is enabled and the histogram has data to work with.
> Histogram without capture does not work in this hardware module.
>
> To disable auto white balance feature (first step)
> v4l2-ctl --set-ctrl=white_balance_automatic=0
>
> To start the one time white balance procedure:
> v4l2-ctl --set-ctrl=do_white_balance=1
>
> User controls now include the do_white_balance ctrl:
> User Controls
>
> brightness 0x00980900 (int) : min=-1024 max=1023 step=1 default=0 value=0 flags=slider
> contrast 0x00980901 (int) : min=-2048 max=2047 step=1 default=256 value=256 flags=slider
> white_balance_automatic 0x0098090c (bool) : default=1 value=1
> do_white_balance 0x0098090d (button) : flags=write-only, execute-on-write
> gamma 0x00980910 (int) : min=0 max=2 step=1 default=2 value=2 flags=slider
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> drivers/media/platform/atmel/atmel-isc.c | 74 +++++++++++++++++++++++++++++---
> 1 file changed, 69 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc.c b/drivers/media/platform/atmel/atmel-isc.c
> index f6b8b00e..e516805 100644
> --- a/drivers/media/platform/atmel/atmel-isc.c
> +++ b/drivers/media/platform/atmel/atmel-isc.c
> @@ -167,6 +167,9 @@ struct isc_ctrls {
> u32 brightness;
> u32 contrast;
> u8 gamma_index;
> +#define ISC_WB_NONE 0
> +#define ISC_WB_AUTO 1
> +#define ISC_WB_ONETIME 2
> u8 awb;
>
> /* one for each component : GR, R, GB, B */
> @@ -210,6 +213,7 @@ struct isc_device {
> struct fmt_config try_config;
>
> struct isc_ctrls ctrls;
> + struct v4l2_ctrl *do_wb_ctrl;
> struct work_struct awb_work;
>
> struct mutex lock;
> @@ -809,7 +813,7 @@ static void isc_set_pipeline(struct isc_device *isc, u32 pipeline)
>
> bay_cfg = isc->config.sd_format->cfa_baycfg;
>
> - if (!ctrls->awb)
> + if (ctrls->awb == ISC_WB_NONE)
> isc_reset_awb_ctrls(isc);
>
> regmap_write(regmap, ISC_WB_CFG, bay_cfg);
> @@ -1928,7 +1932,7 @@ static void isc_awb_work(struct work_struct *w)
> baysel = isc->config.sd_format->cfa_baycfg << ISC_HIS_CFG_BAYSEL_SHIFT;
>
> /* if no more auto white balance, reset controls. */
> - if (!ctrls->awb)
> + if (ctrls->awb == ISC_WB_NONE)
> isc_reset_awb_ctrls(isc);
>
> pm_runtime_get_sync(isc->dev);
> @@ -1937,7 +1941,7 @@ static void isc_awb_work(struct work_struct *w)
> * only update if we have all the required histograms and controls
> * if awb has been disabled, we need to reset registers as well.
> */
> - if (hist_id == ISC_HIS_CFG_MODE_GR || !ctrls->awb) {
> + if (hist_id == ISC_HIS_CFG_MODE_GR || ctrls->awb == ISC_WB_NONE) {
> /*
> * It may happen that DMA Done IRQ will trigger while we are
> * updating white balance registers here.
> @@ -1947,6 +1951,16 @@ static void isc_awb_work(struct work_struct *w)
> spin_lock_irqsave(&isc->awb_lock, flags);
> isc_update_awb_ctrls(isc);
> spin_unlock_irqrestore(&isc->awb_lock, flags);
> +
> + /*
> + * if we are doing just the one time white balance adjustment,
> + * we are basically done.
> + */
> + if (ctrls->awb == ISC_WB_ONETIME) {
> + v4l2_info(&isc->v4l2_dev,
> + "Completed one time white-balance adjustment.\n");
> + ctrls->awb = ISC_WB_NONE;
> + }
> }
> regmap_write(regmap, ISC_HIS_CFG, hist_id | baysel | ISC_HIS_CFG_RAR);
> isc_update_profile(isc);
> @@ -1974,10 +1988,56 @@ static int isc_s_ctrl(struct v4l2_ctrl *ctrl)
> ctrls->gamma_index = ctrl->val;
> break;
> case V4L2_CID_AUTO_WHITE_BALANCE:
> - ctrls->awb = ctrl->val;
> + if (ctrl->val == 1) {
> + ctrls->awb = ISC_WB_AUTO;
> + v4l2_ctrl_activate(isc->do_wb_ctrl, false);
> + } else {
> + ctrls->awb = ISC_WB_NONE;
> + v4l2_ctrl_activate(isc->do_wb_ctrl, true);
> + }
> + /* we did not configure ISC yet */
> + if (!isc->config.sd_format)
> + break;
> +
> + if (!ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code)) {
> + v4l2_err(&isc->v4l2_dev,
> + "White balance adjustments available only if sensor is in RAW mode.\n");
This isn't an error, instead if the format isn't raw, then deactivate
the control (see v4l2_ctrl_activate()). That way the control framework
will handle this.
> + return 0;
> + }
> +
> if (ctrls->hist_stat != HIST_ENABLED) {
> isc_reset_awb_ctrls(isc);
> }
> +
> + if (isc->ctrls.awb && vb2_is_streaming(&isc->vb2_vidq) &&
> + ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
> + isc_set_histogram(isc, true);
> +
> + break;
> + case V4L2_CID_DO_WHITE_BALANCE:
> + /* we did not configure ISC yet */
> + if (!isc->config.sd_format)
> + break;
> +
> + if (ctrls->awb == ISC_WB_AUTO) {
> + v4l2_err(&isc->v4l2_dev,
> + "To use one time white-balance adjustment, disable auto white balance first.\n");
I'd do this differently: if auto whitebalance is already on, then just do
nothing for V4L2_CID_DO_WHITE_BALANCE.
> + return -EAGAIN;
> + }
> + if (!vb2_is_streaming(&isc->vb2_vidq)) {
> + v4l2_err(&isc->v4l2_dev,
> + "One time white-balance adjustment requires streaming to be enabled.\n");
This too should use v4l2_ctrl_activate(): activate the control in start_streaming,
deactivate in stop_streaming (and when the control is created).
> + return -EAGAIN;
> + }
> +
> + if (!ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code)) {
> + v4l2_err(&isc->v4l2_dev,
> + "White balance adjustments available only if sensor is in RAW mode.\n");
Same note as above: use v4l2_ctrl_activate() for this.
> + return -EAGAIN;
> + }
> + ctrls->awb = ISC_WB_ONETIME;
> + isc_set_histogram(isc, true);
> + v4l2_info(&isc->v4l2_dev, "One time white-balance started.\n");
Make this v4l2_dbg.
> break;
> default:
> return -EINVAL;
> @@ -2000,7 +2060,7 @@ static int isc_ctrl_init(struct isc_device *isc)
> ctrls->hist_stat = HIST_INIT;
> isc_reset_awb_ctrls(isc);
>
> - ret = v4l2_ctrl_handler_init(hdl, 4);
> + ret = v4l2_ctrl_handler_init(hdl, 5);
> if (ret < 0)
> return ret;
>
> @@ -2012,6 +2072,10 @@ static int isc_ctrl_init(struct isc_device *isc)
> v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAMMA, 0, GAMMA_MAX, 1, 2);
> v4l2_ctrl_new_std(hdl, ops, V4L2_CID_AUTO_WHITE_BALANCE, 0, 1, 1, 1);
>
> + /* do_white_balance is a button, so min,max,step,default are ignored */
> + isc->do_wb_ctrl = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_DO_WHITE_BALANCE,
> + 0, 0, 0, 0);
> +
> v4l2_ctrl_handler_setup(hdl);
>
> return 0;
>
Regards,
Hans
^ permalink raw reply
* Re: [PATCH 08/10] media: coda: allow encoder to set colorimetry on the output queue
From: Philipp Zabel @ 2019-04-10 14:23 UTC (permalink / raw)
To: Hans Verkuil, linux-media; +Cc: kernel
In-Reply-To: <1c5e978c-eecb-8847-8746-51e83e48ae85@xs4all.nl>
On Wed, 2019-04-10 at 15:48 +0200, Hans Verkuil wrote:
[...]
> Isn't the colorimetry information encoded in the stream's metadata?
That depends on the codec. Colorimetry information can be stored in
optional headers (for example Sequence Display Extension for MPEG-2,
Video Usability Information for H.264) but I don't know that the CODA
firmware can parse or generate any of them.
> So should it be set i an encoder register so it ends up in the right
> place in the bitstream?
I could produce the necessary headers manually and inject them into the
bitstream in the driver.
But for example for JPEG it is not clear to me if there even is a
correct way to do that. Can V4L2 colorimetry settings be translated into
ICC profiles?
> And for decoders it should then be read back from a register.
Same as above, I'd have to parse the headers in the driver.
> Just curious how this would work for this codec and if it is even possible.
>
> For now this patch is OK, but I think more work is needed.
Agreed.
regards
Philipp
^ permalink raw reply
* Re: [PATCH 07/10] media: coda: limit frame interval enumeration to supported frame sizes
From: Philipp Zabel @ 2019-04-10 14:22 UTC (permalink / raw)
To: Hans Verkuil, linux-media; +Cc: kernel
In-Reply-To: <ebbc3d03-e59e-32d5-c943-f9e426414848@xs4all.nl>
On Wed, 2019-04-10 at 15:43 +0200, Hans Verkuil wrote:
[...]
> > @@ -1126,12 +1127,32 @@ static int coda_enum_frameintervals(struct file *file, void *fh,
[...]
>
> Why support VIDIOC_ENUM_FRAMEINTERVALS at all? It makes no sense for a codec.
> I'd remove it altogether.
It returns the range supported by the frame rate registers that can be
set for constant bitrate encoding. I think the idea was to let the
GStreamer v4l2 elements know about possible frame rate range.
I think I should be able to remove it without any negative effects.
regards
Philipp
^ permalink raw reply
* Re: [RFC PATCH] media/doc: Allow sizeimage to be set by v4l clients
From: Stanimir Varbanov @ 2019-04-10 14:21 UTC (permalink / raw)
To: Hans Verkuil, linux-media
Cc: Mauro Carvalho Chehab, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot, Malathi Gottam
In-Reply-To: <c103b0d8-0020-17e0-4584-e5c3ca6bbc51@xs4all.nl>
Hi Hans,
On 3/14/19 3:11 PM, Hans Verkuil wrote:
> On 1/21/19 11:48 AM, Stanimir Varbanov wrote:
>> Hi Hans,
>>
>> On 1/18/19 11:13 AM, Hans Verkuil wrote:
>>> On 1/16/19 1:37 PM, Stanimir Varbanov wrote:
>>>> This changes v4l2_pix_format and v4l2_plane_pix_format sizeimage
>>>> field description to allow v4l clients to set bigger image size
>>>> in case of variable length compressed data.
>>>>
>>>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>>>> ---
>>>> Documentation/media/uapi/v4l/pixfmt-v4l2-mplane.rst | 5 ++++-
>>>> Documentation/media/uapi/v4l/pixfmt-v4l2.rst | 3 ++-
>>>> 2 files changed, 6 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-v4l2-mplane.rst b/Documentation/media/uapi/v4l/pixfmt-v4l2-mplane.rst
>>>> index 7f82dad9013a..dbe0b74e9ba4 100644
>>>> --- a/Documentation/media/uapi/v4l/pixfmt-v4l2-mplane.rst
>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-v4l2-mplane.rst
>>>> @@ -30,7 +30,10 @@ describing all planes of that format.
>>>>
>>>> * - __u32
>>>> - ``sizeimage``
>>>> - - Maximum size in bytes required for image data in this plane.
>>>> + - Maximum size in bytes required for image data in this plane,
>>>> + set by the driver. When the image consists of variable length
>>>> + compressed data this is the maximum number of bytes required
>>>> + to hold an image, and it is allowed to be set by the client.
>>>> * - __u32
>>>> - ``bytesperline``
>>>> - Distance in bytes between the leftmost pixels in two adjacent
>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-v4l2.rst b/Documentation/media/uapi/v4l/pixfmt-v4l2.rst
>>>> index 71eebfc6d853..54b6d2b67bd7 100644
>>>> --- a/Documentation/media/uapi/v4l/pixfmt-v4l2.rst
>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-v4l2.rst
>>>> @@ -89,7 +89,8 @@ Single-planar format structure
>>>> - Size in bytes of the buffer to hold a complete image, set by the
>>>> driver. Usually this is ``bytesperline`` times ``height``. When
>>>> the image consists of variable length compressed data this is the
>>>> - maximum number of bytes required to hold an image.
>>>> + maximum number of bytes required to hold an image, and it is
>>>> + allowed to be set by the client.
>>>> * - __u32
>>>> - ``colorspace``
>>>> - Image colorspace, from enum :c:type:`v4l2_colorspace`.
>>>>
>>>
>>> Hmm. "maximum number of bytes required to hold an image": that's not actually true
>>> for bitstream formats like MPEG. It's just the size of the buffer used to store the
>>> bitstream, i.e. one buffer may actually contain multiple compressed images, or a
>>> compressed image is split over multiple buffers.
>>>
>>
>> Do you want me to change something in the current documentation, i.e.
>> the quoted above?
>
> Hmm, it looks like this discussion stalled (i.e. I forgot to reply).
>
> How about this:
>
> "When the image consists of variable length compressed data this is the
> number of bytes required by the encoder to support the worst-case
I don't think 'encoder' is the right word here:
s/encoder/encoder or decoder
> compression scenario. Clients are allowed to set this field. However,
> drivers may ignore the value or modify it."
Can we rephrase to:
"Clients are allowed to set sizeimage field, but however drivers my
ignore the value or modify it."
--
regards,
Stan
^ permalink raw reply
* Re: [PATCH 1/7] media: atmel: atmel-isc: add safe checks and fixed wrong ISC state in error case
From: Hans Verkuil @ 2019-04-10 14:19 UTC (permalink / raw)
To: Eugen.Hristev, linux-media, Nicolas.Ferre, linux-arm-kernel,
linux-kernel, mchehab
Cc: ksloat
In-Reply-To: <1554807715-2353-2-git-send-email-eugen.hristev@microchip.com>
On 4/9/19 1:07 PM, Eugen.Hristev@microchip.com wrote:
> From: Eugen Hristev <eugen.hristev@microchip.com>
>
> This adds safety checks on some scenarios:
> - start streaming but streaming is already started
Can't happen. vb2 checks for that.
> - start streaming but no buffer in the dma queue
Can't happen since min_buffers_needed is > 0. So start_streaming will
never be called unless at least one buffer is queued.
> - spin lock is not released in error scenario
> - no frame is configured but dma is requested to start
Can this ever happen? It's set by start_streaming when you know there is
at least one buffer, so this seems overkill.
Regards,
Hans
> - configure ISC may have been called without need, before checking if buffer is
> ok.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> drivers/media/platform/atmel/atmel-isc.c | 32 +++++++++++++++++++++++++++-----
> 1 file changed, 27 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc.c b/drivers/media/platform/atmel/atmel-isc.c
> index a10db16..3c19761 100644
> --- a/drivers/media/platform/atmel/atmel-isc.c
> +++ b/drivers/media/platform/atmel/atmel-isc.c
> @@ -722,6 +722,11 @@ static void isc_start_dma(struct isc_device *isc)
> u32 dctrl_dview;
> dma_addr_t addr0;
>
> + if (!isc->cur_frm) {
> + v4l2_err(&isc->v4l2_dev, "Video buffer not available\n");
> + return;
> + }
> +
> addr0 = vb2_dma_contig_plane_dma_addr(&isc->cur_frm->vb.vb2_buf, 0);
> regmap_write(regmap, ISC_DAD0, addr0);
>
> @@ -886,6 +891,9 @@ static int isc_start_streaming(struct vb2_queue *vq, unsigned int count)
> unsigned long flags;
> int ret;
>
> + if (vb2_is_streaming(&isc->vb2_vidq))
> + return -EBUSY;
> +
> /* Enable stream on the sub device */
> ret = v4l2_subdev_call(isc->current_subdev->sd, video, s_stream, 1);
> if (ret && ret != -ENOIOCTLCMD) {
> @@ -896,6 +904,20 @@ static int isc_start_streaming(struct vb2_queue *vq, unsigned int count)
>
> pm_runtime_get_sync(isc->dev);
>
> + spin_lock_irqsave(&isc->dma_queue_lock, flags);
> +
> + isc->sequence = 0;
> + isc->stop = false;
> + reinit_completion(&isc->comp);
> +
> + if (list_empty(&isc->dma_queue)) {
> + v4l2_err(&isc->v4l2_dev, "dma queue empty\n");
> + ret = -EINVAL;
> + goto err_configure_unlock;
> + }
> +
> + spin_unlock_irqrestore(&isc->dma_queue_lock, flags);
> +
> ret = isc_configure(isc);
> if (unlikely(ret))
> goto err_configure;
> @@ -905,10 +927,6 @@ static int isc_start_streaming(struct vb2_queue *vq, unsigned int count)
>
> spin_lock_irqsave(&isc->dma_queue_lock, flags);
>
> - isc->sequence = 0;
> - isc->stop = false;
> - reinit_completion(&isc->comp);
> -
> isc->cur_frm = list_first_entry(&isc->dma_queue,
> struct isc_buffer, list);
> list_del(&isc->cur_frm->list);
> @@ -919,8 +937,11 @@ static int isc_start_streaming(struct vb2_queue *vq, unsigned int count)
>
> return 0;
>
> +err_configure_unlock:
> + spin_unlock_irqrestore(&isc->dma_queue_lock, flags);
> +
> err_configure:
> - pm_runtime_put_sync(isc->dev);
> + isc->stop = true;
>
> v4l2_subdev_call(isc->current_subdev->sd, video, s_stream, 0);
>
> @@ -931,6 +952,7 @@ static int isc_start_streaming(struct vb2_queue *vq, unsigned int count)
> INIT_LIST_HEAD(&isc->dma_queue);
> spin_unlock_irqrestore(&isc->dma_queue_lock, flags);
>
> + pm_runtime_put_sync(isc->dev);
> return ret;
> }
>
>
^ permalink raw reply
* RE: [EXTERNAL] Re: [PATCH] media: omap_vout: potential buffer overflow in vidioc_dqbuf()
From: Scheurer, Amber @ 2019-04-10 14:05 UTC (permalink / raw)
To: Dan Carpenter, Hans Verkuil
Cc: Mauro Carvalho Chehab, Niklas Söderlund, Philipp Zabel,
Parrot, Benoit, linux-media@vger.kernel.org,
kernel-janitors@vger.kernel.org, Andrzej Hajda
In-Reply-To: <20190410111415.GB31633@kadam>
I think you might have the wrong Amber copied on this email.
Amber
-----Original Message-----
From: Dan Carpenter [mailto:dan.carpenter@oracle.com]
Sent: Wednesday, April 10, 2019 6:14 AM
To: Hans Verkuil
Cc: Mauro Carvalho Chehab; Scheurer, Amber; Niklas Söderlund; Philipp Zabel; Parrot, Benoit; linux-media@vger.kernel.org; kernel-janitors@vger.kernel.org; Andrzej Hajda
Subject: [EXTERNAL] Re: [PATCH] media: omap_vout: potential buffer overflow in vidioc_dqbuf()
On Wed, Apr 10, 2019 at 12:50:31PM +0200, Hans Verkuil wrote:
> On 4/9/19 1:29 PM, Dan Carpenter wrote:
> > diff --git a/drivers/media/platform/omap/omap_vout.c b/drivers/media/platform/omap/omap_vout.c
> > index 37f0d7146dfa..15e38990e85a 100644
> > --- a/drivers/media/platform/omap/omap_vout.c
> > +++ b/drivers/media/platform/omap/omap_vout.c
> > @@ -1527,8 +1527,6 @@ static int vidioc_dqbuf(struct file *file, void *fh, struct v4l2_buffer *b)
> > unsigned long size;
> > struct videobuf_buffer *vb;
> >
> > - vb = q->bufs[b->index];
> > -
> > if (!vout->streaming)
> > return -EINVAL;
> >
> > @@ -1539,6 +1537,8 @@ static int vidioc_dqbuf(struct file *file, void *fh, struct v4l2_buffer *b)
> > /* Call videobuf_dqbuf for blocking mode */
> > ret = videobuf_dqbuf(q, (struct v4l2_buffer *)b, 0);
>
> We need a:
>
> if (ret)
> return ret;
>
> here. Or alternatively, add 'if (!ret)' around the next five lines.
>
> b->index is only valid if the videobuf_dqbuf call returned 0.
>
Doh. Thanks.
regards,
dan carpenter
^ permalink raw reply
* Re: [PATCH v5 05/13] media: tvp5150: add input source selection of_graph support
From: Marco Felsch @ 2019-04-10 14:04 UTC (permalink / raw)
To: Jacopo Mondi
Cc: mchehab, sakari.ailus, hans.verkuil, jacopo+renesas, robh+dt,
p.zabel, javierm, laurent.pinchart, linux-media, devicetree,
kernel
In-Reply-To: <20190405144542.rinp3jrczlxo3lwf@uno.localdomain>
Hi Jacopo,
On 19-04-05 16:45, Jacopo Mondi wrote:
> Hi Marco,
>
> On Fri, Apr 05, 2019 at 08:03:09AM +0200, Marco Felsch wrote:
> > This patch adds the of_graph support to describe the tvp connections.
> > Physical the TVP5150 has three ports: AIP1A, AIP1B and YOUT. As result
> > of discussion [1],[2] the device-tree maps these ports 1:1. The svideo
> > connector must be conneted to port@0/endpoint@1, look at the Documentation
> > for more information. Since the TVP5150 is a converter the device-tree
> > must contain at least 1-input and 1-output port. The mc-connectors and
> > mc-links are only created if the device-tree contains the corresponding
> > connector nodes. If more than one connector is available the
> > media_entity_operations.link_setup() callback ensures that only one
> > connector is active.
> >
> > [1] https://www.spinics.net/lists/linux-media/msg138545.html
> > [2] https://www.spinics.net/lists/linux-media/msg138546.html
> >
> > Signed-off-by: Marco Felsch <m.felsch@pengutronix.de>
> > ---
> > Changelog:
> >
> > [1] https://patchwork.kernel.org/cover/10794703/
> > [2] https://patchwork.kernel.org/cover/10786553/
> >
> > v5:
> > - Fixing build deps:
> > - tvp5150_mc_init: fix CONFIG_MEDIA_CONTROLLER deps
> > - struct tvp5150: drop CONFIG_MEDIA_CONTROLLER conditional property
> > includes. This leads into to complex deps for futher development.
> > - tvp5150_dt_cleanup: enable function only if CONFIG_OF is enabled
> > - tvp5150_parse_dt: enable function only if CONFIG_OF is enabled
> > - tvp5150_probe: call tvp5150_dt_cleanup only if CONFIG_OF is enabled
> >
> > - Simplify link_setup routine:
> > - use generic connector parsing since both series [1,2] are squashed into
> > one
> > - struct tvp5150: drop pads_state and modify_second_link property
> > due to link_setup() rework.
> > - tvp5150_link_setup: add more comments
> > - tvp5150_link_setup: simply the link setup routine a lot. Edit the 2nd
> > link directly within the driver instead of a recursive media-framework
> > call (__media_entity_setup_link). This improves the readability and
> > shrinks the driver code.
> > - tvp5150_link_setup: disable all active links in case user switches
> > connectors without disable it first.
> > - tvp5150_registered: simplify default link enable path due to link_setup()
> > rework.
> >
> > - General cleanups
> > - tvp5150_parse_dt: drop unecessary test
> > - tvp5150_parse_dt: add err message due to misconfiguration
> > - tvp5150_parse_dt: make use of V4L2_MBUS_UNKNOWN definition
> > - s/dev_dbg/dev_dbg_lvl
> >
>
> Great, this looks much nicer!
>
> > v4:
> > - rebase on top of media_tree/master, fix merge conflict due to commit
> > 60359a28d592 ("media: v4l: fwnode: Initialise the V4L2 fwnode endpoints
> > to zero")
> >
> > v3:
> > - probe(): s/err/err_free_v4l2_ctrls
> > - drop MC dependency for tvp5150_pads
> >
> > v2:
> > - adapt commit message
> > - unify ifdef switches
> > - rename tvp5150_valid_input -> tvp5150_of_valid_input, to be more precise
> > - mc: use 2-input and 1-output pad
> > - mc: link svideo connector to both input pads
> > - mc: enable/disable svideo links in one go
> > - mc: change link_setup() behaviour, switch the input src don't require a
> > explicite disable before.
> > - mc: rename 'local' media_pad param to tvp5150_pad to avoid confusion
> > - mc: enable link to the first available connector and set the
> > corresponding tvp5150 input src per default during registered()
> > - mc/of: factor out oftree connector allocation
> > - of: drop svideo dt port
> > - of: move svideo connector to port@0/endpoint@1
> > - of: require at least 1-in and 1-out endpoint
> >
> > drivers/media/i2c/tvp5150.c | 408 ++++++++++++++++++++++++++++++++----
> > 1 file changed, 368 insertions(+), 40 deletions(-)
> >
> > diff --git a/drivers/media/i2c/tvp5150.c b/drivers/media/i2c/tvp5150.c
> > index 89da921c8886..91504fddb551 100644
> > --- a/drivers/media/i2c/tvp5150.c
> > +++ b/drivers/media/i2c/tvp5150.c
> > @@ -44,16 +44,29 @@ MODULE_PARM_DESC(debug, "Debug level (0-2)");
> > #define dprintk0(__dev, __arg...) dev_dbg_lvl(__dev, 0, 0, __arg)
> >
> > enum tvp5150_pads {
> > - TVP5150_PAD_IF_INPUT,
> > + TVP5150_PAD_AIP1A = TVP5150_COMPOSITE0,
> > + TVP5150_PAD_AIP1B,
> > TVP5150_PAD_VID_OUT,
> > TVP5150_NUM_PADS
> > };
> >
> > +struct tvp5150_connector {
> > + struct v4l2_fwnode_connector base;
> > + struct media_entity ent;
> > + struct media_pad pad;
> > +};
> > +
> > struct tvp5150 {
> > struct v4l2_subdev sd;
> > -#ifdef CONFIG_MEDIA_CONTROLLER
> > + /* additional additional endpoint for the svideo connector */
>
> s/additional additional/additional/
Damn, fixed it.
> > + struct device_node *endpoints[TVP5150_NUM_PADS + 1];
> > + unsigned int endpoints_num;
> > +
> > + /* media-ctl properties */
> > struct media_pad pads[TVP5150_NUM_PADS];
> > -#endif
> > + struct tvp5150_connector *connectors;
> > + int connectors_num;
> > +
> > struct v4l2_ctrl_handler hdl;
> > struct v4l2_rect rect;
> > struct regmap *regmap;
> > @@ -1167,6 +1180,129 @@ static int tvp5150_enum_frame_size(struct v4l2_subdev *sd,
> > return 0;
> > }
> >
> > +/****************************************************************************
> > + * Media entity ops
> > + ****************************************************************************/
> > +#if defined(CONFIG_MEDIA_CONTROLLER)
> > +static int tvp5150_set_link(struct media_pad *connector_pad,
> > + struct media_pad *tvp5150_pad, u32 flags)
> > +{
> > + struct media_link *link;
> > +
> > + link = media_entity_find_link(connector_pad, tvp5150_pad);
> > + if (!link)
> > + return -EINVAL;
> > +
> > + link->flags = flags;
> > + link->reverse->flags = link->flags;
> > +
> > + return 0;
> > +}
> > +
> > +static int tvp5150_disable_all_input_links(struct tvp5150 *decoder)
> > +{
> > + struct media_pad *connector_pad;
> > + int i, err;
>
> i can be unsigned here
Okay
> > +
> > + for (i = 0; i < TVP5150_NUM_PADS - 1; i++) {
> > + connector_pad = media_entity_remote_pad(&decoder->pads[i]);
> > + if (!connector_pad)
> > + continue;
> > +
> > + err = tvp5150_set_link(connector_pad, &decoder->pads[i], 0);
> > + if (err)
> > + return err;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int tvp5150_s_routing(struct v4l2_subdev *sd, u32 input, u32 output,
> > + u32 config);
> > +
> > +static int tvp5150_link_setup(struct media_entity *entity,
> > + const struct media_pad *tvp5150_pad,
> > + const struct media_pad *remote, u32 flags)
> > +{
> > + struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
> > + struct tvp5150 *decoder = to_tvp5150(sd);
> > + struct media_pad *other_tvp5150_pad =
> > + &decoder->pads[tvp5150_pad->index ^ 1];
> > + bool is_svideo = false;
> > + int i, err;
>
> i can be unsigned
Okay
> > +
> > + /*
> > + * The TVP5150 state is determined by the enabled sink pad link(s).
> > + * Enabling or disabling the source pad link has no effect.
> > + */
> > + if (tvp5150_pad->flags & MEDIA_PAD_FL_SOURCE)
> > + return 0;
> > +
> > + /* Check if the svideo connector should be enabled */
> > + for (i = 0; i < decoder->connectors_num; i++) {
> > + if (remote->entity == &decoder->connectors[i].ent) {
> > + is_svideo =
> > + decoder->connectors[i].base.type == V4L2_CON_SVIDEO;
> > + break;
> > + }
> > + }
> > +
> > + dev_dbg_lvl(sd->dev, 1, debug, "link setup '%s':%d->'%s':%d[%d]",
> > + remote->entity->name, remote->index,
> > + tvp5150_pad->entity->name, tvp5150_pad->index,
> > + flags & MEDIA_LNK_FL_ENABLED);
> > + if (is_svideo)
> > + dev_dbg_lvl(sd->dev, 1, debug,
> > + "link setup '%s':%d->'%s':%d[%d]",
> > + remote->entity->name, remote->index,
> > + other_tvp5150_pad->entity->name,
> > + other_tvp5150_pad->index,
> > + flags & MEDIA_LNK_FL_ENABLED);
> > +
> > + /*
> > + * The TVP5150 has a internal mux which allows the following setup:
>
> an internal
Yes
> > + *
> > + * comp-connector1 --\
> > + * |---> AIP1A
> > + * /
> > + * svideo-connector -|
> > + * \
> > + * |---> AIP1B
> > + * comp-connector2 --/
> > + *
> > + * We can't rely on user space that the current connector gets disabled
> > + * first before enabling the new connector. Disable all active
> > + * connector links to be on the safe side.
> > + */
> > + err = tvp5150_disable_all_input_links(decoder);
> > + if (err)
> > + return err;
>
> Not sure here... I think you should refuse, say, to enable
> comp-connector2->AIP1B if S_VIDEO is enabled on AIP1A, or maybe that's not
> possible, as you have a single linke to AIP1B... Another example: you
> should refuse to enable comp-connector1->AIP1A if
> svideo-connector->AIP1A is enabled. These are two distinct links, so
> this should be possible. Am I wrong?
I tought the diagram and prose makes it cleaner, damn.. Thanks for
asking maybe I should think about rewording it..
In short, the svideo needs both links (AIP1A, AIP1B) enabled so there is
no AIP1A or AIP1B case since both are on. In a previouse version of this
series I added the refusing mechanism but Mauro had some concerns about
that. Since the media-api don't cover that he said the dynamic switching
should be the way to go since it is easier for the user space.
So the current solution is that there can be one link enabled (comp1 or
comp2) or two links (svideo). To handle these cases and avoid much
complexity I disable all active links first since it isn't forbidden by
the media-api.
> > +
> > + tvp5150_s_routing(sd, is_svideo ? TVP5150_SVIDEO : tvp5150_pad->index,
> > + flags & MEDIA_LNK_FL_ENABLED ? TVP5150_NORMAL :
> > + TVP5150_BLACK_SCREEN, 0);
> > +
> > + if (flags & MEDIA_LNK_FL_ENABLED) {
> > + /*
> > + * S-Video connector is conneted to both ports AIP1A and AIP1B.
> > + * Both links must be enabled in one-shot regardless which link
> > + * the user requests.
> > + */
> > + if (is_svideo) {
> > + err = tvp5150_set_link((struct media_pad *) remote,
> > + other_tvp5150_pad, flags);
> > + if (err)
> > + return err;
> > + }
> > + }
>
> This is much much nicer than v4! Great job!
>
> > +
> > + return 0;
> > +}
> > +
> > +static const struct media_entity_operations tvp5150_sd_media_ops = {
> > + .link_setup = tvp5150_link_setup,
> > +};
> > +#endif
> > /****************************************************************************
> > I2C Command
> > ****************************************************************************/
> > @@ -1314,6 +1450,65 @@ static int tvp5150_g_tuner(struct v4l2_subdev *sd, struct v4l2_tuner *vt)
> > return 0;
> > }
> >
> > +static int tvp5150_registered(struct v4l2_subdev *sd)
> > +{
> > +#if defined(CONFIG_MEDIA_CONTROLLER)
>
> I wonder if it wouldn't make more sense to select MEDIA_CONTROLLER...
> How do you configure the device without MC, since I haven't seen support for
> platform data?
@Mauro
Can you say something about the non MC case?
I know that the em28xx device is using this driver and if I get it right
the em28xx calls the s_routing callback to set the specific input
source.
So in that case the MC mustn't available/enabled.
Luckily the em28xx can use most of the tvp5150 default config values.
>
> > + struct tvp5150 *decoder = to_tvp5150(sd);
> > + unsigned int i;
> > + int ret;
> > +
> > + /*
> > + * Setup connector pads and links. Enable the link to the first
> > + * available connector per default.
> > + */
> > + for (i = 0; i < decoder->connectors_num; i++) {
> > + struct media_entity *con = &decoder->connectors[i].ent;
> > + struct media_pad *pad = &decoder->connectors[i].pad;
> > + unsigned int port = decoder->connectors[i].base.remote_port;
> > + bool is_svideo =
> > + decoder->connectors[i].base.type == V4L2_CON_SVIDEO;
> > + int flags = i ? 0 : MEDIA_LNK_FL_ENABLED;
> > +
> > + pad->flags = MEDIA_PAD_FL_SOURCE;
> > + ret = media_entity_pads_init(con, 1, pad);
> > + if (ret < 0)
> > + return ret;
> > +
> > + ret = media_device_register_entity(sd->v4l2_dev->mdev, con);
> > + if (ret < 0)
> > + return ret;
> > +
> > + ret = media_create_pad_link(con, 0, &sd->entity, port, flags);
> > + if (ret < 0) {
> > + media_device_unregister_entity(con);
> > + return ret;
> > + }
> > +
> > + if (is_svideo) {
> > + /* svideo links to both aip1a and aip1b */
> > + ret = media_create_pad_link(con, 0, &sd->entity,
> > + port + 1, flags);
> > + if (ret < 0) {
> > + media_device_unregister_entity(con);
> > + return ret;
> > + }
> > + }
> > +
> > + /* enable default input */
> > + if (flags == MEDIA_LNK_FL_ENABLED) {
> > + decoder->input =
> > + is_svideo ? TVP5150_SVIDEO :
> > + port == 0 ? TVP5150_COMPOSITE0 :
> > + TVP5150_COMPOSITE1;
> > +
> > + tvp5150_selmux(sd);
> > + }
> > + }
> > +#endif
> > + return 0;
> > +}
> > +
> > +
> > /* ----------------------------------------------------------------------- */
> >
> > static const struct v4l2_ctrl_ops tvp5150_ctrl_ops = {
> > @@ -1367,6 +1562,10 @@ static const struct v4l2_subdev_ops tvp5150_ops = {
> > .pad = &tvp5150_pad_ops,
> > };
> >
> > +static const struct v4l2_subdev_internal_ops tvp5150_internal_ops = {
> > + .registered = tvp5150_registered,
> > +};
> > +
> > /****************************************************************************
> > I2C Client & Driver
> > ****************************************************************************/
> > @@ -1515,38 +1714,171 @@ static int tvp5150_init(struct i2c_client *c)
> > return 0;
> > }
> >
> > -static int tvp5150_parse_dt(struct tvp5150 *decoder, struct device_node *np)
> > +#if defined(CONFIG_MEDIA_CONTROLLER)
> > +static int tvp5150_add_of_connectors(struct tvp5150 *decoder)
> > {
> > - struct v4l2_fwnode_endpoint bus_cfg = { .bus_type = 0 };
> > - struct device_node *ep;
> > - unsigned int flags;
> > - int ret = 0;
> > + struct device *dev = decoder->sd.dev;
> > + struct tvp5150_connector *connectors;
> > + unsigned int connectors_num = decoder->connectors_num;
> > + int i, ret;
> > +
> > + /* Allocate and initialize all available input connectors */
> > + connectors = devm_kcalloc(dev, connectors_num, sizeof(*connectors),
> > + GFP_KERNEL);
> > + if (!connectors)
> > + return -ENOMEM;
> >
> > - ep = of_graph_get_next_endpoint(np, NULL);
> > - if (!ep)
> > - return -EINVAL;
> > + for (i = 0; i < connectors_num; i++) {
> > + struct v4l2_fwnode_connector *c = &connectors[i].base;
> >
> > - ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep), &bus_cfg);
> > - if (ret)
> > - goto err;
> > + ret = v4l2_fwnode_parse_connector(
> > + of_fwnode_handle(decoder->endpoints[i]), c);
> > +
> > + connectors[i].ent.flags = MEDIA_ENT_FL_CONNECTOR;
> > + connectors[i].ent.function = c->type == V4L2_CON_SVIDEO ?
> > + MEDIA_ENT_F_CONN_SVIDEO : MEDIA_ENT_F_CONN_COMPOSITE;
> > + connectors[i].ent.name = c->label;
>
> This is a bit of a duplication of what you're doing at parse_dt time,
> where you parse_connector() already. I guess you do this in two steps
> because you need to count connectors first, as you allocate their
> descriptor structure dynamically.
>
> Have you considered allocating them statically (it's up to 4
> connectors, not a big waste) and populate them at parse_dt time? (you
> could re-alloc too, but let's not consider that for now).
>
> Bonus point, if you parse_dt() then CONFIG_OF is enabled, and you can
> save the
> if (IS_ENABLED(CONFIG_OF))
> ret = tvp5150_add_of_connectors(decoder);
>
> you now have in tvp5150_mc_init()
Good point and your're right I parse it twice a time. I tought about
your suggestion but then I also tought about the case where it can be
get more complicated. Imagine you have 5 composite connectors: two
connected to AIP1A and 3 connected to AIP1B. Still the same behaviour,
only one link at the time can be enbaled. In that case you can't use
the static allocation. Anyway it isn't supported by the driver right now
but it is easier to add the support if we alloc them dynamic.
My idea between splitting was to have a correct abstraction.
Parsing/validation is done by parse_dt, the tvp5150_add_of_connectors()
relies on that result since it didn't check it again. Of course
v4l2_fwnode_parse_connector() gets parsed twice but this is done during
probe.
But your're right with that
> if (IS_ENABLED(CONFIG_OF))
> ret = tvp5150_add_of_connectors(decoder);
and your last review note. That check isn't correct since it can be the
case where CONFIG_OF and CONFIG_MEDIA_CONTROLLER is enabled but the
device using the driver isn't OF-capable (e.g. em28xx usb-device). So
the check must be:
if (decoder->connectors_num)
ret = tvp5150_add_of_connectors(decoder);
This ensures that tvp5150_add_of_connectors() gets called only for
OF-capable devices.
>
> > + }
> > +
> > + decoder->connectors = connectors;
> >
> > - flags = bus_cfg.bus.parallel.flags;
> > + return 0;
> > +}
> > +#endif
> >
> > - if (bus_cfg.bus_type == V4L2_MBUS_PARALLEL &&
> > - !(flags & V4L2_MBUS_HSYNC_ACTIVE_HIGH &&
> > - flags & V4L2_MBUS_VSYNC_ACTIVE_HIGH &&
> > - flags & V4L2_MBUS_FIELD_EVEN_LOW)) {
> > +static int tvp5150_mc_init(struct v4l2_subdev *sd)
> > +{
> > +#if defined(CONFIG_MEDIA_CONTROLLER)
> > + struct tvp5150 *decoder = to_tvp5150(sd);
> > + unsigned int i;
> > + int ret;
> > +
> > + sd->entity.ops = &tvp5150_sd_media_ops;
> > + sd->entity.function = MEDIA_ENT_F_ATV_DECODER;
> > +
> > + /* Initialize all TVP5150 pads */
> > + for (i = 0; i < TVP5150_NUM_PADS; i++) {
> > + if (i < TVP5150_NUM_PADS - 1) {
> > + decoder->pads[i].flags = MEDIA_PAD_FL_SINK;
> > + decoder->pads[i].sig_type = PAD_SIGNAL_ANALOG;
> > + } else {
> > + decoder->pads[i].flags = MEDIA_PAD_FL_SOURCE;
> > + decoder->pads[i].sig_type = PAD_SIGNAL_DV;
> > + }
> > + }
> > + ret = media_entity_pads_init(&sd->entity, TVP5150_NUM_PADS,
> > + decoder->pads);
> > + if (ret < 0)
> > + goto out;
> > +
> > + if (IS_ENABLED(CONFIG_OF))
> > + ret = tvp5150_add_of_connectors(decoder);
> > +out:
> > + return ret;
> > +#else
> > + return 0;
> > +#endif
>
> This would really read better as:
Good point, I will change that or should I rather change the
tvp5150_mc_init() call during the probe to
tvp5150_probe()
{
...
if(IS_ENABLED(CONFIG_MEDIA_CONTROLLER)
tvp5150_mc_init()
...
}
> #if defined(CONFIG_MEDIA_CONTROLLER)
> static int tvp5150_mc_init(struct v4l2_subdev *sd)
> {
> ...
>
> }
> static int tvp5150_mc_init(struct v4l2_subdev *sd)
> {
> ....
> }
> #else /* defined(CONFIG_MEDIA_CONTROLLER) */
> static int tvp5150_mc_init(struct v4l2_subdev *sd)
> {
> return 0;
> }
> static int tvp5150_mc_init(struct v4l2_subdev *sd)
> {
> return 0;
> }
> #endif /* defined(CONFIG_MEDIA_CONTROLLER) */
>
> > +}
> > +
> > +#if defined(CONFIG_OF)
>
> Do you need this? I tried compiling with OF support disabled and this
> #if guard removed and all went fine. After all, all calls to parse_dt
> and dt_cleanup are already guarded by an IS_ENABLED(CONFIG_OF)
My goal was to reduce the code size in case OF isn't enabled since the
call don't depend only on the CONFIG_OF this function won't be threw
away in case CONFIG_OF is disabled if I got it right.
if (IS_ENABLED(CONFIG_OF) && np)
But you're right I can remove it to reducing the ifdef switches.
> > +static int tvp5150_parse_dt(struct tvp5150 *decoder, struct device_node *np)
> > +{
> > + struct device *dev = decoder->sd.dev;
> > + struct v4l2_fwnode_endpoint bus_cfg = { .bus_type = V4L2_MBUS_UNKNOWN };
> > + struct v4l2_fwnode_connector c;
>
> Could you declare this inside the for loop?
Of course.
> > + struct device_node *ep_np;
> > + unsigned int flags;
Moved flags inside the for loop too.
> > + int ret, i = 0, in = 0;
>
> i and in could be unsigned
okay
> > + bool found = false;
> > +
> > + /* at least 1 output and 1 input */
> > + decoder->endpoints_num = of_graph_get_endpoint_count(np);
> > + if (decoder->endpoints_num < 2 || decoder->endpoints_num > 4) {
> > + dev_err(dev, "At least 1 input and 1 output must be connected to the device.\n");
> > ret = -EINVAL;
> > goto err;
> > }
> >
> > - decoder->mbus_type = bus_cfg.bus_type;
> > + for_each_endpoint_of_node(np, ep_np) {
> > + struct of_endpoint ep;
> > +
> > + of_graph_parse_endpoint(ep_np, &ep);
> > + switch (ep.port) {
> > + /* fall through */
>
> Move this below the case please
Yes.
> > + case TVP5150_PAD_AIP1A:
> > + case TVP5150_PAD_AIP1B:
> > + ret = v4l2_fwnode_parse_connector(
> > + of_fwnode_handle(ep_np), &c);
> > + if (c.type != V4L2_CON_COMPOSITE &&
> > + c.type != V4L2_CON_SVIDEO) {
> > + dev_err(dev,
> > + "Invalid endpoint %d on port %d\n",
> > + c.remote_id, c.remote_port);
> > + ret = -EINVAL;
> > + goto err;
> > + }
> > + in++;
> > + break;
> > + case TVP5150_PAD_VID_OUT:
> > + ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep_np),
> > + &bus_cfg);
> > + if (ret)
> > + goto err;
> > +
> > + flags = bus_cfg.bus.parallel.flags;
> > +
> > + if (bus_cfg.bus_type == V4L2_MBUS_PARALLEL &&
> > + !(flags & V4L2_MBUS_HSYNC_ACTIVE_HIGH &&
> > + flags & V4L2_MBUS_VSYNC_ACTIVE_HIGH &&
> > + flags & V4L2_MBUS_FIELD_EVEN_LOW)) {
> > + ret = -EINVAL;
> > + goto err;
> > + }
> > +
> > + decoder->mbus_type = bus_cfg.bus_type;
> > + break;
> > + default:
> > + dev_err(dev, "Invalid port %d for endpoint %pOF\n",
> > + ep.port, ep.local_node);
> > + ret = -EINVAL;
> > + goto err;
> > + }
> > +
> > + of_node_get(ep_np);
> > + decoder->endpoints[i] = ep_np;
> > + i++;
> >
> > + found = true;
> > + }
> > +
> > + decoder->connectors_num = in;
> > + return found ? 0 : -ENODEV;
> > err:
> > - of_node_put(ep);
> > return ret;
> > }
> >
> > +static void tvp5150_dt_cleanup(struct tvp5150 *decoder)
> > +{
> > + unsigned int i;
> > +
> > + for (i = 0; i < TVP5150_NUM_PADS; i++)
> > + of_node_put(decoder->endpoints[i]);
> > +}
> > +
> > +#else /* !defined(CONFIG_OF) */
> > +
> > +static inline int
> > +tvp5150_parse_dt(struct tvp5150 *decoder, struct device_node *np)
> > +{
> > + return 0;
> > +}
> > +
> > +static inline void tvp5150_dt_cleanup(struct tvp5150 *decoder)
> > +{
> > +
> > +}
> > +#endif
> > +
> > static const char * const tvp5150_test_patterns[2] = {
> > "Disabled",
> > "Black screen"
> > @@ -1585,7 +1917,7 @@ static int tvp5150_probe(struct i2c_client *c,
> > res = tvp5150_parse_dt(core, np);
> > if (res) {
> > dev_err(sd->dev, "DT parsing error: %d\n", res);
> > - return res;
> > + goto err_cleanup_dt;
> > }
> > } else {
> > /* Default to BT.656 embedded sync */
> > @@ -1593,25 +1925,16 @@ static int tvp5150_probe(struct i2c_client *c,
> > }
> >
> > v4l2_i2c_subdev_init(sd, c, &tvp5150_ops);
> > + sd->internal_ops = &tvp5150_internal_ops;
> > sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> >
> > -#if defined(CONFIG_MEDIA_CONTROLLER)
> > - core->pads[TVP5150_PAD_IF_INPUT].flags = MEDIA_PAD_FL_SINK;
> > - core->pads[TVP5150_PAD_IF_INPUT].sig_type = PAD_SIGNAL_ANALOG;
> > - core->pads[TVP5150_PAD_VID_OUT].flags = MEDIA_PAD_FL_SOURCE;
> > - core->pads[TVP5150_PAD_VID_OUT].sig_type = PAD_SIGNAL_DV;
> > -
> > - sd->entity.function = MEDIA_ENT_F_ATV_DECODER;
> > -
> > - res = media_entity_pads_init(&sd->entity, TVP5150_NUM_PADS, core->pads);
> > - if (res < 0)
> > - return res;
> > -
> > -#endif
> > + res = tvp5150_mc_init(sd);
> > + if (res)
> > + goto err_cleanup_dt;
> >
> > res = tvp5150_detect_version(core);
> > if (res < 0)
> > - return res;
> > + goto err_cleanup_dt;
> >
> > core->norm = V4L2_STD_ALL; /* Default is autodetect */
> > core->detected_norm = V4L2_STD_UNKNOWN;
> > @@ -1637,7 +1960,7 @@ static int tvp5150_probe(struct i2c_client *c,
> > sd->ctrl_handler = &core->hdl;
> > if (core->hdl.error) {
> > res = core->hdl.error;
> > - goto err;
> > + goto err_free_v4l2_ctrls;
> > }
> >
> > tvp5150_set_default(tvp5150_read_std(sd), &core->rect);
> > @@ -1649,19 +1972,24 @@ static int tvp5150_probe(struct i2c_client *c,
> > tvp5150_isr, IRQF_TRIGGER_HIGH |
> > IRQF_ONESHOT, "tvp5150", core);
> > if (res)
> > - goto err;
> > + goto err_free_v4l2_ctrls;
> > }
> >
> > res = v4l2_async_register_subdev(sd);
> > if (res < 0)
> > - goto err;
> > + goto err_free_v4l2_ctrls;
> >
> > if (debug > 1)
> > tvp5150_log_status(sd);
> > +
> > return 0;
> >
> > -err:
> > +err_free_v4l2_ctrls:
> > v4l2_ctrl_handler_free(&core->hdl);
> > +err_cleanup_dt:
> > + if (IS_ENABLED(CONFIG_OF) && np)
>
> is there any case where CONFIG_OF is enabled and dev->of_node might be
> NULL ? How did the driver probed in first place if that was the case.
I think that can be the case if you have an embedded device (of-support)
where a tvp5150 is connected directly and an attached em28xx usb device.
>
> I made to the end of this big patch :p quite a nice one!
Thanks for your feedback =)
> Thanks
> j
>
Regards,
Marco
> > + tvp5150_dt_cleanup(core);
> > +
> > return res;
> > }
> >
> > --
> > 2.20.1
> >
--
Pengutronix e.K. | |
Industrial Linux Solutions | http://www.pengutronix.de/ |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
^ permalink raw reply
* Re: [PATCH v2 3/3] rcar-csi2: Move setting of Field Detection Control Register
From: Hans Verkuil @ 2019-04-10 13:59 UTC (permalink / raw)
To: Niklas Söderlund, Laurent Pinchart, linux-media
Cc: linux-renesas-soc, Ulrich Hecht, Kieran Bingham
In-Reply-To: <20190308235702.27057-4-niklas.soderlund+renesas@ragnatech.se>
On 3/9/19 12:57 AM, Niklas Söderlund wrote:
> Latest datasheet (rev 1.50) clarifies that the FLD register should be
> set after LINKCNT.
>
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Ulrich Hecht <uli+renesas@fpond.eu>
> Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> ---
> drivers/media/platform/rcar-vin/rcar-csi2.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/rcar-vin/rcar-csi2.c b/drivers/media/platform/rcar-vin/rcar-csi2.c
> index 07d5c8c66b7cd382..077e0d344b395b54 100644
> --- a/drivers/media/platform/rcar-vin/rcar-csi2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-csi2.c
> @@ -529,7 +529,6 @@ static int rcsi2_start_receiver(struct rcar_csi2 *priv)
> rcsi2_write(priv, PHTC_REG, 0);
>
> /* Configure */
> - rcsi2_write(priv, FLD_REG, fld);
> rcsi2_write(priv, VCDT_REG, vcdt);
> if (vcdt2)
> rcsi2_write(priv, VCDT2_REG, vcdt2);
> @@ -560,6 +559,7 @@ static int rcsi2_start_receiver(struct rcar_csi2 *priv)
> rcsi2_write(priv, PHYCNT_REG, phycnt);
> rcsi2_write(priv, LINKCNT_REG, LINKCNT_MONITOR_EN |
> LINKCNT_REG_MONI_PACT_EN | LINKCNT_ICLK_NONSTOP);
> + rcsi2_write(priv, FLD_REG, fld);
> rcsi2_write(priv, PHYCNT_REG, phycnt | PHYCNT_SHUTDOWNZ);
> rcsi2_write(priv, PHYCNT_REG, phycnt | PHYCNT_SHUTDOWNZ | PHYCNT_RSTZ);
>
>
This patch doesn't apply because the fld variable no longer exists.
I think the fix is easy, but I prefer it if you rebase and repost this patch.
I've merged the others, so it's just this patch that's the problem.
Regards,
Hans
^ permalink raw reply
* Re: [PATCH 10/10] media: coda: require all decoder command flags to be cleared
From: Hans Verkuil @ 2019-04-10 13:53 UTC (permalink / raw)
To: Philipp Zabel, linux-media; +Cc: kernel
In-Reply-To: <1554829061.5799.7.camel@pengutronix.de>
On 4/9/19 6:57 PM, Philipp Zabel wrote:
> On Mon, 2019-04-08 at 14:32 +0200, Philipp Zabel wrote:
>> The memory-to-memory stateful video decoder interface documentation
>> requires the decoder stop command initiating the drain sequence to have
>> flags set to zero.
>> Stop to black makes no sense as stopped memory-to-memory decoders do not
>> produce any frames, and stopping immediately can be achieved by stopping
>> the output video queue with VIDIOC_STREAMOFF.
>>
>> The mute audio start command flag serves no purpose as the coda driver
>> does not handle any audio formats, and does not support playback at
>> non-standard speeds.
>>
>> Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
>> ---
>> drivers/media/platform/coda/coda-common.c | 4 ++--
>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
>> index 318f0be103bb..96798f98734a 100644
>> --- a/drivers/media/platform/coda/coda-common.c
>> +++ b/drivers/media/platform/coda/coda-common.c
>> @@ -1050,10 +1050,10 @@ static int coda_try_decoder_cmd(struct file *file, void *fh,
>> if (dc->cmd != V4L2_DEC_CMD_STOP)
>> return -EINVAL;
>>
>> - if (dc->flags & V4L2_DEC_CMD_STOP_TO_BLACK)
>> + if (dc->stop.pts != 0)
>> return -EINVAL;
>>
>> - if (!(dc->flags & V4L2_DEC_CMD_STOP_IMMEDIATELY) && (dc->stop.pts != 0))
>> + if (dc->flags != 0)
>> return -EINVAL;
>
> This change currently causes a v4l2-compliance failure
>
> fail: v4l2-test-codecs.cpp(104): ret != 0
> test VIDIOC_(TRY_)DECODER_CMD: FAIL
>
> because it still expects V4L2_DEC_CMD_STOP_IMMEDIATELY to be supported.
I've fixed this in my v4l-utils work-in-progress branch:
https://git.linuxtv.org/hverkuil/v4l-utils.git/log/?h=vicodec
But I want to hold off on this patch a little bit.
I think we need to add helpers for the try_en/decoder_cmd ioctls
to v4l2-mem2mem.c since it will be the same for all codecs.
I also had to fix vicodec since the checks it did weren't complete:
https://git.linuxtv.org/hverkuil/media_tree.git/commit/?h=vicodec&id=4e95bff9a15b63179fab20ac2113c238dc20665b
Regards,
Hans
^ permalink raw reply
* Re: [PATCH 08/10] media: coda: allow encoder to set colorimetry on the output queue
From: Hans Verkuil @ 2019-04-10 13:48 UTC (permalink / raw)
To: Philipp Zabel, linux-media; +Cc: kernel
In-Reply-To: <20190408123256.22868-8-p.zabel@pengutronix.de>
On 4/8/19 2:32 PM, Philipp Zabel wrote:
> v4l2-compliance sets colorimetry on the output queue and then verifies
> that querying colorimetry on the capture queue returns the same
> configuration. For this to work, the encoder must allow setting context
> colorimetry parameters on the output queue.
>
> Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
> ---
> drivers/media/platform/coda/coda-common.c | 10 +++++-----
> 1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
> index 2966eb1c4d2d..7b89dbae938d 100644
> --- a/drivers/media/platform/coda/coda-common.c
> +++ b/drivers/media/platform/coda/coda-common.c
> @@ -819,6 +819,11 @@ static int coda_s_fmt_vid_out(struct file *file, void *priv,
> if (ret)
> return ret;
>
> + ctx->colorspace = f->fmt.pix.colorspace;
> + ctx->xfer_func = f->fmt.pix.xfer_func;
> + ctx->ycbcr_enc = f->fmt.pix.ycbcr_enc;
> + ctx->quantization = f->fmt.pix.quantization;
> +
> if (ctx->inst_type != CODA_INST_DECODER)
> return 0;
>
> @@ -831,11 +836,6 @@ static int coda_s_fmt_vid_out(struct file *file, void *priv,
> }
> ctx->codec = codec;
>
> - ctx->colorspace = f->fmt.pix.colorspace;
> - ctx->xfer_func = f->fmt.pix.xfer_func;
> - ctx->ycbcr_enc = f->fmt.pix.ycbcr_enc;
> - ctx->quantization = f->fmt.pix.quantization;
> -
> dst_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
> if (!dst_vq)
> return -EINVAL;
>
Isn't the colorimetry information encoded in the stream's metadata? So should
it be set in an encoder register so it ends up in the right place in the bitstream?
And for decoders it should then be read back from a register.
Just curious how this would work for this codec and if it is even possible.
For now this patch is OK, but I think more work is needed.
Regards,
Hans
^ permalink raw reply
* Re: [PATCH v8 2/2] media: cedrus: Add H264 decoding support
From: Paul Kocialkowski @ 2019-04-10 13:45 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus,
Laurent Pinchart
Cc: tfiga, posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, jernej.skrabec, jonas,
ezequiel, linux-sunxi, Thomas Petazzoni, Jernej Skrabec
In-Reply-To: <157519b5571e24c9ef4189d30f8434b5b61121b1.1554382670.git-series.maxime.ripard@bootlin.com>
Hi,
Le jeudi 04 avril 2019 à 14:59 +0200, Maxime Ripard a écrit :
> Introduce some basic H264 decoding support in cedrus. So far, only the
> baseline profile videos have been tested, and some more advanced features
> used in higher profiles are not even implemented.
With the change to rename V4L2_PIX_FMT_H264_SLICE_RAW and make it
private, this is:
Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Cheers,
Paul
> Reviewed-by: Jernej Skrabec <jernej.skrabec@siol.net>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> drivers/staging/media/sunxi/cedrus/Makefile | 3 +-
> drivers/staging/media/sunxi/cedrus/cedrus.c | 31 +-
> drivers/staging/media/sunxi/cedrus/cedrus.h | 38 +-
> drivers/staging/media/sunxi/cedrus/cedrus_dec.c | 13 +-
> drivers/staging/media/sunxi/cedrus/cedrus_h264.c | 574 +++++++++++++++-
> drivers/staging/media/sunxi/cedrus/cedrus_hw.c | 4 +-
> drivers/staging/media/sunxi/cedrus/cedrus_regs.h | 91 ++-
> drivers/staging/media/sunxi/cedrus/cedrus_video.c | 9 +-
> 8 files changed, 761 insertions(+), 2 deletions(-)
> create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_h264.c
>
> diff --git a/drivers/staging/media/sunxi/cedrus/Makefile b/drivers/staging/media/sunxi/cedrus/Makefile
> index 808842f0119e..c85ac6db0302 100644
> --- a/drivers/staging/media/sunxi/cedrus/Makefile
> +++ b/drivers/staging/media/sunxi/cedrus/Makefile
> @@ -1,4 +1,5 @@
> # SPDX-License-Identifier: GPL-2.0
> obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi-cedrus.o
>
> -sunxi-cedrus-y = cedrus.o cedrus_video.o cedrus_hw.o cedrus_dec.o cedrus_mpeg2.o
> +sunxi-cedrus-y = cedrus.o cedrus_video.o cedrus_hw.o cedrus_dec.o \
> + cedrus_mpeg2.o cedrus_h264.o
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c
> index b98add3cdedd..d613f5c24a2f 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus.c
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c
> @@ -40,6 +40,36 @@ static const struct cedrus_control cedrus_controls[] = {
> .codec = CEDRUS_CODEC_MPEG2,
> .required = false,
> },
> + {
> + .id = V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS,
> + .elem_size = sizeof(struct v4l2_ctrl_h264_decode_params),
> + .codec = CEDRUS_CODEC_H264,
> + .required = true,
> + },
> + {
> + .id = V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS,
> + .elem_size = sizeof(struct v4l2_ctrl_h264_slice_params),
> + .codec = CEDRUS_CODEC_H264,
> + .required = true,
> + },
> + {
> + .id = V4L2_CID_MPEG_VIDEO_H264_SPS,
> + .elem_size = sizeof(struct v4l2_ctrl_h264_sps),
> + .codec = CEDRUS_CODEC_H264,
> + .required = true,
> + },
> + {
> + .id = V4L2_CID_MPEG_VIDEO_H264_PPS,
> + .elem_size = sizeof(struct v4l2_ctrl_h264_pps),
> + .codec = CEDRUS_CODEC_H264,
> + .required = true,
> + },
> + {
> + .id = V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX,
> + .elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix),
> + .codec = CEDRUS_CODEC_H264,
> + .required = true,
> + },
> };
>
> #define CEDRUS_CONTROLS_COUNT ARRAY_SIZE(cedrus_controls)
> @@ -278,6 +308,7 @@ static int cedrus_probe(struct platform_device *pdev)
> }
>
> dev->dec_ops[CEDRUS_CODEC_MPEG2] = &cedrus_dec_ops_mpeg2;
> + dev->dec_ops[CEDRUS_CODEC_H264] = &cedrus_dec_ops_h264;
>
> mutex_init(&dev->dev_mutex);
>
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.h b/drivers/staging/media/sunxi/cedrus/cedrus.h
> index c57c04b41d2e..bef79f630520 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus.h
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus.h
> @@ -32,7 +32,7 @@
>
> enum cedrus_codec {
> CEDRUS_CODEC_MPEG2,
> -
> + CEDRUS_CODEC_H264,
> CEDRUS_CODEC_LAST,
> };
>
> @@ -42,6 +42,12 @@ enum cedrus_irq_status {
> CEDRUS_IRQ_OK,
> };
>
> +enum cedrus_h264_pic_type {
> + CEDRUS_H264_PIC_TYPE_FRAME = 0,
> + CEDRUS_H264_PIC_TYPE_FIELD,
> + CEDRUS_H264_PIC_TYPE_MBAFF,
> +};
> +
> struct cedrus_control {
> u32 id;
> u32 elem_size;
> @@ -49,6 +55,14 @@ struct cedrus_control {
> unsigned char required:1;
> };
>
> +struct cedrus_h264_run {
> + const struct v4l2_ctrl_h264_decode_params *decode_params;
> + const struct v4l2_ctrl_h264_pps *pps;
> + const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix;
> + const struct v4l2_ctrl_h264_slice_params *slice_params;
> + const struct v4l2_ctrl_h264_sps *sps;
> +};
> +
> struct cedrus_mpeg2_run {
> const struct v4l2_ctrl_mpeg2_slice_params *slice_params;
> const struct v4l2_ctrl_mpeg2_quantization *quantization;
> @@ -59,12 +73,20 @@ struct cedrus_run {
> struct vb2_v4l2_buffer *dst;
>
> union {
> + struct cedrus_h264_run h264;
> struct cedrus_mpeg2_run mpeg2;
> };
> };
>
> struct cedrus_buffer {
> struct v4l2_m2m_buffer m2m_buf;
> +
> + union {
> + struct {
> + unsigned int position;
> + enum cedrus_h264_pic_type pic_type;
> + } h264;
> + } codec;
> };
>
> struct cedrus_ctx {
> @@ -79,6 +101,19 @@ struct cedrus_ctx {
> struct v4l2_ctrl **ctrls;
>
> struct vb2_buffer *dst_bufs[VIDEO_MAX_FRAME];
> +
> + union {
> + struct {
> + void *mv_col_buf;
> + dma_addr_t mv_col_buf_dma;
> + ssize_t mv_col_buf_field_size;
> + ssize_t mv_col_buf_size;
> + void *pic_info_buf;
> + dma_addr_t pic_info_buf_dma;
> + void *neighbor_info_buf;
> + dma_addr_t neighbor_info_buf_dma;
> + } h264;
> + } codec;
> };
>
> struct cedrus_dec_ops {
> @@ -121,6 +156,7 @@ struct cedrus_dev {
> };
>
> extern struct cedrus_dec_ops cedrus_dec_ops_mpeg2;
> +extern struct cedrus_dec_ops cedrus_dec_ops_h264;
>
> static inline void cedrus_write(struct cedrus_dev *dev, u32 reg, u32 val)
> {
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> index 4d6d602cdde6..bdad87eb9d79 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> @@ -46,6 +46,19 @@ void cedrus_device_run(void *priv)
> V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION);
> break;
>
> + case V4L2_PIX_FMT_H264_SLICE_RAW:
> + run.h264.decode_params = cedrus_find_control_data(ctx,
> + V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS);
> + run.h264.pps = cedrus_find_control_data(ctx,
> + V4L2_CID_MPEG_VIDEO_H264_PPS);
> + run.h264.scaling_matrix = cedrus_find_control_data(ctx,
> + V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX);
> + run.h264.slice_params = cedrus_find_control_data(ctx,
> + V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS);
> + run.h264.sps = cedrus_find_control_data(ctx,
> + V4L2_CID_MPEG_VIDEO_H264_SPS);
> + break;
> +
> default:
> break;
> }
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c
> new file mode 100644
> index 000000000000..2c98a3e46d2b
> --- /dev/null
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c
> @@ -0,0 +1,574 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Cedrus VPU driver
> + *
> + * Copyright (c) 2013 Jens Kuske <jenskuske@gmail.com>
> + * Copyright (c) 2018 Bootlin
> + */
> +
> +#include <linux/types.h>
> +
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "cedrus.h"
> +#include "cedrus_hw.h"
> +#include "cedrus_regs.h"
> +
> +enum cedrus_h264_sram_off {
> + CEDRUS_SRAM_H264_PRED_WEIGHT_TABLE = 0x000,
> + CEDRUS_SRAM_H264_FRAMEBUFFER_LIST = 0x100,
> + CEDRUS_SRAM_H264_REF_LIST_0 = 0x190,
> + CEDRUS_SRAM_H264_REF_LIST_1 = 0x199,
> + CEDRUS_SRAM_H264_SCALING_LIST_8x8_0 = 0x200,
> + CEDRUS_SRAM_H264_SCALING_LIST_8x8_1 = 0x210,
> + CEDRUS_SRAM_H264_SCALING_LIST_4x4 = 0x220,
> +};
> +
> +struct cedrus_h264_sram_ref_pic {
> + __le32 top_field_order_cnt;
> + __le32 bottom_field_order_cnt;
> + __le32 frame_info;
> + __le32 luma_ptr;
> + __le32 chroma_ptr;
> + __le32 mv_col_top_ptr;
> + __le32 mv_col_bot_ptr;
> + __le32 reserved;
> +} __packed;
> +
> +#define CEDRUS_H264_FRAME_NUM 18
> +
> +#define CEDRUS_NEIGHBOR_INFO_BUF_SIZE (16 * SZ_1K)
> +#define CEDRUS_PIC_INFO_BUF_SIZE (128 * SZ_1K)
> +
> +static void cedrus_h264_write_sram(struct cedrus_dev *dev,
> + enum cedrus_h264_sram_off off,
> + const void *data, size_t len)
> +{
> + const u32 *buffer = data;
> + size_t count = DIV_ROUND_UP(len, 4);
> +
> + cedrus_write(dev, VE_AVC_SRAM_PORT_OFFSET, off << 2);
> +
> + while (count--)
> + cedrus_write(dev, VE_AVC_SRAM_PORT_DATA, *buffer++);
> +}
> +
> +static dma_addr_t cedrus_h264_mv_col_buf_addr(struct cedrus_ctx *ctx,
> + unsigned int position,
> + unsigned int field)
> +{
> + dma_addr_t addr = ctx->codec.h264.mv_col_buf_dma;
> +
> + /* Adjust for the position */
> + addr += position * ctx->codec.h264.mv_col_buf_field_size * 2;
> +
> + /* Adjust for the field */
> + addr += field * ctx->codec.h264.mv_col_buf_field_size;
> +
> + return addr;
> +}
> +
> +static void cedrus_fill_ref_pic(struct cedrus_ctx *ctx,
> + struct cedrus_buffer *buf,
> + unsigned int top_field_order_cnt,
> + unsigned int bottom_field_order_cnt,
> + struct cedrus_h264_sram_ref_pic *pic)
> +{
> + struct vb2_buffer *vbuf = &buf->m2m_buf.vb.vb2_buf;
> + unsigned int position = buf->codec.h264.position;
> +
> + pic->top_field_order_cnt = top_field_order_cnt;
> + pic->bottom_field_order_cnt = bottom_field_order_cnt;
> + pic->frame_info = buf->codec.h264.pic_type << 8;
> +
> + pic->luma_ptr = cedrus_buf_addr(vbuf, &ctx->dst_fmt, 0);
> + pic->chroma_ptr = cedrus_buf_addr(vbuf, &ctx->dst_fmt, 1);
> + pic->mv_col_top_ptr = cedrus_h264_mv_col_buf_addr(ctx, position, 0);
> + pic->mv_col_bot_ptr = cedrus_h264_mv_col_buf_addr(ctx, position, 1);
> +}
> +
> +static void cedrus_write_frame_list(struct cedrus_ctx *ctx,
> + struct cedrus_run *run)
> +{
> + struct cedrus_h264_sram_ref_pic pic_list[CEDRUS_H264_FRAME_NUM];
> + const struct v4l2_ctrl_h264_decode_params *decode = run->h264.decode_params;
> + const struct v4l2_ctrl_h264_slice_params *slice = run->h264.slice_params;
> + const struct v4l2_ctrl_h264_sps *sps = run->h264.sps;
> + struct vb2_queue *cap_q = &ctx->fh.m2m_ctx->cap_q_ctx.q;
> + struct cedrus_buffer *output_buf;
> + struct cedrus_dev *dev = ctx->dev;
> + unsigned long used_dpbs = 0;
> + unsigned int position;
> + unsigned int output = 0;
> + unsigned int i;
> +
> + memset(pic_list, 0, sizeof(pic_list));
> +
> + for (i = 0; i < ARRAY_SIZE(decode->dpb); i++) {
> + const struct v4l2_h264_dpb_entry *dpb = &decode->dpb[i];
> + struct cedrus_buffer *cedrus_buf;
> + int buf_idx;
> +
> + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_VALID))
> + continue;
> +
> + buf_idx = vb2_find_timestamp(cap_q, dpb->reference_ts, 0);
> + if (buf_idx < 0)
> + continue;
> +
> + cedrus_buf = vb2_to_cedrus_buffer(ctx->dst_bufs[buf_idx]);
> + position = cedrus_buf->codec.h264.position;
> + used_dpbs |= BIT(position);
> +
> + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
> + continue;
> +
> + cedrus_fill_ref_pic(ctx, cedrus_buf,
> + dpb->top_field_order_cnt,
> + dpb->bottom_field_order_cnt,
> + &pic_list[position]);
> +
> + output = max(position, output);
> + }
> +
> + position = find_next_zero_bit(&used_dpbs, CEDRUS_H264_FRAME_NUM,
> + output);
> + if (position >= CEDRUS_H264_FRAME_NUM)
> + position = find_first_zero_bit(&used_dpbs, CEDRUS_H264_FRAME_NUM);
> +
> + output_buf = vb2_to_cedrus_buffer(&run->dst->vb2_buf);
> + output_buf->codec.h264.position = position;
> +
> + if (slice->flags & V4L2_H264_SLICE_FLAG_FIELD_PIC)
> + output_buf->codec.h264.pic_type = CEDRUS_H264_PIC_TYPE_FIELD;
> + else if (sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)
> + output_buf->codec.h264.pic_type = CEDRUS_H264_PIC_TYPE_MBAFF;
> + else
> + output_buf->codec.h264.pic_type = CEDRUS_H264_PIC_TYPE_FRAME;
> +
> + cedrus_fill_ref_pic(ctx, output_buf,
> + decode->top_field_order_cnt,
> + decode->bottom_field_order_cnt,
> + &pic_list[position]);
> +
> + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_FRAMEBUFFER_LIST,
> + pic_list, sizeof(pic_list));
> +
> + cedrus_write(dev, VE_H264_OUTPUT_FRAME_IDX, position);
> +}
> +
> +#define CEDRUS_MAX_REF_IDX 32
> +
> +static void _cedrus_write_ref_list(struct cedrus_ctx *ctx,
> + struct cedrus_run *run,
> + const u8 *ref_list, u8 num_ref,
> + enum cedrus_h264_sram_off sram)
> +{
> + const struct v4l2_ctrl_h264_decode_params *decode = run->h264.decode_params;
> + struct vb2_queue *cap_q = &ctx->fh.m2m_ctx->cap_q_ctx.q;
> + struct cedrus_dev *dev = ctx->dev;
> + u8 sram_array[CEDRUS_MAX_REF_IDX];
> + unsigned int i;
> + size_t size;
> +
> + memset(sram_array, 0, sizeof(sram_array));
> +
> + for (i = 0; i < num_ref; i++) {
> + const struct v4l2_h264_dpb_entry *dpb;
> + const struct cedrus_buffer *cedrus_buf;
> + const struct vb2_v4l2_buffer *ref_buf;
> + unsigned int position;
> + int buf_idx;
> + u8 dpb_idx;
> +
> + dpb_idx = ref_list[i];
> + dpb = &decode->dpb[dpb_idx];
> +
> + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
> + continue;
> +
> + buf_idx = vb2_find_timestamp(cap_q, dpb->reference_ts, 0);
> + if (buf_idx < 0)
> + continue;
> +
> + ref_buf = to_vb2_v4l2_buffer(ctx->dst_bufs[buf_idx]);
> + cedrus_buf = vb2_v4l2_to_cedrus_buffer(ref_buf);
> + position = cedrus_buf->codec.h264.position;
> +
> + sram_array[i] |= position << 1;
> + if (ref_buf->field == V4L2_FIELD_BOTTOM)
> + sram_array[i] |= BIT(0);
> + }
> +
> + size = min_t(size_t, ALIGN(num_ref, 4), sizeof(sram_array));
> + cedrus_h264_write_sram(dev, sram, &sram_array, size);
> +}
> +
> +static void cedrus_write_ref_list0(struct cedrus_ctx *ctx,
> + struct cedrus_run *run)
> +{
> + const struct v4l2_ctrl_h264_slice_params *slice = run->h264.slice_params;
> +
> + _cedrus_write_ref_list(ctx, run,
> + slice->ref_pic_list0,
> + slice->num_ref_idx_l0_active_minus1 + 1,
> + CEDRUS_SRAM_H264_REF_LIST_0);
> +}
> +
> +static void cedrus_write_ref_list1(struct cedrus_ctx *ctx,
> + struct cedrus_run *run)
> +{
> + const struct v4l2_ctrl_h264_slice_params *slice = run->h264.slice_params;
> +
> + _cedrus_write_ref_list(ctx, run,
> + slice->ref_pic_list1,
> + slice->num_ref_idx_l1_active_minus1 + 1,
> + CEDRUS_SRAM_H264_REF_LIST_1);
> +}
> +
> +static void cedrus_write_scaling_lists(struct cedrus_ctx *ctx,
> + struct cedrus_run *run)
> +{
> + const struct v4l2_ctrl_h264_scaling_matrix *scaling =
> + run->h264.scaling_matrix;
> + struct cedrus_dev *dev = ctx->dev;
> +
> + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_SCALING_LIST_8x8_0,
> + scaling->scaling_list_8x8[0],
> + sizeof(scaling->scaling_list_8x8[0]));
> +
> + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_SCALING_LIST_8x8_1,
> + scaling->scaling_list_8x8[3],
> + sizeof(scaling->scaling_list_8x8[3]));
> +
> + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_SCALING_LIST_4x4,
> + scaling->scaling_list_4x4,
> + sizeof(scaling->scaling_list_4x4));
> +}
> +
> +static void cedrus_write_pred_weight_table(struct cedrus_ctx *ctx,
> + struct cedrus_run *run)
> +{
> + const struct v4l2_ctrl_h264_slice_params *slice =
> + run->h264.slice_params;
> + const struct v4l2_h264_pred_weight_table *pred_weight =
> + &slice->pred_weight_table;
> + struct cedrus_dev *dev = ctx->dev;
> + int i, j, k;
> +
> + cedrus_write(dev, VE_H264_SHS_WP,
> + ((pred_weight->chroma_log2_weight_denom & 0x7) << 4) |
> + ((pred_weight->luma_log2_weight_denom & 0x7) << 0));
> +
> + cedrus_write(dev, VE_AVC_SRAM_PORT_OFFSET,
> + CEDRUS_SRAM_H264_PRED_WEIGHT_TABLE << 2);
> +
> + for (i = 0; i < ARRAY_SIZE(pred_weight->weight_factors); i++) {
> + const struct v4l2_h264_weight_factors *factors =
> + &pred_weight->weight_factors[i];
> +
> + for (j = 0; j < ARRAY_SIZE(factors->luma_weight); j++) {
> + u32 val;
> +
> + val = (((u32)factors->luma_offset[j] & 0x1ff) << 16) |
> + (factors->luma_weight[j] & 0x1ff);
> + cedrus_write(dev, VE_AVC_SRAM_PORT_DATA, val);
> + }
> +
> + for (j = 0; j < ARRAY_SIZE(factors->chroma_weight); j++) {
> + for (k = 0; k < ARRAY_SIZE(factors->chroma_weight[0]); k++) {
> + u32 val;
> +
> + val = (((u32)factors->chroma_offset[j][k] & 0x1ff) << 16) |
> + (factors->chroma_weight[j][k] & 0x1ff);
> + cedrus_write(dev, VE_AVC_SRAM_PORT_DATA, val);
> + }
> + }
> + }
> +}
> +
> +static void cedrus_set_params(struct cedrus_ctx *ctx,
> + struct cedrus_run *run)
> +{
> + const struct v4l2_ctrl_h264_decode_params *decode = run->h264.decode_params;
> + const struct v4l2_ctrl_h264_slice_params *slice = run->h264.slice_params;
> + const struct v4l2_ctrl_h264_pps *pps = run->h264.pps;
> + const struct v4l2_ctrl_h264_sps *sps = run->h264.sps;
> + struct vb2_buffer *src_buf = &run->src->vb2_buf;
> + struct cedrus_dev *dev = ctx->dev;
> + dma_addr_t src_buf_addr;
> + u32 offset = slice->header_bit_size;
> + u32 len = (slice->size * 8) - offset;
> + u32 reg;
> +
> + cedrus_write(dev, VE_H264_VLD_LEN, len);
> + cedrus_write(dev, VE_H264_VLD_OFFSET, offset);
> +
> + src_buf_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
> + cedrus_write(dev, VE_H264_VLD_END,
> + src_buf_addr + vb2_get_plane_payload(src_buf, 0));
> + cedrus_write(dev, VE_H264_VLD_ADDR,
> + VE_H264_VLD_ADDR_VAL(src_buf_addr) |
> + VE_H264_VLD_ADDR_FIRST | VE_H264_VLD_ADDR_VALID |
> + VE_H264_VLD_ADDR_LAST);
> +
> + /*
> + * FIXME: Since the bitstream parsing is done in software, and
> + * in userspace, this shouldn't be needed anymore. But it
> + * turns out that removing it breaks the decoding process,
> + * without any clear indication why.
> + */
> + cedrus_write(dev, VE_H264_TRIGGER_TYPE,
> + VE_H264_TRIGGER_TYPE_INIT_SWDEC);
> +
> + if (((pps->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED) &&
> + (slice->slice_type == V4L2_H264_SLICE_TYPE_P ||
> + slice->slice_type == V4L2_H264_SLICE_TYPE_SP)) ||
> + (pps->weighted_bipred_idc == 1 &&
> + slice->slice_type == V4L2_H264_SLICE_TYPE_B))
> + cedrus_write_pred_weight_table(ctx, run);
> +
> + if ((slice->slice_type == V4L2_H264_SLICE_TYPE_P) ||
> + (slice->slice_type == V4L2_H264_SLICE_TYPE_SP) ||
> + (slice->slice_type == V4L2_H264_SLICE_TYPE_B))
> + cedrus_write_ref_list0(ctx, run);
> +
> + if (slice->slice_type == V4L2_H264_SLICE_TYPE_B)
> + cedrus_write_ref_list1(ctx, run);
> +
> + // picture parameters
> + reg = 0;
> + /*
> + * FIXME: the kernel headers are allowing the default value to
> + * be passed, but the libva doesn't give us that.
> + */
> + reg |= (slice->num_ref_idx_l0_active_minus1 & 0x1f) << 10;
> + reg |= (slice->num_ref_idx_l1_active_minus1 & 0x1f) << 5;
> + reg |= (pps->weighted_bipred_idc & 0x3) << 2;
> + if (pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE)
> + reg |= VE_H264_PPS_ENTROPY_CODING_MODE;
> + if (pps->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED)
> + reg |= VE_H264_PPS_WEIGHTED_PRED;
> + if (pps->flags & V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED)
> + reg |= VE_H264_PPS_CONSTRAINED_INTRA_PRED;
> + if (pps->flags & V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE)
> + reg |= VE_H264_PPS_TRANSFORM_8X8_MODE;
> + cedrus_write(dev, VE_H264_PPS, reg);
> +
> + // sequence parameters
> + reg = 0;
> + reg |= (sps->chroma_format_idc & 0x7) << 19;
> + reg |= (sps->pic_width_in_mbs_minus1 & 0xff) << 8;
> + reg |= sps->pic_height_in_map_units_minus1 & 0xff;
> + if (sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY)
> + reg |= VE_H264_SPS_MBS_ONLY;
> + if (sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)
> + reg |= VE_H264_SPS_MB_ADAPTIVE_FRAME_FIELD;
> + if (sps->flags & V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE)
> + reg |= VE_H264_SPS_DIRECT_8X8_INFERENCE;
> + cedrus_write(dev, VE_H264_SPS, reg);
> +
> + // slice parameters
> + reg = 0;
> + reg |= decode->nal_ref_idc ? BIT(12) : 0;
> + reg |= (slice->slice_type & 0xf) << 8;
> + reg |= slice->cabac_init_idc & 0x3;
> + reg |= VE_H264_SHS_FIRST_SLICE_IN_PIC;
> + if (slice->flags & V4L2_H264_SLICE_FLAG_FIELD_PIC)
> + reg |= VE_H264_SHS_FIELD_PIC;
> + if (slice->flags & V4L2_H264_SLICE_FLAG_BOTTOM_FIELD)
> + reg |= VE_H264_SHS_BOTTOM_FIELD;
> + if (slice->flags & V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED)
> + reg |= VE_H264_SHS_DIRECT_SPATIAL_MV_PRED;
> + cedrus_write(dev, VE_H264_SHS, reg);
> +
> + reg = 0;
> + reg |= VE_H264_SHS2_NUM_REF_IDX_ACTIVE_OVRD;
> + reg |= (slice->num_ref_idx_l0_active_minus1 & 0x1f) << 24;
> + reg |= (slice->num_ref_idx_l1_active_minus1 & 0x1f) << 16;
> + reg |= (slice->disable_deblocking_filter_idc & 0x3) << 8;
> + reg |= (slice->slice_alpha_c0_offset_div2 & 0xf) << 4;
> + reg |= slice->slice_beta_offset_div2 & 0xf;
> + cedrus_write(dev, VE_H264_SHS2, reg);
> +
> + reg = 0;
> + reg |= (pps->second_chroma_qp_index_offset & 0x3f) << 16;
> + reg |= (pps->chroma_qp_index_offset & 0x3f) << 8;
> + reg |= (pps->pic_init_qp_minus26 + 26 + slice->slice_qp_delta) & 0x3f;
> + cedrus_write(dev, VE_H264_SHS_QP, reg);
> +
> + // clear status flags
> + cedrus_write(dev, VE_H264_STATUS, cedrus_read(dev, VE_H264_STATUS));
> +
> + // enable int
> + cedrus_write(dev, VE_H264_CTRL,
> + VE_H264_CTRL_SLICE_DECODE_INT |
> + VE_H264_CTRL_DECODE_ERR_INT |
> + VE_H264_CTRL_VLD_DATA_REQ_INT);
> +}
> +
> +static enum cedrus_irq_status
> +cedrus_h264_irq_status(struct cedrus_ctx *ctx)
> +{
> + struct cedrus_dev *dev = ctx->dev;
> + u32 reg = cedrus_read(dev, VE_H264_STATUS);
> +
> + if (reg & (VE_H264_STATUS_DECODE_ERR_INT |
> + VE_H264_STATUS_VLD_DATA_REQ_INT))
> + return CEDRUS_IRQ_ERROR;
> +
> + if (reg & VE_H264_CTRL_SLICE_DECODE_INT)
> + return CEDRUS_IRQ_OK;
> +
> + return CEDRUS_IRQ_NONE;
> +}
> +
> +static void cedrus_h264_irq_clear(struct cedrus_ctx *ctx)
> +{
> + struct cedrus_dev *dev = ctx->dev;
> +
> + cedrus_write(dev, VE_H264_STATUS,
> + VE_H264_STATUS_INT_MASK);
> +}
> +
> +static void cedrus_h264_irq_disable(struct cedrus_ctx *ctx)
> +{
> + struct cedrus_dev *dev = ctx->dev;
> + u32 reg = cedrus_read(dev, VE_H264_CTRL);
> +
> + cedrus_write(dev, VE_H264_CTRL,
> + reg & ~VE_H264_CTRL_INT_MASK);
> +}
> +
> +static void cedrus_h264_setup(struct cedrus_ctx *ctx,
> + struct cedrus_run *run)
> +{
> + struct cedrus_dev *dev = ctx->dev;
> +
> + cedrus_engine_enable(dev, CEDRUS_CODEC_H264);
> +
> + cedrus_write(dev, VE_H264_SDROT_CTRL, 0);
> + cedrus_write(dev, VE_H264_EXTRA_BUFFER1,
> + ctx->codec.h264.pic_info_buf_dma);
> + cedrus_write(dev, VE_H264_EXTRA_BUFFER2,
> + ctx->codec.h264.neighbor_info_buf_dma);
> +
> + cedrus_write_scaling_lists(ctx, run);
> + cedrus_write_frame_list(ctx, run);
> +
> + cedrus_set_params(ctx, run);
> +}
> +
> +static int cedrus_h264_start(struct cedrus_ctx *ctx)
> +{
> + struct cedrus_dev *dev = ctx->dev;
> + unsigned int field_size;
> + unsigned int mv_col_size;
> + int ret;
> +
> + /*
> + * FIXME: It seems that the H6 cedarX code is using a formula
> + * here based on the size of the frame, while all the older
> + * code is using a fixed size, so that might need to be
> + * changed at some point.
> + */
> + ctx->codec.h264.pic_info_buf =
> + dma_alloc_coherent(dev->dev, CEDRUS_PIC_INFO_BUF_SIZE,
> + &ctx->codec.h264.pic_info_buf_dma,
> + GFP_KERNEL);
> + if (!ctx->codec.h264.pic_info_buf)
> + return -ENOMEM;
> +
> + /*
> + * That buffer is supposed to be 16kiB in size, and be aligned
> + * on 16kiB as well. However, dma_alloc_coherent provides the
> + * guarantee that we'll have a CPU and DMA address aligned on
> + * the smallest page order that is greater to the requested
> + * size, so we don't have to overallocate.
> + */
> + ctx->codec.h264.neighbor_info_buf =
> + dma_alloc_coherent(dev->dev, CEDRUS_NEIGHBOR_INFO_BUF_SIZE,
> + &ctx->codec.h264.neighbor_info_buf_dma,
> + GFP_KERNEL);
> + if (!ctx->codec.h264.neighbor_info_buf) {
> + ret = -ENOMEM;
> + goto err_pic_buf;
> + }
> +
> + field_size = DIV_ROUND_UP(ctx->src_fmt.width, 16) *
> + DIV_ROUND_UP(ctx->src_fmt.height, 16) * 16;
> +
> + /*
> + * FIXME: This is actually conditional to
> + * V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE not being set, we
> + * might have to rework this if memory efficiency ever is
> + * something we need to work on.
> + */
> + field_size = field_size * 2;
> +
> + /*
> + * FIXME: This is actually conditional to
> + * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY not being set, we might
> + * have to rework this if memory efficiency ever is something
> + * we need to work on.
> + */
> + field_size = field_size * 2;
> + ctx->codec.h264.mv_col_buf_field_size = field_size;
> +
> + mv_col_size = field_size * 2 * CEDRUS_H264_FRAME_NUM;
> + ctx->codec.h264.mv_col_buf_size = mv_col_size;
> + ctx->codec.h264.mv_col_buf = dma_alloc_coherent(dev->dev,
> + ctx->codec.h264.mv_col_buf_size,
> + &ctx->codec.h264.mv_col_buf_dma,
> + GFP_KERNEL);
> + if (!ctx->codec.h264.mv_col_buf) {
> + ret = -ENOMEM;
> + goto err_neighbor_buf;
> + }
> +
> + return 0;
> +
> +err_neighbor_buf:
> + dma_free_coherent(dev->dev, CEDRUS_NEIGHBOR_INFO_BUF_SIZE,
> + ctx->codec.h264.neighbor_info_buf,
> + ctx->codec.h264.neighbor_info_buf_dma);
> +
> +err_pic_buf:
> + dma_free_coherent(dev->dev, CEDRUS_PIC_INFO_BUF_SIZE,
> + ctx->codec.h264.pic_info_buf,
> + ctx->codec.h264.pic_info_buf_dma);
> + return ret;
> +}
> +
> +static void cedrus_h264_stop(struct cedrus_ctx *ctx)
> +{
> + struct cedrus_dev *dev = ctx->dev;
> +
> + dma_free_coherent(dev->dev, ctx->codec.h264.mv_col_buf_size,
> + ctx->codec.h264.mv_col_buf,
> + ctx->codec.h264.mv_col_buf_dma);
> + dma_free_coherent(dev->dev, CEDRUS_NEIGHBOR_INFO_BUF_SIZE,
> + ctx->codec.h264.neighbor_info_buf,
> + ctx->codec.h264.neighbor_info_buf_dma);
> + dma_free_coherent(dev->dev, CEDRUS_PIC_INFO_BUF_SIZE,
> + ctx->codec.h264.pic_info_buf,
> + ctx->codec.h264.pic_info_buf_dma);
> +}
> +
> +static void cedrus_h264_trigger(struct cedrus_ctx *ctx)
> +{
> + struct cedrus_dev *dev = ctx->dev;
> +
> + cedrus_write(dev, VE_H264_TRIGGER_TYPE,
> + VE_H264_TRIGGER_TYPE_AVC_SLICE_DECODE);
> +}
> +
> +struct cedrus_dec_ops cedrus_dec_ops_h264 = {
> + .irq_clear = cedrus_h264_irq_clear,
> + .irq_disable = cedrus_h264_irq_disable,
> + .irq_status = cedrus_h264_irq_status,
> + .setup = cedrus_h264_setup,
> + .start = cedrus_h264_start,
> + .stop = cedrus_h264_stop,
> + .trigger = cedrus_h264_trigger,
> +};
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
> index fbfff7c1c771..748f7f673547 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
> @@ -46,6 +46,10 @@ int cedrus_engine_enable(struct cedrus_dev *dev, enum cedrus_codec codec)
> reg |= VE_MODE_DEC_MPEG;
> break;
>
> + case CEDRUS_CODEC_H264:
> + reg |= VE_MODE_DEC_H264;
> + break;
> +
> default:
> return -EINVAL;
> }
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h
> index de2d6b6f64bf..3e9931416e45 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h
> @@ -232,4 +232,95 @@
> #define VE_DEC_MPEG_ROT_LUMA (VE_ENGINE_DEC_MPEG + 0xcc)
> #define VE_DEC_MPEG_ROT_CHROMA (VE_ENGINE_DEC_MPEG + 0xd0)
>
> +#define VE_H264_SPS 0x200
> +#define VE_H264_SPS_MBS_ONLY BIT(18)
> +#define VE_H264_SPS_MB_ADAPTIVE_FRAME_FIELD BIT(17)
> +#define VE_H264_SPS_DIRECT_8X8_INFERENCE BIT(16)
> +
> +#define VE_H264_PPS 0x204
> +#define VE_H264_PPS_ENTROPY_CODING_MODE BIT(15)
> +#define VE_H264_PPS_WEIGHTED_PRED BIT(4)
> +#define VE_H264_PPS_CONSTRAINED_INTRA_PRED BIT(1)
> +#define VE_H264_PPS_TRANSFORM_8X8_MODE BIT(0)
> +
> +#define VE_H264_SHS 0x208
> +#define VE_H264_SHS_FIRST_SLICE_IN_PIC BIT(5)
> +#define VE_H264_SHS_FIELD_PIC BIT(4)
> +#define VE_H264_SHS_BOTTOM_FIELD BIT(3)
> +#define VE_H264_SHS_DIRECT_SPATIAL_MV_PRED BIT(2)
> +
> +#define VE_H264_SHS2 0x20c
> +#define VE_H264_SHS2_NUM_REF_IDX_ACTIVE_OVRD BIT(12)
> +
> +#define VE_H264_SHS_WP 0x210
> +
> +#define VE_H264_SHS_QP 0x21c
> +#define VE_H264_SHS_QP_SCALING_MATRIX_DEFAULT BIT(24)
> +
> +#define VE_H264_CTRL 0x220
> +#define VE_H264_CTRL_VLD_DATA_REQ_INT BIT(2)
> +#define VE_H264_CTRL_DECODE_ERR_INT BIT(1)
> +#define VE_H264_CTRL_SLICE_DECODE_INT BIT(0)
> +
> +#define VE_H264_CTRL_INT_MASK (VE_H264_CTRL_VLD_DATA_REQ_INT | \
> + VE_H264_CTRL_DECODE_ERR_INT | \
> + VE_H264_CTRL_SLICE_DECODE_INT)
> +
> +#define VE_H264_TRIGGER_TYPE 0x224
> +#define VE_H264_TRIGGER_TYPE_AVC_SLICE_DECODE (8 << 0)
> +#define VE_H264_TRIGGER_TYPE_INIT_SWDEC (7 << 0)
> +
> +#define VE_H264_STATUS 0x228
> +#define VE_H264_STATUS_VLD_DATA_REQ_INT VE_H264_CTRL_VLD_DATA_REQ_INT
> +#define VE_H264_STATUS_DECODE_ERR_INT VE_H264_CTRL_DECODE_ERR_INT
> +#define VE_H264_STATUS_SLICE_DECODE_INT VE_H264_CTRL_SLICE_DECODE_INT
> +
> +#define VE_H264_STATUS_INT_MASK VE_H264_CTRL_INT_MASK
> +
> +#define VE_H264_CUR_MB_NUM 0x22c
> +
> +#define VE_H264_VLD_ADDR 0x230
> +#define VE_H264_VLD_ADDR_FIRST BIT(30)
> +#define VE_H264_VLD_ADDR_LAST BIT(29)
> +#define VE_H264_VLD_ADDR_VALID BIT(28)
> +#define VE_H264_VLD_ADDR_VAL(x) (((x) & 0x0ffffff0) | ((x) >> 28))
> +
> +#define VE_H264_VLD_OFFSET 0x234
> +#define VE_H264_VLD_LEN 0x238
> +#define VE_H264_VLD_END 0x23c
> +#define VE_H264_SDROT_CTRL 0x240
> +#define VE_H264_OUTPUT_FRAME_IDX 0x24c
> +#define VE_H264_EXTRA_BUFFER1 0x250
> +#define VE_H264_EXTRA_BUFFER2 0x254
> +#define VE_H264_BASIC_BITS 0x2dc
> +#define VE_AVC_SRAM_PORT_OFFSET 0x2e0
> +#define VE_AVC_SRAM_PORT_DATA 0x2e4
> +
> +#define VE_ISP_INPUT_SIZE 0xa00
> +#define VE_ISP_INPUT_STRIDE 0xa04
> +#define VE_ISP_CTRL 0xa08
> +#define VE_ISP_INPUT_LUMA 0xa78
> +#define VE_ISP_INPUT_CHROMA 0xa7c
> +
> +#define VE_AVC_PARAM 0xb04
> +#define VE_AVC_QP 0xb08
> +#define VE_AVC_MOTION_EST 0xb10
> +#define VE_AVC_CTRL 0xb14
> +#define VE_AVC_TRIGGER 0xb18
> +#define VE_AVC_STATUS 0xb1c
> +#define VE_AVC_BASIC_BITS 0xb20
> +#define VE_AVC_UNK_BUF 0xb60
> +#define VE_AVC_VLE_ADDR 0xb80
> +#define VE_AVC_VLE_END 0xb84
> +#define VE_AVC_VLE_OFFSET 0xb88
> +#define VE_AVC_VLE_MAX 0xb8c
> +#define VE_AVC_VLE_LENGTH 0xb90
> +#define VE_AVC_REF_LUMA 0xba0
> +#define VE_AVC_REF_CHROMA 0xba4
> +#define VE_AVC_REC_LUMA 0xbb0
> +#define VE_AVC_REC_CHROMA 0xbb4
> +#define VE_AVC_REF_SLUMA 0xbb8
> +#define VE_AVC_REC_SLUMA 0xbbc
> +#define VE_AVC_MB_INFO 0xbc0
> +
> #endif
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_video.c b/drivers/staging/media/sunxi/cedrus/cedrus_video.c
> index 9673874ece10..e2b530b1a956 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus_video.c
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus_video.c
> @@ -38,6 +38,10 @@ static struct cedrus_format cedrus_formats[] = {
> .directions = CEDRUS_DECODE_SRC,
> },
> {
> + .pixelformat = V4L2_PIX_FMT_H264_SLICE_RAW,
> + .directions = CEDRUS_DECODE_SRC,
> + },
> + {
> .pixelformat = V4L2_PIX_FMT_SUNXI_TILED_NV12,
> .directions = CEDRUS_DECODE_DST,
> },
> @@ -100,6 +104,7 @@ static void cedrus_prepare_format(struct v4l2_pix_format *pix_fmt)
>
> switch (pix_fmt->pixelformat) {
> case V4L2_PIX_FMT_MPEG2_SLICE:
> + case V4L2_PIX_FMT_H264_SLICE_RAW:
> /* Zero bytes per line for encoded source. */
> bytesperline = 0;
>
> @@ -464,6 +469,10 @@ static int cedrus_start_streaming(struct vb2_queue *vq, unsigned int count)
> ctx->current_codec = CEDRUS_CODEC_MPEG2;
> break;
>
> + case V4L2_PIX_FMT_H264_SLICE_RAW:
> + ctx->current_codec = CEDRUS_CODEC_H264;
> + break;
> +
> default:
> return -EINVAL;
> }
--
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
^ permalink raw reply
* Re: [PATCH v8 1/2] media: uapi: Add H264 low-level decoder API compound controls.
From: Paul Kocialkowski @ 2019-04-10 13:44 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus,
Laurent Pinchart
Cc: tfiga, posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, jernej.skrabec, jonas,
ezequiel, linux-sunxi, Thomas Petazzoni, Guenter Roeck
In-Reply-To: <f537c99d8b5a974f8687f4f0d018c025a4220f7a.1554382670.git-series.maxime.ripard@bootlin.com>
Hi,
Le jeudi 04 avril 2019 à 14:59 +0200, Maxime Ripard a écrit :
> From: Pawel Osciak <posciak@chromium.org>
>
> Stateless video codecs will require both the H264 metadata and slices in
> order to be able to decode frames.
>
> This introduces the definitions for a new pixel format for H264 slices that
> have been parsed, as well as the structures used to pass the metadata from
> the userspace to the kernel.
With V4L2_PIX_FMT_H264_SLICE_RAW renamed to V4L2_PIX_FMT_H264_SLICE and
moved to the private h264-ctrls.h, this is:
Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Cheers,
Paul
> Reviewed-by: Tomasz Figa <tfiga@chromium.org>
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> Signed-off-by: Guenter Roeck <groeck@chromium.org>
> Co-developed-by: Maxime Ripard <maxime.ripard@bootlin.com>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> Documentation/media/uapi/v4l/biblio.rst | 9 +-
> Documentation/media/uapi/v4l/ext-ctrls-codec.rst | 569 ++++++++++++++-
> Documentation/media/uapi/v4l/pixfmt-compressed.rst | 19 +-
> Documentation/media/uapi/v4l/vidioc-queryctrl.rst | 30 +-
> Documentation/media/videodev2.h.rst.exceptions | 5 +-
> drivers/media/v4l2-core/v4l2-ctrls.c | 42 +-
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +-
> include/media/h264-ctrls.h | 192 +++++-
> include/media/v4l2-ctrls.h | 13 +-
> include/uapi/linux/videodev2.h | 1 +-
> 10 files changed, 880 insertions(+), 1 deletion(-)
> create mode 100644 include/media/h264-ctrls.h
>
> diff --git a/Documentation/media/uapi/v4l/biblio.rst b/Documentation/media/uapi/v4l/biblio.rst
> index ec33768c055e..8f4eb8823d82 100644
> --- a/Documentation/media/uapi/v4l/biblio.rst
> +++ b/Documentation/media/uapi/v4l/biblio.rst
> @@ -122,6 +122,15 @@ ITU BT.1119
>
> :author: International Telecommunication Union (http://www.itu.ch)
>
> +.. _h264:
> +
> +ITU-T Rec. H.264 Specification (04/2017 Edition)
> +================================================
> +
> +:title: ITU-T Recommendation H.264 "Advanced Video Coding for Generic Audiovisual Services"
> +
> +:author: International Telecommunication Union (http://www.itu.ch)
> +
> .. _jfif:
>
> JFIF
> diff --git a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst
> index 67a122339c0e..fe720f239f70 100644
> --- a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst
> +++ b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst
> @@ -1371,6 +1371,575 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
> - Layer number
>
>
> +.. _v4l2-mpeg-h264:
> +
> +``V4L2_CID_MPEG_VIDEO_H264_SPS (struct)``
> + Specifies the sequence parameter set (as extracted from the
> + bitstream) for the associated H264 slice data. This includes the
> + necessary parameters for configuring a stateless hardware decoding
> + pipeline for H264. The bitstream parameters are defined according
> + to :ref:`h264`, section 7.4.2.1.1 "Sequence Parameter Set Data
> + Semantics". For further documentation, refer to the above
> + specification, unless there is an explicit comment stating
> + otherwise.
> +
> + .. note::
> +
> + This compound control is not yet part of the public kernel API and
> + it is expected to change.
> +
> +.. c:type:: v4l2_ctrl_h264_sps
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_ctrl_h264_sps
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u8
> + - ``profile_idc``
> + -
> + * - __u8
> + - ``constraint_set_flags``
> + - See :ref:`Sequence Parameter Set Constraints Set Flags <h264_sps_constraints_set_flags>`
> + * - __u8
> + - ``level_idc``
> + -
> + * - __u8
> + - ``seq_parameter_set_id``
> + -
> + * - __u8
> + - ``chroma_format_idc``
> + -
> + * - __u8
> + - ``bit_depth_luma_minus8``
> + -
> + * - __u8
> + - ``bit_depth_chroma_minus8``
> + -
> + * - __u8
> + - ``log2_max_frame_num_minus4``
> + -
> + * - __u8
> + - ``pic_order_cnt_type``
> + -
> + * - __u8
> + - ``log2_max_pic_order_cnt_lsb_minus4``
> + -
> + * - __u8
> + - ``max_num_ref_frames``
> + -
> + * - __u8
> + - ``num_ref_frames_in_pic_order_cnt_cycle``
> + -
> + * - __s32
> + - ``offset_for_ref_frame[255]``
> + -
> + * - __s32
> + - ``offset_for_non_ref_pic``
> + -
> + * - __s32
> + - ``offset_for_top_to_bottom_field``
> + -
> + * - __u16
> + - ``pic_width_in_mbs_minus1``
> + -
> + * - __u16
> + - ``pic_height_in_map_units_minus1``
> + -
> + * - __u32
> + - ``flags``
> + - See :ref:`Sequence Parameter Set Flags <h264_sps_flags>`
> +
> +.. _h264_sps_constraints_set_flags:
> +
> +``Sequence Parameter Set Constraints Set Flags``
> +
> +.. cssclass:: longtable
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - ``V4L2_H264_SPS_CONSTRAINT_SET0_FLAG``
> + - 0x00000001
> + -
> + * - ``V4L2_H264_SPS_CONSTRAINT_SET1_FLAG``
> + - 0x00000002
> + -
> + * - ``V4L2_H264_SPS_CONSTRAINT_SET2_FLAG``
> + - 0x00000004
> + -
> + * - ``V4L2_H264_SPS_CONSTRAINT_SET3_FLAG``
> + - 0x00000008
> + -
> + * - ``V4L2_H264_SPS_CONSTRAINT_SET4_FLAG``
> + - 0x00000010
> + -
> + * - ``V4L2_H264_SPS_CONSTRAINT_SET5_FLAG``
> + - 0x00000020
> + -
> +
> +.. _h264_sps_flags:
> +
> +``Sequence Parameter Set Flags``
> +
> +.. cssclass:: longtable
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - ``V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE``
> + - 0x00000001
> + -
> + * - ``V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS``
> + - 0x00000002
> + -
> + * - ``V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO``
> + - 0x00000004
> + -
> + * - ``V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED``
> + - 0x00000008
> + -
> + * - ``V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY``
> + - 0x00000010
> + -
> + * - ``V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD``
> + - 0x00000020
> + -
> + * - ``V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE``
> + - 0x00000040
> + -
> +
> +``V4L2_CID_MPEG_VIDEO_H264_PPS (struct)``
> + Specifies the picture parameter set (as extracted from the
> + bitstream) for the associated H264 slice data. This includes the
> + necessary parameters for configuring a stateless hardware decoding
> + pipeline for H264. The bitstream parameters are defined according
> + to :ref:`h264`, section 7.4.2.2 "Picture Parameter Set RBSP
> + Semantics". For further documentation, refer to the above
> + specification, unless there is an explicit comment stating
> + otherwise.
> +
> + .. note::
> +
> + This compound control is not yet part of the public kernel API and
> + it is expected to change.
> +
> +.. c:type:: v4l2_ctrl_h264_pps
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_ctrl_h264_pps
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u8
> + - ``pic_parameter_set_id``
> + -
> + * - __u8
> + - ``seq_parameter_set_id``
> + -
> + * - __u8
> + - ``num_slice_groups_minus1``
> + -
> + * - __u8
> + - ``num_ref_idx_l0_default_active_minus1``
> + -
> + * - __u8
> + - ``num_ref_idx_l1_default_active_minus1``
> + -
> + * - __u8
> + - ``weighted_bipred_idc``
> + -
> + * - __s8
> + - ``pic_init_qp_minus26``
> + -
> + * - __s8
> + - ``pic_init_qs_minus26``
> + -
> + * - __s8
> + - ``chroma_qp_index_offset``
> + -
> + * - __s8
> + - ``second_chroma_qp_index_offset``
> + -
> + * - __u16
> + - ``flags``
> + - See :ref:`Picture Parameter Set Flags <h264_pps_flags>`
> +
> +.. _h264_pps_flags:
> +
> +``Picture Parameter Set Flags``
> +
> +.. cssclass:: longtable
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - ``V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE``
> + - 0x00000001
> + -
> + * - ``V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT``
> + - 0x00000002
> + -
> + * - ``V4L2_H264_PPS_FLAG_WEIGHTED_PRED``
> + - 0x00000004
> + -
> + * - ``V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT``
> + - 0x00000008
> + -
> + * - ``V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED``
> + - 0x00000010
> + -
> + * - ``V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT``
> + - 0x00000020
> + -
> + * - ``V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE``
> + - 0x00000040
> + -
> + * - ``V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT``
> + - 0x00000080
> + -
> +
> +``V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (struct)``
> + Specifies the scaling matrix (as extracted from the bitstream) for
> + the associated H264 slice data. The bitstream parameters are
> + defined according to :ref:`h264`, section 7.4.2.1.1.1 "Scaling
> + List Semantics".For further documentation, refer to the above
> + specification, unless there is an explicit comment stating
> + otherwise.
> +
> + .. note::
> +
> + This compound control is not yet part of the public kernel API and
> + it is expected to change.
> +
> +.. c:type:: v4l2_ctrl_h264_scaling_matrix
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_ctrl_h264_scaling_matrix
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u8
> + - ``scaling_list_4x4[6][16]``
> + -
> + * - __u8
> + - ``scaling_list_8x8[6][64]``
> + -
> +
> +``V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS (struct)``
> + Specifies the slice parameters (as extracted from the bitstream)
> + for the associated H264 slice data. This includes the necessary
> + parameters for configuring a stateless hardware decoding pipeline
> + for H264. The bitstream parameters are defined according to
> + :ref:`h264`, section 7.4.3 "Slice Header Semantics". For further
> + documentation, refer to the above specification, unless there is
> + an explicit comment stating otherwise.
> +
> + .. note::
> +
> + This compound control is not yet part of the public kernel API
> + and it is expected to change.
> +
> + This structure is expected to be passed as an array, with one
> + entry for each slice included in the bitstream buffer.
> +
> +.. c:type:: v4l2_ctrl_h264_slice_params
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_ctrl_h264_slice_params
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u32
> + - ``size``
> + -
> + * - __u32
> + - ``header_bit_size``
> + -
> + * - __u16
> + - ``first_mb_in_slice``
> + -
> + * - __u8
> + - ``slice_type``
> + -
> + * - __u8
> + - ``pic_parameter_set_id``
> + -
> + * - __u8
> + - ``colour_plane_id``
> + -
> + * - __u8
> + - ``redundant_pic_cnt``
> + -
> + * - __u16
> + - ``frame_num``
> + -
> + * - __u16
> + - ``idr_pic_id``
> + -
> + * - __u16
> + - ``pic_order_cnt_lsb``
> + -
> + * - __s32
> + - ``delta_pic_order_cnt_bottom``
> + -
> + * - __s32
> + - ``delta_pic_order_cnt0``
> + -
> + * - __s32
> + - ``delta_pic_order_cnt1``
> + -
> + * - struct :c:type:`v4l2_h264_pred_weight_table`
> + - ``pred_weight_table``
> + -
> + * - __u32
> + - ``dec_ref_pic_marking_bit_size``
> + -
> + * - __u32
> + - ``pic_order_cnt_bit_size``
> + -
> + * - __u8
> + - ``cabac_init_idc``
> + -
> + * - __s8
> + - ``slice_qp_delta``
> + -
> + * - __s8
> + - ``slice_qs_delta``
> + -
> + * - __u8
> + - ``disable_deblocking_filter_idc``
> + -
> + * - __s8
> + - ``slice_alpha_c0_offset_div2``
> + -
> + * - __s8
> + - ``slice_beta_offset_div2``
> + -
> + * - __u8
> + - ``num_ref_idx_l0_active_minus1``
> + -
> + * - __u8
> + - ``num_ref_idx_l1_active_minus1``
> + -
> + * - __u32
> + - ``slice_group_change_cycle``
> + -
> + * - __u8
> + - ``ref_pic_list0[32]``
> + - Reference picture list after applying the per-slice modifications
> + * - __u8
> + - ``ref_pic_list1[32]``
> + - Reference picture list after applying the per-slice modifications
> + * - __u32
> + - ``flags``
> + - See :ref:`Slice Parameter Flags <h264_slice_flags>`
> +
> +.. _h264_slice_flags:
> +
> +``Slice Parameter Set Flags``
> +
> +.. cssclass:: longtable
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - ``V4L2_H264_SLICE_FLAG_FIELD_PIC``
> + - 0x00000001
> + -
> + * - ``V4L2_H264_SLICE_FLAG_BOTTOM_FIELD``
> + - 0x00000002
> + -
> + * - ``V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED``
> + - 0x00000004
> + -
> + * - ``V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH``
> + - 0x00000008
> + -
> +
> +``Prediction Weight Table``
> +
> + The bitstream parameters are defined according to :ref:`h264`,
> + section 7.4.3.2 "Prediction Weight Table Semantics". For further
> + documentation, refer to the above specification, unless there is
> + an explicit comment stating otherwise.
> +
> +.. c:type:: v4l2_h264_pred_weight_table
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_h264_pred_weight_table
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u16
> + - ``luma_log2_weight_denom``
> + -
> + * - __u16
> + - ``chroma_log2_weight_denom``
> + -
> + * - struct :c:type:`v4l2_h264_weight_factors`
> + - ``weight_factors[2]``
> + - The weight factors at index 0 are the weight factors for the reference
> + list 0, the one at index 1 for the reference list 1.
> +
> +.. c:type:: v4l2_h264_weight_factors
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_h264_weight_factors
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __s16
> + - ``luma_weight[32]``
> + -
> + * - __s16
> + - ``luma_offset[32]``
> + -
> + * - __s16
> + - ``chroma_weight[32][2]``
> + -
> + * - __s16
> + - ``chroma_offset[32][2]``
> + -
> +
> +``V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (struct)``
> + Specifies the decode parameters (as extracted from the bitstream)
> + for the associated H264 slice data. This includes the necessary
> + parameters for configuring a stateless hardware decoding pipeline
> + for H264. The bitstream parameters are defined according to
> + :ref:`h264`. For further documentation, refer to the above
> + specification, unless there is an explicit comment stating
> + otherwise.
> +
> + .. note::
> +
> + This compound control is not yet part of the public kernel API and
> + it is expected to change.
> +
> +.. c:type:: v4l2_ctrl_h264_decode_params
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_ctrl_h264_decode_params
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u32
> + - ``num_slices``
> + - Number of slices needed to decode the current frame
> + * - __u32
> + - ``nal_ref_idc``
> + - NAL reference ID value coming from the NAL Unit header
> + * - __u8
> + - ``ref_pic_list_p0[32]``
> + - Backward reference list used by P-frames in the original bitstream order
> + * - __u8
> + - ``ref_pic_list_b0[32]``
> + - Backward reference list used by B-frames in the original bitstream order
> + * - __u8
> + - ``ref_pic_list_b1[32]``
> + - Forward reference list used by B-frames in the original bitstream order
> + * - __s32
> + - ``top_field_order_cnt``
> + - Picture Order Count for the coded top field
> + * - __s32
> + - ``bottom_field_order_cnt``
> + - Picture Order Count for the coded bottom field
> + * - __u32
> + - ``flags``
> + - See :ref:`Decode Parameters Flags <h264_decode_params_flags>`
> + * - struct :c:type:`v4l2_h264_dpb_entry`
> + - ``dpb[16]``
> + -
> +
> +.. _h264_decode_params_flags:
> +
> +``Decode Parameters Flags``
> +
> +.. cssclass:: longtable
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - ``V4L2_H264_DECODE_PARAM_FLAG_IDR_PIC``
> + - 0x00000001
> + - That picture is an IDR picture
> +
> +.. c:type:: v4l2_h264_dpb_entry
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_h264_dpb_entry
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u64
> + - ``reference_ts``
> + - Timestamp of the V4L2 capture buffer to use as reference, used
> + with B-coded and P-coded frames. The timestamp refers to the
> + ``timestamp`` field in struct :c:type:`v4l2_buffer`. Use the
> + :c:func:`v4l2_timeval_to_ns()` function to convert the struct
> + :c:type:`timeval` in struct :c:type:`v4l2_buffer` to a __u64.
> + * - __u16
> + - ``frame_num``
> + -
> + * - __u16
> + - ``pic_num``
> + -
> + * - __s32
> + - ``top_field_order_cnt``
> + -
> + * - __s32
> + - ``bottom_field_order_cnt``
> + -
> + * - __u32
> + - ``flags``
> + - See :ref:`DPB Entry Flags <h264_dpb_flags>`
> +
> +.. _h264_dpb_flags:
> +
> +``DPB Entries Flags``
> +
> +.. cssclass:: longtable
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - ``V4L2_H264_DPB_ENTRY_FLAG_VALID``
> + - 0x00000001
> + - The DPB entry is valid and should be considered
> + * - ``V4L2_H264_DPB_ENTRY_FLAG_ACTIVE``
> + - 0x00000002
> + - The DPB entry is currently being used as a reference frame
> + * - ``V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM``
> + - 0x00000004
> + - The DPB entry is a long term reference frame
>
> .. _v4l2-mpeg-mpeg2:
>
> diff --git a/Documentation/media/uapi/v4l/pixfmt-compressed.rst b/Documentation/media/uapi/v4l/pixfmt-compressed.rst
> index 6c961cfb74da..ea0a8a68759b 100644
> --- a/Documentation/media/uapi/v4l/pixfmt-compressed.rst
> +++ b/Documentation/media/uapi/v4l/pixfmt-compressed.rst
> @@ -52,6 +52,25 @@ Compressed Formats
> - ``V4L2_PIX_FMT_H264_MVC``
> - 'M264'
> - H264 MVC video elementary stream.
> + * .. _V4L2-PIX-FMT-H264-SLICE:
> +
> + - ``V4L2_PIX_FMT_H264_SLICE_RAW``
> + - 'S264'
> + - H264 parsed slice data, as extracted from the H264 bitstream.
> + This format is adapted for stateless video decoders that
> + implement an H264 pipeline (using the :ref:`codec` and
> + :ref:`media-request-api`). Metadata associated with the frame
> + to decode are required to be passed through the
> + ``V4L2_CID_MPEG_VIDEO_H264_SPS``,
> + ``V4L2_CID_MPEG_VIDEO_H264_PPS``,
> + ``V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX``,
> + ``V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS`` and
> + ``V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS`` controls. See the
> + :ref:`associated Codec Control IDs <v4l2-mpeg-h264>`.
> + Exactly one output and one capture buffer must be provided for
> + use with this pixel format. The output buffer must contain the
> + appropriate number of macroblocks to decode a full
> + corresponding frame to the matching capture buffer.
> * .. _V4L2-PIX-FMT-H263:
>
> - ``V4L2_PIX_FMT_H263``
> diff --git a/Documentation/media/uapi/v4l/vidioc-queryctrl.rst b/Documentation/media/uapi/v4l/vidioc-queryctrl.rst
> index f824162d0ea9..dc500632095d 100644
> --- a/Documentation/media/uapi/v4l/vidioc-queryctrl.rst
> +++ b/Documentation/media/uapi/v4l/vidioc-queryctrl.rst
> @@ -443,6 +443,36 @@ See also the examples in :ref:`control`.
> - n/a
> - A struct :c:type:`v4l2_ctrl_mpeg2_quantization`, containing MPEG-2
> quantization matrices for stateless video decoders.
> + * - ``V4L2_CTRL_TYPE_H264_SPS``
> + - n/a
> + - n/a
> + - n/a
> + - A struct :c:type:`v4l2_ctrl_h264_sps`, containing H264
> + sequence parameters for stateless video decoders.
> + * - ``V4L2_CTRL_TYPE_H264_PPS``
> + - n/a
> + - n/a
> + - n/a
> + - A struct :c:type:`v4l2_ctrl_h264_pps`, containing H264
> + picture parameters for stateless video decoders.
> + * - ``V4L2_CTRL_TYPE_H264_SCALING_MATRIX``
> + - n/a
> + - n/a
> + - n/a
> + - A struct :c:type:`v4l2_ctrl_h264_scaling_matrix`, containing H264
> + scaling matrices for stateless video decoders.
> + * - ``V4L2_CTRL_TYPE_H264_SLICE_PARAMS``
> + - n/a
> + - n/a
> + - n/a
> + - A struct :c:type:`v4l2_ctrl_h264_slice_params`, containing H264
> + slice parameters for stateless video decoders.
> + * - ``V4L2_CTRL_TYPE_H264_DECODE_PARAMS``
> + - n/a
> + - n/a
> + - n/a
> + - A struct :c:type:`v4l2_ctrl_h264_decode_params`, containing H264
> + decode parameters for stateless video decoders.
>
> .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
>
> diff --git a/Documentation/media/videodev2.h.rst.exceptions b/Documentation/media/videodev2.h.rst.exceptions
> index 64d348e67df9..55cbe324b9fc 100644
> --- a/Documentation/media/videodev2.h.rst.exceptions
> +++ b/Documentation/media/videodev2.h.rst.exceptions
> @@ -136,6 +136,11 @@ replace symbol V4L2_CTRL_TYPE_U32 :c:type:`v4l2_ctrl_type`
> replace symbol V4L2_CTRL_TYPE_U8 :c:type:`v4l2_ctrl_type`
> replace symbol V4L2_CTRL_TYPE_MPEG2_SLICE_PARAMS :c:type:`v4l2_ctrl_type`
> replace symbol V4L2_CTRL_TYPE_MPEG2_QUANTIZATION :c:type:`v4l2_ctrl_type`
> +replace symbol V4L2_CTRL_TYPE_H264_SPS :c:type:`v4l2_ctrl_type`
> +replace symbol V4L2_CTRL_TYPE_H264_PPS :c:type:`v4l2_ctrl_type`
> +replace symbol V4L2_CTRL_TYPE_H264_SCALING_MATRIX :c:type:`v4l2_ctrl_type`
> +replace symbol V4L2_CTRL_TYPE_H264_SLICE_PARAMS :c:type:`v4l2_ctrl_type`
> +replace symbol V4L2_CTRL_TYPE_H264_DECODE_PARAMS :c:type:`v4l2_ctrl_type`
>
> # V4L2 capability defines
> replace define V4L2_CAP_VIDEO_CAPTURE device-capabilities
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index b1ae2e555c68..46aec8c3acde 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -828,6 +828,11 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_MPEG_VIDEO_H264_CONSTRAINED_INTRA_PREDICTION:
> return "H264 Constrained Intra Pred";
> case V4L2_CID_MPEG_VIDEO_H264_CHROMA_QP_INDEX_OFFSET: return "H264 Chroma QP Index Offset";
> + case V4L2_CID_MPEG_VIDEO_H264_SPS: return "H264 Sequence Parameter Set";
> + case V4L2_CID_MPEG_VIDEO_H264_PPS: return "H264 Picture Parameter Set";
> + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: return "H264 Scaling Matrix";
> + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS: return "H264 Slice Parameters";
> + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS: return "H264 Decode Parameters";
> case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4 I-Frame QP Value";
> case V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP: return "MPEG4 P-Frame QP Value";
> case V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP: return "MPEG4 B-Frame QP Value";
> @@ -1309,6 +1314,21 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> case V4L2_CID_MPEG_VIDEO_FWHT_PARAMS:
> *type = V4L2_CTRL_TYPE_FWHT_PARAMS;
> break;
> + case V4L2_CID_MPEG_VIDEO_H264_SPS:
> + *type = V4L2_CTRL_TYPE_H264_SPS;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_PPS:
> + *type = V4L2_CTRL_TYPE_H264_PPS;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX:
> + *type = V4L2_CTRL_TYPE_H264_SCALING_MATRIX;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS:
> + *type = V4L2_CTRL_TYPE_H264_SLICE_PARAMS;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS:
> + *type = V4L2_CTRL_TYPE_H264_DECODE_PARAMS;
> + break;
> default:
> *type = V4L2_CTRL_TYPE_INTEGER;
> break;
> @@ -1678,6 +1698,13 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
> case V4L2_CTRL_TYPE_FWHT_PARAMS:
> return 0;
>
> + case V4L2_CTRL_TYPE_H264_SPS:
> + case V4L2_CTRL_TYPE_H264_PPS:
> + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
> + case V4L2_CTRL_TYPE_H264_SLICE_PARAMS:
> + case V4L2_CTRL_TYPE_H264_DECODE_PARAMS:
> + return 0;
> +
> default:
> return -EINVAL;
> }
> @@ -2261,6 +2288,21 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
> case V4L2_CTRL_TYPE_FWHT_PARAMS:
> elem_size = sizeof(struct v4l2_ctrl_fwht_params);
> break;
> + case V4L2_CTRL_TYPE_H264_SPS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_sps);
> + break;
> + case V4L2_CTRL_TYPE_H264_PPS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_pps);
> + break;
> + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
> + elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix);
> + break;
> + case V4L2_CTRL_TYPE_H264_SLICE_PARAMS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_slice_params);
> + break;
> + case V4L2_CTRL_TYPE_H264_DECODE_PARAMS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_decode_params);
> + break;
> default:
> if (type < V4L2_CTRL_COMPOUND_TYPES)
> elem_size = sizeof(s32);
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index ac87c3e37280..f6e1254064d2 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1325,6 +1325,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> case V4L2_PIX_FMT_H264: descr = "H.264"; break;
> case V4L2_PIX_FMT_H264_NO_SC: descr = "H.264 (No Start Codes)"; break;
> case V4L2_PIX_FMT_H264_MVC: descr = "H.264 MVC"; break;
> + case V4L2_PIX_FMT_H264_SLICE_RAW: descr = "H.264 Parsed Slice Data"; break;
> case V4L2_PIX_FMT_H263: descr = "H.263"; break;
> case V4L2_PIX_FMT_MPEG1: descr = "MPEG-1 ES"; break;
> case V4L2_PIX_FMT_MPEG2: descr = "MPEG-2 ES"; break;
> diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h
> new file mode 100644
> index 000000000000..e2f83b3cdbef
> --- /dev/null
> +++ b/include/media/h264-ctrls.h
> @@ -0,0 +1,192 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * These are the H.264 state controls for use with stateless H.264
> + * codec drivers.
> + *
> + * It turns out that these structs are not stable yet and will undergo
> + * more changes. So keep them private until they are stable and ready to
> + * become part of the official public API.
> + */
> +
> +#ifndef _H264_CTRLS_H_
> +#define _H264_CTRLS_H_
> +
> +/*
> + * This is put insanely high to avoid conflicting with controls that
> + * would be added during the phase where those controls are not
> + * stable. It should be fixed eventually.
> + */
> +#define V4L2_CID_MPEG_VIDEO_H264_SPS (V4L2_CID_MPEG_BASE+1000)
> +#define V4L2_CID_MPEG_VIDEO_H264_PPS (V4L2_CID_MPEG_BASE+1001)
> +#define V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (V4L2_CID_MPEG_BASE+1002)
> +#define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS (V4L2_CID_MPEG_BASE+1003)
> +#define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (V4L2_CID_MPEG_BASE+1004)
> +
> +/* enum v4l2_ctrl_type type values */
> +#define V4L2_CTRL_TYPE_H264_SPS 0x0110
> +#define V4L2_CTRL_TYPE_H264_PPS 0x0111
> +#define V4L2_CTRL_TYPE_H264_SCALING_MATRIX 0x0112
> +#define V4L2_CTRL_TYPE_H264_SLICE_PARAMS 0x0113
> +#define V4L2_CTRL_TYPE_H264_DECODE_PARAMS 0x0114
> +
> +#define V4L2_H264_SPS_CONSTRAINT_SET0_FLAG 0x01
> +#define V4L2_H264_SPS_CONSTRAINT_SET1_FLAG 0x02
> +#define V4L2_H264_SPS_CONSTRAINT_SET2_FLAG 0x04
> +#define V4L2_H264_SPS_CONSTRAINT_SET3_FLAG 0x08
> +#define V4L2_H264_SPS_CONSTRAINT_SET4_FLAG 0x10
> +#define V4L2_H264_SPS_CONSTRAINT_SET5_FLAG 0x20
> +
> +#define V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE 0x01
> +#define V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS 0x02
> +#define V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO 0x04
> +#define V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED 0x08
> +#define V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY 0x10
> +#define V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD 0x20
> +#define V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE 0x40
> +
> +struct v4l2_ctrl_h264_sps {
> + __u8 profile_idc;
> + __u8 constraint_set_flags;
> + __u8 level_idc;
> + __u8 seq_parameter_set_id;
> + __u8 chroma_format_idc;
> + __u8 bit_depth_luma_minus8;
> + __u8 bit_depth_chroma_minus8;
> + __u8 log2_max_frame_num_minus4;
> + __u8 pic_order_cnt_type;
> + __u8 log2_max_pic_order_cnt_lsb_minus4;
> + __u8 max_num_ref_frames;
> + __u8 num_ref_frames_in_pic_order_cnt_cycle;
> + __s32 offset_for_ref_frame[255];
> + __s32 offset_for_non_ref_pic;
> + __s32 offset_for_top_to_bottom_field;
> + __u16 pic_width_in_mbs_minus1;
> + __u16 pic_height_in_map_units_minus1;
> + __u32 flags;
> +};
> +
> +#define V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE 0x0001
> +#define V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT 0x0002
> +#define V4L2_H264_PPS_FLAG_WEIGHTED_PRED 0x0004
> +#define V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT 0x0008
> +#define V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED 0x0010
> +#define V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT 0x0020
> +#define V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE 0x0040
> +#define V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT 0x0080
> +
> +struct v4l2_ctrl_h264_pps {
> + __u8 pic_parameter_set_id;
> + __u8 seq_parameter_set_id;
> + __u8 num_slice_groups_minus1;
> + __u8 num_ref_idx_l0_default_active_minus1;
> + __u8 num_ref_idx_l1_default_active_minus1;
> + __u8 weighted_bipred_idc;
> + __s8 pic_init_qp_minus26;
> + __s8 pic_init_qs_minus26;
> + __s8 chroma_qp_index_offset;
> + __s8 second_chroma_qp_index_offset;
> + __u16 flags;
> +};
> +
> +struct v4l2_ctrl_h264_scaling_matrix {
> + __u8 scaling_list_4x4[6][16];
> + __u8 scaling_list_8x8[6][64];
> +};
> +
> +struct v4l2_h264_weight_factors {
> + __s16 luma_weight[32];
> + __s16 luma_offset[32];
> + __s16 chroma_weight[32][2];
> + __s16 chroma_offset[32][2];
> +};
> +
> +struct v4l2_h264_pred_weight_table {
> + __u16 luma_log2_weight_denom;
> + __u16 chroma_log2_weight_denom;
> + struct v4l2_h264_weight_factors weight_factors[2];
> +};
> +
> +#define V4L2_H264_SLICE_TYPE_P 0
> +#define V4L2_H264_SLICE_TYPE_B 1
> +#define V4L2_H264_SLICE_TYPE_I 2
> +#define V4L2_H264_SLICE_TYPE_SP 3
> +#define V4L2_H264_SLICE_TYPE_SI 4
> +
> +#define V4L2_H264_SLICE_FLAG_FIELD_PIC 0x01
> +#define V4L2_H264_SLICE_FLAG_BOTTOM_FIELD 0x02
> +#define V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED 0x04
> +#define V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH 0x08
> +
> +struct v4l2_ctrl_h264_slice_params {
> + /* Size in bytes, including header */
> + __u32 size;
> + /* Offset in bits to slice_data() from the beginning of this slice. */
> + __u32 header_bit_size;
> +
> + __u16 first_mb_in_slice;
> + __u8 slice_type;
> + __u8 pic_parameter_set_id;
> + __u8 colour_plane_id;
> + __u8 redundant_pic_cnt;
> + __u16 frame_num;
> + __u16 idr_pic_id;
> + __u16 pic_order_cnt_lsb;
> + __s32 delta_pic_order_cnt_bottom;
> + __s32 delta_pic_order_cnt0;
> + __s32 delta_pic_order_cnt1;
> +
> + struct v4l2_h264_pred_weight_table pred_weight_table;
> + /* Size in bits of dec_ref_pic_marking() syntax element. */
> + __u32 dec_ref_pic_marking_bit_size;
> + /* Size in bits of pic order count syntax. */
> + __u32 pic_order_cnt_bit_size;
> +
> + __u8 cabac_init_idc;
> + __s8 slice_qp_delta;
> + __s8 slice_qs_delta;
> + __u8 disable_deblocking_filter_idc;
> + __s8 slice_alpha_c0_offset_div2;
> + __s8 slice_beta_offset_div2;
> + __u8 num_ref_idx_l0_active_minus1;
> + __u8 num_ref_idx_l1_active_minus1;
> + __u32 slice_group_change_cycle;
> +
> + /*
> + * Entries on each list are indices into
> + * v4l2_ctrl_h264_decode_params.dpb[].
> + */
> + __u8 ref_pic_list0[32];
> + __u8 ref_pic_list1[32];
> +
> + __u32 flags;
> +};
> +
> +#define V4L2_H264_DPB_ENTRY_FLAG_VALID 0x01
> +#define V4L2_H264_DPB_ENTRY_FLAG_ACTIVE 0x02
> +#define V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM 0x04
> +
> +struct v4l2_h264_dpb_entry {
> + __u64 reference_ts;
> + __u16 frame_num;
> + __u16 pic_num;
> + /* Note that field is indicated by v4l2_buffer.field */
> + __s32 top_field_order_cnt;
> + __s32 bottom_field_order_cnt;
> + __u32 flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */
> +};
> +
> +#define V4L2_H264_DECODE_PARAM_FLAG_IDR_PIC 0x01
> +
> +struct v4l2_ctrl_h264_decode_params {
> + struct v4l2_h264_dpb_entry dpb[16];
> + __u16 num_slices;
> + __u16 nal_ref_idc;
> + __u8 ref_pic_list_p0[32];
> + __u8 ref_pic_list_b0[32];
> + __u8 ref_pic_list_b1[32];
> + __s32 top_field_order_cnt;
> + __s32 bottom_field_order_cnt;
> + __u32 flags; /* V4L2_H264_DECODE_PARAM_FLAG_* */
> +};
> +
> +#endif
> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> index bd621cec65a5..dce6f33fd749 100644
> --- a/include/media/v4l2-ctrls.h
> +++ b/include/media/v4l2-ctrls.h
> @@ -23,11 +23,12 @@
> #include <media/media-request.h>
>
> /*
> - * Include the mpeg2 and fwht stateless codec compound control definitions.
> + * Include the stateless codec compound control definitions.
> * This will move to the public headers once this API is fully stable.
> */
> #include <media/mpeg2-ctrls.h>
> #include <media/fwht-ctrls.h>
> +#include <media/h264-ctrls.h>
>
> /* forward references */
> struct file;
> @@ -51,6 +52,11 @@ struct poll_table_struct;
> * @p_mpeg2_slice_params: Pointer to a MPEG2 slice parameters structure.
> * @p_mpeg2_quantization: Pointer to a MPEG2 quantization data structure.
> * @p_fwht_params: Pointer to a FWHT stateless parameters structure.
> + * @p_h264_sps: Pointer to a struct v4l2_ctrl_h264_sps.
> + * @p_h264_pps: Pointer to a struct v4l2_ctrl_h264_pps.
> + * @p_h264_scaling_matrix: Pointer to a struct v4l2_ctrl_h264_scaling_matrix.
> + * @p_h264_slice_param: Pointer to a struct v4l2_ctrl_h264_slice_params.
> + * @p_h264_decode_param: Pointer to a struct v4l2_ctrl_h264_decode_params.
> * @p: Pointer to a compound value.
> */
> union v4l2_ctrl_ptr {
> @@ -63,6 +69,11 @@ union v4l2_ctrl_ptr {
> struct v4l2_ctrl_mpeg2_slice_params *p_mpeg2_slice_params;
> struct v4l2_ctrl_mpeg2_quantization *p_mpeg2_quantization;
> struct v4l2_ctrl_fwht_params *p_fwht_params;
> + struct v4l2_ctrl_h264_sps *p_h264_sps;
> + struct v4l2_ctrl_h264_pps *p_h264_pps;
> + struct v4l2_ctrl_h264_scaling_matrix *p_h264_scaling_matrix;
> + struct v4l2_ctrl_h264_slice_params *p_h264_slice_param;
> + struct v4l2_ctrl_h264_decode_params *p_h264_decode_param;
> void *p;
> };
>
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 496e6453450c..838732acdefc 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -657,6 +657,7 @@ struct v4l2_pix_format {
> #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> +#define V4L2_PIX_FMT_H264_SLICE_RAW v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> #define V4L2_PIX_FMT_H263 v4l2_fourcc('H', '2', '6', '3') /* H263 */
> #define V4L2_PIX_FMT_MPEG1 v4l2_fourcc('M', 'P', 'G', '1') /* MPEG-1 ES */
> #define V4L2_PIX_FMT_MPEG2 v4l2_fourcc('M', 'P', 'G', '2') /* MPEG-2 ES */
--
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
^ permalink raw reply
* Re: [PATCH 07/10] media: coda: limit frame interval enumeration to supported frame sizes
From: Hans Verkuil @ 2019-04-10 13:43 UTC (permalink / raw)
To: Philipp Zabel, linux-media; +Cc: kernel
In-Reply-To: <20190408123256.22868-7-p.zabel@pengutronix.de>
On 4/8/19 2:32 PM, Philipp Zabel wrote:
> Let VIDIOC_ENUM_FRAMEINTERVALS return -EINVAL if userspace queries
> frame intervals for unsupported frame sizes.
>
> Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
> ---
> drivers/media/platform/coda/coda-common.c | 33 ++++++++++++++++++-----
> 1 file changed, 27 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
> index 943f003c26c4..2966eb1c4d2d 100644
> --- a/drivers/media/platform/coda/coda-common.c
> +++ b/drivers/media/platform/coda/coda-common.c
> @@ -1117,7 +1117,8 @@ static int coda_enum_frameintervals(struct file *file, void *fh,
> struct v4l2_frmivalenum *f)
> {
> struct coda_ctx *ctx = fh_to_ctx(fh);
> - int i;
> + struct coda_q_data *q_data;
> + const struct coda_codec *codec;
>
> if (f->index)
> return -EINVAL;
> @@ -1126,12 +1127,32 @@ static int coda_enum_frameintervals(struct file *file, void *fh,
> if (!ctx->vdoa && f->pixel_format == V4L2_PIX_FMT_YUYV)
> return -EINVAL;
>
> - for (i = 0; i < CODA_MAX_FORMATS; i++) {
> - if (f->pixel_format == ctx->cvd->src_formats[i] ||
> - f->pixel_format == ctx->cvd->dst_formats[i])
> - break;
> + if (ctx->inst_type == CODA_INST_ENCODER) {
> + if (coda_format_normalize_yuv(f->pixel_format) ==
> + V4L2_PIX_FMT_YUV420) {
> + q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
> + codec = coda_find_codec(ctx->dev, f->pixel_format,
> + q_data->fourcc);
> + } else {
> + codec = coda_find_codec(ctx->dev, V4L2_PIX_FMT_YUV420,
> + f->pixel_format);
> + }
> + } else {
> + if (coda_format_normalize_yuv(f->pixel_format) ==
> + V4L2_PIX_FMT_YUV420) {
> + q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
> + codec = coda_find_codec(ctx->dev, q_data->fourcc,
> + f->pixel_format);
> + } else {
> + codec = coda_find_codec(ctx->dev, f->pixel_format,
> + V4L2_PIX_FMT_YUV420);
> + }
> }
> - if (i == CODA_MAX_FORMATS)
> + if (!codec)
> + return -EINVAL;
> +
> + if (f->width < MIN_W || f->width > codec->max_w ||
> + f->height < MIN_H || f->height > codec->max_h)
> return -EINVAL;
>
> f->type = V4L2_FRMIVAL_TYPE_CONTINUOUS;
>
Why support VIDIOC_ENUM_FRAMEINTERVALS at all? It makes no sense for a codec.
I'd remove it altogether.
Regards,
Hans
^ permalink raw reply
* Re: [PATCH v8 1/2] media: uapi: Add H264 low-level decoder API compound controls.
From: Hans Verkuil @ 2019-04-10 13:30 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus,
Laurent Pinchart
Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel,
linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske,
jernej.skrabec, jonas, ezequiel, linux-sunxi, Thomas Petazzoni,
Guenter Roeck
In-Reply-To: <f537c99d8b5a974f8687f4f0d018c025a4220f7a.1554382670.git-series.maxime.ripard@bootlin.com>
On 4/4/19 2:59 PM, Maxime Ripard wrote:
> From: Pawel Osciak <posciak@chromium.org>
>
> Stateless video codecs will require both the H264 metadata and slices in
> order to be able to decode frames.
>
> This introduces the definitions for a new pixel format for H264 slices that
> have been parsed, as well as the structures used to pass the metadata from
> the userspace to the kernel.
>
> Reviewed-by: Tomasz Figa <tfiga@chromium.org>
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> Signed-off-by: Guenter Roeck <groeck@chromium.org>
> Co-developed-by: Maxime Ripard <maxime.ripard@bootlin.com>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> Documentation/media/uapi/v4l/biblio.rst | 9 +-
> Documentation/media/uapi/v4l/ext-ctrls-codec.rst | 569 ++++++++++++++-
> Documentation/media/uapi/v4l/pixfmt-compressed.rst | 19 +-
> Documentation/media/uapi/v4l/vidioc-queryctrl.rst | 30 +-
> Documentation/media/videodev2.h.rst.exceptions | 5 +-
> drivers/media/v4l2-core/v4l2-ctrls.c | 42 +-
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +-
> include/media/h264-ctrls.h | 192 +++++-
> include/media/v4l2-ctrls.h | 13 +-
> include/uapi/linux/videodev2.h | 1 +-
> 10 files changed, 880 insertions(+), 1 deletion(-)
> create mode 100644 include/media/h264-ctrls.h
>
> diff --git a/Documentation/media/uapi/v4l/biblio.rst b/Documentation/media/uapi/v4l/biblio.rst
> index ec33768c055e..8f4eb8823d82 100644
> --- a/Documentation/media/uapi/v4l/biblio.rst
> +++ b/Documentation/media/uapi/v4l/biblio.rst
> @@ -122,6 +122,15 @@ ITU BT.1119
>
> :author: International Telecommunication Union (http://www.itu.ch)
>
> +.. _h264:
> +
> +ITU-T Rec. H.264 Specification (04/2017 Edition)
> +================================================
> +
> +:title: ITU-T Recommendation H.264 "Advanced Video Coding for Generic Audiovisual Services"
> +
> +:author: International Telecommunication Union (http://www.itu.ch)
> +
> .. _jfif:
>
> JFIF
> diff --git a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst
> index 67a122339c0e..fe720f239f70 100644
> --- a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst
> +++ b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst
> @@ -1371,6 +1371,575 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
> - Layer number
>
>
> +.. _v4l2-mpeg-h264:
> +
> +``V4L2_CID_MPEG_VIDEO_H264_SPS (struct)``
> + Specifies the sequence parameter set (as extracted from the
> + bitstream) for the associated H264 slice data. This includes the
> + necessary parameters for configuring a stateless hardware decoding
> + pipeline for H264. The bitstream parameters are defined according
> + to :ref:`h264`, section 7.4.2.1.1 "Sequence Parameter Set Data
> + Semantics". For further documentation, refer to the above
> + specification, unless there is an explicit comment stating
> + otherwise.
> +
> + .. note::
> +
> + This compound control is not yet part of the public kernel API and
> + it is expected to change.
> +
> +.. c:type:: v4l2_ctrl_h264_sps
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_ctrl_h264_sps
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u8
> + - ``profile_idc``
> + -
> + * - __u8
> + - ``constraint_set_flags``
> + - See :ref:`Sequence Parameter Set Constraints Set Flags <h264_sps_constraints_set_flags>`
> + * - __u8
> + - ``level_idc``
> + -
> + * - __u8
> + - ``seq_parameter_set_id``
> + -
> + * - __u8
> + - ``chroma_format_idc``
> + -
> + * - __u8
> + - ``bit_depth_luma_minus8``
> + -
> + * - __u8
> + - ``bit_depth_chroma_minus8``
> + -
> + * - __u8
> + - ``log2_max_frame_num_minus4``
> + -
> + * - __u8
> + - ``pic_order_cnt_type``
> + -
> + * - __u8
> + - ``log2_max_pic_order_cnt_lsb_minus4``
> + -
> + * - __u8
> + - ``max_num_ref_frames``
> + -
> + * - __u8
> + - ``num_ref_frames_in_pic_order_cnt_cycle``
> + -
> + * - __s32
> + - ``offset_for_ref_frame[255]``
> + -
> + * - __s32
> + - ``offset_for_non_ref_pic``
> + -
> + * - __s32
> + - ``offset_for_top_to_bottom_field``
> + -
> + * - __u16
> + - ``pic_width_in_mbs_minus1``
> + -
> + * - __u16
> + - ``pic_height_in_map_units_minus1``
> + -
> + * - __u32
> + - ``flags``
> + - See :ref:`Sequence Parameter Set Flags <h264_sps_flags>`
> +
> +.. _h264_sps_constraints_set_flags:
> +
> +``Sequence Parameter Set Constraints Set Flags``
> +
> +.. cssclass:: longtable
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - ``V4L2_H264_SPS_CONSTRAINT_SET0_FLAG``
> + - 0x00000001
> + -
> + * - ``V4L2_H264_SPS_CONSTRAINT_SET1_FLAG``
> + - 0x00000002
> + -
> + * - ``V4L2_H264_SPS_CONSTRAINT_SET2_FLAG``
> + - 0x00000004
> + -
> + * - ``V4L2_H264_SPS_CONSTRAINT_SET3_FLAG``
> + - 0x00000008
> + -
> + * - ``V4L2_H264_SPS_CONSTRAINT_SET4_FLAG``
> + - 0x00000010
> + -
> + * - ``V4L2_H264_SPS_CONSTRAINT_SET5_FLAG``
> + - 0x00000020
> + -
> +
> +.. _h264_sps_flags:
> +
> +``Sequence Parameter Set Flags``
> +
> +.. cssclass:: longtable
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - ``V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE``
> + - 0x00000001
> + -
> + * - ``V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS``
> + - 0x00000002
> + -
> + * - ``V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO``
> + - 0x00000004
> + -
> + * - ``V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED``
> + - 0x00000008
> + -
> + * - ``V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY``
> + - 0x00000010
> + -
> + * - ``V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD``
> + - 0x00000020
> + -
> + * - ``V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE``
> + - 0x00000040
> + -
> +
> +``V4L2_CID_MPEG_VIDEO_H264_PPS (struct)``
> + Specifies the picture parameter set (as extracted from the
> + bitstream) for the associated H264 slice data. This includes the
> + necessary parameters for configuring a stateless hardware decoding
> + pipeline for H264. The bitstream parameters are defined according
> + to :ref:`h264`, section 7.4.2.2 "Picture Parameter Set RBSP
> + Semantics". For further documentation, refer to the above
> + specification, unless there is an explicit comment stating
> + otherwise.
> +
> + .. note::
> +
> + This compound control is not yet part of the public kernel API and
> + it is expected to change.
> +
> +.. c:type:: v4l2_ctrl_h264_pps
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_ctrl_h264_pps
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u8
> + - ``pic_parameter_set_id``
> + -
> + * - __u8
> + - ``seq_parameter_set_id``
> + -
> + * - __u8
> + - ``num_slice_groups_minus1``
> + -
> + * - __u8
> + - ``num_ref_idx_l0_default_active_minus1``
> + -
> + * - __u8
> + - ``num_ref_idx_l1_default_active_minus1``
> + -
> + * - __u8
> + - ``weighted_bipred_idc``
> + -
> + * - __s8
> + - ``pic_init_qp_minus26``
> + -
> + * - __s8
> + - ``pic_init_qs_minus26``
> + -
> + * - __s8
> + - ``chroma_qp_index_offset``
> + -
> + * - __s8
> + - ``second_chroma_qp_index_offset``
> + -
> + * - __u16
> + - ``flags``
> + - See :ref:`Picture Parameter Set Flags <h264_pps_flags>`
> +
> +.. _h264_pps_flags:
> +
> +``Picture Parameter Set Flags``
> +
> +.. cssclass:: longtable
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - ``V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE``
> + - 0x00000001
> + -
> + * - ``V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT``
> + - 0x00000002
> + -
> + * - ``V4L2_H264_PPS_FLAG_WEIGHTED_PRED``
> + - 0x00000004
> + -
> + * - ``V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT``
> + - 0x00000008
> + -
> + * - ``V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED``
> + - 0x00000010
> + -
> + * - ``V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT``
> + - 0x00000020
> + -
> + * - ``V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE``
> + - 0x00000040
> + -
> + * - ``V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT``
> + - 0x00000080
> + -
> +
> +``V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (struct)``
> + Specifies the scaling matrix (as extracted from the bitstream) for
> + the associated H264 slice data. The bitstream parameters are
> + defined according to :ref:`h264`, section 7.4.2.1.1.1 "Scaling
> + List Semantics".For further documentation, refer to the above
> + specification, unless there is an explicit comment stating
> + otherwise.
> +
> + .. note::
> +
> + This compound control is not yet part of the public kernel API and
> + it is expected to change.
> +
> +.. c:type:: v4l2_ctrl_h264_scaling_matrix
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_ctrl_h264_scaling_matrix
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u8
> + - ``scaling_list_4x4[6][16]``
> + -
> + * - __u8
> + - ``scaling_list_8x8[6][64]``
> + -
> +
> +``V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS (struct)``
> + Specifies the slice parameters (as extracted from the bitstream)
> + for the associated H264 slice data. This includes the necessary
> + parameters for configuring a stateless hardware decoding pipeline
> + for H264. The bitstream parameters are defined according to
> + :ref:`h264`, section 7.4.3 "Slice Header Semantics". For further
> + documentation, refer to the above specification, unless there is
> + an explicit comment stating otherwise.
> +
> + .. note::
> +
> + This compound control is not yet part of the public kernel API
> + and it is expected to change.
> +
> + This structure is expected to be passed as an array, with one
> + entry for each slice included in the bitstream buffer.
> +
> +.. c:type:: v4l2_ctrl_h264_slice_params
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_ctrl_h264_slice_params
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u32
> + - ``size``
> + -
> + * - __u32
> + - ``header_bit_size``
> + -
> + * - __u16
> + - ``first_mb_in_slice``
> + -
> + * - __u8
> + - ``slice_type``
> + -
> + * - __u8
> + - ``pic_parameter_set_id``
> + -
> + * - __u8
> + - ``colour_plane_id``
> + -
> + * - __u8
> + - ``redundant_pic_cnt``
> + -
> + * - __u16
> + - ``frame_num``
> + -
> + * - __u16
> + - ``idr_pic_id``
> + -
> + * - __u16
> + - ``pic_order_cnt_lsb``
> + -
> + * - __s32
> + - ``delta_pic_order_cnt_bottom``
> + -
> + * - __s32
> + - ``delta_pic_order_cnt0``
> + -
> + * - __s32
> + - ``delta_pic_order_cnt1``
> + -
> + * - struct :c:type:`v4l2_h264_pred_weight_table`
> + - ``pred_weight_table``
> + -
> + * - __u32
> + - ``dec_ref_pic_marking_bit_size``
> + -
> + * - __u32
> + - ``pic_order_cnt_bit_size``
> + -
> + * - __u8
> + - ``cabac_init_idc``
> + -
> + * - __s8
> + - ``slice_qp_delta``
> + -
> + * - __s8
> + - ``slice_qs_delta``
> + -
> + * - __u8
> + - ``disable_deblocking_filter_idc``
> + -
> + * - __s8
> + - ``slice_alpha_c0_offset_div2``
> + -
> + * - __s8
> + - ``slice_beta_offset_div2``
> + -
> + * - __u8
> + - ``num_ref_idx_l0_active_minus1``
> + -
> + * - __u8
> + - ``num_ref_idx_l1_active_minus1``
> + -
> + * - __u32
> + - ``slice_group_change_cycle``
> + -
> + * - __u8
> + - ``ref_pic_list0[32]``
> + - Reference picture list after applying the per-slice modifications
> + * - __u8
> + - ``ref_pic_list1[32]``
> + - Reference picture list after applying the per-slice modifications
> + * - __u32
> + - ``flags``
> + - See :ref:`Slice Parameter Flags <h264_slice_flags>`
> +
> +.. _h264_slice_flags:
> +
> +``Slice Parameter Set Flags``
> +
> +.. cssclass:: longtable
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - ``V4L2_H264_SLICE_FLAG_FIELD_PIC``
> + - 0x00000001
> + -
> + * - ``V4L2_H264_SLICE_FLAG_BOTTOM_FIELD``
> + - 0x00000002
> + -
> + * - ``V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED``
> + - 0x00000004
> + -
> + * - ``V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH``
> + - 0x00000008
> + -
> +
> +``Prediction Weight Table``
> +
> + The bitstream parameters are defined according to :ref:`h264`,
> + section 7.4.3.2 "Prediction Weight Table Semantics". For further
> + documentation, refer to the above specification, unless there is
> + an explicit comment stating otherwise.
> +
> +.. c:type:: v4l2_h264_pred_weight_table
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_h264_pred_weight_table
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u16
> + - ``luma_log2_weight_denom``
> + -
> + * - __u16
> + - ``chroma_log2_weight_denom``
> + -
> + * - struct :c:type:`v4l2_h264_weight_factors`
> + - ``weight_factors[2]``
> + - The weight factors at index 0 are the weight factors for the reference
> + list 0, the one at index 1 for the reference list 1.
> +
> +.. c:type:: v4l2_h264_weight_factors
> +
> +.. cssclass:: longtable
> +
/> +.. flat-table:: struct v4l2_h264_weight_factors
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __s16
> + - ``luma_weight[32]``
> + -
> + * - __s16
> + - ``luma_offset[32]``
> + -
> + * - __s16
> + - ``chroma_weight[32][2]``
> + -
> + * - __s16
> + - ``chroma_offset[32][2]``
> + -
> +
> +``V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (struct)``
> + Specifies the decode parameters (as extracted from the bitstream)
> + for the associated H264 slice data. This includes the necessary
> + parameters for configuring a stateless hardware decoding pipeline
> + for H264. The bitstream parameters are defined according to
> + :ref:`h264`. For further documentation, refer to the above
> + specification, unless there is an explicit comment stating
> + otherwise.
> +
> + .. note::
> +
> + This compound control is not yet part of the public kernel API and
> + it is expected to change.
> +
> +.. c:type:: v4l2_ctrl_h264_decode_params
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_ctrl_h264_decode_params
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u32
> + - ``num_slices``
> + - Number of slices needed to decode the current frame
> + * - __u32
> + - ``nal_ref_idc``
> + - NAL reference ID value coming from the NAL Unit header
> + * - __u8
> + - ``ref_pic_list_p0[32]``
> + - Backward reference list used by P-frames in the original bitstream order
> + * - __u8
> + - ``ref_pic_list_b0[32]``
> + - Backward reference list used by B-frames in the original bitstream order
> + * - __u8
> + - ``ref_pic_list_b1[32]``
> + - Forward reference list used by B-frames in the original bitstream order
> + * - __s32
> + - ``top_field_order_cnt``
> + - Picture Order Count for the coded top field
> + * - __s32
> + - ``bottom_field_order_cnt``
> + - Picture Order Count for the coded bottom field
> + * - __u32
> + - ``flags``
> + - See :ref:`Decode Parameters Flags <h264_decode_params_flags>`
> + * - struct :c:type:`v4l2_h264_dpb_entry`
> + - ``dpb[16]``
> + -
> +
> +.. _h264_decode_params_flags:
> +
> +``Decode Parameters Flags``
> +
> +.. cssclass:: longtable
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - ``V4L2_H264_DECODE_PARAM_FLAG_IDR_PIC``
> + - 0x00000001
> + - That picture is an IDR picture
> +
> +.. c:type:: v4l2_h264_dpb_entry
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_h264_dpb_entry
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u64
> + - ``reference_ts``
> + - Timestamp of the V4L2 capture buffer to use as reference, used
> + with B-coded and P-coded frames. The timestamp refers to the
> + ``timestamp`` field in struct :c:type:`v4l2_buffer`. Use the
> + :c:func:`v4l2_timeval_to_ns()` function to convert the struct
> + :c:type:`timeval` in struct :c:type:`v4l2_buffer` to a __u64.
> + * - __u16
> + - ``frame_num``
> + -
> + * - __u16
> + - ``pic_num``
> + -
> + * - __s32
> + - ``top_field_order_cnt``
> + -
> + * - __s32
> + - ``bottom_field_order_cnt``
> + -
> + * - __u32
> + - ``flags``
> + - See :ref:`DPB Entry Flags <h264_dpb_flags>`
> +
> +.. _h264_dpb_flags:
> +
> +``DPB Entries Flags``
> +
> +.. cssclass:: longtable
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - ``V4L2_H264_DPB_ENTRY_FLAG_VALID``
> + - 0x00000001
> + - The DPB entry is valid and should be considered
> + * - ``V4L2_H264_DPB_ENTRY_FLAG_ACTIVE``
> + - 0x00000002
> + - The DPB entry is currently being used as a reference frame
> + * - ``V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM``
> + - 0x00000004
> + - The DPB entry is a long term reference frame
>
> .. _v4l2-mpeg-mpeg2:
>
> diff --git a/Documentation/media/uapi/v4l/pixfmt-compressed.rst b/Documentation/media/uapi/v4l/pixfmt-compressed.rst
> index 6c961cfb74da..ea0a8a68759b 100644
> --- a/Documentation/media/uapi/v4l/pixfmt-compressed.rst
> +++ b/Documentation/media/uapi/v4l/pixfmt-compressed.rst
> @@ -52,6 +52,25 @@ Compressed Formats
> - ``V4L2_PIX_FMT_H264_MVC``
> - 'M264'
> - H264 MVC video elementary stream.
> + * .. _V4L2-PIX-FMT-H264-SLICE:
Should be _V4L2-PIX-FMT-H264-SLICE-RAW.
> +
> + - ``V4L2_PIX_FMT_H264_SLICE_RAW``
> + - 'S264'
> + - H264 parsed slice data, as extracted from the H264 bitstream.
> + This format is adapted for stateless video decoders that
> + implement an H264 pipeline (using the :ref:`codec` and
codec is now mem2mem.
> + :ref:`media-request-api`). Metadata associated with the frame
> + to decode are required to be passed through the
> + ``V4L2_CID_MPEG_VIDEO_H264_SPS``,
> + ``V4L2_CID_MPEG_VIDEO_H264_PPS``,
> + ``V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX``,
> + ``V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS`` and
> + ``V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS`` controls. See the
> + :ref:`associated Codec Control IDs <v4l2-mpeg-h264>`.
> + Exactly one output and one capture buffer must be provided for
> + use with this pixel format. The output buffer must contain the
> + appropriate number of macroblocks to decode a full
> + corresponding frame to the matching capture buffer.
> * .. _V4L2-PIX-FMT-H263:
>
> - ``V4L2_PIX_FMT_H263``
> diff --git a/Documentation/media/uapi/v4l/vidioc-queryctrl.rst b/Documentation/media/uapi/v4l/vidioc-queryctrl.rst
> index f824162d0ea9..dc500632095d 100644
> --- a/Documentation/media/uapi/v4l/vidioc-queryctrl.rst
> +++ b/Documentation/media/uapi/v4l/vidioc-queryctrl.rst
> @@ -443,6 +443,36 @@ See also the examples in :ref:`control`.
> - n/a
> - A struct :c:type:`v4l2_ctrl_mpeg2_quantization`, containing MPEG-2
> quantization matrices for stateless video decoders.
> + * - ``V4L2_CTRL_TYPE_H264_SPS``
> + - n/a
> + - n/a
> + - n/a
> + - A struct :c:type:`v4l2_ctrl_h264_sps`, containing H264
> + sequence parameters for stateless video decoders.
> + * - ``V4L2_CTRL_TYPE_H264_PPS``
> + - n/a
> + - n/a
> + - n/a
> + - A struct :c:type:`v4l2_ctrl_h264_pps`, containing H264
> + picture parameters for stateless video decoders.
> + * - ``V4L2_CTRL_TYPE_H264_SCALING_MATRIX``
> + - n/a
> + - n/a
> + - n/a
> + - A struct :c:type:`v4l2_ctrl_h264_scaling_matrix`, containing H264
> + scaling matrices for stateless video decoders.
> + * - ``V4L2_CTRL_TYPE_H264_SLICE_PARAMS``
> + - n/a
> + - n/a
> + - n/a
> + - A struct :c:type:`v4l2_ctrl_h264_slice_params`, containing H264
> + slice parameters for stateless video decoders.
> + * - ``V4L2_CTRL_TYPE_H264_DECODE_PARAMS``
> + - n/a
> + - n/a
> + - n/a
> + - A struct :c:type:`v4l2_ctrl_h264_decode_params`, containing H264
> + decode parameters for stateless video decoders.
>
> .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
>
> diff --git a/Documentation/media/videodev2.h.rst.exceptions b/Documentation/media/videodev2.h.rst.exceptions
> index 64d348e67df9..55cbe324b9fc 100644
> --- a/Documentation/media/videodev2.h.rst.exceptions
> +++ b/Documentation/media/videodev2.h.rst.exceptions
> @@ -136,6 +136,11 @@ replace symbol V4L2_CTRL_TYPE_U32 :c:type:`v4l2_ctrl_type`
> replace symbol V4L2_CTRL_TYPE_U8 :c:type:`v4l2_ctrl_type`
> replace symbol V4L2_CTRL_TYPE_MPEG2_SLICE_PARAMS :c:type:`v4l2_ctrl_type`
> replace symbol V4L2_CTRL_TYPE_MPEG2_QUANTIZATION :c:type:`v4l2_ctrl_type`
> +replace symbol V4L2_CTRL_TYPE_H264_SPS :c:type:`v4l2_ctrl_type`
> +replace symbol V4L2_CTRL_TYPE_H264_PPS :c:type:`v4l2_ctrl_type`
> +replace symbol V4L2_CTRL_TYPE_H264_SCALING_MATRIX :c:type:`v4l2_ctrl_type`
> +replace symbol V4L2_CTRL_TYPE_H264_SLICE_PARAMS :c:type:`v4l2_ctrl_type`
> +replace symbol V4L2_CTRL_TYPE_H264_DECODE_PARAMS :c:type:`v4l2_ctrl_type`
>
> # V4L2 capability defines
> replace define V4L2_CAP_VIDEO_CAPTURE device-capabilities
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index b1ae2e555c68..46aec8c3acde 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -828,6 +828,11 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_MPEG_VIDEO_H264_CONSTRAINED_INTRA_PREDICTION:
> return "H264 Constrained Intra Pred";
> case V4L2_CID_MPEG_VIDEO_H264_CHROMA_QP_INDEX_OFFSET: return "H264 Chroma QP Index Offset";
> + case V4L2_CID_MPEG_VIDEO_H264_SPS: return "H264 Sequence Parameter Set";
> + case V4L2_CID_MPEG_VIDEO_H264_PPS: return "H264 Picture Parameter Set";
> + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: return "H264 Scaling Matrix";
> + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS: return "H264 Slice Parameters";
> + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS: return "H264 Decode Parameters";
> case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4 I-Frame QP Value";
> case V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP: return "MPEG4 P-Frame QP Value";
> case V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP: return "MPEG4 B-Frame QP Value";
> @@ -1309,6 +1314,21 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> case V4L2_CID_MPEG_VIDEO_FWHT_PARAMS:
> *type = V4L2_CTRL_TYPE_FWHT_PARAMS;
> break;
> + case V4L2_CID_MPEG_VIDEO_H264_SPS:
> + *type = V4L2_CTRL_TYPE_H264_SPS;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_PPS:
> + *type = V4L2_CTRL_TYPE_H264_PPS;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX:
> + *type = V4L2_CTRL_TYPE_H264_SCALING_MATRIX;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS:
> + *type = V4L2_CTRL_TYPE_H264_SLICE_PARAMS;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS:
> + *type = V4L2_CTRL_TYPE_H264_DECODE_PARAMS;
> + break;
> default:
> *type = V4L2_CTRL_TYPE_INTEGER;
> break;
> @@ -1678,6 +1698,13 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
> case V4L2_CTRL_TYPE_FWHT_PARAMS:
> return 0;
>
> + case V4L2_CTRL_TYPE_H264_SPS:
> + case V4L2_CTRL_TYPE_H264_PPS:
> + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
> + case V4L2_CTRL_TYPE_H264_SLICE_PARAMS:
> + case V4L2_CTRL_TYPE_H264_DECODE_PARAMS:
> + return 0;
> +
> default:
> return -EINVAL;
> }
> @@ -2261,6 +2288,21 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
> case V4L2_CTRL_TYPE_FWHT_PARAMS:
> elem_size = sizeof(struct v4l2_ctrl_fwht_params);
> break;
> + case V4L2_CTRL_TYPE_H264_SPS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_sps);
> + break;
> + case V4L2_CTRL_TYPE_H264_PPS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_pps);
> + break;
> + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
> + elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix);
> + break;
> + case V4L2_CTRL_TYPE_H264_SLICE_PARAMS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_slice_params);
> + break;
> + case V4L2_CTRL_TYPE_H264_DECODE_PARAMS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_decode_params);
> + break;
> default:
> if (type < V4L2_CTRL_COMPOUND_TYPES)
> elem_size = sizeof(s32);
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index ac87c3e37280..f6e1254064d2 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1325,6 +1325,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> case V4L2_PIX_FMT_H264: descr = "H.264"; break;
> case V4L2_PIX_FMT_H264_NO_SC: descr = "H.264 (No Start Codes)"; break;
> case V4L2_PIX_FMT_H264_MVC: descr = "H.264 MVC"; break;
> + case V4L2_PIX_FMT_H264_SLICE_RAW: descr = "H.264 Parsed Slice Data"; break;
> case V4L2_PIX_FMT_H263: descr = "H.263"; break;
> case V4L2_PIX_FMT_MPEG1: descr = "MPEG-1 ES"; break;
> case V4L2_PIX_FMT_MPEG2: descr = "MPEG-2 ES"; break;
> diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h
> new file mode 100644
> index 000000000000..e2f83b3cdbef
> --- /dev/null
> +++ b/include/media/h264-ctrls.h
> @@ -0,0 +1,192 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * These are the H.264 state controls for use with stateless H.264
> + * codec drivers.
> + *
> + * It turns out that these structs are not stable yet and will undergo
> + * more changes. So keep them private until they are stable and ready to
> + * become part of the official public API.
> + */
> +
> +#ifndef _H264_CTRLS_H_
> +#define _H264_CTRLS_H_
> +
> +/*
> + * This is put insanely high to avoid conflicting with controls that
> + * would be added during the phase where those controls are not
> + * stable. It should be fixed eventually.
> + */
> +#define V4L2_CID_MPEG_VIDEO_H264_SPS (V4L2_CID_MPEG_BASE+1000)
> +#define V4L2_CID_MPEG_VIDEO_H264_PPS (V4L2_CID_MPEG_BASE+1001)
> +#define V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (V4L2_CID_MPEG_BASE+1002)
> +#define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS (V4L2_CID_MPEG_BASE+1003)
> +#define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (V4L2_CID_MPEG_BASE+1004)
> +
> +/* enum v4l2_ctrl_type type values */
> +#define V4L2_CTRL_TYPE_H264_SPS 0x0110
> +#define V4L2_CTRL_TYPE_H264_PPS 0x0111
> +#define V4L2_CTRL_TYPE_H264_SCALING_MATRIX 0x0112
> +#define V4L2_CTRL_TYPE_H264_SLICE_PARAMS 0x0113
> +#define V4L2_CTRL_TYPE_H264_DECODE_PARAMS 0x0114
> +
> +#define V4L2_H264_SPS_CONSTRAINT_SET0_FLAG 0x01
> +#define V4L2_H264_SPS_CONSTRAINT_SET1_FLAG 0x02
> +#define V4L2_H264_SPS_CONSTRAINT_SET2_FLAG 0x04
> +#define V4L2_H264_SPS_CONSTRAINT_SET3_FLAG 0x08
> +#define V4L2_H264_SPS_CONSTRAINT_SET4_FLAG 0x10
> +#define V4L2_H264_SPS_CONSTRAINT_SET5_FLAG 0x20
> +
> +#define V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE 0x01
> +#define V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS 0x02
> +#define V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO 0x04
> +#define V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED 0x08
> +#define V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY 0x10
> +#define V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD 0x20
> +#define V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE 0x40
> +
> +struct v4l2_ctrl_h264_sps {
> + __u8 profile_idc;
> + __u8 constraint_set_flags;
> + __u8 level_idc;
> + __u8 seq_parameter_set_id;
> + __u8 chroma_format_idc;
> + __u8 bit_depth_luma_minus8;
> + __u8 bit_depth_chroma_minus8;
> + __u8 log2_max_frame_num_minus4;
> + __u8 pic_order_cnt_type;
> + __u8 log2_max_pic_order_cnt_lsb_minus4;
> + __u8 max_num_ref_frames;
> + __u8 num_ref_frames_in_pic_order_cnt_cycle;
> + __s32 offset_for_ref_frame[255];
> + __s32 offset_for_non_ref_pic;
> + __s32 offset_for_top_to_bottom_field;
> + __u16 pic_width_in_mbs_minus1;
> + __u16 pic_height_in_map_units_minus1;
> + __u32 flags;
> +};
> +
> +#define V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE 0x0001
> +#define V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT 0x0002
> +#define V4L2_H264_PPS_FLAG_WEIGHTED_PRED 0x0004
> +#define V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT 0x0008
> +#define V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED 0x0010
> +#define V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT 0x0020
> +#define V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE 0x0040
> +#define V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT 0x0080
> +
> +struct v4l2_ctrl_h264_pps {
> + __u8 pic_parameter_set_id;
> + __u8 seq_parameter_set_id;
> + __u8 num_slice_groups_minus1;
> + __u8 num_ref_idx_l0_default_active_minus1;
> + __u8 num_ref_idx_l1_default_active_minus1;
> + __u8 weighted_bipred_idc;
> + __s8 pic_init_qp_minus26;
> + __s8 pic_init_qs_minus26;
> + __s8 chroma_qp_index_offset;
> + __s8 second_chroma_qp_index_offset;
> + __u16 flags;
> +};
> +
> +struct v4l2_ctrl_h264_scaling_matrix {
> + __u8 scaling_list_4x4[6][16];
> + __u8 scaling_list_8x8[6][64];
> +};
> +
> +struct v4l2_h264_weight_factors {
> + __s16 luma_weight[32];
> + __s16 luma_offset[32];
> + __s16 chroma_weight[32][2];
> + __s16 chroma_offset[32][2];
> +};
> +
> +struct v4l2_h264_pred_weight_table {
> + __u16 luma_log2_weight_denom;
> + __u16 chroma_log2_weight_denom;
> + struct v4l2_h264_weight_factors weight_factors[2];
> +};
> +
> +#define V4L2_H264_SLICE_TYPE_P 0
> +#define V4L2_H264_SLICE_TYPE_B 1
> +#define V4L2_H264_SLICE_TYPE_I 2
> +#define V4L2_H264_SLICE_TYPE_SP 3
> +#define V4L2_H264_SLICE_TYPE_SI 4
> +
> +#define V4L2_H264_SLICE_FLAG_FIELD_PIC 0x01
> +#define V4L2_H264_SLICE_FLAG_BOTTOM_FIELD 0x02
> +#define V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED 0x04
> +#define V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH 0x08
> +
> +struct v4l2_ctrl_h264_slice_params {
> + /* Size in bytes, including header */
> + __u32 size;
> + /* Offset in bits to slice_data() from the beginning of this slice. */
> + __u32 header_bit_size;
> +
> + __u16 first_mb_in_slice;
> + __u8 slice_type;
> + __u8 pic_parameter_set_id;
> + __u8 colour_plane_id;
> + __u8 redundant_pic_cnt;
> + __u16 frame_num;
> + __u16 idr_pic_id;
> + __u16 pic_order_cnt_lsb;
> + __s32 delta_pic_order_cnt_bottom;
> + __s32 delta_pic_order_cnt0;
> + __s32 delta_pic_order_cnt1;
> +
> + struct v4l2_h264_pred_weight_table pred_weight_table;
> + /* Size in bits of dec_ref_pic_marking() syntax element. */
> + __u32 dec_ref_pic_marking_bit_size;
> + /* Size in bits of pic order count syntax. */
> + __u32 pic_order_cnt_bit_size;
> +
> + __u8 cabac_init_idc;
> + __s8 slice_qp_delta;
> + __s8 slice_qs_delta;
> + __u8 disable_deblocking_filter_idc;
> + __s8 slice_alpha_c0_offset_div2;
> + __s8 slice_beta_offset_div2;
> + __u8 num_ref_idx_l0_active_minus1;
> + __u8 num_ref_idx_l1_active_minus1;
> + __u32 slice_group_change_cycle;
> +
> + /*
> + * Entries on each list are indices into
> + * v4l2_ctrl_h264_decode_params.dpb[].
> + */
> + __u8 ref_pic_list0[32];
> + __u8 ref_pic_list1[32];
> +
> + __u32 flags;
> +};
> +
> +#define V4L2_H264_DPB_ENTRY_FLAG_VALID 0x01
> +#define V4L2_H264_DPB_ENTRY_FLAG_ACTIVE 0x02
> +#define V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM 0x04
> +
> +struct v4l2_h264_dpb_entry {
> + __u64 reference_ts;
> + __u16 frame_num;
> + __u16 pic_num;
> + /* Note that field is indicated by v4l2_buffer.field */
> + __s32 top_field_order_cnt;
> + __s32 bottom_field_order_cnt;
> + __u32 flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */
> +};
> +
> +#define V4L2_H264_DECODE_PARAM_FLAG_IDR_PIC 0x01
> +
> +struct v4l2_ctrl_h264_decode_params {
> + struct v4l2_h264_dpb_entry dpb[16];
> + __u16 num_slices;
> + __u16 nal_ref_idc;
> + __u8 ref_pic_list_p0[32];
> + __u8 ref_pic_list_b0[32];
> + __u8 ref_pic_list_b1[32];
> + __s32 top_field_order_cnt;
> + __s32 bottom_field_order_cnt;
> + __u32 flags; /* V4L2_H264_DECODE_PARAM_FLAG_* */
> +};
> +
> +#endif
> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> index bd621cec65a5..dce6f33fd749 100644
> --- a/include/media/v4l2-ctrls.h
> +++ b/include/media/v4l2-ctrls.h
> @@ -23,11 +23,12 @@
> #include <media/media-request.h>
>
> /*
> - * Include the mpeg2 and fwht stateless codec compound control definitions.
> + * Include the stateless codec compound control definitions.
> * This will move to the public headers once this API is fully stable.
> */
> #include <media/mpeg2-ctrls.h>
> #include <media/fwht-ctrls.h>
> +#include <media/h264-ctrls.h>
>
> /* forward references */
> struct file;
> @@ -51,6 +52,11 @@ struct poll_table_struct;
> * @p_mpeg2_slice_params: Pointer to a MPEG2 slice parameters structure.
> * @p_mpeg2_quantization: Pointer to a MPEG2 quantization data structure.
> * @p_fwht_params: Pointer to a FWHT stateless parameters structure.
> + * @p_h264_sps: Pointer to a struct v4l2_ctrl_h264_sps.
> + * @p_h264_pps: Pointer to a struct v4l2_ctrl_h264_pps.
> + * @p_h264_scaling_matrix: Pointer to a struct v4l2_ctrl_h264_scaling_matrix.
> + * @p_h264_slice_param: Pointer to a struct v4l2_ctrl_h264_slice_params.
> + * @p_h264_decode_param: Pointer to a struct v4l2_ctrl_h264_decode_params.
p_h264_slice_param -> p_h264_slice_params
p_h264_decode_param -> p_h264_decode_params
> * @p: Pointer to a compound value.
> */
> union v4l2_ctrl_ptr {
> @@ -63,6 +69,11 @@ union v4l2_ctrl_ptr {
> struct v4l2_ctrl_mpeg2_slice_params *p_mpeg2_slice_params;
> struct v4l2_ctrl_mpeg2_quantization *p_mpeg2_quantization;
> struct v4l2_ctrl_fwht_params *p_fwht_params;
> + struct v4l2_ctrl_h264_sps *p_h264_sps;
> + struct v4l2_ctrl_h264_pps *p_h264_pps;
> + struct v4l2_ctrl_h264_scaling_matrix *p_h264_scaling_matrix;
> + struct v4l2_ctrl_h264_slice_params *p_h264_slice_param;
> + struct v4l2_ctrl_h264_decode_params *p_h264_decode_param;
Ditto.
Regards,
Hans
> void *p;
> };
>
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 496e6453450c..838732acdefc 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -657,6 +657,7 @@ struct v4l2_pix_format {
> #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> +#define V4L2_PIX_FMT_H264_SLICE_RAW v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> #define V4L2_PIX_FMT_H263 v4l2_fourcc('H', '2', '6', '3') /* H263 */
> #define V4L2_PIX_FMT_MPEG1 v4l2_fourcc('M', 'P', 'G', '1') /* MPEG-1 ES */
> #define V4L2_PIX_FMT_MPEG2 v4l2_fourcc('M', 'P', 'G', '2') /* MPEG-2 ES */
>
^ permalink raw reply
* [PATCH v3 9/9] arm64: dts: rockchip: add video codec for rk3328
From: ayaka @ 2019-04-10 12:42 UTC (permalink / raw)
To: linux-media
Cc: Randy Li, randy.li, hverkuil, maxime.ripard, joro, nicolas,
jernej.skrabec, paul.kocialkowski, thomas.petazzoni, mchehab,
ezequiel, posciak, groeck, linux-rockchip
In-Reply-To: <20190410124226.8612-1-ayaka@soulik.info>
From: Randy Li <ayaka@soulik.info>
Having problem with vepu2
The sclk_vdec_core for RKVDEC is in gpll at vendor kernel.
Signed-off-by: Randy Li <ayaka@soulik.info>
---
arch/arm64/boot/dts/rockchip/rk3328-evb.dts | 32 ++++++
.../arm64/boot/dts/rockchip/rk3328-rock64.dts | 32 ++++++
arch/arm64/boot/dts/rockchip/rk3328.dtsi | 108 +++++++++++++++++-
3 files changed, 170 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/boot/dts/rockchip/rk3328-evb.dts b/arch/arm64/boot/dts/rockchip/rk3328-evb.dts
index 8302d86d35c4..c89714f79f93 100644
--- a/arch/arm64/boot/dts/rockchip/rk3328-evb.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3328-evb.dts
@@ -213,6 +213,18 @@
};
};
+&rkvdec {
+ status = "okay";
+};
+
+&rkvdec_mmu {
+ status = "okay";
+};
+
+&rkvdec_srv {
+ status = "okay";
+};
+
&sdio {
bus-width = <4>;
cap-sd-highspeed;
@@ -269,3 +281,23 @@
&usb_host0_ohci {
status = "okay";
};
+
+&vdpu {
+ status = "okay";
+};
+
+&vop {
+ status = "okay";
+};
+
+&vop_mmu {
+ status = "okay";
+};
+
+&vpu_mmu {
+ status = "okay";
+};
+
+&vpu_service{
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/rockchip/rk3328-rock64.dts b/arch/arm64/boot/dts/rockchip/rk3328-rock64.dts
index 2157a528276b..520e444806cc 100644
--- a/arch/arm64/boot/dts/rockchip/rk3328-rock64.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3328-rock64.dts
@@ -256,6 +256,14 @@
};
};
+&h265e {
+ status = "okay";
+};
+
+&h265e_mmu {
+ status = "okay";
+};
+
&i2s1 {
status = "okay";
@@ -300,6 +308,18 @@
};
};
+&rkvdec {
+ status = "okay";
+};
+
+&rkvdec_mmu {
+ status = "okay";
+};
+
+&rkvdec_srv {
+ status = "okay";
+};
+
&sdmmc {
bus-width = <4>;
cap-mmc-highspeed;
@@ -370,6 +390,10 @@
status = "okay";
};
+&vdpu {
+ status = "okay";
+};
+
&vop {
status = "okay";
};
@@ -377,3 +401,11 @@
&vop_mmu {
status = "okay";
};
+
+&vpu_mmu {
+ status = "okay";
+};
+
+&vpu_service{
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/rockchip/rk3328.dtsi b/arch/arm64/boot/dts/rockchip/rk3328.dtsi
index 55a72abed6e7..9b3f8d22b60a 100644
--- a/arch/arm64/boot/dts/rockchip/rk3328.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3328.dtsi
@@ -573,6 +573,27 @@
resets = <&cru SRST_GPU_A>;
};
+ venc_srv: venc-srv {
+ compatible = "rockchip,mpp-service";
+ status = "disabled";
+ };
+
+ h265e: h265e@ff330000 {
+ compatible = "rockchip,hevc-encoder-v1";
+ reg = <0x0 0xff330000 0 0x200>;
+ interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cru ACLK_H265>, <&cru PCLK_H265>,
+ <&cru SCLK_VENC_CORE>, <&cru SCLK_VENC_DSP>,
+ <&cru ACLK_RKVENC>, <&cru ACLK_AXISRAM>;
+ clock-names = "aclk_h265", "pclk_h265", "clk_core",
+ "clk_dsp", "aclk_venc", "aclk_axi2sram";
+ iommus = <&h265e_mmu>;
+ rockchip,srv = <&venc_srv>;
+ syscon = <&grf 0x040c 0x8000800 0x80000>;
+ power-domains = <&power RK3328_PD_HEVC>;
+ status = "disabled";
+ };
+
h265e_mmu: iommu@ff330200 {
compatible = "rockchip,iommu";
reg = <0x0 0xff330200 0 0x100>;
@@ -584,6 +605,25 @@
status = "disabled";
};
+ vepu: vepu@ff340000 {
+ compatible = "rockchip,vpu-encoder-v2";
+ reg = <0x0 0xff340000 0x0 0x400>;
+ interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cru ACLK_H264>, <&cru HCLK_H264>,
+ <&cru SCLK_VENC_CORE>;
+ clock-names = "aclk_vcodec", "hclk_vcodec",
+ "clk_core";
+ resets = <&cru SRST_RKVENC_H264_A>,
+ <&cru SRST_RKVENC_H264_H>;
+ reset-names = "video_a", "video_h";
+ iommus = <&vepu_mmu>;
+ rockchip,srv = <&venc_srv>;
+ syscon = <&grf 0x040c 0x8000800 0x80000>;
+ power-domains = <&power RK3328_PD_HEVC>;
+ status = "disabled";
+ };
+
+
vepu_mmu: iommu@ff340800 {
compatible = "rockchip,iommu";
reg = <0x0 0xff340800 0x0 0x40>;
@@ -595,6 +635,42 @@
status = "disabled";
};
+
+ vpu_service: vdpu-srv {
+ compatible = "rockchip,mpp-service";
+ status = "disabled";
+ };
+
+ vdpu: vpu-decoder@ff350000 {
+ compatible = "rockchip,vpu-decoder-v2";
+ reg = <0x0 0xff350400 0x0 0x400>;
+ interrupts = <GIC_SPI 9 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "irq_dec";
+ resets = <&cru SRST_VCODEC_A>, <&cru SRST_VCODEC_H>;
+ reset-names = "video_a", "video_h";
+ clocks = <&cru ACLK_VPU>, <&cru HCLK_VPU>;
+ clock-names = "aclk_vcodec", "hclk_vcodec";
+ iommus = <&vpu_mmu>;
+ power-domains = <&power RK3328_PD_VPU>;
+ rockchip,srv = <&vpu_service>;
+ status = "disabled";
+ };
+
+ avsd: avs-decoder@ff351000 {
+ compatible = "rockchip,avs-decoder-v1";
+ reg = <0x0 0xff351000 0x0 0x200>;
+ interrupts = <GIC_SPI 9 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "irq_dec";
+ clocks = <&cru ACLK_VPU>, <&cru HCLK_VPU>;
+ clock-names = "aclk_vcodec", "hclk_vcodec";
+ resets = <&cru SRST_VCODEC_A>, <&cru SRST_VCODEC_H>;
+ reset-names = "video_a", "video_h";
+ power-domains = <&power RK3328_PD_VPU>;
+ iommus = <&vpu_mmu>;
+ rockchip,srv = <&vpu_service>;
+ status = "disabled";
+ };
+
vpu_mmu: iommu@ff350800 {
compatible = "rockchip,iommu";
reg = <0x0 0xff350800 0x0 0x40>;
@@ -606,6 +682,34 @@
status = "disabled";
};
+ rkvdec_srv: rkvdec-srv {
+ compatible = "rockchip,mpp-service";
+ status = "disabled";
+ };
+
+ rkvdec: rkvdec@ff36000 {
+ compatible = "rockchip,video-decoder-v1";
+ reg = <0x0 0xff360000 0x0 0x400>;
+ interrupts = <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "irq_dec";
+ clocks = <&cru ACLK_RKVDEC>, <&cru HCLK_RKVDEC>,
+ <&cru SCLK_VDEC_CABAC>, <&cru SCLK_VDEC_CORE>;
+ clock-names = "aclk_vcodec", "hclk_vcodec", "clk_cabac",
+ "clk_core";
+ assigned-clocks = <&cru ACLK_RKVDEC_PRE>, <&cru SCLK_VDEC_CORE>;
+ assigned-clock-parents = <&cru PLL_GPLL>, <&cru PLL_GPLL>;
+ assigned-clock-rates = <500000000>, <245760000>;
+ resets = <&cru SRST_VDEC_A>, <&cru SRST_VDEC_H>,
+ <&cru SRST_VDEC_NIU_A>, <&cru SRST_VDEC_NIU_H>,
+ <&cru SRST_VDEC_CABAC>, <&cru SRST_VDEC_CORE>;
+ reset-names = "video_a", "video_h", "niu_a", "niu_h",
+ "video_cabac", "video_core";
+ iommus = <&rkvdec_mmu>;
+ power-domains = <&power RK3328_PD_VIDEO>;
+ rockchip,srv = <&rkvdec_srv>;
+ status = "disabled";
+ };
+
rkvdec_mmu: iommu@ff360480 {
compatible = "rockchip,iommu";
reg = <0x0 0xff360480 0x0 0x40>, <0x0 0xff3604c0 0x0 0x40>;
@@ -740,8 +844,8 @@
<15000000>, <15000000>,
<300000000>, <100000000>,
<400000000>, <100000000>,
- <50000000>, <100000000>,
- <100000000>, <100000000>,
+ <50000000>, <300000000>,
+ <300000000>, <300000000>,
<50000000>, <50000000>,
<50000000>, <50000000>,
<24000000>, <600000000>,
--
2.20.1
^ permalink raw reply related
* [PATCH v3 8/9] arm64: dts: rockchip: boost clocks for rk3328
From: ayaka @ 2019-04-10 12:42 UTC (permalink / raw)
To: linux-media
Cc: Randy Li, hverkuil, maxime.ripard, joro, nicolas, jernej.skrabec,
paul.kocialkowski, thomas.petazzoni, mchehab, ezequiel, posciak,
groeck, linux-rockchip
In-Reply-To: <20190410124226.8612-1-ayaka@soulik.info>
From: Randy Li <randy.li@rock-chips.com>
Or VOP won't work well.
Signed-off-by: Randy Li <randy.li@rock-chips.com>
---
arch/arm64/boot/dts/rockchip/rk3328.dtsi | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/boot/dts/rockchip/rk3328.dtsi b/arch/arm64/boot/dts/rockchip/rk3328.dtsi
index 84f14b132e8f..55a72abed6e7 100644
--- a/arch/arm64/boot/dts/rockchip/rk3328.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3328.dtsi
@@ -738,8 +738,8 @@
<0>, <24000000>,
<24000000>, <24000000>,
<15000000>, <15000000>,
- <100000000>, <100000000>,
- <100000000>, <100000000>,
+ <300000000>, <100000000>,
+ <400000000>, <100000000>,
<50000000>, <100000000>,
<100000000>, <100000000>,
<50000000>, <50000000>,
--
2.20.1
^ permalink raw reply related
* [PATCH v3 7/9] [TEST]: rkvdec: spspps address alignment
From: ayaka @ 2019-04-10 12:42 UTC (permalink / raw)
To: linux-media
Cc: ayaka, randy.li, hverkuil, maxime.ripard, joro, nicolas,
jernej.skrabec, paul.kocialkowski, thomas.petazzoni, mchehab,
ezequiel, posciak, groeck, linux-rockchip
In-Reply-To: <20190410124226.8612-1-ayaka@soulik.info>
I found the offset for cpu access is not equal to the DMA
opeartion.
Signed-off-by: ayaka <ayaka@soulik.info>
---
drivers/staging/rockchip-mpp/mpp_dev_common.h | 3 +++
drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c | 3 +++
drivers/staging/rockchip-mpp/rkvdec/avc.c | 14 +++++++-------
3 files changed, 13 insertions(+), 7 deletions(-)
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_common.h b/drivers/staging/rockchip-mpp/mpp_dev_common.h
index 36770af53a95..6718bcccde1f 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_common.h
+++ b/drivers/staging/rockchip-mpp/mpp_dev_common.h
@@ -100,6 +100,9 @@ struct mpp_session {
struct v4l2_ctrl_handler ctrl_handler;
/* TODO: FIXME: slower than helper function ? */
struct v4l2_ctrl **ctrls;
+
+ void *aux_vaddr;
+ dma_addr_t aux_addr;
};
/* The context for the a task */
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c b/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c
index 97abfdfc344f..cc1fa9737bc1 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c
+++ b/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c
@@ -382,6 +382,9 @@ static int rkvdec_open(struct file *filp)
filp->private_data = &session->fh;
pm_runtime_get_sync(mpp_dev->dev);
+ session->aux_vaddr = dmam_alloc_coherent(mpp_dev->dev, SZ_1M,
+ &session->aux_addr, GFP_KERNEL);
+
mpp_debug_leave();
return 0;
}
diff --git a/drivers/staging/rockchip-mpp/rkvdec/avc.c b/drivers/staging/rockchip-mpp/rkvdec/avc.c
index 1cb5b2208bfa..8266a990dca6 100644
--- a/drivers/staging/rockchip-mpp/rkvdec/avc.c
+++ b/drivers/staging/rockchip-mpp/rkvdec/avc.c
@@ -26,8 +26,8 @@
#include "regs.h"
#include "avc-data.h"
-static void generate_input_data(struct rkvdec_regs *p_regs,
- struct vb2_v4l2_buffer *src_buf,
+static void generate_input_data(struct mpp_session *session,
+ struct rkvdec_regs *p_regs,
const struct v4l2_ctrl_h264_sps *sps,
const struct v4l2_ctrl_h264_pps *pps,
const struct v4l2_ctrl_h264_scaling_matrix
@@ -44,8 +44,8 @@ static void generate_input_data(struct rkvdec_regs *p_regs,
stream_len = slice_param->size + 64;
- r_data = vb2_plane_vaddr(&src_buf->vb2_buf, 0);
- r_scaling_offs = ALIGN(stream_len, 16);
+ r_data = session->aux_vaddr;
+ r_scaling_offs = 0;
r_data += r_scaling_offs;
if (pps->flags & V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT) {
@@ -62,7 +62,7 @@ static void generate_input_data(struct rkvdec_regs *p_regs,
rkvdec_avc_write_sps(&rbsp, sps);
rkvdec_avc_write_pps(&rbsp, pps);
rkvdec_avc_write_pps_tail(&rbsp, scaling_addr, decode_param);
- p_regs->sw_pps_base = p_regs->sw_strm_rlc_base + r_sps_offs;
+ p_regs->sw_pps_base = session->aux_addr + r_sps_offs;
for (i = 1; i < 256; i++)
memset(r_data + r_sps_offs + i * 32, 0, 32);
@@ -72,7 +72,7 @@ static void generate_input_data(struct rkvdec_regs *p_regs,
r_rps_offs = ALIGN(r_rps_offs, 16);
rbsp_init(&rbsp, r_data + r_rps_offs, SZ_2M - r_rps_offs, 0);
rkvdec_avc_write_rps(&rbsp, sps, slice_param, decode_param);
- p_regs->sw_rps_base = p_regs->sw_strm_rlc_base + r_rps_offs;
+ p_regs->sw_rps_base = session->aux_addr + r_rps_offs;
}
static void init_hw_cfg(struct rkvdec_regs *p_regs)
@@ -238,7 +238,7 @@ int rkvdec_avc_gen_reg(struct mpp_session *session, void *regs,
dst_buf = v4l2_m2m_next_dst_buf(session->fh.m2m_ctx);
rkvdec_avc_gen_ref(p_regs, dst_buf, decode_param);
- generate_input_data(p_regs, src_buf, sps, pps, scaling, slice_param,
+ generate_input_data(session, p_regs, sps, pps, scaling, slice_param,
decode_param);
return 0;
--
2.20.1
^ permalink raw reply related
* [PATCH v3 6/9] [TEST]: rockchip: mpp: vdpu2: move qtable to input buffer
From: ayaka @ 2019-04-10 12:42 UTC (permalink / raw)
To: linux-media
Cc: Randy Li, hverkuil, maxime.ripard, joro, nicolas, jernej.skrabec,
paul.kocialkowski, thomas.petazzoni, mchehab, ezequiel, posciak,
groeck, linux-rockchip
In-Reply-To: <20190410124226.8612-1-ayaka@soulik.info>
From: Randy Li <randy.li@rock-chips.com>
I want the memory region !!!
It can save more time if those data are prepared in userspace.
Signed-off-by: Randy Li <randy.li@rock-chips.com>
---
drivers/staging/rockchip-mpp/mpp_dev_common.c | 3 +--
drivers/staging/rockchip-mpp/mpp_dev_common.h | 3 ---
drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c | 3 +++
drivers/staging/rockchip-mpp/vdpu2/mpeg2.c | 20 +++++++++++--------
4 files changed, 16 insertions(+), 13 deletions(-)
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_common.c b/drivers/staging/rockchip-mpp/mpp_dev_common.c
index 21816ad8a43b..97c4d897f168 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_common.c
+++ b/drivers/staging/rockchip-mpp/mpp_dev_common.c
@@ -1217,8 +1217,7 @@ static int rockchip_mpp_queue_init(void *priv, struct vb2_queue *src_vq,
src_vq->io_modes = VB2_MMAP | VB2_DMABUF;
src_vq->drv_priv = session;
src_vq->mem_ops = &vb2_dma_contig_memops;
- src_vq->dma_attrs = DMA_ATTR_ALLOC_SINGLE_PAGES |
- DMA_ATTR_NO_KERNEL_MAPPING;
+ src_vq->dma_attrs = DMA_ATTR_ALLOC_SINGLE_PAGES;
src_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
src_vq->min_buffers_needed = 1;
src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_common.h b/drivers/staging/rockchip-mpp/mpp_dev_common.h
index 33d7725be67b..36770af53a95 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_common.h
+++ b/drivers/staging/rockchip-mpp/mpp_dev_common.h
@@ -100,9 +100,6 @@ struct mpp_session {
struct v4l2_ctrl_handler ctrl_handler;
/* TODO: FIXME: slower than helper function ? */
struct v4l2_ctrl **ctrls;
-
- dma_addr_t qtable_addr;
- void *qtable_vaddr;
};
/* The context for the a task */
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c b/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
index 1be73ab9c2be..92d68b962fe1 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
+++ b/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
@@ -176,6 +176,9 @@ static int rkvdpu_s_fmt_vid_out_mplane(struct file *filp, void *priv,
if (sizes >= SZ_16M)
return -EINVAL;
+ /* For those slice header data */
+ pix_mp->plane_fmt[pix_mp->num_planes - 1].sizeimage += SZ_1M;
+
if (vdpu_setup_ctrls(session, pix_mp->pixelformat))
return -EINVAL;
diff --git a/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c b/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
index 837ee4a4a000..c12d1a8ef2da 100644
--- a/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
+++ b/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
@@ -113,7 +113,10 @@ int rkvdpu_mpeg2_gen_reg(struct mpp_session *session, void *regs,
const struct v4l2_ctrl_mpeg2_slice_params *params;
const struct v4l2_mpeg2_sequence *sequence;
const struct v4l2_mpeg2_picture *picture;
+ const struct v4l2_ctrl_mpeg2_quantization *quantization;
struct vdpu2_regs *p_regs = regs;
+ void *qtable = NULL;
+ size_t stream_len = 0;
params = rockchip_mpp_get_cur_ctrl(session,
V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS);
@@ -122,6 +125,8 @@ int rkvdpu_mpeg2_gen_reg(struct mpp_session *session, void *regs,
sequence = ¶ms->sequence;
picture = ¶ms->picture;
+ quantization = rockchip_mpp_get_cur_ctrl(session,
+ V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION);
init_hw_cfg(p_regs);
@@ -198,7 +203,13 @@ int rkvdpu_mpeg2_gen_reg(struct mpp_session *session, void *regs,
p_regs->sw64.rlc_vlc_base =
vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf, 0);
p_regs->sw122.strm_start_bit = params->data_bit_offset;
- p_regs->sw51.stream_len = vb2_get_plane_payload(&src_buf->vb2_buf, 0);
+ stream_len = vb2_get_plane_payload(&src_buf->vb2_buf, 0);
+ p_regs->sw51.stream_len = stream_len;
+
+ qtable = vb2_plane_vaddr(&src_buf->vb2_buf, 0) + ALIGN(stream_len, 8);
+ mpeg2_dec_copy_qtable(qtable, quantization);
+ p_regs->sw61.qtable_base = p_regs->sw64.rlc_vlc_base
+ + ALIGN(stream_len, 8);
return 0;
}
@@ -207,7 +218,6 @@ int rkvdpu_mpeg2_prepare_buf(struct mpp_session *session, void *regs)
{
const struct v4l2_ctrl_mpeg2_slice_params *params;
const struct v4l2_mpeg2_sequence *sequence;
- const struct v4l2_ctrl_mpeg2_quantization *quantization;
const struct v4l2_mpeg2_picture *picture;
struct vb2_v4l2_buffer *dst_buf;
dma_addr_t cur_addr, fwd_addr, bwd_addr;
@@ -220,9 +230,6 @@ int rkvdpu_mpeg2_prepare_buf(struct mpp_session *session, void *regs)
picture = ¶ms->picture;
sequence = ¶ms->sequence;
- quantization = rockchip_mpp_get_cur_ctrl(session,
- V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION);
-
dst_buf = v4l2_m2m_next_dst_buf(session->fh.m2m_ctx);
cur_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
@@ -266,8 +273,5 @@ int rkvdpu_mpeg2_prepare_buf(struct mpp_session *session, void *regs)
p_regs->sw135.refer3_base = cur_addr >> 2;
}
- mpeg2_dec_copy_qtable(session->qtable_vaddr, quantization);
- p_regs->sw61.qtable_base = session->qtable_addr;
-
return 0;
}
--
2.20.1
^ permalink raw reply related
* [PATCH v3 5/9] [TEST]: rockchip: mpp: support qtable
From: ayaka @ 2019-04-10 12:42 UTC (permalink / raw)
To: linux-media
Cc: Randy Li, hverkuil, maxime.ripard, joro, nicolas, jernej.skrabec,
paul.kocialkowski, thomas.petazzoni, mchehab, ezequiel, posciak,
groeck, linux-rockchip
In-Reply-To: <20190410124226.8612-1-ayaka@soulik.info>
From: Randy Li <randy.li@rock-chips.com>
I don't care, I don't want to store buffers for a session.
I just want to use it to verify the FFmpeg.
---
drivers/staging/rockchip-mpp/mpp_dev_common.h | 3 +++
drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c | 5 ++++-
drivers/staging/rockchip-mpp/vdpu2/mpeg2.c | 13 ++++++++-----
3 files changed, 15 insertions(+), 6 deletions(-)
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_common.h b/drivers/staging/rockchip-mpp/mpp_dev_common.h
index 36770af53a95..33d7725be67b 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_common.h
+++ b/drivers/staging/rockchip-mpp/mpp_dev_common.h
@@ -100,6 +100,9 @@ struct mpp_session {
struct v4l2_ctrl_handler ctrl_handler;
/* TODO: FIXME: slower than helper function ? */
struct v4l2_ctrl **ctrls;
+
+ dma_addr_t qtable_addr;
+ void *qtable_vaddr;
};
/* The context for the a task */
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c b/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
index dbd9f334562e..1be73ab9c2be 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
+++ b/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
@@ -246,6 +246,9 @@ static int rkvdpu_open(struct file *filp)
if (IS_ERR_OR_NULL(session))
return PTR_ERR(session);
+ session->qtable_vaddr = dmam_alloc_coherent(mpp_dev->dev, 64 * 4,
+ &session->qtable_addr,
+ GFP_KERNEL);
filp->private_data = &session->fh;
pm_runtime_get_sync(mpp_dev->dev);
@@ -529,7 +532,7 @@ static int rockchip_mpp_rkvdpu_probe(struct platform_device *pdev)
ret = mpp_dev_register_node(mpp_dev, mpp_dev->variant->node_name,
NULL, &rkvdpu_ioctl_ops);
if (ret)
- dev_err(dev, "register char device failed: %d\n", ret);
+ dev_err(dev, "register v4l2/media device failed: %d\n", ret);
memcpy(mpp_dev->fmt_out, fmt_out_templ, sizeof(fmt_out_templ));
memcpy(mpp_dev->fmt_cap, fmt_cap_templ, sizeof(fmt_cap_templ));
diff --git a/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c b/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
index d32958c4cb20..837ee4a4a000 100644
--- a/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
+++ b/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
@@ -52,7 +52,7 @@ static const u8 intra_default_q_matrix[64] = {
static void mpeg2_dec_copy_qtable(u8 * qtable, const struct v4l2_ctrl_mpeg2_quantization
*ctrl)
{
- int i, n;
+ int i;
if (!qtable || !ctrl)
return;
@@ -111,16 +111,12 @@ int rkvdpu_mpeg2_gen_reg(struct mpp_session *session, void *regs,
struct vb2_v4l2_buffer *src_buf)
{
const struct v4l2_ctrl_mpeg2_slice_params *params;
- const struct v4l2_ctrl_mpeg2_quantization *quantization;
const struct v4l2_mpeg2_sequence *sequence;
const struct v4l2_mpeg2_picture *picture;
struct vdpu2_regs *p_regs = regs;
params = rockchip_mpp_get_cur_ctrl(session,
V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS);
- quantization = rockchip_mpp_get_cur_ctrl(session,
- V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION);
-
if (!params)
return -EINVAL;
@@ -211,6 +207,7 @@ int rkvdpu_mpeg2_prepare_buf(struct mpp_session *session, void *regs)
{
const struct v4l2_ctrl_mpeg2_slice_params *params;
const struct v4l2_mpeg2_sequence *sequence;
+ const struct v4l2_ctrl_mpeg2_quantization *quantization;
const struct v4l2_mpeg2_picture *picture;
struct vb2_v4l2_buffer *dst_buf;
dma_addr_t cur_addr, fwd_addr, bwd_addr;
@@ -223,6 +220,9 @@ int rkvdpu_mpeg2_prepare_buf(struct mpp_session *session, void *regs)
picture = ¶ms->picture;
sequence = ¶ms->sequence;
+ quantization = rockchip_mpp_get_cur_ctrl(session,
+ V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION);
+
dst_buf = v4l2_m2m_next_dst_buf(session->fh.m2m_ctx);
cur_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
@@ -266,5 +266,8 @@ int rkvdpu_mpeg2_prepare_buf(struct mpp_session *session, void *regs)
p_regs->sw135.refer3_base = cur_addr >> 2;
}
+ mpeg2_dec_copy_qtable(session->qtable_vaddr, quantization);
+ p_regs->sw61.qtable_base = session->qtable_addr;
+
return 0;
}
--
2.20.1
^ permalink raw reply related
* [PATCH v3 4/9] [WIP]: rockchip: mpp: H.264 decoder ctrl data
From: ayaka @ 2019-04-10 12:42 UTC (permalink / raw)
To: linux-media
Cc: ayaka, randy.li, hverkuil, maxime.ripard, joro, nicolas,
jernej.skrabec, paul.kocialkowski, thomas.petazzoni, mchehab,
ezequiel, posciak, groeck, linux-rockchip
In-Reply-To: <20190410124226.8612-1-ayaka@soulik.info>
I really don't want to do this.
Signed-off-by: Randy Li <randy.li@rock-chips.com>
Signed-off-by: ayaka <ayaka@soulik.info>
---
drivers/staging/rockchip-mpp/Makefile | 2 +-
drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c | 8 +-
.../staging/rockchip-mpp/rkvdec/avc-data.c | 239 ++++++++++++++++++
.../staging/rockchip-mpp/rkvdec/avc-data.h | 40 +++
drivers/staging/rockchip-mpp/rkvdec/avc.c | 71 +++++-
drivers/staging/rockchip-mpp/vdpu2/avc.c | 165 ++++++++++++
6 files changed, 514 insertions(+), 11 deletions(-)
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/avc-data.c
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/avc-data.h
create mode 100644 drivers/staging/rockchip-mpp/vdpu2/avc.c
diff --git a/drivers/staging/rockchip-mpp/Makefile b/drivers/staging/rockchip-mpp/Makefile
index 8da33fa5142d..e2c2bf297812 100644
--- a/drivers/staging/rockchip-mpp/Makefile
+++ b/drivers/staging/rockchip-mpp/Makefile
@@ -2,7 +2,7 @@
rk-mpp-service-objs := mpp_service.o
rk-mpp-device-objs := mpp_dev_common.o
rk-mpp-vdec-objs := mpp_dev_rkvdec.o
-rk-mpp-vdec-objs += rkvdec/avc.o
+rk-mpp-vdec-objs += rkvdec/avc.o rkvdec/avc-data.o
rk-mpp-vdec-objs += rkvdec/hevc.o rkvdec/hevc-data.o rkvdec/rbsp.o
rk-mpp-vdpu2-objs := mpp_dev_vdpu2.o vdpu2/mpeg2.o
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c b/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c
index 756821dbf829..97abfdfc344f 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c
+++ b/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c
@@ -284,13 +284,15 @@ static int rkvdec_s_fmt_vid_cap_mplane(struct file *filp, void *priv,
pix_mp->plane_fmt[1].bytesperline * ALIGN(pix_mp->height,
8);
#else
- /* TODO: HEVC only request the height is aligned with 8 */
+ /*
+ * TODO: H.264 would use 16 alignment while the resolution is under HD,
+ * HEVC only request the height is aligned with 8
+ */
pix_mp->plane_fmt[0].sizeimage =
pix_mp->plane_fmt[0].bytesperline * ALIGN(pix_mp->height,
- 16);
+ 8);
/* Additional space for motion vector */
pix_mp->plane_fmt[0].sizeimage *= 2;
- pix_mp->plane_fmt[0].sizeimage += SZ_4M;
pix_mp->plane_fmt[1].sizeimage = SZ_2M;
#endif
pix_mp->num_planes = 2;
diff --git a/drivers/staging/rockchip-mpp/rkvdec/avc-data.c b/drivers/staging/rockchip-mpp/rkvdec/avc-data.c
new file mode 100644
index 000000000000..57172528f988
--- /dev/null
+++ b/drivers/staging/rockchip-mpp/rkvdec/avc-data.c
@@ -0,0 +1,239 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2019 Randy Li, <ayaka@soulik.info>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include "avc-data.h"
+
+static const u32 zig_zag_4x4[16] = {
+ 0, 1, 4, 8, 5, 2, 3, 6, 9, 12, 13, 10, 7, 11, 14, 15
+};
+
+static const u32 zig_zag_8x8[64] = {
+ 0, 1, 8, 16, 9, 2, 3, 10, 17, 24, 32, 25, 18, 11, 4, 5,
+ 12, 19, 26, 33, 40, 48, 41, 34, 27, 20, 13, 6, 7, 14, 21, 28,
+ 35, 42, 49, 56, 57, 50, 43, 36, 29, 22, 15, 23, 30, 37, 44, 51,
+ 58, 59, 52, 45, 38, 31, 39, 46, 53, 60, 61, 54, 47, 55, 62, 63
+};
+
+static void fill_is_long_term(struct rbsp *rbsp, const struct v4l2_ctrl_h264_decode_param
+ *decode_param)
+{
+ u16 is_long_term = 0;
+ u8 i;
+
+ for (i = 0; i < 16; i++)
+ if (decode_param->dpb[i].
+ flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
+ is_long_term |= (1 << i);
+
+ rbsp_write_bits(rbsp, 16, is_long_term);
+}
+
+/* in zig-zag order */
+void rkvdec_avc_update_scaling_list(u8 * buf, const struct
+ v4l2_ctrl_h264_scaling_matrix
+ *scaling)
+{
+ u8 i, j;
+
+ for (i = 0; i < 6; i++)
+ for (j = 0; j < 16; j++)
+ buf[zig_zag_4x4[j] + (i << 4)] =
+ scaling->scaling_list_4x4[i][j];
+
+ for (i = 0; i < 2; i++)
+ for (j = 0; j < 64; j++)
+ buf[zig_zag_8x8[j] + (i << 6)] =
+ scaling->scaling_list_8x8[i][j];
+}
+
+int rkvdec_avc_write_sps(struct rbsp *rbsp,
+ const struct v4l2_ctrl_h264_sps *sps)
+{
+ /* TODO: seq_parameter_set_id */
+ rbsp_write_bits(rbsp, 4, 0);
+ rbsp_write_bits(rbsp, 8, sps->profile_idc);
+ /* constraint_set3_flag */
+ rbsp_write_flag(rbsp, sps->constraint_set_flags >> 3);
+ rbsp_write_bits(rbsp, 2, sps->chroma_format_idc);
+ /* bit_depth_luma Not used */
+ rbsp_write_bits(rbsp, 3, sps->bit_depth_luma_minus8);
+ /* bit_depth_chroma Not used */
+ rbsp_write_bits(rbsp, 3, sps->bit_depth_chroma_minus8);
+ /* TODO: qpprime_y_zero_transform_bypass_flag */
+ rbsp_write_flag(rbsp, 0);
+ rbsp_write_bits(rbsp, 4, sps->log2_max_frame_num_minus4);
+ rbsp_write_bits(rbsp, 5, sps->max_num_ref_frames);
+ rbsp_write_bits(rbsp, 2, sps->pic_order_cnt_type);
+ rbsp_write_bits(rbsp, 4, sps->log2_max_pic_order_cnt_lsb_minus4);
+ /* delta_pic_order_always_zero_flag */
+ rbsp_write_flag(rbsp,
+ sps->flags &
+ V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO);
+ rbsp_write_bits(rbsp, 9, sps->pic_width_in_mbs_minus1 + 1);
+ /* TODO: check whether it work for field coding */
+ rbsp_write_bits(rbsp, 9, sps->pic_height_in_map_units_minus1 + 1);
+ rbsp_write_flag(rbsp, sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY);
+ rbsp_write_flag(rbsp, sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD);
+ rbsp_write_flag(rbsp,
+ sps->flags & V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE);
+
+ /* TODO: mvc */
+ rbsp_write_flag(rbsp, 0);
+ /* num_views_minus1 */
+ rbsp_write_bits(rbsp, 2, 0);
+ /* view_id[0] */
+ rbsp_write_bits(rbsp, 10, 0);
+ /* view_id[1] */
+ rbsp_write_bits(rbsp, 10, 0);
+ /* num_anchor_refs_l0 */
+ rbsp_write_flag(rbsp, 0);
+ /* anchor_ref_l0 */
+ rbsp_write_bits(rbsp, 10, 0);
+ rbsp_write_flag(rbsp, 0);
+ rbsp_write_bits(rbsp, 10, 0);
+ /* num_non_anchor_refs_l0 */
+ rbsp_write_flag(rbsp, 0);
+ rbsp_write_bits(rbsp, 10, 0);
+ rbsp_write_flag(rbsp, 0);
+ rbsp_write_bits(rbsp, 10, 0);
+ /* Align with 128 bit */
+ rbsp_write_bits(rbsp, 2, 0);
+
+ return 0;
+}
+
+int rkvdec_avc_write_pps(struct rbsp *rbsp,
+ const struct v4l2_ctrl_h264_pps *pps)
+{
+ /* TODO: pps_pic_parameter_set_id */
+ rbsp_write_bits(rbsp, 8, 0);
+ rbsp_write_bits(rbsp, 5, 0);
+ rbsp_write_flag(rbsp,
+ pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE);
+ rbsp_write_flag(rbsp,
+ pps->flags &
+ V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT);
+ rbsp_write_bits(rbsp, 5, pps->num_ref_idx_l0_default_active_minus1);
+ rbsp_write_bits(rbsp, 5, pps->num_ref_idx_l1_default_active_minus1);
+ rbsp_write_flag(rbsp, pps->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED);
+ rbsp_write_bits(rbsp, 2, pps->weighted_bipred_idc);
+ rbsp_write_bits(rbsp, 7, pps->pic_init_qp_minus26);
+ rbsp_write_bits(rbsp, 6, pps->pic_init_qs_minus26);
+ rbsp_write_bits(rbsp, 5, pps->chroma_qp_index_offset);
+ rbsp_write_flag(rbsp,
+ pps->flags &
+ V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT);
+ rbsp_write_flag(rbsp,
+ pps->flags & V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED);
+ rbsp_write_flag(rbsp,
+ pps->flags &
+ V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT);
+ rbsp_write_flag(rbsp,
+ pps->flags & V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE);
+ rbsp_write_bits(rbsp, 5, pps->second_chroma_qp_index_offset);
+ rbsp_write_flag(rbsp,
+ pps->flags &
+ V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT);
+
+ return 0;
+}
+
+int rkvdec_avc_write_pps_tail(struct rbsp *rbsp, dma_addr_t scaling_addr, const struct v4l2_ctrl_h264_decode_param
+ *decode_param)
+{
+ /* scaling list buffer */
+ rbsp_write_bits(rbsp, 32, scaling_addr);
+
+ /* DPB */
+ fill_is_long_term(rbsp, decode_param);
+
+ /* TODO: VOIdx, Layer id */
+ rbsp_write_bits(rbsp, 16, 0);
+
+ /* Align with 128 bit */
+ rbsp_write_bits(rbsp, 8, 0);
+
+ return 0;
+}
+
+static inline void fill_rps_list(struct rbsp *rbsp, const struct v4l2_ctrl_h264_decode_param
+ *decode_param, const u8 * list)
+{
+ u8 i;
+
+ for (i = 0; i < 32; i++) {
+ u8 idx, active_flag;
+
+ idx = list[i];
+
+ active_flag = decode_param->dpb[idx].flags &
+ V4L2_H264_DPB_ENTRY_FLAG_ACTIVE;
+ if (!active_flag) {
+ rbsp_write_bits(rbsp, 7, 0);
+ } else {
+ rbsp_write_bits(rbsp, 5, idx | BIT(5));
+ /* TODO: bottom flag */
+ rbsp_write_flag(rbsp, 0);
+ /* TODO: view id */
+ rbsp_write_flag(rbsp, 0);
+ }
+ }
+}
+
+int rkvdec_avc_write_rps(struct rbsp *rbsp,
+ const struct v4l2_ctrl_h264_sps *sps,
+ const struct v4l2_ctrl_h264_slice_param *slice_param,
+ const struct v4l2_ctrl_h264_decode_param *decode_param)
+{
+ int max_frame_num = 1 << (sps->log2_max_frame_num_minus4 + 4);
+ u8 i;
+
+ for (i = 0; i < 16; i++) {
+ u16 frame_num = decode_param->dpb[i].frame_num;
+
+ rbsp_write_bits(rbsp, 16, frame_num > max_frame_num ?
+ frame_num - max_frame_num : frame_num);
+ }
+
+ /* reserved */
+ rbsp_write_bits(rbsp, 16, 0);
+ /* TODO: VoidX */
+ rbsp_write_bits(rbsp, 16, 0);
+
+ switch (slice_param->slice_type) {
+ case V4L2_H264_SLICE_TYPE_P:
+ fill_rps_list(rbsp, decode_param, slice_param->ref_pic_list0);
+ for (i = 0; i < 14; i++)
+ rbsp_write_bits(rbsp, 32, 0);
+ break;
+ case V4L2_H264_SLICE_TYPE_B:
+ for (i = 0; i < 7; i++)
+ rbsp_write_bits(rbsp, 32, 0);
+ fill_rps_list(rbsp, decode_param, slice_param->ref_pic_list0);
+ fill_rps_list(rbsp, decode_param, slice_param->ref_pic_list1);
+ break;
+ case V4L2_H264_SLICE_TYPE_I:
+ /* TODO: SVC */
+ default:
+ for (i = 0; i < 21; i++)
+ rbsp_write_bits(rbsp, 32, 0);
+ break;
+ }
+
+ rbsp_write_bits(rbsp, 32, 0);
+ rbsp_write_bits(rbsp, 32, 0);
+
+ return 0;
+}
diff --git a/drivers/staging/rockchip-mpp/rkvdec/avc-data.h b/drivers/staging/rockchip-mpp/rkvdec/avc-data.h
new file mode 100644
index 000000000000..38ad17273b8a
--- /dev/null
+++ b/drivers/staging/rockchip-mpp/rkvdec/avc-data.h
@@ -0,0 +1,40 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2019 Randy Li, <ayaka@soulik.info>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <linux/types.h>
+
+#include <linux/videodev2.h>
+#include <media/v4l2-ctrls.h>
+
+#include "rbsp.h"
+
+void rkvdec_avc_update_scaling_list(u8 * buf, const struct
+ v4l2_ctrl_h264_scaling_matrix
+ *scaling);
+
+int rkvdec_avc_write_sps(struct rbsp *rbsp,
+ const struct v4l2_ctrl_h264_sps *sps);
+
+int rkvdec_avc_write_pps(struct rbsp *rbsp,
+ const struct v4l2_ctrl_h264_pps *pps);
+
+int rkvdec_avc_write_pps_tail(struct rbsp *rbsp, dma_addr_t scaling_addr,
+ const struct v4l2_ctrl_h264_decode_param *decode_param);
+
+int rkvdec_avc_write_rps(struct rbsp *rbsp,
+ const struct v4l2_ctrl_h264_sps *sps,
+ const struct v4l2_ctrl_h264_slice_param *slice_param,
+ const struct v4l2_ctrl_h264_decode_param
+ *decode_param);
diff --git a/drivers/staging/rockchip-mpp/rkvdec/avc.c b/drivers/staging/rockchip-mpp/rkvdec/avc.c
index 3d91a119e533..1cb5b2208bfa 100644
--- a/drivers/staging/rockchip-mpp/rkvdec/avc.c
+++ b/drivers/staging/rockchip-mpp/rkvdec/avc.c
@@ -24,6 +24,56 @@
#include "mpp_dev_common.h"
#include "hal.h"
#include "regs.h"
+#include "avc-data.h"
+
+static void generate_input_data(struct rkvdec_regs *p_regs,
+ struct vb2_v4l2_buffer *src_buf,
+ const struct v4l2_ctrl_h264_sps *sps,
+ const struct v4l2_ctrl_h264_pps *pps,
+ const struct v4l2_ctrl_h264_scaling_matrix
+ *scaling, const struct v4l2_ctrl_h264_slice_param
+ *slice_param, const struct v4l2_ctrl_h264_decode_param
+ *decode_param)
+{
+ struct rbsp rbsp = { 0, };
+ size_t r_scaling_offs, r_sps_offs, r_rps_offs;
+ size_t stream_len = 0;
+ dma_addr_t scaling_addr = 0;
+ void *r_data = NULL;
+ int i;
+
+ stream_len = slice_param->size + 64;
+
+ r_data = vb2_plane_vaddr(&src_buf->vb2_buf, 0);
+ r_scaling_offs = ALIGN(stream_len, 16);
+ r_data += r_scaling_offs;
+
+ if (pps->flags & V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT) {
+ rkvdec_avc_update_scaling_list(r_data, scaling);
+ r_sps_offs = r_scaling_offs + 6 * 16 + 2 * 64 + 128;
+ r_sps_offs = ALIGN(r_sps_offs, 16);
+ scaling_addr = p_regs->sw_strm_rlc_base + r_scaling_offs;
+ } else {
+ r_sps_offs = r_scaling_offs;
+ scaling_addr = 0;
+ }
+
+ rbsp_init(&rbsp, r_data + r_sps_offs, SZ_2M - r_sps_offs, 0);
+ rkvdec_avc_write_sps(&rbsp, sps);
+ rkvdec_avc_write_pps(&rbsp, pps);
+ rkvdec_avc_write_pps_tail(&rbsp, scaling_addr, decode_param);
+ p_regs->sw_pps_base = p_regs->sw_strm_rlc_base + r_sps_offs;
+
+ for (i = 1; i < 256; i++)
+ memset(r_data + r_sps_offs + i * 32, 0, 32);
+
+ /* 256 bits */
+ r_rps_offs = r_sps_offs + 32 * 256 + 128;
+ r_rps_offs = ALIGN(r_rps_offs, 16);
+ rbsp_init(&rbsp, r_data + r_rps_offs, SZ_2M - r_rps_offs, 0);
+ rkvdec_avc_write_rps(&rbsp, sps, slice_param, decode_param);
+ p_regs->sw_rps_base = p_regs->sw_strm_rlc_base + r_rps_offs;
+}
static void init_hw_cfg(struct rkvdec_regs *p_regs)
{
@@ -152,18 +202,23 @@ int rkvdec_avc_gen_reg(struct mpp_session *session, void *regs,
{
const struct v4l2_ctrl_h264_sps *sps;
const struct v4l2_ctrl_h264_pps *pps;
+ const struct v4l2_ctrl_h264_scaling_matrix *scaling;
const struct v4l2_ctrl_h264_slice_param *slice_param;
const struct v4l2_ctrl_h264_decode_param *decode_param;
struct vb2_v4l2_buffer *dst_buf;
struct rkvdec_regs *p_regs = regs;
- size_t stream_len = 0;
sps = rockchip_mpp_get_cur_ctrl(session, V4L2_CID_MPEG_VIDEO_H264_SPS);
pps = rockchip_mpp_get_cur_ctrl(session, V4L2_CID_MPEG_VIDEO_H264_PPS);
- slice_param = rockchip_mpp_get_cur_ctrl(session,
- V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS);
- decode_param = rockchip_mpp_get_cur_ctrl(session,
- V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS);
+ scaling =
+ rockchip_mpp_get_cur_ctrl(session,
+ V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS);
+ slice_param =
+ rockchip_mpp_get_cur_ctrl(session,
+ V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS);
+ decode_param =
+ rockchip_mpp_get_cur_ctrl(session,
+ V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS);
if (!sps || !pps || !slice_param || !decode_param)
return -EINVAL;
@@ -178,12 +233,14 @@ int rkvdec_avc_gen_reg(struct mpp_session *session, void *regs,
p_regs->sw_sysctrl.strm_start_bit = slice_param->header_bit_size;
/* hardware wants a zerod memory at the stream end */
- stream_len = slice_param->size + 64;
- p_regs->sw_stream_len = stream_len;
+ p_regs->sw_stream_len = slice_param->size + 64;
dst_buf = v4l2_m2m_next_dst_buf(session->fh.m2m_ctx);
rkvdec_avc_gen_ref(p_regs, dst_buf, decode_param);
+ generate_input_data(p_regs, src_buf, sps, pps, scaling, slice_param,
+ decode_param);
+
return 0;
}
diff --git a/drivers/staging/rockchip-mpp/vdpu2/avc.c b/drivers/staging/rockchip-mpp/vdpu2/avc.c
new file mode 100644
index 000000000000..f77bb8ef810a
--- /dev/null
+++ b/drivers/staging/rockchip-mpp/vdpu2/avc.c
@@ -0,0 +1,165 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2019 Randy Li, <ayaka@soulik.info>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <linux/types.h>
+
+#include <linux/videodev2.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mpp_dev_common.h"
+#include "hal.h"
+#include "regs.h"
+
+#define DEC_LITTLE_ENDIAN (1)
+
+static const u8 zigzag[64] = {
+ 0, 1, 8, 16, 9, 2, 3, 10,
+ 17, 24, 32, 25, 18, 11, 4, 5,
+ 12, 19, 26, 33, 40, 48, 41, 34,
+ 27, 20, 13, 6, 7, 14, 21, 28,
+ 35, 42, 49, 56, 57, 50, 43, 36,
+ 29, 22, 15, 23, 30, 37, 44, 51,
+ 58, 59, 52, 45, 38, 31, 39, 46,
+ 53, 60, 61, 54, 47, 55, 62, 63
+};
+
+static const u8 intra_default_q_matrix[64] = {
+ 8, 16, 19, 22, 26, 27, 29, 34,
+ 16, 16, 22, 24, 27, 29, 34, 37,
+ 19, 22, 26, 27, 29, 34, 34, 38,
+ 22, 22, 26, 27, 29, 34, 37, 40,
+ 22, 26, 27, 29, 32, 35, 40, 48,
+ 26, 27, 29, 32, 35, 40, 48, 58,
+ 26, 27, 29, 34, 38, 46, 56, 69,
+ 27, 29, 35, 38, 46, 56, 69, 83
+};
+
+static void init_hw_cfg(struct vdpu2_regs *p_regs)
+{
+ p_regs->sw54.dec_strm_wordsp = 1;
+ p_regs->sw54.dec_strendian_e = DEC_LITTLE_ENDIAN;
+ p_regs->sw54.dec_in_wordsp = 1;
+ p_regs->sw54.dec_out_wordsp = 1;
+ p_regs->sw54.dec_in_endian = DEC_LITTLE_ENDIAN; //change
+ p_regs->sw54.dec_out_endian = DEC_LITTLE_ENDIAN;
+ p_regs->sw57.dec_timeout = 1;
+ p_regs->sw57.dec_timeout_e = 1;
+
+ p_regs->sw57.dec_clk_gate_e = 1;
+ p_regs->sw57.pref_sigchan = 1;
+ p_regs->sw57.bus_pos_sel = 1;
+ p_regs->sw57.intra_dbl3t = 1;
+ p_regs->sw57.inter_dblspeed = 1;
+ p_regs->sw57.intra_dblspeed = 1;
+
+ p_regs->sw50.tiled_mode_msb = 0;
+ p_regs->sw56.dec_max_burst = 16;
+ p_regs->sw50.dec_scmd_dis = 0;
+ p_regs->sw50.dec_adv_pre_dis = 0;
+ p_regs->sw52.apf_threshold = 8;
+
+ p_regs->sw50.dec_latency = 0;
+ p_regs->sw56.dec_data_disc_e = 0;
+
+ p_regs->sw55.dec_irq = 0;
+ p_regs->sw56.dec_axi_rd_id = 0;
+ p_regs->sw56.dec_axi_wr_id = 0;
+
+ p_reg->sw59.pred_bc_tap_0_0 = 1;
+ /* -5 */
+ p_reg->sw59.pred_bc_tap_0_1 = 0x3fb;
+ p_reg->sw59.pred_bc_tap_0_2 = 20;
+
+ p_regs->sw53.dec_mode = RKVDPU2_FMT_H264D;
+}
+
+int rkvdpu_mpeg2_gen_reg(struct mpp_session *session, void *regs,
+ struct vb2_v4l2_buffer *src_buf)
+{
+ const struct v4l2_ctrl_h264_sps *sps;
+ const struct v4l2_ctrl_h264_pps *pps;
+ const struct v4l2_ctrl_h264_scaling_matrix *scaling;
+ const struct v4l2_ctrl_h264_slice_param *slice_param;
+ const struct v4l2_ctrl_h264_decode_param *decode_param;
+ struct vb2_v4l2_buffer *dst_buf;
+ struct rkvdec_regs *p_regs = regs;
+
+ sps = rockchip_mpp_get_cur_ctrl(session, V4L2_CID_MPEG_VIDEO_H264_SPS);
+ pps = rockchip_mpp_get_cur_ctrl(session, V4L2_CID_MPEG_VIDEO_H264_PPS);
+ scaling =
+ rockchip_mpp_get_cur_ctrl(session,
+ V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS);
+ slice_param =
+ rockchip_mpp_get_cur_ctrl(session,
+ V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS);
+ decode_param =
+ rockchip_mpp_get_cur_ctrl(session,
+ V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS);
+
+ if (!sps || !pps || !slice_param || !decode_param)
+ return -EINVAL;
+
+ init_hw_cfg(p_regs);
+
+ p_regs->sw120.pic_mb_width = sps->pic_width_in_mbs_minus1 + 1;
+ p_regs->sw120.pic_mb_height_p = sps->pic_height_in_map_units_minus1 + 1;
+
+#if 0
+ /* PICT_FRAME */
+ if (picture->picture_structure == 3) {
+ p_regs->sw57.pic_fieldmode_e = 0;
+ } else {
+ p_regs->sw57.pic_fieldmode_e = 1;
+ /* PICT_TOP_FIEL */
+ if (picture->picture_structure == 1)
+ p_regs->sw57.pic_topfield_e = 1;
+ }
+#endif
+ p_regs->sw51.qp_init = pps->pic_init_qp_minus26 + 26;
+ p_regs->sw114.max_refidx0 = slice_params->num_ref_idx_l0_active_minus1 + 1;
+ p_regs->sw111.max_refnum = sps->num_ref_frames;
+
+ p_regs->sw115.const_intra_en = pps->constrained_intra_pred_flag;
+
+#if 0
+ p_regs->sw112.dblk_ctrl_flag = pp->deblocking_filter_control_present_flag;
+ p_regs->sw112.rpcp_flag = pp->redundant_pic_cnt_present_flag;
+ p_regs->sw113.refpic_mk_len = p_hal->slice_long[0].drpm_used_bitlen;
+ p_regs->sw115.idr_pic_flag = p_hal->slice_long[0].idr_flag;
+ p_regs->sw113.idr_pic_id = p_hal->slice_long[0].idr_pic_id;
+ p_regs->sw114.pps_id = p_hal->slice_long[0].active_pps_id;
+ p_regs->sw114.poc_field_len = p_hal->slice_long[0].poc_used_bitlen;
+#endif
+
+ p_regs->sw52.startmb_x = 0;
+ p_regs->sw52.startmb_y = 0;
+ p_regs->sw57.dec_out_dis = 0;
+ p_regs->sw50.filtering_dis = 1;
+
+ p_regs->sw64.rlc_vlc_base =
+ vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf, 0);
+ p_regs->sw122.strm_start_bit = params->data_bit_offset;
+ stream_len = vb2_get_plane_payload(&src_buf->vb2_buf, 0);
+ p_regs->sw51.stream_len = stream_len;
+
+ qtable = vb2_plane_vaddr(&src_buf->vb2_buf, 0) + ALIGN(stream_len, 8);
+ p_regs->sw61.qtable_base = p_regs->sw64.rlc_vlc_base
+ + ALIGN(stream_len, 8);
+
+ return 0;
+}
--
2.20.1
^ permalink raw reply related
* [PATCH v3 3/9] [WIP]: rockchip: mpp: HEVC decoder ctrl data
From: ayaka @ 2019-04-10 12:42 UTC (permalink / raw)
To: linux-media
Cc: Randy Li, hverkuil, maxime.ripard, joro, nicolas, jernej.skrabec,
paul.kocialkowski, thomas.petazzoni, mchehab, ezequiel, posciak,
groeck, linux-rockchip
In-Reply-To: <20190410124226.8612-1-ayaka@soulik.info>
From: Randy Li <randy.li@rock-chips.com>
Not done yet, not enough data.
Signed-off-by: Randy Li <randy.li@rock-chips.com>
---
drivers/staging/rockchip-mpp/Makefile | 4 +-
.../staging/rockchip-mpp/rkvdec/hevc-data.c | 208 ++++++++++++++++++
.../staging/rockchip-mpp/rkvdec/hevc-data.h | 27 +++
drivers/staging/rockchip-mpp/rkvdec/hevc.c | 2 +
4 files changed, 240 insertions(+), 1 deletion(-)
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hevc-data.c
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hevc-data.h
diff --git a/drivers/staging/rockchip-mpp/Makefile b/drivers/staging/rockchip-mpp/Makefile
index 9722b0059563..8da33fa5142d 100644
--- a/drivers/staging/rockchip-mpp/Makefile
+++ b/drivers/staging/rockchip-mpp/Makefile
@@ -1,7 +1,9 @@
# SPDX-License-Identifier: GPL-2.0
rk-mpp-service-objs := mpp_service.o
rk-mpp-device-objs := mpp_dev_common.o
-rk-mpp-vdec-objs := mpp_dev_rkvdec.o rkvdec/hevc.o rkvdec/avc.o
+rk-mpp-vdec-objs := mpp_dev_rkvdec.o
+rk-mpp-vdec-objs += rkvdec/avc.o
+rk-mpp-vdec-objs += rkvdec/hevc.o rkvdec/hevc-data.o rkvdec/rbsp.o
rk-mpp-vdpu2-objs := mpp_dev_vdpu2.o vdpu2/mpeg2.o
obj-$(CONFIG_ROCKCHIP_MPP_SERVICE) += rk-mpp-service.o
diff --git a/drivers/staging/rockchip-mpp/rkvdec/hevc-data.c b/drivers/staging/rockchip-mpp/rkvdec/hevc-data.c
new file mode 100644
index 000000000000..26694a2f46c5
--- /dev/null
+++ b/drivers/staging/rockchip-mpp/rkvdec/hevc-data.c
@@ -0,0 +1,208 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2019 Randy Li, <ayaka@soulik.info>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include "hevc-data.h"
+
+/* 7.3.2.2.1 General sequence parameter set RBSP syntax */
+int rkvdec_hevc_write_sps(struct rbsp *rbsp,
+ const struct v4l2_ctrl_hevc_sps *sps)
+{
+ /* TODO: sps_video_parameter_set_id */
+ rbsp_write_bits(rbsp, 4, 0);
+ /* TODO: sps_seq_parameter_set_id */
+ rbsp_write_bits(rbsp, 4, 0);
+ /* chroma_format_idc */
+ rbsp_write_bits(rbsp, 2, sps->chroma_format_idc);
+ rbsp_write_bits(rbsp, 13, sps->pic_width_in_luma_samples);
+ rbsp_write_bits(rbsp, 13, sps->pic_height_in_luma_samples);
+ /* bit_depth_luma */
+ rbsp_write_bits(rbsp, 4, sps->bit_depth_luma_minus8 + 8);
+ rbsp_write_bits(rbsp, 4, sps->bit_depth_chroma_minus8 + 8);
+ /* log2_max_pic_order_cnt_lsb */
+ rbsp_write_bits(rbsp, 5, sps->log2_max_pic_order_cnt_lsb_minus4 + 4);
+ /* FIXME it is 3 bits in document */
+ rbsp_write_bits(rbsp, 2, sps->log2_diff_max_min_luma_coding_block_size);
+ /* log2_min_luma_coding_block_size */
+ rbsp_write_bits(rbsp, 3,
+ sps->log2_min_luma_coding_block_size_minus3 + 3);
+ /* log2_min_transform_block_size */
+ rbsp_write_bits(rbsp, 3,
+ sps->log2_min_luma_transform_block_size_minus2 + 2);
+ rbsp_write_bits(rbsp, 2,
+ sps->log2_diff_max_min_luma_transform_block_size);
+ rbsp_write_bits(rbsp, 3, sps->max_transform_hierarchy_depth_inter);
+ rbsp_write_bits(rbsp, 3, sps->max_transform_hierarchy_depth_intra);
+
+ rbsp_write_flag(rbsp, sps->scaling_list_enabled_flag);
+ rbsp_write_flag(rbsp, sps->amp_enabled_flag);
+ rbsp_write_flag(rbsp, sps->sample_adaptive_offset_enabled_flag);
+ rbsp_write_flag(rbsp, sps->pcm_enabled_flag);
+
+ /* pcm_sample_bit_depth_luma */
+ rbsp_write_bits(rbsp, 4, sps->pcm_sample_bit_depth_luma_minus1 + 1);
+ /* pcm_sample_bit_depth_chroma */
+ rbsp_write_bits(rbsp, 4, sps->pcm_sample_bit_depth_chroma_minus1 + 1);
+ rbsp_write_flag(rbsp, sps->pcm_loop_filter_disabled_flag);
+
+ rbsp_write_bits(rbsp, 3,
+ sps->log2_diff_max_min_pcm_luma_coding_block_size);
+ /* log2_min_pcm_luma_coding_block_size */
+ rbsp_write_bits(rbsp, 3,
+ sps->log2_min_pcm_luma_coding_block_size_minus3 + 3);
+ rbsp_write_bits(rbsp, 7, sps->num_short_term_ref_pic_sets);
+ rbsp_write_flag(rbsp, sps->long_term_ref_pics_present_flag);
+ rbsp_write_bits(rbsp, 6, sps->num_long_term_ref_pics_sps);
+ rbsp_write_flag(rbsp, sps->sps_temporal_mvp_enabled_flag);
+ rbsp_write_flag(rbsp, sps->strong_intra_smoothing_enabled_flag);
+ /* Above is 100 bits total */
+#if 0
+ /* transform_skip_rotation_enabled_flag to intra_smoothing_disabled_flag */
+ rbsp_write_bits(rbsp, 7, 0);
+ /* sps_max_dec_pic_buffering_minus1 */
+ rbsp_write_bits(rbsp, 4, sps->sps_max_dec_pic_buffering_minus1);
+ rbsp_write_flag(rbsp, sps->separate_colour_plane_flag);
+ /* TODO: high_precision_offsets_enabled */
+ rbsp_write_flag(rbsp, 0);
+ /* TODO: persistent_rice_adaptation_enabled_flag */
+ rbsp_write_flag(rbsp, 0);
+ /* reserved */
+ rbsp_write_bits(rbsp, 14, 0xffffffff);
+#else
+ rbsp_write_bits(rbsp, 7, 0);
+ /* padding */
+ rbsp_write_bits(rbsp, 21, 0xffffffff);
+#endif
+
+ return 0;
+}
+
+int rkvdec_hevc_write_pps(struct rbsp *rbsp,
+ const struct v4l2_ctrl_hevc_sps *sps,
+ const struct v4l2_ctrl_hevc_pps *pps,
+ const struct v4l2_ctrl_hevc_slice_params *slice_params)
+{
+
+ u32 min_cb_log2_size_y, ctb_log2_size_y, log2_min_cu_qp_delta_size;
+ u16 column_width[20] = { 0, };
+ u16 row_height[22] = { 0, };
+ u8 i;
+
+ min_cb_log2_size_y = sps->log2_min_luma_coding_block_size_minus3 + 3;
+ ctb_log2_size_y = min_cb_log2_size_y +
+ sps->log2_diff_max_min_luma_coding_block_size;
+
+ log2_min_cu_qp_delta_size = ctb_log2_size_y - pps->diff_cu_qp_delta_depth;
+
+ /* pps_pic_parameter_set_id */
+ rbsp_write_bits(rbsp, 6, 0);
+ /* pps_seq_parameter_set_id */
+ rbsp_write_bits(rbsp, 4, 0);
+ /* dependent_slice_segments_enabled_flag */
+ rbsp_write_flag(rbsp, pps->dependent_slice_segment_flag);
+ rbsp_write_flag(rbsp, pps->output_flag_present_flag);
+ /* FIXME it is 3 bits in document */
+ rbsp_write_bits(rbsp, 13, pps->num_extra_slice_header_bits);
+ /* sign_data_hiding_flag */
+ rbsp_write_flag(rbsp, pps->sign_data_hiding_enabled_flag);
+ rbsp_write_flag(rbsp, pps->cabac_init_present_flag);
+ /* FIXME: from slice params ? */
+ rbsp_write_bits(rbsp, 4, slice_params->num_ref_idx_l0_active_minus1 + 1);
+ rbsp_write_bits(rbsp, 4, slice_params->num_ref_idx_l1_active_minus1 + 1);
+ /* FIXME it is 6 bits in document init_qp_minus26 */
+ rbsp_write_bits(rbsp, 7, pps->init_qp_minus26);
+ rbsp_write_flag(rbsp, pps->constrained_intra_pred_flag);
+ rbsp_write_flag(rbsp, pps->transform_skip_enabled_flag);
+ rbsp_write_flag(rbsp, pps->cu_qp_delta_enabled_flag);
+ /* Log2MinCuQpDeltaSize */
+ rbsp_write_bits(rbsp, 3, log2_min_cu_qp_delta_size);
+ rbsp_write_bits(rbsp, 5, pps->pps_cb_qp_offset);
+ rbsp_write_bits(rbsp, 5, pps->pps_cr_qp_offset);
+ rbsp_write_flag(rbsp, pps->pps_slice_chroma_qp_offsets_present_flag);
+ rbsp_write_flag(rbsp, pps->weighted_pred_flag);
+ rbsp_write_flag(rbsp, pps->weighted_bipred_flag);
+ rbsp_write_flag(rbsp, pps->transquant_bypass_enabled_flag);
+ rbsp_write_flag(rbsp, pps->tiles_enabled_flag);
+ rbsp_write_flag(rbsp, pps->entropy_coding_sync_enabled_flag);
+ rbsp_write_flag(rbsp, pps->pps_loop_filter_across_slices_enabled_flag);
+ rbsp_write_flag(rbsp, pps->loop_filter_across_tiles_enabled_flag);
+ rbsp_write_flag(rbsp, pps->deblocking_filter_override_enabled_flag);
+ /* pps_deblocking_filter_disabled_flag */
+ rbsp_write_flag(rbsp, pps->pps_disable_deblocking_filter_flag);
+ rbsp_write_bits(rbsp, 4, pps->pps_beta_offset_div2);
+ rbsp_write_bits(rbsp, 4, pps->pps_tc_offset_div2);
+ rbsp_write_flag(rbsp, pps->lists_modification_present_flag);
+ rbsp_write_bits(rbsp, 3, pps->log2_parallel_merge_level_minus2 + 2);
+ rbsp_write_flag(rbsp, pps->slice_segment_header_extension_present_flag);
+ /* reserved, log2_transform_skip_max_size_minus2 */
+ rbsp_write_bits(rbsp, 3, 0);
+ /* num_tile_columns */
+ rbsp_write_bits(rbsp, 5, pps->num_tile_columns_minus1 + 1);
+ /* num_tile_rows */
+ rbsp_write_bits(rbsp, 5, pps->num_tile_rows_minus1 + 1);
+ /* ? */
+ rbsp_write_bits(rbsp, 3, 2);
+ /* align 30 ? */
+ rbsp_write_bits(rbsp, 32, 0xffffffff);
+
+ /* TODO: support tile video */
+ column_width[0] = 0;
+ row_height[0] = 0;
+
+ for (i = 0; i < 20; i++) {
+ if (column_width[i])
+ column_width[i]--;
+ rbsp_write_bits(rbsp, column_width[i], 8);
+ }
+ for (i = 0; i < 22; i++) {
+ if (row_height[i])
+ row_height[i]--;
+ rbsp_write_bits(rbsp, row_height[i], 8);
+ }
+
+ /* TODO: scaleing_address */
+
+ return 0;
+}
+
+int rkvdec_hevc_write_soft_rps(struct rbsp *rbsp,
+ const struct v4l2_ctrl_hevc_slice_params *slice_params)
+{
+ int i;
+
+ for (i = 0; i < 15; i++) {
+ /* FIXME: is long term */
+ rbsp_write_flag(rbsp, 0);
+ rbsp_write_bits(rbsp, 4, slice_params->ref_idx_l0[i]);
+ }
+
+ for (i = 0; i < 15; i++) {
+ /* FIXME: is long term */
+ rbsp_write_flag(rbsp, 0);
+ rbsp_write_bits(rbsp, 4, slice_params->ref_idx_l1[i]);
+ }
+ /* TODO: lowdelay_flag */
+ rbsp_write_flag(rbsp, 1);
+
+ /* TODO: Rps_bit_offset_include_lt */
+ rbsp_write_bits(rbsp, 10, 0);
+ rbsp_write_bits(rbsp, 9, 0);
+}
+
+/* 7.3.7 Short-term reference picture set syntax */
+int rkvdec_hevc_write_rps(struct rbsp *rbsp,
+ const struct v4l2_ctrl_hevc_pps *pps)
+{
+
+}
diff --git a/drivers/staging/rockchip-mpp/rkvdec/hevc-data.h b/drivers/staging/rockchip-mpp/rkvdec/hevc-data.h
new file mode 100644
index 000000000000..6b94cd41d377
--- /dev/null
+++ b/drivers/staging/rockchip-mpp/rkvdec/hevc-data.h
@@ -0,0 +1,27 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2019 Randy Li, <ayaka@soulik.info>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <linux/types.h>
+
+#include <linux/videodev2.h>
+#include <media/v4l2-ctrls.h>
+
+#include "rbsp.h"
+
+int rkvdec_hevc_write_sps(struct rbsp *rbsp,
+ const struct v4l2_ctrl_hevc_sps *sps);
+
+int rkvdec_hevc_write_rps(struct rbsp *rbsp,
+ const struct v4l2_ctrl_hevc_pps *pps);
diff --git a/drivers/staging/rockchip-mpp/rkvdec/hevc.c b/drivers/staging/rockchip-mpp/rkvdec/hevc.c
index 78f150000128..6f74ce45533a 100644
--- a/drivers/staging/rockchip-mpp/rkvdec/hevc.c
+++ b/drivers/staging/rockchip-mpp/rkvdec/hevc.c
@@ -25,6 +25,8 @@
#include "hal.h"
#include "regs.h"
+#include "hevc-data.h"
+
static void init_hw_cfg(struct rkvdec_regs *p_regs)
{
p_regs->sw_interrupt.dec_e = 1;
--
2.20.1
^ permalink raw reply related
* [PATCH v3 2/9] rockchip: mpp: rkvdec: rbsp
From: ayaka @ 2019-04-10 12:42 UTC (permalink / raw)
To: linux-media
Cc: Randy Li, hverkuil, maxime.ripard, joro, nicolas, jernej.skrabec,
paul.kocialkowski, thomas.petazzoni, mchehab, ezequiel, posciak,
groeck, linux-rockchip
In-Reply-To: <20190410124226.8612-1-ayaka@soulik.info>
From: Randy Li <randy.li@rock-chips.com>
It is a bit writer.
Signed-off-by: Randy Li <randy.li@rock-chips.com>
---
drivers/staging/rockchip-mpp/rkvdec/rbsp.c | 96 ++++++++++++++++++++++
drivers/staging/rockchip-mpp/rkvdec/rbsp.h | 30 +++++++
2 files changed, 126 insertions(+)
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/rbsp.c
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/rbsp.h
diff --git a/drivers/staging/rockchip-mpp/rkvdec/rbsp.c b/drivers/staging/rockchip-mpp/rkvdec/rbsp.c
new file mode 100644
index 000000000000..3fbc50e9bca1
--- /dev/null
+++ b/drivers/staging/rockchip-mpp/rkvdec/rbsp.c
@@ -0,0 +1,96 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2012 Vista Silicon S.L.
+ * Copyright (C) 2019 Randy Li <ayaka@soulik.info>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <asm-generic/errno-base.h>
+#include <linux/kernel.h>
+#include <linux/types.h>
+
+#include "rbsp.h"
+
+int rbsp_init(struct rbsp *rbsp, void *buf, int size, int bit_pos)
+{
+ if (!buf)
+ return -EINVAL;
+ if (DIV_ROUND_UP(bit_pos, 32) >= size)
+ return -EINVAL;
+
+ rbsp->buf = buf;
+ rbsp->size = size >> 2;
+ rbsp->pos = bit_pos;
+
+ return 0;
+}
+
+static inline int rbsp_read_bit(struct rbsp *rbsp)
+{
+ int shift = rbsp->pos % 32;
+ int ofs = rbsp->pos++ / 32;
+
+ if (ofs >= rbsp->size)
+ return -EINVAL;
+
+ return (rbsp->buf[ofs] >> shift) & 1;
+}
+
+static inline int rbsp_write_bit(struct rbsp *rbsp, int bit)
+{
+ int shift = rbsp->pos % 32;
+ int ofs = rbsp->pos++ / 32;
+
+ if (ofs >= rbsp->size)
+ return -EINVAL;
+
+ rbsp->buf[ofs] &= ~(1 << shift);
+ rbsp->buf[ofs] |= bit << shift;
+
+ return 0;
+}
+
+int rbsp_write_bits(struct rbsp *rbsp, int num, int value)
+{
+ int shift = rbsp->pos % 32;
+ int ofs = rbsp->pos / 32;
+
+ if (ofs >= rbsp->size)
+ return -EINVAL;
+
+ if (num + shift >= 32) {
+ u32 lbits = 32 - shift;
+ u32 hbits = num + shift - 32;
+
+ rbsp->buf[ofs] &= ~(((1 << lbits) - 1) << shift);
+ rbsp->buf[ofs] |= value << shift;
+
+ value >>= (32 - shift);
+ rbsp->buf[ofs + 1] &= ~(((1 << hbits) - 1));
+ rbsp->buf[ofs + 1] |= value;
+ } else {
+ rbsp->buf[ofs] &= ~(((1 << num) - 1) << shift);
+ rbsp->buf[ofs] |= value << shift;
+ }
+
+ rbsp->pos += num;
+
+ return 0;
+}
+
+int rbsp_write_flag(struct rbsp *rbsp, int value)
+{
+ if (value)
+ return rbsp_write_bit(rbsp, BIT(0));
+ else
+ return rbsp_write_bit(rbsp, 0);
+}
diff --git a/drivers/staging/rockchip-mpp/rkvdec/rbsp.h b/drivers/staging/rockchip-mpp/rkvdec/rbsp.h
new file mode 100644
index 000000000000..d87c582bfd41
--- /dev/null
+++ b/drivers/staging/rockchip-mpp/rkvdec/rbsp.h
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2012 Vista Silicon S.L.
+ * Copyright (C) 2019 Randy Li <ayaka@soulik.info>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#ifndef _RBSP_H_
+#define _RBSP_H_
+
+struct rbsp {
+ u32 *buf;
+ int size;
+ int pos;
+};
+
+int rbsp_init(struct rbsp *rbsp, void *buf, int size, int bit_pos);
+int rbsp_write_flag(struct rbsp *rbsp, int bit);
+int rbsp_write_bits(struct rbsp *rbsp, int num, int value);
+
+#endif
--
2.20.1
^ permalink raw reply related
* [PATCH v3 0/9] [WIP]: rockchip mpp for v4l2 video deocder
From: ayaka @ 2019-04-10 12:42 UTC (permalink / raw)
To: linux-media
Cc: Randy 'ayaka' Li, randy.li, hverkuil, maxime.ripard, joro,
nicolas, jernej.skrabec, paul.kocialkowski, thomas.petazzoni,
mchehab, ezequiel, posciak, groeck, linux-rockchip
From: Randy 'ayaka' Li <ayaka@soulik.info>
Although I really hate the bitstream construction in kernel and I think
many people realise its problems, I still take the advise from ndufresne to
release this version. This should be released in a early week but
I was sick that time.
After reviewed the documents from Rockchip and I have confirmed that with
some rockchip staff, those documents are not update to date. So you may
find some part is different comparing to official document.
The v4l2-request-test from bootlin won't work. Its slice data doesn't
have a start code which making it not a complete nal unit. And I found
its slice header information may not be correct. Even comparing to the
Big buck bunny's origin files, neither the slice data nor sequence
information matches.
I extracted a slice data from Rockchip mpp to verify my driver, it work
fine, you can find it on my github. I only verified the I slice now,
I have not verified P or B slice. Hopefully it would work.
I have the same problem with v4l2-request-test on HEVC as well so even
this version shipped with HEVC bitstream construction, I didn't know
whether it would work.
I need some time to prepare the userspace tool or it is really hard for
HEVC to write slice info manually.
Changlog
v3: add AVC support for rkvdec
v2: add MPEG-2 support for vdpu2
v1: inital version
Randy Li (7):
staging: video: rockchip: add v4l2 decoder
rockchip: mpp: rkvdec: rbsp
[WIP]: rockchip: mpp: HEVC decoder ctrl data
[TEST]: rockchip: mpp: support qtable
[TEST]: rockchip: mpp: vdpu2: move qtable to input buffer
arm64: dts: rockchip: boost clocks for rk3328
arm64: dts: rockchip: add video codec for rk3328
ayaka (2):
[WIP]: rockchip: mpp: H.264 decoder ctrl data
[TEST]: rkvdec: spspps address alignment
arch/arm64/boot/dts/rockchip/rk3328-evb.dts | 32 +
.../arm64/boot/dts/rockchip/rk3328-rock64.dts | 32 +
arch/arm64/boot/dts/rockchip/rk3328.dtsi | 112 +-
drivers/staging/Kconfig | 2 +
drivers/staging/Makefile | 1 +
drivers/staging/rockchip-mpp/Kconfig | 33 +
drivers/staging/rockchip-mpp/Makefile | 12 +
drivers/staging/rockchip-mpp/mpp_debug.h | 87 +
drivers/staging/rockchip-mpp/mpp_dev_common.c | 1390 +++++++
drivers/staging/rockchip-mpp/mpp_dev_common.h | 215 +
drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c | 924 +++++
drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c | 607 +++
drivers/staging/rockchip-mpp/mpp_service.c | 197 +
drivers/staging/rockchip-mpp/mpp_service.h | 38 +
.../staging/rockchip-mpp/rkvdec/avc-data.c | 239 ++
.../staging/rockchip-mpp/rkvdec/avc-data.h | 40 +
drivers/staging/rockchip-mpp/rkvdec/avc.c | 259 ++
drivers/staging/rockchip-mpp/rkvdec/cabac.h | 3614 +++++++++++++++++
drivers/staging/rockchip-mpp/rkvdec/hal.h | 70 +
.../staging/rockchip-mpp/rkvdec/hevc-data.c | 208 +
.../staging/rockchip-mpp/rkvdec/hevc-data.h | 27 +
drivers/staging/rockchip-mpp/rkvdec/hevc.c | 169 +
drivers/staging/rockchip-mpp/rkvdec/rbsp.c | 96 +
drivers/staging/rockchip-mpp/rkvdec/rbsp.h | 30 +
drivers/staging/rockchip-mpp/rkvdec/regs.h | 377 ++
drivers/staging/rockchip-mpp/vdpu2/avc.c | 165 +
drivers/staging/rockchip-mpp/vdpu2/hal.h | 52 +
drivers/staging/rockchip-mpp/vdpu2/mpeg2.c | 277 ++
drivers/staging/rockchip-mpp/vdpu2/regs.h | 670 +++
include/uapi/video/rk_vpu_service.h | 101 +
30 files changed, 10072 insertions(+), 4 deletions(-)
create mode 100644 drivers/staging/rockchip-mpp/Kconfig
create mode 100644 drivers/staging/rockchip-mpp/Makefile
create mode 100644 drivers/staging/rockchip-mpp/mpp_debug.h
create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_common.c
create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_common.h
create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c
create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
create mode 100644 drivers/staging/rockchip-mpp/mpp_service.c
create mode 100644 drivers/staging/rockchip-mpp/mpp_service.h
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/avc-data.c
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/avc-data.h
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/avc.c
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/cabac.h
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hal.h
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hevc-data.c
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hevc-data.h
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hevc.c
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/rbsp.c
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/rbsp.h
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/regs.h
create mode 100644 drivers/staging/rockchip-mpp/vdpu2/avc.c
create mode 100644 drivers/staging/rockchip-mpp/vdpu2/hal.h
create mode 100644 drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
create mode 100644 drivers/staging/rockchip-mpp/vdpu2/regs.h
create mode 100644 include/uapi/video/rk_vpu_service.h
--
2.20.1
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox