From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 26911C369C2 for ; Tue, 22 Apr 2025 20:36:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version: Content-Transfer-Encoding:Content-Type:References:In-Reply-To:Date:Cc:To:From :Subject:Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=QB/EZxvi3t87PK18i4g1vhWVTtw3rMf9PNCebJ3iyMk=; b=tDOzUWkwfgPvnztS/cM+6moGWM nZHfb0/d2/R3GMOsKTTkBWdrHxC2FLIHY48Z1UxXp40NqJBtMxq67T1jN8StPvKIUnTMrovbAMY2i zm8ovy5a271vKjhBxyeAdwOA2TV3JcBCQkawyo7ZmKaIBtTFCLFymbGuktq+MTteulf2tP8zagoi1 mErhbrXJ3QEa/nu5uzrsB3522zCdcGjcdSchc/nn5Sq0oUaDbgLzyXKNKOtYg9yRbm8e4qSOfXEwV +EEqHfld5jYxtuEL4PEJkYIh2mLb7eI10Zvu8+7JvvGn6Zv/D6VVOIQY6mCPVHHdM3vrNr9TXQfOo 2l5sp61A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1u7KLc-00000008TTv-0bxb; Tue, 22 Apr 2025 20:36:04 +0000 Received: from mail-qv1-xf2a.google.com ([2607:f8b0:4864:20::f2a]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1u7KJf-00000008TNi-28ew for linux-arm-kernel@lists.infradead.org; Tue, 22 Apr 2025 20:34:08 +0000 Received: by mail-qv1-xf2a.google.com with SMTP id 6a1803df08f44-6e8f43c1fa0so71295366d6.3 for ; Tue, 22 Apr 2025 13:34:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ndufresne-ca.20230601.gappssmtp.com; s=20230601; t=1745354042; x=1745958842; darn=lists.infradead.org; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject :date:message-id:reply-to; bh=QB/EZxvi3t87PK18i4g1vhWVTtw3rMf9PNCebJ3iyMk=; b=2UOwW8xxEWcZPj9Yqfb7ivHDKOvJDq0vyvozWmJe/UuxZ9lYxoSMevh7L006Vw9Uyy IUpzrPI6gE3ucY+82XgSm6oLPj8MuQgN673f9gou8IgMbnlyqngv9AbKkP8x1KUr7NdU trr4Zo9rw00kFBxdIQJj6cflbrp5KoYiCl3OBy3W40e+QuhzYjJW6boM7Vii2AXyPc6Q BgVNDfDo0qWsKp4uRMRGNq/WgBXNRGPvPYR8j0EoGNIhJ0vj5Iet8jlBD1cQ/YxT6UJ3 VcHtNbC1M4ZGxt8wi7LB/WriLHsjQjFmVJFyHjQ4jDnKQu6kNQ/CtcQvP8BzOQp31Wvc QW4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745354042; x=1745958842; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=QB/EZxvi3t87PK18i4g1vhWVTtw3rMf9PNCebJ3iyMk=; b=RljZhrAUcUmNeNHq8dMAl5mi/6X6RiJCQl5FsKBUeA4XlU9nNqZnykROFkL0IuPi/h dEPHrgy9kXBQdML6suQtK3hQV576BNqKjz+wvBqUZhyznzQ0tuoAYKf5j6KvcOZZ8tHB ePdj7VywGaEm5j/bwFyi2XMiIx1Pjct43W7B9p9HQEIP+Xg/412zMcwwkgU1hxfUSY8U MUspwMVX3mqTlqeoRDl+uw4lPo4y1SCKMWGrlTHclHCowsTKBo4wTAjE7+7HgVoFm5J7 X/9gA2iTZK05Y671hI4YgMcVI0iPjkWreJFVZ8TqS6OwF8m0Wrm7iBuFE4FkHXcnlKwc pGug== X-Forwarded-Encrypted: i=1; AJvYcCVvAIF/yRiiy9PCTWtLyYsmii2b+EECjR1cnN8jLwWKKGd2ORESv1+ZRz5fm3GXD8Ud+eo8+MZM3xyC9wJq44RP@lists.infradead.org X-Gm-Message-State: AOJu0YwRzR2rBIfnzxROd9sgqrw8qMiL/aTCJ6DGLWGk6i+Oe/xmUyGz oqqnNNXw6lTZjvML1M3WhYS8I+jSX5WabcdXaYxYiE7I1ImYlHZXtdaRyu2ZOzc= X-Gm-Gg: ASbGncvR0cOPL/se0DDocOfvpErGflkJpgu1MnQ3ts/cQm7i2vffIE5Na73xYIYEKQy 7oISduLHYK8R6xLn3cEkGIVaY6Ye4Nbet641H7TdnH5YkQgDKl/7tg7UhzAzoHwBs0ke8ZXilqX USG7XpaX5rYJDnCvpoofHfzkuHkSmIIJmOf7K4V7nUn2KHPLvgC4EpMQVS5YvAjDKelZ15aMhg0 OpytDm2kLzsLmGYLMY7FFvzCE7oCO/ArM0r24rwggJzynvALwMripe6kme5dMpQpgZ4RAANIbTa eiI16WsSLEXiqdKSVqFJPI7odKDivSZ/JaSoeYnZGmH8cA== X-Google-Smtp-Source: AGHT+IF2NY8NWZhRDDTGP3IWCtm2obgw3XdCPc+7lNS25z2oTNzCLuKcPU7yqB4NlMxM4ZoFISN2xw== X-Received: by 2002:a05:6214:5085:b0:6eb:28e4:8519 with SMTP id 6a1803df08f44-6f2c4562ff4mr302755556d6.21.1745354041933; Tue, 22 Apr 2025 13:34:01 -0700 (PDT) Received: from ?IPv6:2606:6d00:15:9913::5ac? ([2606:6d00:15:9913::5ac]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6f2c2bfcfc5sm61803746d6.75.2025.04.22.13.34.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Apr 2025 13:34:01 -0700 (PDT) Message-ID: <7864deb3dc090bc52cbdb4c646d0155ecc256076.camel@ndufresne.ca> Subject: Re: [PATCH v5] media: amphion: Support dmabuf and v4l2 buffer without binding From: Nicolas Dufresne To: Ming Qian , mchehab@kernel.org, hverkuil-cisco@xs4all.nl Cc: shawnguo@kernel.org, robh+dt@kernel.org, s.hauer@pengutronix.de, kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, xiahong.bao@nxp.com, eagle.zhou@nxp.com, tao.jiang_2@nxp.com, imx@lists.linux.dev, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Date: Tue, 22 Apr 2025 16:34:00 -0400 In-Reply-To: <20250120082105.2960717-1-ming.qian@oss.nxp.com> References: <20250120082105.2960717-1-ming.qian@oss.nxp.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.56.0 (3.56.0-1.fc42) MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250422_133403_561360_11ED549E X-CRM114-Status: GOOD ( 28.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Le lundi 20 janvier 2025 =C3=A0 17:21 +0900, Ming Qian a =C3=A9crit=C2=A0: > When using VB2_DMABUF, the relationship between dma-buf and v4l2 buffer > may not one-to-one, a single dma-buf may be queued via different > v4l2 buffers, and different dma-bufs may be queued via the same > v4l2 buffer, so it's not appropriate to use the v4l2 buffer index > as the frame store id. >=20 > We can generate a frame store id according to the dma address. > Then for a given dma-buf, the id is fixed. >=20 > Driver now manages the frame store and vb2-buffer states independently. >=20 > When a dmabuf is queued via another v4l2 buffer before the buffer is > released by firmware, need to pend it until firmware release it. >=20 > Signed-off-by: Ming Qian > --- > v5 > - Avoid dynamic size calculation in memory allocation, use kmalloc_array > =C2=A0 instead > v4 > - remove unnecessary 'out of memory' message > v3 > -- fix a typo in NULL pointer check > v2 > -- fix an uninitialized issue reported by media-ci >=20 > =C2=A0drivers/media/platform/amphion/vdec.c=C2=A0=C2=A0=C2=A0=C2=A0 | 235= +++++++++++++++++----- > =C2=A0drivers/media/platform/amphion/vpu.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = |=C2=A0=C2=A0 7 +- > =C2=A0drivers/media/platform/amphion/vpu_dbg.c=C2=A0 |=C2=A0 15 +- > =C2=A0drivers/media/platform/amphion/vpu_v4l2.c |=C2=A0 11 + > =C2=A04 files changed, 220 insertions(+), 48 deletions(-) >=20 > diff --git a/drivers/media/platform/amphion/vdec.c b/drivers/media/platfo= rm/amphion/vdec.c > index b3bc9eb16d6a..f4979d537b97 100644 > --- a/drivers/media/platform/amphion/vdec.c > +++ b/drivers/media/platform/amphion/vdec.c > @@ -26,6 +26,7 @@ > =C2=A0#include "vpu_cmds.h" > =C2=A0#include "vpu_rpc.h" > =C2=A0 > +#define VDEC_SLOT_CNT_DFT 32 > =C2=A0#define VDEC_MIN_BUFFER_CAP 8 > =C2=A0#define VDEC_MIN_BUFFER_OUT 8 > =C2=A0 > @@ -41,6 +42,14 @@ struct vdec_fs_info { > =C2=A0 u32 tag; > =C2=A0}; > =C2=A0 > +struct vdec_frame_store_t { > + struct vpu_vb2_buffer *curr; > + struct vpu_vb2_buffer *pend; > + dma_addr_t addr; > + unsigned int state; > + u32 tag; > +}; > + > =C2=A0struct vdec_t { > =C2=A0 u32 seq_hdr_found; > =C2=A0 struct vpu_buffer udata; > @@ -48,7 +57,8 @@ struct vdec_t { > =C2=A0 struct vpu_dec_codec_info codec_info; > =C2=A0 enum vpu_codec_state state; > =C2=A0 > - struct vpu_vb2_buffer *slots[VB2_MAX_FRAME]; > + struct vdec_frame_store_t *slots; > + u32 slot_count; > =C2=A0 u32 req_frame_count; > =C2=A0 struct vdec_fs_info mbi; > =C2=A0 struct vdec_fs_info dcp; > @@ -289,6 +299,63 @@ static int vdec_ctrl_init(struct vpu_inst *inst) > =C2=A0 return 0; > =C2=A0} > =C2=A0 > +static void vdec_attach_frame_store(struct vpu_inst *inst, struct vb2_bu= ffer *vb) > +{ > + struct vb2_v4l2_buffer *vbuf =3D to_vb2_v4l2_buffer(vb); > + struct vpu_vb2_buffer *vpu_buf =3D to_vpu_vb2_buffer(vbuf); > + struct vdec_t *vdec =3D inst->priv; > + struct vdec_frame_store_t *new_slots =3D NULL; > + dma_addr_t addr; > + int i; > + > + addr =3D vpu_get_vb_phy_addr(vb, 0); > + for (i =3D 0; i < vdec->slot_count; i++) { > + if (addr =3D=3D vdec->slots[i].addr) { > + if (vdec->slots[i].curr && vdec->slots[i].curr !=3D vpu_buf) { > + vpu_set_buffer_state(vbuf, VPU_BUF_STATE_CHANGED); > + vdec->slots[i].pend =3D vpu_buf; > + } else { > + vpu_set_buffer_state(vbuf, vdec->slots[i].state); > + } > + vpu_buf->fs_id =3D i; > + return; > + } > + } > + > + for (i =3D 0; i < vdec->slot_count; i++) { > + if (!vdec->slots[i].addr) { > + vdec->slots[i].addr =3D addr; > + vpu_buf->fs_id =3D i; > + return; > + } > + } > + > + new_slots =3D krealloc_array(vdec->slots, vdec->slot_count * 2, > + =C2=A0=C2=A0 sizeof(*vdec->slots), > + =C2=A0=C2=A0 GFP_KERNEL | __GFP_ZERO); > + if (!new_slots) { > + vpu_set_buffer_state(vbuf, VPU_BUF_STATE_ERROR); > + return; > + } > + > + vdec->slots =3D new_slots; > + vdec->slot_count *=3D 2; > + > + vdec->slots[i].addr =3D addr; > + vpu_buf->fs_id =3D i; > +} > + > +static void vdec_reset_frame_store(struct vpu_inst *inst) > +{ > + struct vdec_t *vdec =3D inst->priv; > + > + if (!vdec->slots || !vdec->slot_count) > + return; > + > + vpu_trace(inst->dev, "inst[%d] reset slots\n", inst->id); > + memset(vdec->slots, 0, sizeof(*vdec->slots) * vdec->slot_count); > +} > + > =C2=A0static void vdec_handle_resolution_change(struct vpu_inst *inst) > =C2=A0{ > =C2=A0 struct vdec_t *vdec =3D inst->priv; > @@ -750,11 +817,11 @@ static int vdec_frame_decoded(struct vpu_inst *inst= , void *arg) > =C2=A0 struct vb2_v4l2_buffer *src_buf; > =C2=A0 int ret =3D 0; > =C2=A0 > - if (!info || info->id >=3D ARRAY_SIZE(vdec->slots)) > + if (!info || info->id >=3D vdec->slot_count) > =C2=A0 return -EINVAL; > =C2=A0 > =C2=A0 vpu_inst_lock(inst); > - vpu_buf =3D vdec->slots[info->id]; > + vpu_buf =3D vdec->slots[info->id].curr; > =C2=A0 if (!vpu_buf) { > =C2=A0 dev_err(inst->dev, "[%d] decoded invalid frame[%d]\n", inst->id, = info->id); > =C2=A0 ret =3D -EINVAL; > @@ -775,11 +842,13 @@ static int vdec_frame_decoded(struct vpu_inst *inst= , void *arg) > =C2=A0 if (vpu_get_buffer_state(vbuf) =3D=3D VPU_BUF_STATE_DECODED) > =C2=A0 dev_info(inst->dev, "[%d] buf[%d] has been decoded\n", inst->id, = info->id); > =C2=A0 vpu_set_buffer_state(vbuf, VPU_BUF_STATE_DECODED); > + vdec->slots[info->id].state =3D VPU_BUF_STATE_DECODED; > =C2=A0 vdec->decoded_frame_count++; > =C2=A0 if (vdec->params.display_delay_enable) { > =C2=A0 struct vpu_format *cur_fmt; > =C2=A0 > =C2=A0 cur_fmt =3D vpu_get_format(inst, inst->cap_format.type); > + vdec->slots[info->id].state =3D VPU_BUF_STATE_READY; > =C2=A0 vpu_set_buffer_state(vbuf, VPU_BUF_STATE_READY); > =C2=A0 for (int i =3D 0; i < vbuf->vb2_buf.num_planes; i++) > =C2=A0 vb2_set_plane_payload(&vbuf->vb2_buf, > @@ -802,11 +871,11 @@ static struct vpu_vb2_buffer *vdec_find_buffer(stru= ct vpu_inst *inst, u32 luma) > =C2=A0 struct vdec_t *vdec =3D inst->priv; > =C2=A0 int i; > =C2=A0 > - for (i =3D 0; i < ARRAY_SIZE(vdec->slots); i++) { > - if (!vdec->slots[i]) > + for (i =3D 0; i < vdec->slot_count; i++) { > + if (!vdec->slots[i].curr) > =C2=A0 continue; > - if (luma =3D=3D vdec->slots[i]->luma) > - return vdec->slots[i]; > + if (luma =3D=3D vdec->slots[i].addr) > + return vdec->slots[i].curr; > =C2=A0 } > =C2=A0 > =C2=A0 return NULL; > @@ -840,11 +909,11 @@ static void vdec_buf_done(struct vpu_inst *inst, st= ruct vpu_frame_info *frame) > =C2=A0 > =C2=A0 cur_fmt =3D vpu_get_format(inst, inst->cap_format.type); > =C2=A0 vbuf =3D &vpu_buf->m2m_buf.vb; > - if (vbuf->vb2_buf.index !=3D frame->id) > - dev_err(inst->dev, "[%d] buffer id(%d, %d) dismatch\n", > - inst->id, vbuf->vb2_buf.index, frame->id); > + if (vpu_buf->fs_id !=3D frame->id) > + dev_err(inst->dev, "[%d] buffer id(%d(%d), %d) dismatch\n", > + inst->id, vpu_buf->fs_id, vbuf->vb2_buf.index, frame->id); Was not newly introduced, so I'll take anyway, but can you send a patch later to fix dismatch -> mismatch. > =C2=A0 > - if (vpu_get_buffer_state(vbuf) =3D=3D VPU_BUF_STATE_READY && vdec->para= ms.display_delay_enable) > + if (vdec->params.display_delay_enable) > =C2=A0 return; > =C2=A0 > =C2=A0 if (vpu_get_buffer_state(vbuf) !=3D VPU_BUF_STATE_DECODED) > @@ -857,10 +926,11 @@ static void vdec_buf_done(struct vpu_inst *inst, st= ruct vpu_frame_info *frame) > =C2=A0 vbuf->sequence =3D vdec->sequence; > =C2=A0 dev_dbg(inst->dev, "[%d][OUTPUT TS]%32lld\n", inst->id, vbuf->vb2_= buf.timestamp); > =C2=A0 > - v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE); > =C2=A0 vpu_inst_lock(inst); > + vdec->slots[vpu_buf->fs_id].state =3D VPU_BUF_STATE_READY; > =C2=A0 vdec->display_frame_count++; > =C2=A0 vpu_inst_unlock(inst); > + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE); > =C2=A0 dev_dbg(inst->dev, "[%d] decoded : %d, display : %d, sequence : %d= \n", > =C2=A0 inst->id, vdec->decoded_frame_count, vdec->display_frame_count, v= dec->sequence); > =C2=A0} > @@ -1103,18 +1173,30 @@ static int vdec_response_frame(struct vpu_inst *i= nst, struct vb2_v4l2_buffer *vb > =C2=A0 if (!vbuf) > =C2=A0 return -EINVAL; > =C2=A0 > - if (vdec->slots[vbuf->vb2_buf.index]) { > - dev_err(inst->dev, "[%d] repeat alloc fs %d\n", > - inst->id, vbuf->vb2_buf.index); > + vpu_buf =3D to_vpu_vb2_buffer(vbuf); > + if (vpu_buf->fs_id < 0 || vpu_buf->fs_id >=3D vdec->slot_count) { > + dev_err(inst->dev, "invalid fs %d for v4l2 buffer %d\n", > + vpu_buf->fs_id, vbuf->vb2_buf.index); > =C2=A0 return -EINVAL; > =C2=A0 } > =C2=A0 > + if (vdec->slots[vpu_buf->fs_id].curr) { > + if (vdec->slots[vpu_buf->fs_id].curr !=3D vpu_buf) { > + vpu_set_buffer_state(vbuf, VPU_BUF_STATE_CHANGED); > + vdec->slots[vpu_buf->fs_id].pend =3D vpu_buf; > + } else { > + vpu_set_buffer_state(vbuf, vdec->slots[vpu_buf->fs_id].state); > + } > + dev_err(inst->dev, "[%d] repeat alloc fs %d (v4l2 index %d)\n", > + inst->id, vpu_buf->fs_id, vbuf->vb2_buf.index); > + return -EAGAIN; > + } > + > =C2=A0 dev_dbg(inst->dev, "[%d] state =3D %s, alloc fs %d, tag =3D 0x%x\n= ", > =C2=A0 inst->id, vpu_codec_state_name(inst->state), vbuf->vb2_buf.index,= vdec->seq_tag); > - vpu_buf =3D to_vpu_vb2_buffer(vbuf); > =C2=A0 > =C2=A0 memset(&info, 0, sizeof(info)); > - info.id =3D vbuf->vb2_buf.index; > + info.id =3D vpu_buf->fs_id; > =C2=A0 info.type =3D MEM_RES_FRAME; > =C2=A0 info.tag =3D vdec->seq_tag; > =C2=A0 info.luma_addr =3D vpu_get_vb_phy_addr(&vbuf->vb2_buf, 0); > @@ -1129,12 +1211,13 @@ static int vdec_response_frame(struct vpu_inst *i= nst, struct vb2_v4l2_buffer *vb > =C2=A0 if (ret) > =C2=A0 return ret; > =C2=A0 > - vpu_buf->tag =3D info.tag; > =C2=A0 vpu_buf->luma =3D info.luma_addr; > =C2=A0 vpu_buf->chroma_u =3D info.chroma_addr; > =C2=A0 vpu_buf->chroma_v =3D 0; > =C2=A0 vpu_set_buffer_state(vbuf, VPU_BUF_STATE_INUSE); > - vdec->slots[info.id] =3D vpu_buf; > + vdec->slots[info.id].tag =3D info.tag; > + vdec->slots[info.id].curr =3D vpu_buf; > + vdec->slots[info.id].state =3D VPU_BUF_STATE_INUSE; > =C2=A0 vdec->req_frame_count--; > =C2=A0 > =C2=A0 return 0; > @@ -1195,25 +1278,47 @@ static void vdec_recycle_buffer(struct vpu_inst *= inst, struct vb2_v4l2_buffer *v > =C2=A0 v4l2_m2m_buf_queue(inst->fh.m2m_ctx, vbuf); > =C2=A0} > =C2=A0 > -static void vdec_clear_slots(struct vpu_inst *inst) > +static void vdec_release_curr_frame_store(struct vpu_inst *inst, u32 id) > =C2=A0{ > =C2=A0 struct vdec_t *vdec =3D inst->priv; > =C2=A0 struct vpu_vb2_buffer *vpu_buf; > =C2=A0 struct vb2_v4l2_buffer *vbuf; > + > + if (id >=3D vdec->slot_count) > + return; > + if (!vdec->slots[id].curr) > + return; > + > + vpu_buf =3D vdec->slots[id].curr; > + vbuf =3D &vpu_buf->m2m_buf.vb; > + > + vdec_response_fs_release(inst, id, vdec->slots[id].tag); > + if (vpu_buf->fs_id =3D=3D id) { > + if (vpu_buf->state !=3D VPU_BUF_STATE_READY) > + vdec_recycle_buffer(inst, vbuf); > + vpu_set_buffer_state(vbuf, VPU_BUF_STATE_IDLE); > + } > + > + vdec->slots[id].curr =3D NULL; > + vdec->slots[id].state =3D VPU_BUF_STATE_IDLE; > + > + if (vdec->slots[id].pend) { > + vpu_set_buffer_state(&vdec->slots[id].pend->m2m_buf.vb, VPU_BUF_STATE_= IDLE); > + vdec->slots[id].pend =3D NULL; > + } > +} > + > +static void vdec_clear_slots(struct vpu_inst *inst) > +{ > + struct vdec_t *vdec =3D inst->priv; > =C2=A0 int i; > =C2=A0 > - for (i =3D 0; i < ARRAY_SIZE(vdec->slots); i++) { > - if (!vdec->slots[i]) > + for (i =3D 0; i < vdec->slot_count; i++) { > + if (!vdec->slots[i].curr) > =C2=A0 continue; > =C2=A0 > - vpu_buf =3D vdec->slots[i]; > - vbuf =3D &vpu_buf->m2m_buf.vb; > - > =C2=A0 vpu_trace(inst->dev, "clear slot %d\n", i); > - vdec_response_fs_release(inst, i, vpu_buf->tag); > - vdec_recycle_buffer(inst, vbuf); > - vdec->slots[i]->state =3D VPU_BUF_STATE_IDLE; > - vdec->slots[i] =3D NULL; > + vdec_release_curr_frame_store(inst, i); > =C2=A0 } > =C2=A0} > =C2=A0 > @@ -1354,39 +1459,29 @@ static void vdec_event_req_fs(struct vpu_inst *in= st, struct vpu_fs_info *fs) > =C2=A0static void vdec_evnet_rel_fs(struct vpu_inst *inst, struct vpu_fs_= info *fs) > =C2=A0{ > =C2=A0 struct vdec_t *vdec =3D inst->priv; > - struct vpu_vb2_buffer *vpu_buf; > - struct vb2_v4l2_buffer *vbuf; > =C2=A0 > - if (!fs || fs->id >=3D ARRAY_SIZE(vdec->slots)) > + if (!fs || fs->id >=3D vdec->slot_count) > =C2=A0 return; > =C2=A0 if (fs->type !=3D MEM_RES_FRAME) > =C2=A0 return; > =C2=A0 > - if (fs->id >=3D vpu_get_num_buffers(inst, inst->cap_format.type)) { > + if (fs->id >=3D vdec->slot_count) { > =C2=A0 dev_err(inst->dev, "[%d] invalid fs(%d) to release\n", inst->id, = fs->id); > =C2=A0 return; > =C2=A0 } > =C2=A0 > =C2=A0 vpu_inst_lock(inst); > - vpu_buf =3D vdec->slots[fs->id]; > - vdec->slots[fs->id] =3D NULL; > - > - if (!vpu_buf) { > + if (!vdec->slots[fs->id].curr) { > =C2=A0 dev_dbg(inst->dev, "[%d] fs[%d] has bee released\n", inst->id, fs= ->id); > =C2=A0 goto exit; > =C2=A0 } > =C2=A0 > - vbuf =3D &vpu_buf->m2m_buf.vb; > - if (vpu_get_buffer_state(vbuf) =3D=3D VPU_BUF_STATE_DECODED) { > + if (vdec->slots[fs->id].state =3D=3D VPU_BUF_STATE_DECODED) { > =C2=A0 dev_dbg(inst->dev, "[%d] frame skip\n", inst->id); > =C2=A0 vdec->sequence++; > =C2=A0 } > =C2=A0 > - vdec_response_fs_release(inst, fs->id, vpu_buf->tag); > - if (vpu_get_buffer_state(vbuf) !=3D VPU_BUF_STATE_READY) > - vdec_recycle_buffer(inst, vbuf); > - > - vpu_set_buffer_state(vbuf, VPU_BUF_STATE_IDLE); > + vdec_release_curr_frame_store(inst, fs->id); > =C2=A0 vpu_process_capture_buffer(inst); > =C2=A0 > =C2=A0exit: > @@ -1582,6 +1677,11 @@ static void vdec_cleanup(struct vpu_inst *inst) > =C2=A0 return; > =C2=A0 > =C2=A0 vdec =3D inst->priv; > + if (vdec) { > + kfree(vdec->slots); > + vdec->slots =3D NULL; > + vdec->slot_count =3D 0; > + } > =C2=A0 vfree(vdec); > =C2=A0 inst->priv =3D NULL; > =C2=A0 vfree(inst); > @@ -1713,11 +1813,43 @@ static int vdec_stop_session(struct vpu_inst *ins= t, u32 type) > =C2=A0 return 0; > =C2=A0} > =C2=A0 > -static int vdec_get_debug_info(struct vpu_inst *inst, char *str, u32 siz= e, u32 i) > +static int vdec_get_slot_debug_info(struct vpu_inst *inst, char *str, u3= 2 size, u32 i) > =C2=A0{ > =C2=A0 struct vdec_t *vdec =3D inst->priv; > + struct vpu_vb2_buffer *vpu_buf; > =C2=A0 int num =3D -1; > =C2=A0 > + vpu_inst_lock(inst); > + if (i >=3D vdec->slot_count || !vdec->slots[i].addr) > + goto exit; > + > + vpu_buf =3D vdec->slots[i].curr; > + > + num =3D scnprintf(str, size, "slot[%2d] :", i); > + if (vpu_buf) { > + num +=3D scnprintf(str + num, size - num, " %2d", > + vpu_buf->m2m_buf.vb.vb2_buf.index); > + num +=3D scnprintf(str + num, size - num, "; state =3D %d", vdec->slot= s[i].state); > + } else { > + num +=3D scnprintf(str + num, size - num, " -1"); > + } > + > + if (vdec->slots[i].pend) > + num +=3D scnprintf(str + num, size - num, "; %d", > + vdec->slots[i].pend->m2m_buf.vb.vb2_buf.index); > + > + num +=3D scnprintf(str + num, size - num, "\n"); > +exit: > + vpu_inst_unlock(inst); > + > + return num; > +} > + > +static int vdec_get_debug_info(struct vpu_inst *inst, char *str, u32 siz= e, u32 i) > +{ > + struct vdec_t *vdec =3D inst->priv; > + int num; > + > =C2=A0 switch (i) { > =C2=A0 case 0: > =C2=A0 num =3D scnprintf(str, size, > @@ -1771,6 +1903,7 @@ static int vdec_get_debug_info(struct vpu_inst *ins= t, char *str, u32 size, u32 i > =C2=A0 vdec->codec_info.vui_present); > =C2=A0 break; > =C2=A0 default: > + num =3D vdec_get_slot_debug_info(inst, str, size, i - 10); > =C2=A0 break; > =C2=A0 } > =C2=A0 > @@ -1794,6 +1927,8 @@ static struct vpu_inst_ops vdec_inst_ops =3D { > =C2=A0 .get_debug_info =3D vdec_get_debug_info, > =C2=A0 .wait_prepare =3D vpu_inst_unlock, > =C2=A0 .wait_finish =3D vpu_inst_lock, > + .attach_frame_store =3D vdec_attach_frame_store, > + .reset_frame_store =3D vdec_reset_frame_store, > =C2=A0}; > =C2=A0 > =C2=A0static void vdec_init(struct file *file) > @@ -1834,6 +1969,16 @@ static int vdec_open(struct file *file) > =C2=A0 return -ENOMEM; > =C2=A0 } > =C2=A0 > + vdec->slots =3D kmalloc_array(VDEC_SLOT_CNT_DFT, > + =C2=A0=C2=A0=C2=A0 sizeof(*vdec->slots), > + =C2=A0=C2=A0=C2=A0 GFP_KERNEL | __GFP_ZERO); > + if (!vdec->slots) { > + vfree(vdec); > + vfree(inst); > + return -ENOMEM; > + } > + vdec->slot_count =3D VDEC_SLOT_CNT_DFT; > + > =C2=A0 inst->ops =3D &vdec_inst_ops; > =C2=A0 inst->formats =3D vdec_formats; > =C2=A0 inst->type =3D VPU_CORE_TYPE_DEC; > diff --git a/drivers/media/platform/amphion/vpu.h b/drivers/media/platfor= m/amphion/vpu.h > index 978971712742..d8100da160d1 100644 > --- a/drivers/media/platform/amphion/vpu.h > +++ b/drivers/media/platform/amphion/vpu.h > @@ -223,6 +223,8 @@ struct vpu_inst_ops { > =C2=A0 int (*get_debug_info)(struct vpu_inst *inst, char *str, u32 size, = u32 i); > =C2=A0 void (*wait_prepare)(struct vpu_inst *inst); > =C2=A0 void (*wait_finish)(struct vpu_inst *inst); > + void (*attach_frame_store)(struct vpu_inst *inst, struct vb2_buffer *vb= ); > + void (*reset_frame_store)(struct vpu_inst *inst); > =C2=A0}; > =C2=A0 > =C2=A0struct vpu_inst { > @@ -297,7 +299,8 @@ enum { > =C2=A0 VPU_BUF_STATE_DECODED, > =C2=A0 VPU_BUF_STATE_READY, > =C2=A0 VPU_BUF_STATE_SKIP, > - VPU_BUF_STATE_ERROR > + VPU_BUF_STATE_ERROR, > + VPU_BUF_STATE_CHANGED > =C2=A0}; > =C2=A0 > =C2=A0struct vpu_vb2_buffer { > @@ -306,8 +309,8 @@ struct vpu_vb2_buffer { > =C2=A0 dma_addr_t chroma_u; > =C2=A0 dma_addr_t chroma_v; > =C2=A0 unsigned int state; > - u32 tag; > =C2=A0 u32 average_qp; > + s32 fs_id; > =C2=A0}; > =C2=A0 > =C2=A0void vpu_writel(struct vpu_dev *vpu, u32 reg, u32 val); > diff --git a/drivers/media/platform/amphion/vpu_dbg.c b/drivers/media/pla= tform/amphion/vpu_dbg.c > index 940e5bda5fa3..497ae4e8a229 100644 > --- a/drivers/media/platform/amphion/vpu_dbg.c > +++ b/drivers/media/platform/amphion/vpu_dbg.c > @@ -48,6 +48,7 @@ static char *vpu_stat_name[] =3D { > =C2=A0 [VPU_BUF_STATE_READY] =3D "ready", > =C2=A0 [VPU_BUF_STATE_SKIP] =3D "skip", > =C2=A0 [VPU_BUF_STATE_ERROR] =3D "error", > + [VPU_BUF_STATE_CHANGED] =3D "changed", > =C2=A0}; > =C2=A0 > =C2=A0static inline const char *to_vpu_stat_name(int state) > @@ -164,6 +165,7 @@ static int vpu_dbg_instance(struct seq_file *s, void = *data) > =C2=A0 for (i =3D 0; i < vb2_get_num_buffers(vq); i++) { > =C2=A0 struct vb2_buffer *vb; > =C2=A0 struct vb2_v4l2_buffer *vbuf; > + struct vpu_vb2_buffer *vpu_buf; > =C2=A0 > =C2=A0 vb =3D vb2_get_buffer(vq, i); > =C2=A0 if (!vb) > @@ -173,13 +175,24 @@ static int vpu_dbg_instance(struct seq_file *s, voi= d *data) > =C2=A0 continue; > =C2=A0 > =C2=A0 vbuf =3D to_vb2_v4l2_buffer(vb); > + vpu_buf =3D to_vpu_vb2_buffer(vbuf); > =C2=A0 > =C2=A0 num =3D scnprintf(str, sizeof(str), > - "capture[%2d] state =3D %10s, %8s\n", > + "capture[%2d] state =3D %10s, %8s", > =C2=A0 i, vb2_stat_name[vb->state], > =C2=A0 to_vpu_stat_name(vpu_get_buffer_state(vbuf))); > =C2=A0 if (seq_write(s, str, num)) > =C2=A0 return 0; > + > + if (vpu_buf->fs_id >=3D 0) { > + num =3D scnprintf(str, sizeof(str), "; fs %d", vpu_buf->fs_id); > + if (seq_write(s, str, num)) > + return 0; > + } > + > + num =3D scnprintf(str, sizeof(str), "\n"); > + if (seq_write(s, str, num)) > + return 0; > =C2=A0 } > =C2=A0 > =C2=A0 num =3D scnprintf(str, sizeof(str), "sequence =3D %d\n", inst->seq= uence); > diff --git a/drivers/media/platform/amphion/vpu_v4l2.c b/drivers/media/pl= atform/amphion/vpu_v4l2.c > index 50aeb69d2c66..37ef706c29dd 100644 > --- a/drivers/media/platform/amphion/vpu_v4l2.c > +++ b/drivers/media/platform/amphion/vpu_v4l2.c > @@ -500,14 +500,25 @@ static int vpu_vb2_queue_setup(struct vb2_queue *vq= , > =C2=A0 call_void_vop(inst, release); > =C2=A0 } > =C2=A0 > + if (V4L2_TYPE_IS_CAPTURE(vq->type)) > + call_void_vop(inst, reset_frame_store); > + > =C2=A0 return 0; > =C2=A0} > =C2=A0 > =C2=A0static int vpu_vb2_buf_init(struct vb2_buffer *vb) > =C2=A0{ > =C2=A0 struct vb2_v4l2_buffer *vbuf =3D to_vb2_v4l2_buffer(vb); > + struct vpu_vb2_buffer *vpu_buf =3D to_vpu_vb2_buffer(vbuf); > + struct vpu_inst *inst =3D vb2_get_drv_priv(vb->vb2_queue); > =C2=A0 > + vpu_buf->fs_id =3D -1; > =C2=A0 vpu_set_buffer_state(vbuf, VPU_BUF_STATE_IDLE); > + > + if (!inst->ops->attach_frame_store || V4L2_TYPE_IS_OUTPUT(vb->type)) > + return 0; > + > + call_void_vop(inst, attach_frame_store, vb); > =C2=A0 return 0; > =C2=A0} > =C2=A0 Reviewed-by: Nicolas Dufresne