From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C9D37CD37AE for ; Wed, 4 Sep 2024 07:09:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=mcyOGk3MDu708YOZSkXWyzVvtrOpNHUpauTz79vJE50=; b=Cr7oa0mbAzSKgZXVFo2wjt1jql Emlw9Vf86/QecTqDwgcNgmaT+Sv6MkEL03WUlRQczQwYVvb2Loqs7JZgKzM4Nn8drkEnLyX4Azywr GosfIiB/beLUncnQQO6W/xyxNPYR8c3dk/ZD4GdzgRoifAEbJz9QUZNKFQ9/aneZTa357WBWFmh+8 azk1ClLitggeiBgdSrTS4JxBGOSBh9Bfeu96CQ1GDgS5x4OXHx/eGbh4JGnhbt0jbqNTB6dWU6vUZ 08nLgkxnC/H6KtT4Eq6JkrotVH3rxORKi+vSWgj88WAeEBXMvz7MauDDeh9qnnVlwtxvFXdlQfDqK PDL5vduw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1slk8R-000000037e1-1eZk; Wed, 04 Sep 2024 07:08:59 +0000 Received: from mgamail.intel.com ([198.175.65.15]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1slk7T-000000037Qk-3FvA; Wed, 04 Sep 2024 07:08:01 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1725433679; x=1756969679; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=ujkfgFcta+jaZo3W65HHpjN0mTvqdn/KfBCJvNnjwZ0=; b=HaDQiCd8dMal8qu+9TnB6eK1xho90YX4Gbh1KvCiQ47IopCBdcs2eMN4 ocoUGj9aoMD2AViTBJ+wVkS2PO3VBssn99GpUyeD2GUH4/qWKC+WkTaO9 9VCUBu0WjcWAveqks9yP9GYMF3uYHd0igwrzmN0w14ytsBaGPAxpLSK3I Ob2IqVA5Uxr24hR1i4sI6ZrBWPc2WFrGZ/BNhyHbX7eUJEGpuAn61CgBO nB7i9thXIHuhIZyIYdaeIlMzuvf5q6/Qy+hAvlEzDD3l8GtrGAiIdG3o8 QJvuaBJd5wfpmxrBffWJEZdQEEQy7xX+DgkcAT89XIgHvwN5AuWANWCgL Q==; X-CSE-ConnectionGUID: 44fSG8n9T7aoJ/27bz+KjQ== X-CSE-MsgGUID: 2VbxKgOHRzGqBZnXw7oo7w== X-IronPort-AV: E=McAfee;i="6700,10204,11184"; a="27824606" X-IronPort-AV: E=Sophos;i="6.10,201,1719903600"; d="scan'208";a="27824606" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Sep 2024 00:07:57 -0700 X-CSE-ConnectionGUID: istFUcuDT9aLhJqRl4HKng== X-CSE-MsgGUID: h1/a3uwrTjKQrqngGD06Ug== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,201,1719903600"; d="scan'208";a="69801822" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmviesa004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Sep 2024 00:07:53 -0700 Received: from kekkonen.localdomain (localhost [127.0.0.1]) by kekkonen.fi.intel.com (Postfix) with SMTP id 75EF311F781; Wed, 4 Sep 2024 10:07:50 +0300 (EEST) Date: Wed, 4 Sep 2024 07:07:50 +0000 From: Sakari Ailus To: Tomi Valkeinen Cc: Mauro Carvalho Chehab , Raspberry Pi Kernel Maintenance , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Florian Fainelli , Broadcom internal kernel review list , Rob Herring , Krzysztof Kozlowski , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Naushir Patuck , Laurent Pinchart , Jacopo Mondi , Kieran Bingham Subject: Re: [PATCH v3 3/4] media: raspberrypi: Add support for RP1-CFE Message-ID: References: <20240815-rp1-cfe-v3-0-e15a979db327@ideasonboard.com> <20240815-rp1-cfe-v3-3-e15a979db327@ideasonboard.com> <0a9f8470-606b-47e2-a622-038a2c070b5e@ideasonboard.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <0a9f8470-606b-47e2-a622-038a2c070b5e@ideasonboard.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240904_000759_956132_7D3601BA X-CRM114-Status: GOOD ( 55.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Moi, On Mon, Sep 02, 2024 at 01:05:42PM +0300, Tomi Valkeinen wrote: > Hi Sakari, > > Thanks for the review! You're welcome. > > > +#define cfe_dbg(fmt, arg...) dev_dbg(&cfe->pdev->dev, fmt, ##arg) > > > > cfe should be an argument to cfe_dbg(). > > Why? This, and the ones below, is an internal macro to make it easier and > shorter to do prints. Adding the parameter gives no benefit that I can see. Generally macros shouldn't expect certain variables not defined on the same level the macros themselves. It gets harder to maintain this way. > > > +#define node_supports_image_output(node) \ > > > + (!!(node_desc[(node)->id].caps & V4L2_CAP_VIDEO_CAPTURE)) > > > > No need to cast to bool through !!. Same below. > > I like my bools to be bools, not ints... But at the same time, I don't see > how that would cause issues in the uses we have in this driver. So I'll drop > these. Alternatively, explicitly cast to bool. But I don't think it's needed. > > > > +#define node_supports_meta_output(node) \ > > > + (!!(node_desc[(node)->id].caps & V4L2_CAP_META_CAPTURE)) > > > +#define node_supports_image_input(node) \ > > > + (!!(node_desc[(node)->id].caps & V4L2_CAP_VIDEO_OUTPUT)) > > > +#define node_supports_meta_input(node) \ > > > + (!!(node_desc[(node)->id].caps & V4L2_CAP_META_OUTPUT)) > > > +#define node_supports_image(node) \ > > > + (node_supports_image_output(node) || node_supports_image_input(node)) > > > +#define node_supports_meta(node) \ > > > + (node_supports_meta_output(node) || node_supports_meta_input(node)) > > > + > > > +#define is_image_output_node(node) \ > > > + ((node)->buffer_queue.type == V4L2_BUF_TYPE_VIDEO_CAPTURE) > > > +#define is_image_input_node(node) \ > > > + ((node)->buffer_queue.type == V4L2_BUF_TYPE_VIDEO_OUTPUT) > > > +#define is_image_node(node) \ > > > + (is_image_output_node(node) || is_image_input_node(node)) > > > +#define is_meta_output_node(node) \ > > > + ((node)->buffer_queue.type == V4L2_BUF_TYPE_META_CAPTURE) > > > +#define is_meta_input_node(node) \ > > > + ((node)->buffer_queue.type == V4L2_BUF_TYPE_META_OUTPUT) > > > +#define is_meta_node(node) \ > > > + (is_meta_output_node(node) || is_meta_input_node(node)) > > > + > > > +/* To track state across all nodes. */ > > > +#define NUM_STATES 5 This might be nicer if declared as last. > > > +#define NODE_REGISTERED BIT(0) > > > +#define NODE_ENABLED BIT(1) > > > +#define NODE_STREAMING BIT(2) > > > +#define FS_INT BIT(3) > > > +#define FE_INT BIT(4) ... > > > +static int cfe_start_channel(struct cfe_node *node) > > > +{ > > > + struct cfe_device *cfe = node->cfe; > > > + struct v4l2_subdev_state *state; > > > + struct v4l2_mbus_framefmt *source_fmt; > > > + const struct cfe_fmt *fmt; > > > + unsigned long flags; > > > + bool start_fe; > > > + int ret; > > > + > > > + cfe_dbg("%s: [%s]\n", __func__, node_desc[node->id].name); > > > > This looks like a development time leftover. There are quite a few such > > prints that provide little information anyway. How about removing them all? > > These are very valuable when testing, fixing or improving the driver. If I > were to remove them, I would just have to add them back whenever I'd be > doing something with the driver and things would not work perfectly. > > The debug prints we have are all low frequency. There's a bunch printed when > starting the streaming and when stopping it, but the debug prints are not > used while streaming is on-going, and instead we have trace events for that. I'm fine with debug prints when they do print useful information but you have many of them just in the beginning of the function, printing only the function name (and possibly the node name). I'd remove those, except in cases where calling the function itself is useful information, such as on returning the buffers. > > > +static int cfe_link_node_pads(struct cfe_device *cfe) > > > +{ > > > + int ret; > > > + int pad; > > > + > > > + /* Source -> CSI2 */ > > > + > > > + pad = media_entity_get_fwnode_pad(&cfe->source_sd->entity, > > > + cfe->remote_ep_fwnode, > > > + MEDIA_PAD_FL_SOURCE); > > > + if (pad < 0) { > > > + cfe_err("Source %s has no connected source pad\n", > > > + cfe->source_sd->name); > > > + return pad; > > > + } > > > + > > > + cfe->source_pad = pad; > > > + > > > + ret = media_create_pad_link(&cfe->source_sd->entity, pad, > > > + &cfe->csi2.sd.entity, CSI2_PAD_SINK, > > > + MEDIA_LNK_FL_IMMUTABLE | > > > + MEDIA_LNK_FL_ENABLED); > > > + if (ret) > > > + return ret; > > > + > > > + for (unsigned int i = 0; i < CSI2_NUM_CHANNELS; i++) { > > > + struct cfe_node *node = &cfe->node[i]; > > > + > > > + if (!check_state(cfe, NODE_REGISTERED, i)) > > > + continue; > > > + > > > + /* CSI2 channel # -> /dev/video# */ > > > + ret = media_create_pad_link(&cfe->csi2.sd.entity, > > > + node_desc[i].link_pad, > > > + &node->video_dev.entity, 0, 0); > > > + if (ret) > > > + return ret; > > > + > > > + if (node_supports_image(node)) { > > > + /* CSI2 channel # -> FE Input */ > > > + ret = media_create_pad_link(&cfe->csi2.sd.entity, > > > + node_desc[i].link_pad, > > > + &cfe->fe.sd.entity, > > > + FE_STREAM_PAD, 0); > > > + if (ret) > > > + return ret; > > > + } > > > + } > > > + > > > + for (unsigned int i = CSI2_NUM_CHANNELS; i < NUM_NODES; i++) { > > > + struct cfe_node *node = &cfe->node[i]; > > > + struct media_entity *src, *dst; > > > + unsigned int src_pad, dst_pad; > > > + > > > + if (node_desc[i].pad_flags & MEDIA_PAD_FL_SINK) { > > > + /* FE -> /dev/video# */ > > > + src = &cfe->fe.sd.entity; > > > + src_pad = node_desc[i].link_pad; > > > + dst = &node->video_dev.entity; > > > + dst_pad = 0; > > > + } else { > > > + /* /dev/video# -> FE */ > > > + dst = &cfe->fe.sd.entity; > > > + dst_pad = node_desc[i].link_pad; > > > + src = &node->video_dev.entity; > > > + src_pad = 0; > > > + } > > > + > > > + ret = media_create_pad_link(src, src_pad, dst, dst_pad, 0); > > > + if (ret) > > > + return ret; > > > + } > > > + > > > + return 0; > > > +} > > > + > > > +static int cfe_probe_complete(struct cfe_device *cfe) > > > +{ > > > + int ret; > > > + > > > + cfe->v4l2_dev.notify = cfe_notify; > > > + > > > + for (unsigned int i = 0; i < NUM_NODES; i++) { > > > + ret = cfe_register_node(cfe, i); > > > + if (ret) { > > > + cfe_err("Unable to register video node %u.\n", i); > > > + goto unregister; > > > + } > > > + } > > > + > > > + ret = cfe_link_node_pads(cfe); > > > + if (ret) { > > > + cfe_err("Unable to link node pads.\n"); > > > + goto unregister; > > > + } > > > + > > > + ret = v4l2_device_register_subdev_nodes(&cfe->v4l2_dev); > > > + if (ret) { > > > + cfe_err("Unable to register subdev nodes.\n"); > > > + goto unregister; > > > + } > > > + > > > + return 0; > > > + > > > +unregister: > > > + cfe_unregister_nodes(cfe); > > > + return ret; > > > +} > > > + > > > +static int cfe_async_bound(struct v4l2_async_notifier *notifier, > > > + struct v4l2_subdev *subdev, > > > + struct v4l2_async_connection *asd) > > > +{ > > > + struct cfe_device *cfe = to_cfe_device(notifier->v4l2_dev); > > > + > > > + if (cfe->source_sd) { > > > + cfe_err("Rejecting subdev %s (Already set!!)", subdev->name); > > > + return 0; > > > + } > > > + > > > + cfe->source_sd = subdev; > > > + > > > + cfe_dbg("Using source %s for capture\n", subdev->name); > > > + > > > + return 0; > > > +} > > > + > > > +static int cfe_async_complete(struct v4l2_async_notifier *notifier) > > > +{ > > > + struct cfe_device *cfe = to_cfe_device(notifier->v4l2_dev); > > > + > > > + return cfe_probe_complete(cfe); > > > +} > > > + > > > +static const struct v4l2_async_notifier_operations cfe_async_ops = { > > > + .bound = cfe_async_bound, > > > + .complete = cfe_async_complete, > > > +}; > > > + > > > +static int cfe_register_async_nf(struct cfe_device *cfe) > > > +{ > > > + struct platform_device *pdev = cfe->pdev; > > > + struct v4l2_fwnode_endpoint ep = { .bus_type = V4L2_MBUS_CSI2_DPHY }; > > > + int ret = -EINVAL; Is the assignment necessary? > > > + struct fwnode_handle *local_ep_fwnode; > > > + struct fwnode_handle *remote_ep_fwnode; > > > + struct v4l2_async_connection *asd; > > > + > > > + local_ep_fwnode = fwnode_graph_get_endpoint_by_id(pdev->dev.fwnode, 0, 0, 0); > > > + if (!local_ep_fwnode) { > > > + cfe_err("Failed to find local endpoint fwnode\n"); > > > + return -ENODEV; > > > + } > > > + > > > + remote_ep_fwnode = fwnode_graph_get_remote_endpoint(local_ep_fwnode); > > > + if (!remote_ep_fwnode) { > > > + cfe_err("Failed to find remote endpoint fwnode\n"); > > > + ret = -ENODEV; > > > + goto err_put_local_fwnode; > > > + } > > > + > > > + /* Parse the local endpoint and validate its configuration. */ > > > + v4l2_fwnode_endpoint_parse(local_ep_fwnode, &ep); You'll need to check the return value here. > > > + > > > + if (ep.bus_type != V4L2_MBUS_CSI2_DPHY) { This check is redundant. > > > + cfe_err("endpoint node type != CSI2\n"); > > > + ret = -EINVAL; > > > + goto err_put_remote_fwnode; > > > + } > > > + > > > + for (unsigned int lane = 0; lane < ep.bus.mipi_csi2.num_data_lanes; lane++) { > > > + if (ep.bus.mipi_csi2.data_lanes[lane] != lane + 1) { > > > + cfe_err("subdevice %pfwf: data lanes reordering not supported\n", > > > + remote_ep_fwnode); > > > + ret = -EINVAL; > > > + goto err_put_remote_fwnode; > > > + } > > > + } > > > + > > > + cfe->csi2.dphy.max_lanes = ep.bus.mipi_csi2.num_data_lanes; > > > + cfe->csi2.bus_flags = ep.bus.mipi_csi2.flags; > > > + > > > + cfe->remote_ep_fwnode = remote_ep_fwnode; > > > + > > > + cfe_dbg("source %pfwf: %u data lanes, flags=0x%08x\n", > > > + remote_ep_fwnode, cfe->csi2.dphy.max_lanes, cfe->csi2.bus_flags); > > > + > > > + /* Initialize and register the async notifier. */ > > > + v4l2_async_nf_init(&cfe->notifier, &cfe->v4l2_dev); > > > + cfe->notifier.ops = &cfe_async_ops; > > > + > > > + asd = v4l2_async_nf_add_fwnode(&cfe->notifier, remote_ep_fwnode, > > > + struct v4l2_async_connection); > > > > Could you use v4l2_async_nf_add_fwnode_remote() and just not bother with > > remote_ep_fwnode at all? > > I need the remote_ep_fwnode in cfe_link_node_pads() when creating the links. > > Is there some other way to get the remote pad so that I can call the > media_create_pad_link()? Could you use v4l2_create_fwnode_links_to_pad() for that? > > > > + if (IS_ERR(asd)) { > > > + ret = PTR_ERR(asd); > > > + cfe_err("Error adding subdevice: %d\n", ret); > > > + goto err_put_remote_fwnode; > > > + } > > > + > > > + ret = v4l2_async_nf_register(&cfe->notifier); > > > + if (ret) { > > > + cfe_err("Error registering async notifier: %d\n", ret); > > > + goto err_nf_cleanup; > > > + } > > > + > > > + fwnode_handle_put(local_ep_fwnode); > > > + > > > + return 0; > > > + > > > +err_nf_cleanup: > > > + v4l2_async_nf_cleanup(&cfe->notifier); > > > +err_put_remote_fwnode: > > > + fwnode_handle_put(remote_ep_fwnode); > > > +err_put_local_fwnode: > > > + fwnode_handle_put(local_ep_fwnode); > > > + > > > + return ret; > > > +} ... > > > +static void cfe_remove(struct platform_device *pdev) > > > +{ > > > + struct cfe_device *cfe = platform_get_drvdata(pdev); > > > + > > > + debugfs_remove(cfe->debugfs); > > > + > > > + v4l2_async_nf_unregister(&cfe->notifier); > > > + v4l2_async_nf_cleanup(&cfe->notifier); > > > + > > > + media_device_unregister(&cfe->mdev); > > > > Do you think you might have the time to write patch that would convert the > > driver to use media device refcounting? It's not needed for upstreaming > > though, but would be nice to have such a new driver to be converted when we > > (hopefully not in too distant future) could merge that set. > > I had a quick try. I noticed that the current driver will crash if streaming > is active when unbinding/unloading, so that needs fixing anyway. > > But with the v4 series ([PATCH v4 00/26] Media device lifetime management), > I hit WARN_ON(!vdev->release) in __video_register_device(). I haven't > studied the series that much, but I understood vdev's release will be > handled via the media device. Is that not right? Let's discuss this online. > > > + debugfs_create_file("csi2_regs", 0444, debugfs, csi2, &csi2_regs_fops); > > > > Should this reflect the device name? This will currently only work with one > > such device in the system. > > These files are in a device specific directory, e.g. > "rp1-cfe:1f00110000.csi/". Ack. -- Kind regards, Sakari Ailus