From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from alsa0.perex.cz (alsa0.perex.cz [77.48.224.243]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DF795C00528 for ; Wed, 2 Aug 2023 12:10:04 +0000 (UTC) Received: from alsa1.perex.cz (alsa1.perex.cz [207.180.221.201]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by alsa0.perex.cz (Postfix) with ESMTPS id B6EFB6C0; Wed, 2 Aug 2023 14:09:12 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.11.0 alsa0.perex.cz B6EFB6C0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=alsa-project.org; s=default; t=1690978202; bh=whQlH0dOc48bO3JEIL0F7sPMnnLRJntQO65G1/ZHeWQ=; h=Date:From:To:Cc:Subject:In-Reply-To:References:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=rz03YAX5NiETJ7WX2bklNi1Lf2U0w7sKEF+ABMsAI2QCS/S/vICMIin26/FUCShyi pKu3NIIuRMgV12/PdiYwub3SS2sAz0rpawCK5Z1JOmix8S4Fqy1tX/aC69s4mKOIXL +toW3V4wVSOUFURbkW0+v+63PapPWfNjFI/6lTVQ= Received: by alsa1.perex.cz (Postfix, from userid 50401) id 5D676F80425; Wed, 2 Aug 2023 14:08:45 +0200 (CEST) Received: from mailman-core.alsa-project.org (mailman-core.alsa-project.org [10.254.200.10]) by alsa1.perex.cz (Postfix) with ESMTP id 17890F8025A; Wed, 2 Aug 2023 14:08:45 +0200 (CEST) Received: by alsa1.perex.cz (Postfix, from userid 50401) id F0C17F8025A; Wed, 2 Aug 2023 14:08:40 +0200 (CEST) Received: from smtp-out2.suse.de (smtp-out2.suse.de [IPv6:2001:67c:2178:6::1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by alsa1.perex.cz (Postfix) with ESMTPS id 17702F80149 for ; Wed, 2 Aug 2023 14:08:36 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.11.0 alsa1.perex.cz 17702F80149 Authentication-Results: alsa1.perex.cz; dkim=pass (1024-bit key, unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=AXjMJDzt; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=a5+VLwV3 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 122E81F38D; Wed, 2 Aug 2023 12:08:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1690978116; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wxPHjo3+ia+QJjBrjbeie8E5g2hZQ55e92AAzrGOhEI=; b=AXjMJDzt8LQ5Aws+NBUjfUEoMB9Yz7f6O0HMW0it0ifUd7WEIaIHEP4qncTGjhhD6EDoxb F7gnLl+5o8NiHW19hpC3ep6emXodiyGqOw2WhbgfiVBTf4qGBKd7jTJ1qGRp85rauiY6Oa d1aiZV+hUmtFcwdfyiPu5SoMzIcBsZw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1690978116; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wxPHjo3+ia+QJjBrjbeie8E5g2hZQ55e92AAzrGOhEI=; b=a5+VLwV3cvmphzmj2zlwrNKt4nmwdNbp2bySub5cb7YvNuOmBZ5b0GmUbqi8u7jfqmS9X0 to8kobwjwCyaFLAg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id DB90A13909; Wed, 2 Aug 2023 12:08:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id WG4jNUNHymR8PwAAMHmgww (envelope-from ); Wed, 02 Aug 2023 12:08:35 +0000 Date: Wed, 02 Aug 2023 14:08:35 +0200 Message-ID: <87il9xoddo.wl-tiwai@suse.de> From: Takashi Iwai To: Shengjiu Wang Cc: Hans Verkuil , Shengjiu Wang , sakari.ailus@iki.fi, tfiga@chromium.org, m.szyprowski@samsung.com, mchehab@kernel.org, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, Xiubo.Lee@gmail.com, festevam@gmail.com, nicoleotsuka@gmail.com, lgirdwood@gmail.com, broonie@kernel.org, perex@perex.cz, tiwai@suse.com, alsa-devel@alsa-project.org, linuxppc-dev@lists.ozlabs.org Subject: Re: [RFC PATCH v2 0/7] Add audio support in v4l2 framework In-Reply-To: References: <1690265540-25999-1-git-send-email-shengjiu.wang@nxp.com> <47d66c28-1eb2-07f5-d6f9-779d675aefe8@xs4all.nl> <87il9xu1ro.wl-tiwai@suse.de> User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/27.2 Mule/6.0 MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Message-ID-Hash: CSGO3GYA4VB3EKOOAFRIW3WLKDWVJ6WL X-Message-ID-Hash: CSGO3GYA4VB3EKOOAFRIW3WLKDWVJ6WL X-MailFrom: tiwai@suse.de X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-alsa-devel.alsa-project.org-0; header-match-alsa-devel.alsa-project.org-1; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.8 Precedence: list List-Id: "Alsa-devel mailing list for ALSA developers - http://www.alsa-project.org" Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Wed, 02 Aug 2023 14:02:29 +0200, Shengjiu Wang wrote: > > On Wed, Aug 2, 2023 at 7:22 PM Takashi Iwai wrote: > > > > On Wed, 02 Aug 2023 09:32:37 +0200, > > Hans Verkuil wrote: > > > > > > Hi all, > > > > > > On 25/07/2023 08:12, Shengjiu Wang wrote: > > > > Audio signal processing has the requirement for memory to > > > > memory similar as Video. > > > > > > > > This patch is to add this support in v4l2 framework, defined > > > > new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and > > > > V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format > > > > for audio case usage. > > > > > > > > The created audio device is named "/dev/audioX". > > > > > > > > And add memory to memory support for two kinds of i.MX ASRC > > > > module > > > > > > Before I spend time on this: are the audio maintainers OK with doing > > > this in V4L2? > > > > > > I do want to have a clear statement on this as it is not something I > > > can decide. > > > > Well, I personally don't mind to have some audio capability in v4l2 > > layer. But, the only uncertain thing for now is whether this is a > > must-have or not. > > > > Thanks, I am also not sure about this. I am also confused that why > there is no m2m implementation for audio in the kernel. Audio also > has similar decoder encoder post-processing as video. > > > > > IIRC, the implementation in the sound driver side was never done just > > because there was no similar implementation? If so, and if the > > extension to the v4l2 core layer is needed, shouldn't it be more > > considered for the possible other route? > > > > Actually I'd like someone could point me to the other route. I'd like to > try. > > The reason why I select to extend v4l2 for such audio usage is that v4l2 > looks best for this audio m2m implementation. v4l2 is designed for m2m > usage. if we need implement another 'route', I don't think it can do better > that v4l2. > > I appreciate that someone can share his ideas or doable solutions. > And please don't ignore my request, ignore my patch. Can you explain a bit more details of your demand? At least, a "big picture" showing how your hardware is implemented and what is exactly necessary would be helpful for understanding the problem. thanks, Takashi From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 87A26C04E69 for ; Wed, 2 Aug 2023 12:09:40 +0000 (UTC) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=AXjMJDzt; dkim=fail reason="signature verification failed" header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=a5+VLwV3; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4RG9l62Y5xz3cRy for ; Wed, 2 Aug 2023 22:09:38 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=AXjMJDzt; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=a5+VLwV3; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=suse.de (client-ip=195.135.220.29; helo=smtp-out2.suse.de; envelope-from=tiwai@suse.de; receiver=lists.ozlabs.org) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4RG9k54TMLz30Kg for ; Wed, 2 Aug 2023 22:08:45 +1000 (AEST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 122E81F38D; Wed, 2 Aug 2023 12:08:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1690978116; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wxPHjo3+ia+QJjBrjbeie8E5g2hZQ55e92AAzrGOhEI=; b=AXjMJDzt8LQ5Aws+NBUjfUEoMB9Yz7f6O0HMW0it0ifUd7WEIaIHEP4qncTGjhhD6EDoxb F7gnLl+5o8NiHW19hpC3ep6emXodiyGqOw2WhbgfiVBTf4qGBKd7jTJ1qGRp85rauiY6Oa d1aiZV+hUmtFcwdfyiPu5SoMzIcBsZw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1690978116; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wxPHjo3+ia+QJjBrjbeie8E5g2hZQ55e92AAzrGOhEI=; b=a5+VLwV3cvmphzmj2zlwrNKt4nmwdNbp2bySub5cb7YvNuOmBZ5b0GmUbqi8u7jfqmS9X0 to8kobwjwCyaFLAg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id DB90A13909; Wed, 2 Aug 2023 12:08:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id WG4jNUNHymR8PwAAMHmgww (envelope-from ); Wed, 02 Aug 2023 12:08:35 +0000 Date: Wed, 02 Aug 2023 14:08:35 +0200 Message-ID: <87il9xoddo.wl-tiwai@suse.de> From: Takashi Iwai To: Shengjiu Wang Subject: Re: [RFC PATCH v2 0/7] Add audio support in v4l2 framework In-Reply-To: References: <1690265540-25999-1-git-send-email-shengjiu.wang@nxp.com> <47d66c28-1eb2-07f5-d6f9-779d675aefe8@xs4all.nl> <87il9xu1ro.wl-tiwai@suse.de> User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/27.2 Mule/6.0 MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: nicoleotsuka@gmail.com, alsa-devel@alsa-project.org, lgirdwood@gmail.com, Xiubo.Lee@gmail.com, linux-kernel@vger.kernel.org, Shengjiu Wang , tiwai@suse.com, linux-media@vger.kernel.org, tfiga@chromium.org, Hans Verkuil , linuxppc-dev@lists.ozlabs.org, broonie@kernel.org, sakari.ailus@iki.fi, perex@perex.cz, mchehab@kernel.org, festevam@gmail.com, m.szyprowski@samsung.com Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Wed, 02 Aug 2023 14:02:29 +0200, Shengjiu Wang wrote: > > On Wed, Aug 2, 2023 at 7:22 PM Takashi Iwai wrote: > > > > On Wed, 02 Aug 2023 09:32:37 +0200, > > Hans Verkuil wrote: > > > > > > Hi all, > > > > > > On 25/07/2023 08:12, Shengjiu Wang wrote: > > > > Audio signal processing has the requirement for memory to > > > > memory similar as Video. > > > > > > > > This patch is to add this support in v4l2 framework, defined > > > > new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and > > > > V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format > > > > for audio case usage. > > > > > > > > The created audio device is named "/dev/audioX". > > > > > > > > And add memory to memory support for two kinds of i.MX ASRC > > > > module > > > > > > Before I spend time on this: are the audio maintainers OK with doing > > > this in V4L2? > > > > > > I do want to have a clear statement on this as it is not something I > > > can decide. > > > > Well, I personally don't mind to have some audio capability in v4l2 > > layer. But, the only uncertain thing for now is whether this is a > > must-have or not. > > > > Thanks, I am also not sure about this. I am also confused that why > there is no m2m implementation for audio in the kernel. Audio also > has similar decoder encoder post-processing as video. > > > > > IIRC, the implementation in the sound driver side was never done just > > because there was no similar implementation? If so, and if the > > extension to the v4l2 core layer is needed, shouldn't it be more > > considered for the possible other route? > > > > Actually I'd like someone could point me to the other route. I'd like to > try. > > The reason why I select to extend v4l2 for such audio usage is that v4l2 > looks best for this audio m2m implementation. v4l2 is designed for m2m > usage. if we need implement another 'route', I don't think it can do better > that v4l2. > > I appreciate that someone can share his ideas or doable solutions. > And please don't ignore my request, ignore my patch. Can you explain a bit more details of your demand? At least, a "big picture" showing how your hardware is implemented and what is exactly necessary would be helpful for understanding the problem. thanks, Takashi