From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from alsa0.perex.cz (alsa0.perex.cz [77.48.224.243]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 20FE9C3DA66 for ; Fri, 25 Aug 2023 13:55:55 +0000 (UTC) Received: from alsa1.perex.cz (alsa1.perex.cz [207.180.221.201]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by alsa0.perex.cz (Postfix) with ESMTPS id C4959847; Fri, 25 Aug 2023 15:55:02 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.11.0 alsa0.perex.cz C4959847 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=alsa-project.org; s=default; t=1692971752; bh=x/wTDi1KQtRy3GO7+v7RDFR7PmlMwycWHdaLaMbRYT8=; h=Date:From:To:Cc:Subject:In-Reply-To:References:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=sLol4e0cvS8zmkR7BMqaFXcmDv1djbz7KaMACOWwPg7XCQaq0i0OGs37QtBkwPCO2 62Wdrk1QQfm98x2GBeBoxYDu78E6KRJDpMfJ4xNpaHGMsuHI2QUpe+uLRISGXvB2H6 JbCgacsP92DWQqdHL0KCWCkbCzPYy7INhgQK+S90= Received: by alsa1.perex.cz (Postfix, from userid 50401) id 4C963F800D1; Fri, 25 Aug 2023 15:54:32 +0200 (CEST) Received: from mailman-core.alsa-project.org (mailman-core.alsa-project.org [10.254.200.10]) by alsa1.perex.cz (Postfix) with ESMTP id 0A81FF800BF; Fri, 25 Aug 2023 15:54:32 +0200 (CEST) Received: by alsa1.perex.cz (Postfix, from userid 50401) id 307FFF800F5; Fri, 25 Aug 2023 15:54:27 +0200 (CEST) Received: from smtp-out2.suse.de (smtp-out2.suse.de [IPv6:2001:67c:2178:6::1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by alsa1.perex.cz (Postfix) with ESMTPS id B3A40F800AE for ; Fri, 25 Aug 2023 15:54:15 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.11.0 alsa1.perex.cz B3A40F800AE Authentication-Results: alsa1.perex.cz; dkim=pass (1024-bit key, unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=oaXz39cK; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=49KR5S3p Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id B05A11F45E; Fri, 25 Aug 2023 13:54:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1692971653; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XKT17FbYypZcEOfeXYaNtZ6Vwk1v8XxDX315rlANfbM=; b=oaXz39cK3qcNjEBK5WwfqkPFOhZhOdCfDdzvyCvqfMG8kYdvk/WFEM8AQgd4KY9W+/Xo2/ E+14Ro0lcO4OSf9Zxf0w6da0XFtLFm0z2H4ik4XsNMVFtKFhMI6QmdOl5ydZYav7rGyV2j qQEgupRXOTNkG1M7/ff3FJeiWUtYUfk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1692971653; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XKT17FbYypZcEOfeXYaNtZ6Vwk1v8XxDX315rlANfbM=; b=49KR5S3p5IqEhp4UE32uf1qWdQM0x6sRKyRhKKMSfQBcjoZj/rr1e+PqWIINOHy9PPiXBz Tf09mag9blyJ2gBA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 39122138F9; Fri, 25 Aug 2023 13:54:13 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id tc/pDIWy6GTCBgAAMHmgww (envelope-from ); Fri, 25 Aug 2023 13:54:13 +0000 Date: Fri, 25 Aug 2023 15:54:12 +0200 Message-ID: <8735076xdn.wl-tiwai@suse.de> From: Takashi Iwai To: Shengjiu Wang Cc: Mark Brown , Hans Verkuil , Shengjiu Wang , sakari.ailus@iki.fi, tfiga@chromium.org, m.szyprowski@samsung.com, mchehab@kernel.org, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, Xiubo.Lee@gmail.com, festevam@gmail.com, nicoleotsuka@gmail.com, lgirdwood@gmail.com, perex@perex.cz, tiwai@suse.com, alsa-devel@alsa-project.org, linuxppc-dev@lists.ozlabs.org Subject: Re: [RFC PATCH v2 0/7] Add audio support in v4l2 framework In-Reply-To: References: <47d66c28-1eb2-07f5-d6f9-779d675aefe8@xs4all.nl> <87il9xu1ro.wl-tiwai@suse.de> <87il9xoddo.wl-tiwai@suse.de> <844ef9b6-d5e2-46a9-b7a5-7ee86a2e449c@sirena.org.uk> <87wmxk8jaq.wl-tiwai@suse.de> User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/27.2 Mule/6.0 MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Message-ID-Hash: IN7QISWVTHSUHEHK5H2ZHJQ2VAMTV7JG X-Message-ID-Hash: IN7QISWVTHSUHEHK5H2ZHJQ2VAMTV7JG X-MailFrom: tiwai@suse.de X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-alsa-devel.alsa-project.org-0; header-match-alsa-devel.alsa-project.org-1; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.8 Precedence: list List-Id: "Alsa-devel mailing list for ALSA developers - http://www.alsa-project.org" Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Fri, 25 Aug 2023 05:46:43 +0200, Shengjiu Wang wrote: > > On Fri, Aug 25, 2023 at 4:21 AM Mark Brown wrote: > > > > On Thu, Aug 24, 2023 at 07:03:09PM +0200, Takashi Iwai wrote: > > > Shengjiu Wang wrote: > > > > > > But there are several issues: > > > > 1. Need to create sound cards. ASRC module support multi instances, then > > > > need to create multi sound cards for each instance. > > > > > Hm, why can't it be multiple PCM instances instead? > > > > I'm having a hard time following this one too. > > > > > > 2. The ASRC is an entirety but with DPCM we need to separate input port and > > > > output port to playback substream and capture stream. Synchronous between > > > > playback substream and capture substream is a problem. > > > > How to start them and stop them at the same time. > > > > > This could be done by enforcing the full duplex and linking the both > > > PCM streams, I suppose. > > > > > > So shall we make the decision that we can go to the V4L2 solution? > > > > > Honestly speaking, I don't mind much whether it's implemented in V2L4 > > > or not -- at least for the kernel part, we can reorganize / refactor > > > things internally. But, the biggest remaining question to me is > > > whether this user-space interface is the most suitable one. Is it > > > well defined, usable and maintained for the audio applications? Or > > > is it meant to be a stop-gap for a specific use case? > > > > I'm having a really hard time summoning much enthusiasm for using v4l > > here, it feels like this is heading down the same bodge route as DPCM > > but directly as ABI so even harder to fix properly. That said all the > > ALSA APIs are really intended to be used in real time and this sounds > > like a non real time application? I don't fully understand what the > > actual use case is here. > > Thanks for your reply. > > This asrc memory to memory (memory ->asrc->memory) case is a non > real time use case. > > User fills the input buffer to the asrc module, after conversion, then asrc > sends back the output buffer to user. So it is not a traditional ALSA playback > and capture case. I think it is not good to create sound card for it, it is > not a sound card actually. > > It is a specific use case, there is no reference in current kernel. > v4l2 memory to memory is the closed implementation, v4l2 current > support video, image, radio, tuner, touch devices, so it is not > complicated to add support for this specific audio case. > > Maybe you can go through these patches first. Because we > had implemented the "memory -> asrc ->i2s device-> codec" > use case in ALSA. Now the "memory->asrc->memory" needs > to reuse the code in asrc driver, so the first 3 patches is for refining > the code to make it can be shared by the "memory->asrc->memory" > driver. > > The main change is in the v4l2 side, A /dev/vl42-audio will be created, > user applications only use the ioctl of v4l2 framework. Ah, now I'm slowly understanding. So, what you want is to have an interface to perform the batch conversion of a data stream from an input to an output? And ALSA PCM interface doesn't fit fully for that purpose because the data handling is batched and it's not like a normal PCM streaming? Basically the whole M2M arguments are rather subtle. Those are implementation details that can be resolved in several different ways in the kernel side. But the design of the operation is the crucial point. Maybe we can consider implementing a similar feature in ALSA API, too. But it's too far-stretched for now. So, if v4l2 interface provides the requested feature (the batched audio stream conversion), it's OK to ride on it. thanks, Takashi