From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D6DFC3DA6F for ; Thu, 24 Aug 2023 17:03:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242835AbjHXRDV (ORCPT ); Thu, 24 Aug 2023 13:03:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44352 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242827AbjHXRDS (ORCPT ); Thu, 24 Aug 2023 13:03:18 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC82710F9; Thu, 24 Aug 2023 10:03:11 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 9112622088; Thu, 24 Aug 2023 17:03:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1692896590; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jrAG7m96VPC1mkTaluLxh7W982yxT13Stt9OhGrTnBs=; b=WrjeBYzEG/Pn5m6AeyEiknOSIMqtAdDqClNOS/3SjpN58WUjMFzLtjWQPgshHDcl79tLiI pxGoPHEF3b0DYnskNcEh8VSmQw6Xa2qWdmmlc4u5l1QWP7QDLGB92Bk0sKT8MxcMJxZcIx 44n9rZ7Dpuf3fD+u1C6AVuwrd/5ldKo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1692896590; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jrAG7m96VPC1mkTaluLxh7W982yxT13Stt9OhGrTnBs=; b=P0cx/WiwaKOXPVf85RzezDUPaF9gG2B68rMGFZe+R6tSPL2WBYVwB1RO8q432Pfsjn6t3C sMP+sAcwlmMjXFDQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 2AAA11336F; Thu, 24 Aug 2023 17:03:10 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 7QA8CU6N52Q8bgAAMHmgww (envelope-from ); Thu, 24 Aug 2023 17:03:10 +0000 Date: Thu, 24 Aug 2023 19:03:09 +0200 Message-ID: <87wmxk8jaq.wl-tiwai@suse.de> From: Takashi Iwai To: Shengjiu Wang Cc: Mark Brown , Hans Verkuil , Shengjiu Wang , sakari.ailus@iki.fi, tfiga@chromium.org, m.szyprowski@samsung.com, mchehab@kernel.org, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, Xiubo.Lee@gmail.com, festevam@gmail.com, nicoleotsuka@gmail.com, lgirdwood@gmail.com, perex@perex.cz, tiwai@suse.com, alsa-devel@alsa-project.org, linuxppc-dev@lists.ozlabs.org Subject: Re: [RFC PATCH v2 0/7] Add audio support in v4l2 framework In-Reply-To: References: <1690265540-25999-1-git-send-email-shengjiu.wang@nxp.com> <47d66c28-1eb2-07f5-d6f9-779d675aefe8@xs4all.nl> <87il9xu1ro.wl-tiwai@suse.de> <87il9xoddo.wl-tiwai@suse.de> <844ef9b6-d5e2-46a9-b7a5-7ee86a2e449c@sirena.org.uk> User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/27.2 Mule/6.0 MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 23 Aug 2023 16:33:19 +0200, Shengjiu Wang wrote: > > On Fri, Aug 11, 2023 at 7:05 PM Shengjiu Wang wrote: > > > > Hi Mark, Takashi > > > > On Thu, Aug 3, 2023 at 9:11 PM Shengjiu Wang wrote: > > > > > > On Thu, Aug 3, 2023 at 1:28 AM Mark Brown wrote: > > > > > > > > On Wed, Aug 02, 2023 at 10:41:43PM +0800, Shengjiu Wang wrote: > > > > > > > > > Currently the ASRC in ALSA is to connect to another I2S device as > > > > > a sound card. But we'd like to the ASRC can be used by user space directly > > > > > that user space application can get the output after conversion from ASRC. > > > > > > > > That sort of use case would be handled via DPCM at the minute, though > > > > persuading it to connect two front ends together might be fun (which is > > > > the sort of reason why we want to push digital information down into > > > > DAPM and make everything a component). > > > > > > Thanks. > > > > > > ASRC M2M case needs to run as fast as possible, no sync clock control. > > > If use sound card to handle ASRC M2M case, the user application > > > should be aplay/arecord, then we need to consider xrun issue, buffer > > > may timeout, sync between aplay and arecord, these should't be > > > considered by pure memory to memory operation. > > > > > > DPCM may achitect all the audio things in components and sound > > > card, it is good. but for the M2M case, it is complcated. not sure > > > it is doable. > > > > > > > Beside the concern in previous mail, > > > > DPCM needs to separate ASRC to be two substreams (playback and capture). > > > > But the ASRC needs the sample rate & format of input and output first > > then start conversion. > > > > If the playback controls the rate & format of input, capture substream > > controls the rate & format of output, as a result > > one substream needs to get information(dma buffer address, size... > > rate, format) from another substream, then start both substreams in the > > last substream. How to synchronize these two substreams is a problem. > > One stream can be released but another stream doesn't know . > > > > So I don't think it is a good idea to use DPCM for pure M2M case. > > > > So can I persuade you to consider the V4L2 solution? > > > > Just a summary: > > Basic M2M conversion can work with DPCM, I have tried with some > workaround to make it work. > > But there are several issues: > 1. Need to create sound cards. ASRC module support multi instances, then > need to create multi sound cards for each instance. Hm, why can't it be multiple PCM instances instead? > 2. The ASRC is an entirety but with DPCM we need to separate input port and > output port to playback substream and capture stream. Synchronous between > playback substream and capture substream is a problem. > How to start them and stop them at the same time. This could be done by enforcing the full duplex and linking the both PCM streams, I suppose. > 3. How to handle the xrun issue. pause or resume. which are brought by ALSA. Doesn't V4L2 handle the overrun/underrun at all? Also, no resume support? Pause and resume are optional in ALSA frame work, you don't need to implement them unless you want/need. > So shall we make the decision that we can go to the V4L2 solution? Honestly speaking, I don't mind much whether it's implemented in V2L4 or not -- at least for the kernel part, we can reorganize / refactor things internally. But, the biggest remaining question to me is whether this user-space interface is the most suitable one. Is it well defined, usable and maintained for the audio applications? Or is it meant to be a stop-gap for a specific use case? thanks, Takashi