All of lore.kernel.org
 help / color / mirror / Atom feed
From: Mike Brady <mikebrady@eircom.net>
To: eric@anholt.net, stefan.wahren@i2se.com, gregkh@linuxfoundation.org
Cc: linux-arm-kernel@lists.infradead.org, devel@driverdev.osuosl.org,
	alsa-devel@alsa-project.org, f.fainelli@gmail.com, tiwai@suse.de,
	mikebrady@eircom.net, linux-kernel@vger.kernel.org,
	julia.lawall@lip6.fr, linux-rpi-kernel@lists.infradead.org,
	nishka.dasgupta_ug18@ashoka.edu.in, k.marinushkin@gmail.com
Subject: [PATCH v2] staging: bcm2835-audio: interpolate audio delay
Date: Mon, 22 Oct 2018 20:17:08 +0100	[thread overview]
Message-ID: <20181022191708.GA4659@ubuntu> (raw)
In-Reply-To: <20181018105715.GA6660@ubuntu>

When the BCM2835 audio output is used, userspace sees a jitter up to 10ms
in the audio position, aka "delay" -- the number of frames that must
be output before a new frame would be played.
Make this a bit nicer for userspace by interpolating the position
using the CPU clock.
The overhead is small -- an extra ktime_get() every time a GPU message
is sent -- and another call and a few calculations whenever the delay
is sought from userland.
At 48,000 frames per second, i.e. approximately 20 microseconds per
frame, it would take a clock inaccuracy of
20 microseconds in 10 milliseconds -- 2,000 parts per million --
to result in an inaccurate estimate, whereas
crystal- or resonator-based clocks typically have an
inaccuracy of 10s to 100s of parts per million.

Signed-off-by: Mike Brady <mikebrady@eircom.net>
---
Changes in v2 -- remove inappropriate addition of SNDRV_PCM_INFO_BATCH flag

 .../vc04_services/bcm2835-audio/bcm2835-pcm.c | 20 +++++++++++++++++++
 .../vc04_services/bcm2835-audio/bcm2835.h     |  1 +
 2 files changed, 21 insertions(+)

diff --git a/drivers/staging/vc04_services/bcm2835-audio/bcm2835-pcm.c b/drivers/staging/vc04_services/bcm2835-audio/bcm2835-pcm.c
index e66da11af5cf..9053b996cada 100644
--- a/drivers/staging/vc04_services/bcm2835-audio/bcm2835-pcm.c
+++ b/drivers/staging/vc04_services/bcm2835-audio/bcm2835-pcm.c
@@ -74,6 +74,7 @@ void bcm2835_playback_fifo(struct bcm2835_alsa_stream *alsa_stream,
 	atomic_set(&alsa_stream->pos, pos);
 
 	alsa_stream->period_offset += bytes;
+	alsa_stream->interpolate_start = ktime_get();
 	if (alsa_stream->period_offset >= alsa_stream->period_size) {
 		alsa_stream->period_offset %= alsa_stream->period_size;
 		snd_pcm_period_elapsed(substream);
@@ -243,6 +244,7 @@ static int snd_bcm2835_pcm_prepare(struct snd_pcm_substream *substream)
 	atomic_set(&alsa_stream->pos, 0);
 	alsa_stream->period_offset = 0;
 	alsa_stream->draining = false;
+	alsa_stream->interpolate_start = ktime_get();
 
 	return 0;
 }
@@ -292,6 +294,24 @@ snd_bcm2835_pcm_pointer(struct snd_pcm_substream *substream)
 {
 	struct snd_pcm_runtime *runtime = substream->runtime;
 	struct bcm2835_alsa_stream *alsa_stream = runtime->private_data;
+	ktime_t now = ktime_get();
+
+	/* Give userspace better delay reporting by interpolating between GPU
+	 * notifications, assuming audio speed is close enough to the clock
+	 * used for ktime
+	 */
+
+	if ((ktime_to_ns(alsa_stream->interpolate_start)) &&
+	    (ktime_compare(alsa_stream->interpolate_start, now) < 0)) {
+		u64 interval =
+			(ktime_to_ns(ktime_sub(now,
+				alsa_stream->interpolate_start)));
+		u64 frames_output_in_interval =
+			div_u64((interval * runtime->rate), 1000000000);
+		snd_pcm_sframes_t frames_output_in_interval_sized =
+			-frames_output_in_interval;
+		runtime->delay = frames_output_in_interval_sized;
+	}
 
 	return snd_pcm_indirect_playback_pointer(substream,
 		&alsa_stream->pcm_indirect,
diff --git a/drivers/staging/vc04_services/bcm2835-audio/bcm2835.h b/drivers/staging/vc04_services/bcm2835-audio/bcm2835.h
index e13435d1c205..595ad584243f 100644
--- a/drivers/staging/vc04_services/bcm2835-audio/bcm2835.h
+++ b/drivers/staging/vc04_services/bcm2835-audio/bcm2835.h
@@ -78,6 +78,7 @@ struct bcm2835_alsa_stream {
 	unsigned int period_offset;
 	unsigned int buffer_size;
 	unsigned int period_size;
+	ktime_t interpolate_start;
 
 	struct bcm2835_audio_instance *instance;
 	int idx;
-- 
2.17.1

WARNING: multiple messages have this Message-ID (diff)
From: mikebrady@eircom.net (Mike Brady)
To: linux-arm-kernel@lists.infradead.org
Subject: [PATCH v2] staging: bcm2835-audio: interpolate audio delay
Date: Mon, 22 Oct 2018 20:17:08 +0100	[thread overview]
Message-ID: <20181022191708.GA4659@ubuntu> (raw)
In-Reply-To: <20181018105715.GA6660@ubuntu>

When the BCM2835 audio output is used, userspace sees a jitter up to 10ms
in the audio position, aka "delay" -- the number of frames that must
be output before a new frame would be played.
Make this a bit nicer for userspace by interpolating the position
using the CPU clock.
The overhead is small -- an extra ktime_get() every time a GPU message
is sent -- and another call and a few calculations whenever the delay
is sought from userland.
At 48,000 frames per second, i.e. approximately 20 microseconds per
frame, it would take a clock inaccuracy of
20 microseconds in 10 milliseconds -- 2,000 parts per million --
to result in an inaccurate estimate, whereas
crystal- or resonator-based clocks typically have an
inaccuracy of 10s to 100s of parts per million.

Signed-off-by: Mike Brady <mikebrady@eircom.net>
---
Changes in v2 -- remove inappropriate addition of SNDRV_PCM_INFO_BATCH flag

 .../vc04_services/bcm2835-audio/bcm2835-pcm.c | 20 +++++++++++++++++++
 .../vc04_services/bcm2835-audio/bcm2835.h     |  1 +
 2 files changed, 21 insertions(+)

diff --git a/drivers/staging/vc04_services/bcm2835-audio/bcm2835-pcm.c b/drivers/staging/vc04_services/bcm2835-audio/bcm2835-pcm.c
index e66da11af5cf..9053b996cada 100644
--- a/drivers/staging/vc04_services/bcm2835-audio/bcm2835-pcm.c
+++ b/drivers/staging/vc04_services/bcm2835-audio/bcm2835-pcm.c
@@ -74,6 +74,7 @@ void bcm2835_playback_fifo(struct bcm2835_alsa_stream *alsa_stream,
 	atomic_set(&alsa_stream->pos, pos);
 
 	alsa_stream->period_offset += bytes;
+	alsa_stream->interpolate_start = ktime_get();
 	if (alsa_stream->period_offset >= alsa_stream->period_size) {
 		alsa_stream->period_offset %= alsa_stream->period_size;
 		snd_pcm_period_elapsed(substream);
@@ -243,6 +244,7 @@ static int snd_bcm2835_pcm_prepare(struct snd_pcm_substream *substream)
 	atomic_set(&alsa_stream->pos, 0);
 	alsa_stream->period_offset = 0;
 	alsa_stream->draining = false;
+	alsa_stream->interpolate_start = ktime_get();
 
 	return 0;
 }
@@ -292,6 +294,24 @@ snd_bcm2835_pcm_pointer(struct snd_pcm_substream *substream)
 {
 	struct snd_pcm_runtime *runtime = substream->runtime;
 	struct bcm2835_alsa_stream *alsa_stream = runtime->private_data;
+	ktime_t now = ktime_get();
+
+	/* Give userspace better delay reporting by interpolating between GPU
+	 * notifications, assuming audio speed is close enough to the clock
+	 * used for ktime
+	 */
+
+	if ((ktime_to_ns(alsa_stream->interpolate_start)) &&
+	    (ktime_compare(alsa_stream->interpolate_start, now) < 0)) {
+		u64 interval =
+			(ktime_to_ns(ktime_sub(now,
+				alsa_stream->interpolate_start)));
+		u64 frames_output_in_interval =
+			div_u64((interval * runtime->rate), 1000000000);
+		snd_pcm_sframes_t frames_output_in_interval_sized =
+			-frames_output_in_interval;
+		runtime->delay = frames_output_in_interval_sized;
+	}
 
 	return snd_pcm_indirect_playback_pointer(substream,
 		&alsa_stream->pcm_indirect,
diff --git a/drivers/staging/vc04_services/bcm2835-audio/bcm2835.h b/drivers/staging/vc04_services/bcm2835-audio/bcm2835.h
index e13435d1c205..595ad584243f 100644
--- a/drivers/staging/vc04_services/bcm2835-audio/bcm2835.h
+++ b/drivers/staging/vc04_services/bcm2835-audio/bcm2835.h
@@ -78,6 +78,7 @@ struct bcm2835_alsa_stream {
 	unsigned int period_offset;
 	unsigned int buffer_size;
 	unsigned int period_size;
+	ktime_t interpolate_start;
 
 	struct bcm2835_audio_instance *instance;
 	int idx;
-- 
2.17.1

WARNING: multiple messages have this Message-ID (diff)
From: Mike Brady <mikebrady@eircom.net>
To: eric@anholt.net, stefan.wahren@i2se.com, gregkh@linuxfoundation.org
Cc: f.fainelli@gmail.com, tiwai@suse.de,
	nishka.dasgupta_ug18@ashoka.edu.in, julia.lawall@lip6.fr,
	mikebrady@eircom.net, k.marinushkin@gmail.com,
	linux-rpi-kernel@lists.infradead.org,
	linux-arm-kernel@lists.infradead.org, devel@driverdev.osuosl.org,
	linux-kernel@vger.kernel.org, alsa-devel@alsa-project.org
Subject: [PATCH v2] staging: bcm2835-audio: interpolate audio delay
Date: Mon, 22 Oct 2018 20:17:08 +0100	[thread overview]
Message-ID: <20181022191708.GA4659@ubuntu> (raw)
In-Reply-To: <20181018105715.GA6660@ubuntu>

When the BCM2835 audio output is used, userspace sees a jitter up to 10ms
in the audio position, aka "delay" -- the number of frames that must
be output before a new frame would be played.
Make this a bit nicer for userspace by interpolating the position
using the CPU clock.
The overhead is small -- an extra ktime_get() every time a GPU message
is sent -- and another call and a few calculations whenever the delay
is sought from userland.
At 48,000 frames per second, i.e. approximately 20 microseconds per
frame, it would take a clock inaccuracy of
20 microseconds in 10 milliseconds -- 2,000 parts per million --
to result in an inaccurate estimate, whereas
crystal- or resonator-based clocks typically have an
inaccuracy of 10s to 100s of parts per million.

Signed-off-by: Mike Brady <mikebrady@eircom.net>
---
Changes in v2 -- remove inappropriate addition of SNDRV_PCM_INFO_BATCH flag

 .../vc04_services/bcm2835-audio/bcm2835-pcm.c | 20 +++++++++++++++++++
 .../vc04_services/bcm2835-audio/bcm2835.h     |  1 +
 2 files changed, 21 insertions(+)

diff --git a/drivers/staging/vc04_services/bcm2835-audio/bcm2835-pcm.c b/drivers/staging/vc04_services/bcm2835-audio/bcm2835-pcm.c
index e66da11af5cf..9053b996cada 100644
--- a/drivers/staging/vc04_services/bcm2835-audio/bcm2835-pcm.c
+++ b/drivers/staging/vc04_services/bcm2835-audio/bcm2835-pcm.c
@@ -74,6 +74,7 @@ void bcm2835_playback_fifo(struct bcm2835_alsa_stream *alsa_stream,
 	atomic_set(&alsa_stream->pos, pos);
 
 	alsa_stream->period_offset += bytes;
+	alsa_stream->interpolate_start = ktime_get();
 	if (alsa_stream->period_offset >= alsa_stream->period_size) {
 		alsa_stream->period_offset %= alsa_stream->period_size;
 		snd_pcm_period_elapsed(substream);
@@ -243,6 +244,7 @@ static int snd_bcm2835_pcm_prepare(struct snd_pcm_substream *substream)
 	atomic_set(&alsa_stream->pos, 0);
 	alsa_stream->period_offset = 0;
 	alsa_stream->draining = false;
+	alsa_stream->interpolate_start = ktime_get();
 
 	return 0;
 }
@@ -292,6 +294,24 @@ snd_bcm2835_pcm_pointer(struct snd_pcm_substream *substream)
 {
 	struct snd_pcm_runtime *runtime = substream->runtime;
 	struct bcm2835_alsa_stream *alsa_stream = runtime->private_data;
+	ktime_t now = ktime_get();
+
+	/* Give userspace better delay reporting by interpolating between GPU
+	 * notifications, assuming audio speed is close enough to the clock
+	 * used for ktime
+	 */
+
+	if ((ktime_to_ns(alsa_stream->interpolate_start)) &&
+	    (ktime_compare(alsa_stream->interpolate_start, now) < 0)) {
+		u64 interval =
+			(ktime_to_ns(ktime_sub(now,
+				alsa_stream->interpolate_start)));
+		u64 frames_output_in_interval =
+			div_u64((interval * runtime->rate), 1000000000);
+		snd_pcm_sframes_t frames_output_in_interval_sized =
+			-frames_output_in_interval;
+		runtime->delay = frames_output_in_interval_sized;
+	}
 
 	return snd_pcm_indirect_playback_pointer(substream,
 		&alsa_stream->pcm_indirect,
diff --git a/drivers/staging/vc04_services/bcm2835-audio/bcm2835.h b/drivers/staging/vc04_services/bcm2835-audio/bcm2835.h
index e13435d1c205..595ad584243f 100644
--- a/drivers/staging/vc04_services/bcm2835-audio/bcm2835.h
+++ b/drivers/staging/vc04_services/bcm2835-audio/bcm2835.h
@@ -78,6 +78,7 @@ struct bcm2835_alsa_stream {
 	unsigned int period_offset;
 	unsigned int buffer_size;
 	unsigned int period_size;
+	ktime_t interpolate_start;
 
 	struct bcm2835_audio_instance *instance;
 	int idx;
-- 
2.17.1


  parent reply	other threads:[~2018-10-22 19:17 UTC|newest]

Thread overview: 54+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-18 10:57 [PATCH] staging: bcm2835-audio: interpolate audio delay Mike Brady
2018-10-18 10:57 ` Mike Brady
2018-10-18 11:02 ` Takashi Iwai
2018-10-18 11:02   ` Takashi Iwai
2018-10-18 19:48 ` Kirill Marinushkin
2018-10-18 19:48   ` Kirill Marinushkin
2018-10-22 19:08   ` Mike Brady
2018-10-22 19:08     ` Mike Brady
2018-10-22 19:17 ` Mike Brady [this message]
2018-10-22 19:17   ` [PATCH v2] " Mike Brady
2018-10-22 19:17   ` Mike Brady
2018-10-22 22:25   ` Kirill Marinushkin
2018-10-22 22:25     ` Kirill Marinushkin
2018-10-24  8:20     ` Mike Brady
2018-10-24  8:20       ` [alsa-devel] " Mike Brady
2018-10-24  8:20       ` Mike Brady
2018-10-24 18:06       ` Kirill Marinushkin
2018-10-24 18:06         ` Kirill Marinushkin
2018-10-24 19:54         ` Mike Brady
2018-10-24 19:54           ` [alsa-devel] " Mike Brady
2018-10-24 19:54           ` Mike Brady
2018-10-24 22:02           ` Kirill Marinushkin
2018-10-24 22:02             ` Kirill Marinushkin
2018-10-25  7:37             ` Takashi Iwai
2018-10-25  7:37               ` [alsa-devel] " Takashi Iwai
2018-10-25  7:37               ` Takashi Iwai
2018-10-25 17:20               ` Kirill Marinushkin
2018-10-25 17:20                 ` Kirill Marinushkin
2018-10-28 14:24                 ` Mike Brady
2018-10-28 14:24                   ` [alsa-devel] " Mike Brady
2018-10-28 14:24                   ` Mike Brady
2018-10-28 14:26               ` Mike Brady
2018-10-28 14:26                 ` [alsa-devel] " Mike Brady
2018-10-28 14:26                 ` Mike Brady
2018-11-05 15:57                 ` Mike Brady
2018-11-05 15:57                   ` [alsa-devel] " Mike Brady
2018-11-05 15:57                   ` Mike Brady
2018-11-05 16:11                   ` Takashi Iwai
2018-11-05 16:11                     ` Takashi Iwai
2018-11-06 21:05                     ` Mike Brady
2018-11-06 21:05                       ` Mike Brady
2018-11-06 21:31                       ` Takashi Iwai
2018-11-06 21:31                         ` Takashi Iwai
2018-11-11 18:08                         ` Mike Brady
2018-11-11 18:21                           ` [PATCH v2] ARM: " Mike Brady
2018-11-11 18:08                           ` [alsa-devel] [PATCH v2] " Mike Brady
2018-11-11 20:18                           ` [PATCH v2] ARM: " Stefan Wahren
2018-11-11 20:18                             ` Stefan Wahren
2018-11-11 20:18                             ` Stefan Wahren
2018-11-13 16:50                           ` Takashi Iwai
2018-11-13 16:50                             ` Takashi Iwai
2019-01-01 10:02                             ` Mike Brady
2019-01-01 10:02                               ` Mike Brady
2019-01-01 10:02                               ` Mike Brady

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181022191708.GA4659@ubuntu \
    --to=mikebrady@eircom.net \
    --cc=alsa-devel@alsa-project.org \
    --cc=devel@driverdev.osuosl.org \
    --cc=eric@anholt.net \
    --cc=f.fainelli@gmail.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=julia.lawall@lip6.fr \
    --cc=k.marinushkin@gmail.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rpi-kernel@lists.infradead.org \
    --cc=nishka.dasgupta_ug18@ashoka.edu.in \
    --cc=stefan.wahren@i2se.com \
    --cc=tiwai@suse.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.