Re: Audio and Video sync. Getting Timestamps

Re: Audio and Video sync. Getting Timestamps

[granda]

Mensaje citado por Takashi Iwai <tiwai@suse.de>:

> At Tue, 27 Apr 2004 20:36:29 +0200,
> Juan Carlos Granda wrote:
> > 
> > [1  <text/plain; iso-8859-1 (quoted-printable)>]
> > 
> > [2  <text/html; iso-8859-1 (quoted-printable)>]
> > Hi,
> >  
> > I'm developing a Multi-thread app for capturing audio+video and i have a
> few
> > questions. The app is based on 3 threads:
> >  
> >  - 1 thread for capturing audio using ALSA lib.
> >  - 1 thread for capturing video using V4L2.
> >  - 1 thread for saving audio+video into an AVI file.
> >  
> > Both of capturing threads store captured data into a queue and the last
> thread picks
> > the data and sync A/V.
> >  
> > I start audio capturing and then the video. So, i need to known the point
> of the
> > stream of audio the video begins.
> > I'm readings frames by period size times. My prob is how to get valid ALSA
> timestamps
> > to eliminate the previous frames of the period the video starts. I've
> tried
> > snd_pcm_status_get_trigger_tstamp but it returns the same timestamp for a
> long period
> > of time (13 secs) (is it related to the period or buffer time?). I've
> tried
> > snd_pcm_status_get_tstamp too but i obtain a delay of several microseconds
> from the
> > period time (50 - 100 more or less) (is it because the clock resolution?).
> 
> the alsa-driver can put timestamp at each period update.
> set SND_TSTAMP_MMAP in snd_pcm_sw_params_set_tstamp_mode().
> 
> 
> Takashi
> 
> 

I set SND_TSTAMP_MMAP like you told me. What function may i use? I've tried
snd_pcm_status_get_trigger_tstamp and snd_pcm_status_get_tstamp, and the first
always return the same timestamp or it changes every 4 or 6 seconds. And the
second returns the "now" timestamp. What am i doing wrong?

Could you explain me the all secuence to make it works?

Thanl, Juan Carlos


> -------------------------------------------------------
> This SF.Net email is sponsored by: Oracle 10g
> Get certified on the hottest thing ever to hit the market... Oracle 10g. 
> Take an Oracle 10g class now, and we'll give you the exam FREE.
> http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
> _______________________________________________
> Alsa-devel mailing list
> Alsa-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/alsa-devel
> 




-------------------------------------------------
Este mensaje fue enviado
utilizando el servicio Webmail de:
http://www.telecable.es


-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g. 
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click

Re: Audio and Video sync. Getting Timestamps

[Takashi Iwai]

At Wed,  5 May 2004 11:13:20 +0200,
"" <granda@telecable.es> wrote:
> 
> Mensaje citado por Takashi Iwai <tiwai@suse.de>:
> 
> > At Tue, 27 Apr 2004 20:36:29 +0200,
> > Juan Carlos Granda wrote:
> > > 
> > > [1  <text/plain; iso-8859-1 (quoted-printable)>]
> > > 
> > > [2  <text/html; iso-8859-1 (quoted-printable)>]
> > > Hi,
> > >  
> > > I'm developing a Multi-thread app for capturing audio+video and i have a
> > few
> > > questions. The app is based on 3 threads:
> > >  
> > >  - 1 thread for capturing audio using ALSA lib.
> > >  - 1 thread for capturing video using V4L2.
> > >  - 1 thread for saving audio+video into an AVI file.
> > >  
> > > Both of capturing threads store captured data into a queue and the last
> > thread picks
> > > the data and sync A/V.
> > >  
> > > I start audio capturing and then the video. So, i need to known the point
> > of the
> > > stream of audio the video begins.
> > > I'm readings frames by period size times. My prob is how to get valid ALSA
> > timestamps
> > > to eliminate the previous frames of the period the video starts. I've
> > tried
> > > snd_pcm_status_get_trigger_tstamp but it returns the same timestamp for a
> > long period
> > > of time (13 secs) (is it related to the period or buffer time?). I've
> > tried
> > > snd_pcm_status_get_tstamp too but i obtain a delay of several microseconds
> > from the
> > > period time (50 - 100 more or less) (is it because the clock resolution?).
> > 
> > the alsa-driver can put timestamp at each period update.
> > set SND_TSTAMP_MMAP in snd_pcm_sw_params_set_tstamp_mode().
> > 
> > 
> > Takashi
> > 
> > 
> 
> I set SND_TSTAMP_MMAP like you told me. What function may i use? I've tried
> snd_pcm_status_get_trigger_tstamp and snd_pcm_status_get_tstamp, and the first
> always return the same timestamp or it changes every 4 or 6 seconds. And the
> second returns the "now" timestamp. What am i doing wrong?

no, you're not wrong, i overlooked that the SND_PCM_STATUS ioctl calls
the update of timestamp.  so, apparently there is no this timestamp
mode for the period-updates.

but, anyway, you can get the sound frame position and the timestamp at
the same time.  it should suffice to calculate the sync, i believe.


Takashi


-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g. 
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click

Re: Audio and Video sync. Getting Timestamps

[Jaroslav Kysela]

On Wed, 5 May 2004, Takashi Iwai wrote:

> > I set SND_TSTAMP_MMAP like you told me. What function may i use? I've tried
> > snd_pcm_status_get_trigger_tstamp and snd_pcm_status_get_tstamp, and the first
> > always return the same timestamp or it changes every 4 or 6 seconds. And the
> > second returns the "now" timestamp. What am i doing wrong?
> 
> no, you're not wrong, i overlooked that the SND_PCM_STATUS ioctl calls
> the update of timestamp.  so, apparently there is no this timestamp
> mode for the period-updates.
> 
> but, anyway, you can get the sound frame position and the timestamp at
> the same time.  it should suffice to calculate the sync, i believe.

That's right. The semantics is - mark timestamp when the stream pointer
is updated - not at period boundary (which can be shifted with interrupt 
latency issues).

						Jaroslav

-----
Jaroslav Kysela <perex@suse.cz>
Linux Kernel Sound Maintainer
ALSA Project, SuSE Labs


-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g. 
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click

Re: Audio and Video sync. Getting Timestamps

[Juan Carlos Granda]

That's my app does:

1.- Open the device for capture
2.- Set the access mode SND_PCM_ACCESS_RW_INTERLEAVED
3.- Set format 16 bits (SND_PCM_FORMAT_S16_LE)
4.- Set channels 2 (stereo)
5.- Set buffer time near 1 second
6.- Set period time near 0.1 seconds
7.- Copy the hardware params to alsa
8.- Set start threshold (0x7fffffffff -> Explicit start)
9.- Set transfer align 1
10.- Copy the software params to alsa

read loop:
    - Read 'period size' frames (snd_pcm_readi)

How can i get the timestamp of the first frame read? If i use
snd_pcm_status_get_trigger_tstamp i obtain the same timestamp all the times.
If i use snd_pcm_status_get_tstamp i obtain the "now" time. Is there any way
to get the timestamp of every period.

Thanks






-------------------------------------------------------
This SF.Net email is sponsored by Sleepycat Software
Learn developer strategies Cisco, Motorola, Ericsson & Lucent use to deliver
higher performing products faster, at low TCO.
http://www.sleepycat.com/telcomwpreg.php?From=osdnemail3

Re: Re: Audio and Video sync. Getting Timestamps

[James Courtier-Dutton]

Juan Carlos Granda wrote:
> That's my app does:
> 
> 1.- Open the device for capture
> 2.- Set the access mode SND_PCM_ACCESS_RW_INTERLEAVED
> 3.- Set format 16 bits (SND_PCM_FORMAT_S16_LE)
> 4.- Set channels 2 (stereo)
> 5.- Set buffer time near 1 second
> 6.- Set period time near 0.1 seconds
> 7.- Copy the hardware params to alsa
> 8.- Set start threshold (0x7fffffffff -> Explicit start)
> 9.- Set transfer align 1
> 10.- Copy the software params to alsa
> 
> read loop:
>     - Read 'period size' frames (snd_pcm_readi)
> 
> How can i get the timestamp of the first frame read? If i use
> snd_pcm_status_get_trigger_tstamp i obtain the same timestamp all the times.
> If i use snd_pcm_status_get_tstamp i obtain the "now" time. Is there any way
> to get the timestamp of every period.
> 
> Thanks
> 
> 

I help develope xinehq.de.
We have needed to do audio, video sync, but we don't need any audio 
timestamps using the get_trigger_tstamp etc.
We just use the snd_pcm_delay() call.

Take an example.
Sample rate 10000 Samples a second (10000 just to make things easy to 
explain.)

We call snd_pcm_delay() and it returns 4000.
4000 samples at a sample rate of 10000 = 400ms.
If we now write some samples to the sound device, we know that they will 
arrive at the speakers in 400ms time.

If we have some audio that needs to be played in 500ms time, we just 
insert 100 samples(100ms of samples) before sending our audio. We just 
know that those samples will reach the speakers in 500ms time.

In xine, we do more compilcated things like resampling, so that samples 
are removed and added without the user noticing.

So, for your application, if you just call snd_pcm_delay() just before 
you write your samples, or just after you write them, you will get a 
measure of when they will play.

Cheers
James


-------------------------------------------------------
This SF.Net email is sponsored by Sleepycat Software
Learn developer strategies Cisco, Motorola, Ericsson & Lucent use to deliver
higher performing products faster, at low TCO.
http://www.sleepycat.com/telcomwpreg.php?From=osdnemail3

Re: Audio and Video sync. Getting Timestamps

[Måns Rullgård]

James Courtier-Dutton <James@superbug.demon.co.uk> writes:

> Juan Carlos Granda wrote:
>> That's my app does:
>> 1.- Open the device for capture
>> 2.- Set the access mode SND_PCM_ACCESS_RW_INTERLEAVED
>> 3.- Set format 16 bits (SND_PCM_FORMAT_S16_LE)
>> 4.- Set channels 2 (stereo)
>> 5.- Set buffer time near 1 second
>> 6.- Set period time near 0.1 seconds
>> 7.- Copy the hardware params to alsa
>> 8.- Set start threshold (0x7fffffffff -> Explicit start)
>> 9.- Set transfer align 1
>> 10.- Copy the software params to alsa
>> read loop:
>>     - Read 'period size' frames (snd_pcm_readi)
>> How can i get the timestamp of the first frame read? If i use
>> snd_pcm_status_get_trigger_tstamp i obtain the same timestamp all the times.
>> If i use snd_pcm_status_get_tstamp i obtain the "now" time. Is there any way
>> to get the timestamp of every period.
>> Thanks
>>
>
> I help develope xinehq.de.
> We have needed to do audio, video sync, but we don't need any audio
> timestamps using the get_trigger_tstamp etc.
> We just use the snd_pcm_delay() call.
>
> Take an example.
> Sample rate 10000 Samples a second (10000 just to make things easy to
> explain.)
>
> We call snd_pcm_delay() and it returns 4000.
> 4000 samples at a sample rate of 10000 = 400ms.
> If we now write some samples to the sound device, we know that they
> will arrive at the speakers in 400ms time.

The OP was recording.

-- 
Måns Rullgård
mru@kth.se



-------------------------------------------------------
This SF.Net email is sponsored by Sleepycat Software
Learn developer strategies Cisco, Motorola, Ericsson & Lucent use to deliver
higher performing products faster, at low TCO.
http://www.sleepycat.com/telcomwpreg.php?From=osdnemail3

Re: Re: Audio and Video sync. Getting Timestamps

[James Courtier-Dutton]

Måns Rullgård wrote:
> James Courtier-Dutton <James@superbug.demon.co.uk> writes:
<snip>

> 
> The OP was recording.
> 

Oops. My explanation only covers playback, not capture.
But if you use the snd_pcm_avail_update() just before you capture some 
samples from the buffer, then do a gettimeofday(), you will get an 
accurate measure of when the samples arrived in the buffer.
If you also use a thread and get it to do poll(), you could also keep 
the latency down, with it calling the routine on each period elapsed.

Summary: Use a method similar to the playback method, but just modify it 
a be for record.

Cheers
James


-------------------------------------------------------
This SF.Net email is sponsored by Sleepycat Software
Learn developer strategies Cisco, Motorola, Ericsson & Lucent use to deliver
higher performing products faster, at low TCO.
http://www.sleepycat.com/telcomwpreg.php?From=osdnemail3

Audio and Video sync. Getting Timestamps

[Juan Carlos Granda]

[-- Attachment #1: Type: text/plain, Size: 1048 bytes --]

Hi,

I'm developing a Multi-thread app for capturing audio+video and i have a few questions. The app is based on 3 threads:

 - 1 thread for capturing audio using ALSA lib.
 - 1 thread for capturing video using V4L2.
 - 1 thread for saving audio+video into an AVI file.

Both of capturing threads store captured data into a queue and the last thread picks the data and sync A/V.

I start audio capturing and then the video. So, i need to known the point of the stream of audio the video begins. 
I'm readings frames by period size times. My prob is how to get valid ALSA timestamps to eliminate the previous frames of the period the video starts. I've tried snd_pcm_status_get_trigger_tstamp but it returns the same timestamp for a long period of time (13 secs) (is it related to the period or buffer time?). I've tried snd_pcm_status_get_tstamp too but i obtain a delay of several microseconds from the period time (50 - 100 more or less) (is it because the clock resolution?).

Thanks

pd. Sorry for my little knowledge of english.

[-- Attachment #2: Type: text/html, Size: 2114 bytes --]

Re: Audio and Video sync. Getting Timestamps

[Takashi Iwai]

At Tue, 27 Apr 2004 20:36:29 +0200,
Juan Carlos Granda wrote:
> 
> [1  <text/plain; iso-8859-1 (quoted-printable)>]
> 
> [2  <text/html; iso-8859-1 (quoted-printable)>]
> Hi,
>  
> I'm developing a Multi-thread app for capturing audio+video and i have a few
> questions. The app is based on 3 threads:
>  
>  - 1 thread for capturing audio using ALSA lib.
>  - 1 thread for capturing video using V4L2.
>  - 1 thread for saving audio+video into an AVI file.
>  
> Both of capturing threads store captured data into a queue and the last thread picks
> the data and sync A/V.
>  
> I start audio capturing and then the video. So, i need to known the point of the
> stream of audio the video begins.
> I'm readings frames by period size times. My prob is how to get valid ALSA timestamps
> to eliminate the previous frames of the period the video starts. I've tried
> snd_pcm_status_get_trigger_tstamp but it returns the same timestamp for a long period
> of time (13 secs) (is it related to the period or buffer time?). I've tried
> snd_pcm_status_get_tstamp too but i obtain a delay of several microseconds from the
> period time (50 - 100 more or less) (is it because the clock resolution?).

the alsa-driver can put timestamp at each period update.
set SND_TSTAMP_MMAP in snd_pcm_sw_params_set_tstamp_mode().


Takashi


-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g. 
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click