How to deal with cards with a large on-board buffer?

How to deal with cards with a large on-board buffer?

[Nicolas Boullis]

Hi,

I'm trying to write an ALSA interface for the audio part of MPEG2 
decoder boards Sigma Designs Hollywod+ and Creative Labs DXR3, both 
based on Sigma Designs EM8300 chip.

To make it short, the trouble is that, to get correct working, it seems 
to require that the audio buffer be kept at some minimal fill level. Is 
there a way I can enforce this?

Browsing at the code, I found out the fifo_size element of the 
snd_pcm_hardware_t type, that is not documented in Takashi Iwai's 
excellent "Writing an ALSA Driver" guide. Is it something that might be 
used to enforce this? (I tried, of course, but my test was 
unsuccessful...)

I also discovered the SNDRV_PCM_INFO_DOUBLE, SNDRV_PCM_INFO_BATCH 
and SNDRV_PCM_INFO_BLOCK_TRANSFER that are give little information, and 
that sound to me like things that might be useful. Any explanation about 
those?


Now, just in case someone has a better idea how to make things work, 
here is a more comprehensive explanation about how those cards work, *as 
far as I could understand* (since I got absolutely no explanation from 
the manufacturer):
   - there is a rather large on-chip buffer (0x3c00 bytes), that we can 
     neither write to or read directly
   - we can read pointers to the current read and write positions in 
     this buffer
   - this buffer is fed by chip-initiated DMA transfers from the main 
     memory
   - the buffers for DMA transfers are declared in a FIFO; they mustn't 
     be too large (8kB is ok; 16kB is not since it's larger that the 
     on-chip buffer)
   - if the on-chip buffer is "too empty" (don't know exactly when the 
     chip decides it is too empty) and no buffer is available for 
     DMA transfer, it "sends" some junk to the internal buffer
   - interruptions are not raised when buffers are DMA-transfered, but 
     at fixed 20.2ms interval (stable experimental value)
   - it might be required that DMA buffers are aligned on 4kB boundaried 
     (or page boundaries)

Any idea how to correctly deal with such a beast? I tried a few ones, 
but none was really successful...


Cheers,

Nicolas Boullis


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: How to deal with cards with a large on-board buffer?

[Takashi Iwai]

At Wed, 3 May 2006 01:41:25 +0200,
Nicolas Boullis wrote:
> 
> Hi,
> 
> I'm trying to write an ALSA interface for the audio part of MPEG2 
> decoder boards Sigma Designs Hollywod+ and Creative Labs DXR3, both 
> based on Sigma Designs EM8300 chip.
> 
> To make it short, the trouble is that, to get correct working, it seems 
> to require that the audio buffer be kept at some minimal fill level. Is 
> there a way I can enforce this?
 
It's no way because the data is fed from the application.
If less than watermark (usually equivalent with period size) is
available, it's simply an underrun.

> Browsing at the code, I found out the fifo_size element of the 
> snd_pcm_hardware_t type, that is not documented in Takashi Iwai's 
> excellent "Writing an ALSA Driver" guide. Is it something that might be 
> used to enforce this? (I tried, of course, but my test was 
> unsuccessful...)

No, it's an obsoleted field and actually not used in the driver.

> I also discovered the SNDRV_PCM_INFO_DOUBLE, SNDRV_PCM_INFO_BATCH 
> and SNDRV_PCM_INFO_BLOCK_TRANSFER that are give little information, and 
> that sound to me like things that might be useful. Any explanation about 
> those?

INFO_DOUBLE is just a hint to user-space and not used in the driver.
INFO_BATCH and INFO_BLOCK_TRANSFER are evaluated only in the OSS
emulation code.  OSS DSP_CAP_REALTIME bit is reset when either of them
is set.  But no influence on the behavior of ALSA native apps.


> Now, just in case someone has a better idea how to make things work, 
> here is a more comprehensive explanation about how those cards work, *as 
> far as I could understand* (since I got absolutely no explanation from 
> the manufacturer):
>    - there is a rather large on-chip buffer (0x3c00 bytes), that we can 
>      neither write to or read directly
>    - we can read pointers to the current read and write positions in 
>      this buffer
>    - this buffer is fed by chip-initiated DMA transfers from the main 
>      memory
>    - the buffers for DMA transfers are declared in a FIFO; they mustn't 
>      be too large (8kB is ok; 16kB is not since it's larger that the 
>      on-chip buffer)
>    - if the on-chip buffer is "too empty" (don't know exactly when the 
>      chip decides it is too empty) and no buffer is available for 
>      DMA transfer, it "sends" some junk to the internal buffer
>    - interruptions are not raised when buffers are DMA-transfered, but 
>      at fixed 20.2ms interval (stable experimental value)
>    - it might be required that DMA buffers are aligned on 4kB boundaried 
>      (or page boundaries)
> 
> Any idea how to correctly deal with such a beast? I tried a few ones, 
> but none was really successful...

Are you using a system timer or any dedicated irq source?

The first thing to do is to find out the bottleneck.  You have two
transfers running asynchronously: on the hardware buffer and between
memory <-> h/w buffer.  I guess the former is running continuously,
but how the latter transfer is implemented?

Anyway, the source code would reveal problems better.


Takashi


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: How to deal with cards with a large on-board buffer?

[Clemens Ladisch]

Nicolas Boullis wrote:
> I'm trying to write an ALSA interface for the audio part of MPEG2 
> decoder boards Sigma Designs Hollywod+ and Creative Labs DXR3, both 
> based on Sigma Designs EM8300 chip.
> 
> To make it short, the trouble is that, to get correct working, it seems 
> to require that the audio buffer be kept at some minimal fill level. Is 
> there a way I can enforce this?

The card should do this automatically as it initiates the DMA transfers
by itself.

> Browsing at the code, I found out the fifo_size element of the 
> snd_pcm_hardware_t type, that is not documented

because it isn't used.

> I also discovered the SNDRV_PCM_INFO_DOUBLE, SNDRV_PCM_INFO_BATCH 
> and SNDRV_PCM_INFO_BLOCK_TRANSFER that are give little information, and 
> that sound to me like things that might be useful.

These are just informational flags that describe how the driver does
things.

> Now, just in case someone has a better idea how to make things work, 
> here is a more comprehensive explanation about how those cards work, *as 
> far as I could understand* (since I got absolutely no explanation from 
> the manufacturer):
>    - there is a rather large on-chip buffer (0x3c00 bytes), that we can 
>      neither write to or read directly

So it's just a FIFO that is more or less transparent to the computer?

>    - we can read pointers to the current read and write positions in 
>      this buffer
>    - this buffer is fed by chip-initiated DMA transfers from the main 
>      memory

How big are these transfers?

>    - the buffers for DMA transfers are declared in a FIFO;

You mean this chip supports scatter/gather buffers?

>    - if the on-chip buffer is "too empty" (don't know exactly when the 
>      chip decides it is too empty) and no buffer is available for 
>      DMA transfer, it "sends" some junk to the internal buffer

Most sound card FIFOs operate like this.

>    - interruptions are not raised when buffers are DMA-transfered, but 
>      at fixed 20.2ms interval (stable experimental value)

The ymfpci and usb drivers deal with a similar problem.


Regards,
Clemens


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: How to deal with cards with a large on-board buffer?

[Nicolas Boullis]

Hi,

On Wed, May 03, 2006 at 02:26:34PM +0200, Takashi Iwai wrote:
> At Wed, 3 May 2006 01:41:25 +0200, Nicolas Boullis wrote:
> > 
> > To make it short, the trouble is that, to get correct working, it seems 
> > to require that the audio buffer be kept at some minimal fill level. Is 
> > there a way I can enforce this?
>  
> It's no way because the data is fed from the application.
> If less than watermark (usually equivalent with period size) is
> available, it's simply an underrun.

Do you mean that userspace applications are expected to always have a 
period worth of data available? Then I guess I should force the period 
size to be larger than the buffer size...


> > Browsing at the code, I found out the fifo_size element of the 
> > snd_pcm_hardware_t type, that is not documented in Takashi Iwai's 
> > excellent "Writing an ALSA Driver" guide. Is it something that might be 
> > used to enforce this? (I tried, of course, but my test was 
> > unsuccessful...)
> 
> No, it's an obsoleted field and actually not used in the driver.

OK, I see.


> > Any idea how to correctly deal with such a beast? I tried a few ones, 
> > but none was really successful...
> 
> Are you using a system timer or any dedicated irq source?

It's a dedicated irq source for the chip.


> The first thing to do is to find out the bottleneck.

Sorry, but what bottleneck?


> You have two
> transfers running asynchronously: on the hardware buffer and between
> memory <-> h/w buffer.  I guess the former is running continuously,

You're right.


> but how the latter transfer is implemented?

The CPU-side mmaps some on-chip memory. Within this memory, there is a 
32-slot circular buffer, used with a "read pointer" and a "write 
pointer" to implement a FIFO. The CPU-side only updates the write 
pointer, and writes to the buffer. The elements in the buffer are 
pointers to and size of buffers that contain the actual data, and that 
are meant to be trasfered to the hardware buffer. The chip itself 
decides when to transfer a buffer, and updates the "read pointer".


> Anyway, the source code would reveal problems better.

Of course. You can grab the relevant part of the source code at 
http://dxr3.sourceforge.net/download/em8300_alsa.c but you should be 
warned that the code is not really clean, and that I'm somewhat ashamed 
to show it.

If you're willing to help me, feel free to ask any question about my 
code. Some explanation anyway: I wanted to avoid memory transfers 
within the driver, so I though I would allocate a large buffer, split it 
in 32 chunks, and set up the DMA slots each to one of those buffers. 
Whenever an interrupt is raised, I set the "write pointer" such that it 
considers the FIFO to be full.


Thanks,

Nicolas


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: How to deal with cards with a large on-board buffer?

[Takashi Iwai]

At Thu, 4 May 2006 03:18:06 +0200,
Nicolas Boullis wrote:
> 
> Hi,
> 
> On Wed, May 03, 2006 at 02:26:34PM +0200, Takashi Iwai wrote:
> > At Wed, 3 May 2006 01:41:25 +0200, Nicolas Boullis wrote:
> > > 
> > > To make it short, the trouble is that, to get correct working, it seems 
> > > to require that the audio buffer be kept at some minimal fill level. Is 
> > > there a way I can enforce this?
> >  
> > It's no way because the data is fed from the application.
> > If less than watermark (usually equivalent with period size) is
> > available, it's simply an underrun.
> 
> Do you mean that userspace applications are expected to always have a 
> period worth of data available? Then I guess I should force the period 
> size to be larger than the buffer size...

It's controlled via avail_min of snd_pcm_sw_params struct, and it's
set to period_size as default.  So, in most cases, it's guaranteed
that the data is more than period_size -- or falls into an underrun.

In theory the driver can refuses the wrong request by apps (avail_min
< period_size), but currently there is no callback to the driver for
sw_params.

I'd suggest to leave as it is right now.  If underruns occur too
often, something is wrong in another place, e.g. the timing of
transfer between HOST and on-chip memories.


Takashi


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: How to deal with cards with a large on-board buffer?

[Nicolas Boullis]

On Wed, May 03, 2006 at 02:26:58PM +0200, Clemens Ladisch wrote:
> > 
> > To make it short, the trouble is that, to get correct working, it seems 
> > to require that the audio buffer be kept at some minimal fill level. Is 
> > there a way I can enforce this?
> 
> The card should do this automatically as it initiates the DMA transfers
> by itself.

It does. But then I have problems if the transfered buffer has not been 
filled by the application...


> > Now, just in case someone has a better idea how to make things work, 
> > here is a more comprehensive explanation about how those cards work, *as 
> > far as I could understand* (since I got absolutely no explanation from 
> > the manufacturer):
> >    - there is a rather large on-chip buffer (0x3c00 bytes), that we can 
> >      neither write to or read directly
> 
> So it's just a FIFO that is more or less transparent to the computer?

I guess so.

> >    - we can read pointers to the current read and write positions in 
> >      this buffer
> >    - this buffer is fed by chip-initiated DMA transfers from the main 
> >      memory
> 
> How big are these transfers?

In the current code, they are 4kB (1ksample) or 8kB (2ksample). I can 
specify the size I want, but it does not work with large buffers (8kB is 
already more than half the size of the on-chip buffer). And it seems 
that there are problems when the buffers are not 4kB-aligned.


> >    - the buffers for DMA transfers are declared in a FIFO;
> 
> You mean this chip supports scatter/gather buffers?

I'm not sure what you mean exactly by scatter/gather buffers, but I 
guess it is something like that.


> >    - if the on-chip buffer is "too empty" (don't know exactly when the 
> >      chip decides it is too empty) and no buffer is available for 
> >      DMA transfer, it "sends" some junk to the internal buffer
> 
> Most sound card FIFOs operate like this.

OK, I did not know.


> >    - interruptions are not raised when buffers are DMA-transfered, but 
> >      at fixed 20.2ms interval (stable experimental value)
> 
> The ymfpci and usb drivers deal with a similar problem.

I'll try to give them a look.
But if I understand things correctly, I must ensure that the periods are 
large enough to be played in more than 20.2ms, if I want 
snd_pcm_period_elapsed to be called often enough. Is that right?


Cheers,

Nicolas

PS: sorry if I'm being stupid, I'm not used to writing audio drivers...


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: How to deal with cards with a large on-board buffer?

[Nicolas Boullis]

On Thu, May 04, 2006 at 04:49:43PM +0200, Takashi Iwai wrote:
> 
> I'd suggest to leave as it is right now.  If underruns occur too
> often, something is wrong in another place, e.g. the timing of
> transfer between HOST and on-chip memories.

Thre certainly is some problem... ;-)
The trouble is that I don't know how to fix them, and I don't even 
really understand what happens...

Have you tried to give a look to my code?


Regards,

Nicolas


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: How to deal with cards with a large on-board buffer?

[Takashi Iwai]

At Fri, 5 May 2006 01:56:53 +0200,
Nicolas Boullis wrote:
> 
> On Thu, May 04, 2006 at 04:49:43PM +0200, Takashi Iwai wrote:
> > 
> > I'd suggest to leave as it is right now.  If underruns occur too
> > often, something is wrong in another place, e.g. the timing of
> > transfer between HOST and on-chip memories.
> 
> Thre certainly is some problem... ;-)
> The trouble is that I don't know how to fix them, and I don't even 
> really understand what happens...

What is the symptom?

> Have you tried to give a look to my code?

Not yet.  I couldn't access to the web page.


Takashi


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642