Re: [v4l-dvb-maintainer] bttv driver errors

Re: [v4l-dvb-maintainer] bttv driver errors

[Trent Piepho]

On Mon, 18 Aug 2008, Jean Delvare wrote:
>
> My initial guess was that the PCI bandwidth was insufficient, however
> a quick computation shows that the problems start far below the limit:
> (5cams*4cif*yuv420)=(5*640*480*30*1.5)=66 MB/s, only half of the PCI
> bandwidth. The vendor claims that the adapter should be able to capture

Analog video doesn't transmit at a constant rate.  There are empty spaces
between lines and a large delay between frames.

What's more, yuv420 works by sending one line at 2 bytes/pixel and the next
line at 1 byte/pixel.  If the bt878 had a large fifo, the bit rate might
average out, but it doesn't.  It's something microscopic like 128 bytes.

An NTSC line has active data for about 52.6 us out of 63.5 us.  For square
pixel (640x480) it should be 52.15 and R.601 should be 53.33, but I won't
count on the bttv driver getting that right.

640 pixels/line * 2 bytes/pixel / 52.15 us/line = 24.5 bytes/us (MB/s)

With 5 cams you're over 122 MB/s.  Obviously the peak datarate is too high.

Even if it wasn't, you'd still want to do a latency calculation based on
the bt848 fifo size and trigger point.  The FIFO can overflow even well
below max PCI bandwidth.

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: bttv driver errors

[Trent Piepho]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=X-UNKNOWN, Size: 2407 bytes --]

On Wed, 20 Aug 2008, Jean Delvare wrote:

> Hi Trent,
>
> Le lundi 18 août 2008, Trent Piepho a écrit :
> > What's more, yuv420 works by sending one line at 2 bytes/pixel and the next
> > line at 1 byte/pixel.  If the bt878 had a large fifo, the bit rate might
> > average out, but it doesn't.  It's something microscopic like 128 bytes.
>
> You're right, it's (in practice) 128 bytes.
>
> While reading the BT878 datasheet to try to better understand how
> this all happens, I came to wonder how the chip can actually handle
> planar formats with vertical subsampling.
>
> To do vertical subsampling, you obviously need to handle several
> lines together (2 in the case of yuv420). However, the FIFO is too

2 is needed for simple averaging.  Higher quality requires even more lines
for a multi-tap FIR filter for some kind.

> small to contain a complete line of data, and there doesn't seem to
> be any RISC instruction for fetching chroma information back from
> memory either. This suggests that the BT878 is cheating on vertical
> subsampling, and instead of averaging on 2 lines (2x2 chroma
> subsampling), it averages on 1 line (2x1 chroma subsampling) and just
> skips the chroma information of the next line. Can you please confirm
> or infirm this? If I'm wrong then I would be grateful if you could
> explain how the BT878 achieves vertical subsampling.

4:2:0 YUV is achieved by setting the chip to 4:2:2 mode and then dropping
"every other" chroma line with RISC DMA program.

Note that every other line means something totally different if you are
talking about fields vs frames.  Historically bttv got this wrong for field
capture, and didn't take field dominance into account for frame capture.  I
don't know if it's been fixed or not.

I suppose one could drop the just the U samples for one line, then drop
just the V for the next line, to get a better average bit rate.

A better question would be how does the bt878 do horizontal and vertical
scaling?  If you look at the description of ultralock and the number of
taps avaiable for the vertical scaling filters, the chip must have some
kind of multi-line buffer before the scaler.  But this buffer, and the
delay it must introduce, is never mentioned in the datasheet.

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: bttv driver errors

[Trent Piepho]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=X-UNKNOWN, Size: 3455 bytes --]

On Thu, 21 Aug 2008, Jean Delvare wrote:
> Le jeudi 21 août 2008, Trent Piepho a écrit :
> > On Wed, 20 Aug 2008, Jean Delvare wrote:
> > > While reading the BT878 datasheet to try to better understand how
> > > this all happens, I came to wonder how the chip can actually handle
> > > planar formats with vertical subsampling.
> > >
> > > To do vertical subsampling, you obviously need to handle several
> > > lines together (2 in the case of yuv420). However, the FIFO is too
> >
> > 2 is needed for simple averaging.  Higher quality requires even more lines
> > for a multi-tap FIR filter for some kind.
> >
> > > small to contain a complete line of data, and there doesn't seem to
> > > be any RISC instruction for fetching chroma information back from
> > > memory either. This suggests that the BT878 is cheating on vertical
> > > subsampling, and instead of averaging on 2 lines (2x2 chroma
> > > subsampling), it averages on 1 line (2x1 chroma subsampling) and just
> > > skips the chroma information of the next line. Can you please confirm
> > > or infirm this? If I'm wrong then I would be grateful if you could
> > > explain how the BT878 achieves vertical subsampling.
> >
> > 4:2:0 YUV is achieved by setting the chip to 4:2:2 mode and then dropping
> > "every other" chroma line with RISC DMA program.
>
> OK. So you agree with me that this is approximative and not "real"
> vertical subsampling. I'm curious if more recent video chips are

Depends on what you mean by "real".  Think of the top line of the chroma as
x(0), the line below it as x(1), the next line x(2) and so on.  The line
above would be x(-1), then x(-2), etc.

It sounds like you think the formula for the subsampled chroma should be:
0.5 * x(0) + 0.5 * x(1)

And this is a pretty common way to do it.  The formula the bt878 uses is:
1.0 * x(0)

That's also a perfectly valid and real formula to use, though not a
particularly good one.  You could also use something like:
0.125 * x(-1) + 0.375 * x(0) + 0.375 * x(1) + 0.125 * x(2)

> > I suppose one could drop the just the U samples for one line, then drop
> > just the V for the next line, to get a better average bit rate.
>
> This would make some sense indeed, but I suspect that the BT878
> doesn't actually do this. Looking at the available RISC instructions,
> I see an instruction copying Y data from FIFO to RAM and skipping U
> and V data, and another instruction copying all of Y, U and V data
> from FIFO to RAM. But no instruction copying Y and U or Y and V. So

You're right, I forgot about that.  There is only WRITE123 and WRITE1S23,
no WRITE12S3.

You might try switching to packed mode instead of planar.  In that mode the
three FIFOs are merged.  Another thing I've heard about is using shorter
write commands.  When the bt878 processes a WRITE123, it doesn't write one
pixel of Y, one pixel of U, one pixel of V, then the next Y, the next V,
etc.  What it does is write all the Y, then all the U, then all the V.  So
while all the Y data is getting written, the U and V FIFOs are still
getting filled.  What that means is the latency for the V FIFO is the time
it took the bt878 to be granted bus access, plus the time it takes to empty
the Y FIFO (or finish the current RISC instruction), plus the time it takes
to empty the U FIFO.

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: [v4l-dvb-maintainer] bttv driver errors

[Andy Walls]

On Thu, 2008-08-21 at 15:50 -0700, Trent Piepho wrote:
> On Thu, 21 Aug 2008, Jean Delvare wrote:
> > Le jeudi 21 aot 2008, Trent Piepho a crit:
> > > On Wed, 20 Aug 2008, Jean Delvare wrote:

> > > 4:2:0 YUV is achieved by setting the chip to 4:2:2 mode and then dropping
> > > "every other" chroma line with RISC DMA program.
> >
> > OK. So you agree with me that this is approximative and not "real"
> > vertical subsampling. I'm curious if more recent video chips are

Strictly speaking subsampling happens at the A to D conversion.  Follow
on processes are filtering and, in this case, decimation.


> Depends on what you mean by "real".  Think of the top line of the chroma as
> x(0), the line below it as x(1), the next line x(2) and so on.  The line
> above would be x(-1), then x(-2), etc.
> 
> It sounds like you think the formula for the subsampled chroma should be:
> 0.5 * x(0) + 0.5 * x(1)

That's a short moving average (low pass) filter on the 4:2:2 subsampled
chroma prior to decimation.  It may have the effect of removing the
higher spatial frequencies in the chroma, but so does the decimation
(with the required anti-aliasing filter after re interpolation later).

>From a signal processing perspective, I wouldn't expect a low pass
filter (i.e. the anti-alias filter) to be employed until after the
interpolation to reconstruct the chroma signals for display.



> And this is a pretty common way to do it.  The formula the bt878 uses is:
> 1.0 * x(0)

This is straight decimation, which is what I would think is the "real"
way to do things - for non-video signals at least.  It limits the
highest spatial frequency that can be accurately recovered for the
chroma.


> That's also a perfectly valid and real formula to use, though not a
> particularly good one.

What are the measures you are using to make a good/bad declaration?  If
only the complexity of a capture implementation is the measure, then
straight decimation is the best, I would think.


>   You could also use something like:
> 0.125 * x(-1) + 0.375 * x(0) + 0.375 * x(1) + 0.125 * x(2)

(Another type of moving average filter before decimation.)


What are the benefits of using the moving average filters before
decimation?  


Regards,
Andy


--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: [v4l-dvb-maintainer] bttv driver errors

[Andy Walls]

On Fri, 2008-08-22 at 08:26 -0400, Andy Walls wrote:
> On Thu, 2008-08-21 at 15:50 -0700, Trent Piepho wrote:
> > On Thu, 21 Aug 2008, Jean Delvare wrote:
> > > Le jeudi 21 aot 2008, Trent Piepho a crit:
> > > > On Wed, 20 Aug 2008, Jean Delvare wrote:
> 
> > > > 4:2:0 YUV is achieved by setting the chip to 4:2:2 mode and then dropping
> > > > "every other" chroma line with RISC DMA program.
> > >
> > > OK. So you agree with me that this is approximative and not "real"
> > > vertical subsampling. I'm curious if more recent video chips are
> 
> Strictly speaking subsampling happens at the A to D conversion.  Follow
> on processes are filtering and, in this case, decimation.

Correcting myself on this point: "undersampling" happens at the A to D
conversion.  Sorry for the misstatement.

Regards,
Andy

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: [v4l-dvb-maintainer] bttv driver errors

[Daniel Glöckner]

On Fri, Aug 22, 2008 at 08:26:34AM -0400, Andy Walls wrote:
> What are the benefits of using the moving average filters before
> decimation?

The benefit is that the upper spectrum is attenuated before it is
mirrored into the lower spectrum by the decimation. After the decimation
you have no way of removing those artifacts in a mathematically correct
way.

There are better FIR filters than 0.5 * x(0) + 0.5 * x(1), which will
avoid aliasing even more and preserve more of the lower spectrum.
This is just the simplest.

Btw., enabling the chroma comb filter in bttv will result in alternating
0.5*x(-2)+0.5*x(0) and 0.5*x(-1)+0.5*x(1) for frame captures and
0.5*x(-1)+0.5*x(0) for field captures. I have once tried to modify
bttv_risc_planar to be closer to MPEG chroma subsampling when the comb
filter is enabled, because I was annoyed by the gray first line.

On Thu, Aug 21, 2008 at 01:50:05AM -0700, Trent Piepho wrote:
> A better question would be how does the bt878 do horizontal and vertical
> scaling?  If you look at the description of ultralock and the number of
> taps avaiable for the vertical scaling filters, the chip must have some
> kind of multi-line buffer before the scaler.  But this buffer, and the
> delay it must introduce, is never mentioned in the datasheet.

The buffer is "mentioned" in figure 2-4 of the Fusion 878A datasheet
available on the Conexant website. Its size can be derived from the
restrictions placed on the luma filters. It should be 768 luma samples.
The decimation then has another full line buffer for luma and chroma
to perform the linear luma interpolation and the chroma comb filter.

  Daniel

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: [v4l-dvb-maintainer] bttv driver errors

[Andy Walls]

On Fri, 2008-08-22 at 22:32 +0200, Daniel Glöckner wrote:
> On Fri, Aug 22, 2008 at 08:26:34AM -0400, Andy Walls wrote:
> > What are the benefits of using the moving average filters before
> > decimation?
> 
> The benefit is that the upper spectrum is attenuated before it is
> mirrored into the lower spectrum by the decimation. After the decimation
> you have no way of removing those artifacts in a mathematically correct
> way.

Ah, of course. Thank you.  

I should have known that. :P  I didn't mentally connect decimation with
pulling the alias spectra in closer to overlap the main spectrum.


Regards,
Andy


> There are better FIR filters than 0.5 * x(0) + 0.5 * x(1), which will
> avoid aliasing even more and preserve more of the lower spectrum.
> This is just the simplest.
> 
> Btw., enabling the chroma comb filter in bttv will result in alternating
> 0.5*x(-2)+0.5*x(0) and 0.5*x(-1)+0.5*x(1) for frame captures and
> 0.5*x(-1)+0.5*x(0) for field captures. I have once tried to modify
> bttv_risc_planar to be closer to MPEG chroma subsampling when the comb
> filter is enabled, because I was annoyed by the gray first line.
> 
> On Thu, Aug 21, 2008 at 01:50:05AM -0700, Trent Piepho wrote:
> > A better question would be how does the bt878 do horizontal and vertical
> > scaling?  If you look at the description of ultralock and the number of
> > taps avaiable for the vertical scaling filters, the chip must have some
> > kind of multi-line buffer before the scaler.  But this buffer, and the
> > delay it must introduce, is never mentioned in the datasheet.
> 
> The buffer is "mentioned" in figure 2-4 of the Fusion 878A datasheet
> available on the Conexant website. Its size can be derived from the
> restrictions placed on the luma filters. It should be 768 luma samples.
> The decimation then has another full line buffer for luma and chroma
> to perform the linear luma interpolation and the chroma comb filter.
> 
>   Daniel
> 
> _______________________________________________
> v4l-dvb-maintainer mailing list
> v4l-dvb-maintainer@linuxtv.org
> http://www.linuxtv.org/cgi-bin/mailman/listinfo/v4l-dvb-maintainer
> 

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: [v4l-dvb-maintainer] bttv driver errors

[Trent Piepho]

On Fri, 22 Aug 2008, Andy Walls wrote:
> On Thu, 2008-08-21 at 15:50 -0700, Trent Piepho wrote:
> > And this is a pretty common way to do it.  The formula the bt878 uses is:
> > 1.0 * x(0)
>
> This is straight decimation, which is what I would think is the "real"
> way to do things - for non-video signals at least.  It limits the
> highest spatial frequency that can be accurately recovered for the
> chroma.

You have to do a low-pass filter before decimation, otherwise the high
frequencies will alias.

I think the mathematically correct filter would be a sinc function with an
infinite base, though in practice such sinc filters can produce undesirable
ringing effects.

> > That's also a perfectly valid and real formula to use, though not a
> > particularly good one.
>
> What are the measures you are using to make a good/bad declaration?  If
> only the complexity of a capture implementation is the measure, then
> straight decimation is the best, I would think.

Quality.  For example, see
http://www.glennchan.info/articles/technical/chroma/chroma1.htm

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list