video4linux user land API concern

video4linux user land API concern

[Hubert Tonneau]

Any Video4Linux driver should support both native hardware color encoding
(for maximum performances) and rgb (for maximum flexibility).

Requiering user land tools to be prepared to match the webcam native color
encoding is poor kernel API design for several reasons:
. some software will say 'I will work only if your camera support the encoding
  I want' (surf the Internet and you will discover that it's already sadely true)
. small tools that make Unix flexibility get hard to write and test without
  relying on glue libraries, and glue libraries are evil
. if new color models appear in new cameras, the current design will require
  to map them anyway to some existing encodings not to break existing softwares,
  so the end result will be even more confusing because the driver supporting a
  non rgb encoding will not necessary mean that selecting the encoding is better
  from the performances point of view

Re: video4linux user land API concern

[Alan Cox]

On Mer, 2005-11-09 at 10:26 +0000, Hubert Tonneau wrote:
> Any Video4Linux driver should support both native hardware color encoding
> (for maximum performances) and rgb (for maximum flexibility).
> 
> Requiering user land tools to be prepared to match the webcam native color
> encoding is poor kernel API design for several reasons:

The kernel API was designed on the basis that someone would one day have
the sense to write a nice user space library of formats. 

> . if new color models appear in new cameras, the current design will require
>   to map them anyway to some existing encodings not to break existing softwares,
>   so the end result will be even more confusing because the driver supporting a
>   non rgb encoding will not necessary mean that selecting the encoding is better
>   from the performances point of view

Many of the encodings done by hardware are extremely complicated and
tricky to unpack in kernel space. If a camera captures jpeg for example
you don't want in kernel jpeg decoders, let alone mpeg decoders etc

Re: video4linux user land API concern

[Hubert Tonneau]

Alan Cox wrote:
>
> Many of the encodings done by hardware are extremely complicated and
> tricky to unpack in kernel space. If a camera captures jpeg for example
> you don't want in kernel jpeg decoders, let alone mpeg decoders etc

So you fall back on the current main bad point about the Linux kernel design.

One quality of Linux, as opposed to some other proposed kernels, is that all
drivers are part of the kernel tarball, so you can expect to always get drivers
that are consistent with the kernel internal API.

On the other hand, over time, more and more kernel only related features
have been rejected to user land, for various reasons mostly related to
stability and security. I mean 'ifconfig', raid tools, etc.
Your agument about video decoding falls in the stability category.

Now, if you look for documentation about the interface between the kernel
and the user land tool that have has been written at the same time to provide
an overall services (let think about software raid) then you discover that
it's mosty inexisting, and that the interface in the middle is just an
unengeneered nightmare.
So, in facts you can't use the kernel function without adopting the user
land tool associated, but then you discover that very fiew eyes watched and
checked the user land part of the tool, and that it's quality is really really
poor (opens a file, but not check the error code, etc, etc)

The proper solution seems clear: it is to have an in kernel tarball
minimalistic glibc, in kernel tarball set of kernel related user land tools so
that all of them benefit from the magic Linus taste.
Building the kernel should build a new /kbin/ and /klib/ directory with all
kernel related user land tools and libraries.
Then distibutions should just set softlinks to these.

As a summary, the fact that you say you don't want to have the decoding
running at ring 0 does not mean that it should not be part of the kernel
tarball.

Re: video4linux user land API concern

[linux-os (Dick Johnson)]

On Wed, 9 Nov 2005, Hubert Tonneau wrote:

> Alan Cox wrote:
>>
>> Many of the encodings done by hardware are extremely complicated and
>> tricky to unpack in kernel space. If a camera captures jpeg for example
>> you don't want in kernel jpeg decoders, let alone mpeg decoders etc
>
> So you fall back on the current main bad point about the Linux kernel design.
>

It isn't Linux kernel design. The Linux kernel emulates the Unix
architecture which, by definition, has the kernel performing tasks
on behalf of user programs. The kernel doesn't support the kernel!
It is designed to support user programs as efficiently as possible.
If you need a jpeg decoder, it is simply part of a user program,
hopefully shared with others as a runtime library. You just don't
put user stuff inside the kernel. It has no place there. Look at
Microsoft and see how they did things. They munged user-mode and
kernel stuff all into a slime-ball that gets linked together with
a database (the registry). That's why such systems perform poorly
and will never be reliable or secure.

If the techies (pronunced idiots) at Microsoft had left the Gordon
Bell kernel alone (he also designed VAX/VMS), and let it support
user-mode programs like it should, Linux probably would have died
out when Linus left Helsinki. Instead, people like you convinced
Microsoft that the "good stuff" should be inside the kernel.

People like Alan Cox are going to make sure that it doesn't
happen like that with Linux.

> One quality of Linux, as opposed to some other proposed kernels, is that all
> drivers are part of the kernel tarball, so you can expect to always get drivers
> that are consistent with the kernel internal API.
>
> On the other hand, over time, more and more kernel only related features
> have been rejected to user land, for various reasons mostly related to
> stability and security. I mean 'ifconfig', raid tools, etc.
> Your agument about video decoding falls in the stability category.
>

Again, learn about operating systems in general and what the
Unix/Linux platform is. These things didn't get rejected to user-land
they got put in user-land where they belong for the most efficient
operation.

> Now, if you look for documentation about the interface between the kernel
> and the user land tool that have has been written at the same time to provide
> an overall services (let think about software raid) then you discover that
> it's mosty inexisting, and that the interface in the middle is just an
> unengeneered nightmare.

No. We have a standard API that's well documented.

    Hint: struct file_operations

... for inside the kernel, and the usual open()/close()/read()/write()
/mmap()/poll()/ioctl() for the user-land interface. These are all
things that people who know about this architecture don't even have
to look up.

> So, in facts you can't use the kernel function without adopting the user
> land tool associated, but then you discover that very fiew eyes watched and
> checked the user land part of the tool, and that it's quality is really really
> poor (opens a file, but not check the error code, etc, etc)
>

So fix the user-land tool and send a patch to the maintainer.

> The proper solution seems clear: it is to have an in kernel tarball
> minimalistic glibc, in kernel tarball set of kernel related user land tools so
> that all of them benefit from the magic Linus taste.
> Building the kernel should build a new /kbin/ and /klib/ directory with all
> kernel related user land tools and libraries.
> Then distibutions should just set softlinks to these.
>

The proper solution is to learn what you are talking about,
then contribute something of value.

> As a summary, the fact that you say you don't want to have the decoding
> running at ring 0 does not mean that it should not be part of the kernel
> tarball.
>


Cheers,
Dick Johnson
Penguin : Linux version 2.6.13.4 on an i686 machine (5589.55 BogoMips).
Warning : 98.36% of all statistics are fiction.
.

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: video4linux user land API concern

[Alan Cox]

On Mer, 2005-11-09 at 12:58 +0000, Hubert Tonneau wrote:
> The proper solution seems clear: it is to have an in kernel tarball
> minimalistic glibc, in kernel tarball set of kernel related user land tools so
> that all of them benefit from the magic Linus taste.

And this is different to a "linux distribution" how ?

Re: video4linux user land API concern

[Hubert Tonneau]

Alan Cox wrote:
>
> And this is different to a "linux distribution" how ?

Think about two entry level utilities, I mean 'ls' and 'ifconfig'.

'ls' is clearly part of the Linux distribution because it should not care
the underlying kernel.

On the other hand 'ifconfig' has no meaning if the underlying is not Linux.

So, what I'm saying is that bringing all these utilities that have no meaning
if the underlying kernel is not Linux to /usr/src/linux/userland/
would help.

Most of us once falled on the problem you switch from 2.4 to 2.6 or the other
way round, and the system start not to work properly because the distribution
needs a different package for 2.4 and 2.6
(I've also discovered more subtil troubles such a 32 bits iptables will not
 be abble to configure a 64 bits Opteron kernel with 32 bits support,
 probably because the data structure at the interface of the kernel and iptable
 are long size dependent)

If all the kernel centric user land utilities are part of the kernel tree,
then:
1) when you change of kernel, you change of utilities set
2) it can be mandatory that these must not rely on external distribution files

As a result, you get a properly layered distribution, with:
. a kernel
. a mini SELF CONTAINED distibution with all kernel centric utilities and
  libraries
. the full distribution with true mostly kernel agnostic tools and applications

Such a design makes Linux
. more robust: fiewer computers are silently running with user land tools that
  don't match the kernel
. more flexible: packaging Linux for strange non Unix usage get's cheaper

The underlying reasoning is that even if most of the time the right
boundary between kernel and application is the kernel API, as you pointed
out, sometime it cannot (you don't want MPEG2 decoder in the kernel) so
the boundary should be between the kernel distribution and the overall
distribution, not between two parts (the kernel and the bloated distribution)
that we have no serious reason to assert that they will always match each
other.

Ok, this does not append if you assert that end users install new kernel
through their distribution packages, since the distribution packages will
(should) contain the dependencies to keep an overall consistent system.

So another question might be: do you prefer power users to compile and run
Linux kernel from the kernel.org plain source tarball, so send easy to use
feedback, or run only distribution packaged kernels.
In the first case, packing the low level kernel centric user land tools with
the kernel tarball makes power users life much easier, and improves feedback
since it helps to keep the system consistent.

Once upon a time, akpm stated that he has no way to reward people that mostly
provide feedback. That's ok since a mostly crash free kernel is a serious
reward.
Also helping them with a packaging troubles free kernel would be a serious
bonus, so I'm cc ing him this message.