Re: [V4L] Re: [RFC] alternative kernel multimedia API

Re: [V4L] Re: [RFC] alternative kernel multimedia API

[volodya]

On 26 Oct 2001, Gerd Knorr wrote:

> > > I would like to see a detailed explanation of how the capabilities 
> > > differ from those of v4l2.
> >  
> >  The major difference is in increased flexibility and stability of
> >  kernel-user space interface.
> >  
> >  On the application side:
> >         * ability to utilize advanced driver features without explicit
> >           knowledge of their behaviour
> >         * wider compatibility without the need for recompile
> 
> what exactly is bad with the v4l2 controls?

a) no way to use them to set non-integer data (like gamma tables)
b) labels are limited to 32 chars, no way to provide "comment" fields.
  (I would like the driver to provide a short label and longer,
   user-readable and understable comment field).
c) no way to present control dependencies.
  For example in ATI cards there are several ways you can control gamma:
  gamma table, gamma overlay setting, gamma setting in the decoder chip.
  same goes for brightness. I would rather expose all the controls and
  show their relationship then bundle them up in one control
  (as to why this is relevant to kernel and not only Xv see below)

> 
> >  On the kernel side:
> >         * elimination of interfaced specific headers
> 
> Ok, you replace the '#define V4L2_CID_HUE 0x???' with the magic string
> "HUE" then.  The point is?

ioctl are kernel specific. My scheme proposes a set of mmaped buffers and
character stream. The plus is that I can make Xmultimedia extension that
also presents a set of shared buffers and character stream. The
application code will be exactly the same.

Additionally nonone will need to include kernel headers to compile the
application. Also magic string HUE is a bigger namespace then an
integer number. 

> 
> >  On the driver side:
> >         * be compatible with a wide range of applications
> 
> ???
> Sorry, but I don't see why your approach handles this better than v4l2.

Since the application accesses the driver symbolically they only have to
worry about agreeing on semantics. For example the driver can layout its
control structs any way it likes (say, to mirror hardware registers) and 
the application will bind to them. 

Also, mmaped buffers + character device fits a larger category then just
the devices mentioned in v4l2 specifications. Granted the specification
can be extended, but the new driver will have to distribute not only its
source but a patch to the kernel headers. The scheme will avoid this.

> 
> >         * introduce support for new features without the need to modify
> >           kernel interfaces
> 
> Hmm.  I don't like the idea to add new stuff that easily.  People tend
> to do quick&dirty hacks + crazy stuff with it, leading to more
> incompatibilies between drivers and applications.

And people will always be able to write a driver outside video4linux
framework. In fact, plenty of drivers and applications are incompatible as
is. The same is true about audio drivers. 

The compatibility is achived best by specifications that are clear and
easy to implement. This proposal helps it by providing a well defined
interface and by separating semantic meaning from actual interface. 
First the internal kernel library will insure that the control interface
is standard across drivers. Secondly, the drivers are free to use whatever
fields they want - no restriction their. Thirdly, the way the applications
know they deal, say, with a grabber device is when the driver says it
supports a certain INTERFACE_CLASS. Hence, to make sure the drivers
behave, all we have to do is bar any driver which declares itself
compliant with a certain INTERFACE_CLASS but isn't from entering the
kernel. And the added advantage is that we can have many INTERFACE_CLASSes
which vary in implementation difficulty, so that BASIC_GRABBER-YUV422
would be very easy to implement (just expose the buffers) and something
more advanced could be added later. This solves the conflict of people
wanting to have _some_ driver and the time consumption of writing a driver
that supports all features.

> 
> >  The last point is the very important in my opinion. For example, in
> >  current specification, v4l2 has no support for TV-out in graphics cards.
> >  It has no support for setting complex parameters like gamma tables. 
> >  Using memory-mapped buffers requires device specific ioctls.
> 
> Which device specific ioctls?  They are common for _all_ v4l2 devices.

Quote from http://www.thedirks.org/v4l2/v4l2dev.htm :

A common use for memory-mapped buffers is for streaming data to and from
drivers with minimum overhead. Drivers will maintain internal queues of
buffers and process them asynchonously. The ioctl commands for doing that
are described in the device-specific documents. Memory-mapping can also be
used for a variety of other purposes. Drivers can define hardware-specific
memory buffer types if needed. Use V4L2_BUF_TYPE_PRIVATE_BASE and greater
values for such buffer types. 


> 
> >  The goal is to create an interface that does not rely on structures
> >  defined in kernel headers for communication. 
> 
> Why do you want to do that?

 * not elegant
 * kernel structures force a specific model onto a driver
 * can cause problems with different compilers
 * confuse applications when a driver does not implement a field

Here is another way one may arrive at this kind of interface:
 
Consider capability flags. Wouldn't it be nice to have the driver present
them symbolically instead of bits ? Besides everything else you get around
the issue of running out of bits when adding new ones. 

So you can query device and get a string of the form:

 PREVIEW,SELECT,TUNER

However, this is hard to implement with ioctl's as they rely on fixed
length structures. You would have to call first to find out the size of
the buffer you need (which the driver would have to compute) and then the
second time to get the data. Eeeks. Instead just make a device and let the
application do something like:

write:  ?QUERY_CAPABILITY\n
read: PREVIEW,SELECT,TUNER

Much better, ha ? Now if you agree that this is nice we can use the same
scheme for other things. For example the whole control (i.e. hue,
etc) query can be done thru this device. No need for structs. Same goes
for width,height,picture format etc. 

Now what is left are the structs describing the format of memory buffers. 
But the driver typically has this info in a struct itself. Instead of
getting it via ioctl simply mmap the area - you'll be getting the exact
same information yourself with less context switches.

This gets rid of the structs. 

What is left of v4l now are synchronization ioctls. But we can replace
them with read/writes on the control device. With the added benifit that
we can select on it to wait for an event.

                      Vladimir Dergachev

> 
>   Gerd
> 
> -- 
> Netscape is unable to locate the server localhost:8000.
> Please check the server name and try again.
> 
> 
> 
> _______________________________________________
> Video4linux-list mailing list
> Video4linux-list@redhat.com
> https://listman.redhat.com/mailman/listinfo/video4linux-list
> 

Re: [V4L] Re: [RFC] alternative kernel multimedia API

[Gerd Knorr]

> > what exactly is bad with the v4l2 controls?
>  
>  a) no way to use them to set non-integer data (like gamma tables)

Yes, they aren't designed for complex stuff.  They can do boolean,
multiple choice and integer options.  These ones an application can
easily present a reasonable GUI to the user without knowing what the
control exactly is good for.  For more complex stuff like gamma tables
(list of integers?) it is IMHO nearly impossible to build a reasonable
GUI without knowing what these values are actually good for.  Or even
know whenever it is useful to build a GUI for it or not.

>  b) labels are limited to 32 chars, no way to provide "comment" fields.
>    (I would like the driver to provide a short label and longer,
>     user-readable and understable comment field).

IMHO this isn't the job of the device driver.  Think about translations
of these descriptions for example ...

>  c) no way to present control dependencies.

I don't see how your approach handles this.

>    For example in ATI cards there are several ways you can control gamma:
>    gamma table, gamma overlay setting, gamma setting in the decoder chip.
>    same goes for brightness. I would rather expose all the controls and
>    show their relationship then bundle them up in one control

... and thus trying to make the driver tell the applications how the GUI
should present the controls to the user?

> > Ok, you replace the '#define V4L2_CID_HUE 0x???' with the magic string
> > "HUE" then.  The point is?
>  
>  ioctl are kernel specific. My scheme proposes a set of mmaped buffers and
>  character stream. The plus is that I can make Xmultimedia extension that
>  also presents a set of shared buffers and character stream. The
>  application code will be exactly the same.

That's your main point I assume?  You are trying to kill the ioctls
because you want to use the same protocol between application + driver
and application + other application (say a X-Server)?

>  Additionally nonone will need to include kernel headers to compile the
>  application.

Why do you want to avoid this?

>  Also magic string HUE is a bigger namespace then an
>  integer number. 

I doubt we will ever run out of space with 2^32 integer numbers.  I have
yet to see a v4l2 driver which has more than 2^8 controls ...

> > >         * be compatible with a wide range of applications
> > 
> > ???
> > Sorry, but I don't see why your approach handles this better than v4l2.
>  
>  Since the application accesses the driver symbolically they only have to
>  worry about agreeing on semantics. For example the driver can layout its
>  control structs any way it likes (say, to mirror hardware registers) and 
>  the application will bind to them. 

I don't see any difference to v4l2 here ...

>  Also, mmaped buffers + character device fits a larger category then just
>  the devices mentioned in v4l2 specifications. Granted the specification
>  can be extended, but the new driver will have to distribute not only its
>  source but a patch to the kernel headers. The scheme will avoid this.

And why this is better?  You likely still have to teach the applications
about the new features of the driver if you actually want to use them
(unless there are just a few new controls, but this case can easily
handled with v4l2 too).

> > >         * introduce support for new features without the need to modify
> > >           kernel interfaces
> > 
> > Hmm.  I don't like the idea to add new stuff that easily.  People tend
> > to do quick&dirty hacks + crazy stuff with it, leading to more
> > incompatibilies between drivers and applications.
>  
>  And people will always be able to write a driver outside video4linux
>  framework. In fact, plenty of drivers and applications are incompatible as
>  is. The same is true about audio drivers. 

Yes, these incompatibility issues are bad.  But I doubt the situation
will become better if every driver can easily add new stuff.

>  The compatibility is achived best by specifications that are clear and
>  easy to implement. This proposal helps it by providing a well defined
>  interface and by separating semantic meaning from actual interface. 

The semantic needs to be documented too, otherwise it easily gets worse
than v4l2.  One big plus for v4l2 is that it actually has very good API
specs.

>  First the internal kernel library will insure that the control interface
>  is standard across drivers. Secondly, the drivers are free to use whatever
>  fields they want - no restriction their.  Thirdly, the way the applications
>  know they deal, say, with a grabber device is when the driver says it
>  supports a certain INTERFACE_CLASS. Hence, to make sure the drivers
>  behave, all we have to do is bar any driver which declares itself
>  compliant with a certain INTERFACE_CLASS but isn't from entering the
>  kernel.

I don't see how this is different from the v4l2 ioctl interface.  We
have capability flags, ...

>  And the added advantage is that we can have many INTERFACE_CLASSes
>  which vary in implementation difficulty, so that BASIC_GRABBER-YUV422
>  would be very easy to implement (just expose the buffers) and something
>  more advanced could be added later. This solves the conflict of people
>  wanting to have _some_ driver and the time consumption of writing a driver
>  that supports all features.

I doubt this fixes the problem (driver devel time) you've mentioned.
Most of the time of the driver development is needed for the code which
talks to the hardware.  Building a simple v4l(2) ioctl interface which
just says "I can nothing but capture, and the format list is YUV422" is
easy compared to the other work which needs to be done for a working
driver.  v4l(2) doesn't force you to wait with the release until you
have a full-featured driver ...

> > >  The last point is the very important in my opinion. For example, in
> > >  current specification, v4l2 has no support for TV-out in graphics cards.
> > >  It has no support for setting complex parameters like gamma tables. 
> > >  Using memory-mapped buffers requires device specific ioctls.
> > 
> > Which device specific ioctls?  They are common for _all_ v4l2 devices.
>  
>  Quote from http://www.thedirks.org/v4l2/v4l2dev.htm :
>  
>  A common use for memory-mapped buffers is for streaming data to and from
>  drivers with minimum overhead. Drivers will maintain internal queues of
>  buffers and process them asynchonously. The ioctl commands for doing that
>  are described in the device-specific documents. Memory-mapping can also be
>  used for a variety of other purposes. Drivers can define hardware-specific
>  memory buffer types if needed. Use V4L2_BUF_TYPE_PRIVATE_BASE and greater
>  values for such buffer types. 

Ok, for different device types.  Currently only capture drivers exist
for v4l2 btw.  I doubt that your application will use the _same_ code
path for a _different_ device type.

> > >  The goal is to create an interface that does not rely on structures
> > >  defined in kernel headers for communication. 
> > 
> > Why do you want to do that?
>  
>   * not elegant

"elegant" is hardly a argument because it is subjective.

>   * kernel structures force a specific model onto a driver

The control strings you want to read() and write() to the driver do
exactly the same, because they must have some clear defined semantic to
make the whole model actually workable.

>   * can cause problems with different compilers

Then your compiler is buggy.

>   * confuse applications when a driver does not implement a field

With v4l2 controls this shouldn't be a issue any more.

>  However, this is hard to implement with ioctl's as they rely on fixed
>  length structures. You would have to call first to find out the size of
>  the buffer you need (which the driver would have to compute) and then the
>  second time to get the data. Eeeks.

Wrong.  Look at the specs for the v4l2 controls.

>  Now what is left are the structs describing the format of memory buffers. 
>  But the driver typically has this info in a struct itself. Instead of
>  getting it via ioctl simply mmap the area - you'll be getting the exact
>  same information yourself with less context switches.

read() + write() need context switches too.  And ioctl() can actually
pass informations in both directions, there are cases where you need
_more_ context switches with read()/write() than with ioctl() because
with ioctl one system calls does the job, but with read()/write() you
need two:  one for the request and one to get back the actual data.

>  This gets rid of the structs. 

Using strings instead adds a overhead to both driver and applications
for string parsing.  And the application likely puts the parsed data
into a struct again anyway ...

>  What is left of v4l now are synchronization ioctls. But we can replace
>  them with read/writes on the control device. With the added benifit that
>  we can select on it to wait for an event.

v4l2 expects drivers to support select too, so you can wait for your
capture requests using select.

  Gerd

-- 
Netscape is unable to locate the server localhost:8000.
Please check the server name and try again.

Re: [V4L] Re: [RFC] alternative kernel multimedia API

[volodya]

On Fri, 26 Oct 2001, Gerd Knorr wrote:

> > > what exactly is bad with the v4l2 controls?
> >  
> >  a) no way to use them to set non-integer data (like gamma tables)
> 
> Yes, they aren't designed for complex stuff.  They can do boolean,
> multiple choice and integer options.  These ones an application can
> easily present a reasonable GUI to the user without knowing what the
> control exactly is good for.  For more complex stuff like gamma tables
> (list of integers?) it is IMHO nearly impossible to build a reasonable
> GUI without knowing what these values are actually good for.  Or even
> know whenever it is useful to build a GUI for it or not.

First of all such a GUI is relatively easy to build: see gamma controls in
XV. Secondly, you are presenting a shaky argument: GUI is hard to built
hence do not let user adjust them at all..

> 
> >  b) labels are limited to 32 chars, no way to provide "comment" fields.
> >    (I would like the driver to provide a short label and longer,
> >     user-readable and understable comment field).
> 
> IMHO this isn't the job of the device driver.  Think about translations
> of these descriptions for example ...
> 

I think it is a job of the interface presented to the user. Since we want
the application to find this information from the driver it should know
it.

> >  c) no way to present control dependencies.
> 
> I don't see how your approach handles this.

Ahh, it is not there yet. I did not want to do it until there is a simple
driver to experiment with. It would be along the lines of
HUE_DEPENDENT_ON=....

> 
> >    For example in ATI cards there are several ways you can control gamma:
> >    gamma table, gamma overlay setting, gamma setting in the decoder chip.
> >    same goes for brightness. I would rather expose all the controls and
> >    show their relationship then bundle them up in one control
> 
> ... and thus trying to make the driver tell the applications how the GUI
> should present the controls to the user?

No, but allow the driver to expose the structure to the application. 

> 
> > > Ok, you replace the '#define V4L2_CID_HUE 0x???' with the magic string
> > > "HUE" then.  The point is?
> >  
> >  ioctl are kernel specific. My scheme proposes a set of mmaped buffers and
> >  character stream. The plus is that I can make Xmultimedia extension that
> >  also presents a set of shared buffers and character stream. The
> >  application code will be exactly the same.
> 
> That's your main point I assume?  You are trying to kill the ioctls
> because you want to use the same protocol between application + driver
> and application + other application (say a X-Server)?
> 

One of them. The cards I am writing v4l driver for are graphics cards with
integrated capture. You get:
  * v4l/v4l2 capture
  * Xv video overlay
  * interfaces not supported by current infrastructure

This is too complex, if you consider that all these things do is adjust
some values and transfer chunks of data.

> >  Additionally nonone will need to include kernel headers to compile the
> >  application.
> 
> Why do you want to avoid this?
> 

So I can distribute the driver and the application to use it and not
depend on the kernel version.

> >  Also magic string HUE is a bigger namespace then an
> >  integer number. 
> 
> I doubt we will ever run out of space with 2^32 integer numbers.  I have
> yet to see a v4l2 driver which has more than 2^8 controls ...
> 

We did run out of TCP/IP addresses and major/minor numbers. I can see
myself introducing GAMMA_xxx which start from 0x10000000 and go another
0x1000 numbers.

> > > >         * be compatible with a wide range of applications
> > > 
> > > ???
> > > Sorry, but I don't see why your approach handles this better than v4l2.
> >  
> >  Since the application accesses the driver symbolically they only have to
> >  worry about agreeing on semantics. For example the driver can layout its
> >  control structs any way it likes (say, to mirror hardware registers) and 
> >  the application will bind to them. 
> 
> I don't see any difference to v4l2 here ...
> 
> >  Also, mmaped buffers + character device fits a larger category then just
> >  the devices mentioned in v4l2 specifications. Granted the specification
> >  can be extended, but the new driver will have to distribute not only its
> >  source but a patch to the kernel headers. The scheme will avoid this.
> 
> And why this is better?  You likely still have to teach the applications
> about the new features of the driver if you actually want to use them
> (unless there are just a few new controls, but this case can easily
> handled with v4l2 too).
> 

You don't have to invent a new kernel header just to support a new set of
devices.

> > > >         * introduce support for new features without the need to modify
> > > >           kernel interfaces
> > > 
> > > Hmm.  I don't like the idea to add new stuff that easily.  People tend
> > > to do quick&dirty hacks + crazy stuff with it, leading to more
> > > incompatibilies between drivers and applications.
> >  
> >  And people will always be able to write a driver outside video4linux
> >  framework. In fact, plenty of drivers and applications are incompatible as
> >  is. The same is true about audio drivers. 
> 
> Yes, these incompatibility issues are bad.  But I doubt the situation
> will become better if every driver can easily add new stuff.
> 

If this does not interfere with other drivers -  yes.

> >  The compatibility is achived best by specifications that are clear and
> >  easy to implement. This proposal helps it by providing a well defined
> >  interface and by separating semantic meaning from actual interface. 
> 
> The semantic needs to be documented too, otherwise it easily gets worse
> than v4l2.  One big plus for v4l2 is that it actually has very good API
> specs.

v4l2 is quite a lot bettern than v4l. But it is quite static. Think about
what needs to be done to enable gamma tables for video out - this is not
trivial..

> 
> >  First the internal kernel library will insure that the control interface
> >  is standard across drivers. Secondly, the drivers are free to use whatever
> >  fields they want - no restriction their.  Thirdly, the way the applications
> >  know they deal, say, with a grabber device is when the driver says it
> >  supports a certain INTERFACE_CLASS. Hence, to make sure the drivers
> >  behave, all we have to do is bar any driver which declares itself
> >  compliant with a certain INTERFACE_CLASS but isn't from entering the
> >  kernel.
> 
> I don't see how this is different from the v4l2 ioctl interface.  We
> have capability flags, ...
> 
> >  And the added advantage is that we can have many INTERFACE_CLASSes
> >  which vary in implementation difficulty, so that BASIC_GRABBER-YUV422
> >  would be very easy to implement (just expose the buffers) and something
> >  more advanced could be added later. This solves the conflict of people
> >  wanting to have _some_ driver and the time consumption of writing a driver
> >  that supports all features.
> 
> I doubt this fixes the problem (driver devel time) you've mentioned.
> Most of the time of the driver development is needed for the code which
> talks to the hardware.  Building a simple v4l(2) ioctl interface which
> just says "I can nothing but capture, and the format list is YUV422" is
> easy compared to the other work which needs to be done for a working
> driver.  v4l(2) doesn't force you to wait with the release until you
> have a full-featured driver ...

It's not. What I want to be able to do is to release a driver that
supports basic functionality plus a bunch of device-specific stuff for
which no API exists. Then, with more experience, we can make an API. 
With current approach we make an API first, and then write the
driver. That's why we need v4l2: v4l was an API designed for bt848 cards
which was impossible to extend without breaking applications.

> 
> > > >  The last point is the very important in my opinion. For example, in
> > > >  current specification, v4l2 has no support for TV-out in graphics cards.
> > > >  It has no support for setting complex parameters like gamma tables. 
> > > >  Using memory-mapped buffers requires device specific ioctls.
> > > 
> > > Which device specific ioctls?  They are common for _all_ v4l2 devices.
> >  
> >  Quote from http://www.thedirks.org/v4l2/v4l2dev.htm :
> >  
> >  A common use for memory-mapped buffers is for streaming data to and from
> >  drivers with minimum overhead. Drivers will maintain internal queues of
> >  buffers and process them asynchonously. The ioctl commands for doing that
> >  are described in the device-specific documents. Memory-mapping can also be
> >  used for a variety of other purposes. Drivers can define hardware-specific
> >  memory buffer types if needed. Use V4L2_BUF_TYPE_PRIVATE_BASE and greater
> >  values for such buffer types. 
> 
> Ok, for different device types.  Currently only capture drivers exist
> for v4l2 btw.  I doubt that your application will use the _same_ code
> path for a _different_ device type.
> 

A good deal of the code path could be preserved. What I want more is to
eliminate patching generic kernel interfaces to implement support
for new devices.

> > > >  The goal is to create an interface that does not rely on structures
> > > >  defined in kernel headers for communication. 
> > > 
> > > Why do you want to do that?
> >  
> >   * not elegant
> 
> "elegant" is hardly a argument because it is subjective.
> 

But explains why I want it ;)

> >   * kernel structures force a specific model onto a driver
> 
> The control strings you want to read() and write() to the driver do
> exactly the same, because they must have some clear defined semantic to
> make the whole model actually workable.
> 

The benifit is that we dont have to define it in the kernel headers. Right
now if I want to add a new ioctl I have tons of problems:

      * how do I communicate to the application source the new structs
        I am using ?
      * what do I do if someone uses the same ioctl in the kernel source
      * what happens if I want to add a new field during driver
        development ? (users will see crashing applications..)

> >   * can cause problems with different compilers
> 
> Then your compiler is buggy.

No, I may have simply used different compilers for the kernel and the
application.

> 
> >   * confuse applications when a driver does not implement a field
> 
> With v4l2 controls this shouldn't be a issue any more.

What if the driver does not support counting dropped frames ? What if
there is a control with no min/max limits ?

> 
> >  However, this is hard to implement with ioctl's as they rely on fixed
> >  length structures. You would have to call first to find out the size of
> >  the buffer you need (which the driver would have to compute) and then the
> >  second time to get the data. Eeeks.
> 
> Wrong.  Look at the specs for the v4l2 controls.

Can I return a string as a value ? (For a device specific control).
Can I set/read gamma values ?

> 
> >  Now what is left are the structs describing the format of memory buffers. 
> >  But the driver typically has this info in a struct itself. Instead of
> >  getting it via ioctl simply mmap the area - you'll be getting the exact
> >  same information yourself with less context switches.
> 
> read() + write() need context switches too.  And ioctl() can actually
> pass informations in both directions, there are cases where you need
> _more_ context switches with read()/write() than with ioctl() because
> with ioctl one system calls does the job, but with read()/write() you
> need two:  one for the request and one to get back the actual data.
> 

control interface is not meant to be high bandwidth. this pefectly fine +
the benifits mentioned earlier.

> >  This gets rid of the structs. 
> 
> Using strings instead adds a overhead to both driver and applications
> for string parsing.  And the application likely puts the parsed data
> into a struct again anyway ...
> 

I don't think this true as parsing is really not that hard. I don't expect
you to believe me though until there is some code to work with.

> >  What is left of v4l now are synchronization ioctls. But we can replace
> >  them with read/writes on the control device. With the added benifit that
> >  we can select on it to wait for an event.
> 
> v4l2 expects drivers to support select too, so you can wait for your
> capture requests using select.

Which means one more thing in the driver to support.

                         Vladimir Dergachev

> 
>   Gerd
> 
> -- 
> Netscape is unable to locate the server localhost:8000.
> Please check the server name and try again.
> 

Re: [V4L] Re: [RFC] alternative kernel multimedia API

[Gerd Knorr]

> > GUI without knowing what these values are actually good for.  Or even
> > know whenever it is useful to build a GUI for it or not.
> 
> First of all such a GUI is relatively easy to build: see gamma controls in
> XV. Secondly, you are presenting a shaky argument: GUI is hard to built
> hence do not let user adjust them at all..

There are cases where it is quite useless to build a GUI:  If you have a
device with hardware compression support and want to set DCT
coefficients for example.  It is (like gamma table) just a list of
values, but you certainly don't want present the same GUI for them to
the user.  For the DCT coefficients it probably would be reasonable to
present no GUI at all, or let the user only pick one of a few reasonable
choices.

> > I don't see how your approach handles this.
> 
> Ahh, it is not there yet. I did not want to do it until there is a simple
> driver to experiment with. It would be along the lines of
> HUE_DEPENDENT_ON=....

And this means?  If hue depends on foo, the application should read back
the hue value and update the GUI if foo has changed?

> > >  Additionally nonone will need to include kernel headers to compile the
> > >  application.
> > 
> > Why do you want to avoid this?
> > 
> 
> So I can distribute the driver and the application to use it and not
> depend on the kernel version.

The exported API must not depend on the kernel version.  If the API
changes in a way which isn't backward compatible, it is a kernel bug.

> > >  Also, mmaped buffers + character device fits a larger category then just
> > >  the devices mentioned in v4l2 specifications. Granted the specification
> > >  can be extended, but the new driver will have to distribute not only its
> > >  source but a patch to the kernel headers. The scheme will avoid this.
> > 
> > And why this is better?  You likely still have to teach the applications
> > about the new features of the driver if you actually want to use them
> > (unless there are just a few new controls, but this case can easily
> > handled with v4l2 too).
> 
> You don't have to invent a new kernel header just to support a new set of
> devices.

I can't see why it is a problem to add a new header or new ioctls to
a existing header file.  I like it this way, because the kernel headers
with all the #defines and structs are providing at least a minimum of
documentation.  I do often read header files when programming stuff.

> v4l2 is quite a lot bettern than v4l. But it is quite static.

I consider a static API a good thing.  Code often lives for years, and
it is nice if your five year old source code build and works just fine
...

> Think about
> what needs to be done to enable gamma tables for video out - this is not
> trivial..

Yes, we'll basically need new v4l2 ioctls for it.

> > I doubt this fixes the problem (driver devel time) you've mentioned.
> > Most of the time of the driver development is needed for the code which
> > talks to the hardware.  Building a simple v4l(2) ioctl interface which
> > just says "I can nothing but capture, and the format list is YUV422" is
> > easy compared to the other work which needs to be done for a working
> > driver.  v4l(2) doesn't force you to wait with the release until you
> > have a full-featured driver ...
> 
> It's not. What I want to be able to do is to release a driver that
> supports basic functionality plus a bunch of device-specific stuff for
> which no API exists. Then, with more experience, we can make an API. 

Nobody stops you from creating some temporary device-specific ioctls for
playing and testing ...

Of course old applications don't know about the new stuff (neither the
temporary nor the final versions which make in into the official API
some day maybe).  But I don't see how your approach handles this better:
Applications still need to be hacked to use the new stuff ...


> > >   * kernel structures force a specific model onto a driver
> > 
> > The control strings you want to read() and write() to the driver do
> > exactly the same, because they must have some clear defined semantic to
> > make the whole model actually workable.
> > 
> 
> The benifit is that we dont have to define it in the kernel headers. Right
> now if I want to add a new ioctl I have tons of problems:
> 
>       * how do I communicate to the application source the new structs
>         I am using ?

You can't.  But I don't see why this is a issue:  The only thing a
application can handle easily are controls like contrast/hue where the
only thing a application needs to do is to map it to a GUI and let the
user understand and adjust stuff.  The other stuff has way to much
non-trivial dependences, I doubt a application can blindly use new
driver features.

>       * what do I do if someone uses the same ioctl in the kernel source

???

>       * what happens if I want to add a new field during driver
>         development ? (users will see crashing applications..)

No.

v4l2 has a few reserved[x] in the structs to allow such extentions
without breaking binary comparibility.

If you add a new field to a struct the ioctl magic number changes
because the size of the struct has changed.  This can be used to add new
fields without breaking stuff (althrought it starts to become ugly here,
drivers have to support both old and new versions of the ioctl).  Look
at SOUND_MIXER_INFO vs. SOUND_OLD_MIXER_INFO in linux/soundcard.h for a
example.

> > >   * can cause problems with different compilers
> > 
> > Then your compiler is buggy.
> 
> No, I may have simply used different compilers for the kernel and the
> application.

I doubt that.

> > >   * confuse applications when a driver does not implement a field
> > 
> > With v4l2 controls this shouldn't be a issue any more.
> 
> What if the driver does not support counting dropped frames ?

-EINVAL or something like that.

> What if there is a control with no min/max limits ?

Do you have a example?

> > >  However, this is hard to implement with ioctl's as they rely on fixed
> > >  length structures. You would have to call first to find out the size of
> > >  the buffer you need (which the driver would have to compute) and then the
> > >  second time to get the data. Eeeks.
> > 
> > Wrong.  Look at the specs for the v4l2 controls.
> 
> Can I return a string as a value ?  (For a device specific control).

Multiple choice controls have strings for each value.

> Can I set/read gamma values ?

Gamma values yes (tables no).

> > >  them with read/writes on the control device. With the added benifit that
> > >  we can select on it to wait for an event.
> > 
> > v4l2 expects drivers to support select too, so you can wait for your
> > capture requests using select.
> 
> Which means one more thing in the driver to support.

Of course a driver needs some code to support select(), it doesn't work
with black magic.  What else do you expect?

  Gerd

Re: [V4L] Re: [RFC] alternative kernel multimedia API

[volodya]

On Mon, 29 Oct 2001, Gerd Knorr wrote:

> > > GUI without knowing what these values are actually good for.  Or even
> > > know whenever it is useful to build a GUI for it or not.
> > 
> > First of all such a GUI is relatively easy to build: see gamma controls in
> > XV. Secondly, you are presenting a shaky argument: GUI is hard to built
> > hence do not let user adjust them at all..
> 
> There are cases where it is quite useless to build a GUI:  If you have a
> device with hardware compression support and want to set DCT
> coefficients for example.  It is (like gamma table) just a list of
> values, but you certainly don't want present the same GUI for them to
> the user.  For the DCT coefficients it probably would be reasonable to
> present no GUI at all, or let the user only pick one of a few reasonable
> choices.
> 

Well, I am not saying that GUI needs to be presented for everything that
can be adjusted, at least not unless requested explicitly by the user.

> > > I don't see how your approach handles this.
> > 
> > Ahh, it is not there yet. I did not want to do it until there is a simple
> > driver to experiment with. It would be along the lines of
> > HUE_DEPENDENT_ON=....
> 
> And this means?  If hue depends on foo, the application should read back
> the hue value and update the GUI if foo has changed?

Let me do it with mixers ;)

VOLUME1

VOLUME2

VOLUME2_DEPENDENT_ON=MIXER1

This means that VOLUME2 is the master control of VOLUME1.

Perhaps I should change it to VOLUME1_MASTER=VOLUME2 ...

> 
> > > >  Additionally nonone will need to include kernel headers to compile the
> > > >  application.
> > > 
> > > Why do you want to avoid this?
> > > 
> > 
> > So I can distribute the driver and the application to use it and not
> > depend on the kernel version.
> 
> The exported API must not depend on the kernel version.  If the API
> changes in a way which isn't backward compatible, it is a kernel bug.

I already got a report from the user that in his kernel
video_register_device has 2 arguments and not 3. This is pain to deal with
in drivers distributed separately from kernel tree.

> 
> > > >  Also, mmaped buffers + character device fits a larger category then just
> > > >  the devices mentioned in v4l2 specifications. Granted the specification
> > > >  can be extended, but the new driver will have to distribute not only its
> > > >  source but a patch to the kernel headers. The scheme will avoid this.
> > > 
> > > And why this is better?  You likely still have to teach the applications
> > > about the new features of the driver if you actually want to use them
> > > (unless there are just a few new controls, but this case can easily
> > > handled with v4l2 too).
> > 
> > You don't have to invent a new kernel header just to support a new set of
> > devices.
> 
> I can't see why it is a problem to add a new header or new ioctls to
> a existing header file.  I like it this way, because the kernel headers
> with all the #defines and structs are providing at least a minimum of
> documentation.  I do often read header files when programming stuff.
> 

Because we don't know which interface is best until we experiment with
it. And I can't experiment without people being able to test. And the
easier it is to compile and install code the more testers (and developers
!) we get.

> > v4l2 is quite a lot bettern than v4l. But it is quite static.
> 
> I consider a static API a good thing.  Code often lives for years, and
> it is nice if your five year old source code build and works just fine
> ...
> 

This sounds more as an argument between assembler and C. Each has its
place.

> > Think about
> > what needs to be done to enable gamma tables for video out - this is not
> > trivial..
> 
> Yes, we'll basically need new v4l2 ioctls for it.
> 
> > > I doubt this fixes the problem (driver devel time) you've mentioned.
> > > Most of the time of the driver development is needed for the code which
> > > talks to the hardware.  Building a simple v4l(2) ioctl interface which
> > > just says "I can nothing but capture, and the format list is YUV422" is
> > > easy compared to the other work which needs to be done for a working
> > > driver.  v4l(2) doesn't force you to wait with the release until you
> > > have a full-featured driver ...
> > 
> > It's not. What I want to be able to do is to release a driver that
> > supports basic functionality plus a bunch of device-specific stuff for
> > which no API exists. Then, with more experience, we can make an API. 
> 
> Nobody stops you from creating some temporary device-specific ioctls for
> playing and testing ...
> 
> Of course old applications don't know about the new stuff (neither the
> temporary nor the final versions which make in into the official API
> some day maybe).  But I don't see how your approach handles this better:
> Applications still need to be hacked to use the new stuff ...
> 

The way you write device specific appliacation is by including kernel
headers. If the stuff we want is not there makes a lot trouble for
installing and maintaining code.

> 
> > > >   * kernel structures force a specific model onto a driver
> > > 
> > > The control strings you want to read() and write() to the driver do
> > > exactly the same, because they must have some clear defined semantic to
> > > make the whole model actually workable.
> > > 
> > 
> > The benifit is that we dont have to define it in the kernel headers. Right
> > now if I want to add a new ioctl I have tons of problems:
> > 
> >       * how do I communicate to the application source the new structs
> >         I am using ?
> 
> You can't.  But I don't see why this is a issue:  The only thing a
> application can handle easily are controls like contrast/hue where the
> only thing a application needs to do is to map it to a GUI and let the
> user understand and adjust stuff.  The other stuff has way to much
> non-trivial dependences, I doubt a application can blindly use new
> driver features.
> 

Have you ever thought that the reason we only use these controls is
because they are the only ones easy to implement now ?

> >       * what do I do if someone uses the same ioctl in the kernel source
> 
> ???

Same ioctl number, in the kernel headers..

> 
> >       * what happens if I want to add a new field during driver
> >         development ? (users will see crashing applications..)
> 
> No.
> 
> v4l2 has a few reserved[x] in the structs to allow such extentions
> without breaking binary comparibility.
> 
> If you add a new field to a struct the ioctl magic number changes
> because the size of the struct has changed.  This can be used to add new
> fields without breaking stuff (althrought it starts to become ugly here,
> drivers have to support both old and new versions of the ioctl).  Look
> at SOUND_MIXER_INFO vs. SOUND_OLD_MIXER_INFO in linux/soundcard.h for a
> example.
> 

And then we run out of them.. and we can run out of magic number too.. 

> > > >   * can cause problems with different compilers
> > > 
> > > Then your compiler is buggy.
> > 
> > No, I may have simply used different compilers for the kernel and the
> > application.
> 
> I doubt that.

You are kidding, aren't you ? Most distributions now come out with egcs
compiler 1.1.x "recommended for compiling" kernels and something newer.

> 
> > > >   * confuse applications when a driver does not implement a field
> > > 
> > > With v4l2 controls this shouldn't be a issue any more.
> > 
> > What if the driver does not support counting dropped frames ?
> 
> -EINVAL or something like that.

But supports every other field.

> 
> > What if there is a control with no min/max limits ?
> 
> Do you have a example?
> 

Overlay color key - this is basically an RGB pixel value.


> > > >  However, this is hard to implement with ioctl's as they rely on fixed
> > > >  length structures. You would have to call first to find out the size of
> > > >  the buffer you need (which the driver would have to compute) and then the
> > > >  second time to get the data. Eeeks.
> > > 
> > > Wrong.  Look at the specs for the v4l2 controls.
> > 
> > Can I return a string as a value ?  (For a device specific control).
> 
> Multiple choice controls have strings for each value.
> 

With no way to pass strings (and their meaning) from the driver to the
user applications.

> > Can I set/read gamma values ?
> 
> Gamma values yes (tables no).
> 
> > > >  them with read/writes on the control device. With the added benifit that
> > > >  we can select on it to wait for an event.
> > > 
> > > v4l2 expects drivers to support select too, so you can wait for your
> > > capture requests using select.
> > 
> > Which means one more thing in the driver to support.
> 
> Of course a driver needs some code to support select(), it doesn't work
> with black magic.  What else do you expect?

I would prefer minimum effort on the part of driver writer ;) At the
moment all I see in bttv and my own code for select is looking on some
already existing fields. Heck, the code is very similar to what needs to
be checked for a blocking/non-blocking read. Why duplicate it ?

Also, v4l select will not work (as far as I understand) when the driver is
using memory-mapped buffers.

                        Vladimir Dergachev

> 
>   Gerd
> 

Re: [V4L] Re: [RFC] alternative kernel multimedia API

[Gerd Knorr]

> > > So I can distribute the driver and the application to use it and not
> > > depend on the kernel version.
> > 
> > The exported API must not depend on the kernel version.  If the API
> > changes in a way which isn't backward compatible, it is a kernel bug.
> 
> I already got a report from the user that in his kernel
> video_register_device has 2 arguments and not 3. This is pain to deal with
> in drivers distributed separately from kernel tree.

Internal kernel interfaces are another story, they are not fixed and are
allways subject to change and not limited to v4l.  See the major pci
subsystem changes from 2.2 => 2.4 for example.  The API visible to the
application matters, this must stay backward compatible.

> > I can't see why it is a problem to add a new header or new ioctls to
> > a existing header file.  I like it this way, because the kernel headers
> > with all the #defines and structs are providing at least a minimum of
> > documentation.  I do often read header files when programming stuff.
> > 
> 
> Because we don't know which interface is best until we experiment with
> it. And I can't experiment without people being able to test. And the
> easier it is to compile and install code the more testers (and developers
> !) we get.

I can't see why ioctls don't allow you to experiment.

> > Of course old applications don't know about the new stuff (neither the
> > temporary nor the final versions which make in into the official API
> > some day maybe).  But I don't see how your approach handles this better:
> > Applications still need to be hacked to use the new stuff ...
> > 
> 
> The way you write device specific appliacation is by including kernel
> headers. If the stuff we want is not there makes a lot trouble for
> installing and maintaining code.

No, it doesn't.  I never had trouble with xawtv.  I simply shipped a
private copy of videodev.h with the xawtv tarball (which allowed to
build it without hassle on 2.0.x systems which had no linux/videodev.h
yet).

> > You can't.  But I don't see why this is a issue:  The only thing a
> > application can handle easily are controls like contrast/hue where the
> > only thing a application needs to do is to map it to a GUI and let the
> > user understand and adjust stuff.  The other stuff has way to much
> > non-trivial dependences, I doubt a application can blindly use new
> > driver features.
> 
> Have you ever thought that the reason we only use these controls is
> because they are the only ones easy to implement now ?

I doubt this is just a implementation issue.  I don't believe in AI.

> > > > >   * can cause problems with different compilers
> > > > 
> > > > Then your compiler is buggy.
> > > 
> > > No, I may have simply used different compilers for the kernel and the
> > > application.
> > 
> > I doubt that.
> 
> You are kidding, aren't you ? Most distributions now come out with egcs
> compiler 1.1.x "recommended for compiling" kernels and something newer.

I doubt that this creates trouble, not that you might use different
compilers (I know RH ships or used to ship kgcc ...).

> > > What if the driver does not support counting dropped frames ?
> > 
> > -EINVAL or something like that.
> 
> But supports every other field.

Can't happen, there is VIDIOC_G_PERF only for this performance
monitoring ...

> > > What if there is a control with no min/max limits ?
> > 
> > Do you have a example?
> > 
> 
> Overlay color key - this is basically an RGB pixel value.

0x000000 => 0xffffff ?

> I would prefer minimum effort on the part of driver writer ;) At the
> moment all I see in bttv and my own code for select is looking on some
> already existing fields. Heck, the code is very similar to what needs to
> be checked for a blocking/non-blocking read. Why duplicate it ?

bttv's poll function is very short for exactly this reason.  Basically I
only have to call poll_wait with the (existing) wait queue which the IRQ
handler will wake up once the frame capture is finished.

> Also, v4l select will not work (as far as I understand) when the driver is
> using memory-mapped buffers.

v4l2 expects drivers to support select for the mmap() case too.

  Gerd

-- 
Netscape is unable to locate the server localhost:8000.
Please check the server name and try again.

Re: [V4L] Re: [RFC] alternative kernel multimedia API

[volodya]

On Tue, 30 Oct 2001, Gerd Knorr wrote:

> > > > So I can distribute the driver and the application to use it and not
> > > > depend on the kernel version.
> > > 
> > > The exported API must not depend on the kernel version.  If the API
> > > changes in a way which isn't backward compatible, it is a kernel bug.
> > 
> > I already got a report from the user that in his kernel
> > video_register_device has 2 arguments and not 3. This is pain to deal with
> > in drivers distributed separately from kernel tree.
> 
> Internal kernel interfaces are another story, they are not fixed and are
> allways subject to change and not limited to v4l.  See the major pci
> subsystem changes from 2.2 => 2.4 for example.  The API visible to the
> application matters, this must stay backward compatible.

You are starting to see my point. I want API flexible enough not to
require #ifdef's each time a new feature is added. Driver is an
application in it's own right. While 2.2=>2.4 change is ok, I really
dislike such changes in the _stable_ kernel. 

> 
> > > I can't see why it is a problem to add a new header or new ioctls to
> > > a existing header file.  I like it this way, because the kernel headers
> > > with all the #defines and structs are providing at least a minimum of
> > > documentation.  I do often read header files when programming stuff.
> > > 
> > 
> > Because we don't know which interface is best until we experiment with
> > it. And I can't experiment without people being able to test. And the
> > easier it is to compile and install code the more testers (and developers
> > !) we get.
> 
> I can't see why ioctls don't allow you to experiment.
> 

Well, could you add then an ioctl to the current kernel so I can
experiment with it ? I want it in the kernel headers so that the
applications will compile whether or not the driver sourcecode has been
installed. Nothing fancy, just an ioctl number something along

#define V4L_VLADIMIR_DERGACHEV_PRIVATE_IOCTL     _IOWR(....)

I'll take care of the structs ;)

> > > Of course old applications don't know about the new stuff (neither the
> > > temporary nor the final versions which make in into the official API
> > > some day maybe).  But I don't see how your approach handles this better:
> > > Applications still need to be hacked to use the new stuff ...
> > > 
> > 
> > The way you write device specific appliacation is by including kernel
> > headers. If the stuff we want is not there makes a lot trouble for
> > installing and maintaining code.
> 
> No, it doesn't.  I never had trouble with xawtv.  I simply shipped a
> private copy of videodev.h with the xawtv tarball (which allowed to
> build it without hassle on 2.0.x systems which had no linux/videodev.h
> yet).
> 

Well, if it worked for you - good. I am not going that way, the versioning
issues alone are reason enough.

> > > You can't.  But I don't see why this is a issue:  The only thing a
> > > application can handle easily are controls like contrast/hue where the
> > > only thing a application needs to do is to map it to a GUI and let the
> > > user understand and adjust stuff.  The other stuff has way to much
> > > non-trivial dependences, I doubt a application can blindly use new
> > > driver features.
> > 
> > Have you ever thought that the reason we only use these controls is
> > because they are the only ones easy to implement now ?
> 
> I doubt this is just a implementation issue.  I don't believe in AI.
> 

:)))) There is a thin line between real intellegence and clever
algorithm. My bet is that a clever algorithm will suffice.

> > > > > >   * can cause problems with different compilers
> > > > > 
> > > > > Then your compiler is buggy.
> > > > 
> > > > No, I may have simply used different compilers for the kernel and the
> > > > application.
> > > 
> > > I doubt that.
> > 
> > You are kidding, aren't you ? Most distributions now come out with egcs
> > compiler 1.1.x "recommended for compiling" kernels and something newer.
> 
> I doubt that this creates trouble, not that you might use different
> compilers (I know RH ships or used to ship kgcc ...).
> 

It might. I have no desire of debugging such things.

> > > > What if the driver does not support counting dropped frames ?
> > > 
> > > -EINVAL or something like that.
> > 
> > But supports every other field.
> 
> Can't happen, there is VIDIOC_G_PERF only for this performance
> monitoring ...

You mean you won't allow a driver like that. This is precisely the
inflexibility I was talking about.

> 
> > > > What if there is a control with no min/max limits ?
> > > 
> > > Do you have a example?
> > > 
> > 
> > Overlay color key - this is basically an RGB pixel value.
> 
> 0x000000 => 0xffffff ?

If you use alpha value too you'll need all 32bits. And imagine a GUI
slider for that.. ;) 

> 
> > I would prefer minimum effort on the part of driver writer ;) At the
> > moment all I see in bttv and my own code for select is looking on some
> > already existing fields. Heck, the code is very similar to what needs to
> > be checked for a blocking/non-blocking read. Why duplicate it ?
> 
> bttv's poll function is very short for exactly this reason.  Basically I
> only have to call poll_wait with the (existing) wait queue which the IRQ
> handler will wake up once the frame capture is finished.
> 

Ohh, bttv _is_ good. I only argue against v4l :)

                            Vladimir Dergachev

> > Also, v4l select will not work (as far as I understand) when the driver is
> > using memory-mapped buffers.
> 
> v4l2 expects drivers to support select for the mmap() case too.

> 
>   Gerd
> 
> -- 
> Netscape is unable to locate the server localhost:8000.
> Please check the server name and try again.
> 

Re: [V4L] Re: [RFC] alternative kernel multimedia API

[Gerd Knorr]

> > Internal kernel interfaces are another story, they are not fixed and are
> > allways subject to change and not limited to v4l.  See the major pci
> > subsystem changes from 2.2 => 2.4 for example.  The API visible to the
> > application matters, this must stay backward compatible.
> 
> You are starting to see my point. I want API flexible enough not to
> require #ifdef's each time a new feature is added. Driver is an
> application in it's own right. While 2.2=>2.4 change is ok, I really
> dislike such changes in the _stable_ kernel. 

Bad luck for you.  See Linus comments about binary-only modules +
compatibility.  There is simply no way around a ifdef here and there for
different kernel versions because a driver IMHO should be maintained for
both current stable and current hacker kernels.

> > I can't see why ioctls don't allow you to experiment.
> > 
> 
> Well, could you add then an ioctl to the current kernel so I can
> experiment with it ? I want it in the kernel headers so that the
> applications will compile whether or not the driver sourcecode has been
> installed. Nothing fancy, just an ioctl number something along
> 
> #define V4L_VLADIMIR_DERGACHEV_PRIVATE_IOCTL     _IOWR(....)
> 
> I'll take care of the structs ;)

Why this needs to be in the kernel?  Simply ship a copy of the header
file with both application and driver or require the driver being
installed to build the application.  Once you've worked out good,
working interfaces they can go into the kernel headers.  You don't need
that for experimental stuff.

> > No, it doesn't.  I never had trouble with xawtv.  I simply shipped a
> > private copy of videodev.h with the xawtv tarball (which allowed to
> > build it without hassle on 2.0.x systems which had no linux/videodev.h
> > yet).
> > 
> 
> Well, if it worked for you - good. I am not going that way,

If you want to put yourself in trouble ...

> the versioning
> issues alone are reason enough.

With argument passing using strings instead of structs you have the
versioning issues to, they only show up in other ways.  I'd expect it
becomes harder to debug because the failures are more subtile.  With a
obsolete ioctl struct you likely get back -EINVAL, which is quite
obvious if the application does sane error checking.  Or the application
doesn't even compile.  Both are IMHO much better than some stange
behaviour at runtime, which is the failure pattern I'd expect with API
changes when passing parameters using strings.

> > I doubt this is just a implementation issue.  I don't believe in AI.
> > 
> 
> :)))) There is a thin line between real intellegence and clever
> algorithm. My bet is that a clever algorithm will suffice.

I don't believe this until I've seen that working in real live.

> > > I would prefer minimum effort on the part of driver writer ;) At the
> > > moment all I see in bttv and my own code for select is looking on some
> > > already existing fields. Heck, the code is very similar to what needs to
> > > be checked for a blocking/non-blocking read. Why duplicate it ?
> > 
> > bttv's poll function is very short for exactly this reason.  Basically I
> > only have to call poll_wait with the (existing) wait queue which the IRQ
> > handler will wake up once the frame capture is finished.
> > 
> 
> Ohh, bttv _is_ good. I only argue against v4l :)

But bttv supports the v4l(2) API.  If bttv can handle the poll without
much extra code, what exactly was the point in arguing against the
select support defined in the v4l2 API?

  Gerd

Re: [V4L] Re: [RFC] alternative kernel multimedia API

[volodya]

On Mon, 5 Nov 2001, Gerd Knorr wrote:

> > > Internal kernel interfaces are another story, they are not fixed and are
> > > allways subject to change and not limited to v4l.  See the major pci
> > > subsystem changes from 2.2 => 2.4 for example.  The API visible to the
> > > application matters, this must stay backward compatible.
> > 
> > You are starting to see my point. I want API flexible enough not to
> > require #ifdef's each time a new feature is added. Driver is an
> > application in it's own right. While 2.2=>2.4 change is ok, I really
> > dislike such changes in the _stable_ kernel. 
> 
> Bad luck for you.  See Linus comments about binary-only modules +
> compatibility.  There is simply no way around a ifdef here and there for
> different kernel versions because a driver IMHO should be maintained for
> both current stable and current hacker kernels.

The change I was talking about has occured someplace between 2.4.2 and
2.4.9. On the other hand some interface did not change at all - for
example serial devices /dev/ttySx. I do not see anything too special to
video capture to warrant constanly changing interfaces.

> 
> > > I can't see why ioctls don't allow you to experiment.
> > > 
> > 
> > Well, could you add then an ioctl to the current kernel so I can
> > experiment with it ? I want it in the kernel headers so that the
> > applications will compile whether or not the driver sourcecode has been
> > installed. Nothing fancy, just an ioctl number something along
> > 
> > #define V4L_VLADIMIR_DERGACHEV_PRIVATE_IOCTL     _IOWR(....)
> > 
> > I'll take care of the structs ;)
> 
> Why this needs to be in the kernel?  Simply ship a copy of the header
> file with both application and driver or require the driver being
> installed to build the application.  Once you've worked out good,
> working interfaces they can go into the kernel headers.  You don't need
> that for experimental stuff.

And what am I to do if someone introduces the exact same ioctl number into
the kernel ? I will get instant breakage. People will start saying: this
does not work with kernele 2.4.(N+x). So, I'll change the number and will
get bugreports of the kind "it does not work with 2.4.(N-1-y)". I do not
want that.

> 
> > > No, it doesn't.  I never had trouble with xawtv.  I simply shipped a
> > > private copy of videodev.h with the xawtv tarball (which allowed to
> > > build it without hassle on 2.0.x systems which had no linux/videodev.h
> > > yet).
> > > 
> > 
> > Well, if it worked for you - good. I am not going that way,
> 
> If you want to put yourself in trouble ...
> 
> > the versioning
> > issues alone are reason enough.
> 
> With argument passing using strings instead of structs you have the
> versioning issues to, they only show up in other ways.  I'd expect it
> becomes harder to debug because the failures are more subtile.  With a
> obsolete ioctl struct you likely get back -EINVAL, which is quite
> obvious if the application does sane error checking.  Or the application
> doesn't even compile.  Both are IMHO much better than some stange

This is a separate issue.. Just keep in mind that there are plenty of
applications that ignore return values from ioctl's.

> behaviour at runtime, which is the failure pattern I'd expect with API
> changes when passing parameters using strings.
> 
> > > I doubt this is just a implementation issue.  I don't believe in AI.
> > > 
> > 
> > :)))) There is a thin line between real intellegence and clever
> > algorithm. My bet is that a clever algorithm will suffice.
> 
> I don't believe this until I've seen that working in real live.

This is something I agree with ;) I posted the RFC to get some feedback
and check whether I missed something important. It does not look this way 
and the question boiled down to one API versus another. I"ll try
implementing the ideas from RFC.. Once this is ready I'll post it for
further discussion.

                 thanks for prompt replies

                                  Vladimir Dergachev

> 
> > > > I would prefer minimum effort on the part of driver writer ;) At the
> > > > moment all I see in bttv and my own code for select is looking on some
> > > > already existing fields. Heck, the code is very similar to what needs to
> > > > be checked for a blocking/non-blocking read. Why duplicate it ?
> > > 
> > > bttv's poll function is very short for exactly this reason.  Basically I
> > > only have to call poll_wait with the (existing) wait queue which the IRQ
> > > handler will wake up once the frame capture is finished.
> > > 
> > 
> > Ohh, bttv _is_ good. I only argue against v4l :)
> 
> But bttv supports the v4l(2) API.  If bttv can handle the poll without
> much extra code, what exactly was the point in arguing against the
> select support defined in the v4l2 API?
> 
>   Gerd
> 

Re: [V4L] Re: [RFC] alternative kernel multimedia API

[Gerd Knorr]

> > different kernel versions because a driver IMHO should be maintained for
> > both current stable and current hacker kernels.
> 
> The change I was talking about has occured someplace between 2.4.2 and
> 2.4.9. On the other hand some interface did not change at all - for
> example serial devices /dev/ttySx. I do not see anything too special to
> video capture to warrant constanly changing interfaces.

That was me in kernel 2.4.3 IIRC.  Added one more argument to allow
drivers to ask for specific minor numbers (so you can give your devices
fixed minor numbers using insmod options).  And this has _NOTHING_ to do
with the API visible to the applications.


> > Why this needs to be in the kernel?  Simply ship a copy of the header
> > file with both application and driver or require the driver being
> > installed to build the application.  Once you've worked out good,
> > working interfaces they can go into the kernel headers.  You don't need
> > that for experimental stuff.
> 
> And what am I to do if someone introduces the exact same ioctl number into
> the kernel ? I will get instant breakage. People will start saying: this
> does not work with kernele 2.4.(N+x). So, I'll change the number and will
> get bugreports of the kind "it does not work with 2.4.(N-1-y)". I do not
> want that.

Such clashes shouldn't happen as v4l has ioctl number ranges for driver
private stuff which can be used for such tests and shouldn't cause
clashes with new, official ioctls.

Beside that I don't see why breaking applications is a problem for
_experimental_ interfaces.  On the one hand you want to have the
flexibility to change interfaces easily to test them, on the other hand
you care alot about compatibility and stuff.  You can't get both, I
don't see a way to do that without making either the drivers or the
applications (or both) very complex.

That is the price users will have to pay for playing with bleeding edge
stuff.

> > becomes harder to debug because the failures are more subtile.  With a
> > obsolete ioctl struct you likely get back -EINVAL, which is quite
> > obvious if the application does sane error checking.  Or the application
> > doesn't even compile.  Both are IMHO much better than some stange
> 
> This is a separate issue.. Just keep in mind that there are plenty of
> applications that ignore return values from ioctl's.

s/applications/broken applications/

  Gerd

-- 
Netscape is unable to locate the server localhost:8000.
Please check the server name and try again.

Re: [V4L] Re: [RFC] alternative kernel multimedia API

[volodya]

On Mon, 5 Nov 2001, Gerd Knorr wrote:

> > > different kernel versions because a driver IMHO should be maintained for
> > > both current stable and current hacker kernels.
> > 
> > The change I was talking about has occured someplace between 2.4.2 and
> > 2.4.9. On the other hand some interface did not change at all - for
> > example serial devices /dev/ttySx. I do not see anything too special to
> > video capture to warrant constanly changing interfaces.
> 
> That was me in kernel 2.4.3 IIRC.  Added one more argument to allow
> drivers to ask for specific minor numbers (so you can give your devices
> fixed minor numbers using insmod options).  And this has _NOTHING_ to do
> with the API visible to the applications.
> 

It has to do with the API visible to the driver.

> 
> > > Why this needs to be in the kernel?  Simply ship a copy of the header
> > > file with both application and driver or require the driver being
> > > installed to build the application.  Once you've worked out good,
> > > working interfaces they can go into the kernel headers.  You don't need
> > > that for experimental stuff.
> > 
> > And what am I to do if someone introduces the exact same ioctl number into
> > the kernel ? I will get instant breakage. People will start saying: this
> > does not work with kernele 2.4.(N+x). So, I'll change the number and will
> > get bugreports of the kind "it does not work with 2.4.(N-1-y)". I do not
> > want that.
> 
> Such clashes shouldn't happen as v4l has ioctl number ranges for driver
> private stuff which can be used for such tests and shouldn't cause
> clashes with new, official ioctls.

I did not know that - thanks. Where do I find notes on this ?

> 
> Beside that I don't see why breaking applications is a problem for
> _experimental_ interfaces.  On the one hand you want to have the

It is a problem because I want as many people as possible to try them.
This is the only way to work out installation dependent bugs. There is a
lot of variety out there: Redhat, Mandrake, Slackware, Suse, ix86,
PowerPC, Alpha, Sparc.. Each is a little different.

> flexibility to change interfaces easily to test them, on the other hand
> you care alot about compatibility and stuff.  You can't get both, I
> don't see a way to do that without making either the drivers or the
> applications (or both) very complex.

Now here you are wrong. C have not changed in a while and you can still
write any programs in it ;) As for complexity.. I don't mind 10000 line
file if it is backed up by good algorithm. The good news is that with this
approach we separate interface stuff from driver dependent stuff - and,
hence, the most complex part can be easily tested.

> 
> That is the price users will have to pay for playing with bleeding edge
> stuff.
> 
> > > becomes harder to debug because the failures are more subtile.  With a
> > > obsolete ioctl struct you likely get back -EINVAL, which is quite
> > > obvious if the application does sane error checking.  Or the application
> > > doesn't even compile.  Both are IMHO much better than some stange
> > 
> > This is a separate issue.. Just keep in mind that there are plenty of
> > applications that ignore return values from ioctl's.
> 
> s/applications/broken applications/

The line here is even finer than between AI and clever algorithm ;)

                                  Vladimir Dergachev

> 
>   Gerd
> 
> -- 
> Netscape is unable to locate the server localhost:8000.
> Please check the server name and try again.
> 

Re: [V4L] Re: [RFC] alternative kernel multimedia API

[Gerd Knorr]

> > drivers to ask for specific minor numbers (so you can give your devices
> > fixed minor numbers using insmod options).  And this has _NOTHING_ to do
> > with the API visible to the applications.
> 
> It has to do with the API visible to the driver.

Yes.  Why do you mix the two?  These are completely separate issues.

> > Such clashes shouldn't happen as v4l has ioctl number ranges for driver
> > private stuff which can be used for such tests and shouldn't cause
> > clashes with new, official ioctls.
> 
> I did not know that - thanks. Where do I find notes on this ?

BASE_VIDIOCPRIVATE is defined in videodev.h and used by various drivers
in drivers/media/video

> > Beside that I don't see why breaking applications is a problem for
> > _experimental_ interfaces.  On the one hand you want to have the
> 
> It is a problem because I want as many people as possible to try them.
> This is the only way to work out installation dependent bugs. There is a
> lot of variety out there: Redhat, Mandrake, Slackware, Suse, ix86,
> PowerPC, Alpha, Sparc.. Each is a little different.

And how installation issues are related to API design / testing?

> > flexibility to change interfaces easily to test them, on the other hand
> > you care alot about compatibility and stuff.  You can't get both, I
> > don't see a way to do that without making either the drivers or the
> > applications (or both) very complex.
> 
> Now here you are wrong. C have not changed in a while and you can still
> write any programs in it ;) As for complexity.. I don't mind 10000 line
> file if it is backed up by good algorithm. The good news is that with this
> approach we separate interface stuff from driver dependent stuff - and,
> hence, the most complex part can be easily tested.

I doubt this.  IMHO the complex part isn't the read/write interface and
the string parsing (I'd expect this can easily separated out into some
kind of library / kernel module / whatever).  The complex part is to
keep the backward compatibility while changing/improving the interfaces
(which is one of your goals with the new approach, right?), and I don't
see a way to handle that in generic, driver-independant code ...

  Gerd

-- 
Netscape is unable to locate the server localhost:8000.
Please check the server name and try again.

Re: [V4L] Re: [RFC] alternative kernel multimedia API

[Mark McClelland]

I realize Gerd already replied to this, but I just had to throw in my 2 
cents...

volodya@mindspring.com wrote:

>On Mon, 29 Oct 2001, Gerd Knorr wrote:
>
>I already got a report from the user that in his kernel
>video_register_device has 2 arguments and not 3. This is pain to deal with
>in drivers distributed separately from kernel tree.
>
It's an internal kernel interface, not an ioctl. Yeah, it's a PITA, but 
that's the price of progress. Just #ifdef it like I did in 
http://alpha.dyndns.org/ov511/download/ov511.c

>The way you write device specific appliacation is by including kernel
>headers. If the stuff we want is not there makes a lot trouble for
>installing and maintaining code.
>
That's flat-out wrong. Read the linux-kernel archives and see what Linus 
thinks about including kernel headers, esp. regarding glibc. Apps should 
keep a private copy of the header(s). This is what I do with my (yet to 
materialize) ov511-control app, and it works OK, even across ioctl changes.

>>You can't.  But I don't see why this is a issue:  The only thing a
>>application can handle easily are controls like contrast/hue where the
>>only thing a application needs to do is to map it to a GUI and let the
>>user understand and adjust stuff.  The other stuff has way to much
>>non-trivial dependences, I doubt a application can blindly use new
>>driver features.
>>
>Have you ever thought that the reason we only use these controls is
>because they are the only ones easy to implement now ?
>
What I don't understand is how will your driver implement these controls 
in a generic V4L3  GUI control app automatically? No matter how powerful 
the semantic information you give to the app is, it can still only build 
interfaces from standard GUI components that it already knows about. The 
app cannot build a gamma curve control on its own. If it could, we 
wouldn't need programmers anymore :)

So, the real issue here is not whether a string-parsing API is 
ultimately more powerful than ioctls. It probably is, but at the expense 
of code size, speed, and things like:

SET GAMMA_TABLE_ENTRY 56742="exec("/bin/sh")"

The real issue is which is more maintainable. Since you have to update 
the kernel when you add new features to your (in-kernel) driver, adding 
#defines to videodev.h doesn't take much extra work.

>>>      * what do I do if someone uses the same ioctl in the kernel source
>>>
>>???
>>
>Same ioctl number, in the kernel headers..
>
Conflicts can happen, but they will usually be caught when merging 
patches into the mainline kernel. If not, then it is probably because 
someone screwed up and redefined ioctls from public ranges in their 
driver's header. That's what private ranges are for.

The same thing can happen with any name/number space, including the one 
you have devised.

>>>What if the driver does not support counting dropped frames ?
>>>
>>-EINVAL or something like that.
>>
>But supports every other field.
>
Then don't report any dropped frames (hardcode it to zero). If the 
driver can't detect dropped frames, then how will telling the app that 
it can't make any difference? The app can detect it via the frames' 
timecodes if it has to.

>>>What if there is a control with no min/max limits ?
>>>
>>Do you have a example?
>>
>Overlay color key - this is basically an RGB pixel value.
>
Export it as three integers, with range of 0-255. Remember, most 
settings values ultimately end up in integer (or less) sized hardware 
registers. Ones that don't are special cases, which can be set via a 
private ioctl. Regardless of how settings are communicated between the 
control-app and driver, you will have to build a custom GUI component 
for any oddball setting.

>>Multiple choice controls have strings for each value.
>>
>With no way to pass strings (and their meaning) from the driver to the
>user applications.
>
See struct v4l2_querymenu

>>>Can I set/read gamma values ?
>>>
>>Gamma values yes (tables no).
>>
While I appreciate the creativity and originality of your API proposal 
(not to mention the time you took to write it; you'd have to pay me $$$ 
to do that much work), I must ask: Why should we dump V4L/V4L2 and 
rewrite all of our apps and drivers, just to save you the trouble of 
adding one (private) ioctl? Is your hardware really that difficult to 
fit into V4L2?

-- 
Mark McClelland
mmcclell@bigfoot.com
PGP public key fingerprint: 317C 58AC 1B39 2AB0 AB96  EB38 0B6F 731F 3573 75CC

Re: [V4L] Re: [RFC] alternative kernel multimedia API

[Albert D. Cahalan]

Mark McClelland writes:
> volodya@mindspring.com wrote:
>> On Mon, 29 Oct 2001, Gerd Knorr wrote:

>>> You can't.  But I don't see why this is a issue:  The only thing a
>>> application can handle easily are controls like contrast/hue where the
>>> only thing a application needs to do is to map it to a GUI and let the
>>> user understand and adjust stuff.  The other stuff has way to much
>>> non-trivial dependences, I doubt a application can blindly use new
>>> driver features.
>>
>> Have you ever thought that the reason we only use these controls is
>> because they are the only ones easy to implement now ?
>
> What I don't understand is how will your driver implement these controls 
> in a generic V4L3  GUI control app automatically? No matter how powerful 
> the semantic information you give to the app is, it can still only build 
> interfaces from standard GUI components that it already knows about. The 
> app cannot build a gamma curve control on its own. If it could, we 
> wouldn't need programmers anymore :)

The driver provides this:  /proc/v4l3/vid0/gamma.java

:-)

The very idea of a gamma table is featuritis. Just a single number
will do for most anyone. You get from 0.01 to 2.55 and "off" with
just 8 bits.

If you want to get fancy, I guess you need:

1 gamma value per primary
3 tri-stimulus values for each primary  ("What color is red?")
3 tri-stimulus values for the whitepoint (maybe)
1 black level for each primary

That is 18 values at most.

Re: [V4L] Re: [RFC] alternative kernel multimedia API

[volodya]

On Sun, 4 Nov 2001, Albert D. Cahalan wrote:

> Mark McClelland writes:
> > volodya@mindspring.com wrote:
> >> On Mon, 29 Oct 2001, Gerd Knorr wrote:
> 
> >>> You can't.  But I don't see why this is a issue:  The only thing a
> >>> application can handle easily are controls like contrast/hue where the
> >>> only thing a application needs to do is to map it to a GUI and let the
> >>> user understand and adjust stuff.  The other stuff has way to much
> >>> non-trivial dependences, I doubt a application can blindly use new
> >>> driver features.
> >>
> >> Have you ever thought that the reason we only use these controls is
> >> because they are the only ones easy to implement now ?
> >
> > What I don't understand is how will your driver implement these controls 
> > in a generic V4L3  GUI control app automatically? No matter how powerful 

kmultimedia, not V4L3. The interface is meant to be used with _all_
multimedia devices, not just video ones.

> > the semantic information you give to the app is, it can still only build 
> > interfaces from standard GUI components that it already knows about. The 
> > app cannot build a gamma curve control on its own. If it could, we 
> > wouldn't need programmers anymore :)
> 
> The driver provides this:  /proc/v4l3/vid0/gamma.java
> 
> :-)
> 
> The very idea of a gamma table is featuritis. Just a single number
> will do for most anyone. You get from 0.01 to 2.55 and "off" with
> just 8 bits.

And the color correction is unnecessary too. In fact I see a big
opportunity for you to try selling IBM PCs with DOS and generic monitors
to artistic community. Just imagine the slogan "Be free from the confines
of color!" ;)

But on a more serious note: the hardware already has it. All I want is to
export the interface to the userspace so that users can decide whether
they want to or not.

> 
> If you want to get fancy, I guess you need:
> 
> 1 gamma value per primary
> 3 tri-stimulus values for each primary  ("What color is red?")
> 3 tri-stimulus values for the whitepoint (maybe)
> 1 black level for each primary
> 
> That is 18 values at most.

See ? You are in the same pit as everyone else. You are trying to
construct a single model that fits everything. This is the exact 
problem that makes Windows so awkward: if you are doing what Microsoft
thought you might do you are ok(more or less), if you are trying something
new you are screwed.

Leave the choice to the user and application programmer. Make it easy for
them to support it.

Regardless, I am going to give it a try implementing it. It looks like the
main argument boiled down to: why bother while we have v4l2 and if you
really want we can extend it, which is more or less what I expected, I
don't believe any further discussion will make sense until there is some
code to look at.

Big thanks to everyone for comments :)

                              Vladimir Dergachev

> 
>