RFCv1: v4l-dvb development models & old kernel support

RFCv1: v4l-dvb development models & old kernel support

[Hans Verkuil]

RFC: v4l-dvb Development Models
Version 1, February 21st 2009
-------------------------------

Introduction
------------

The current v4l-dvb development model is based on a mercurial repository. 
Patches are fed into the repo by developers, the v4l-dvb maintainer will 
backport relevant patches from the upstream git repository into the v4l-dvb 
master and push essential fixes from the v4l-dvb repo into the upstream git 
repo. When a new kernel merge window opens all our accumulated changes are 
fed into the new kernel and the cycle starts again.

What makes our v4l-dvb special is the ability to support older kernels. 
Through the use of smart compat headers, scripts and #if KERNEL_VERSION 
lines in the code the backwards compatibility support is done in a way that 
keeps the code relatively untouched. When our code is fed to the kernel, 
all the compat code is stripped so none of this is visible upstream.

The obvious advantage is that people can use and test our drivers on older 
kernels without having to upgrade to the latest bleeding edge kernel. 
Currently we support kernels from 2.6.16 onwards.

However, the v4l-dvb tree has seen considerable growth. Below is a simple 
statistic:

$ du -bs linux-2.6.*/drivers/media/
 5569219 linux-2.6.16.61/drivers/media/
 7189484 linux-2.6.17.14/drivers/media/
 7910318 linux-2.6.18.8/drivers/media/
 8081530 linux-2.6.19.5/drivers/media/
 8419446 linux-2.6.20.21/drivers/media/
 8556067 linux-2.6.21.7/drivers/media/
 9121077 linux-2.6.22.19/drivers/media/
 9334386 linux-2.6.23.12/drivers/media/
 9667264 linux-2.6.24.7/drivers/media/
10121497 linux-2.6.25.11/drivers/media/
11244052 linux-2.6.26/drivers/media/
12322637 linux-2.6.27/drivers/media/
12947608 linux-2.6.28/drivers/media/
13719917 linux-2.6.29-rc4/drivers/media/

So the total amount of code in the 2.6.29 tree is about 2.5 times that of 
the 2.6.16 tree and that tree is almost three years old.

The high rate of changes and new drivers means that keeping up the backwards 
compatibility becomes an increasingly heavy burden.

This leads to two questions:

1) Can we change our development model in such a way that this burden is 
reduced?
2) How far back do we want to support older kernels anyway?

These questions are related, since changes in the development model has 
implications for the answer to the second question.


1: Alternatives to the development model
----------------------------------------

I see the following options:

A) Keep our current model. This also keeps the way we do our backwards 
compatibility unchanged.

B) Switch to a git tree that tracks Linus' tree closely and we drop 
backwards compatibility completely.

C) Switch to the ALSA approach (http://git.alsa-project.org/).

ALSA uses three trees: alsa-kernel tracks Linus' git tree, alsa-kmirror 
contains only the alsa part (similar to the layout of our repository), and 
alsa-driver is equal to alsa-kmirror, but with compatibility patches.

If I understand their method correctly new changes are pushed to alsa-kernel 
and a script merges the changes into alsa-kmirror. Patches meant for 
alsa-specific files (the equivalent to our v4l/scripts or v4l2-spec 
directories) have to be pushed to alsa-kmirror directly because there is of 
course no equivalent in alsa-kernel.

Every so often the alsa-driver repository is merged with the latest changes 
from alsa-kmirror and someone goes in, checks what broke this time and 
fixes any problems manually through the creation of patches.

D) Anyone? I can't think of other serious alternatives.


1.1 Pros and Cons
-----------------

A: Keep our current model

Pros:

- we know it works.
- having backwards compatibility is very helpful in testing the drivers, it 
makes it very easy to point testers to the latest code base and have them 
install it.

Cons:

- no direct access to a full kernel git tree which makes it hard to make 
changes that go outside the drivers/media directory (e.g. platform related 
changes).
- received patches against the latest git tree can be hard to apply if they 
conflict with our compat code in the source.
- over time the code becomes ugly and harder to maintain with all the 
backwards compatibility changes.
- the lack of a kernel git tree as the main repository makes it laborious 
for the v4l-dvb maintainer to create patches for the upstream git tree.
- it's a continuous effort to try and keep the compatibility working. 
Because the repository is such a quickly changing codebase any fixed compat 
issue is promptly replaced by new compat issues.


B: Switch to a git tree and drop compatibility completely

Pros:

- makes driver development and v4l-dvb maintenance very easy.
- no compatibility issues anymore, this saves a lot of time.
- ability to change kernel code outside the driver/media part.
- received patches against the latest git tree are easy to apply.

Cons:

- no compatibility means that the initial testbase will be reduced 
substantially since it will be too difficult for many users to install the 
bleeding-edge kernel. So real feedback won't come in until the code is 
released as part of the main distros kernels.
- the same is true for ourselves: we need to continuously upgrade our 
kernel, which is not always an option.


C: The ALSA model

Pros:

- ability to change kernel code outside the driver/media part.
- the core code contains no compatibility code, thus making it very clean.
- received patches against the latest git tree are easy to apply.
- sending patches upstream is much easier.
- if patches go in that would break compatibility, then that doesn't 
immediately affect the compatibility git tree since these patches aren't 
merged automatically.
- the compatibility patches do not have to be that concerned about the mess 
they make of the code since the patched code will never go upstream.
- for the compatibility tree we can reuse our existing compatibility 
mechanism. No need to reinvent the wheel.

Cons:

- someone has to go in on a regular basis and merge the latest changes into 
the compatibility tree and fix up everything that broke. This is a 
commitment that may be hard to keep up.
- it is a fair amount of work to convert our model into this new model.
- developers will probably have to run the latest kernel as well in order to 
test whether their changes can be compiled and run correctly on the latest 
git tree. This might be a Pro as well, since it will also allow you to 
catch bugs quicker.
- if a change breaks compat in the compatibility tree, then that will make 
it hard for end users to test. You probably need to make a private copy of 
the compat tree just to fix that up, allowing end users to test your code.


1.2 Overall conclusion
----------------------

Looking at the options I think that the ALSA model is a very good candidate. 
Since it uses the standard git tree the maintenance of the repository when 
it comes to 'interfacing' with the Linus' tree is much simplified. Since 
getting fixes and new drivers into the mainline kernel is our prime task it 
is right that this part should be easy.

I think it is also good that the compat tree is more reliable. Now our code 
breaks too often for older kernels due to new patches that were committed. 
Replacing this by a periodic merge-and-fix cycle will make it less fragile. 
And the fact that it isn't code that goes to the mainline kernel allows us 
to be less concerned about how ugly the patch will look like.

We do need some way of assigning the fixing of broken compatibility to 
several people. It is not a task that should fall to only one or two 
persons. Whoever broke it should probably also fix it, in particular if 
that person was the maintainer of the driver as well. It's a matter of 
sharing the pain.


2 How many kernels to support?
------------------------------

Currently we go back three years to kernel 2.6.16. This is tested daily by 
running a compile on all versions from 2.6.16 (and on multiple 
architectures as well). While a successful compile doesn't mean that it 
will also run, it actually is a pretty good indicator.

A quick check (v4l/scripts/gentree.pl 2.6.29 linux/ gentree; du -sb 
linux/drivers/ gentree/drivers/) shows that we have about 200 kB of 
compatibility code in the driver sources:

14249297        linux/drivers/
14055008        gentree/drivers/

Keeping support for older kernels should come with an expiry date as at some 
point in time the effort involved outweighs the benefits. It also keeps the 
amount of compat code down and you know that particularly ugly patches can 
eventually be removed. We are not paid to maintain this code, and we have 
better things to do with our time than to keep the maintenance of an old 
kernel up.

What are the benefits of having compatibility code?

1) It increases the test coverage: more users can try out the code since 
upgrading to a bleeding edge kernel is usually not an option.
2) It eases our own development efforts for pretty much the same reason: no 
need to upgrade to the latest git kernel to test your code.

The drawbacks are obvious:

1) it's a lot of work to keep the compatibility working.
2) sometimes really large changes are made in the kernel that makes it very 
hard to keep support older kernels at all.

Usually the changes are renamed data structures, fields that are 
removed/added or moved to another struct altogether. While annoying, it is 
usually not too hard to fix.

However, in the particular case of the i2c model that was changed in kernel 
2.6.22 (and that sparked this discussion), the changes are so intrusive 
that they could no longer be fixed with a simple #if KERNEL_VERSION. It 
would probably still be possible if we had only a handfew of i2c modules, 
but i2c is a integral part of the design of many devices and we currently 
have 50 of these to maintain (and more coming in all the time).

At the moment keeping the compatibility alive for kernels older than 2.6.22 
actually slows down development, both for us and for the i2c core. And due 
to the code needed to support these older kernels the upstream code is 
actually affected and it looks really weird in these newer kernels.

2.6.22 is now over 1.5 years old. It is readily available in all the major 
distros with the exception of the enterprise versions.

I would like to propose some rules that we can use to decide how far we go 
in supporting older kernels.

1) As long as the three major distros do not yet support a certain kernel, 
we obviously have to keep compatibility code in.

2) Support for kernels older than the oldest supported kernel by the three 
major distros should be dropped. This explicitly excludes enterprise 
kernels (more about that below).

3) In case of *major* technical problems we should have the option to drop 
support for kernels older than the oldest kernel supported by the latest 
distro releases. This is as a last resort only, but in all the time I've 
been developing for v4l-dvb the i2c core changes are the first that would 
qualify for this rule.

In my view enterprise distros are *not* our responsibility. First of all 
people pay for them, so if they want to have webcams supported they should 
go to their vendor. We are not paid for doing this, nor are we supported by 
the vendors of the distros. I am convinced that the group involved is 
small: I can't remember seeing any bug reports for kernels from enterprise 
distros here. And if there are a few, then it is such a small number that 
it really doesn't pay to accomodate them.

I think it is also completely fair to expect that users of old unsupported 
non-enterprise distros should upgrade to something newer first. If only 
because for many developers it will can be hard or impossible to even 
install an old kernel on there machine anymore.

Note that development model C would still make it possible to support 
enterprise kernels without compromising the code quality of the main tree. 
However, that will be a substantial amount of work. If RedHat or Suse 
thinks there is enough demand for that, then they are free to have one of 
their own developers to do the work. But that's not what we should spend 
our unpaid time on.

Currently the oldest supported kernels are:

Oldest supported Fedora kernel is 2.6.25 (Fedora 9): 
http://fedoraproject.org/wiki/Releases

Oldest supported openSUSE kernel is 2.6.22 (10.3). I don't know when the 
end-of-life due date is for this openSUSE release. It might not be too long 
from now. (I can't find a decent webpage with the openSUSE releases, does 
anyone have one?)

Oldest supported Ubuntu kernel is 2.6.22 (7.10): 
https://wiki.ubuntu.com/Releases. End of life for this one will be in April 
2009, after that the oldest kernel for Ubuntu will be 2.6.27 (8.10).

In my view these criteria strike a good balance between supporting our users 
so we can good test coverage, and limiting the effort involved in 
supporting the compatibility code. And they are also based on simple facts: 
whenever the oldest regularly supported kernel changes, we can go in and 
remove some of the compat code. No need for discussions, the rules are 
clear and consistent.

And luckily, since the oldest kernel currently in regular use is 2.6.22 that 
makes a very good argument for dropping the i2c compatibility mess.

In addition, limiting the number of supported kernels will ease the effort 
needed to switch development models, should we actually decide to do so.

Comments?

	Hans

-- 
Hans Verkuil - video4linux developer - sponsored by TANDBERG

Re: RFCv1: v4l-dvb development models & old kernel support

[wk]

Hans Verkuil wrote:
> Comments?
>
> 	Hans
>   

As only beeing reader of this list.., why not simply reduce the work load by

- reducing the number of supported kernel versions to five major 
versions? Currently 2.6.28 would mean down to 2.6.23,
this would be enough cover all nearly up-to-date distributions. Users 
from embedded devices are anyway mostly not able to compile or use newer 
drivers.

- not changing to git, already since this generates a lot of work
Not too far away dev was changed from cvs to hg, and already there some 
pieces are left over (for example that api stuff).

- force users to upgrade their kernel if (breaking these backward 
compat, and only if) *major* upgrades inside standard kernel would 
require a very huge amount of backporting, for example that i2c stuff.

I guess such solution would help immediately.

--Winfried

Re: RFCv1: v4l-dvb development models & old kernel support

[Trent Piepho]

On Sat, 21 Feb 2009, wk wrote:
> Hans Verkuil wrote:
> > Comments?
> >
> > 	Hans
> >
>
> As only beeing reader of this list.., why not simply reduce the work load by
>
> - reducing the number of supported kernel versions to five major
> versions? Currently 2.6.28 would mean down to 2.6.23,
> this would be enough cover all nearly up-to-date distributions. Users
> from embedded devices are anyway mostly not able to compile or use newer
> drivers.

Supporting versions between 2.6.23 and 2.6.28 isn't much more work than
supporting just 2.6.23 and 2.6.28.  There aren't many compat issues that
appeared in say 2.6.25 and disappeared in 2.6.27 that could be ignored.

Re: RFCv1: v4l-dvb development models & old kernel support

[Adam Baker]

On Saturday 21 February 2009, Hans Verkuil wrote:

> The high rate of changes and new drivers means that keeping up the
> backwards compatibility becomes an increasingly heavy burden.
>
> This leads to two questions:
>
> 1) Can we change our development model in such a way that this burden is
> reduced?

Possibly but even just spreading the burden better (and avoiding the compat 
code affecting the main tree in the case of i2c) would be a worthwhile 
change.

> 2) How far back do we want to support older kernels anyway?
>

To the point that the effort expended on the compat work is balanced by the 
benefit of more testers.

> These questions are related, since changes in the development model has
> implications for the answer to the second question.
>
>
> 1: Alternatives to the development model
> ----------------------------------------
>
> I see the following options:
>
> A) Keep our current model. This also keeps the way we do our backwards
> compatibility unchanged.
>
> B) Switch to a git tree that tracks Linus' tree closely and we drop
> backwards compatibility completely.
>
> C) Switch to the ALSA approach (http://git.alsa-project.org/).
>

Another example of this approach can be seen with the linux-wireless git tree. 
There is a description of the process at 
http://linuxwireless.org/en/users/Download#Developers

It might be a more relevant example as there are changes in 2.6.27 that make 
it difficult to support older kernels. They therefore made a decision at that 
point to restrict the automated backporting to 2.6.27 onwards and say patches 
will be accepted to the compat tree that covers 2.6.21 to 2.6.26 if a driver 
change is compatible but they must be manually flagged as being suitable 
(I've no idea how many are).

It does require one person (who isn't the main wireless maintainer) to be the 
maintainer of the compat tree.

> B: Switch to a git tree and drop compatibility completely
>
> Pros:
>
> - makes driver development and v4l-dvb maintenance very easy.
> - no compatibility issues anymore, this saves a lot of time.
> - ability to change kernel code outside the driver/media part.
> - received patches against the latest git tree are easy to apply.
>

Ensures that driver changes get tested in the kernel they will be released in 
so there is less chance of a change elsewhere breaking your change. Also 
means more people are testing pre release kernels so might have stopped the 
USB bug in 2.6.28 making it to the released kernel.

> Cons:
>
> - no compatibility means that the initial testbase will be reduced
> substantially since it will be too difficult for many users to install the
> bleeding-edge kernel. So real feedback won't come in until the code is
> released as part of the main distros kernels.
> - the same is true for ourselves: we need to continuously upgrade our
> kernel, which is not always an option.
>

It also means that a git pull can result in a long long rebuild if the 
upstream has just pulled a load of changes to other subsystems.

>
> 2 How many kernels to support?
> ------------------------------

> Keeping support for older kernels should come with an expiry date as at
> some point in time the effort involved outweighs the benefits.

I think the outweighs the benefits point is critical here and indicates what 
the break point should be.

>
> Oldest supported Ubuntu kernel is 2.6.22 (7.10):
> https://wiki.ubuntu.com/Releases. End of life for this one will be in April
> 2009, after that the oldest kernel for Ubuntu will be 2.6.27 (8.10).
>
You missed 8.04 which EOLs in April 2011 for Desktop use and uses 2.6.24


> In my view these criteria strike a good balance between supporting our
> users so we can good test coverage, and limiting the effort involved in
> supporting the compatibility code. And they are also based on simple facts:
> whenever the oldest regularly supported kernel changes, we can go in and
> remove some of the compat code. No need for discussions, the rules are
> clear and consistent.

I don't think anyone will (or need) actively track theses dates unless the 
code is causing them a headache but the rule is still a useful guide - if the 
compat code is causing you a problem and the 3 most popular distros have 
EOLed the relevant kernel it can be dropped without discussion by whoever 
administers the compat repository, otherwise it should be discussed on the ML 
first.

The only question then would be how to choose the 3. I don't think 1 and 2 are 
in much dispute but I suspect OpenSUSE is slugging it out for 3rd place with 
Debian.

>
> And luckily, since the oldest kernel currently in regular use is 2.6.22
> that makes a very good argument for dropping the i2c compatibility mess.
>

Unfortunately this all omits one important point, are there any key developers 
for whom dropping support for old kernels will cause them a problem which 
could reduce their productivity.  Mauro has stated that it would cause him a 
problem but I can't tell how big a problem it would really be.

Adam

Re: RFCv1: v4l-dvb development models & old kernel support

[hermann pitton]

Hi,

Am Samstag, den 21.02.2009, 23:47 +0000 schrieb Adam Baker:
> On Saturday 21 February 2009, Hans Verkuil wrote:
> 
> > The high rate of changes and new drivers means that keeping up the
> > backwards compatibility becomes an increasingly heavy burden.
> >
> > This leads to two questions:
> >
> > 1) Can we change our development model in such a way that this burden is
> > reduced?
> 
> Possibly but even just spreading the burden better (and avoiding the compat 
> code affecting the main tree in the case of i2c) would be a worthwhile 
> change.
> 
> > 2) How far back do we want to support older kernels anyway?
> >
> 
> To the point that the effort expended on the compat work is balanced by the 
> benefit of more testers.
> 
> > These questions are related, since changes in the development model has
> > implications for the answer to the second question.
> >
> >
> > 1: Alternatives to the development model
> > ----------------------------------------
> >
> > I see the following options:
> >
> > A) Keep our current model. This also keeps the way we do our backwards
> > compatibility unchanged.
> >
> > B) Switch to a git tree that tracks Linus' tree closely and we drop
> > backwards compatibility completely.
> >
> > C) Switch to the ALSA approach (http://git.alsa-project.org/).
> >
> 
> Another example of this approach can be seen with the linux-wireless git tree. 
> There is a description of the process at 
> http://linuxwireless.org/en/users/Download#Developers
> 
> It might be a more relevant example as there are changes in 2.6.27 that make 
> it difficult to support older kernels. They therefore made a decision at that 
> point to restrict the automated backporting to 2.6.27 onwards and say patches 
> will be accepted to the compat tree that covers 2.6.21 to 2.6.26 if a driver 
> change is compatible but they must be manually flagged as being suitable 
> (I've no idea how many are).
> 
> It does require one person (who isn't the main wireless maintainer) to be the 
> maintainer of the compat tree.
> 
> > B: Switch to a git tree and drop compatibility completely
> >
> > Pros:
> >
> > - makes driver development and v4l-dvb maintenance very easy.
> > - no compatibility issues anymore, this saves a lot of time.
> > - ability to change kernel code outside the driver/media part.
> > - received patches against the latest git tree are easy to apply.
> >
> 
> Ensures that driver changes get tested in the kernel they will be released in 
> so there is less chance of a change elsewhere breaking your change. Also 
> means more people are testing pre release kernels so might have stopped the 
> USB bug in 2.6.28 making it to the released kernel.
> 
> > Cons:
> >
> > - no compatibility means that the initial testbase will be reduced
> > substantially since it will be too difficult for many users to install the
> > bleeding-edge kernel. So real feedback won't come in until the code is
> > released as part of the main distros kernels.
> > - the same is true for ourselves: we need to continuously upgrade our
> > kernel, which is not always an option.
> >
> 
> It also means that a git pull can result in a long long rebuild if the 
> upstream has just pulled a load of changes to other subsystems.
> 
> >
> > 2 How many kernels to support?
> > ------------------------------
> 
> > Keeping support for older kernels should come with an expiry date as at
> > some point in time the effort involved outweighs the benefits.
> 
> I think the outweighs the benefits point is critical here and indicates what 
> the break point should be.
> 
> >
> > Oldest supported Ubuntu kernel is 2.6.22 (7.10):
> > https://wiki.ubuntu.com/Releases. End of life for this one will be in April
> > 2009, after that the oldest kernel for Ubuntu will be 2.6.27 (8.10).
> >
> You missed 8.04 which EOLs in April 2011 for Desktop use and uses 2.6.24
> 
> 
> > In my view these criteria strike a good balance between supporting our
> > users so we can good test coverage, and limiting the effort involved in
> > supporting the compatibility code. And they are also based on simple facts:
> > whenever the oldest regularly supported kernel changes, we can go in and
> > remove some of the compat code. No need for discussions, the rules are
> > clear and consistent.
> 
> I don't think anyone will (or need) actively track theses dates unless the 
> code is causing them a headache but the rule is still a useful guide - if the 
> compat code is causing you a problem and the 3 most popular distros have 
> EOLed the relevant kernel it can be dropped without discussion by whoever 
> administers the compat repository, otherwise it should be discussed on the ML 
> first.
> 
> The only question then would be how to choose the 3. I don't think 1 and 2 are 
> in much dispute but I suspect OpenSUSE is slugging it out for 3rd place with 
> Debian.
> 
> >
> > And luckily, since the oldest kernel currently in regular use is 2.6.22
> > that makes a very good argument for dropping the i2c compatibility mess.
> >
> 
> Unfortunately this all omits one important point, are there any key developers 
> for whom dropping support for old kernels will cause them a problem which 
> could reduce their productivity.  Mauro has stated that it would cause him a 
> problem but I can't tell how big a problem it would really be.
> 
> Adam
> --

all nice, but ...

First, we have the best out of kernel build and compat system ever.

This has reasons.

Second, Jean and Hans gave arguments for some limitations after we
previously worked in the other direction.

I and others tried to express, that we can't provide support for latest
drivers for commercial distributions endlessly and that there must be a
limit for what is released at kernel.org and how far they can fall back.

Even if they are most productive contributors, Greg has statistics,
they are it _not_ here at all.

I would leave this entirely to those who contributed, Mauro, Trent, Mike
and Hans to help Jean further.

In fact, we had a breakage first for 2.6.18 and later for below 2.6.22
and nobody did care anymore. I repeat that Hans caused his own problems
now going further down :)

Cheers,
Hermann  

Re: RFCv1: v4l-dvb development models & old kernel support

[kilgota]

On Sat, 21 Feb 2009, Adam Baker wrote:

> On Saturday 21 February 2009, Hans Verkuil wrote:
>
>> The high rate of changes and new drivers means that keeping up the
>> backwards compatibility becomes an increasingly heavy burden.
>>
>> This leads to two questions:
>>
>> 1) Can we change our development model in such a way that this burden is
>> reduced?
>
> Possibly but even just spreading the burden better (and avoiding the compat
> code affecting the main tree in the case of i2c) would be a worthwhile
> change.
>
>> 2) How far back do we want to support older kernels anyway?

<long snip>

>> B: Switch to a git tree and drop compatibility completely
>>
>> Pros:
>>
>> - makes driver development and v4l-dvb maintenance very easy.
>> - no compatibility issues anymore, this saves a lot of time.
>> - ability to change kernel code outside the driver/media part.
>> - received patches against the latest git tree are easy to apply.
>>
>
> Ensures that driver changes get tested in the kernel they will be released in
> so there is less chance of a change elsewhere breaking your change. Also
> means more people are testing pre release kernels so might have stopped the
> USB bug in 2.6.28 making it to the released kernel.

This is a valid counter-point to something I said earlier. OTOH, there is 
still the question about whether "modular" development has benefits which 
outweigh the monolithic approach. I had never thought about the issue 
myself until confronted with the way that it is done here. But there are 
benefits to the "modular" approach, too. I did not know them before trying 
it out.

>
>> Cons:
>>
>> - no compatibility means that the initial testbase will be reduced
>> substantially since it will be too difficult for many users to install the
>> bleeding-edge kernel. So real feedback won't come in until the code is
>> released as part of the main distros kernels.
>> - the same is true for ourselves: we need to continuously upgrade our
>> kernel, which is not always an option.
>>

Yes.

>
> It also means that a git pull can result in a long long rebuild if the
> upstream has just pulled a load of changes to other subsystems.

Again yes. One is trying to get work done, and someone else did big 
changes. Well, so you can't just compile blindly. You really do have to 
look at those changes. OK, so do "make menuconfig" and read some of the 
docs about the new stuff you never heard of before, some of which might 
make your machine work ever so much better -- or break it if you change 
this and don't change that, and you see where this is going. I said 
earlier that anyone who is a developer is pleased to keep current with the 
kernel, but at least that should mean that not more than once every couple 
of weeks should one be forced to spend time on that???

>
>>
>> 2 How many kernels to support?
>> ------------------------------
>
>> Keeping support for older kernels should come with an expiry date as at
>> some point in time the effort involved outweighs the benefits.
>
> I think the outweighs the benefits point is critical here and indicates what
> the break point should be.
>
>>
>> Oldest supported Ubuntu kernel is 2.6.22 (7.10):

This is a bit optimistic.

Matter of fact, I just bought a brand new eeePC in January, on which Asus 
chose to install Xandros. The response to uname -r is (I put this on a 
separate line in order to highlight it)

2.6.21.4-eeepc

Now, some might not think of Xandros as a leading distro. It certainly 
would not have been my first choice. The choice of such an old kernel 
confirms that impression. But the netbook hardware platform, I would say, 
is a rather important one. The point is, if one is going to start looking 
for kernels that are obviously too old to mess with but are in common use 
then one has to go back even beyond 2.6.22. If it were my choice, I 
wouldn't.

>> In my view these criteria strike a good balance between supporting our
>> users so we can good test coverage, and limiting the effort involved in
>> supporting the compatibility code. And they are also based on simple facts:
>> whenever the oldest regularly supported kernel changes, we can go in and
>> remove some of the compat code. No need for discussions, the rules are
>> clear and consistent.
>
> I don't think anyone will (or need) actively track theses dates unless the
> code is causing them a headache but the rule is still a useful guide - if the
> compat code is causing you a problem and the 3 most popular distros have
> EOLed the relevant kernel it can be dropped without discussion by whoever
> administers the compat repository, otherwise it should be discussed on the ML
> first.
>
> The only question then would be how to choose the 3. I don't think 1 and 2 are
> in much dispute but I suspect OpenSUSE is slugging it out for 3rd place with
> Debian.
>
>>
>> And luckily, since the oldest kernel currently in regular use is 2.6.22
>> that makes a very good argument for dropping the i2c compatibility mess.

Well, it is not the oldest kernel in "regular use" as I have just 
demonstrated. I like this pragmatic approach, but that is the trouble with 
it. OTOH, I can also point out that the eeePC runs beautifully on Slax 
from an SD card slid into the slot, and Slax uses 2.6.27.8 and furthermore 
quite recent copies of this project's software compile and install nicely 
on that "guest" operating system and run not only gspca cameras but also 
the built-in webcam. So, from this two points. They are already made, but 
the example ought to make both of them over again.

1. I was smart enough to install a distro on an SD card. I don't claim big 
credit for that. All of you who read this obviously are, too. So are lots 
of other people who do not frequent kernel-related lists. It was not even 
what I would call difficult to do that. The pont is, you did not and do 
not need to cater to those who would feel forced to use the 2.6.21.4 that 
came on the machine. Not at all. Those people are not in the first 
place the users who are going to test your new software by trying to 
compile it.

2. If in addition to everything else I would have been forced to compile a 
kernel right away on that machine, on the grounds that the 2.6.27.8 which 
came with Slax was not recent enough, and I need a current git tree 
instead, then the testing which I just reported as completely successful 
would probably not have happened. Not that I could not do that, too, but 
it starts to become a matter of time spent.

> Unfortunately this all omits one important point, are there any key developers
> for whom dropping support for old kernels will cause them a problem which
> could reduce their productivity.  Mauro has stated that it would cause him a
> problem but I can't tell how big a problem it would really be.

I will not claim to be a key developer. But I just gave an example in 
which dropping a 2.6.27.x kernel (which some would want now to call "old") 
would have reduced my productivity. I just tested this stuff in an 
environment and on a hardware platform where some of you might be pleased 
to know it all works.

Again, my solution is keep the development process modular. Unless there 
is some really sudden and huge change, nothing will suddenly break. One or 
two versions back will continue to work 95% of the time, at least. If 
there is a big change which will break everything which was done before 
day before yesterday, then publicize it and explain. I would hope that a 
solution like this could obviate the need for most of the #ifdef s for 
compatibility and save a lot of trouble.

Theodore Kilgore

Re: RFCv1: v4l-dvb development models & old kernel support

[Hans Verkuil]

On Sunday 22 February 2009 02:15:51 kilgota@banach.math.auburn.edu wrote:
> >> Oldest supported Ubuntu kernel is 2.6.22 (7.10):
>
> This is a bit optimistic.
>
> Matter of fact, I just bought a brand new eeePC in January, on which Asus
> chose to install Xandros. The response to uname -r is (I put this on a
> separate line in order to highlight it)
>
> 2.6.21.4-eeepc
>
> Now, some might not think of Xandros as a leading distro. It certainly
> would not have been my first choice. The choice of such an old kernel
> confirms that impression. But the netbook hardware platform, I would say,
> is a rather important one. The point is, if one is going to start looking
> for kernels that are obviously too old to mess with but are in common use
> then one has to go back even beyond 2.6.22. If it were my choice, I
> wouldn't.

I don't think these netbooks are relevant for us for the simple fact that 
the main use case of v4l-dvb on devices like this is the webcam, and that 
will obviously be supported by whatever linux version the manufactorer has 
installed.

But it does raise the point that if we decide to drop support for kernels < 
2.6.22 then it is probably a good idea to make a snapshot first so people 
can still have the option to upgrade their v4l-dvb, even though that 
version isn't maintained anymore by us.

Thank you for your feedback!

Regards,

	Hans

-- 
Hans Verkuil - video4linux developer - sponsored by TANDBERG

Re: RFCv1: v4l-dvb development models & old kernel support

[Hans Verkuil]

On Sunday 22 February 2009 00:47:46 Adam Baker wrote:
> On Saturday 21 February 2009, Hans Verkuil wrote:
> > The high rate of changes and new drivers means that keeping up the
> > backwards compatibility becomes an increasingly heavy burden.
> >
> > This leads to two questions:
> >
> > 1) Can we change our development model in such a way that this burden
> > is reduced?
>
> Possibly but even just spreading the burden better (and avoiding the
> compat code affecting the main tree in the case of i2c) would be a
> worthwhile change.
>
> > 2) How far back do we want to support older kernels anyway?
>
> To the point that the effort expended on the compat work is balanced by
> the benefit of more testers.
>
> > These questions are related, since changes in the development model has
> > implications for the answer to the second question.
> >
> >
> > 1: Alternatives to the development model
> > ----------------------------------------
> >
> > I see the following options:
> >
> > A) Keep our current model. This also keeps the way we do our backwards
> > compatibility unchanged.
> >
> > B) Switch to a git tree that tracks Linus' tree closely and we drop
> > backwards compatibility completely.
> >
> > C) Switch to the ALSA approach (http://git.alsa-project.org/).
>
> Another example of this approach can be seen with the linux-wireless git
> tree. There is a description of the process at
> http://linuxwireless.org/en/users/Download#Developers
>
> It might be a more relevant example as there are changes in 2.6.27 that
> make it difficult to support older kernels. They therefore made a
> decision at that point to restrict the automated backporting to 2.6.27
> onwards and say patches will be accepted to the compat tree that covers
> 2.6.21 to 2.6.26 if a driver change is compatible but they must be
> manually flagged as being suitable (I've no idea how many are).
>
> It does require one person (who isn't the main wireless maintainer) to be
> the maintainer of the compat tree.

This would basically mean making a snapshot of the v4l-dvb repository, 
calling it v4l-dvb-old and relying on people to update it with fixes. I did 
think about this myself but I thought it unlikely that the old tree would 
see much work, if at all. It's what they are admitting to on the wireless 
site as well. This could be an option if we are faced with an incompatible 
kernel change, but in this particular case it is my gut-feeling that 2.6.22 
is old enough that people can just upgrade to that release.

> > And luckily, since the oldest kernel currently in regular use is 2.6.22
> > that makes a very good argument for dropping the i2c compatibility
> > mess.
>
> Unfortunately this all omits one important point, are there any key
> developers for whom dropping support for old kernels will cause them a
> problem which could reduce their productivity.  Mauro has stated that it
> would cause him a problem but I can't tell how big a problem it would
> really be.

I'll start a poll. Let's see what the opinion is.

Regards,

	Hans

-- 
Hans Verkuil - video4linux developer - sponsored by TANDBERG

Re: RFCv1: v4l-dvb development models & old kernel support

[Andrea Venturi]

Hans Verkuil wrote:
> ...
> This would basically mean making a snapshot of the v4l-dvb repository, 
> calling it v4l-dvb-old and relying on people to update it with fixes. I did 
> think about this myself but I thought it unlikely that the old tree would 
> see much work, if at all. It's what they are admitting to on the wireless 
> site as well. This could be an option if we are faced with an incompatible 
> kernel change, but in this particular case it is my gut-feeling that 2.6.22 
> is old enough that people can just upgrade to that release.
>   

hi,
i know of some platforms where the linux kernel is stuck on a way older 
release as the HW producer is a bit picky (to say the least..).

i think about the embedded multimedia SOCs where linux is already 
winning hands down with regard to coverage and support BUT the industry 
still doesn't get really the idea behind the free software environment 
[*] and still gives away lots of binary blobs for the "proprietary HW" 
(so end users can tweak something but not really move over to modern 
kernels..).

here there are three examples i know of quite a bit:

- ST Microelectronics STi710x, DVB SOC, SH4 CPU based stuck on kernel 
2.6.17..

- Sigma Design SMP8634,  MIPS  based stuck on 2.6.15

- IBM Stb25xx  DVB SOC  PowerPc based stuck on 2.6.17

i'm sure there are really more like these and as they are targeting the 
multimedia environment (media center?), the large part of end users 
would like to add some gadget running on v4l-dvb drivers..

this is in my point of view at least a good reason to keep a snapshot of 
the latest v4l-dvb tree who has a bit of support for older kernel 
versions; the so called v4l-dvb-old idea already written about.

of course, the newer development should focus only on newer kernel to 
keep as low as possible the compatibility burden.

just my 2 cents. bye

Andrea Venturi



[*] it's of course the slow and painful path to consciousness of the 
limits of closed development model  when the open source and free (as 
freedom) scenario starts to reveal all the win-win convenience for all 
the parties involved, but it's same ol story and we are all well 
involved here in this kind of mindset, so don't want to repeat..

Re: RFCv1: v4l-dvb development models & old kernel support

[Tobias Stöber]

Hi together,

following this dicussion for some time, let me just add my solely 
personal view. I'll have to prepend that I'm only a user in terms of 
v4l-dvb, but do for instance quite a lot support for Eee PC netbooks. So 
some comments here in some sense urged me to jump into this discussion.

Hans Verkuil schrieb:
> On Sunday 22 February 2009 02:15:51 kilgota@banach.math.auburn.edu wrote:
>>>> Oldest supported Ubuntu kernel is 2.6.22 (7.10):
>> This is a bit optimistic.
>>
>> Matter of fact, I just bought a brand new eeePC in January, on which Asus
>> chose to install Xandros. The response to uname -r is (I put this on a
>> separate line in order to highlight it)
>>
>> 2.6.21.4-eeepc
>>
>> Now, some might not think of Xandros as a leading distro. It certainly
>> would not have been my first choice. The choice of such an old kernel
>> confirms that impression. But the netbook hardware platform, I would say,
>> is a rather important one. The point is, if one is going to start looking
>> for kernels that are obviously too old to mess with but are in common use
>> then one has to go back even beyond 2.6.22. If it were my choice, I
>> wouldn't.
> 
> I don't think these netbooks are relevant for us for the simple fact that 
> the main use case of v4l-dvb on devices like this is the webcam, and that 
> will obviously be supported by whatever linux version the manufactorer has 
> installed.

First to the "netbook" questions ...)

I suppose that you Hans don't use any of these so called netbooks?

Nevertheless these devices are common today, just for the fact, that 
they offer a good value for money, are powerful enough for all the day 
to day tasks a normal user may want to do and are sometimes the first 
ever experience of normal users with linux. Furthermore you have a 
device which weights around 1 kg and has roughly the size of an A5 piece 
of paper or slightly larger.

Besides the netbooks there are other devices, that share the same kind 
of linux distributions - or to make it more specific - say the Asus Eee 
Box models, that also comes (if they come with linux) with a Asus 
specific kind of Xandros linux, which itself is based on Debian etch and 
also newer parts of Debian lenny / sid.

Just looking at the people I know (sometimes consider as friends), 
nearly 1/3 of them own such a device. More than 3/4 of them sticks to 
the default linux, which certainly has its quirks and problems.

Using any other linux distribution may introduce a very recent kernel or 
other recent parts of software - but it just swaps Asus Xandros specific 
problems with that of the particular distro or kernel.

The "netbook" itself is - just my personal view of it - just a small 
kind of notebook device with sometime older (Celeron M353 cpu etc.) or 
in newer models Intel Atom based hardware.

This also means, that a lot of people just use it as a notebook, which 
means they use some sort of peripherals. Nearly all of the people I know 
owning a "netbook", don't use their netbook just as it came, all of them 
use some additional hardware, most prominently a lot using DVB usb 
devices, 3G modems, bluetooth or IrDA dongles, external video or webcam 
devices or adapters to provide legacy serial or parallel interface ports.

A lot of such devices are not supported by the kernel modules, that are 
provides with the netbook, so one has to build additional kernel 
modules. Sometime from other source, that the actual kernel source. In 
this case it is very handy to be able to compile a fresh hg checkout.

I don't want to bother you with a listing of cases in which people I 
know to use a specific device or me even helping them to get it running 
on such netbooks....

The point just is, that regardless of a device being classified as 
netbook or notebook it is a very naive point of view to say:

a) that it will only be used with to integrated software,

b) that a manufacturer and or distribution will jump every few weeks 
onto a recent kernel (which would be in some case a major task to 
install) and

c) that every normal user has the abilities to master such a - in some 
sense complicated task - as building and installing a complete kernel 
(building a v4l-dvb checkout is a lot more easier).

Second ...)

... you just to a more wider viewpoint, or even just try to understand 
that there may be just more things to consider as "development".

The kernel itself is a great piece of work and the development it takes 
is in some parts amazing. But - be honest - a kernel as such leads you 
nowhere. You have to have other kind of software to build upon it ... 
say a linux distribution.

Here the first problem occurs, as distributions use different kernels 
and most often introduce there own changes (not only regarding the 
kernel). Also - if you have a running linux system, with sometimes very 
specific software needs, which in most cases takes a lot of work to get 
*it all together* running, you aren't easily changing lot of this, let 
it be kernel, distro or whatever.

So apart from embedded devices, the number of people using using older 
distributions / kernels for a reason, isn't that small as you might believe.

I personally thing that the people that jump on every new kernel are 
just the *minority* here.

Even distributions stick to a kernel version, that proved to be of a 
"good quality" and apply later changes to that. Long time ago there was 
a "stable" (2.2, 2.4) line of kernels and "developemt" kernels (2.3, 
2.5). With the way the kernel is managed today, there are certain 
kernels of - in my view - a very bad quality.

So the question here about old kernels is just, how you developers 
defined yourself:

- do you care more about the development process, speed and ease of 
work, including an easier way of new people into 4vl-dvb development, 
than it is certainly wise to abandon all those things, that interfere 
with that

OR

- do you find it important, that there are actually more than the 
minority of "new-kernel-users" people to actually use your work  .

In this latter case you might have to admit, that it maybe a good to 
idea to support more than just the recent kernel (or the 3 fewer kernels).

> But it does raise the point that if we decide to drop support for kernels < 
> 2.6.22 then it is probably a good idea to make a snapshot first so people 
> can still have the option to upgrade their v4l-dvb, even though that 
> version isn't maintained anymore by us.

The problem here is, that a normal person will take a few dollars, euros 
whatever ... goes into an electronics store or visits an online shop and 
buys some device, let it be a webcam or DVB stick.

A lot of hardware would require a very recent kernel, if it wouldn't be 
possible to compile v4l-dvb itself out of the kernel. For Eee PC 
netbooks with a 2.6.21.4 kernel it applies e.g. to DVB sticks with 
AF9015 chipsets etc.

And a lot of hardware, freshly supported by kernel x.y.z. 1 1/2 years 
ago, will not be available from that store as a new product.

Whatever way you're choosing, it will be fine.

If you decide to drop support for kernels below 2.6.22, I then know, 
that I don't have to try it on out netbooks, and I know (and also can 
tell people), that a device won't ever be supported with the default 
linux distro / netbook until we use kernel 2.6.34 ... whatever.

I don't actually know, how important feedback from "normal" users is for 
you and how much you actually get.

But, again, in my view, the user that every few weeks upgrades his 
kernel is the minority.

Best regards, Tobias

Re: RFCv1: v4l-dvb development models & old kernel support

[CityK]

HI Hans,

Hans Verkuil wrote:
> Oldest supported openSUSE kernel is 2.6.22 (10.3). I don't know when the 
> end-of-life due date is for this openSUSE release. It might not be too long 
> from now. (I can't find a decent webpage with the openSUSE releases, does 
> anyone have one?)

Here's a couple of sources:
- http://en.opensuse.org/SUSE_Linux_Lifetime  ... spells out their firm
dates and projections
- http://en.opensuse.org/Roadmap  ... clicking some of the release
"product highlights" also convey the info (but the key word was "some",
as opposed to the ideal case where "all" the release product highlight
descriptions prominently convey such info)

Hopefully someone from the SUSE community will work towards making such
information both more prominently displayed and a little easier to find

Re: RFCv1: v4l-dvb development models & old kernel support

[kilgota]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 14354 bytes --]



On Sun, 22 Feb 2009, Tobias Stöber wrote:

> Hi together,
>
> following this dicussion for some time, let me just add my solely personal 
> view. I'll have to prepend that I'm only a user in terms of v4l-dvb, but do 
> for instance quite a lot support for Eee PC netbooks. So some comments here 
> in some sense urged me to jump into this discussion.

You raise interesting points and complicated issues. I do not know if 
there is any ideal solution -- except, perhaps, time. I, unfortunately, 
can can see all sides of the issue, too...

>
> Hans Verkuil schrieb:
>> On Sunday 22 February 2009 02:15:51 kilgota@banach.math.auburn.edu wrote:
>>>>> Oldest supported Ubuntu kernel is 2.6.22 (7.10):
>>> This is a bit optimistic.
>>> 
>>> Matter of fact, I just bought a brand new eeePC in January, on which Asus
>>> chose to install Xandros. The response to uname -r is (I put this on a
>>> separate line in order to highlight it)
>>> 
>>> 2.6.21.4-eeepc

<snip>

>> 
>> I don't think these netbooks are relevant for us for the simple fact that 
>> the main use case of v4l-dvb on devices like this is the webcam, and that 
>> will obviously be supported by whatever linux version the manufactorer has 
>> installed.
>
> First to the "netbook" questions ...)
>
> I suppose that you Hans don't use any of these so called netbooks?
>
> Nevertheless these devices are common today, just for the fact, that they 
> offer a good value for money, are powerful enough for all the day to day 
> tasks a normal user may want to do and are sometimes the first ever 
> experience of normal users with linux. Furthermore you have a device which 
> weights around 1 kg and has roughly the size of an A5 piece of paper or 
> slightly larger.

Yes, indeed. And the Atom model, especially when upgraded to 2 gig of RAM, 
is far from a toy. It is a very powerful and fast machine. It compares 
quite well with the desktop system I was using until quite recently 
(single-core Sempron 2600+ with 1 gig of RAM) and also with the heavy 
laptop I was using until quite recently (old Dell Latitude C600 with 
800mhz Mobile Celeron and 384 meg of PC100 RAM, max capacity 512 meg of 
RAM). So I fully expect the eeePC to be the machine that I use while 
traveling, going to conferences, giving presentations, and this means as 
well that it has to be, ultimately, a second (or third or fourth) 
development platform for me because that is what I am doing.

For me, of course, this means that it will ultimately have to have a 
systematic kernel and software upgrade. But other problems impinge there. 
I mentioned that I am running a very contemporary distro on it, with a 
very contemporary kernel. But I have not removed the original installation 
yet. For example, right now the wireless works with the original 
installation, but the support for the RaLink2860 is not yet in a stock 
kernel. So an important hardware component is not easy to support. I have 
the ability and the patience to deal with a problem like this one. But I 
am obviously in a minority.

>
> Besides the netbooks there are other devices, that share the same kind of 
> linux distributions - or to make it more specific - say the Asus Eee Box 
> models, that also comes (if they come with linux) with a Asus specific kind 
> of Xandros linux, which itself is based on Debian etch and also newer parts 
> of Debian lenny / sid.

Well, now the other side of me is speaking. I admire Debian. But I have 
never used it. Debian has been too often too far behind on too many things 
-- kernel, libc version, libtool version, you name it. Then there are 
distros which base themselves on Debian, but are not Debian, and thereby 
seem to gain the attention of hardware manufacturers. I do not understand.

Have I mentioned a user who reported a new stillcam to me two months ago, 
and the first thing to do was to see if it is supported by a recent 
libgphoto2? Well, on his Debian workstation a newer libgphoto2 would have 
required a newer libtool. He uses the workstation in his job, so he did 
not want to mess with that. So at my urging he put Slax on an SD card for 
his eeePC, and the Slackware package libgphoto2-2.4.3 (and a couple of 
missing dependencies) and then finally was able to test the camera and yes 
it works and I posted an update to SVN yesterday evening, after 2 months 
of hassle.

I have had other experiences with users of certain distros, experiences 
which were not nearly so positive.

Part of my point here is that it is not only kernel development which is 
affected by dilatory distros. Userspace development is affected, 
too. Users are affected. Everyone is affected. Not just the kernel 
version is in question. Basic libraries are behind on their versions. 
An attempt to upgrade anything causes dependency hell. From my point 
of view, the only thing to do about the problem is to publicize it. One 
possible way to publicize it is to point out just how much trouble it 
causes.

>
> Just looking at the people I know (sometimes consider as friends), nearly 1/3 
> of them own such a device. More than 3/4 of them sticks to the default linux, 
> which certainly has its quirks and problems.

Indeed.


> Using any other linux distribution may introduce a very recent kernel or 
> other recent parts of software - but it just swaps Asus Xandros specific 
> problems with that of the particular distro or kernel.

Such as, for example, supporting the wireless card ...

Why don't you encourage your friends to put a live distro on an SD card 
to support some of the new stuff they want to install? It's easy. It's 
fun. It's cheap. And it does not have to be Slax if you or they prefer 
something else.

>
> The "netbook" itself is - just my personal view of it - just a small kind of 
> notebook device with sometime older (Celeron M353 cpu etc.) or in newer 
> models Intel Atom based hardware.
>
> This also means, that a lot of people just use it as a notebook, which means 
> they use some sort of peripherals. Nearly all of the people I know owning a 
> "netbook", don't use their netbook just as it came, all of them use some 
> additional hardware, most prominently a lot using DVB usb devices, 3G modems, 
> bluetooth or IrDA dongles, external video or webcam devices or adapters to 
> provide legacy serial or parallel interface ports.
>
> A lot of such devices are not supported by the kernel modules, that are 
> provides with the netbook, so one has to build additional kernel modules. 
> Sometime from other source, that the actual kernel source. In this case it is 
> very handy to be able to compile a fresh hg checkout.

Again, use an SD card with another distro on it. Me, I did not even try to 
compile that on the original distro with the original kernel. One look at 
that kernel version and I just *assumed* it would never work. But it ought 
to compile with any version that would pass as "current" which was my 
original point. So this says don't switch over to git and current kernel 
source.

>
> I don't want to bother you with a listing of cases in which people I know to 
> use a specific device or me even helping them to get it running on such 
> netbooks....

If you used a live distro, it would be even easier to do that.

>
> The point just is, that regardless of a device being classified as netbook or 
> notebook it is a very naive point of view to say:
>
> a) that it will only be used with to integrated software,

agree.


> b) that a manufacturer and or distribution will jump every few weeks onto a 
> recent kernel (which would be in some case a major task to install) and

also true, but some of the manufacturers and some of the distros *do* need 
a bit of education. Their behavior is really out of line. Perhaps we can 
be idealists and believe that if they actually could be made to understand 
how much trouble is caused, there might be a change?

>
> c) that every normal user has the abilities to master such a - in some sense 
> complicated task - as building and installing a complete kernel (building a 
> v4l-dvb checkout is a lot more easier).

Also true.

>
> Second ...)
>
> ... you just to a more wider viewpoint, or even just try to understand that 
> there may be just more things to consider as "development".
>
> The kernel itself is a great piece of work and the development it takes is in 
> some parts amazing. But - be honest - a kernel as such leads you nowhere. You 
> have to have other kind of software to build upon it ... say a linux 
> distribution.
>
> Here the first problem occurs, as distributions use different kernels and 
> most often introduce there own changes (not only regarding the kernel). Also 
> - if you have a running linux system, with sometimes very specific software 
> needs, which in most cases takes a lot of work to get *it all together* 
> running, you aren't easily changing lot of this, let it be kernel, distro or 
> whatever.

That is why I consider it a very bad thing to put together a distro which 
is full of old this and that which works well with the other pieces of 
itself but the attempt to upgrade any single piece of it requires one to 
jack up the hood ornament and roll a new car under it.

>
> So apart from embedded devices, the number of people using using older 
> distributions / kernels for a reason, isn't that small as you might believe.

This is true, and we should all understand that it it true. It is also a 
sad state of affairs.

>
> I personally thing that the people that jump on every new kernel are just the 
> *minority* here.
>
> Even distributions stick to a kernel version, that proved to be of a "good 
> quality" and apply later changes to that. Long time ago there was a "stable" 
> (2.2, 2.4) line of kernels and "developemt" kernels (2.3, 2.5). With the way 
> the kernel is managed today, there are certain kernels of - in my view - a 
> very bad quality.

All of this is also true. Equally, it is the case that some distros are 
not sufficiently paying attention to upgrading the kernel -- or for that 
matter anything else.

>
> So the question here about old kernels is just, how you developers defined 
> yourself:
>
> - do you care more about the development process, speed and ease of work, 
> including an easier way of new people into 4vl-dvb development, than it is 
> certainly wise to abandon all those things, that interfere with that
>
> OR
>
> - do you find it important, that there are actually more than the minority of 
> "new-kernel-users" people to actually use your work  .
>
> In this latter case you might have to admit, that it maybe a good to idea to 
> support more than just the recent kernel (or the 3 fewer kernels).

Any distro which is so far behind that it becomes impossible for any 
average user to upgrade anything without getting into dependency hell over 
one thing after another, because the distro is behind on too many fronts 
at the same time, seems to me can be left out in the cold. One cannot 
support distros like that in userspace, either. This is not an either-or 
question of elitism on the part of developers.

>
>> But it does raise the point that if we decide to drop support for kernels < 
>> 2.6.22 then it is probably a good idea to make a snapshot first so people 
>> can still have the option to upgrade their v4l-dvb, even though that 
>> version isn't maintained anymore by us.
>
> The problem here is, that a normal person will take a few dollars, euros 
> whatever ... goes into an electronics store or visits an online shop and buys 
> some device, let it be a webcam or DVB stick.
>
> A lot of hardware would require a very recent kernel, if it wouldn't be 
> possible to compile v4l-dvb itself out of the kernel. For Eee PC netbooks 
> with a 2.6.21.4 kernel it applies e.g. to DVB sticks with AF9015 chipsets 
> etc.

But why is the eeePC using the 2.6.21.4 kernel? What is it exactly that 
makes this difficult for the people who made that choice, to understand? I 
am not speaking of the people who bought one. I bought one. I did not buy 
one with Windows on it. I proudly bought Linux. But when I saw that this 
is kernel version 2.6.21.4 I was shocked.

>
> And a lot of hardware, freshly supported by kernel x.y.z. 1 1/2 years ago, 
> will not be available from that store as a new product.
>
> Whatever way you're choosing, it will be fine.
>
> If you decide to drop support for kernels below 2.6.22, I then know, that I 
> don't have to try it on out netbooks, and I know (and also can tell people), 
> that a device won't ever be supported with the default linux distro / netbook 
> until we use kernel 2.6.34 ... whatever.

Again, use a live distro on USB stick or SD card. Slax on SD card runs 
like a scalded dog. No perceptible performance hit, at all.

>
> I don't actually know, how important feedback from "normal" users is for you 
> and how much you actually get.

I have had to deal with lots of them. I just mentioned one of them in this 
letter. He was a very nice and very patient and very intelligent person. 
He also said that my instructions were very clear, which makes me feel 
good, too. I have had years of experience in writing instructions to 
people in similar situations, so I would hope that I got better at it. But 
this man, like some others, was caught in the situation that he was using 
a distro which is just too far behind on too many things at the same time, 
and the distro seems unable to catch up even when doing a major upgrade.

So the problem is not those users. It is those distros.

>
> But, again, in my view, the user that every few weeks upgrades his kernel is 
> the minority.

This is again true. The fact that I might do that puts me in the developer 
category, I suppose. But even people like me might have several machines 
and only do that on just one of them.


> Best regards, Tobias

Sorry for the long letter. It seems that there are things on which we 
agree and things on which we disagree. Both of us are concerned with 
users. But the users have not caused the problems, have they? Distros and 
hardware manufacturers have caused these problems, and the problems are 
really serious problems both for users and for developers. So a better 
solution has to be thought of, than just to indulge the dilatory behavior 
of the guilty in order to avoid displeasing some of the innocent.

Theodore Kilgore