Is the Linux kernel underfunded? Lack of quality and security?

Is the Linux kernel underfunded? Lack of quality and security?

[Evan Rudford]

The problem of underfunding plagues many open source projects.
I wonder whether the Linux kernel suffers from underfunding in
comparison to its global reach.
Although code reviews and technical discussions are working well, I
argue that the testing infrastructure of the kernel is lacking.
Severe bugs are discovered late, and they are discovered by developers
that should not be exposed to that amount of breakage.
Moreover, I feel that security issues do not receive enough resources.

I argue that the cost of those bugs is vastly higher than the cost
that it would take to setup a better quality assurance.
With sufficient funding, the kernel might do all of the following:

- Make serious efforts to rewrite code with a bad security track
record, instead of only fixing security vulnerabilities on an ad hoc
basis.
- Although the kernel will always remain in C, make serious efforts to
introduce a safe language for kernel modules and perhaps for some
subsystems.
- Build an efficient continuous integration (CI) infrastructure.
- Run a fast subset of the CI tests as a gatekeeper for all patch sets.
- Run strict CI tests to ensure that userspace compatibility does not break.
- Run CI tests not only in virtual environments, but also on real hardware.
- Run CI tests that aim to detect performance regressions.

I realize that some companies are already running kernel testing
infrastructure like this.
However, the development process seems to either lack the resources or
the willingness to build a better quality assurance?

Re: Is the Linux kernel underfunded? Lack of quality and security?

[Al Viro]

On Sun, Jan 05, 2020 at 04:47:33AM +0100, Evan Rudford wrote:

> - Although the kernel will always remain in C, make serious efforts to
> introduce a safe language for kernel modules and perhaps for some
> subsystems.

Let me see if I've got it right - you suggest introducing an infrastructure
that would provide the bindings between the core kernel and those "safe
language modules" and maintaining its safety (from those languages' point
of view) through the changes of said core kernel *and* through the changes
of ABI of the languages in question?  That takes an impressive skillset
from the poor sods in question - on the level of people actively working
on the language implementation, _in_ _addition_ _to_ what's normally
required for the kernel work.  And that happy group would have to keep
track of the kernel changes.  That will certainly make everything more
secure; I just wonder where have you found the funding to cover the costs
of psychiatric care for the victims^Wproud members of that august group.
You do have that funding lined up, right?

Re: Is the Linux kernel underfunded? Lack of quality and security?

[Theodore Y. Ts'o]

On Sun, Jan 05, 2020 at 04:47:33AM +0100, Evan Rudford wrote:
> The problem of underfunding plagues many open source projects.
> I wonder whether the Linux kernel suffers from underfunding in
> comparison to its global reach.
> Although code reviews and technical discussions are working well, I
> argue that the testing infrastructure of the kernel is lacking.
> Severe bugs are discovered late, and they are discovered by developers
> that should not be exposed to that amount of breakage.
> Moreover, I feel that security issues do not receive enough resources.

It sounds like you are unaware of the Kernel Self Protection Project
(KSPP), which is focused on proactively improving the kernel's
security features, and the KernelCI project.  There is quite a lot of
work happening already.

One of the challenges is that is an extremely large number of
different ways a kernel can be configured, and that a *very* large
number of the bugs tend to be hardware specific.  Running CI on all
possible hardware that might run Linux is really not practical; but
there is a very large number of tests being run on both VM's and on
those hardware platforms that companies who are donating hardware to
KernelCI care about.

Keep in mind that there is *always* the opportunity to do more testing
and QA work.  Companies which care about specific hardware and
software configurations are contributing resources (both money and
engineering headcount) to improve the quality for those specific
configurations.  So there is *always* opportunities where more
resources can improve any product.  This is true whether you are
talking about, say, a $15,000 Ford Fiesta or a $115,000 Porsche 911.

If you have access to resources that you would like to contribute, and
have some specific areas where you would like to see improvement, we
can certainly put you in touch with the various organizations, such as
the Linux Foundation, which are organizing efforts such as KernelCI.
There are also a number of engineers from a goodly number of companies
contributing to the Kernel Self Protection Project.  If you are
interested in getting involved, please see:

    https://kernsec.org/wiki/index.php/Kernel_Self_Protection_Project

Cheers,

					- Ted