[TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

TL;DR: We (NVIDIA) want to explore the possibility of upstreaming
mechanisms currently reserved for a safety-oriented fork of the
kernel.
Should there be a consensus that they can be valuable, we can plan for
the work required for upstreaming, but first we would like to discuss
the content, the expectations, and what it would entail to reach a
mergeable status.

--------------------------

The long pitch:

Intro

In the field of Functional Safety, there is a strong demand for use of
the Linux kernel as base operating system for safety applications.

Examples:
Assisted/Autonomous driving, industrial/humanoid robots, medical
equipment, aerospace.

Linux, though, was not designed for safety, and it cannot be used as-is.

While there are attempts at claiming safety compliance through
process, these are insufficient for advanced safety applications.

The reason why said claims are insufficient is that on one hand the
vanilla kernel would not survive negative testing (e.g. fault
injection) and on the other hand a process-based approach would
require demonstrable absence of safety-compromising bugs.

Which is more or less in the same ballpark as proving absence of bugs
across the entire code base. Not very realistic.

At NVIDIA we are developing kernel extensions that support Linux-based
safety through actual hardening mechanisms that can withstand that
sort of validation based on simulation of faults.

--------------------------

References

Some of the problems addressed:

Linux Virtual Address Space Safety
https://youtu.be/pe9OVjWdF-w

Identifying Safety Weaknesses and Fault Propagation in the Linux Kernel
[https://www.youtube.com/watch?v=gW0W9YPnsjI]


What we are working on:

NVIDIA Linux for Safety – A System-Level View
https://www.youtube.com/watch?v=9GPQVu7KDTw

NVIDIA Approach for Achieving ASIL B Qualified Linux
[https://www.youtube.com/watch?v=ueIPuhcyAUo]

--------------------------

Opportunity

While Safety was the primary goal for this exercise, it has become
apparent that the very same mechanisms could be employed, with
different policies, also for more traditional purposes, like integrity
and security. This is something that could benefit a broader number of
users than we had initially anticipated.

--------------------------

Question/Topic for discussion

Would the upstream community be interested in this sort of hardening?

We would like to understand if there could be a mutual interest.

Safety extensions were initially thought to be a standalone fork, due
to the low interest that they would probably command.

However, if there is an interest due to the adjacent applications,
like integrity and safety, a path to upstreaming can be considered.

We also want to explain the criteria we need to meet, in order to
retain compliance with safety standards (it's not as boring as it
might sound).

 --------------------------

Desired outcome from the discussion

The feature-set is not small; there is a significant number of changes involved.

It is expected that any attempt at upstreaming will require
significant - probably multi-year - effort, both for working on the
patches and for reviewing them.

Before embarking in such an endeavour, we believe it would be
beneficial to everyone to assess what these features can bring to
Linux, if they are seen as useful, and what implementing them would
imply.


--
thanks,
igor stoppa

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Greg KH]

On Mon, Sep 07, 2026 at 07:53:14PM +0300, igor.stoppa@gmail.com wrote:
> TL;DR: We (NVIDIA) want to explore the possibility of upstreaming
> mechanisms currently reserved for a safety-oriented fork of the
> kernel.
> Should there be a consensus that they can be valuable, we can plan for
> the work required for upstreaming, but first we would like to discuss
> the content, the expectations, and what it would entail to reach a
> mergeable status.
> 
> --------------------------
> 
> The long pitch:
> 
> Intro
> 
> In the field of Functional Safety, there is a strong demand for use of
> the Linux kernel as base operating system for safety applications.
> 
> Examples:
> Assisted/Autonomous driving, industrial/humanoid robots, medical
> equipment, aerospace.
> 
> Linux, though, was not designed for safety, and it cannot be used as-is.
> 
> While there are attempts at claiming safety compliance through
> process, these are insufficient for advanced safety applications.
> 
> The reason why said claims are insufficient is that on one hand the
> vanilla kernel would not survive negative testing (e.g. fault
> injection) and on the other hand a process-based approach would
> require demonstrable absence of safety-compromising bugs.
> 
> Which is more or less in the same ballpark as proving absence of bugs
> across the entire code base. Not very realistic.
> 
> At NVIDIA we are developing kernel extensions that support Linux-based
> safety through actual hardening mechanisms that can withstand that
> sort of validation based on simulation of faults.
> 
> --------------------------
> 
> References
> 
> Some of the problems addressed:
> 
> Linux Virtual Address Space Safety
> https://youtu.be/pe9OVjWdF-w
> 
> Identifying Safety Weaknesses and Fault Propagation in the Linux Kernel
> [https://www.youtube.com/watch?v=gW0W9YPnsjI]
> 
> 
> What we are working on:
> 
> NVIDIA Linux for Safety – A System-Level View
> https://www.youtube.com/watch?v=9GPQVu7KDTw
> 
> NVIDIA Approach for Achieving ASIL B Qualified Linux
> [https://www.youtube.com/watch?v=ueIPuhcyAUo]
> 
> --------------------------
> 
> Opportunity
> 
> While Safety was the primary goal for this exercise, it has become
> apparent that the very same mechanisms could be employed, with
> different policies, also for more traditional purposes, like integrity
> and security. This is something that could benefit a broader number of
> users than we had initially anticipated.
> 
> --------------------------
> 
> Question/Topic for discussion
> 
> Would the upstream community be interested in this sort of hardening?

Why not just submit your patches like other projects do to get review
that way?  If you don't have working patches, there's not much we can
really comment on, right?

And as you are working on this with the other ELISA people, why not work
with them to get the changes merged upstream?  Or is this separate from
that effort?

thanks,

greg k-h

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

On Mon, 7 Sept 2026 at 23:56, Greg KH <gregkh@linuxfoundation.org> wrote:

> Why not just submit your patches like other projects do to get review
> that way?

Because:

- the effort I mentioned is about safety
AFAIK safety is not something that has attracted much interest
in the past, from upstream.
Especially when it comes to alignment with requirements from safety standards.
It seems unlikely that things would change much in the near future.
This is simply an observation of past exchanges between safety folks
and the upstream community.

- we have code that is arranged for survivability to recurring rebase,
as a fork. It's far from ideal, but it's an acknowledgement of the
previous point.
And it allows us to decouple internal schedules from
whatever time it might take for upstreaming anything.

- as I wrote, perhaps there might be a common interest in extending
the properties of the code we have, toward topics that seem to be more
resonating with upstream, like integrity and security.

- if indeed there is interest, then we can try to prioritize this rework
But we would also seek an agreement that the final shape must
still satisfy our safety-oriented needs.
After all, what would be the point of upstreaming something, if then
the end result turned out to be unusable for our main purpose?
It needs to remain compatible with its original intended use.
That is one of the points I'd like to explain.

>  If you don't have working patches, there's not much we can
> really comment on, right?

I did not propose a code review, though.
I proposed a discussion about the architectural changes supporting the
features we have implemented.

For that, knowledge of kernel architecture should suffice.
And a tad of ARM64 MMU architecture. Not much else.
I can provide the safety perspective.

We do have working patches, at least working for our own purposes.
As I wrote, the effort of massaging them into LKLM compliant shape
would not be insignificant.

With this I absolutely do not mean that we are seeking a carte blanche
for merging whatever.
Precisely because we take seriously the requirements for upstreaming,
we would like to first seek at least alignment of expectations about
both functionality of the code, and its shape.
It would not be very useful for us to prepare said patchset only for seeing
it rejected because safety needs might be considered unacceptable.

If there is a will to discuss it, and an agreement at least on intent and
expectations of the exercise can be found, then we can proceed with
the canonical rework.

In case no agreement can be found, at least we tried it.
And who knows, maybe some parts can still be acceptable, if not the entirety.

> And as you are working on this with the other ELISA people, why not work
> with them to get the changes merged upstream?  Or is this separate from
> that effort?

This is an NVIDIA effort, currently.
We are exploring the possibility of anticipating its publication, or at least of
parts of it that might be of interest for the kernel community.

Our activity in ELISA so far has been focused on describing the
problem we are trying to solve.
But ELISA's intent is not to maintain a fork of the kernel or anyhow
replace upstream.

Thus, I am asking here, where people who maintain the subsystems that would
be impacted can be found.

I can ask ELISA people to lobby for the proposal, if it helps, but in the end
what is being sought for is an architectural discussion with kernel maintainers.

-- 
igor stoppa

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Theodore Tso]

Looking at some of the videos which you cited, and your description,
what the kernel safety effort reminds me is the PREEMPT_RT patches[1],
so the path to upstream that was followed by the PREEMPT_RT patches
might be instructive.

[1] https://en.wikipedia.org/wiki/PREEMPT_RT

There were plenty of people who were people doing soft realtime
without needing PREEMPT_RT, but if what your project required
something stronger --- hard realtime guarantees --- then you needed
the PREEMPT_RT patch set[2].

[2] https://www.socallinuxexpo.org/scale7x/sites/scale7x.socallinuxexpo.org/files/Bryan-che-IntroToRealtime-Scale2009.pdf

I was quite familiar with PREEMPT_RT, since I led the team at IBM
which productized the PREEMPT_RT patches for use by the US Navy's
DD(G)-1000 Zumwalt class destroyer.  Also involved included John
Stultz and Darren Hart, who are still involved with Linux Kernel
development, and we got an IBM Systems Journal publication out of
it[3].

[3] https://www.researchgate.net/publication/220354037_Real-time_Linux_in_real_time

The path to upstream took place over many, many years, and involved
individual features that were useful not just for PREEMPT_RT patch
set, but also for other use cases.  For example, Futexes came out of
the real-time linux patchset.

So that's what I would recommend.  You will need to have a business
case to justify the huge amount of work to (a) implement Linux with
your safety certification requirements, and (b) get that work
upstream.  Since the company(s) funding the work will need to be able
to business income to justify continuing to underwrite your work,
having an out-of-tree patch set is going to be necessary.  There's no
getting around that.

Along the way, if you can find that various features in your
out-of-tree patchset can be useful for solving multiple problems
(which was the case with futexes) you can get those features lifted
out and upstreamed, thus reducing the size of the out-of-tree patch
set, and reducing your work to maintain that patch set.

Cheers,

					- Ted

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

On Tue, 8 Sept 2026 at 06:52, Theodore Tso <tytso@mit.edu> wrote:
>
> Looking at some of the videos which you cited, and your description,
> what the kernel safety effort reminds me is the PREEMPT_RT patches[1],
> so the path to upstream that was followed by the PREEMPT_RT patches
> might be instructive.

Yes, that path is the one we expected from the beginning.
But OTOH it's perhaps the slowest.

And it's not mutually exclusive with trying to accelerate all or part
of the process.
Through this discussion, for example.

Slow path as a fallback is always available.
We are definitely not betting the house on getting feature X
merged by any date Y, which often seems to be the cause
for friction when someone tries to upstream something.

So we can afford perhaps more flexibility

But there is an extremely important aspect, where what we are doing for safety
diverges from the typical project. Even from the PREEMPT_R one.

AFAIK PREEMPT_RT was driven by specific functional needs only.
We need to deal also with additional regulatory requirements that affect
the shape of the code, not only its behavior.

To the best of my knowledge, this aspect is perhaps a new addition to the
typical kernel coding criteria.
And that is what I wanted to discuss.

Like: how the measured (and here there are lots of opinions) complexity
of a given code component affects its suitability for being utilized
for a functionality
that must carry safety goals.

A comment like "don't use function F, use function G" or don't implement
"function H, because function J already exists" might not be compatible
with the needs we have, to qualify the code.

Has there been any previous occurrence of this sort of constraint?
I couldn't find any.

-- 
thanks, igor

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Theodore Tso]

On Tue, Sep 08, 2026 at 01:14:48PM -0500, igor.stoppa@gmail.com wrote:
> AFAIK PREEMPT_RT was driven by specific functional needs only.
> We need to deal also with additional regulatory requirements that affect
> the shape of the code, not only its behavior.
>
> To the best of my knowledge, this aspect is perhaps a new addition to the
> typical kernel coding criteria.
> And that is what I wanted to discuss.

If the certification requires restricting and slowing down what 99% of
the other Linux developers need to do, supporting 99% of the business
value of Linux, it's not going to be something that people will be
enthusiastic about.

In practice, certification requires paying $$$$$ to a certification
agency, so realistically, the vast majority of Linux kernel releases
will not be safety certified.  So tying everyone's hands when going to
have approximately business value most of the time, and only if *all*
of your Linux "safety" patches are accepted, and you would then have
veto power over all future Linux development.... is not going to be
something that most people would consider the worthwhile just to
extend Linux's world-wide domination by 0.00001%, and where your
company would be capturing nearly all of the business value.


Worst, you're asking us to buy a pig-in-a-poke.  There are no patches,
and you want us to agree ahead of time to constrain and inconvenience
tens of thousands of kernel developers for something that won't
benefit most of them or their companies?

It may be that some of your patches will improve safety, without
actually achieving full certification.  Call that "soft" Linux safety
as opposed to "hard" Linux safety, much like "soft" vs "hard"
real-time.  If there cheap ways that we can make Linux more robust
without compromising speed (so no checks on fast paths) and without
compromising maintability, great!

Yes, it's slower, but there are no shortcuts.  The PREEMPT_RT patches
show that it is possible, however.  (But only if the business value
makes it worthwhile.  And if business value isn't strong enough, they
why should we pay the cost ahead of time, and accept a huge amount of
developer velocity and aggravation?)

Cheers,

							- Ted

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

On Tue, 8 Sept 2026 at 16:56, Theodore Tso <tytso@mit.edu> wrote:

> If the certification requires restricting and slowing down what 99% of
> the other Linux developers need to do,

No. It doesn't. At least not in our case.
I've replied also in the other mail to @Miguel Ojeda about it.
The need is mostly about preserving certain properties of the "fences mechanism"
that we would be providing.

What you describe is the "pure-process-based" approach that has been
chosen by many.
There, one tries to prove that something is safe, through evidence
related to the
development processes, because it's needed to be safe.
Or else there is no product.

What we do, instead, is to let the 99.9% of the kernel to be non
safety qualified.
But behind provable barriers. We qualify the barriers as a fraction of
the code base.
The barriers might use some simple kernel functionality.
"Simple" as in "easy to evaluate its safety" e.g. like linked list management.
Mostly, stateles/lib level code.

So the real concern, there, would be that in case the fences get
merged, their qualifiability
should be preserved.

Most of your objections seem to be inferred based on others' previous attempts,
but they do not actually apply to what I've described.

In fact, we truly do want the kernel to stick to its present process,
because we can
rebase easily on it, since we are not tied to qualification of
extensive portions of it.

The idea is that once a core safety framework is present, then whoever
wants to use a
specific driver can choose to go the extra mile and make it compliant
with the framework.

Fully voluntary opt-in.

Or if they do not care about its safety, they can still use it, just
not as a safe component.
So it would be really on a case-by-case basis.

And for those who do not care about safety at all, even the core
framework can be fully compiled off.

--
igor

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Steven Rostedt]

[ Adding a couple of people working with ELISA ]

Bringing back my memory of working at Lockheed Martin in the 1990s on
helping it achieve level 5 CMMI rating[1] ...

On Tue, 8 Sep 2026 17:32:40 +0300
"igor.stoppa@gmail.com" <igor.stoppa@gmail.com> wrote:

> On Tue, 8 Sept 2026 at 16:56, Theodore Tso <tytso@mit.edu> wrote:
> 
> > If the certification requires restricting and slowing down what 99% of
> > the other Linux developers need to do,  
> 
> No. It doesn't. At least not in our case.
> I've replied also in the other mail to @Miguel Ojeda about it.
> The need is mostly about preserving certain properties of the "fences mechanism"
> that we would be providing.

What my LM manager always told me, the best process is the one you don't
realize you are doing. That stuck with me. It's to make the process do
things that "make sense" (although ironically, the CMMI level 5 process was
definitely *not* that!).

For instance, my process is to look at patches in patchwork for tracing,
review them, ask for updates, iterate, and when I'm happy I apply them to
my test tree. I then run my full test suite on these patches and if it
passes, I then push them to my linux-next tree. When Linus opens up the
merge window, I write the change log explaining the changes, and send a
pull request to Linus.

The above is a process. If I document it and then always follow it, it can
be something that can be "certified" (if it passes the requirements). But
I'll be following that process regardless of if it is a certified process
or not.

Hence, a process that you don't realize is a process!

> 
> What you describe is the "pure-process-based" approach that has been
> chosen by many.
> There, one tries to prove that something is safe, through evidence
> related to the
> development processes, because it's needed to be safe.
> Or else there is no product.

From what I remember, certification requires documenting the process and
showing that you are following it. And have artifacts to prove it (git
commits, email trail, etc). Repeatability is crucial. Being able to create
tests may also be needed.

> 
> What we do, instead, is to let the 99.9% of the kernel to be non
> safety qualified.
> But behind provable barriers. We qualify the barriers as a fraction of
> the code base.
> The barriers might use some simple kernel functionality.
> "Simple" as in "easy to evaluate its safety" e.g. like linked list management.
> Mostly, stateles/lib level code.

I'm guessing that this will affect the core kernel the most (scheduler,
interrupt handling, memory management) which should be treated with a bit
more scrutiny than normal device drivers as if they go wrong, everything
goes wrong.

File systems have a similar requirement because if they go wrong, people
lose data, which is never good. Funny enough, I'm not sure if file systems
are part of the safety portion, as file systems are mostly used for logging
in safety environments, but may not be part of the safety path.

> 
> So the real concern, there, would be that in case the fences get
> merged, their qualifiability
> should be preserved.

What you should strive to achieve is a way to make it easy for Linux kernel
developers to follow the process you need. If it makes sense to follow and
not a bunch of bureaucratic BS, you may have people following it.
Especially if it helps make their code better.

> 
> Most of your objections seem to be inferred based on others' previous attempts,
> but they do not actually apply to what I've described.
> 
> In fact, we truly do want the kernel to stick to its present process,
> because we can
> rebase easily on it, since we are not tied to qualification of
> extensive portions of it.
> 
> The idea is that once a core safety framework is present, then whoever
> wants to use a
> specific driver can choose to go the extra mile and make it compliant
> with the framework.
> 
> Fully voluntary opt-in.
> 
> Or if they do not care about its safety, they can still use it, just
> not as a safe component.
> So it would be really on a case-by-case basis.
> 
> And for those who do not care about safety at all, even the core
> framework can be fully compiled off.

It sounds like you want to do what I mentioned above. To have a process that
kernel developers can follow that's not too different than what they do
today, but just in a way that it helps satisfy your requirements.

-- Steve

[1] https://investors.lockheedmartin.com/news-releases/news-release-details/lockheed-martin-achieves-record-fourth-cmmi-level-5-rating

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

On Tue, 8 Sept 2026 at 22:28, Steven Rostedt <rostedt@goodmis.org> wrote:

> [ Adding a couple of people working with ELISA ]

[...]

> Hence, a process that you don't realize is a process!

Even intentionally not having a process can be seen as a process :)

[...]

> From what I remember, certification requires documenting the process and
> showing that you are following it. And have artifacts to prove it (git
> commits, email trail, etc). Repeatability is crucial. Being able to create
> tests may also be needed.

Indeed, and having a process, test cases, repeatability and what not,
it is all necessary to get to the doorsteps of a safety qualification.
That's what is called QM - Quality Managed.
Which in itself is already a significant achievement.
CMM belongs more or less there.

The problems come when one wants to move into the field of actual safety
qualification.

To name what is perhaps the most recurring term: FFI, Freedom From Interference.
It means that a given component which has been identified as the recipient
of safety goals, must not be silently altered in its safety-related properties.

Alteration can be both spatial (e.g. scribbling over its memory)
and temporal (e.g. altering its execution timing in whatever way).

Contrary to what one might guess at first impression, though,
this doesn't mean that said interference cannot be allowed.

It can, provided that it is somehow managed timely.
The classical case is a process not being scheduled and failing to ping its WD.

(I'm not missing the irony of me preaching to the choir, but if I have
to say this, I might as well
try to make it as accessible as possible to everyone who might be reading it)

Watchdogging done right is in itself not trivial, but often overlooked.
However more obvious problems arise with the other aspect, spatial interference.

Assuming a bug can lurk in one of the many kernel components,
there is a possibility that it can compromise the integrity of some information
that is writable in kernel space.
Information that might be fulfilling a safety goal.

How to ensure that, if this happens, such an event will be either prevented
or detected within the allotted temporal window?

The bug can be anywhere, and it can manifest itself only once in 10
blue moons, or less.
And it might not even be normally noticeable, because under most
conditions it scribbles
over memory that is irrelevant or even unused.

But that one time that the bug hits the jackpot, what is gonna prevent
it from causing
an unsafe situation? E.g. wrong angle passed to the steering actuator,
car that doesn't follow a turn.

These situations must be covered, explicitly.

The surface to answer for can be very large: from private data of
device drivers to
parts of the linear maps providing backing for processes.

Some argue that by virtue of a process, they can guarantee that a
specific build
of a specific configuration won't cause that sort of problem.

It more or less amounts to claiming that the entirety of the kernel is
devoid of bugs that
might cause that unwanted behavior, under a very diverse set of conditions.
That, and statistical data obtained from very different builds,
configured in very different ways,
running on very different systems, under very different loads.

Without entering the merit of the claim itself, one observation that
can be made, objectively,
about that sort of approach.

If even one bug of the wrong type has escaped the process, then the
system is not safe anymore.
If it has ever been. I leave it to the reader to decide what is the
likelihood of such a bug-free system.

What said so far might sound like paranoia, but it is what is
necessary to provide safety.
Besides, there are various levels of qualification, for safety.
Any process-based approach won't go very far.

Hence the approach I described earlier, where we are perfectly happy
to let the vast majority
of the kernel as it is, not a recipient of any safety requirement.
If maintainers want to improve their processes, that's great,
but we wouldn't be really asking for any extra effort, for safety.

> I'm guessing that this will affect the core kernel the most (scheduler,
> interrupt handling, memory management) which should be treated with a bit
> more scrutiny than normal device drivers as if they go wrong, everything
> goes wrong.

Actually, no. You are describing availability. MTBF.
Great to have for a marketable product, but not mandatory from a safety angle.
What matters for safety is to detect and report timely any safety violation.
That's all.
Each system can be designed to cope with it in its own way.
E.g. some cars will disengage the main unit and activate an auxiliary
system with reduced capability.
Others will simply turn on an indicator and tell you that a certain
functionality (e.g. ABS) is unavailable.

> File systems have a similar requirement because if they go wrong, people
> lose data, which is never good. Funny enough, I'm not sure if file systems
> are part of the safety portion, as file systems are mostly used for logging
> in safety environments, but may not be part of the safety path.

What we want to ensure is the absence of unsafe operations.
In many cases one can tie it to integrity and timing.

But, without downplaying filesystems, the first and foremost integrity
problem is
memory integrity, because what good is it to have a safe filesystem, if then
things can get silently altered once they are in memory, due to a bug?

> What you should strive to achieve is a way to make it easy for Linux kernel
> developers to follow the process you need. If it makes sense to follow and
> not a bunch of bureaucratic BS, you may have people following it.
> Especially if it helps make their code better.

I know you have been exposed to what Kate and Gab are pursuing.
That is the process-based-approach.

While we do not disagree with the benefits it can bring, we don't
really rely on it.
As I wrote earlier, for us it's perfectly fine if the present
development model stays as it is,
we don't have any dependency on it, for claiming safety.

We have fences to take care of unwanted behavior.
Which means that we don't have to worry about where the interference comes from,
as long as we can model and manage it.

[...]

> It sounds like you want to do what I mentioned above. To have a process that
> kernel developers can follow that's not too different than what they do
> today, but just in a way that it helps satisfy your requirements.

Our method doesn't need anything like that.
It's been designed specifically to not have that need.
We certainly won't oppose any process improvement.
But there's also the problem that what you describe would not be found
to be sufficient,
by certain stricter assessing bodies.

OTOH our safety concept has been found to be adequate, provided that we
bring it to completion.

-- 
thanks, igor

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Steven Rostedt]

[ Adding another Gabriele to the mix ]

On Wed, 9 Sep 2026 00:13:19 +0300
"igor.stoppa@gmail.com" <igor.stoppa@gmail.com> wrote:

> On Tue, 8 Sept 2026 at 22:28, Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > [ Adding a couple of people working with ELISA ]  
> 
> [...]
> 
> > Hence, a process that you don't realize is a process!  
> 
> Even intentionally not having a process can be seen as a process :)
> 
> [...]
> 
> > From what I remember, certification requires documenting the process and
> > showing that you are following it. And have artifacts to prove it (git
> > commits, email trail, etc). Repeatability is crucial. Being able to create
> > tests may also be needed.  
> 
> Indeed, and having a process, test cases, repeatability and what not,
> it is all necessary to get to the doorsteps of a safety qualification.
> That's what is called QM - Quality Managed.
> Which in itself is already a significant achievement.
> CMM belongs more or less there.
> 
> The problems come when one wants to move into the field of actual safety
> qualification.
> 
> To name what is perhaps the most recurring term: FFI, Freedom From Interference.
> It means that a given component which has been identified as the recipient
> of safety goals, must not be silently altered in its safety-related properties.
> 
> Alteration can be both spatial (e.g. scribbling over its memory)
> and temporal (e.g. altering its execution timing in whatever way).
> 
> Contrary to what one might guess at first impression, though,
> this doesn't mean that said interference cannot be allowed.
> 
> It can, provided that it is somehow managed timely.
> The classical case is a process not being scheduled and failing to ping its WD.
> 
> (I'm not missing the irony of me preaching to the choir, but if I have
> to say this, I might as well
> try to make it as accessible as possible to everyone who might be reading it)
> 
> Watchdogging done right is in itself not trivial, but often overlooked.
> However more obvious problems arise with the other aspect, spatial interference.
> 
> Assuming a bug can lurk in one of the many kernel components,
> there is a possibility that it can compromise the integrity of some information
> that is writable in kernel space.
> Information that might be fulfilling a safety goal.
> 
> How to ensure that, if this happens, such an event will be either prevented
> or detected within the allotted temporal window?
> 
> The bug can be anywhere, and it can manifest itself only once in 10
> blue moons, or less.
> And it might not even be normally noticeable, because under most
> conditions it scribbles
> over memory that is irrelevant or even unused.
> 
> But that one time that the bug hits the jackpot, what is gonna prevent
> it from causing
> an unsafe situation? E.g. wrong angle passed to the steering actuator,
> car that doesn't follow a turn.
> 
> These situations must be covered, explicitly.

Sounds very much what the Runtime Verifier[1] is used for. It is code that
adds a model represented by a state machine and hooks to tracepoints within
the kernel. It moves the state along based on the information it monitors
and if it were to ever detect an anomaly it would trigger a reaction (could
be simply panic the kernel to go into a safety critical mode).

-- Steve


> 
> The surface to answer for can be very large: from private data of
> device drivers to
> parts of the linear maps providing backing for processes.
> 
> Some argue that by virtue of a process, they can guarantee that a
> specific build
> of a specific configuration won't cause that sort of problem.
> 
> It more or less amounts to claiming that the entirety of the kernel is
> devoid of bugs that
> might cause that unwanted behavior, under a very diverse set of conditions.
> That, and statistical data obtained from very different builds,
> configured in very different ways,
> running on very different systems, under very different loads.
> 
> Without entering the merit of the claim itself, one observation that
> can be made, objectively,
> about that sort of approach.
> 
> If even one bug of the wrong type has escaped the process, then the
> system is not safe anymore.
> If it has ever been. I leave it to the reader to decide what is the
> likelihood of such a bug-free system.
> 
> What said so far might sound like paranoia, but it is what is
> necessary to provide safety.
> Besides, there are various levels of qualification, for safety.
> Any process-based approach won't go very far.
> 
> Hence the approach I described earlier, where we are perfectly happy
> to let the vast majority
> of the kernel as it is, not a recipient of any safety requirement.
> If maintainers want to improve their processes, that's great,
> but we wouldn't be really asking for any extra effort, for safety.
> 
> > I'm guessing that this will affect the core kernel the most (scheduler,
> > interrupt handling, memory management) which should be treated with a bit
> > more scrutiny than normal device drivers as if they go wrong, everything
> > goes wrong.  
> 
> Actually, no. You are describing availability. MTBF.
> Great to have for a marketable product, but not mandatory from a safety angle.
> What matters for safety is to detect and report timely any safety violation.
> That's all.
> Each system can be designed to cope with it in its own way.
> E.g. some cars will disengage the main unit and activate an auxiliary
> system with reduced capability.
> Others will simply turn on an indicator and tell you that a certain
> functionality (e.g. ABS) is unavailable.
> 
> > File systems have a similar requirement because if they go wrong, people
> > lose data, which is never good. Funny enough, I'm not sure if file systems
> > are part of the safety portion, as file systems are mostly used for logging
> > in safety environments, but may not be part of the safety path.  
> 
> What we want to ensure is the absence of unsafe operations.
> In many cases one can tie it to integrity and timing.
> 
> But, without downplaying filesystems, the first and foremost integrity
> problem is
> memory integrity, because what good is it to have a safe filesystem, if then
> things can get silently altered once they are in memory, due to a bug?
> 
> > What you should strive to achieve is a way to make it easy for Linux kernel
> > developers to follow the process you need. If it makes sense to follow and
> > not a bunch of bureaucratic BS, you may have people following it.
> > Especially if it helps make their code better.  
> 
> I know you have been exposed to what Kate and Gab are pursuing.
> That is the process-based-approach.
> 
> While we do not disagree with the benefits it can bring, we don't
> really rely on it.
> As I wrote earlier, for us it's perfectly fine if the present
> development model stays as it is,
> we don't have any dependency on it, for claiming safety.
> 
> We have fences to take care of unwanted behavior.
> Which means that we don't have to worry about where the interference comes from,
> as long as we can model and manage it.
> 
> [...]
> 
> > It sounds like you want to do what I mentioned above. To have a process that
> > kernel developers can follow that's not too different than what they do
> > today, but just in a way that it helps satisfy your requirements.  
> 
> Our method doesn't need anything like that.
> It's been designed specifically to not have that need.
> We certainly won't oppose any process improvement.
> But there's also the problem that what you describe would not be found
> to be sufficient,
> by certain stricter assessing bodies.
> 
> OTOH our safety concept has been found to be adequate, provided that we
> bring it to completion.
> 

[1] https://docs.kernel.org/trace/rv/runtime-verification.html

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

On Wed, 9 Sept 2026 at 02:13, Steven Rostedt <rostedt@goodmis.org> wrote:
>
>
> [ Adding another Gabriele to the mix ]
>
> On Wed, 9 Sep 2026 00:13:19 +0300
> "igor.stoppa@gmail.com" <igor.stoppa@gmail.com> wrote:

[...]

> > But that one time that the bug hits the jackpot, what is gonna prevent
> > it from causing
> > an unsafe situation? E.g. wrong angle passed to the steering actuator,
> > car that doesn't follow a turn.
> >
> > These situations must be covered, explicitly.
>
> Sounds very much what the Runtime Verifier[1] is used for. It is code that
> adds a model represented by a state machine and hooks to tracepoints within
> the kernel. It moves the state along based on the information it monitors
> and if it were to ever detect an anomaly it would trigger a reaction (could
> be simply panic the kernel to go into a safety critical mode).

> [1] https://docs.kernel.org/trace/rv/runtime-verification.html

Monitoring is a way of doing it, yes, and I am familiar with the
runtime verification monitor.
We actually did consider it, some time ago when we were feeling the waters.

But it won't cut it, for most practical safety cases.

In order to detect a deviation, first of all you need to be able to model it.
Modeling it adds extra complexity and detecting it adds overhead.
Even if minimal, once it starts to be applied to potentially large
targets, it adds up.
And that's for the drivers.
If I have a SR (Safety Related) process, it is still exposed to interference
from the kernel through the linear map.

Adding monitoring to pages worth of data used by the process would
kill the system
(and I'm intentionally leaving out code, but that's exposed to
interference as well).

The humble MMU is the simplest and most effective way to protect
things with minimal overhead,
once one has in place the support for individually selecting which
memory map any given kernel
thread needs to use at a given moment. And for marshalling allocations
so that SR data gets
backed by pages that are not writable by non-safety-qualified code.
We happen to have it.

The other problem with these sort of readily-available components from
the kernel is that usually
they take freely from any available API (and why shouldn't they, if it
is compatible with their intended purpose)
which leads to a dependency hell when it comes to qualifying the code
supporting safety goal.
Because they would pull in more than half of the kernel.

Qualifying rigorously a single kmalloc invocation would spiral into
endless work.

BTW, "qualifying" means reviewing the code and analyzing it and
"proving" that it has
no bugs that would be compromising the intended safe behavior.

Frankly I would never even entertain the idea that we could seriously
claim to have done
carpet review on the kmalloc function call graph and proven that all
of them are safe.

So, we intentionally limit ourselves to having, as dependency of our
custom fences,
only functions that are simple enough that we can honestly and credibly claim to
have reviewed adequately.

-- 
thanks, igor

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Gabriele Monaco]

On Wed, 2026-09-09 at 02:48 +0300, igor.stoppa@gmail.com wrote:
> Monitoring is a way of doing it, yes, and I am familiar with the
> runtime verification monitor.
> We actually did consider it, some time ago when we were feeling the waters.
> 
> But it won't cut it, for most practical safety cases.
> 
> In order to detect a deviation, first of all you need to be able to model it.
> Modeling it adds extra complexity and detecting it adds overhead.
> Even if minimal, once it starts to be applied to potentially large
> targets, it adds up.
> And that's for the drivers.
> If I have a SR (Safety Related) process, it is still exposed to interference
> from the kernel through the linear map.
> 
> Adding monitoring to pages worth of data used by the process would kill the
> system (and I'm intentionally leaving out code, but that's exposed to
> interference as well).
> 
> The humble MMU is the simplest and most effective way to protect things with
> minimal overhead, once one has in place the support for individually selecting
> which memory map any given kernel thread needs to use at a given moment. And
> for marshalling allocations so that SR data gets backed by pages that are not
> writable by non-safety-qualified code.
> We happen to have it.
> 
> The other problem with these sort of readily-available components from
> the kernel is that usually they take freely from any available API (and why
> shouldn't they, if it is compatible with their intended purpose) which leads
> to a dependency hell when it comes to qualifying the code supporting safety
> goal.
> Because they would pull in more than half of the kernel.
> 
> Qualifying rigorously a single kmalloc invocation would spiral into endless
> work.
> 
> BTW, "qualifying" means reviewing the code and analyzing it and "proving" that
> it has no bugs that would be compromising the intended safe behavior.
> 
> Frankly I would never even entertain the idea that we could seriously
> claim to have done carpet review on the kmalloc function call graph and proven
> that all of them are safe.
> 
> So, we intentionally limit ourselves to having, as dependency of our
> custom fences, only functions that are simple enough that we can honestly and
> credibly claim to have reviewed adequately.

About this last point you make, RV was originally designed with this specific
idea in mind. The simple Deterministic Automata have very few dependencies and
the code path is as simple as possible to be verified.

Monitors can, optionally, add more bells and whistles like timers (most timed
automata) or a dynamically allocated hashmap (per-object monitors).
Those are not even compiled on monitors not using them.

There is also Linear Temporal Logic that sits somewhere in the middle, with
slightly more complicated code but still few dependencies (bitmaps).

Obviously all types of RV monitors rely on tracepoints, that's how they attach
to kernel code and it isn't negotiable.

You may be aware of it, but the issues and requirements of RV in the safety
domain was discussed in [1].
Whether RV can be made free of interference to fit the functional safety use
cases is still an open problem, though.

And of course, as you mentioned, in some cases RV just doesn't fit the
requirements, but where the logic is complex for hardware-only solutions RV
could help, if we figure the other issues out.

Gabriele

[1] - https://github.com/elisa-tech/Safety_Architecture_WG/pull/14

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

On Wed, 9 Sept 2026 at 10:44, Gabriele Monaco <gmonaco@redhat.com> wrote:
>
> On Wed, 2026-09-09 at 02:48 +0300, igor.stoppa@gmail.com wrote:
[...]

> > So, we intentionally limit ourselves to having, as dependency of our
> > custom fences, only functions that are simple enough that we can honestly and
> > credibly claim to have reviewed adequately.
>
> About this last point you make, RV was originally designed with this specific
> idea in mind. The simple Deterministic Automata have very few dependencies and
> the code path is as simple as possible to be verified.

Maybe you are unaware of this, but you are picking up a discussion we had
with the late Daniel Bristot de Oliveira.
I'm glad for the unexpected chance to remember him.

> Monitors can, optionally, add more bells and whistles like timers (most timed
> automata) or a dynamically allocated hashmap (per-object monitors).
> Those are not even compiled on monitors not using them.

To be clear: I have nothing ideological against the RV
Different tools have different capabilities, and what matters is to do the right
matchmaking between tools and use cases.
And the safety problem is too complex for affording any form of NIH-ism
I have spent some time studying how the RV works, and I am convinced that it
can deliver on its intended initial purpose. And well.

> There is also Linear Temporal Logic that sits somewhere in the middle, with
> slightly more complicated code but still few dependencies (bitmaps).

Yeah, bitmaps are one of the basic tools.
There is only so much variety one can achieve, given a certain HW architecture.

> Obviously all types of RV monitors rely on tracepoints, that's how they attach
> to kernel code and it isn't negotiable.

It is a limitation that - as expected - comes from trying to adapt a
tool that was
born with a different purpose in mind.
The idea of leveraging the existing ftrace infrastructure often bubbles up in
FuSa circles. It's undoubtedly tempting.

> You may be aware of it, but the issues and requirements of RV in the safety
> domain was discussed in [1].

Yes, I am aware, I had notified my concerns already.
Both Kate and Gab should have heard them many times =)
But I can do an encore, here below.

> Whether RV can be made free of interference to fit the functional safety use
> cases is still an open problem, though.

This is perhaps one of my favourite pet peeves with the approach taken by
many functional safety initiatives.
They start looking for a tool that might solve the problem, but skip
what should be
instead a mandatory entry vetting:

Is the tool immune from the type of interference it should monitor?
In other words, is it qualifiable?
If not, can it be made compliant with FFI requirements?

If the answer is no, then why bother going forward?
First find a solution to that. Or else there is no safety story.

Instead, often, this aspect is neglected till the very end.

Let me give an example that is similar in spirit:
containers, or rather cgroups.

They are often advertised as the solution to safety problems.
It is undeniable that one can use containers for limiting resource consumption
of certain groups of processes.

But is their implementation safe? They are pervasive, and if _their_ metadata
gets corrupted, they can actually impede the correct execution of those very
processes they were meant to protect.

> And of course, as you mentioned, in some cases RV just doesn't fit the
> requirements, but where the logic is complex for hardware-only solutions RV
> could help, if we figure the other issues out.

Precisely, "_if_ we figure the other issues out".
That should be the starting point. Not something postponed to the future.

An assessor worth their salt will go straight there:
"how does the tool defend itself, in the first place, before thinking
of how to save others?"

From this perspective, safety is worse than security, because security usually
lets you plan your defences so that you can use anything available, to
prevent an attack.
Because the core system is assumed to be trusted, usually.

With safety that assumption doesn't hold.
One needs to prove that the dependencies are acceptable.

You wrote earlier:

> The simple Deterministic Automata have very few dependencies and
> the code path is as simple as possible to be verified.

That is a start, but that statement is formulated in a subjective way.
And it is qualitative.
To be adequate, it should be turned into something objectively quantitative.
Measurable, verifiable.

e.g. if your "only" dependency is kmalloc or get_free_pages(), that is already
pretty much unsustainable, if you expect that those functions must be safe.

And if your automa depends on internal states (what else could it do?),
then what happens if those states are corrupted?

Some people will tell you that what I just described is too strict, and that
the standard allows for "expert opinion" or "expert judgement".

First of all, that sort of approach fails the completeness test:
it is assumed that the expert knows enough to actually assess the whole system
thoroughly. Or that you have a collection of experts.
Same thing: who knows well enough the entirety of the kernel code base
to confidently
assert that it has no bugs that would be harmful in these situations?
Would even the collective of kernel maintainers be comfortable with
that statement?

Another thing that you will hear is that things can be tested adequately.
There you have another completeness/observability problem:
who can ensure that the testing input/output vectors are sufficient to
expose all the
configuration of internal states that are relevant to safety?

It doesn't mean that this will prevent obtaining a safety certificate from some
assessing entity: different entities have different interpretations of
the standard.
One then should ask: what is the purpose of the exercise?
Getting a certificate or reaching a certain level of demonstrable safety?

The risk with the typical top-down approach is that one invests so
much in a certain
solution that it becomes extremely difficult to accept that it might
turn out to be unsuitable.

With bottom-up (ensure FFI of the monitor/fences first), that risk
doesn't exist.

--
thanks, igor

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Gabriele Monaco]

On Wed, 2026-09-09 at 12:40 +0300, igor.stoppa@gmail.com wrote:
> On Wed, 9 Sept 2026 at 10:44, Gabriele Monaco <gmonaco@redhat.com> wrote:
> > 
> > On Wed, 2026-09-09 at 02:48 +0300, igor.stoppa@gmail.com wrote:
> [...]
> 
> > > So, we intentionally limit ourselves to having, as dependency of our
> > > custom fences, only functions that are simple enough that we can honestly
> > > and
> > > credibly claim to have reviewed adequately.
> > 
> > About this last point you make, RV was originally designed with this
> > specific idea in mind. The simple Deterministic Automata have very few
> > dependencies and the code path is as simple as possible to be verified.
> 
> Maybe you are unaware of this, but you are picking up a discussion we had
> with the late Daniel Bristot de Oliveira.
> I'm glad for the unexpected chance to remember him.

I was not aware, but it makes a lot of sense, I was directly quoting him in that
statement.

Unfortunately I didn't get the chance to meet Daniel, but I tried my best to
understand his original intent indirectly. Though I'm surely missing some of
those in-person interactions.

> > Monitors can, optionally, add more bells and whistles like timers (most
> > timed
> > automata) or a dynamically allocated hashmap (per-object monitors).
> > Those are not even compiled on monitors not using them.
> 
> To be clear: I have nothing ideological against the RV
> Different tools have different capabilities, and what matters is to do the
> right matchmaking between tools and use cases.
> And the safety problem is too complex for affording any form of NIH-ism
> I have spent some time studying how the RV works, and I am convinced that it
> can deliver on its intended initial purpose. And well.
> 
> > There is also Linear Temporal Logic that sits somewhere in the middle, with
> > slightly more complicated code but still few dependencies (bitmaps).
> 
> Yeah, bitmaps are one of the basic tools.
> There is only so much variety one can achieve, given a certain HW
> architecture.
> 
> > Obviously all types of RV monitors rely on tracepoints, that's how they
> > attach
> > to kernel code and it isn't negotiable.
> 
> It is a limitation that - as expected - comes from trying to adapt a
> tool that was born with a different purpose in mind.
> The idea of leveraging the existing ftrace infrastructure often bubbles up in
> FuSa circles. It's undoubtedly tempting.
> 
> > You may be aware of it, but the issues and requirements of RV in the safety
> > domain was discussed in [1].
> 
> Yes, I am aware, I had notified my concerns already.
> Both Kate and Gab should have heard them many times =)
> But I can do an encore, here below.

Of course, you even commented there.. I missed that

> > Whether RV can be made free of interference to fit the functional safety use
> > cases is still an open problem, though.
> 
> This is perhaps one of my favourite pet peeves with the approach taken by
> many functional safety initiatives.
> They start looking for a tool that might solve the problem, but skip
> what should be instead a mandatory entry vetting:
> 
> Is the tool immune from the type of interference it should monitor?
> In other words, is it qualifiable?
> If not, can it be made compliant with FFI requirements?
> 
> If the answer is no, then why bother going forward?
> First find a solution to that. Or else there is no safety story.
> 
> Instead, often, this aspect is neglected till the very end.
> 
> Let me give an example that is similar in spirit:
> containers, or rather cgroups.
> 
> They are often advertised as the solution to safety problems.
> It is undeniable that one can use containers for limiting resource consumption
> of certain groups of processes.
> 
> But is their implementation safe? They are pervasive, and if _their_ metadata
> gets corrupted, they can actually impede the correct execution of those very
> processes they were meant to protect.
> 
> > And of course, as you mentioned, in some cases RV just doesn't fit the
> > requirements, but where the logic is complex for hardware-only solutions RV
> > could help, if we figure the other issues out.
> 
> Precisely, "_if_ we figure the other issues out".
> That should be the starting point. Not something postponed to the future.
> 
> An assessor worth their salt will go straight there:
> "how does the tool defend itself, in the first place, before thinking
> of how to save others?"
> 
> From this perspective, safety is worse than security, because security usually
> lets you plan your defences so that you can use anything available, to
> prevent an attack.
> Because the core system is assumed to be trusted, usually.
> 
> With safety that assumption doesn't hold.
> One needs to prove that the dependencies are acceptable.
> 
> You wrote earlier:
> 
> > The simple Deterministic Automata have very few dependencies and
> > the code path is as simple as possible to be verified.
> 
> That is a start, but that statement is formulated in a subjective way.
> And it is qualitative.
> To be adequate, it should be turned into something objectively quantitative.
> Measurable, verifiable.
> 
> e.g. if your "only" dependency is kmalloc or get_free_pages(), that is already
> pretty much unsustainable, if you expect that those functions must be safe.
> 
> And if your automa depends on internal states (what else could it do?),
> then what happens if those states are corrupted?
> 
> Some people will tell you that what I just described is too strict, and that
> the standard allows for "expert opinion" or "expert judgement".
> 
> First of all, that sort of approach fails the completeness test:
> it is assumed that the expert knows enough to actually assess the whole system
> thoroughly. Or that you have a collection of experts.
> Same thing: who knows well enough the entirety of the kernel code base
> to confidently
> assert that it has no bugs that would be harmful in these situations?
> Would even the collective of kernel maintainers be comfortable with
> that statement?
> 
> Another thing that you will hear is that things can be tested adequately.
> There you have another completeness/observability problem:
> who can ensure that the testing input/output vectors are sufficient to
> expose all the configuration of internal states that are relevant to safety?
> 
> It doesn't mean that this will prevent obtaining a safety certificate from
> some assessing entity: different entities have different interpretations of
> the standard.
> One then should ask: what is the purpose of the exercise?
> Getting a certificate or reaching a certain level of demonstrable safety?
> 
> The risk with the typical top-down approach is that one invests so
> much in a certain solution that it becomes extremely difficult to accept that
> it might turn out to be unsuitable.
> 
> With bottom-up (ensure FFI of the monitor/fences first), that risk
> doesn't exist.

That is all very interesting. I'm not too familiar with safety requirements.
I get that ftrace and friends are themselves hard to validate (though arguably
they are the best way to instrument a kernel with minimal stepping on anyone's
feet).

In general I feel anything that tries to validate the kernel from the kernel
itself will have this kind of problems. It cannot really be isolated from what
it tries to verify.

How do you see a tracing-based solution that satisfies these safety
requirements? Something running on an external chip and reading the processor
trace? Something involving clearly separated VMs?
I know that's what some fields do (avionics), but is the overhead worth it?

If we assume metadata can get corrupted in any way, I'm not quite sure how a
software can ever be considered safe.

But thanks for the deep explanation, it makes your point very clear.

Gabriele

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

On Wed, 9 Sept 2026 at 18:22, Gabriele Monaco <gmonaco@redhat.com> wrote:

[...]

> That is all very interesting. I'm not too familiar with safety requirements.
> I get that ftrace and friends are themselves hard to validate (though arguably
> they are the best way to instrument a kernel with minimal stepping on anyone's
> feet).

Indeed, it is good at doing what it was devised for. Like the RVM.
But as the saying goes, once you have a hammer ...

> In general I feel anything that tries to validate the kernel from the kernel
> itself will have this kind of problems. It cannot really be isolated from what
> it tries to verify.

Well, that is where I beg to differ. We have implemented such a mechanism.
But I've been repeatedly told that I should provide first the code,
which right now
woulnd't really help much, because it is far from being in a reviewable shape.
So I won't get back there.

You can look at the references from the root mail in this thread,
or we can move the conversation to a different place, if you want to
discuss more about it.

> How do you see a tracing-based solution that satisfies these safety
> requirements? Something running on an external chip and reading the processor
> trace? Something involving clearly separated VMs?
> I know that's what some fields do (avionics), but is the overhead worth it?

We simply create a protected execution environment within the kernel.
Several, actually.

> If we assume metadata can get corrupted in any way, I'm not quite sure how a
> software can ever be considered safe.

With in-kernel fences, that's what we do it. As presented in the links
from the first mail.

-- 
thanks, igor

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Steven Rostedt]

On Wed, 9 Sep 2026 19:07:40 +0300
"igor.stoppa@gmail.com" <igor.stoppa@gmail.com> wrote:

> Well, that is where I beg to differ. We have implemented such a mechanism.
> But I've been repeatedly told that I should provide first the code,
> which right now
> woulnd't really help much, because it is far from being in a reviewable shape.
> So I won't get back there.

Do you plan on attending Linux Plumbers? This looks like a topic for a BoF.

-- Steve

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

On Wed, 9 Sept 2026 at 19:12, Steven Rostedt <rostedt@goodmis.org> wrote:

> Do you plan on attending Linux Plumbers? This looks like a topic for a BoF.

Shameless plug: actually, I have a presentation [1]

It's not exactly on the same topic, though.

A BoF would be also good.

The reason for the initial post in this thread was to try to establish
a first connection
and see if there was any interest at all, even in the mechanisms
themselves, without the
safety baggage.
Precisely because safety has a very high entrance fee, in terms of
what one must internalize first,
I was looking for an easier hook, like security/integrity.

But if there is any interest in safety itself, that's even more than I
was wishing for.
The catch is that our flavor of safety is more focused on mechanisms
and architecture, than process.
Not because the process doesn't matter, but because it seems that
everyone else is already talking about that.

But I thought (wrongly?) that it was already game over for BoFs ...

[1] https://lpc.events/event/20/contributions/2492/

-- 
thanks, igor

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Steven Rostedt]

On Wed, 9 Sep 2026 19:24:01 +0300
"igor.stoppa@gmail.com" <igor.stoppa@gmail.com> wrote:

> On Wed, 9 Sept 2026 at 19:12, Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > Do you plan on attending Linux Plumbers? This looks like a topic for a BoF.  
> 
> Shameless plug: actually, I have a presentation [1]

Great, I'll likely attend that.

> 
> But I thought (wrongly?) that it was already game over for BoFs ...

Before the CfP ends, accepted BoFs get one free pass. After that, BoFs are
more of a first-come/first-serve ordeal with the amount of rooms we have,
and they are only for those that are attending. We accept BoFs throughout
the conference, as long as there is a room available.

There's just no A/V, unless you use your own laptop and connect to the BBB
server (if you want remote attendees). We also have some jabra speakers we
loan out.

-- Steve

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Gabriele Monaco]

On Wed, 2026-09-09 at 19:07 +0300, igor.stoppa@gmail.com wrote:
> On Wed, 9 Sept 2026 at 18:22, Gabriele Monaco <gmonaco@redhat.com> wrote:
> 
> > In general I feel anything that tries to validate the kernel from the kernel
> > itself will have this kind of problems. It cannot really be isolated from
> > what it tries to verify.
> 
> Well, that is where I beg to differ. We have implemented such a mechanism.
> But I've been repeatedly told that I should provide first the code,
> which right now woulnd't really help much, because it is far from being in a
> reviewable shape.
> So I won't get back there.
> 
> You can look at the references from the root mail in this thread,
> or we can move the conversation to a different place, if you want to
> discuss more about it.

Right, I didn't think I had access to the entire conversation but it's all on
lore.

Now I remember, you discussed this at last year's LPC.
RV is indeed not quite a fit for this.

I'll be trying to follow your presentation this year. Also a BoF on this sounds
interesting to get the ball rolling.

Thanks,
Gabriele

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Greg KH]

On Tue, Sep 08, 2026 at 12:42:32AM +0300, igor.stoppa@gmail.com wrote:
> - if indeed there is interest, then we can try to prioritize this rework
> But we would also seek an agreement that the final shape must
> still satisfy our safety-oriented needs.
> After all, what would be the point of upstreaming something, if then
> the end result turned out to be unusable for our main purpose?
> It needs to remain compatible with its original intended use.
> That is one of the points I'd like to explain.

If you don't even try to create something that can be upstreamed, and
submit it for review, then there's nothing that we can ever say other
than "I guess they never tried" :(

> >  If you don't have working patches, there's not much we can
> > really comment on, right?
> 
> I did not propose a code review, though.

But that's what we care about, working code that solves a problem that
you have, in a format that we can discuss.

> I proposed a discussion about the architectural changes supporting the
> features we have implemented.

I didn't see a specific feature, nor how it was implemented, in your
email, just some videos which aren't really "reviewable" at all.

So, along with what Ted said, turn your ideas and proposals into
something that meets your and your customer's needs, in the format which
we can actually review it in,  and submit it for inclusion and see what
happens!  I don't see anything "special" about your topic from any of
the thousands of other new Linux kernel features/additions that warrents
a change in our development process, do you?

thanks,

greg k-h

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

On Tue, 8 Sept 2026 at 08:05, Greg KH <gregkh@linuxfoundation.org> wrote:

> If you don't even try to create something that can be upstreamed, and
> submit it for review, then there's nothing that we can ever say other
> than "I guess they never tried" :(

The problem is not that we don't want to try.
The problem is that the ways it can go wrong or in the wrong direction
are too many.
And the hope was that we could prune a good deal of dead branches upfront.
But I will try to explain it better a few lines below.


> But that's what we care about, working code that solves a problem that
> you have, in a format that we can discuss.

There are 2 aspects:

1. working code that solves problems that we have, but those are for safety
2. working code that solves problems that upstream has, for example
integrity and security

Here the proposition was about discussing #2 and see if we could leverage #1
Sort of win-win scenario. We don't really expect much interest in #1 per-se.
Not even trying to promote it, at this point.
Hence the more open-ended approach.

But you make a valid point, here below.

> I didn't see a specific feature, nor how it was implemented, in your
> email, just some videos which aren't really "reviewable" at all.

Fair point. For good or for bad, I got so used to interrogating some tools that
I didn't consider they would not be used. Sorry about that.

But it's very easy to fix =)

What we are doing, in a nutshell:
On ARM64, we are enabling per-core, per-function, selective transition between
different kernel memory maps, with different write capabilities.
Kinda of MMU enforced rings.
And we pair that with the ability of controlling memory allocations, so that
specific allocation requests are served from matching memory pools.
The same is extended to userspace creation, of course.
Scheduling and other stuff are involved as well, but I hope this gives
at least a generic idea.

Why?
For "our" use case (#1) we do it because we need to protect selected processes
and/or drivers data from the kernel at large.
This part might be controversial, but in a functional safety world,
the specific user process is what matters, and the monolithic kernel
is assumed to contain
lurking bugs that at some point will bite.
Unlike the security scenario, where a malicious agent is required, here
the simple assumption is that - statistically - if one or more bugs
are present, they will bite.
Given e.g. a large amount of cars (any car, not necessarily self
driving ones) circulating over
a large period of time, the number of failures climbs fast. So,
something needs to be done about that.

But what I was hoping to discuss at the ksummit was "your" case as in:
"would the kernel community might find anything useful in what we
have, to solve some of their open problems?"
This is where I wouldn't even know what problem to tackle, that might
be considered worthy.
Or I'd rather hear it first from those who would then review the patches.


> So, along with what Ted said, turn your ideas and proposals into
> something that meets your and your customer's needs,

We will do that. No worries.
But this round was actually about upstream's needs. Not (only) ours.

> I don't see anything "special" about your topic from any of
> the thousands of other new Linux kernel features/additions that warrents
> a change in our development process, do you?

"special" tends to imply "better" - in that sense, no, we are not special

but "different"? perhaps that we are.

AFAIK we have a few constraints that are not typically present in
upstream development.

The "special" constraint here is "sustained qualifiability", or the
ability of obtaining and preserving
a certificate from a qualified entity. Which is what is necessary, to
support business.

Without getting too much into details, in order to obtain a
certificate, it's not only about the development
process, but it is also about being able to demonstrate how the actual
code fares wrt defects.

As an example, one of the parameters involved is complexity,
which acts as a proxy toward the confidence that whatever analysis
done on it can be trusted.

In practice it means that not only we need to have certain functionality,
but also that the code providing it must be of a certain shape.
Which might not always be aligned with what kernel common practices expect.

As a practical example, the entire memory management we do not deem to be safe.
We work around that by introducing additional barriers and validation.

This might sound bad (and I hope it doesn't come across as
disparaging) but, in reality,
it gives us the opportunity of actually following upstream much more
closely, because we
do not need to put too many expectations (in terms of safety) on the
bulk of upstream code.

The price we must pay, though, is that our add-on code needs to be
kept more isolated
from the rest, so that we can claim that it is unaffected by potential
bugs that might be present there.

Again, AFAIK, this sort of constraint is not addressed by regular
kernel practices.
And even less supported by regression testing that might follow, once
a feature was merged.

We would like to discuss this perhaps "unprecedented" rather than
"special" situation.

Because, while it is true that upfront it makes things harder to
merge, it also offers the opportunity
of tackling problems like enhanced availability, or improved security.

E.g. write protecting certain (meta)data from accidental scribbling,
or preventing an attacker from
gaining read/write access to special kernel (meta)data (keyring?) as
soon as a foothold in the kernel is established.

For us, it's a side effect, because we would normally protect things
from the kernel rather than the kernel from
other things (or itself). But what we have are mechanisms. What to
protect/hide from what is policy.

Preferences about these targets for protection would go a long way in
directing the potential development for an RFC patchset.

With the caveat - as I mentioned - that the mechanisms introduced
would have to remain in a shape that
can attain and retain qualification.

--
thanks, igor

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Greg KH]

On Tue, Sep 08, 2026 at 02:26:04PM +0300, igor.stoppa@gmail.com wrote:
> > But that's what we care about, working code that solves a problem that
> > you have, in a format that we can discuss.
> 
> There are 2 aspects:
> 
> 1. working code that solves problems that we have, but those are for safety
> 2. working code that solves problems that upstream has, for example
> integrity and security

You are forgetting that "you" are "upstream".  There is no us vs. them.
If you want to use Linux, you are "us".  Welcome to the community :)

> What we are doing, in a nutshell:
> On ARM64, we are enabling per-core, per-function, selective transition between
> different kernel memory maps, with different write capabilities.
> Kinda of MMU enforced rings.
> And we pair that with the ability of controlling memory allocations, so that
> specific allocation requests are served from matching memory pools.
> The same is extended to userspace creation, of course.
> Scheduling and other stuff are involved as well, but I hope this gives
> at least a generic idea.

That is very very vague, sorry.  Just send patches showing what you have
done.

> > So, along with what Ted said, turn your ideas and proposals into
> > something that meets your and your customer's needs,
> 
> We will do that. No worries.
> But this round was actually about upstream's needs. Not (only) ours.

Again, if you use Linux, you are upstream.  To think otherwise means you
really don't want to use Linux.  No one is forcing you to use Linux
here, so if you have changes that you feel are required to meet your use
case, wonderful, that's what we all do on a daily basis.  Post them and
we can work through them like anything else.

thanks,

greg k-h

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

On Tue, 8 Sept 2026 at 14:54, Greg KH <gregkh@linuxfoundation.org> wrote:

> You are forgetting that "you" are "upstream".  There is no us vs. them.
> If you want to use Linux, you are "us".  Welcome to the community :)

Decisions about what gets merged in the Linux tree are taken by those who
are part of the hierarchy of maintainers. And, ultimately Linus.
That was the intended meaning.

[...]

> That is very very vague, sorry.  Just send patches showing what you have
> done.

As I replied to Ted, please take this as an attempt to find a faster track.
Having 40 minutes or so of discussion and potentially saving months of
iterations seemed compelling.
Even from the perspective of whoever might want to review it.

> so if you have changes that you feel are required to meet your use
> case, wonderful, that's what we all do on a daily basis.  Post them and
> we can work through them like anything else.

While it's clear the value put on code, I have to confess that it's
less clear to me
where/how to initiate any discussion about the kernel development
process itself,
what it entails, and if/how it can be amended.

I have tried to explain the constraints we have to deal with.
While I'm not expecting an answer, perhaps you can advise where to
bring up this topic?
Clearly you don't think it's suitable for ksummit ;) , but then where?

-- 
igor

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Greg KH]

On Tue, Sep 08, 2026 at 03:25:48PM +0300, igor.stoppa@gmail.com wrote:
> On Tue, 8 Sept 2026 at 14:54, Greg KH <gregkh@linuxfoundation.org> wrote:
> 
> > You are forgetting that "you" are "upstream".  There is no us vs. them.
> > If you want to use Linux, you are "us".  Welcome to the community :)
> 
> Decisions about what gets merged in the Linux tree are taken by those who
> are part of the hierarchy of maintainers. And, ultimately Linus.
> That was the intended meaning.

Maintainers review code, without that, this whole discussion is moot.

> > so if you have changes that you feel are required to meet your use
> > case, wonderful, that's what we all do on a daily basis.  Post them and
> > we can work through them like anything else.
> 
> While it's clear the value put on code, I have to confess that it's
> less clear to me
> where/how to initiate any discussion about the kernel development
> process itself,
> what it entails, and if/how it can be amended.

Why are you trying to change a process that works for 4000+ people and
550+ different companies every year without even a solid real proposal
that we can review?  This is baffling to me.

> I have tried to explain the constraints we have to deal with.

I did not see any concrete constraint, just vague assumptions.

Again, please show the code, without that, none of this means anything
to us, nor should you want it to (otherwise what would keep everyone out
there sending us vague proposals as to why they are different from
everyone else and slow us all down?)

> While I'm not expecting an answer, perhaps you can advise where to
> bring up this topic?
> Clearly you don't think it's suitable for ksummit ;) , but then where?

I would suggest stepping back and learning a bit more about how we do
development here.  The first step in interacting with us, is you posting
working code for your need.  That proves that this all is something that
you really have gotten working and want actual review on how to get it
merged.  Without that, there's nothing we can do here.

thanks,

greg k-h

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

On Tue, 8 Sept 2026 at 15:39, Greg KH <gregkh@linuxfoundation.org> wrote:

> Why are you trying to change a process that works for 4000+ people and
> 550+ different companies every year without even a solid real proposal
> that we can review?  This is baffling to me.

The code will come later. This was truly just an attempt at speedin up
the interaction.
And possibly finding interest. But I acknowledge it did not work.
I hope in due time it will be more clear, once we publish the code, I
won't insist now.

Actually, I want to apologise if this was perceived as an attempt to
force things.
It was never meant to be.

--
thanks, igor

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Miguel Ojeda]

On Tue, Sep 8, 2026 at 2:26 PM igor.stoppa@gmail.com
<igor.stoppa@gmail.com> wrote:
>
> While it's clear the value put on code, I have to confess that it's
> less clear to me
> where/how to initiate any discussion about the kernel development
> process itself,
> what it entails, and if/how it can be amended.
>
> I have tried to explain the constraints we have to deal with.
> While I'm not expecting an answer, perhaps you can advise where to
> bring up this topic?
> Clearly you don't think it's suitable for ksummit ;) , but then where?

I think what you are trying to say is that your process to retain
qualification would require some changes in the upstream development
process (even if we assume the code is merged). Is this correct? It is
at least what I would expect, just guessing from the little bit I know
about functional safety.

If so, the issue is that you will have to convince everyone else to
adapt, at least around the relevant subsystems. It will be hard, and
you will have to be quite clear on what you are aiming for -- it is
unclear to me what you will exactly need. For instance, will you need
the kernel to run a certain process before every release, or to
maintain a traceability matrix of some sort, or...? Will you do all
that on your side only for certain releases instead? etc.

In any case, before any of that, what you are being asked to do is to
show what you have so that maintainers can understand better what kind
of changes (on the technical side, not the process side) you would
intend to put on the kernel. That will likely clarify matters, and if
the technical changes aren't suitable, then there is really little
chance to even attempt to adapt the kernel development process for
what you need.

I hope that helps...

Cheers,
Miguel

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

On Tue, 8 Sept 2026 at 16:11, Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:

> I think what you are trying to say is that your process to retain
> qualification would require some changes in the upstream development
> process (even if we assume the code is merged). Is this correct?

In a broad sense, yes.
The actual development itself, no.
Nor would it entail any generation of artifacts.

Many safety-oriented approaches have tried to steer upstream processes
toward safety-oriented practices. With little results.

We don't need that, because we simply put the portions of the kernel
that are not safety qualified outside of "fences" that they provably
cannot silently cross.

OTOH the fences themselves are what needs to be provably safe.
So they must meet various requirements about complexity, etc.

The change, or rather extension, would consist in broadening the
concept of what constitutes a regression.

Presently, if a new patch introduces either a vulnerability,
or a regression in either functionality or performance,
it gets rejected or at least it must be fixed accordingly.

We would need the safety-qualification criteria to be added to the list of
properties measured and preserved across releases.

> It is
> at least what I would expect, just guessing from the little bit I know
> about functional safety.

Yes, you  sort of hit the mark, but in reality I have a feeling you might
have been expecting more draconian requirements.
One of the advantages of not having a purely monolithic kernel anymore
is that we require less cooperation from the subsystems.

> If so, the issue is that you will have to convince everyone else to
> adapt, at least around the relevant subsystems.

It's mostly about tolerating the fact that the fences will be implemented in a
sort of self-contained way, and they must stay like that.
Adn recognising it as a regression, any attempt to change that property.

> It will be hard, and
> you will have to be quite clear on what you are aiming for -- it is
> unclear to me what you will exactly need. For instance, will you need
> the kernel to run a certain process before every release, or to
> maintain a traceability matrix of some sort, or...? Will you do all
> that on your side only for certain releases instead? etc.

Ideally, the kernel could become safe out-of-the-box.
Once the code is published, there is no value in retaining a standalone fork.
It's mostly a necessary pain.

> In any case, before any of that, what you are being asked to do is to
> show what you have so that maintainers can understand better what kind
> of changes (on the technical side, not the process side) you would
> intend to put on the kernel.

Yes, it's a bit of a chicken-egg problem, in the sense that not having a clear
short term visibility on even the possibility of said qualifiability takes away
momentum from any public work.

We will get to that point of publishing the code, eventually.
At the latest, when the forked tree gets published.

> That will likely clarify matters, and if
> the technical changes aren't suitable, then there is really little
> chance to even attempt to adapt the kernel development process for
> what you need.
>
> I hope that helps...

Yes, thanks.
Having the technical aspect behind, what seemed more problematic was the
normative side. You mentioned you have fusa experience, so you know
what I'm talking about.

-- 
thanks, igor

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Theodore Tso]

On Tue, Sep 08, 2026 at 04:52:13PM -0500, igor.stoppa@gmail.com wrote:
> Presently, if a new patch introduces either a vulnerability,
> or a regression in either functionality or performance,
> it gets rejected or at least it must be fixed accordingly.
> 
> We would need the safety-qualification criteria to be added to the list of
> properties measured and preserved across releases.

So perhaps you need to start with everything out-of-tree, and when
natural changes upstream changes, you can send patches upstream to
"fix" the issue, and we can see how annoyed everyone gets.

Your claim that it's not going to be draconian, but I think you need
to demonstrate whether or not this is actually the case.  Let's see
how much cooperation you will need from the various subsystems, and
whether it's going to involve a performance tax.  (And if there is a
performance tax, how bad is it going to be.)

Hint: if there needs to be a performance tax, it should only be paid
for those systems that care about the safety certification.  Maybe for
"safety certified" systems, people will be willing to pay a 20%
performance tax.  But everyone else.... probably not.

						- Ted

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

On Tue, 8 Sept 2026 at 17:45, Theodore Tso <tytso@mit.edu> wrote:

> So perhaps you need to start with everything out-of-tree, and when
> natural changes upstream changes, you can send patches upstream to
> "fix" the issue, and we can see how annoyed everyone gets.

It would be more about declaring the fences themselves as in need of
additional criteria.
We could probably even provide automation for detecting regressions as
part of the
regular kernel test automation, rather than having a reactive approach.
The potential regressions I'm referring to are related to the fences themselves.

> Your claim that it's not going to be draconian, but I think you need
> to demonstrate whether or not this is actually the case.  Let's see
> how much cooperation you will need from the various subsystems, and
> whether it's going to involve a performance tax.  (And if there is a
> performance tax, how bad is it going to be.)

As I mentioned, once the framework is disabled, there is no overhead,
it is compiled off. Practically binary-identical to a kernel compiled
without the patches.

> Hint: if there needs to be a performance tax, it should only be paid
> for those systems that care about the safety certification.

That's already the case, that features are disabled.

>  Maybe for
> "safety certified" systems, people will be willing to pay a 20%
> performance tax.  But everyone else.... probably not.

That 20%, I guess it comes from the typical microkernel world, where
one runs most
 of the drivers/services in what amounts more or less to a process.

Again, it is understandable that you might infer so, based on what
most systems do, but that's not what happens in our case.

And yes, without knowing the exact architecture anyone can guess what they want.
So, it will have to wait, if other-than-code explanations are not suitable.


-- 
igor

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[James Bottomley]

On Tue, 2026-09-08 at 14:26 +0300, igor.stoppa@gmail.com wrote:
> On Tue, 8 Sept 2026 at 08:05, Greg KH <gregkh@linuxfoundation.org>
> wrote:
> 
> > If you don't even try to create something that can be upstreamed,
> > and submit it for review, then there's nothing that we can ever say
> > other than "I guess they never tried" :(
> 
> The problem is not that we don't want to try.
> The problem is that the ways it can go wrong or in the wrong
> direction are too many.
> And the hope was that we could prune a good deal of dead branches
> upfront.
> But I will try to explain it better a few lines below.

You seem to think some people in upstream magically know the correct
direction: they don't; they need you to explore the space and come up
with concrete proposals.  The best way of doing this is to form a wide
but loose consensus first, so I'd really suggest again that you try and
engage with ELISA.  They know the space better (to guide you) and they
have stuff they want to upstream as well so they can tell you what they
think will fly and what won't.

Regards,

James

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

On Tue, 8 Sept 2026 at 15:41, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:

> You seem to think some people in upstream magically know the correct
> direction:

I was looking for any outstanding known problem, not necessarily
related to safety,
that upstream maintainers might have been already aware of, or interested about.
To see if it could have been a good match for the mechanisms we have developed.

The idea has backfired, so we'll go down the usual route.

-- 
thanks, igor

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Steven Rostedt]

On Tue, 8 Sep 2026 16:03:03 +0300
"igor.stoppa@gmail.com" <igor.stoppa@gmail.com> wrote:

> On Tue, 8 Sept 2026 at 15:41, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
> 
> > You seem to think some people in upstream magically know the correct
> > direction:  
> 
> I was looking for any outstanding known problem, not necessarily
> related to safety,
> that upstream maintainers might have been already aware of, or interested about.
> To see if it could have been a good match for the mechanisms we have developed.
> 
> The idea has backfired, so we'll go down the usual route.
> 

Note, the PREEMPT_RT project made a deliberate point to do all work through
LKML, even though we were basically a fork of the kernel. Several
developers asked us to create our own mailing list for which we refused.
The reason was simple. We wanted our work to be seen by everybody as people
could see the rationale behind decisions we made. It also brought new
people to the project when they saw we were working on something that they
thought could benefit them.

Perhaps if you do your development on LKML with something in the subject
showing that it is for your fork (like we used "[RT PATCH]" to
differentiate our work from normal kernel), but it could elicit those that
find your work interesting, and then you may get the discussion you are
looking for.

-- Steve

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[igor.stoppa]

On Tue, 8 Sept 2026 at 18:39, Steven Rostedt <rostedt@goodmis.org> wrote:

> Perhaps if you do your development on LKML with something in the subject
> showing that it is for your fork (like we used "[RT PATCH]" to
> differentiate our work from normal kernel), but it could elicit those that
> find your work interesting, and then you may get the discussion you are
> looking for.

That's very good advice, thanks.
It seemed a bit intrusive to spam everyone with it, but if that's how
this sort of thing
gets discussed, sure, why not.

-- 
thanks, igor

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Steven Rostedt]

On Tue, 8 Sep 2026 19:09:09 +0300
"igor.stoppa@gmail.com" <igor.stoppa@gmail.com> wrote:

> On Tue, 8 Sept 2026 at 18:39, Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > Perhaps if you do your development on LKML with something in the subject
> > showing that it is for your fork (like we used "[RT PATCH]" to
> > differentiate our work from normal kernel), but it could elicit those that
> > find your work interesting, and then you may get the discussion you are
> > looking for.  
> 
> That's very good advice, thanks.
> It seemed a bit intrusive to spam everyone with it, but if that's how
> this sort of thing
> gets discussed, sure, why not.
> 

Just don't Cc the maintainers of the code you modify, as they may think
it's for inclusion. Cc'ing LKML is one thing, as the only ones that read it
are those that are curious about what is happening around the kernel. If it
has some marking in the subject that denotes the patch is for a fork of the
kernel (that you would be happy to be included one day) it should be fine.

-- Steve

Re: [TECH TOPIC] Improving kernel security & integrity by generalizing ad-hoc safety mechanisms

[Theodore Tso]

On Tue, Sep 08, 2026 at 01:35:45PM -0500, Steven Rostedt wrote:
> Just don't Cc the maintainers of the code you modify, as they may think
> it's for inclusion. Cc'ing LKML is one thing, as the only ones that read it
> are those that are curious about what is happening around the kernel. If it
> has some marking in the subject that denotes the patch is for a fork of the
> kernel (that you would be happy to be included one day) it should be fine.

The one exception I'd make to Steven's good advice is if you have a
pre-established working relationship with the maintainer, and the
subject is prefixed with something like [RFC PATCH] or [PATCH SAFETY],
you can ask them to give their opinion of their approach in your patch
set.  (Especially if this is a feature that you are asking if this is
something which in their opinion might be ripe for inclusion in the
subsystem in the upstream feature.)

						- Ted