Cross Architecture Kernel Modules?

Cross Architecture Kernel Modules?

[Kenneth Adam Miller]

[-- Attachment #1: Type: text/plain, Size: 703 bytes --]

Hello all,

I just want to pose the following problem:

There is a kernel module for a non-native architecture, say, arch 1. For
performance reasons, the rest of all of the software needs to run natively
on a different arch, arch 2. Is there any way to perhaps run multiple QEMU
instances for the different architectures in such a way to minimize the
cross architecture performance penalty? For example, I would like the
kernel module in one (non-native) QEMU instance to be made available,
literally equivalently, in the second (native) QEMU instance. Would there
be any API or way to map across the QEMU instances so that the non native
arch kernel module could be mapped to the native QEMU instance?

[-- Attachment #2: Type: text/html, Size: 739 bytes --]

Re: Cross Architecture Kernel Modules?

[Alex Bennée]

Kenneth Adam Miller <kennethadammiller@gmail.com> writes:

> Hello all,
>
> I just want to pose the following problem: 
>
> There is a kernel module for a non-native architecture, say, arch 1. For performance reasons, the rest of all of the software needs to run
> natively on a different arch, arch 2. Is there any way to perhaps run multiple QEMU instances for the different architectures in such a way
> to minimize the cross architecture performance penalty? For example, I would like the kernel module in one (non-native) QEMU instance to
> be made available, literally equivalently, in the second (native) QEMU instance. Would there be any API or way to map across the QEMU
> instances so that the non native arch kernel module could be mapped to
> the native QEMU instance?

What you are describing sounds like heterogeneous system modelling which
QEMU only supports in a very limited way (all vCPUs must be the same
base architecture). You can link QEMU's together by way of shared memory
but there is no other wiring together done in that case although some
3rd party forks take this approach.

The kernel module sounds confusing - why would you have a kernel module
that wasn't the same architecture as the kernel you are running?

-- 
Alex Bennée

Re: Cross Architecture Kernel Modules?

[Kenneth Adam Miller]

[-- Attachment #1: Type: text/plain, Size: 1563 bytes --]

The source for it isn't available in order that it be compiled to the
desired architecture.

What 3rd party forks take this approach?

On Wed, Jan 19, 2022 at 2:06 PM Alex Bennée <alex.bennee@linaro.org> wrote:

>
> Kenneth Adam Miller <kennethadammiller@gmail.com> writes:
>
> > Hello all,
> >
> > I just want to pose the following problem:
> >
> > There is a kernel module for a non-native architecture, say, arch 1. For
> performance reasons, the rest of all of the software needs to run
> > natively on a different arch, arch 2. Is there any way to perhaps run
> multiple QEMU instances for the different architectures in such a way
> > to minimize the cross architecture performance penalty? For example, I
> would like the kernel module in one (non-native) QEMU instance to
> > be made available, literally equivalently, in the second (native) QEMU
> instance. Would there be any API or way to map across the QEMU
> > instances so that the non native arch kernel module could be mapped to
> > the native QEMU instance?
>
> What you are describing sounds like heterogeneous system modelling which
> QEMU only supports in a very limited way (all vCPUs must be the same
> base architecture). You can link QEMU's together by way of shared memory
> but there is no other wiring together done in that case although some
> 3rd party forks take this approach.
>
> The kernel module sounds confusing - why would you have a kernel module
> that wasn't the same architecture as the kernel you are running?
>
> --
> Alex Bennée
>

[-- Attachment #2: Type: text/html, Size: 1999 bytes --]

Re: Cross Architecture Kernel Modules?

[Kenneth Adam Miller]

[-- Attachment #1: Type: text/plain, Size: 1915 bytes --]

Would it be possible somehow to save the TCG cache, as with user binaries,
but for a kernel module, before then loading that kernel module into memory
the target architecture whether in or outside of QEMU?

On Wed, Jan 19, 2022 at 2:42 PM Kenneth Adam Miller <
kennethadammiller@gmail.com> wrote:

> The source for it isn't available in order that it be compiled to the
> desired architecture.
>
> What 3rd party forks take this approach?
>
> On Wed, Jan 19, 2022 at 2:06 PM Alex Bennée <alex.bennee@linaro.org>
> wrote:
>
>>
>> Kenneth Adam Miller <kennethadammiller@gmail.com> writes:
>>
>> > Hello all,
>> >
>> > I just want to pose the following problem:
>> >
>> > There is a kernel module for a non-native architecture, say, arch 1.
>> For performance reasons, the rest of all of the software needs to run
>> > natively on a different arch, arch 2. Is there any way to perhaps run
>> multiple QEMU instances for the different architectures in such a way
>> > to minimize the cross architecture performance penalty? For example, I
>> would like the kernel module in one (non-native) QEMU instance to
>> > be made available, literally equivalently, in the second (native) QEMU
>> instance. Would there be any API or way to map across the QEMU
>> > instances so that the non native arch kernel module could be mapped to
>> > the native QEMU instance?
>>
>> What you are describing sounds like heterogeneous system modelling which
>> QEMU only supports in a very limited way (all vCPUs must be the same
>> base architecture). You can link QEMU's together by way of shared memory
>> but there is no other wiring together done in that case although some
>> 3rd party forks take this approach.
>>
>> The kernel module sounds confusing - why would you have a kernel module
>> that wasn't the same architecture as the kernel you are running?
>>
>> --
>> Alex Bennée
>>
>

[-- Attachment #2: Type: text/html, Size: 2599 bytes --]

Re: Cross Architecture Kernel Modules?

[Alex Bennée]

Kenneth Adam Miller <kennethadammiller@gmail.com> writes:

> Would it be possible somehow to save the TCG cache, as with user binaries, but for a kernel module, before then loading that kernel
> module into memory the target architecture whether in or outside of
> QEMU?

OK let me stop you right there - TCG generated code needs a bunch of
runtime support from QEMU itself and is not designed to be run
standalone. All the existing issues with binary kernel modules get
multiplied once you have to account for architecture differences. Even
on the same architecture you can run into problems if the modules use
kernel structures that have changed between releases. Even if the kernel
was compiled for the exact same kernel release you are trying to run it
in changes in fields, padding and layout are going to be an issue.

In short this is not the solution you are looking for. Your time would
be better spent reverse engineering the proprietary module and writing a
open source version of it than trying to get something like this to
work.

Sorry.


>
> On Wed, Jan 19, 2022 at 2:42 PM Kenneth Adam Miller <kennethadammiller@gmail.com> wrote:
>
>  The source for it isn't available in order that it be compiled to the desired architecture.
>
>  What 3rd party forks take this approach?
>
>  On Wed, Jan 19, 2022 at 2:06 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>
>  Kenneth Adam Miller <kennethadammiller@gmail.com> writes:
>
>  > Hello all,
>  >
>  > I just want to pose the following problem: 
>  >
>  > There is a kernel module for a non-native architecture, say, arch 1. For performance reasons, the rest of all of the software
>  needs to run
>  > natively on a different arch, arch 2. Is there any way to perhaps run multiple QEMU instances for the different architectures in
>  such a way
>  > to minimize the cross architecture performance penalty? For example, I would like the kernel module in one (non-native) QEMU
>  instance to
>  > be made available, literally equivalently, in the second (native) QEMU instance. Would there be any API or way to map across
>  the QEMU
>  > instances so that the non native arch kernel module could be mapped to
>  > the native QEMU instance?
>
>  What you are describing sounds like heterogeneous system modelling which
>  QEMU only supports in a very limited way (all vCPUs must be the same
>  base architecture). You can link QEMU's together by way of shared memory
>  but there is no other wiring together done in that case although some
>  3rd party forks take this approach.
>
>  The kernel module sounds confusing - why would you have a kernel module
>  that wasn't the same architecture as the kernel you are running?
>
>  -- 
>  Alex Bennée


-- 
Alex Bennée

Re: Cross Architecture Kernel Modules?

[Peter Maydell]

On Wed, 19 Jan 2022 at 19:53, Kenneth Adam Miller
<kennethadammiller@gmail.com> wrote:
>
> The source for it isn't available in order that it be compiled to the desired architecture.

In general kernel modules need to be built not just for the right architecture
but even for the specific kernel version you're going to load them into.

In theory you could perhaps build QEMU into the native system kernel and
have it act as a shim between non-native kernel modules (this has been done
for UEFI modules before). But you would (unlike the UEFI case) run into
incompatibilities in data structure layouts in memory. In general it would
be a lot of work; it is certainly not something QEMU is anywhere near
being able to do today. I doubt it's something that either the upstream
kernel folks nor upstream QEMU would be interested in either.

The best approach would be to get the source for the kernel module.

-- PMM