Vector on Banana Pi BPI-F3

Vector on Banana Pi BPI-F3

[Robbin Ehn]

Hey!

There seems to be a large interest in running openjdk with vector on
this board out-of-the-box.
The vendor kernel have vector from 6.4 back-ported without hwprobe.
And FYI: the vendor kernel for k230 has THEAD vector patches.
Both do the right thing during context switches (sources have been checked).

It seems like we have three options, anymore?

A:
Keep what we do: i.e. no hwprobe no V.
Downside no V (default) on contemporary boards.
Hence no V coverage in CI testing and disappointed early adopters.

B:
Enable it for triplet mvendorid/marchid/mimplid.
Only V for this specific board, other boards and/or updates to this
board require new binaries.

C:
Enable it for boards with V in hwcap.
Downside boards false reporting V (1.0), such as K230 small-CPU and THEAD,
needs to be handled, safe fetch vsetvli and vsetvli v1.0 check can fix
those two examples.
If we miss a board or a new board comes out we may lack proper check thus crash.

In PRs:
https://github.com/openjdk/jdk/pull/19472
https://github.com/openjdk/jdk/pull/19679

Options A was used.

The question is if we can reconsider due to the benefits and use
another option allowing V to automatically work?

Thanks, Robbin

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: Vector on Banana Pi BPI-F3

[Conor Dooley]

[-- Attachment #1.1: Type: text/plain, Size: 4009 bytes --]

Yo,

On Mon, Jul 01, 2024 at 11:08:31AM +0200, Robbin Ehn wrote:
> There seems to be a large interest in running openjdk with vector on
> this board out-of-the-box.
> The vendor kernel have vector from 6.4 back-ported without hwprobe.
> And FYI: the vendor kernel for k230 has THEAD vector patches.
> Both do the right thing during context switches (sources have been checked).
> 
> It seems like we have three options, anymore?
> 
> A:
> Keep what we do: i.e. no hwprobe no V.
> Downside no V (default) on contemporary boards.
> Hence no V coverage in CI testing and disappointed early adopters.

Upside: pressure on vendors not to ship old, heavily modified kernels.
I'd be pretty vociferous about pushing the blame on the vendors were I
in your shoes! There's some work in progress for mainline spacemit k1
support that would allow you to get something usable for CI although
obviously these things move slowly. Palmer did mention that if this
board is helpful for compiler testing that some of the Rivos folks might
work on support for it in mainline though, which would be neat...

> B:
> Enable it for triplet mvendorid/marchid/mimplid.
> Only V for this specific board, other boards and/or updates to this
> board require new binaries.

And, depending on implementation, may break if the kernel running on the
board doesn't support vector. You'd need to check hwcap and the triplet.
m*id all come from the CPU, so you can't be sure that the values there
will uniquely identify a specific board or SoC for you anyway. If you
want to support the can of worms you're opening there with m*id stuff
for vendor kernels, then go for it. If you start it for vector on one
board, where do you draw a line in terms of either extension or boards?

> C:
> Enable it for boards with V in hwcap.
> Downside boards false reporting V (1.0), such as K230 small-CPU and THEAD,

Why does the small K230 CPU falsely report that? Does it have no vector
and report that it does? Or does it support 0.7 like the other T-head
stuff? I suppose the latter given they the patches you say their kernel
has.

One thing that we did in the kernel was, when parsing a devicetree, to
ignore v in riscv,isa where the CPU vendor was T-Head and the archid was
0x0. It wasn't meant to be a perfect detection of incorrect devicetrees,
but a simple best-effort attempt to handle vendor provided dtbs for
Vector 0.7 devices. Best-effort is fine in this case, cos ultimately it
isn't the kernel's problem if someone has incorrectly described their
hardware. Guo Ren claimed that no vector 1.0 devices have a 0x0 marchid,
but I do not know if there are vector 0.7 devices with a non-zero marchid.

I'd be very curious as to what the small cpu reports for marchid/mimpid,
to see if that rule would still hold true. If they happens to be non-zero,
mainline wouldn't even run on it, even ignoring vector, without either an
OpenSBI and devicetree replacement or kernel changes, so the courtesy
disabling of vector wouldn't even trigger. What it would mean, is a
similar approach to disable vector on 0.7 devices wouldn't suffice for
you.

> needs to be handled, safe fetch vsetvli and vsetvli v1.0 check can fix
> those two examples.
> If we miss a board or a new board comes out we may lack proper check thus crash.

With my only mainline matters blinkers on, that's another tick in the
"pressure on vendors to not ship old, heavily modified kernels" box!

> In PRs:
> https://github.com/openjdk/jdk/pull/19472
> https://github.com/openjdk/jdk/pull/19679
> 
> Options A was used.
> 
> The question is if we can reconsider due to the benefits and use
> another option allowing V to automatically work?

I think whether you reconsider isn't really a topic for lkml, it seems
more like a policy decision for your project as there's not a magic bullet
we can offer you that can be sure not to break if you intent supporting
"random" vendor kernels.

Cheers,
Conor.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[-- Attachment #2: Type: text/plain, Size: 161 bytes --]

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: Vector on Banana Pi BPI-F3

[Robbin Ehn]

Hey, thanks for chiming in!

On Mon, Jul 1, 2024 at 2:09 PM Conor Dooley <conor@kernel.org> wrote:
>
> Yo,
>
> On Mon, Jul 01, 2024 at 11:08:31AM +0200, Robbin Ehn wrote:
> > There seems to be a large interest in running openjdk with vector on
> > this board out-of-the-box.
> > The vendor kernel have vector from 6.4 back-ported without hwprobe.
> > And FYI: the vendor kernel for k230 has THEAD vector patches.
> > Both do the right thing during context switches (sources have been checked).
> >
> > It seems like we have three options, anymore?
> >
> > A:
> > Keep what we do: i.e. no hwprobe no V.
> > Downside no V (default) on contemporary boards.
> > Hence no V coverage in CI testing and disappointed early adopters.
>
> Upside: pressure on vendors not to ship old, heavily modified kernels.
> I'd be pretty vociferous about pushing the blame on the vendors were I
> in your shoes! There's some work in progress for mainline spacemit k1
> support that would allow you to get something usable for CI although
> obviously these things move slowly. Palmer did mention that if this
> board is helpful for compiler testing that some of the Rivos folks might
> work on support for it in mainline though, which would be neat...

Yes, I know about the mainline effort, I didn't mention that because
if that work is not picked up by the vendor, most will probably stick
with the vendor kernel.

>
> > B:
> > Enable it for triplet mvendorid/marchid/mimplid.
> > Only V for this specific board, other boards and/or updates to this
> > board require new binaries.
>
> And, depending on implementation, may break if the kernel running on the
> board doesn't support vector. You'd need to check hwcap and the triplet.
> m*id all come from the CPU, so you can't be sure that the values there
> will uniquely identify a specific board or SoC for you anyway. If you
> want to support the can of worms you're opening there with m*id stuff
> for vendor kernels, then go for it. If you start it for vector on one
> board, where do you draw a line in terms of either extension or boards?
>
> > C:
> > Enable it for boards with V in hwcap.
> > Downside boards false reporting V (1.0), such as K230 small-CPU and THEAD,
>
> Why does the small K230 CPU falsely report that? Does it have no vector
> and report that it does? Or does it support 0.7 like the other T-head
> stuff? I suppose the latter given they the patches you say their kernel
> has.

If you are running on the small CPU0 (no V) I was told hwcap still reported V !?
My board just runs the big CPU1 core with V 1.0, so I have not verified this.

>
> One thing that we did in the kernel was, when parsing a devicetree, to
> ignore v in riscv,isa where the CPU vendor was T-Head and the archid was
> 0x0. It wasn't meant to be a perfect detection of incorrect devicetrees,
> but a simple best-effort attempt to handle vendor provided dtbs for
> Vector 0.7 devices. Best-effort is fine in this case, cos ultimately it
> isn't the kernel's problem if someone has incorrectly described their
> hardware. Guo Ren claimed that no vector 1.0 devices have a 0x0 marchid,
> but I do not know if there are vector 0.7 devices with a non-zero marchid.
>

As we are talking about kernels without hwprobe, userspace can only
get it if it's reported via proc. (?)
(my k230 kernel for example does not)

> I'd be very curious as to what the small cpu reports for marchid/mimpid,
> to see if that rule would still hold true. If they happens to be non-zero,
> mainline wouldn't even run on it, even ignoring vector, without either an
> OpenSBI and devicetree replacement or kernel changes, so the courtesy
> disabling of vector wouldn't even trigger. What it would mean, is a
> similar approach to disable vector on 0.7 devices wouldn't suffice for
> you.
>
> > needs to be handled, safe fetch vsetvli and vsetvli v1.0 check can fix
> > those two examples.
> > If we miss a board or a new board comes out we may lack proper check thus crash.
>
> With my only mainline matters blinkers on, that's another tick in the
> "pressure on vendors to not ship old, heavily modified kernels" box!
>
> > In PRs:
> > https://github.com/openjdk/jdk/pull/19472
> > https://github.com/openjdk/jdk/pull/19679
> >
> > Options A was used.
> >
> > The question is if we can reconsider due to the benefits and use
> > another option allowing V to automatically work?
>
> I think whether you reconsider isn't really a topic for lkml, it seems
> more like a policy decision for your project as there's not a magic bullet
> we can offer you that can be sure not to break if you intent supporting
> "random" vendor kernels.

Yes, wrong list :) And I yes that's my opinion also.
But as it will signicant make life easier for many, I'm willing to let it slide.

Added the correct list, but I'll keep lkml if anyone else wants to chime in.

Thanks, Robbin

>
> Cheers,
> Conor.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: Vector on Banana Pi BPI-F3

[Conor Dooley]

[-- Attachment #1.1: Type: text/plain, Size: 5821 bytes --]

On Mon, Jul 01, 2024 at 03:19:51PM +0200, Robbin Ehn wrote:
> On Mon, Jul 1, 2024 at 2:09 PM Conor Dooley <conor@kernel.org> wrote:
> > On Mon, Jul 01, 2024 at 11:08:31AM +0200, Robbin Ehn wrote:
> > > There seems to be a large interest in running openjdk with vector on
> > > this board out-of-the-box.
> > > The vendor kernel have vector from 6.4 back-ported without hwprobe.
> > > And FYI: the vendor kernel for k230 has THEAD vector patches.
> > > Both do the right thing during context switches (sources have been checked).
> > >
> > > It seems like we have three options, anymore?
> > >
> > > A:
> > > Keep what we do: i.e. no hwprobe no V.
> > > Downside no V (default) on contemporary boards.
> > > Hence no V coverage in CI testing and disappointed early adopters.
> >
> > Upside: pressure on vendors not to ship old, heavily modified kernels.
> > I'd be pretty vociferous about pushing the blame on the vendors were I
> > in your shoes! There's some work in progress for mainline spacemit k1
> > support that would allow you to get something usable for CI although
> > obviously these things move slowly. Palmer did mention that if this
> > board is helpful for compiler testing that some of the Rivos folks might
> > work on support for it in mainline though, which would be neat...
> 
> Yes, I know about the mainline effort, I didn't mention that because
> if that work is not picked up by the vendor, most will probably stick
> with the vendor kernel.

Right. I meant that specifically on the CI point.

> > > B:
> > > Enable it for triplet mvendorid/marchid/mimplid.
> > > Only V for this specific board, other boards and/or updates to this
> > > board require new binaries.
> >
> > And, depending on implementation, may break if the kernel running on the
> > board doesn't support vector. You'd need to check hwcap and the triplet.
> > m*id all come from the CPU, so you can't be sure that the values there
> > will uniquely identify a specific board or SoC for you anyway. If you
> > want to support the can of worms you're opening there with m*id stuff
> > for vendor kernels, then go for it. If you start it for vector on one
> > board, where do you draw a line in terms of either extension or boards?
> >
> > > C:
> > > Enable it for boards with V in hwcap.
> > > Downside boards false reporting V (1.0), such as K230 small-CPU and THEAD,
> >
> > Why does the small K230 CPU falsely report that? Does it have no vector
> > and report that it does? Or does it support 0.7 like the other T-head
> > stuff? I suppose the latter given they the patches you say their kernel
> > has.
> 
> If you are running on the small CPU0 (no V) I was told hwcap still reported V !?
> My board just runs the big CPU1 core with V 1.0, so I have not verified this.

"At least" falsely reporting vector when there's none is a pretty simple
fix for the vendor (if they care). Having different versions of vector
on different CPUs would be much more annoying. And ye, I asked on IRC
and someone said it does report V as you say.

> > One thing that we did in the kernel was, when parsing a devicetree, to
> > ignore v in riscv,isa where the CPU vendor was T-Head and the archid was
> > 0x0. It wasn't meant to be a perfect detection of incorrect devicetrees,
> > but a simple best-effort attempt to handle vendor provided dtbs for
> > Vector 0.7 devices. Best-effort is fine in this case, cos ultimately it
> > isn't the kernel's problem if someone has incorrectly described their
> > hardware. Guo Ren claimed that no vector 1.0 devices have a 0x0 marchid,
> > but I do not know if there are vector 0.7 devices with a non-zero marchid.
> >
> 
> As we are talking about kernels without hwprobe, userspace can only
> get it if it's reported via proc. (?)
> (my k230 kernel for example does not)

Oh god, does that come with the crappy old "5.10.4" kernel from T-Head?
I was expecting that info to be available in /proc/cpuinfo, but IIRC it
was only added in 2022's 6.1. That "5.10.4" vendor kernel is a parasite.

> > I'd be very curious as to what the small cpu reports for marchid/mimpid,
> > to see if that rule would still hold true. If they happens to be non-zero,
> > mainline wouldn't even run on it, even ignoring vector, without either an
> > OpenSBI and devicetree replacement or kernel changes, so the courtesy
> > disabling of vector wouldn't even trigger. What it would mean, is a
> > similar approach to disable vector on 0.7 devices wouldn't suffice for
> > you.
> >
> > > needs to be handled, safe fetch vsetvli and vsetvli v1.0 check can fix
> > > those two examples.
> > > If we miss a board or a new board comes out we may lack proper check thus crash.
> >
> > With my only mainline matters blinkers on, that's another tick in the
> > "pressure on vendors to not ship old, heavily modified kernels" box!
> >
> > > In PRs:
> > > https://github.com/openjdk/jdk/pull/19472
> > > https://github.com/openjdk/jdk/pull/19679
> > >
> > > Options A was used.
> > >
> > > The question is if we can reconsider due to the benefits and use
> > > another option allowing V to automatically work?
> >
> > I think whether you reconsider isn't really a topic for lkml, it seems
> > more like a policy decision for your project as there's not a magic bullet
> > we can offer you that can be sure not to break if you intent supporting
> > "random" vendor kernels.
> 
> Yes, wrong list :) And I yes that's my opinion also.
> But as it will signicant make life easier for many, I'm willing to let it slide.
> 
> Added the correct list, but I'll keep lkml if anyone else wants to chime in.

Oh, I was almost not going to mention the list, glad I did if you
actually meant to send it elsewhere.

Cheers,
Conor.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[-- Attachment #2: Type: text/plain, Size: 161 bytes --]

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv