[ANN] net-next is OPEN

[ANN] net-next is OPEN

[Jakub Kicinski]

Hi,

net-next is open again, and accepting changes for v6.10.

We seem to be in a good enough shape. For those trying to run selftests
locally - 9p has a UaF bug, see the testing tree for the fix we're
carrying locally. The changes in the timer subsystem also made a number
of tests flaky, beware.

Re: [TEST] VirtioFS instead of 9p (was: net-next is OPEN)

[Matthieu Baerts]

Hi Jakub,

On 25/03/2024 14:42, Jakub Kicinski wrote:
> net-next is open again, and accepting changes for v6.10.

Thank you for the notification!

> We seem to be in a good enough shape. For those trying to run selftests
> locally - 9p has a UaF bug, see the testing tree for the fix we're
> carrying locally.

With 'virtme-ng' used by NIPA, it is possible to use VirtioFS or
OverlayFS instead of 9p.

VirtioFS seems to perform much better than 9p [1]. All you need is to
install "virtiofsd" daemon in userspace [2]. It looks like Nipa is still
using 9p for the rootfs, it might be good to switch to VirtioFS if it is
easy :)

If you want to use OverlayFS (e.g. to mount the kselftests dir), you can
use "vng --overlay-rwdir". If you use "vng --rwdir" (or "vng --rodir"),
9p will be used. Maybe better to recommend that on the wiki [3]?

(The MPTCP CI didn't hit the bug with 9p, because it now uses vng with
VirtioFS.)

[1] https://lwn.net/Articles/951313/
[2] https://virtio-fs.gitlab.io/
[3]
https://github.com/linux-netdev/nipa/wiki/How-to-run-netdev-selftests-CI-style

> The changes in the timer subsystem also made a number
> of tests flaky, beware.

:'(

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.

Re: [TEST] VirtioFS instead of 9p (was: net-next is OPEN)

[Jakub Kicinski]

On Mon, 25 Mar 2024 17:32:54 +0100 Matthieu Baerts wrote:
> With 'virtme-ng' used by NIPA, it is possible to use VirtioFS or
> OverlayFS instead of 9p.
> 
> VirtioFS seems to perform much better than 9p [1]. All you need is to
> install "virtiofsd" daemon in userspace [2]. It looks like Nipa is still
> using 9p for the rootfs, it might be good to switch to VirtioFS if it is
> easy :)

"All you need is to install" undersells it a little :)

It's not packaged for silly OS^w^w AWS Linux. 
And the Rust that comes with it doesn't seem to be able to build it :(


error[E0658]: use of unstable library feature 'is_some_and'
  --> /home/virtme/.cargo/registry/src/github.com-1ecc6299db9ec823/vhost-user-backend-0.14.0/src/bitmap.rs:87:14
   |
87 |             .is_some_and(|bitmap| bitmap.dirty_at(self.base_address.saturating_add(offset)))
   |              ^^^^^^^^^^^
   |
   = note: see issue #93050 <https://github.com/rust-lang/rust/issues/93050> for more information

For more information about this error, try `rustc --explain E0658`.
error: could not compile `vhost-user-backend` due to previous error
warning: build failed, waiting for other jobs to finish...


Onto the ToDo pile it goes :)

> If you want to use OverlayFS (e.g. to mount the kselftests dir), you can
> use "vng --overlay-rwdir". If you use "vng --rwdir" (or "vng --rodir"),
> 9p will be used. Maybe better to recommend that on the wiki [3]?
> 
> (The MPTCP CI didn't hit the bug with 9p, because it now uses vng with
> VirtioFS.)

Re: [TEST] VirtioFS instead of 9p

[Matthieu Baerts]

Hi Jakub,

On 26/03/2024 02:42, Jakub Kicinski wrote:
> On Mon, 25 Mar 2024 17:32:54 +0100 Matthieu Baerts wrote:
>> With 'virtme-ng' used by NIPA, it is possible to use VirtioFS or
>> OverlayFS instead of 9p.
>>
>> VirtioFS seems to perform much better than 9p [1]. All you need is to
>> install "virtiofsd" daemon in userspace [2]. It looks like Nipa is still
>> using 9p for the rootfs, it might be good to switch to VirtioFS if it is
>> easy :)
> 
> "All you need is to install" undersells it a little :)
> 
> It's not packaged for silly OS^w^w AWS Linux.

Thank you for having tried! That's a shame "virtiofsd" is not packaged!

> And the Rust that comes with it doesn't seem to be able to build it :(
Did you try by installing Rust (rustc, cargo) via rustup [1]? It is even
possible to get the offline installer if it is easier [2]. With rustup,
you can easily install newer versions of the Rust toolchain.

[1] https://www.rust-lang.org/learn/get-started
[2] https://forge.rust-lang.org/infra/other-installation-methods.html


Do you need a hand to update the wiki page?

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.

Re: [TEST] VirtioFS instead of 9p

[Andrew Lunn]

> > And the Rust that comes with it doesn't seem to be able to build it :(
> Did you try by installing Rust (rustc, cargo) via rustup [1]? It is even
> possible to get the offline installer if it is easier [2]. With rustup,
> you can easily install newer versions of the Rust toolchain.

I guess part of the problem is that $RUST does not have a stable
meaning yet. The rust in the kernel seems to need a different
definition of $RUST to this package. And this machine is all about
testing the kernel, so $RUST is set for how the kernel wants $RUST
defined.

Maybe, eventually, rust will become stable, and you just install a
rust compiler like you install a C compiler, and it just works for
everything. But we are not there yet.

	Andrew

Re: [TEST] VirtioFS instead of 9p

[Jakub Kicinski]

On Tue, 26 Mar 2024 13:34:33 +0100 Andrew Lunn wrote:
> > > And the Rust that comes with it doesn't seem to be able to build it :(  
> > Did you try by installing Rust (rustc, cargo) via rustup [1]? It is even
> > possible to get the offline installer if it is easier [2]. With rustup,
> > you can easily install newer versions of the Rust toolchain.  
> 
> I guess part of the problem is that $RUST does not have a stable
> meaning yet. The rust in the kernel seems to need a different
> definition of $RUST to this package. And this machine is all about
> testing the kernel, so $RUST is set for how the kernel wants $RUST
> defined.
> 
> Maybe, eventually, rust will become stable, and you just install a
> rust compiler like you install a C compiler, and it just works for
> everything. But we are not there yet.

Somewhat related, our current build_rust test doesn't work because
I used rustup, and it works by adding stuff (paths mostly?) to bashrc.
Which does not get evaluated when we launch the script from a systemd
unit :( I couldn't find a "please run this as an interactive shell"
switch in bash, should we source ~/.bashrc in build_rust.sh for now?

Re: [TEST] VirtioFS instead of 9p

[Jakub Kicinski]

On Tue, 26 Mar 2024 11:27:17 +0100 Matthieu Baerts wrote:
> > "All you need is to install" undersells it a little :)
> > 
> > It's not packaged for silly OS^w^w AWS Linux.  
> 
> Thank you for having tried! That's a shame "virtiofsd" is not packaged!
> 
> > And the Rust that comes with it doesn't seem to be able to build it :(  
> Did you try by installing Rust (rustc, cargo) via rustup [1]? It is even
> possible to get the offline installer if it is easier [2]. With rustup,
> you can easily install newer versions of the Rust toolchain.
> 
> [1] https://www.rust-lang.org/learn/get-started
> [2] https://forge.rust-lang.org/infra/other-installation-methods.html

I know, I know, but there's only so many minutes ;)

> Do you need a hand to update the wiki page?

Yes please! I believe it's a repo somewhere, but not sure if it's easy
to send PRs against it. If PRs are not trivial feel free to edit the
wiki directly.

Re: [TEST] VirtioFS instead of 9p

[Andrew Lunn]

> Somewhat related, our current build_rust test doesn't work because
> I used rustup, and it works by adding stuff (paths mostly?) to bashrc.
> Which does not get evaluated when we launch the script from a systemd
> unit :( I couldn't find a "please run this as an interactive shell"
> switch in bash, should we source ~/.bashrc in build_rust.sh for now?

The man page says:

       When bash is started non-interactively, to run a shell script, for  ex‐
       ample,  it  looks for the variable BASH_ENV in the environment, expands
       its value if it appears there, and uses the expanded value as the  name
       of  a  file to read and execute.  Bash behaves as if the following com‐
       mand were executed:
              if [ -n "$BASH_ENV" ]; then . "$BASH_ENV"; fi
       but the value of the PATH variable is not used to search for the  file‐
       name.

So something like:

[Service]
Environment="BASH_ENV=~/.bashrc"

assuming the service is running as the same user as rustup was run as,
and it is invoking bash, not sh, to run the script.

	Andrew

Re: [TEST] VirtioFS instead of 9p

[Edward Cree]

On 26/03/2024 14:20, Jakub Kicinski wrote:
> I couldn't find a "please run this as an interactive shell"
> switch in bash
'bash -i' appears to be precisely this, and causes bashrc to be run.

-ed

Re: [TEST] VirtioFS instead of 9p

[Matthieu Baerts]

On 26/03/2024 15:23, Jakub Kicinski wrote:
> On Tue, 26 Mar 2024 11:27:17 +0100 Matthieu Baerts wrote:
>>> "All you need is to install" undersells it a little :)
>>>
>>> It's not packaged for silly OS^w^w AWS Linux.  
>>
>> Thank you for having tried! That's a shame "virtiofsd" is not packaged!
>>
>>> And the Rust that comes with it doesn't seem to be able to build it :(  
>> Did you try by installing Rust (rustc, cargo) via rustup [1]? It is even
>> possible to get the offline installer if it is easier [2]. With rustup,
>> you can easily install newer versions of the Rust toolchain.
>>
>> [1] https://www.rust-lang.org/learn/get-started
>> [2] https://forge.rust-lang.org/infra/other-installation-methods.html
> 
> I know, I know, but there's only so many minutes ;)

I like your idea from the bi-weekly meeting of using Docker (or similar)
for that. It looks like virtiofsd is in Debian testing (not in stable
yet) and in Ubuntu 23.10 (what we are using for the MPTCP CI, from a
Docker image).

- https://github.com/multipath-tcp/mptcp-upstream-virtme-docker
-
https://github.com/multipath-tcp/mptcp_net-next/blob/export/.github/workflows/tests.yml#L78

>> Do you need a hand to update the wiki page?
> 
> Yes please! I believe it's a repo somewhere, but not sure if it's easy
> to send PRs against it. If PRs are not trivial feel free to edit the
> wiki directly.

There is a repo [1], but we cannot send PRs [2] :-/

So I just did a few modifications [3] (thanks for letting me doing
that), feel free to review :)

[1] https://github.com/linux-netdev/nipa.wiki.git
[2] https://github.com/orgs/community/discussions/50163
[3]
https://github.com/linux-netdev/nipa/wiki/How-to-run-netdev-selftests-CI-style/_history

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.