[PATCH 0/2] selftests/nolibc: small simplification of test development phase

[PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Willy Tarreau]

Hello Paul,

while developing and testing the recent changes for errno/environ/auxv, I
found that I wasn't relying on the kernel that much and that I was mostly
using qemu in userland only with my local kernel.

I figured that it was more convenient for this purpose than rebuilding an
initramfs and kernel for a quick test, and decided to make this approach
easier to use for everyone by adding a "run-user" target to the Makefile
to do exactly this. E.g:

Native build:
  $ time make -C tools/testing/selftests/nolibc run-user
  ...
  make: Entering directory '/g/public/linux/master/tools/testing/selftests/nolibc'
    MKDIR   sysroot/x86/include
  make[1]: Entering directory '/g/public/linux/master/tools/include/nolibc'
  make[2]: Entering directory '/g/public/linux/master'
  make[2]: Leaving directory '/g/public/linux/master'
  make[2]: Entering directory '/g/public/linux/master'
    INSTALL /g/public/linux/master/tools/testing/selftests/nolibc/sysroot/sysroot/include
  make[2]: Leaving directory '/g/public/linux/master'
  make[1]: Leaving directory '/g/public/linux/master/tools/include/nolibc'
    CC      nolibc-test
  18 chroot_root = -1 EPERM               [FAIL]
  43 link_dir = -1 EACCES  != (-1 EPERM)  [FAIL]
  See all results in /g/public/linux/master/tools/testing/selftests/nolibc/run.out
  make: Leaving directory '/g/public/linux/master/tools/testing/selftests/nolibc'

  real    0m0.966s
  user    0m0.731s
  sys     0m0.164s

Cross build:
  $ time make -C tools/testing/selftests/nolibc run-user ARCH=s390 CROSS_COMPILE=/f/tc/nolibc/gcc-11.3.0-nolibc/s390-linux/bin/s390-linux-
  make: Entering directory '/g/public/linux/master/tools/testing/selftests/nolibc'
    MKDIR   sysroot/s390/include
  make[1]: Entering directory '/g/public/linux/master/tools/include/nolibc'
  make[2]: Entering directory '/g/public/linux/master'
  make[2]: Leaving directory '/g/public/linux/master'
  make[2]: Entering directory '/g/public/linux/master'
    INSTALL /g/public/linux/master/tools/testing/selftests/nolibc/sysroot/sysroot/include
  make[2]: Leaving directory '/g/public/linux/master'
  make[1]: Leaving directory '/g/public/linux/master/tools/include/nolibc'
    CC      nolibc-test
  18 chroot_root = -1 EPERM               [FAIL]
  43 link_dir = -1 EACCES  != (-1 EPERM)  [FAIL]
  See all results in /g/public/linux/master/tools/testing/selftests/nolibc/run.out
  make: Leaving directory '/g/public/linux/master/tools/testing/selftests/nolibc'

  real    0m1.014s
  user    0m0.732s
  sys     0m0.183s

In addition, the "x86_64" value for ARCH= is now supported as I got caught
too many times with it not working in this subdir while it's used for the
rest of the kernel ("x86" is used instead as coming from subarch.include).
Generally you don't type it as x86_64 probably is the native build for most
users, but when you start to test toolchains it's a different thing.

There's no matter of urgency for these patches, they're just a bit of
user-friendly stuff. As such, if you're fine with stacking them on top of
what you already have for 6.3, that will be great, otherwise they can
easily wait.

Thank you!
Willy

[CCing Ammar who could benefit from this]

---
Willy Tarreau (2):
  selftests/nolibc: support "x86_64" for arch name
  selftests/nolibc: add a "run-user" target to test the program in user
    land

 tools/testing/selftests/nolibc/Makefile | 10 ++++++++++
 1 file changed, 10 insertions(+)

-- 
2.17.5

[PATCH 1/2] selftests/nolibc: support "x86_64" for arch name

[Willy Tarreau]

Building the kernel with ARCH=x86_64 is perfectly supported, but the
nolibc-test only supports "x86", which causes errors when reusing
existing build environment. Let's permit it to be used as well.

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/testing/selftests/nolibc/Makefile | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
index 2bf613ee363d..423598045ff1 100644
--- a/tools/testing/selftests/nolibc/Makefile
+++ b/tools/testing/selftests/nolibc/Makefile
@@ -14,6 +14,7 @@ endif
 
 # kernel image names by architecture
 IMAGE_i386    = arch/x86/boot/bzImage
+IMAGE_x86_64  = arch/x86/boot/bzImage
 IMAGE_x86     = arch/x86/boot/bzImage
 IMAGE_arm64   = arch/arm64/boot/Image
 IMAGE_arm     = arch/arm/boot/zImage
@@ -25,6 +26,7 @@ IMAGE_NAME    = $(notdir $(IMAGE))
 
 # default kernel configurations that appear to be usable
 DEFCONFIG_i386    = defconfig
+DEFCONFIG_x86_64  = defconfig
 DEFCONFIG_x86     = defconfig
 DEFCONFIG_arm64   = defconfig
 DEFCONFIG_arm     = multi_v7_defconfig
@@ -38,6 +40,7 @@ TEST =
 
 # QEMU_ARCH: arch names used by qemu
 QEMU_ARCH_i386    = i386
+QEMU_ARCH_x86_64  = x86_64
 QEMU_ARCH_x86     = x86_64
 QEMU_ARCH_arm64   = aarch64
 QEMU_ARCH_arm     = arm
@@ -48,6 +51,7 @@ QEMU_ARCH         = $(QEMU_ARCH_$(ARCH))
 
 # QEMU_ARGS : some arch-specific args to pass to qemu
 QEMU_ARGS_i386    = -M pc -append "console=ttyS0,9600 i8042.noaux panic=-1 $(TEST:%=NOLIBC_TEST=%)"
+QEMU_ARGS_x86_64  = -M pc -append "console=ttyS0,9600 i8042.noaux panic=-1 $(TEST:%=NOLIBC_TEST=%)"
 QEMU_ARGS_x86     = -M pc -append "console=ttyS0,9600 i8042.noaux panic=-1 $(TEST:%=NOLIBC_TEST=%)"
 QEMU_ARGS_arm64   = -M virt -cpu cortex-a53 -append "panic=-1 $(TEST:%=NOLIBC_TEST=%)"
 QEMU_ARGS_arm     = -M virt -append "panic=-1 $(TEST:%=NOLIBC_TEST=%)"
-- 
2.17.5

[PATCH 2/2] selftests/nolibc: add a "run-user" target to test the program in user land

[Willy Tarreau]

I found that when developing tests I'm not that often building kernels,
and instead I'm often using the much quicker QEMU Linux emulator instead
of the system emulator. It is sufficient to test startup code, stdlib
code and syscall calling convention. As such it is expected that other
users will value this, so let's add a "run-user" target that immediately
executes the prebuilt executable with the user's privilege (thus some
tests may fail due to insufficient permissions or missing features in
the local kernel).

Now running a userland test is as simple as issuing:

  make ARCH=xxx CROSS_COMPILE=xxx run-user

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/testing/selftests/nolibc/Makefile | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
index 423598045ff1..8fe61d3e3cce 100644
--- a/tools/testing/selftests/nolibc/Makefile
+++ b/tools/testing/selftests/nolibc/Makefile
@@ -80,6 +80,7 @@ help:
 	@echo "  help         this help"
 	@echo "  sysroot      create the nolibc sysroot here (uses \$$ARCH)"
 	@echo "  nolibc-test  build the executable (uses \$$CC and \$$CROSS_COMPILE)"
+	@echo "  run-user     runs the executable under QEMU (uses \$$ARCH, \$$TEST)"
 	@echo "  initramfs    prepare the initramfs with nolibc-test"
 	@echo "  defconfig    create a fresh new default config (uses \$$ARCH)"
 	@echo "  kernel       (re)build the kernel with the initramfs (uses \$$ARCH)"
@@ -113,6 +114,11 @@ nolibc-test: nolibc-test.c sysroot/$(ARCH)/include
 	$(QUIET_CC)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ \
 	  -nostdlib -static -Isysroot/$(ARCH)/include $< -lgcc
 
+# qemu user-land test
+run-user: nolibc-test
+	$(Q)qemu-$(QEMU_ARCH) ./nolibc-test > "$(CURDIR)/run.out" || :
+	$(Q)grep -w FAIL "$(CURDIR)/run.out" && echo "See all results in $(CURDIR)/run.out" || echo "$$(grep -c ^[0-9].*OK $(CURDIR)/run.out) test(s) passed."
+
 initramfs: nolibc-test
 	$(QUIET_MKDIR)mkdir -p initramfs
 	$(call QUIET_INSTALL, initramfs/init)
-- 
2.17.5

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Paul E. McKenney]

On Sat, Jan 21, 2023 at 09:53:18AM +0100, Willy Tarreau wrote:
> Hello Paul,
> 
> while developing and testing the recent changes for errno/environ/auxv, I
> found that I wasn't relying on the kernel that much and that I was mostly
> using qemu in userland only with my local kernel.
> 
> I figured that it was more convenient for this purpose than rebuilding an
> initramfs and kernel for a quick test, and decided to make this approach
> easier to use for everyone by adding a "run-user" target to the Makefile
> to do exactly this. E.g:
> 
> Native build:
>   $ time make -C tools/testing/selftests/nolibc run-user
>   ...
>   make: Entering directory '/g/public/linux/master/tools/testing/selftests/nolibc'
>     MKDIR   sysroot/x86/include
>   make[1]: Entering directory '/g/public/linux/master/tools/include/nolibc'
>   make[2]: Entering directory '/g/public/linux/master'
>   make[2]: Leaving directory '/g/public/linux/master'
>   make[2]: Entering directory '/g/public/linux/master'
>     INSTALL /g/public/linux/master/tools/testing/selftests/nolibc/sysroot/sysroot/include
>   make[2]: Leaving directory '/g/public/linux/master'
>   make[1]: Leaving directory '/g/public/linux/master/tools/include/nolibc'
>     CC      nolibc-test
>   18 chroot_root = -1 EPERM               [FAIL]
>   43 link_dir = -1 EACCES  != (-1 EPERM)  [FAIL]
>   See all results in /g/public/linux/master/tools/testing/selftests/nolibc/run.out
>   make: Leaving directory '/g/public/linux/master/tools/testing/selftests/nolibc'
> 
>   real    0m0.966s
>   user    0m0.731s
>   sys     0m0.164s
> 
> Cross build:
>   $ time make -C tools/testing/selftests/nolibc run-user ARCH=s390 CROSS_COMPILE=/f/tc/nolibc/gcc-11.3.0-nolibc/s390-linux/bin/s390-linux-
>   make: Entering directory '/g/public/linux/master/tools/testing/selftests/nolibc'
>     MKDIR   sysroot/s390/include
>   make[1]: Entering directory '/g/public/linux/master/tools/include/nolibc'
>   make[2]: Entering directory '/g/public/linux/master'
>   make[2]: Leaving directory '/g/public/linux/master'
>   make[2]: Entering directory '/g/public/linux/master'
>     INSTALL /g/public/linux/master/tools/testing/selftests/nolibc/sysroot/sysroot/include
>   make[2]: Leaving directory '/g/public/linux/master'
>   make[1]: Leaving directory '/g/public/linux/master/tools/include/nolibc'
>     CC      nolibc-test
>   18 chroot_root = -1 EPERM               [FAIL]
>   43 link_dir = -1 EACCES  != (-1 EPERM)  [FAIL]
>   See all results in /g/public/linux/master/tools/testing/selftests/nolibc/run.out
>   make: Leaving directory '/g/public/linux/master/tools/testing/selftests/nolibc'
> 
>   real    0m1.014s
>   user    0m0.732s
>   sys     0m0.183s
> 
> In addition, the "x86_64" value for ARCH= is now supported as I got caught
> too many times with it not working in this subdir while it's used for the
> rest of the kernel ("x86" is used instead as coming from subarch.include).
> Generally you don't type it as x86_64 probably is the native build for most
> users, but when you start to test toolchains it's a different thing.
> 
> There's no matter of urgency for these patches, they're just a bit of
> user-friendly stuff. As such, if you're fine with stacking them on top of
> what you already have for 6.3, that will be great, otherwise they can
> easily wait.
> 
> Thank you!
> Willy

Nice, thank you!

I have these placed on top of the -rcu "dev" branch initially for further
review and testing.  If things go well over the next week or so, I will
set it up for the upcoming merge window.

One dependency is of course qemu-x86_64, so in the meantime I will figure
out where I get that from.  ;-)

							Thanx, Paul

> [CCing Ammar who could benefit from this]
> 
> ---
> Willy Tarreau (2):
>   selftests/nolibc: support "x86_64" for arch name
>   selftests/nolibc: add a "run-user" target to test the program in user
>     land
> 
>  tools/testing/selftests/nolibc/Makefile | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> -- 
> 2.17.5
> 

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Willy Tarreau]

On Sat, Jan 21, 2023 at 12:00:38PM -0800, Paul E. McKenney wrote:
> > There's no matter of urgency for these patches, they're just a bit of
> > user-friendly stuff. As such, if you're fine with stacking them on top of
> > what you already have for 6.3, that will be great, otherwise they can
> > easily wait.
> > 
> > Thank you!
> > Willy
> 
> Nice, thank you!
> 
> I have these placed on top of the -rcu "dev" branch initially for further
> review and testing.  If things go well over the next week or so, I will
> set it up for the upcoming merge window.

Thanks!

> One dependency is of course qemu-x86_64, so in the meantime I will figure
> out where I get that from.  ;-)

I build it from time to time from the sources, it's not that long and
normally doesn't reserve me any surprises. But if you have it for other
platforms it's likely that you have it for most platforms as well,
including this one.

Best regards,
Willy

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Paul E. McKenney]

On Sat, Jan 21, 2023 at 10:34:55PM +0100, Willy Tarreau wrote:
> On Sat, Jan 21, 2023 at 12:00:38PM -0800, Paul E. McKenney wrote:
> > > There's no matter of urgency for these patches, they're just a bit of
> > > user-friendly stuff. As such, if you're fine with stacking them on top of
> > > what you already have for 6.3, that will be great, otherwise they can
> > > easily wait.
> > > 
> > > Thank you!
> > > Willy
> > 
> > Nice, thank you!
> > 
> > I have these placed on top of the -rcu "dev" branch initially for further
> > review and testing.  If things go well over the next week or so, I will
> > set it up for the upcoming merge window.
> 
> Thanks!
> 
> > One dependency is of course qemu-x86_64, so in the meantime I will figure
> > out where I get that from.  ;-)
> 
> I build it from time to time from the sources, it's not that long and
> normally doesn't reserve me any surprises. But if you have it for other
> platforms it's likely that you have it for most platforms as well,
> including this one.

And building from sources proved to be reasonably easy, so the test
now passes for me.  My initial thought of putting qemu-x86_64 into
my ~/bin directory fails the sudo test, but putting it into /usr/bin
works fine.

Thank you for the hints!

Should I add a sentence to the commit log noting the potential need to
build qemu from the git repo and to install qemu-x86_64, give or take
what architecture one is running?

							Thanx, Paul

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Willy Tarreau]

Hello Paul,

On Mon, Jan 23, 2023 at 08:47:21AM -0800, Paul E. McKenney wrote:
> And building from sources proved to be reasonably easy, so the test
> now passes for me.  My initial thought of putting qemu-x86_64 into
> my ~/bin directory fails the sudo test, but putting it into /usr/bin
> works fine.

Great!

> Thank you for the hints!
> 
> Should I add a sentence to the commit log noting the potential need to
> build qemu from the git repo and to install qemu-x86_64, give or take
> what architecture one is running?

Well, I've always had all the variants for all supported archs and
didn't know that sometimes only part of them could be installed.
I've used and tested qemu-{i386,x86_64,arm,aarch64,mips,s390x,riscv64}
with this with success, and all of them are built by default for me.
Thus I'm not seeing a good reason for making a special case of x86_64.
Or maybe I'm missing the point ?

Thanks,
Willy

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Paul E. McKenney]

On Mon, Jan 23, 2023 at 06:17:12PM +0100, Willy Tarreau wrote:
> Hello Paul,
> 
> On Mon, Jan 23, 2023 at 08:47:21AM -0800, Paul E. McKenney wrote:
> > And building from sources proved to be reasonably easy, so the test
> > now passes for me.  My initial thought of putting qemu-x86_64 into
> > my ~/bin directory fails the sudo test, but putting it into /usr/bin
> > works fine.
> 
> Great!
> 
> > Thank you for the hints!
> > 
> > Should I add a sentence to the commit log noting the potential need to
> > build qemu from the git repo and to install qemu-x86_64, give or take
> > what architecture one is running?
> 
> Well, I've always had all the variants for all supported archs and
> didn't know that sometimes only part of them could be installed.
> I've used and tested qemu-{i386,x86_64,arm,aarch64,mips,s390x,riscv64}
> with this with success, and all of them are built by default for me.
> Thus I'm not seeing a good reason for making a special case of x86_64.
> Or maybe I'm missing the point ?

Fair point, and yes, I am showing the x86-centricity of my test
environment.  ;-)  This might also apply to non-x86 distro setups,
but I have no idea either way.

I would be OK leaving it as is and responding to problems if and when
they actually occur.  But you are quite correct, if we do add some sort of
informative "Just build qemu!" diagnostic, it should be arch-independent.

							Thanx, Paul

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Ammar Faizi]

On Sat, Jan 21, 2023 at 10:34:55PM +0100, Willy Tarreau wrote:
> On Sat, Jan 21, 2023 at 12:00:38PM -0800, Paul E. McKenney wrote:
> > One dependency is of course qemu-x86_64, so in the meantime I will figure
> > out where I get that from.  ;-)
> 
> I build it from time to time from the sources, it's not that long and
> normally doesn't reserve me any surprises. But if you have it for other
> platforms it's likely that you have it for most platforms as well,
> including this one.

Willy,

Is there a way to make it work for the default qemu installation?
Or maybe it's a mandatory requirement to build qemu from the source?

I use the qemu that comes from Ubuntu apt. I have "qemu-system-x86_64",
but no "qemu-x86_64". So, something like this...

  $ which qemu-x86_64
  $ echo $?
  1

  $ which qemu-system-x86_64
  /usr/bin/qemu-system-x86_64

  $ time make run-user
    MKDIR   sysroot/x86/include
  make[1]: Entering directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
  make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
  make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
  make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
    INSTALL /home/ammarfaizi2/work/linux.work.cc/tools/testing/selftests/nolibc/sysroot/sysroot/include
  make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
  make[1]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
    CC      nolibc-test
  /bin/sh: 1: qemu-x86_64: not found
  0 test(s) passed.

  real    0m0.506s
  user    0m0.393s
  sys     0m0.086s

It would be great if we can avoid building qemu from the source. But if
not, let's go with that.

Thanks!

-- 
Ammar Faizi

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Paul E. McKenney]

On Mon, Jan 23, 2023 at 11:57:56PM +0700, Ammar Faizi wrote:
> On Sat, Jan 21, 2023 at 10:34:55PM +0100, Willy Tarreau wrote:
> > On Sat, Jan 21, 2023 at 12:00:38PM -0800, Paul E. McKenney wrote:
> > > One dependency is of course qemu-x86_64, so in the meantime I will figure
> > > out where I get that from.  ;-)
> > 
> > I build it from time to time from the sources, it's not that long and
> > normally doesn't reserve me any surprises. But if you have it for other
> > platforms it's likely that you have it for most platforms as well,
> > including this one.
> 
> Willy,
> 
> Is there a way to make it work for the default qemu installation?
> Or maybe it's a mandatory requirement to build qemu from the source?
> 
> I use the qemu that comes from Ubuntu apt. I have "qemu-system-x86_64",
> but no "qemu-x86_64". So, something like this...
> 
>   $ which qemu-x86_64
>   $ echo $?
>   1
> 
>   $ which qemu-system-x86_64
>   /usr/bin/qemu-system-x86_64
> 
>   $ time make run-user
>     MKDIR   sysroot/x86/include
>   make[1]: Entering directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
>   make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
>   make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
>   make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
>     INSTALL /home/ammarfaizi2/work/linux.work.cc/tools/testing/selftests/nolibc/sysroot/sysroot/include
>   make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
>   make[1]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
>     CC      nolibc-test
>   /bin/sh: 1: qemu-x86_64: not found
>   0 test(s) passed.
> 
>   real    0m0.506s
>   user    0m0.393s
>   sys     0m0.086s
> 
> It would be great if we can avoid building qemu from the source. But if
> not, let's go with that.

I had no problems building it from source.

One possible enhancement would be to make the error message tell the
user to build from source, but I will let you guys debate the merits
and drawbacks of that approach.

							Thanx, Paul

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Willy Tarreau]

Hi Ammar,

On Mon, Jan 23, 2023 at 11:57:56PM +0700, Ammar Faizi wrote:
> Is there a way to make it work for the default qemu installation?
> Or maybe it's a mandatory requirement to build qemu from the source?
> 
> I use the qemu that comes from Ubuntu apt. I have "qemu-system-x86_64",
> but no "qemu-x86_64". So, something like this...
> 
>   $ which qemu-x86_64
>   $ echo $?
>   1
> 
>   $ which qemu-system-x86_64
>   /usr/bin/qemu-system-x86_64

Ah now I think I understand Paul's question. I didn't know that the
userland version was not always provided. I've always had both side
by side.

> It would be great if we can avoid building qemu from the source. But if
> not, let's go with that.

As Paul indicated, it's really trouble-free and I think I've only done
that since the very first day I started to use QEMU, reason why I probably
never noticed that not everything was packaged.

Then at least to respond to Paul, it could make sense to add a note that
on some distros the userland version might not always be provided and
might require a pretty simple rebuild of QEMU.

Thanks!
Willy

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Ammar Faizi]

On Mon, Jan 23, 2023 at 06:20:16PM +0100, Willy Tarreau wrote:
> Ah now I think I understand Paul's question. I didn't know that the
> userland version was not always provided. I've always had both side
> by side.
> 
> > It would be great if we can avoid building qemu from the source. But if
> > not, let's go with that.
> 
> As Paul indicated, it's really trouble-free and I think I've only done
> that since the very first day I started to use QEMU, reason why I probably
> never noticed that not everything was packaged.
> 
> Then at least to respond to Paul, it could make sense to add a note that
> on some distros the userland version might not always be provided and
> might require a pretty simple rebuild of QEMU.

Ah well...

I figured it out. It turned qemu-user is a different package.
So I have "qemu-system" installed, but not "qemu-user".

Without building from source, just do this on Ubuntu:

  $ sudo apt-get install qemu-user -y
  ...
  Preparing to unpack .../qemu-user_1%3a6.2+dfsg-2ubuntu6.6_amd64.deb ...
  Unpacking qemu-user (1:6.2+dfsg-2ubuntu6.6) ...
  Selecting previously unselected package qemu-user-binfmt.
  Preparing to unpack .../qemu-user-binfmt_1%3a6.2+dfsg-2ubuntu6.6_amd64.deb ...
  Unpacking qemu-user-binfmt (1:6.2+dfsg-2ubuntu6.6) ...
  Setting up qemu-user (1:6.2+dfsg-2ubuntu6.6) ...
  Setting up qemu-user-binfmt (1:6.2+dfsg-2ubuntu6.6) ...
  Processing triggers for man-db (2.10.2-1) ...

  $ which qemu-x86_64
  /usr/bin/qemu-x86_64

  $ sudo make run-user
  MKDIR   sysroot/x86/include
  make[1]: Entering directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
  make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
  make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
  make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
    INSTALL /home/ammarfaizi2/work/linux.work.cc/tools/testing/selftests/nolibc/sysroot/sysroot/include
  make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
  make[1]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
    CC      nolibc-test
  83 test(s) passed.

Sorry for that. I didn't know that they come from different packages.
It works fine for me now.

-- 
Ammar Faizi

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Willy Tarreau]

On Tue, Jan 24, 2023 at 12:31:01AM +0700, Ammar Faizi wrote:
> I figured it out. It turned qemu-user is a different package.
> So I have "qemu-system" installed, but not "qemu-user".
> 
> Without building from source, just do this on Ubuntu:
> 
>   $ sudo apt-get install qemu-user -y
>   ...
>   Preparing to unpack .../qemu-user_1%3a6.2+dfsg-2ubuntu6.6_amd64.deb ...
>   Unpacking qemu-user (1:6.2+dfsg-2ubuntu6.6) ...
>   Selecting previously unselected package qemu-user-binfmt.
>   Preparing to unpack .../qemu-user-binfmt_1%3a6.2+dfsg-2ubuntu6.6_amd64.deb ...
>   Unpacking qemu-user-binfmt (1:6.2+dfsg-2ubuntu6.6) ...
>   Setting up qemu-user (1:6.2+dfsg-2ubuntu6.6) ...
>   Setting up qemu-user-binfmt (1:6.2+dfsg-2ubuntu6.6) ...
>   Processing triggers for man-db (2.10.2-1) ...
> 
>   $ which qemu-x86_64
>   /usr/bin/qemu-x86_64

Ah cool! I looked for something like this on a remote ubuntu system but
failed to figure it out by myself. Do you also have the other archs with
it ?

>   $ sudo make run-user
>   MKDIR   sysroot/x86/include
>   make[1]: Entering directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
>   make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
>   make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
>   make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
>     INSTALL /home/ammarfaizi2/work/linux.work.cc/tools/testing/selftests/nolibc/sysroot/sysroot/include
>   make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
>   make[1]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
>     CC      nolibc-test
>   83 test(s) passed.

Great. And without sudo you should see two of them fail. I intend to
implement geteuid() and condition their result to it, that will be
sufficient for most cases. I'd rather avoid seeing users run test
programs under sudo actually, since they're run on the local system :-/

> Sorry for that. I didn't know that they come from different packages.
> It works fine for me now.

No, don't be sorry, quite the opposite, you taught us something useful!

Thank you!
Willy

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Ammar Faizi]

On Mon, Jan 23, 2023 at 06:37:07PM +0100, Willy Tarreau wrote:
> Ah cool! I looked for something like this on a remote ubuntu system but
> failed to figure it out by myself. Do you also have the other archs with
> it ?

Yes, they come with many architectures.

$ ls /usr/bin/qemu-* | cat
/usr/bin/qemu-aarch64
/usr/bin/qemu-aarch64_be
/usr/bin/qemu-alpha
/usr/bin/qemu-arm
/usr/bin/qemu-armeb
/usr/bin/qemu-cris
/usr/bin/qemu-hexagon
/usr/bin/qemu-hppa
/usr/bin/qemu-i386
/usr/bin/qemu-img
/usr/bin/qemu-io
/usr/bin/qemu-m68k
/usr/bin/qemu-microblaze
/usr/bin/qemu-microblazeel
/usr/bin/qemu-mips
/usr/bin/qemu-mips64
/usr/bin/qemu-mips64el
/usr/bin/qemu-mipsel
/usr/bin/qemu-mipsn32
/usr/bin/qemu-mipsn32el
/usr/bin/qemu-nbd
/usr/bin/qemu-nios2
/usr/bin/qemu-or1k
/usr/bin/qemu-ppc
/usr/bin/qemu-ppc64
/usr/bin/qemu-ppc64le
/usr/bin/qemu-pr-helper
/usr/bin/qemu-riscv32
/usr/bin/qemu-riscv64
/usr/bin/qemu-s390x
/usr/bin/qemu-sh4
/usr/bin/qemu-sh4eb
/usr/bin/qemu-sparc
/usr/bin/qemu-sparc32plus
/usr/bin/qemu-sparc64
/usr/bin/qemu-storage-daemon
/usr/bin/qemu-system-i386
/usr/bin/qemu-system-x86_64
/usr/bin/qemu-system-x86_64-microvm
/usr/bin/qemu-system-x86_64-spice
/usr/bin/qemu-x86_64
/usr/bin/qemu-xtensa
/usr/bin/qemu-xtensaeb

> Great. And without sudo you should see two of them fail. I intend to
> implement geteuid() and condition their result to it, that will be
> sufficient for most cases. I'd rather avoid seeing users run test
> programs under sudo actually, since they're run on the local system :-/

I see.

-- 
Ammar Faizi

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Paul E. McKenney]

[-- Attachment #1: Type: text/plain, Size: 3527 bytes --]

On Tue, Jan 24, 2023 at 12:31:01AM +0700, Ammar Faizi wrote:
> On Mon, Jan 23, 2023 at 06:20:16PM +0100, Willy Tarreau wrote:
> > Ah now I think I understand Paul's question. I didn't know that the
> > userland version was not always provided. I've always had both side
> > by side.
> > 
> > > It would be great if we can avoid building qemu from the source. But if
> > > not, let's go with that.
> > 
> > As Paul indicated, it's really trouble-free and I think I've only done
> > that since the very first day I started to use QEMU, reason why I probably
> > never noticed that not everything was packaged.
> > 
> > Then at least to respond to Paul, it could make sense to add a note that
> > on some distros the userland version might not always be provided and
> > might require a pretty simple rebuild of QEMU.
> 
> Ah well...
> 
> I figured it out. It turned qemu-user is a different package.
> So I have "qemu-system" installed, but not "qemu-user".
> 
> Without building from source, just do this on Ubuntu:
> 
>   $ sudo apt-get install qemu-user -y
>   ...
>   Preparing to unpack .../qemu-user_1%3a6.2+dfsg-2ubuntu6.6_amd64.deb ...
>   Unpacking qemu-user (1:6.2+dfsg-2ubuntu6.6) ...
>   Selecting previously unselected package qemu-user-binfmt.
>   Preparing to unpack .../qemu-user-binfmt_1%3a6.2+dfsg-2ubuntu6.6_amd64.deb ...
>   Unpacking qemu-user-binfmt (1:6.2+dfsg-2ubuntu6.6) ...
>   Setting up qemu-user (1:6.2+dfsg-2ubuntu6.6) ...
>   Setting up qemu-user-binfmt (1:6.2+dfsg-2ubuntu6.6) ...
>   Processing triggers for man-db (2.10.2-1) ...
> 
>   $ which qemu-x86_64
>   /usr/bin/qemu-x86_64
> 
>   $ sudo make run-user
>   MKDIR   sysroot/x86/include
>   make[1]: Entering directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
>   make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
>   make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
>   make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
>     INSTALL /home/ammarfaizi2/work/linux.work.cc/tools/testing/selftests/nolibc/sysroot/sysroot/include
>   make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
>   make[1]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
>     CC      nolibc-test
>   83 test(s) passed.
> 
> Sorry for that. I didn't know that they come from different packages.
> It works fine for me now.

I looked for that, but didn't find it, so thank you!

(Yes, I should have used dpkg, but I was lazy.)

Except that when I install Ubuntu 20.04's version, I get this:

------------------------------------------------------------------------

$ sudo make run-user
  MKDIR   sysroot/x86/include
make[1]: Entering directory '/home/git/linux-rcu/tools/include/nolibc'
make[2]: Entering directory '/home/git/linux-rcu'
make[2]: Leaving directory '/home/git/linux-rcu'
make[2]: Entering directory '/home/git/linux-rcu'
  INSTALL /home/git/linux-rcu/tools/testing/selftests/nolibc/sysroot/sysroot/include
make[2]: Leaving directory '/home/git/linux-rcu'
make[1]: Leaving directory '/home/git/linux-rcu/tools/include/nolibc'
  CC      nolibc-test
32 gettimeofday_null = -1 EFAULT        [FAIL]
See all results in /home/git/linux-rcu/tools/testing/selftests/nolibc/run.out

------------------------------------------------------------------------

I have attached run.out.

In contrast, with my hand-built qemu-x86_64, all tests passed.

This might be just a version-related bug, but figured I should let you
guys know.

							Thanx, Paul

[-- Attachment #2: run.out --]
[-- Type: text/plain, Size: 3968 bytes --]

Running test 'syscall'
0 getpid = 2270744                       [OK]
1 getppid = 2270743                      [OK]
3 gettid = 2270744                       [OK]
5 getpgid_self = 2270584                 [OK]
6 getpgid_bad = -1 ESRCH                 [OK]
7 kill_0 = 0                             [OK]
8 kill_CONT = 0                          [OK]
9 kill_BADPID = -1 ESRCH                 [OK]
10 sbrk = 0                              [OK]
11 brk = 0                               [OK]
12 chdir_root = 0                        [OK]
13 chdir_dot = 0                         [OK]
14 chdir_blah = -1 ENOENT                [OK]
15 chmod_net = 0                         [OK]
16 chmod_self = -1 EPERM                 [OK]
17 chown_self = -1 EPERM                 [OK]
18 chroot_root = 0                       [OK]
19 chroot_blah = -1 ENOENT               [OK]
20 chroot_exe = -1 ENOTDIR               [OK]
21 close_m1 = -1 EBADF                   [OK]
22 close_dup = 0                         [OK]
23 dup_0 = 3                             [OK]
24 dup_m1 = -1 EBADF                     [OK]
25 dup2_0 = 100                          [OK]
26 dup2_m1 = -1 EBADF                    [OK]
27 dup3_0 = 100                          [OK]
28 dup3_m1 = -1 EBADF                    [OK]
29 execve_root = -1 EACCES               [OK]
30 getdents64_root = 672                 [OK]
31 getdents64_null = -1 ENOTDIR          [OK]
32 gettimeofday_null = -1 EFAULT        [FAIL]
34 gettimeofday_bad1 = -1 EFAULT         [OK]
35 gettimeofday_bad2 = -1 EFAULT         [OK]
36 gettimeofday_bad2 = -1 EFAULT         [OK]
38 getpagesize = 0                       [OK]
39 ioctl_tiocinq = 0                     [OK]
40 ioctl_tiocinq = 0                     [OK]
41 link_root1 = -1 EEXIST                [OK]
42 link_blah = -1 ENOENT                 [OK]
43 link_dir = -1 EPERM                   [OK]
44 link_cross = -1 EXDEV                 [OK]
45 lseek_m1 = -1 EBADF                   [OK]
46 lseek_0 = -1 ESPIPE                   [OK]
47 mkdir_root = -1 EEXIST                [OK]
48 open_tty = 3                          [OK]
49 open_blah = -1 ENOENT                 [OK]
50 poll_null = 0                         [OK]
51 poll_stdout = 1                       [OK]
52 poll_fault = -1 EFAULT                [OK]
53 read_badf = -1 EBADF                  [OK]
54 sched_yield = 0                       [OK]
55 select_null = 0                       [OK]
56 select_stdout = 1                     [OK]
57 select_fault = -1 EFAULT              [OK]
58 stat_blah = -1 ENOENT                 [OK]
59 stat_fault = -1 EFAULT                [OK]
60 symlink_root = -1 EEXIST              [OK]
61 unlink_root = -1 EISDIR               [OK]
62 unlink_blah = -1 ENOENT               [OK]
63 wait_child = -1 ECHILD                [OK]
64 waitpid_min = -1 ESRCH                [OK]
65 waitpid_child = -1 ECHILD             [OK]
66 write_badf = -1 EBADF                 [OK]
67 write_zero = 0                        [OK]
Errors during this test: 1

Running test 'stdlib'
0 getenv_TERM = <xterm>                  [OK]
1 getenv_blah = <(null)>                 [OK]
2 setcmp_blah_blah = 0                   [OK]
3 setcmp_blah_blah2 = -50                [OK]
4 setncmp_blah_blah = 0                  [OK]
5 setncmp_blah_blah4 = 0                 [OK]
6 setncmp_blah_blah5 = -53               [OK]
7 setncmp_blah_blah6 = -54               [OK]
8 strchr_foobar_o = <oobar>              [OK]
9 strchr_foobar_z = <(null)>             [OK]
10 strrchr_foobar_o = <obar>             [OK]
11 strrchr_foobar_z = <(null)>           [OK]
12 memcmp_20_20 = 0                      [OK]
13 memcmp_20_60 = -64                    [OK]
14 memcmp_60_20 = 64                     [OK]
15 memcmp_20_e0 = -192                   [OK]
16 memcmp_e0_20 = 192                    [OK]
17 memcmp_80_e0 = -96                    [OK]
18 memcmp_e0_80 = 96                     [OK]
Errors during this test: 0

Total number of errors: 1
Exiting with status 1

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Ammar Faizi]

On Mon, Jan 23, 2023 at 09:40:03AM -0800, Paul E. McKenney wrote: 
> I looked for that, but didn't find it, so thank you!
> 
> (Yes, I should have used dpkg, but I was lazy.)
> 
> Except that when I install Ubuntu 20.04's version, I get this:
> 
> ------------------------------------------------------------------------
> 
> $ sudo make run-user
>   MKDIR   sysroot/x86/include
> make[1]: Entering directory '/home/git/linux-rcu/tools/include/nolibc'
> make[2]: Entering directory '/home/git/linux-rcu'
> make[2]: Leaving directory '/home/git/linux-rcu'
> make[2]: Entering directory '/home/git/linux-rcu'
>   INSTALL /home/git/linux-rcu/tools/testing/selftests/nolibc/sysroot/sysroot/include
> make[2]: Leaving directory '/home/git/linux-rcu'
> make[1]: Leaving directory '/home/git/linux-rcu/tools/include/nolibc'
>   CC      nolibc-test
> 32 gettimeofday_null = -1 EFAULT        [FAIL]
> See all results in /home/git/linux-rcu/tools/testing/selftests/nolibc/run.out
> 
> ------------------------------------------------------------------------
> 
> I have attached run.out.
> 
> In contrast, with my hand-built qemu-x86_64, all tests passed.
> 
> This might be just a version-related bug, but figured I should let you
> guys know.

This is an interesting bug.

I'm a bit reluctant to say that this is a qemu bug. But I can't
reproduce it on my machine. I use qemu that comes from Ubuntu 22.04.

FWIW, my qemu version is:

   $ qemu-x86_64 -version
   qemu-x86_64 version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.6)
   Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers

I'll take a look at that gettimeofday() code and see if we have a bug in
it. In meantime, could you send your qemu version info?

-- 
Ammar Faizi

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Paul E. McKenney]

On Tue, Jan 24, 2023 at 02:01:50AM +0700, Ammar Faizi wrote:
> On Mon, Jan 23, 2023 at 09:40:03AM -0800, Paul E. McKenney wrote: 
> > I looked for that, but didn't find it, so thank you!
> > 
> > (Yes, I should have used dpkg, but I was lazy.)
> > 
> > Except that when I install Ubuntu 20.04's version, I get this:
> > 
> > ------------------------------------------------------------------------
> > 
> > $ sudo make run-user
> >   MKDIR   sysroot/x86/include
> > make[1]: Entering directory '/home/git/linux-rcu/tools/include/nolibc'
> > make[2]: Entering directory '/home/git/linux-rcu'
> > make[2]: Leaving directory '/home/git/linux-rcu'
> > make[2]: Entering directory '/home/git/linux-rcu'
> >   INSTALL /home/git/linux-rcu/tools/testing/selftests/nolibc/sysroot/sysroot/include
> > make[2]: Leaving directory '/home/git/linux-rcu'
> > make[1]: Leaving directory '/home/git/linux-rcu/tools/include/nolibc'
> >   CC      nolibc-test
> > 32 gettimeofday_null = -1 EFAULT        [FAIL]
> > See all results in /home/git/linux-rcu/tools/testing/selftests/nolibc/run.out
> > 
> > ------------------------------------------------------------------------
> > 
> > I have attached run.out.
> > 
> > In contrast, with my hand-built qemu-x86_64, all tests passed.
> > 
> > This might be just a version-related bug, but figured I should let you
> > guys know.
> 
> This is an interesting bug.
> 
> I'm a bit reluctant to say that this is a qemu bug. But I can't
> reproduce it on my machine. I use qemu that comes from Ubuntu 22.04.
> 
> FWIW, my qemu version is:
> 
>    $ qemu-x86_64 -version
>    qemu-x86_64 version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.6)
>    Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
> 
> I'll take a look at that gettimeofday() code and see if we have a bug in
> it. In meantime, could you send your qemu version info?

Thank you for looking into this, and here you go:

    $ qemu-x86_64 -version
    qemu-x86_64 version 4.2.1 (Debian 1:4.2-3ubuntu6.24)
    Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

The version that I built by hand (which passed all tests) is as follows:

    $ /home/git/qemu/build/qemu-x86_64 -version
    qemu-x86_64 version 7.2.0 (v7.2.0)
    Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers

							Thanx, Paul

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Willy Tarreau]

On Mon, Jan 23, 2023 at 09:40:03AM -0800, Paul E. McKenney wrote:
> Except that when I install Ubuntu 20.04's version, I get this:
> 
> ------------------------------------------------------------------------
> 
> $ sudo make run-user
>   MKDIR   sysroot/x86/include
> make[1]: Entering directory '/home/git/linux-rcu/tools/include/nolibc'
> make[2]: Entering directory '/home/git/linux-rcu'
> make[2]: Leaving directory '/home/git/linux-rcu'
> make[2]: Entering directory '/home/git/linux-rcu'
>   INSTALL /home/git/linux-rcu/tools/testing/selftests/nolibc/sysroot/sysroot/include
> make[2]: Leaving directory '/home/git/linux-rcu'
> make[1]: Leaving directory '/home/git/linux-rcu/tools/include/nolibc'
>   CC      nolibc-test
> 32 gettimeofday_null = -1 EFAULT        [FAIL]
> See all results in /home/git/linux-rcu/tools/testing/selftests/nolibc/run.out
> 
> ------------------------------------------------------------------------
> 
> I have attached run.out.
> 
> In contrast, with my hand-built qemu-x86_64, all tests passed.
> 
> This might be just a version-related bug, but figured I should let you
> guys know.

Interesting. Maybe something differs in the way it passes expectedly
invalid pointers to some syscalls. Keep in mind that it's using your
local kernel also, that could make a difference. I'm not that much keen
on trying to investigate that one to be honest, given that this user
mode is really meant to ease the life of test developers like Ammar
and myself who just want to focus on the correctness of the test they're
adding and not that much on the validity of the test itself in this
context. I suggest we keep this one in mind without putting too much
effort on it for now.

Thanks!
Willy

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Paul E. McKenney]

On Mon, Jan 23, 2023 at 08:12:50PM +0100, Willy Tarreau wrote:
> On Mon, Jan 23, 2023 at 09:40:03AM -0800, Paul E. McKenney wrote:
> > Except that when I install Ubuntu 20.04's version, I get this:
> > 
> > ------------------------------------------------------------------------
> > 
> > $ sudo make run-user
> >   MKDIR   sysroot/x86/include
> > make[1]: Entering directory '/home/git/linux-rcu/tools/include/nolibc'
> > make[2]: Entering directory '/home/git/linux-rcu'
> > make[2]: Leaving directory '/home/git/linux-rcu'
> > make[2]: Entering directory '/home/git/linux-rcu'
> >   INSTALL /home/git/linux-rcu/tools/testing/selftests/nolibc/sysroot/sysroot/include
> > make[2]: Leaving directory '/home/git/linux-rcu'
> > make[1]: Leaving directory '/home/git/linux-rcu/tools/include/nolibc'
> >   CC      nolibc-test
> > 32 gettimeofday_null = -1 EFAULT        [FAIL]
> > See all results in /home/git/linux-rcu/tools/testing/selftests/nolibc/run.out
> > 
> > ------------------------------------------------------------------------
> > 
> > I have attached run.out.
> > 
> > In contrast, with my hand-built qemu-x86_64, all tests passed.
> > 
> > This might be just a version-related bug, but figured I should let you
> > guys know.
> 
> Interesting. Maybe something differs in the way it passes expectedly
> invalid pointers to some syscalls. Keep in mind that it's using your
> local kernel also, that could make a difference. I'm not that much keen
> on trying to investigate that one to be honest, given that this user
> mode is really meant to ease the life of test developers like Ammar
> and myself who just want to focus on the correctness of the test they're
> adding and not that much on the validity of the test itself in this
> context. I suggest we keep this one in mind without putting too much
> effort on it for now.

Indeed, it is easy for me to remove qemu-user and re-install my
hand-built version.  In fact, I just now did this and verified that
everything now passes.  ;-)

							Thanx, Paul

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Ammar Faizi]

On Sat, Jan 21, 2023 at 12:00:38PM -0800, Paul E. McKenney wrote:
> Nice, thank you!
> 
> I have these placed on top of the -rcu "dev" branch initially for further
> review and testing.  If things go well over the next week or so, I will
> set it up for the upcoming merge window.
> 
> One dependency is of course qemu-x86_64, so in the meantime I will figure
> out where I get that from.  ;-)

Thank you all, I'll base my work on top of the -rcu "dev" branch.

Tested-by: Ammar Faizi <ammarfaizi2@gnuweeb.org>

-- 
Ammar Faizi

Re: [PATCH 0/2] selftests/nolibc: small simplification of test development phase

[Paul E. McKenney]

On Tue, Jan 24, 2023 at 12:34:43AM +0700, Ammar Faizi wrote:
> On Sat, Jan 21, 2023 at 12:00:38PM -0800, Paul E. McKenney wrote:
> > Nice, thank you!
> > 
> > I have these placed on top of the -rcu "dev" branch initially for further
> > review and testing.  If things go well over the next week or so, I will
> > set it up for the upcoming merge window.
> > 
> > One dependency is of course qemu-x86_64, so in the meantime I will figure
> > out where I get that from.  ;-)
> 
> Thank you all, I'll base my work on top of the -rcu "dev" branch.
> 
> Tested-by: Ammar Faizi <ammarfaizi2@gnuweeb.org>

I will apply this on my next rebase, thank you!

							Thanx, Paul