QEMU commit 0a923be2f642 broke my or1k image.

QEMU commit 0a923be2f642 broke my or1k image.

[Rob Landley]

Grab this tarball, extract it, and ./run-qemu.sh. It's a simple
linux+initramfs image that boots to a shell prompt.

  https://landley.net/bin/mkroot/latest/or1k.tgz

QEMU 7.0.0 ran that linux-or1k image, but newer qemu does not. I besected the
issue to qemu commit 0a923be2f642, and it's still broken in current tip of tree.

Rebuilding the image with current linux-git doesn't seem to make a difference?
Either way I get serial output with old qemu and don't with current qemu.

Rob

P.S. Reproduction sequence for the tarball available upon request, kernel config
is in the docs/ directory, userspace is just a toybox mkroot build but "not
getting kernel boot messages" happens way before userspace gets a vote.

Re: QEMU commit 0a923be2f642 broke my or1k image.

[Stafford Horne]

Hi Rob,

Sorry, I haven't had much time to sit down and work on this mail in the last two
weeks but wanted to get somethign back to you.  Here it goes.

On Wed, Sep 11, 2024 at 12:42:58AM -0500, Rob Landley wrote:
> Grab this tarball, extract it, and ./run-qemu.sh. It's a simple
> linux+initramfs image that boots to a shell prompt.
> 
>   https://landley.net/bin/mkroot/latest/or1k.tgz
> 
> QEMU 7.0.0 ran that linux-or1k image, but newer qemu does not. I besected the
> issue to qemu commit 0a923be2f642, and it's still broken in current tip of tree.

Patch is:

 0a923be2f6 ("hw/openrisc: page-align FDT address")

> Rebuilding the image with current linux-git doesn't seem to make a difference?
> Either way I get serial output with old qemu and don't with current qemu.

The bisect looks strange as it's only moving a page boundary, it could be
correct but it seems harmeless.  There is another commit close by that was
causing issues with serial output for the barebox guys and this is a patch I am
working on to fix it.

https://lore.kernel.org/qemu-devel/20240908062756.70514-1-shorne@gmail.com/

I will try to get time today to look at your tarball and run it, but if not have
a look at this patch.

-Stafford

> Rob
> 
> P.S. Reproduction sequence for the tarball available upon request, kernel config
> is in the docs/ directory, userspace is just a toybox mkroot build but "not
> getting kernel boot messages" happens way before userspace gets a vote.
> 

Re: QEMU commit 0a923be2f642 broke my or1k image.

[Rob Landley]

On 9/16/24 02:21, Stafford Horne wrote:
> On Wed, Sep 11, 2024 at 12:42:58AM -0500, Rob Landley wrote:
>> Grab this tarball, extract it, and ./run-qemu.sh. It's a simple
>> linux+initramfs image that boots to a shell prompt.
>>
>>    https://landley.net/bin/mkroot/latest/or1k.tgz
>>
>> QEMU 7.0.0 ran that linux-or1k image, but newer qemu does not. I besected the
>> issue to qemu commit 0a923be2f642, and it's still broken in current tip of tree.
> 
> Patch is:
> 
>   0a923be2f6 ("hw/openrisc: page-align FDT address")
> 
>> Rebuilding the image with current linux-git doesn't seem to make a difference?
>> Either way I get serial output with old qemu and don't with current qemu.
> 
> The bisect looks strange as it's only moving a page boundary, it could be
> correct but it seems harmeless.  There is another commit close by that was
> causing issues with serial output for the barebox guys and this is a patch I am
> working on to fix it.
> 
> https://lore.kernel.org/qemu-devel/20240908062756.70514-1-shorne@gmail.com/
> 
> I will try to get time today to look at your tarball and run it, but if not have
> a look at this patch.

I just regression tested linux-6.12 against qemu-9.2.0-rc1 and I'm still 
getting no output from or1k, with the current image or the old or1k 
release image that worked under old qemu versions.

I tried applying your serial patch to current qemu, and it made no 
difference: still no output booting the image.

Alas I can no longer test that reverting the commit I identified (git 
show $HASH | patch -p1 -r) makes it work again in current qemu (although 
it did at one point, that's a standard sanity check at the end of bisect 
for me), and I can't fix it up by hand either because 
hw/openrisc/openrisc_sim.c no longer contains the string "load_sim", 
looks like it got collated with common code for other architectures.

I'm happy to tweak the kernel config if qemu changed in a way that broke 
compatibility with old images (I assume _you_ have a kernel that boots), 
but I was hoping "this week's kernel release" would have any necessary 
patches to work with QEMU's changed ABI.

Thanks,

Rob

Re: QEMU commit 0a923be2f642 broke my or1k image.

[Stafford Horne]

On Thu, Nov 21, 2024 at 04:32:25PM -0600, Rob Landley wrote:
> On 9/16/24 02:21, Stafford Horne wrote:
> > On Wed, Sep 11, 2024 at 12:42:58AM -0500, Rob Landley wrote:
> > > Grab this tarball, extract it, and ./run-qemu.sh. It's a simple
> > > linux+initramfs image that boots to a shell prompt.
> > > 
> > >    https://landley.net/bin/mkroot/latest/or1k.tgz
> > > 
> > > QEMU 7.0.0 ran that linux-or1k image, but newer qemu does not. I besected the
> > > issue to qemu commit 0a923be2f642, and it's still broken in current tip of tree.
> > 
> > Patch is:
> > 
> >   0a923be2f6 ("hw/openrisc: page-align FDT address")
> > 
> > > Rebuilding the image with current linux-git doesn't seem to make a difference?
> > > Either way I get serial output with old qemu and don't with current qemu.
> > 
> > The bisect looks strange as it's only moving a page boundary, it could be
> > correct but it seems harmeless.  There is another commit close by that was
> > causing issues with serial output for the barebox guys and this is a patch I am
> > working on to fix it.
> > 
> > https://lore.kernel.org/qemu-devel/20240908062756.70514-1-shorne@gmail.com/
> > 
> > I will try to get time today to look at your tarball and run it, but if not have
> > a look at this patch.
> 
> I just regression tested linux-6.12 against qemu-9.2.0-rc1 and I'm still
> getting no output from or1k, with the current image or the old or1k release
> image that worked under old qemu versions.
> 
> I tried applying your serial patch to current qemu, and it made no
> difference: still no output booting the image.

OK, I am trying to get that patch pushed upstream.  I will be sure we fix your
issues before we do that.

> Alas I can no longer test that reverting the commit I identified (git show
> $HASH | patch -p1 -r) makes it work again in current qemu (although it did
> at one point, that's a standard sanity check at the end of bisect for me),
> and I can't fix it up by hand either because hw/openrisc/openrisc_sim.c no
> longer contains the string "load_sim", looks like it got collated with
> common code for other architectures.
> 
> I'm happy to tweak the kernel config if qemu changed in a way that broke
> compatibility with old images (I assume _you_ have a kernel that boots), but
> I was hoping "this week's kernel release" would have any necessary patches
> to work with QEMU's changed ABI.

The patch I have put in 6.12 only helps with earlycon.  Regular output has
always been working for me.  I will work on testing your image again to see what
is special about it.

Note, I did find some issues with the kernel nor properly handling stdout-path.
It seems that if there are multiple uarts the first one will always be used as
the default uart.  Only the console= command line argument can be used to
override that.

So as long as the kernel selects the first uart as the stdout path (it should
have always been the case)  AND qemu picks the first uart as the uart connected
to the console (it should be with my latest patch) we should get serial output.

Another thing that may get me delayed is that I need to rebuild my or1k-elf-gdb
as it's python version is no longer compatible with the kernel's python.

-Stafford

Re: QEMU commit 0a923be2f642 broke my or1k image.

[Rob Landley]

On 11/22/24 10:35, Stafford Horne wrote:
>> I just regression tested linux-6.12 against qemu-9.2.0-rc1 and I'm still
>> getting no output from or1k, with the current image or the old or1k release
>> image that worked under old qemu versions.
>>
>> I tried applying your serial patch to current qemu, and it made no
>> difference: still no output booting the image.
> 
> OK, I am trying to get that patch pushed upstream.  I will be sure we fix your
> issues before we do that.

Thanks. Let me know how/if I can help.

>> Alas I can no longer test that reverting the commit I identified (git show
>> $HASH | patch -p1 -r) makes it work again in current qemu (although it did
>> at one point, that's a standard sanity check at the end of bisect for me),
>> and I can't fix it up by hand either because hw/openrisc/openrisc_sim.c no
>> longer contains the string "load_sim", looks like it got collated with
>> common code for other architectures.
>>
>> I'm happy to tweak the kernel config if qemu changed in a way that broke
>> compatibility with old images (I assume _you_ have a kernel that boots), but
>> I was hoping "this week's kernel release" would have any necessary patches
>> to work with QEMU's changed ABI.
> 
> The patch I have put in 6.12 only helps with earlycon.  Regular output has
> always been working for me.  I will work on testing your image again to see what
> is special about it.

Just a guess, but given the alignment change, I suspect it's barfing on 
the statically linked initramfs? That seems the most likely step to go 
off the rails given the failing patch is a symbol alignment change in 
the flattened device tree plumbing, and I think the initramfs extractor 
parses device trees very early on to find stuff (I forget why). Moving 
"where the data lives" without a corresponding change to the "where to 
look for the data" code seems a bit strange, but it's not my area...

Here's the miniconfig I built 6.12 with (90% of which is generic to all 
the architectures I'm testing, the sections are labeled. The 
console="FIXME" bit is because I can't get qemu-system-or1k -append 
"blah" to go through to linux, so I stuck FIXME in that field for the 
or1k target and it wound up in the output):

# make ARCH=openrisc allnoconfig KCONFIG_ALLCONFIG=linux-miniconfig
# make ARCH=openrisc -j $(nproc)
# boot vmlinux console=FIXME

# architecture independent
CONFIG_PANIC_TIMEOUT=1
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_RD_GZIP=y
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_SCRIPT=y
CONFIG_BLK_DEV=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_EXT4_FS=y
CONFIG_EXT4_USE_FOR_EXT2=y
CONFIG_VFAT_FS=y
CONFIG_FAT_DEFAULT_UTF8=y
CONFIG_MISC_FILESYSTEMS=y
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ISO8859_1=y
CONFIG_SQUASHFS=y
CONFIG_SQUASHFS_XATTR=y
CONFIG_SQUASHFS_ZLIB=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_NET=y
CONFIG_NETDEVICES=y
CONFIG_NET_CORE=y
CONFIG_NETCONSOLE=y
CONFIG_PACKET=y
CONFIG_UNIX=y
CONFIG_INET=y
CONFIG_IPV6=y
CONFIG_ETHERNET=y
CONFIG_COMPAT_32BIT_TIME=y
CONFIG_EARLY_PRINTK=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y

# architecture specific
CONFIG_ETHOC=y
CONFIG_SERIO=y
CONFIG_SERIAL_OF_PLATFORM=y
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_OPENRISC_BUILTIN_DTB="or1ksim"

# architecture extra
CONFIG_INITRAMFS_SOURCE="../../root/or1k/fs"

And then run-qemu.sh is just:

DIR="$(dirname $0)"; qemu-system-or1k -M or1k-sim -m 256 "$@" -nographic 
-no-reboot -kernel "$DIR"/linux-kernel -append "HOST=or1k console=FIXME 
$KARGS"
echo -e '\e[?7h'

(The echo is because some qemu bioses disable wordwrap in the terminal 
and leave it broken, driving bash cursor-up history nuts. My plumbing 
just sticks a reset in all the architectures' scripts...)

The or1k support in mkroot is the 4 lines at:

   https://github.com/landley/toybox/blob/master/mkroot/mkroot.sh#L256

(I'd cut and paste them here but thunderbird wordwraps everything.)

Also, looking at that, I'm using a builtin DTB and you might be passing 
one in via -dtb? Another thing the alignment change might break...

> Note, I did find some issues with the kernel nor properly handling stdout-path.
> It seems that if there are multiple uarts the first one will always be used as
> the default uart.  Only the console= command line argument can be used to
> override that.

I've never managed to get console= to go through to linux in 
qemu-system-or1k. The above tries but is ignored.

It's also doing a statically linked initramfs because -initrd didn't 
work on this target. Happy to update if it's been fixed, the other 
targets are almost all using -initrd to feed in an external cpio.gz

> So as long as the kernel selects the first uart as the stdout path (it should
> have always been the case)  AND qemu picks the first uart as the uart connected
> to the console (it should be with my latest patch) we should get serial output.

The commit in question is not changing UART configuration. It's changing 
the alignment of a symbol. Your patch to the UART plumbing did not fix 
it. I don't see how the UART configuration could have changed between 
the "working" and "not working" states.

> Another thing that may get me delayed is that I need to rebuild my or1k-elf-gdb
> as it's python version is no longer compatible with the kernel's python.

Yeah, Python 3's forced upgrade treadmill is deeply silly. I'm not a 
fan: https://landley.net/notes-2024.html#09-04-2024

(I know they're doing it to make sure they never have another "our 
userbase refuses to move off 2.0 for a full decade" moment by holding 
tightly to the control they wrested away from their users and never 
giving an inch back now they've forced the issue, by making sure their 
code spoils like milk and the installed base is regularly slashed and 
burned, but for me that just means "use lua" or something. Yes your dev 
team had trauma. So did the perl guys, who eventually gave UP forcing 
their transition down an unwilling userbase's throat... But again, not 
my area.)

> -Stafford

Rob

Re: QEMU commit 0a923be2f642 broke my or1k image.

[Stafford Horne]

On Fri, Nov 22, 2024 at 06:54:21PM -0600, Rob Landley wrote:
> On 11/22/24 10:35, Stafford Horne wrote:
> > > I just regression tested linux-6.12 against qemu-9.2.0-rc1 and I'm still
> > > getting no output from or1k, with the current image or the old or1k release
> > > image that worked under old qemu versions.
> > > 
> > > I tried applying your serial patch to current qemu, and it made no
> > > difference: still no output booting the image.
> > 
> > OK, I am trying to get that patch pushed upstream.  I will be sure we fix your
> > issues before we do that.
> 
> Thanks. Let me know how/if I can help.
> 
> > > Alas I can no longer test that reverting the commit I identified (git show
> > > $HASH | patch -p1 -r) makes it work again in current qemu (although it did
> > > at one point, that's a standard sanity check at the end of bisect for me),
> > > and I can't fix it up by hand either because hw/openrisc/openrisc_sim.c no
> > > longer contains the string "load_sim", looks like it got collated with
> > > common code for other architectures.
> > > 
> > > I'm happy to tweak the kernel config if qemu changed in a way that broke
> > > compatibility with old images (I assume _you_ have a kernel that boots), but
> > > I was hoping "this week's kernel release" would have any necessary patches
> > > to work with QEMU's changed ABI.
> > 
> > The patch I have put in 6.12 only helps with earlycon.  Regular output has
> > always been working for me.  I will work on testing your image again to see what
> > is special about it.
> 
> Just a guess, but given the alignment change, I suspect it's barfing on the
> statically linked initramfs? That seems the most likely step to go off the
> rails given the failing patch is a symbol alignment change in the flattened
> device tree plumbing, and I think the initramfs extractor parses device
> trees very early on to find stuff (I forget why). Moving "where the data
> lives" without a corresponding change to the "where to look for the data"
> code seems a bit strange, but it's not my area...

OK, and the broken earlycon may be masking what is going on, as we should at
least see some console output before things fail.  The earlcon fix is in 6.13
not 6.12.

I was able to test your or1k.tgz image and figure out what is wrong.  Your
run-qemu.sh script has 'console=FIXME'.  This command line argument is taken in
and is causing the boot process to not be able to find the console.

Changing it to 'console=ttyS0' allows me to see the output.

I put a branch with the qemu patches I have here:

  - https://github.com/stffrdhrn/qemu/tree/or1k-9.2.0-fixes-1

> Here's the miniconfig I built 6.12 with (90% of which is generic to all the
> architectures I'm testing, the sections are labeled. The console="FIXME" bit
> is because I can't get qemu-system-or1k -append "blah" to go through to
> linux, so I stuck FIXME in that field for the or1k target and it wound up in
> the output):

The kernel command line is injected by qemu into the qemu generated
devicetree.  I notice when I boot your kernel with the reverted FDT alignment
fix the console prints:

    Kernel command line: earlycon

This means that the qemu devicetree is not being used, hence the command line
args are not working.  The qemu device tree not being used is not good, but that
is why reverting the alignment fix 'seems' to fix the issue.  To me the revert
looks to be breaking the qemu devicetree allowing us to fall back to the kernel
supplied devicetree.

This gave me a clue as to what was going on.

> # make ARCH=openrisc allnoconfig KCONFIG_ALLCONFIG=linux-miniconfig
> # make ARCH=openrisc -j $(nproc)
> # boot vmlinux console=FIXME
> 
> # architecture independent
> CONFIG_PANIC_TIMEOUT=1
> CONFIG_NO_HZ=y
> CONFIG_HIGH_RES_TIMERS=y
> CONFIG_RD_GZIP=y
> CONFIG_BINFMT_ELF=y
> CONFIG_BINFMT_SCRIPT=y
> CONFIG_BLK_DEV=y
> CONFIG_BLK_DEV_INITRD=y
> CONFIG_BLK_DEV_LOOP=y
> CONFIG_EXT4_FS=y
> CONFIG_EXT4_USE_FOR_EXT2=y
> CONFIG_VFAT_FS=y
> CONFIG_FAT_DEFAULT_UTF8=y
> CONFIG_MISC_FILESYSTEMS=y
> CONFIG_NLS_CODEPAGE_437=y
> CONFIG_NLS_ISO8859_1=y
> CONFIG_SQUASHFS=y
> CONFIG_SQUASHFS_XATTR=y
> CONFIG_SQUASHFS_ZLIB=y
> CONFIG_TMPFS=y
> CONFIG_TMPFS_POSIX_ACL=y
> CONFIG_DEVTMPFS=y
> CONFIG_DEVTMPFS_MOUNT=y
> CONFIG_NET=y
> CONFIG_NETDEVICES=y
> CONFIG_NET_CORE=y
> CONFIG_NETCONSOLE=y
> CONFIG_PACKET=y
> CONFIG_UNIX=y
> CONFIG_INET=y
> CONFIG_IPV6=y
> CONFIG_ETHERNET=y
> CONFIG_COMPAT_32BIT_TIME=y
> CONFIG_EARLY_PRINTK=y
> CONFIG_IKCONFIG=y
> CONFIG_IKCONFIG_PROC=y
> 
> # architecture specific
> CONFIG_ETHOC=y
> CONFIG_SERIO=y
> CONFIG_SERIAL_OF_PLATFORM=y
> CONFIG_SERIAL_8250=y
> CONFIG_SERIAL_8250_CONSOLE=y
> CONFIG_OPENRISC_BUILTIN_DTB="or1ksim"
> 
> # architecture extra
> CONFIG_INITRAMFS_SOURCE="../../root/or1k/fs"
> 
> And then run-qemu.sh is just:
> 
> DIR="$(dirname $0)"; qemu-system-or1k -M or1k-sim -m 256 "$@" -nographic
> -no-reboot -kernel "$DIR"/linux-kernel -append "HOST=or1k console=FIXME
> $KARGS"
> echo -e '\e[?7h'
> 
> (The echo is because some qemu bioses disable wordwrap in the terminal and
> leave it broken, driving bash cursor-up history nuts. My plumbing just
> sticks a reset in all the architectures' scripts...)
> 
> The or1k support in mkroot is the 4 lines at:
> 
>   https://github.com/landley/toybox/blob/master/mkroot/mkroot.sh#L256
> 
> (I'd cut and paste them here but thunderbird wordwraps everything.)
> 
> Also, looking at that, I'm using a builtin DTB and you might be passing one
> in via -dtb? Another thing the alignment change might break...

Thanks for the steps.  I was just using the or1k.tgz you provided earlier.  The
above will help if I want to try some kernel fixes on my own.

> > Note, I did find some issues with the kernel nor properly handling stdout-path.
> > It seems that if there are multiple uarts the first one will always be used as
> > the default uart.  Only the console= command line argument can be used to
> > override that.
> 
> I've never managed to get console= to go through to linux in
> qemu-system-or1k. The above tries but is ignored.

As I mentioned above this is a good clue and explains why the alignment "fix"
fixes your issue.

> It's also doing a statically linked initramfs because -initrd didn't work on
> this target. Happy to update if it's been fixed, the other targets are
> almost all using -initrd to feed in an external cpio.gz

Using -initrd should work.  But also using the statically linked initramfs
should be fine too.  The setup I use for testing uses virt with a virtio block
driver.

When using qemu with -initrd qemu will back the kernel, initrd and fdt one after
the other into memory as per.

[ kernel ] - Loads from 0x100 (based on elf layout)
[ initrd ] - page aligned
[  fdt   ] - page aligned devicetree (revert moved to 4 bytes aligned)

The fdt address gets placed into r3 which the kernel uses to find the qemu FDT.
Finding the FDT one of the first steps of the boot processes.

> > So as long as the kernel selects the first uart as the stdout path (it should
> > have always been the case)  AND qemu picks the first uart as the uart connected
> > to the console (it should be with my latest patch) we should get serial output.
> 
> The commit in question is not changing UART configuration. It's changing the
> alignment of a symbol. Your patch to the UART plumbing did not fix it. I
> don't see how the UART configuration could have changed between the
> "working" and "not working" states.

As mentioned, it's causing us to switch off access to the QEMU supplied
FDT which is allowing the uart to work.

If you provide command line args console=ttyS0 things will work.

Also console=ttyS0 is not used as all as it should be the default in QEMU.

> > Another thing that may get me delayed is that I need to rebuild my or1k-elf-gdb
> > as it's python version is no longer compatible with the kernel's python.
> 
> Yeah, Python 3's forced upgrade treadmill is deeply silly. I'm not a fan:
> https://landley.net/notes-2024.html#09-04-2024
> 
> (I know they're doing it to make sure they never have another "our userbase
> refuses to move off 2.0 for a full decade" moment by holding tightly to the
> control they wrested away from their users and never giving an inch back now
> they've forced the issue, by making sure their code spoils like milk and the
> installed base is regularly slashed and burned, but for me that just means
> "use lua" or something. Yes your dev team had trauma. So did the perl guys,
> who eventually gave UP forcing their transition down an unwilling userbase's
> throat... But again, not my area.)

It looks like I didnt need this to solve the issue.  But I will need it soon.
sigh.

It looks like the root cause of the issue was the 'console=FIXME'.

I hope it helps.

-Stafford

Re: QEMU commit 0a923be2f642 broke my or1k image.

[Rob Landley]

On 11/23/24 02:28, Stafford Horne wrote:
>> Just a guess, but given the alignment change, I suspect it's barfing on the
>> statically linked initramfs? That seems the most likely step to go off the
>> rails given the failing patch is a symbol alignment change in the flattened
>> device tree plumbing, and I think the initramfs extractor parses device
>> trees very early on to find stuff (I forget why). Moving "where the data
>> lives" without a corresponding change to the "where to look for the data"
>> code seems a bit strange, but it's not my area...
> 
> OK, and the broken earlycon may be masking what is going on, as we should at
> least see some console output before things fail.  The earlcon fix is in 6.13
> not 6.12.
> 
> I was able to test your or1k.tgz image and figure out what is wrong.  Your
> run-qemu.sh script has 'console=FIXME'.  This command line argument is taken in
> and is causing the boot process to not be able to find the console.
> 
> Changing it to 'console=ttyS0' allows me to see the output.

Ha, so it STARTED parsing console= and broke. Oops. (It was there so I'd 
notice...)

> I put a branch with the qemu patches I have here:
> 
>    - https://github.com/stffrdhrn/qemu/tree/or1k-9.2.0-fixes-1
> 
>> Here's the miniconfig I built 6.12 with (90% of which is generic to all the
>> architectures I'm testing, the sections are labeled. The console="FIXME" bit
>> is because I can't get qemu-system-or1k -append "blah" to go through to
>> linux, so I stuck FIXME in that field for the or1k target and it wound up in
>> the output):
> 
> The kernel command line is injected by qemu into the qemu generated
> devicetree.  I notice when I boot your kernel with the reverted FDT alignment
> fix the console prints:
> 
>      Kernel command line: earlycon
> 
> This means that the qemu devicetree is not being used, hence the command line
> args are not working.  The qemu device tree not being used is not good, but that
> is why reverting the alignment fix 'seems' to fix the issue.  To me the revert
> looks to be breaking the qemu devicetree allowing us to fall back to the kernel
> supplied devicetree.

I'm happy to do it the "right" way if I know what that is. I just 
stumbled around and got it to work.

>> Also, looking at that, I'm using a builtin DTB and you might be passing one
>> in via -dtb? Another thing the alignment change might break...
> 
> Thanks for the steps.  I was just using the or1k.tgz you provided earlier.  The
> above will help if I want to try some kernel fixes on my own.

I'm attempting to regression test as many targets as I can to get 
consistent basic behavior out of:

   https://landley.net/bin/mkroot/0.8.11/

I'm trying to get a new release out with the 6.12 kernel which is why 
I'm revisiting this now.

I've even got a test script that runs all the targets under qemu 
(booting them in parallel even) and checks that A) they boot and run 
userspace, B) they can talk to an emulated hard disk, C) they can talk 
to an emulated network, D) the clock gets set reasonably, E) it knows 
how to exit the emulator. You'd be surprised how many regressions there 
are in just that...

Speaking of which, is there a way to get or1k to exit the emulator? I 
told the kernel to reboot but it says "reboot failed, system halted" and 
hangs instead of exiting qemu. (My testroot runs qemu under "timeout -i 
10" to kill it after 10 seconds of inactivity, I.E. nothing written to 
stdout, but it still counts as a failure on one of the criteria.)

>>> Note, I did find some issues with the kernel nor properly handling stdout-path.
>>> It seems that if there are multiple uarts the first one will always be used as
>>> the default uart.  Only the console= command line argument can be used to
>>> override that.
>>
>> I've never managed to get console= to go through to linux in
>> qemu-system-or1k. The above tries but is ignored.
> 
> As I mentioned above this is a good clue and explains why the alignment "fix"
> fixes your issue.

Happy to do it properly. Almost all the other targets can do console=, 
the FIXME was there to highlight the fact it didn't work right. 
(Silently working for the WRONG REASON is still bad when regression 
testing.)

>> It's also doing a statically linked initramfs because -initrd didn't work on
>> this target. Happy to update if it's been fixed, the other targets are
>> almost all using -initrd to feed in an external cpio.gz
> 
> Using -initrd should work.  But also using the statically linked initramfs
> should be fine too.  The setup I use for testing uses virt with a virtio block
> driver.

Most of the other targets _don't_ use builtin initramfs, so you can swap 
them out "aftermarket" as it were. When it's separate you can examine 
and edit the contents without rebuilding the kernel...

> When using qemu with -initrd qemu will back the kernel, initrd and fdt one after
> the other into memory as per.
> 
> [ kernel ] - Loads from 0x100 (based on elf layout)
> [ initrd ] - page aligned
> [  fdt   ] - page aligned devicetree (revert moved to 4 bytes aligned)
> 
> The fdt address gets placed into r3 which the kernel uses to find the qemu FDT.
> Finding the FDT one of the first steps of the boot processes.

I updated my mkroot config:

   https://github.com/landley/toybox/commit/fb3ca98e2faa

I.E. changed the FIXME to ttyS0, removed BUILTIN=1 so it's no longer 
statically linking the initramfs image, and yanked the builtin DTB, and 
the result works with v9.2.0-rc1.

Still doesn't know how to exit qemu, though. (Is there a kernel symbol I 
can add to 6.12, or does qemu still not have an exit mechanism for this 
board yet?)

(FYI: be2csv is a shell function to convert bash's brace expansion 
syntax to a comma separated value list, and then csv2cfg is another 
shell function that turns the CSV into https://lwn.net/Articles/160497/ 
. The CSV is shipped as docs/linux-microconfig in the tarball if you're 
curious. That's how a 400 line bash script can build a Linux system that 
boots to a shell prompt for a dozen architectures. The or1k config is 
now 2 lines, for example. 3 with the "if or1k" check. The variables it 
assigns to are documented around line 190.)

> If you provide command line args console=ttyS0 things will work.
> 
> Also console=ttyS0 is not used as all as it should be the default in QEMU.

I specify it explicitly to be consistent across architectures.

> It looks like the root cause of the issue was the 'console=FIXME'.
> 
> I hope it helps.

Yup, I just had to remove workarounds for old qemu that are no longer 
needed. Thanks for the help. (If you do teach qemu to exit at some 
point, please let me know...)

> -Stafford

Thanks,

Rob

Re: QEMU commit 0a923be2f642 broke my or1k image.

[Stafford Horne]

On Sat, Nov 23, 2024 at 11:18:53PM -0600, Rob Landley wrote:
...
> I'm attempting to regression test as many targets as I can to get consistent
> basic behavior out of:
> 
>   https://landley.net/bin/mkroot/0.8.11/
> 
> I'm trying to get a new release out with the 6.12 kernel which is why I'm
> revisiting this now.
> 
> I've even got a test script that runs all the targets under qemu (booting
> them in parallel even) and checks that A) they boot and run userspace, B)
> they can talk to an emulated hard disk, C) they can talk to an emulated
> network, D) the clock gets set reasonably, E) it knows how to exit the
> emulator. You'd be surprised how many regressions there are in just that...
> 
> Speaking of which, is there a way to get or1k to exit the emulator? I told
> the kernel to reboot but it says "reboot failed, system halted" and hangs
> instead of exiting qemu. (My testroot runs qemu under "timeout -i 10" to
> kill it after 10 seconds of inactivity, I.E. nothing written to stdout, but
> it still counts as a failure on one of the criteria.)

With the or1k-sim board it is not possible.  Traditionally on or1k we use
special NOP instructions to reboot simulators i.e. 'l.nop 1'.  The QEMU team was
not happy to support this mechanism.

  https://lore.kernel.org/openrisc/fb69c137317a365dcb549dfef1ecd2fbff48e92c.1492384862.git.shorne@gmail.com/T/#mef76d30d294aed84548b1e0d3e23aae499deaeec

As an alertnative we can use the 'virt' board which supports shutdown and
restart via the generic syscon drivers.

    $ grep -C1 SYSCON arch/openrisc/configs/virt_defconfig
    CONFIG_POWER_RESET=y
    CONFIG_POWER_RESET_SYSCON=y
    CONFIG_POWER_RESET_SYSCON_POWEROFF=y
    CONFIG_SYSCON_REBOOT_MODE=y
    # CONFIG_HWMON is not set

> > > > Note, I did find some issues with the kernel nor properly handling stdout-path.
> > > > It seems that if there are multiple uarts the first one will always be used as
> > > > the default uart.  Only the console= command line argument can be used to
> > > > override that.
> > > 
> > > I've never managed to get console= to go through to linux in
> > > qemu-system-or1k. The above tries but is ignored.
> > 
> > As I mentioned above this is a good clue and explains why the alignment "fix"
> > fixes your issue.
> 
> Happy to do it properly. Almost all the other targets can do console=, the
> FIXME was there to highlight the fact it didn't work right. (Silently
> working for the WRONG REASON is still bad when regression testing.)
> 
> > > It's also doing a statically linked initramfs because -initrd didn't work on
> > > this target. Happy to update if it's been fixed, the other targets are
> > > almost all using -initrd to feed in an external cpio.gz
> > 
> > Using -initrd should work.  But also using the statically linked initramfs
> > should be fine too.  The setup I use for testing uses virt with a virtio block
> > driver.
> 
> Most of the other targets _don't_ use builtin initramfs, so you can swap
> them out "aftermarket" as it were. When it's separate you can examine and
> edit the contents without rebuilding the kernel...
> 
> > When using qemu with -initrd qemu will back the kernel, initrd and fdt one after
> > the other into memory as per.
> > 
> > [ kernel ] - Loads from 0x100 (based on elf layout)
> > [ initrd ] - page aligned
> > [  fdt   ] - page aligned devicetree (revert moved to 4 bytes aligned)
> > 
> > The fdt address gets placed into r3 which the kernel uses to find the qemu FDT.
> > Finding the FDT one of the first steps of the boot processes.
> 
> I updated my mkroot config:
> 
>   https://github.com/landley/toybox/commit/fb3ca98e2faa
> 
> I.E. changed the FIXME to ttyS0, removed BUILTIN=1 so it's no longer
> statically linking the initramfs image, and yanked the builtin DTB, and the
> result works with v9.2.0-rc1.
> 
> Still doesn't know how to exit qemu, though. (Is there a kernel symbol I can
> add to 6.12, or does qemu still not have an exit mechanism for this board
> yet?)
> 
> (FYI: be2csv is a shell function to convert bash's brace expansion syntax to
> a comma separated value list, and then csv2cfg is another shell function
> that turns the CSV into https://lwn.net/Articles/160497/ . The CSV is
> shipped as docs/linux-microconfig in the tarball if you're curious. That's
> how a 400 line bash script can build a Linux system that boots to a shell
> prompt for a dozen architectures. The or1k config is now 2 lines, for
> example. 3 with the "if or1k" check. The variables it assigns to are
> documented around line 190.)

That looks good. I need to work on getting my integration testing better, Ill
look into your tools for that.

-Stafford

Re: or1k -M virt -hda and net.

[Rob Landley]

On 11/24/24 00:50, Stafford Horne wrote:
>> Speaking of which, is there a way to get or1k to exit the emulator? I told
>> the kernel to reboot but it says "reboot failed, system halted" and hangs
>> instead of exiting qemu. (My testroot runs qemu under "timeout -i 10" to
>> kill it after 10 seconds of inactivity, I.E. nothing written to stdout, but
>> it still counts as a failure on one of the criteria.)
> 
> With the or1k-sim board it is not possible.  Traditionally on or1k we use
> special NOP instructions to reboot simulators i.e. 'l.nop 1'.  The QEMU team was
> not happy to support this mechanism.
> 
>    https://lore.kernel.org/openrisc/fb69c137317a365dcb549dfef1ecd2fbff48e92c.1492384862.git.shorne@gmail.com/T/#mef76d30d294aed84548b1e0d3e23aae499deaeec
> 
> As an alertnative we can use the 'virt' board which supports shutdown and
> restart via the generic syscon drivers.
> 
>      $ grep -C1 SYSCON arch/openrisc/configs/virt_defconfig
>      CONFIG_POWER_RESET=y
>      CONFIG_POWER_RESET_SYSCON=y
>      CONFIG_POWER_RESET_SYSCON_POWEROFF=y
>      CONFIG_SYSCON_REBOOT_MODE=y
>      # CONFIG_HWMON is not set

I built the 6.12 ARCH=openrisc virt_defconfig and got it to boot my 
initramfs, but while it does shut down, it hasn't got network or block 
device support.

The kernel config looks like it should have virt block device support, 
but nether -hda README nor "-drive file=README,format=raw,id=hd0 -device 
virtio-blk-device,drive=hd0" seem to be wiring it up in qemu where the 
kernel can find it?

Haven't tried to get fancy with the network yet. The buildroot config is 
still or1ksim...

Rob

Re: or1k -M virt -hda and net.

[Rob Landley]

On 12/23/24 07:05, Stafford Horne wrote:
>> The kernel config looks like it should have virt block device support, but
>> nether -hda README nor "-drive file=README,format=raw,id=hd0 -device
>> virtio-blk-device,drive=hd0" seem to be wiring it up in qemu where the
>> kernel can find it?
> 
> The default virt_defconfig should have the drivers we need.  The OpenRISC virt
> platform supports PCI and virtio devices, so I select many of the related
> drivers.

For some reason this message did not show up in qemu-devel's web 
archive, but when I reply all qemu-devel is in the list? Weird.

>      $ grep -C1 -e PCI -e VIRT arch/openrisc/configs/virt_defconfig
>      # CONFIG_WIRELESS is not set
>      CONFIG_PCI=y
>      CONFIG_PCIEPORTBUS=y
>      CONFIG_PCI_HOST_GENERIC=y
>      CONFIG_DEVTMPFS=y
>      --
>      CONFIG_BLK_DEV_NBD=y
>      CONFIG_VIRTIO_BLK=y
>      CONFIG_NETDEVICES=y
>      CONFIG_VIRTIO_NET=y
>      CONFIG_ETHOC=y

I've already got all those except VIRTIO_BLK and VIRTIO_NET.

> In my qemu startup I wire in the hard drive and network in my qemu start script:
> 
>   - https://github.com/stffrdhrn/or1k-utils/blob/master/scripts/qemu-or1k-linux
> 
> This uses:
> 
>    -device virtio-net-pci,netdev=user -netdev user,id=user,net=$IPRANGE.1/24,host=$IPRANGE.100

My other virto targets are just doing:

   -netdev user,id=net0 -device virtio-net-device,netdev=net0

And yes, that works here with VIRTIO_NET enabled. (Dunno why it isn't in 
the kernel defconfig...)

>    -device virtio-blk-device,drive=d0 -drive file=${DISK},id=d0,if=none,format=qcow2

-drive file=file.img,format=raw,id=hd0 -device virtio-blk-device,drive=hd0

My -device looks similar but -drive is file=file.img,format=raw,id=hd0

No idea what if= does? I haven't seemed to need it...

>> Haven't tried to get fancy with the network yet. The buildroot config is
>> still or1ksim...
> 
> Yeah, its nothing fancy for me either. The buildroot with systemV init scripts
> get everything working for me.

My init script (the HERE document starting around 
https://github.com/landley/toybox/blob/master/mkroot/mkroot.sh#L102 ) 
ancestrally goes back to 
https://linuxfromscratch.org/hints/downloads/files/OLD/bsd-init.txt and 
boils down to "call stuff like mount/ifconfig/route in a shell script 
until it's time to exec whatever inherits PID 1".

*shrug* Works fine for embedded systems and small automated containers 
that do a thing and then exit. (You'll notice this one tries to mount 
/dev/?da on /mnt and then run /mnt/init if it exists: that way I can 
plug in an -hda blah.squashfs and have it do stuff automatically. 
Toybox's timeout command has a -i option that's an inactivity timer: if 
nothing is produced on stdout for X seconds, kill the child. So it can 
run forever as long as it's DOING something, but if it stops producing 
output for X seconds it gets ended.)

> -Stafford

Thanks, I think this target is good for release. Now to figure out why 
sh2eb network isn't working after the restore (it used to!). Nor is 
microblaze's network...

Rob

Re: or1k -M virt -hda and net.

[Rob Landley]

On 12/31/24 19:19, Rob Landley wrote:
> On 12/23/24 07:05, Stafford Horne wrote:
>>> The kernel config looks like it should have virt block device 
>>> support, but
...
>>    -device virtio-blk-device,drive=d0 -drive 
>> file=${DISK},id=d0,if=none,format=qcow2
> 
> -drive file=file.img,format=raw,id=hd0 -device virtio-blk-device,drive=hd0
> 
> My -device looks similar but -drive is file=file.img,format=raw,id=hd0
> 
> No idea what if= does? I haven't seemed to need it...
...
> Thanks, I think this target is good for release. Now to figure out why 
> sh2eb network isn't working after the restore (it used to!). Nor is 
> microblaze's network...

My test harness appends -hda filename.img to the qemu command line, 
which works for all the other targets, and is awkward to turn into
-device lots-of-stuff,file-filename.img,more-stuff inside a shell 
script. (At best it's a special case parsing and rewriting qemu command 
line plumbing to turn "generic" into an architecture-specific workaround.)

In THEORY I should be able to do something like:

root/or1k/run-qemu.sh -netdev user,id=net0 -device 
virtio-net-device,netdev=net0 -device virtio-blk-device,drive=sd0 -hda 
README

And just have extra arch setup that then accepts the generic appended to 
it. But in practice that says:

qemu-system-or1k: -device virtio-blk-device,drive=sd0: Device needs 
media, but drive is empty

Putting the -hda before the -device doesn't help. (And even if it did, 
the result would barf if run _without_ -hda, which should also work.)

Having -hda by itself is accepted by qemu, but I don't know what 
bus/driver gets added as a result (or1k kernel does not find it).

Rob

Re: or1k -M virt -hda and net.

[Stafford Horne]

On Tue, Jan 07, 2025 at 05:56:52AM -0600, Rob Landley wrote:
> On 12/31/24 19:19, Rob Landley wrote:
> > On 12/23/24 07:05, Stafford Horne wrote:
> > > > The kernel config looks like it should have virt block device
> > > > support, but
> ...
> > >    -device virtio-blk-device,drive=d0 -drive
> > > file=${DISK},id=d0,if=none,format=qcow2
> > 
> > -drive file=file.img,format=raw,id=hd0 -device virtio-blk-device,drive=hd0
> > 
> > My -device looks similar but -drive is file=file.img,format=raw,id=hd0
> > 
> > No idea what if= does? I haven't seemed to need it...
> ...
> > Thanks, I think this target is good for release. Now to figure out why
> > sh2eb network isn't working after the restore (it used to!). Nor is
> > microblaze's network...
> 
> My test harness appends -hda filename.img to the qemu command line, which
> works for all the other targets, and is awkward to turn into
> -device lots-of-stuff,file-filename.img,more-stuff inside a shell script.
> (At best it's a special case parsing and rewriting qemu command line
> plumbing to turn "generic" into an architecture-specific workaround.)

Hi Rob,

Sorry, from the laat email I was under the impression that you had everything
working as expected.

> In THEORY I should be able to do something like:
> 
> root/or1k/run-qemu.sh -netdev user,id=net0 -device
> virtio-net-device,netdev=net0 -device virtio-blk-device,drive=sd0 -hda
> README
> 
> And just have extra arch setup that then accepts the generic appended to it.
> But in practice that says:
> 
> qemu-system-or1k: -device virtio-blk-device,drive=sd0: Device needs media,
> but drive is empty
> 
> Putting the -hda before the -device doesn't help. (And even if it did, the
> result would barf if run _without_ -hda, which should also work.)
> 
> Having -hda by itself is accepted by qemu, but I don't know what bus/driver
> gets added as a result (or1k kernel does not find it).

I am having a hard time understanding the use case.

As you know I use the following to wire in the buildroot image which I turned
into a qcow2 disk using my tool:

  https://github.com/stffrdhrn/or1k-utils/blob/master/scripts/qemu-or1k-mkimg

  -device virtio-blk-device,drive=d0 -drive file=${DISK},id=d0,format=qcow2

But I think you want to use:

  -device virtio-blk-device,drive=sd0 -hda XYZ

I have not used -hda before, do you have it working with other targets?

According to the qemu docs in qemu-options.hx. I see:

    Use file as hard disk 0, 1, 2 or 3 image on the default bus of the
    emulated machine (this is for example the IDE bus on most x86 machines,
    but it can also be SCSI, virtio or something else on other target
    architectures). See also the :ref:`disk images` chapter in the System
    Emulation Users Guide.

I think, since we don't have a "default" bus in openrisc this doesn't work so we
need to specify the -drive explictly.

I checked the x86 machine code and confirm it seems to work like this.  There is
code in the system setup to look for hd* drives and wire them into IDE.  There
is no such code in openrisc.

-Stafford

Re: or1k -M virt -hda and net.

[Alex Bennée]

Stafford Horne <shorne@gmail.com> writes:

> On Tue, Jan 07, 2025 at 05:56:52AM -0600, Rob Landley wrote:
>> On 12/31/24 19:19, Rob Landley wrote:
>> > On 12/23/24 07:05, Stafford Horne wrote:
>> > > > The kernel config looks like it should have virt block device
>> > > > support, but
>> ...
>> > >    -device virtio-blk-device,drive=d0 -drive
>> > > file=${DISK},id=d0,if=none,format=qcow2
>> > 
>> > -drive file=file.img,format=raw,id=hd0 -device virtio-blk-device,drive=hd0
>> > 
>> > My -device looks similar but -drive is file=file.img,format=raw,id=hd0
>> > 
>> > No idea what if= does? I haven't seemed to need it...
>> ...
>> > Thanks, I think this target is good for release. Now to figure out why
>> > sh2eb network isn't working after the restore (it used to!). Nor is
>> > microblaze's network...
>> 
>> My test harness appends -hda filename.img to the qemu command line, which
>> works for all the other targets, and is awkward to turn into
>> -device lots-of-stuff,file-filename.img,more-stuff inside a shell script.
>> (At best it's a special case parsing and rewriting qemu command line
>> plumbing to turn "generic" into an architecture-specific workaround.)
>
> Hi Rob,
>
> Sorry, from the laat email I was under the impression that you had everything
> working as expected.
>
>> In THEORY I should be able to do something like:
>> 
>> root/or1k/run-qemu.sh -netdev user,id=net0 -device
>> virtio-net-device,netdev=net0 -device virtio-blk-device,drive=sd0 -hda
>> README
>> 
>> And just have extra arch setup that then accepts the generic appended to it.
>> But in practice that says:
>> 
>> qemu-system-or1k: -device virtio-blk-device,drive=sd0: Device needs media,
>> but drive is empty
>> 
>> Putting the -hda before the -device doesn't help. (And even if it did, the
>> result would barf if run _without_ -hda, which should also work.)
>> 
>> Having -hda by itself is accepted by qemu, but I don't know what bus/driver
>> gets added as a result (or1k kernel does not find it).
>
> I am having a hard time understanding the use case.
>
> As you know I use the following to wire in the buildroot image which I turned
> into a qcow2 disk using my tool:
>
>   https://github.com/stffrdhrn/or1k-utils/blob/master/scripts/qemu-or1k-mkimg
>
>   -device virtio-blk-device,drive=d0 -drive file=${DISK},id=d0,format=qcow2
>
> But I think you want to use:
>
>   -device virtio-blk-device,drive=sd0 -hda XYZ
>
> I have not used -hda before, do you have it working with other targets?
>
> According to the qemu docs in qemu-options.hx. I see:
>
>     Use file as hard disk 0, 1, 2 or 3 image on the default bus of the
>     emulated machine (this is for example the IDE bus on most x86 machines,
>     but it can also be SCSI, virtio or something else on other target
>     architectures). See also the :ref:`disk images` chapter in the System
>     Emulation Users Guide.
>
> I think, since we don't have a "default" bus in openrisc this doesn't work so we
> need to specify the -drive explictly.
>
> I checked the x86 machine code and confirm it seems to work like this.  There is
> code in the system setup to look for hd* drives and wire them into IDE.  There
> is no such code in openrisc.

Yeah don't use -hdX as they are legacy options with a lot of default
assumptions. As the docs say: https://qemu.readthedocs.io/en/master/system/invocation.html#hxtool-1

  The QEMU block device handling options have a long history and have
  gone through several iterations as the feature set and complexity of
  the block layer have grown. Many online guides to QEMU often reference
  older and deprecated options, which can lead to confusion.

  The most explicit way to describe disks is to use a combination of
  -device to specify the hardware device and -blockdev to describe the
  backend. The device defines what the guest sees and the backend
  describes how QEMU handles the data. It is the only guaranteed stable
  interface for describing block devices and as such is recommended for
  management tools and scripting.

  The -drive option combines the device and backend into a single
  command line option which is a more human friendly. There is however
  no interface stability guarantee although some older board models
  still need updating to work with the modern blockdev forms.

  Older options like -hda are essentially macros which expand into -drive
  options for various drive interfaces. The original forms bake in a lot
  of assumptions from the days when QEMU was emulating a legacy PC, they
  are not recommended for modern configurations.

>
> -Stafford

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: or1k -M virt -hda and net.

[Rob Landley]

On 1/7/25 11:31, Stafford Horne wrote:
> On Tue, Jan 07, 2025 at 05:56:52AM -0600, Rob Landley wrote:
>> On 12/31/24 19:19, Rob Landley wrote:
>>> On 12/23/24 07:05, Stafford Horne wrote:
>>>>> The kernel config looks like it should have virt block device
>>>>> support, but
>> ...
>>>>     -device virtio-blk-device,drive=d0 -drive
>>>> file=${DISK},id=d0,if=none,format=qcow2
>>>
>>> -drive file=file.img,format=raw,id=hd0 -device virtio-blk-device,drive=hd0
>>>
>>> My -device looks similar but -drive is file=file.img,format=raw,id=hd0
>>>
>>> No idea what if= does? I haven't seemed to need it...
>> ...
>>> Thanks, I think this target is good for release. Now to figure out why
>>> sh2eb network isn't working after the restore (it used to!). Nor is
>>> microblaze's network...
>>
>> My test harness appends -hda filename.img to the qemu command line, which
>> works for all the other targets, and is awkward to turn into
>> -device lots-of-stuff,file-filename.img,more-stuff inside a shell script.
>> (At best it's a special case parsing and rewriting qemu command line
>> plumbing to turn "generic" into an architecture-specific workaround.)
> 
> Hi Rob,
> 
> Sorry, from the laat email I was under the impression that you had everything
> working as expected.

I have it all working by hand, but not in the automated test harness. 
See the comment at the start of:

https://github.com/landley/toybox/blob/master/mkroot/testroot.sh

>> In THEORY I should be able to do something like:
>>
>> root/or1k/run-qemu.sh -netdev user,id=net0 -device
>> virtio-net-device,netdev=net0 -device virtio-blk-device,drive=sd0 -hda
>> README
>>
>> And just have extra arch setup that then accepts the generic appended to it.
>> But in practice that says:
>>
>> qemu-system-or1k: -device virtio-blk-device,drive=sd0: Device needs media,
>> but drive is empty
>>
>> Putting the -hda before the -device doesn't help. (And even if it did, the
>> result would barf if run _without_ -hda, which should also work.)
>>
>> Having -hda by itself is accepted by qemu, but I don't know what bus/driver
>> gets added as a result (or1k kernel does not find it).
> 
> I am having a hard time understanding the use case.

I have a script (above) that creates a test.sqf squashfs image and 
attaches it to the emulator's first hard drive. My init script tries to
mount /dev/?da /mnt and then if /mnt/init exists it runs that instead of 
launching an interactive shell prompt.

The wildcard works for /dev/hda, /dev/sda, and /dev/vda, so no matter 
which type of block device this architecture is using, the codepath to 
invoke it and to use it is generic as long as qemu can populate the bus 
and the associated kernel driver can find the device. This way I can 
test many different architectures with the same code.

Here, maybe this will help:

$ for i in root/*/run-qemu.sh
 > do echo -n "$i:"
 > KARGS='HANDOFF="find /dev -name ?da"' setsid -d timeout -i 10 
/bin/bash $i -hda README 2>/dev/null | grep ^/dev/
 > done
root/aarch64/run-qemu.sh:/dev/vda
root/armv4l/run-qemu.sh:/dev/sda
root/armv5l/run-qemu.sh:/dev/sda
root/armv7l/run-qemu.sh:/dev/vda
root/i486/run-qemu.sh:/dev/sda
root/i686/run-qemu.sh:/dev/sda
root/m68k/run-qemu.sh:/dev/sda
root/microblaze/run-qemu.sh:root/mips64/run-qemu.sh:/dev/sda
root/mipsel/run-qemu.sh:/dev/sda
root/mips/run-qemu.sh:/dev/sda
root/or1k/run-qemu.sh:root/powerpc64le/run-qemu.sh:/dev/sda
root/powerpc64/run-qemu.sh:/dev/sda
root/powerpc/run-qemu.sh:/dev/sda
root/riscv32/run-qemu.sh:/dev/vda
root/riscv64/run-qemu.sh:/dev/vda
root/s390x/run-qemu.sh:/dev/vda
root/sh2eb/run-qemu.sh:root/sh4eb/run-qemu.sh:/dev/sda
root/sh4/run-qemu.sh:/dev/sda
root/x86_64/run-qemu.sh:/dev/sda

The three hiccups above are sh2eb (which is building for a physical 
board so can't run that kernel under qemu), microblaze (which I 
mentioned hasn't got a working block device at all), and or1k. All the 
other architectures accept the same:

   run-qemu -hda filename

And the hardware details aren't the caller's problem. The loop calling 
each board invocation script knows nothing about the board, and doesn't 
have to. This means many different architectures can be regression 
tested in a generic fashion using a human-friendly UI option.

(The "setsid -d" is needed because otherwise if you redirect or pipe 
stdout qemu-system hangs before launching the kernel. Longstanding bug 
of many years, my notes say the hang is because qemu casts SIGTTOU on 
itself and winds up as a frog, 
https://landley.net/notes-2017.html#16-06-2017 and I thought I'd poked 
qemu-devel about it at the time...)

> As you know I use the following to wire in the buildroot image which I turned
> into a qcow2 disk using my tool:
> 
>    https://github.com/stffrdhrn/or1k-utils/blob/master/scripts/qemu-or1k-mkimg
> 
>    -device virtio-blk-device,drive=d0 -drive file=${DISK},id=d0,format=qcow2
> 
> But I think you want to use:
> 
>    -device virtio-blk-device,drive=sd0 -hda XYZ

Yes. I want to just use -hda XYZ which for most targets "just works" 
already because the board knows where to put -hda. Unfortunately, when I 
pass -hda to qemu-system-or1k it accepts it but I can't find the 
resulting block device. (I don't know what driver I'd need to install to 
talk to wherever -hda went on or1k. I've tried tracing the code but it's 
callbacks from callbacks, like reading a choose your own adventure novel.)

Your instructions showed me how to use virtio-block-device which in 
theory should create a /dev/vda and thus work fine with the existing 
code, so I tried to supplement the existing or1k behavior to tell it 
that -hda should go to virtio. I'd be happy to change my kernel config 
to add whatever driver -hda is currently attaching the block device to 
instead, but either way I had to try to figure out how the data 
structures were initialized to, and softmmu/vl.c has:

             case QEMU_OPTION_hda:
             case QEMU_OPTION_hdb:
             case QEMU_OPTION_hdc:
             case QEMU_OPTION_hdd:
                 drive_add(IF_DEFAULT, popt->index - QEMU_OPTION_hda, 
optarg, HD_OPTS);

Which is in blockdev.c doing:

QemuOpts *drive_add(BlockInterfaceType type, int index, const char 
*file, const char *optstr)
{
     QemuOpts *opts = qemu_opts_parse_noisily(qemu_find_opts("drive"), 
optstr, false);
...
     if (type != IF_DEFAULT) {
         qemu_opt_set(opts, "if", if_name[type], &error_abort);

And figuring out where THAT's initialized (or interrogating it at 
runtime) got me as far as:

   grep -r '"device"' qapi/block-core.json

And I think I'd need to be able to use the "monitor" mode which isn't 
really compatible with my normal "serial console on stdin/stdout" use 
cases so I've never quite picked up the knack...

At that point I asked on the list, before sticking 8 zillion printf()s 
into qemu to try to figure out what it's doing.

(The OTHER hiccup is I'd like to be able to NOT use -hda and not have 
qemu abort because -device isn't consumed. I don't mind a warning, it's 
not like qemu-system isn't ridiculously chatty already, but refusing to 
start is a blocker. The guts of the scripts currently look like:

$ cat root/mips/run-qemu.sh
DIR="$(dirname $0)"; qemu-system-mips -M malta -m 256 "$@" -nographic 
-no-reboot -kernel "$DIR"/linux-kernel -initrd "$DIR"/initramfs.cpio.gz 
-append "HOST=mips console=ttyS0 $KARGS"
echo -e '\e[?7h'

So adding a couple -device potato=aardvark,micromanagement=extreme isn't 
a big change, but trying to PARSE the command line arguments and rewrite 
them is brittle and icky. (My old linux from scratch build used -hda for 
a build control image and -hdb for writeable scratch space. There was 
even the occasional -hdc...)

The $DIR shenanigans are so "subdir/run-qemu.sh -hda file" can find 
"file" out of the current directory but find the initramfs and kernel 
image out of the subdir.

The echo -e is because more than one qemu bios bootup spew (which you 
CANNOT SUPPRESS, I've tried) disables wordwrap in the terminal and 
leaves it like that on exit, and then bash's command line 
editing/history goes nuts until you type "reset". man 4 console_codes 
search for DECAWM.

> I have not used -hda before, do you have it working with other targets?

Almost all of them, yes.

In a previous life, I used this same trick to do automated native builds 
of Linux From Scratch 6.x under the emulator. I even had the emulated 
build configured to call out through the virtual network to distcc 
running on loopback on the host (which let me do an SMP build at about 
make -j3 from a single threaded host).

https://speakerdeck.com/landley/developing-for-non-x86-targets-using-qemu

Back when I maintained busybox I taught it to build Linux From Scratch, 
which allowed distros like Alpine Linux to rebase on it, and that was my 
project doing that:

https://landley.net/aboriginal/about.html
https://landley.net/aboriginal/control-images/
https://github.com/landley/control-images/tree/master/images/lfs-bootstrap
https://landley.net/aboriginal/downloads/old/binaries/1.3.0/extras/

This mkroot stuff is my successor project 
(https://landley.net/aboriginal/news.html), which I hope to get natively 
building Linux From Scratch under qemu using a toybox host someday, as 
part of my ongoing plan to turn android into a native self-hosting OS:

https://landley.net/toybox/about.html
http://lists.landley.net/pipermail/toybox-landley.net/2020-July/011898.html

Bit older and slower than I used to be. That much caffeine just gives me 
a migraine these days.

> According to the qemu docs in qemu-options.hx. I see:
> 
>      Use file as hard disk 0, 1, 2 or 3 image on the default bus of the
>      emulated machine (this is for example the IDE bus on most x86 machines,
>      but it can also be SCSI, virtio or something else on other target
>      architectures). See also the :ref:`disk images` chapter in the System
>      Emulation Users Guide.
> 
> I think, since we don't have a "default" bus in openrisc this doesn't work so we
> need to specify the -drive explictly.

Exactly. Patching qemu to have virto be the default bus would also be 
great, you'll notice half the above targets already have -hda slot into 
/dev/vda. There used to be several actual /dev/hda in there too but Greg 
KH hates IDE the same way he hates my Pixel 3's USB tethering driver, so 
now those pretend to be scsi.

> I checked the x86 machine code and confirm it seems to work like this.  There is
> code in the system setup to look for hd* drives and wire them into IDE.  There
> is no such code in openrisc.

Which is why I was trying to redirect it. I dunno how to add the 
relevant code to the system setup, and it wouldn't work with the 
existing release anyway..

If "or1k can't work with -hda" I can document it as broken this release, 
but ALMOST all my targets are passing the basic smoketest! I'm so close!

$ mkroot/testroot.sh
.....................
PASS=aarch64 armv4l armv5l armv7l i486 i686 m68k mips64 mipsel mips 
powerpc64le powerpc64 powerpc riscv32 riscv64 s390x sh4 x86_64
NOPASS=microblaze or1k sh4eb
No kernel: armv7m sh2eb

Microblaze has no /dev/?da (as in there's no device I could attach it to 
even with the long option, I'd have to use NBD), sh4eb has -hda is 
working but it's having some endianness hiccup with the network card 
(works in sh4 but not sh4eb, I'm trying to track it down, I thought this 
worked at one point), and the two "No kernel" ones legitimately have no 
qemu kernel config because sh2eb is producing a kernel for a physical 
board qemu doesn't emulate (j-core turtle, I should signal that somehow 
so it doesn't show up as an error, adding a qemu-system-jcore target 
would be great but I just did 
https://landley.net/bin/mkroot/latest/linux-patches/0002-sh4-fdpic.patch 
so I can test its nommu fdpic userspace under sh4eb instead. Not perfect 
but for basic smoketests...)

And of course armv7m is a whole SAGA... 
https://landley.net/notes-2023.html#23-02-2023 and 
https://www.spinics.net/lists/linux-sh/msg64491.html and... ahem. 
Tangent. Haven't got that one working yet. In theory the toolchain 
FINALLY has fdpic support, musl I'm not sure about (the toolchain I'm 
building is static pie), and I couldn't find a good kernel config that 
lined up with a qemu -M last I checked. (I've looked around for 
examples, but there's a bit of a gap between buildroot's 
configs/stm32f429_disco_xip_defconfig and configs/qemu_arm_ebbr_defconfig).

> -Stafford

Thanks,

Rob

Re: or1k -M virt -hda and net.

[Rob Landley]

On 1/7/25 12:05, Alex Bennée wrote:
> Stafford Horne <shorne@gmail.com> writes:
>> I have not used -hda before, do you have it working with other targets?
>>
>> According to the qemu docs in qemu-options.hx. I see:
>>
>>      Use file as hard disk 0, 1, 2 or 3 image on the default bus of the
>>      emulated machine (this is for example the IDE bus on most x86 machines,
>>      but it can also be SCSI, virtio or something else on other target
>>      architectures). See also the :ref:`disk images` chapter in the System
>>      Emulation Users Guide.
>>
>> I think, since we don't have a "default" bus in openrisc this doesn't work so we
>> need to specify the -drive explictly.
>>
>> I checked the x86 machine code and confirm it seems to work like this.  There is
>> code in the system setup to look for hd* drives and wire them into IDE.  There
>> is no such code in openrisc.
> 
> Yeah don't use -hdX as they are legacy options with a lot of default
> assumptions. As the docs say: https://qemu.readthedocs.io/en/master/system/invocation.html#hxtool-1
> 
>    The QEMU block device handling options have a long history and have
>    gone through several iterations as the feature set and complexity of
>    the block layer have grown. Many online guides to QEMU often reference
>    older and deprecated options, which can lead to confusion.

I want "a block device from this file" in a generic way that works the 
same across multiple architectures regardless of the board being 
emulated, where I only have to specify the file not explicitly 
micromanage bus plumbing details, and which is easy for a human to type 
from when explained over a voice call.

What's the alternative to -hda you suggest for that?

Can I do "./run-qemu.sh -drive file=blah.img" without the rest? Perhaps 
specify all the details in the script and then optionally add an extra 
argument at the end? I couldn't get that to work:

$ root/or1k/run-qemu.sh -netdev user,id=net0 -device 
virtio-net-device,netdev=net0 -drive format=raw,id=hd0 -device 
virtio-blk-device,drive=hd0 -drive file=README
qemu-system-or1k: -drive format=raw,id=hd0: A block device must be 
specified for "file"

Also, if you say -device and -drive but do NOT specify a file, qemu 
refuses to start. So I can't set the defaults but only optionally use 
them, the way -hda has defaults built into the image that don't cause a 
problem if I DON'T add a -hda argument to the command line.

>    Older options like -hda are essentially macros which expand into -drive
>    options for various drive interfaces.

Where the knowledge of "what this board needs in order to do that" is 
built into qemu rather than provided by the caller, yes.

> The original forms bake in a lot
>    of assumptions from the days when QEMU was emulating a legacy PC, they
>    are not recommended for modern configurations.

I'm building a kernel. It finds /dev/?da so I can mount it. That is my 
desired outcome.

I am attempting to get generic behavior out of multiple architectures, 
among other reasons so I can cross-test and package up "it fails on X, 
here's a build and test" to point package maintainers at.

"It natively builds under the emulator" is the easiest way to make that 
work, which is why https://landley.net/bin/toolchains/latest/ has a 
native.sqf for each cross.tar.xz.

wget system-image-arch.txz
wget toolchain.sqf
wget test.img

./run-emulator.sh -hda test.img -hdb toolchain.sqf

If I have to explain "-drive virtio-potato-walrus,inkpot=striated 
-device collect=striated,burbank-potato,ireland" at somebody whose 
domain expertise is xfce or something, the barrier to getting them to 
reproduce the issue I'm seeing is noticeably higher. If I have to MAKE a 
bespoke wrapper shell script for them with every bug report, the 
likelihood that it works differently for them than when I tried it is 
noticeably nonzero, and the likelihood of the issue going on my todo 
heap and never getting pursued upstream is also noticeably higher. Which 
is why I try to make generic tools...

(Making a _test_ script to demonstrate the issue is normal. If it's 
their project, usually they can tell if I typoed it and fix it up 
themselves because they know what I MEANT. But if I typo the setup for 
the virtual environment, or are missing a prerequisite package install, 
or they hit qemu version skew, or I said /bin/sh and theirs points to 
dash... Brick wall. It either works or it doesn't.)

(And when I have to set up the long version for a nightly cron job, and 
then when the test fails 6 months later and I look at it and go "huh? 
salad?" that's a bad 3am digression as well. And which is more likely to 
break from version skew during qemu version upgrades: two lines of 
micromanaging --longopts or -hda that gets adjusted by the maintainers?)

Rob

P.S. For some reason -hda grew an "I'm going to make the first block 
read-only just like loopback devices do because you can't be trusted" 
nag a few years back, but it's mostly yet more boot spam. I can tell the 
kernel to be quiet during boot, but never figured out the equivalent for 
qemu-system...

Re: or1k -M virt -hda and net.

[Geert Uytterhoeven]

Hi Rob,

On Tue, Jan 7, 2025 at 11:53 PM Rob Landley <rob@landley.net> wrote:
> Microblaze has no /dev/?da (as in there's no device I could attach it to
> even with the long option, I'd have to use NBD), sh4eb has -hda is
> working but it's having some endianness hiccup with the network card
> (works in sh4 but not sh4eb, I'm trying to track it down, I thought this
> worked at one point), and the two "No kernel" ones legitimately have no

That's using sh_eth, right?
Last time I tried booting a big endian ARM kernel on Koelsch (R-Car
M2-W), it booted fine until "unable to mount root" (I have no suitable
userspace), except for broken Ethernet, so it failed to obtain a
DHCP lease.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: or1k -M virt -hda and net.

[BALATON Zoltan]

[-- Attachment #1: Type: text/plain, Size: 6035 bytes --]

On Tue, 7 Jan 2025, Rob Landley wrote:
> On 1/7/25 12:05, Alex Bennée wrote:
>> Stafford Horne <shorne@gmail.com> writes:
>>> I have not used -hda before, do you have it working with other targets?
>>> 
>>> According to the qemu docs in qemu-options.hx. I see:
>>>
>>>      Use file as hard disk 0, 1, 2 or 3 image on the default bus of the
>>>      emulated machine (this is for example the IDE bus on most x86 
>>> machines,
>>>      but it can also be SCSI, virtio or something else on other target
>>>      architectures). See also the :ref:`disk images` chapter in the System
>>>      Emulation Users Guide.
>>> 
>>> I think, since we don't have a "default" bus in openrisc this doesn't work 
>>> so we
>>> need to specify the -drive explictly.
>>> 
>>> I checked the x86 machine code and confirm it seems to work like this. 
>>> There is
>>> code in the system setup to look for hd* drives and wire them into IDE. 
>>> There
>>> is no such code in openrisc.
>> 
>> Yeah don't use -hdX as they are legacy options with a lot of default
>> assumptions. As the docs say: 
>> https://qemu.readthedocs.io/en/master/system/invocation.html#hxtool-1
>>
>>    The QEMU block device handling options have a long history and have
>>    gone through several iterations as the feature set and complexity of
>>    the block layer have grown. Many online guides to QEMU often reference
>>    older and deprecated options, which can lead to confusion.
>
> I want "a block device from this file" in a generic way that works the same 
> across multiple architectures regardless of the board being emulated, where I 
> only have to specify the file not explicitly micromanage bus plumbing 
> details, and which is easy for a human to type from when explained over a 
> voice call.
>
> What's the alternative to -hda you suggest for that?
>
> Can I do "./run-qemu.sh -drive file=blah.img" without the rest? Perhaps 
> specify all the details in the script and then optionally add an extra 
> argument at the end? I couldn't get that to work:
>
> $ root/or1k/run-qemu.sh -netdev user,id=net0 -device 
> virtio-net-device,netdev=net0 -drive format=raw,id=hd0 -device 
> virtio-blk-device,drive=hd0 -drive file=README

You need '-drive if=none,id=hd0,format=raw,file=README' as a single option 
not split into two. With if=none -drive won't auto-create a device so you 
then also need a corresponding -device option for the drive that you seem 
to have already above. If you want -hda to work you may need something 
like commit d36b2f4e78 (hw/ppc/sam460ex: Support short options for adding 
drives) for the machine you use. In particular the MachineClass 
block_default_type field says what's the default interface that -drive and 
other short options should use (at least I think so, I'm no expert on this 
either but searching for it should at least point to where it's handled).

Regards,
BALATON Zoltan

> qemu-system-or1k: -drive format=raw,id=hd0: A block device must be specified 
> for "file"
>
> Also, if you say -device and -drive but do NOT specify a file, qemu refuses 
> to start. So I can't set the defaults but only optionally use them, the way 
> -hda has defaults built into the image that don't cause a problem if I DON'T 
> add a -hda argument to the command line.
>
>>    Older options like -hda are essentially macros which expand into -drive
>>    options for various drive interfaces.
>
> Where the knowledge of "what this board needs in order to do that" is built 
> into qemu rather than provided by the caller, yes.
>
>> The original forms bake in a lot
>>    of assumptions from the days when QEMU was emulating a legacy PC, they
>>    are not recommended for modern configurations.
>
> I'm building a kernel. It finds /dev/?da so I can mount it. That is my 
> desired outcome.
>
> I am attempting to get generic behavior out of multiple architectures, among 
> other reasons so I can cross-test and package up "it fails on X, here's a 
> build and test" to point package maintainers at.
>
> "It natively builds under the emulator" is the easiest way to make that work, 
> which is why https://landley.net/bin/toolchains/latest/ has a native.sqf for 
> each cross.tar.xz.
>
> wget system-image-arch.txz
> wget toolchain.sqf
> wget test.img
>
> ./run-emulator.sh -hda test.img -hdb toolchain.sqf
>
> If I have to explain "-drive virtio-potato-walrus,inkpot=striated -device 
> collect=striated,burbank-potato,ireland" at somebody whose domain expertise 
> is xfce or something, the barrier to getting them to reproduce the issue I'm 
> seeing is noticeably higher. If I have to MAKE a bespoke wrapper shell script 
> for them with every bug report, the likelihood that it works differently for 
> them than when I tried it is noticeably nonzero, and the likelihood of the 
> issue going on my todo heap and never getting pursued upstream is also 
> noticeably higher. Which is why I try to make generic tools...
>
> (Making a _test_ script to demonstrate the issue is normal. If it's their 
> project, usually they can tell if I typoed it and fix it up themselves 
> because they know what I MEANT. But if I typo the setup for the virtual 
> environment, or are missing a prerequisite package install, or they hit qemu 
> version skew, or I said /bin/sh and theirs points to dash... Brick wall. It 
> either works or it doesn't.)
>
> (And when I have to set up the long version for a nightly cron job, and then 
> when the test fails 6 months later and I look at it and go "huh? salad?" 
> that's a bad 3am digression as well. And which is more likely to break from 
> version skew during qemu version upgrades: two lines of micromanaging 
> --longopts or -hda that gets adjusted by the maintainers?)
>
> Rob
>
> P.S. For some reason -hda grew an "I'm going to make the first block 
> read-only just like loopback devices do because you can't be trusted" nag a 
> few years back, but it's mostly yet more boot spam. I can tell the kernel to 
> be quiet during boot, but never figured out the equivalent for qemu-system...
>
>

Re: or1k -M virt -hda and net.

[Alex Bennée]

Rob Landley <rob@landley.net> writes:

> On 1/7/25 12:05, Alex Bennée wrote:
>> Stafford Horne <shorne@gmail.com> writes:
>>> I have not used -hda before, do you have it working with other targets?
>>>
>>> According to the qemu docs in qemu-options.hx. I see:
>>>
>>>      Use file as hard disk 0, 1, 2 or 3 image on the default bus of the
>>>      emulated machine (this is for example the IDE bus on most x86 machines,
>>>      but it can also be SCSI, virtio or something else on other target
>>>      architectures). See also the :ref:`disk images` chapter in the System
>>>      Emulation Users Guide.
>>>
>>> I think, since we don't have a "default" bus in openrisc this doesn't work so we
>>> need to specify the -drive explictly.

Well if you want a simple drive command you need something. For example
on -M virt for aarch64:

  -drive  driver=raw,file.driver=host_device,file.filename=/dev/zen-ssd2/trixie-arm64,discard=unmap

only really contains backend options. By default this will attach the
block device to the virtio-pci bus, see virt.c:

  mc->block_default_type = IF_VIRTIO;

The backend options might look a bit much, a simpler case with qcow2
would be:

  -drive driver=qcow2,file=trixie-x86_64.qcow2 

However if you don't have any default bus for your block devices you
must use -device/-blockdev pairs. It doesn't add much:

 -device virtio-scsi-pci \
 -device scsi-hd,drive=hd \
 -blockdev driver=raw,node-name=hd,file.driver=host_device,file.filename=/dev/zen-ssd2/trixie-arm64,discard=unmap \

So all I've added is the bus, a device and then linked them with the
drive/node-name ids.

>>>
>>> I checked the x86 machine code and confirm it seems to work like this.  There is
>>> code in the system setup to look for hd* drives and wire them into IDE.  There
>>> is no such code in openrisc.
>> Yeah don't use -hdX as they are legacy options with a lot of default
>> assumptions. As the docs say: https://qemu.readthedocs.io/en/master/system/invocation.html#hxtool-1
>>    The QEMU block device handling options have a long history and
>> have
>>    gone through several iterations as the feature set and complexity of
>>    the block layer have grown. Many online guides to QEMU often reference
>>    older and deprecated options, which can lead to confusion.
>
> I want "a block device from this file" in a generic way that works the
> same across multiple architectures regardless of the board being
> emulated, where I only have to specify the file not explicitly
> micromanage bus plumbing details, and which is easy for a human to
> type from when explained over a voice call.

You shouldn't need to micro manage bus details, you just need to link
the device to the backend via an id.

> What's the alternative to -hda you suggest for that?
>
> Can I do "./run-qemu.sh -drive file=blah.img" without the rest?
> Perhaps specify all the details in the script and then optionally add
> an extra argument at the end? I couldn't get that to work:
>
> $ root/or1k/run-qemu.sh -netdev user,id=net0 -device
> virtio-net-device,netdev=net0 -drive format=raw,id=hd0 -device
> virtio-blk-device,drive=hd0 -drive file=README
> qemu-system-or1k: -drive format=raw,id=hd0: A block device must be
> specified for "file"
>
> Also, if you say -device and -drive but do NOT specify a file, qemu
> refuses to start. So I can't set the defaults but only optionally use
> them, the way -hda has defaults built into the image that don't cause
> a problem if I DON'T add a -hda argument to the command line.

device and blockdev pairs are required. -drive attempts to do both in
one command line option.

>>    Older options like -hda are essentially macros which expand into -drive
>>    options for various drive interfaces.
>
> Where the knowledge of "what this board needs in order to do that" is
> built into qemu rather than provided by the caller, yes.
>
>> The original forms bake in a lot
>>    of assumptions from the days when QEMU was emulating a legacy PC, they
>>    are not recommended for modern configurations.
>
> I'm building a kernel. It finds /dev/?da so I can mount it. That is my
> desired outcome.
>
> I am attempting to get generic behavior out of multiple architectures,
> among other reasons so I can cross-test and package up "it fails on X,
> here's a build and test" to point package maintainers at.

We support a wide variety of boards some with fixed block device buses
and some with the ability to add stuff dynamically. While we certainly
could do better documenting the edge cases (patches welcome ;-) I'm not
sure its possible to come up with a generic command line that works
across all boards. That said any of the VirtIO enabled platforms (often
called virt) will have fairly similar command lines for adding devices
(modulo PCI/MMIO support).

> "It natively builds under the emulator" is the easiest way to make
> that work, which is why https://landley.net/bin/toolchains/latest/ has
> a native.sqf for each cross.tar.xz.
>
> wget system-image-arch.txz
> wget toolchain.sqf
> wget test.img
>
> ./run-emulator.sh -hda test.img -hdb toolchain.sqf
>
> If I have to explain "-drive virtio-potato-walrus,inkpot=striated
> -device collect=striated,burbank-potato,ireland" at somebody whose
> domain expertise is xfce or something, the barrier to getting them to
> reproduce the issue I'm seeing is noticeably higher. If I have to MAKE
> a bespoke wrapper shell script for them with every bug report, the
> likelihood that it works differently for them than when I tried it is
> noticeably nonzero, and the likelihood of the issue going on my todo
> heap and never getting pursued upstream is also noticeably higher.
> Which is why I try to make generic tools...

Just put it in a script then. 

> (Making a _test_ script to demonstrate the issue is normal. If it's
> their project, usually they can tell if I typoed it and fix it up
> themselves because they know what I MEANT. But if I typo the setup for
> the virtual environment, or are missing a prerequisite package
> install, or they hit qemu version skew, or I said /bin/sh and theirs
> points to dash... Brick wall. It either works or it doesn't.)
>
> (And when I have to set up the long version for a nightly cron job,
> and then when the test fails 6 months later and I look at it and go
> "huh? salad?" that's a bad 3am digression as well. And which is more
> likely to break from version skew during qemu version upgrades: two
> lines of micromanaging --longopts or -hda that gets adjusted by the
> maintainers?)

QEMU's command line reputation is not undeserved but it is at least
consistent with the modern composable options. If we can improve the
documentation then let us know:

  https://qemu.readthedocs.io/en/master/system/device-emulation.html

But expanding the use of automagical options is not really a long term
solution.

> Rob
>
> P.S. For some reason -hda grew an "I'm going to make the first block
> read-only just like loopback devices do because you can't be trusted"
> nag a few years back, but it's mostly yet more boot spam. I can tell
> the kernel to be quiet during boot, but never figured out the
> equivalent for qemu-system...

-append passes options to the kernel command line if you are doing a
direct kernel boot or your firmware supports direct kernel booting.

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: or1k -M virt -hda and net.

[Rob Landley]

[-- Attachment #1: Type: text/plain, Size: 2666 bytes --]

On 1/8/25 02:24, Geert Uytterhoeven wrote:
> Hi Rob,
> 
> On Tue, Jan 7, 2025 at 11:53 PM Rob Landley <rob@landley.net> wrote:
>> Microblaze has no /dev/?da (as in there's no device I could attach it to
>> even with the long option, I'd have to use NBD), sh4eb has -hda is
>> working but it's having some endianness hiccup with the network card
>> (works in sh4 but not sh4eb, I'm trying to track it down, I thought this
>> worked at one point), and the two "No kernel" ones legitimately have no
> 
> That's using sh_eth, right?

8139cp: 8139cp: 10/100 PCI Ethernet driver v1.3 (Mar 22, 2004)
8139cp 0000:00:02.0: enabling device (0000 -> 0003)
8139cp 0000:00:02.0 eth0: RTL-8139C+ at 0x(ptrval), 52:54:00:12:34:56, 
IRQ 128

Is there a different driver I should use? That one works on sh4 little 
endian with almost the same config.

> Last time I tried booting a big endian ARM kernel on Koelsch (R-Car
> M2-W), it booted fine until "unable to mount root" (I have no suitable
> userspace), except for broken Ethernet, so it failed to obtain a
> DHCP lease.

Miniconfig I'm using for 6.19 attached.

If you don't want to build mkroot's toybox+musl userspace yourself, the 
initramfs.cpio.gz in the last release from 
https://landley.net/bin/mkroot/0.8.11/sh4eb.tgz presumably still works, 
using the instructions in run-qemu.sh. (I'm trying to get a new release 
out, hence poking at targets that don't quite pass muster.)

You can also use the sh2eb fdpic userspace from the same directory, but 
you'll need the "fdpic with mmu for sh" patch from 
https://landley.net/bin/mkroot/0.8.11/linux-patches/0002-sh4-fdpic.patch 
to enable the fdpic loader in your kernel config. (That's the current sh 
big endian board I know people deploying today, but qemu doesn't emulate 
it directly, so I run its userspace under qemu-system-sh4eb...)

I think I've built a working static busybox defconfig with 
https://landley.net/bin/toolchains/25-03-2024/sh4eb-linux-musl-cross.tar.xz 
too, although not recently. Might need to switch a command or two off?

The boot says "8139cp 0000:00:02.0 eth0: link down" and then pauses for 
3 seconds while ntp fails to talk to time.google.com to set the clock 
newer than 1970 (sadly no battery backed up clock in the board qemu's 
emulating, there's config entries for one in the relevant kernel 
defconfig but nothing in qemu last I checked), and then I get a shell 
prompt that thinks it's 1970 and can't wget or scp with dropbear.

Rob

P.S: If you "./run-emulator.sh -hda file.img" the init script will try 
to mount /dev/?da on /mnt for you, and then run /mnt/init if it exists 
instead of a shell prompt. Easy automated testing.

[-- Attachment #2: linux-miniconfig --]
[-- Type: text/plain, Size: 1380 bytes --]

# make ARCH=sh allnoconfig KCONFIG_ALLCONFIG=linux-miniconfig
# make ARCH=sh -j $(nproc)
# boot zImage console=ttySC1 noiotrap

# architecture independent
CONFIG_PANIC_TIMEOUT=1
CONFIG_NO_HZ_IDLE=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_RD_GZIP=y
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_SCRIPT=y
CONFIG_BLK_DEV=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_EXT4_FS=y
CONFIG_EXT4_USE_FOR_EXT2=y
CONFIG_VFAT_FS=y
CONFIG_FAT_DEFAULT_UTF8=y
CONFIG_MISC_FILESYSTEMS=y
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ISO8859_1=y
CONFIG_SQUASHFS=y
CONFIG_SQUASHFS_XATTR=y
CONFIG_SQUASHFS_ZLIB=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_NET=y
CONFIG_NETDEVICES=y
CONFIG_NET_CORE=y
CONFIG_NETCONSOLE=y
CONFIG_PACKET=y
CONFIG_UNIX=y
CONFIG_INET=y
CONFIG_IPV6=y
CONFIG_ETHERNET=y
CONFIG_COMPAT_32BIT_TIME=y
CONFIG_EARLY_PRINTK=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y

# architecture specific
CONFIG_CPU_SUBTYPE_SH7751R=y
CONFIG_MMU=y
CONFIG_VSYSCALL=y
CONFIG_SH_FPU=y
CONFIG_SH_RTS7751R2D=y
CONFIG_PCI=y
CONFIG_RTS7751R2D_PLUS=y
CONFIG_SERIAL_SH_SCI=y
CONFIG_SERIAL_SH_SCI_CONSOLE=y
CONFIG_NET_VENDOR_REALTEK=y
CONFIG_8139CP=y
CONFIG_BLK_DEV_SD=y
CONFIG_ATA=y
CONFIG_ATA_SFF=y
CONFIG_ATA_BMDMA=y
CONFIG_PATA_PLATFORM=y
CONFIG_BINFMT_ELF_FDPIC=y
CONFIG_CMDLINE_FROM_BOOTLOADER=y
CONFIG_MEMORY_START=0x0c000000
CONFIG_CPU_BIG_ENDIAN=y

# architecture extra

Re: or1k -M virt -hda and net.

[Geert Uytterhoeven]

Hi Rob,

On Wed, Jan 8, 2025 at 5:23 PM Rob Landley <rob@landley.net> wrote:
> On 1/8/25 02:24, Geert Uytterhoeven wrote:
> > On Tue, Jan 7, 2025 at 11:53 PM Rob Landley <rob@landley.net> wrote:
> >> Microblaze has no /dev/?da (as in there's no device I could attach it to
> >> even with the long option, I'd have to use NBD), sh4eb has -hda is
> >> working but it's having some endianness hiccup with the network card
> >> (works in sh4 but not sh4eb, I'm trying to track it down, I thought this
> >> worked at one point), and the two "No kernel" ones legitimately have no
> >
> > That's using sh_eth, right?
>
> 8139cp: 8139cp: 10/100 PCI Ethernet driver v1.3 (Mar 22, 2004)
> 8139cp 0000:00:02.0: enabling device (0000 -> 0003)
> 8139cp 0000:00:02.0 eth0: RTL-8139C+ at 0x(ptrval), 52:54:00:12:34:56,
> IRQ 128
>
> Is there a different driver I should use? That one works on sh4 little
> endian with almost the same config.

Oh, you're using an SH4 SoC without internal Ethernet (sh7751 in landisk
or rts7751r2d, I guess?). That indeed doesn't use sh_eth.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: or1k -M virt -hda and net.

[Rob Landley]

On 1/8/25 08:59, Alex Bennée wrote:
> Rob Landley <rob@landley.net> writes:
> 
>> On 1/7/25 12:05, Alex Bennée wrote:
>>> Stafford Horne <shorne@gmail.com> writes:
>>>> I have not used -hda before, do you have it working with other targets?
>>>>
>>>> According to the qemu docs in qemu-options.hx. I see:
>>>>
>>>>       Use file as hard disk 0, 1, 2 or 3 image on the default bus of the
>>>>       emulated machine (this is for example the IDE bus on most x86 machines,
>>>>       but it can also be SCSI, virtio or something else on other target
>>>>       architectures). See also the :ref:`disk images` chapter in the System
>>>>       Emulation Users Guide.
>>>>
>>>> I think, since we don't have a "default" bus in openrisc this doesn't work so we
>>>> need to specify the -drive explictly.
> 
> Well if you want a simple drive command you need something. For example
> on -M virt for aarch64:
> 
>    -drive  driver=raw,file.driver=host_device,file.filename=/dev/zen-ssd2/trixie-arm64,discard=unmap

echo "We've replaced the dilithium they normally use with Folger's 
Crystals." > file.img
qemu-system-aarch64 -M virt -cpu cortex-a57 -m 256 -nographic \
   -no-reboot -kernel linux-kernel -initrd initramfs.cpio.gz \
   -append "HOST=aarch64 console=ttyAMA0" -hda README
...
Type exit when done.
$ cat /dev/vda
We've replaced the dilithium they normally use with Folger's Crystals.
$ exit
reboot: Restarting system

Seems like a sane default is already there. (And on most other boards.)

I also note that 90% of the above qemu invocation is the same for all 
targets. My build system's entire configuration for the m68k 
architecture is just:

>   elif [ "$CROSS" == m68k ]; then
>     QEMU_M=q800 KARCH=m68k
>     KCONF="$(be2csv MMU M68040 M68KFPU_EMU MAC BLK_DEV_SD MACINTOSH_DRIVERS \
>       NET_VENDOR_NATSEMI MACSONIC SCSI{,_LOWLEVEL,_MAC_ESP} \
>       SERIAL_PMACZILOG{,_TTYS,_CONSOLE})"

And here's the or1k target under discussion:

>   elif [ "$CROSS" == or1k ]; then
>     KARCH=openrisc QEMU_M=virt KARGS=ttyS0
>     KCONF="$(be2csv ETHOC SERIO SERIAL_OF_PLATFORM SERIAL_8250{,_CONSOLE} \
>       VIRTIO_{MENU,NET,BLK,PCI,MMIO} POWER_RESET{,_SYSCON{,_POWEROFF}} SYSCON_REBOOT_MODE)"

There are some targets I have to poke harder, armv5l and armv4tl have
"qemu-system-arm -M versatilepb -net nic,model=rtl8139 -net user" for 
some reason... Huh, apparently I've been doing that since 2007?

https://github.com/landley/aboriginal/commit/5a51e551568a

Why did I do that... https://landley.net/notes-2007.html#04-07-2007 says 
"switch to using the rtl8139 driver because PIO doesn't work on
the qemu-system-arm PCI controller yet so I need something with mmio." 
Maybe that's fixed by now and I can go back to the default network card 
there? Hmmm, hw/arm/versatilepb.c says the default is smc91c111, the 
kernel driver for that is CONFIG_SMC91X... which needs GPIOLIB for some 
reason (why is that a depends not a selects?) And yes, the board's 
default network card works if I flip on the right driver and remove the 
--micromanagement.

Ok, fixed: https://github.com/landley/toybox/commit/65887c2f3cd8

But the point was it _can_ do that. Just seldom needs to. I was asking 
for a "that" prefix which let me use -hda on or1k (and still worked when 
I didn't supply one). I don't mind supplementing or overriding an 
unworkable default in the wrapper script, I've done it before. But what 
I was asking for was something to make -hda work when $@ expanded to it.

> only really contains backend options. By default this will attach the
> block device to the virtio-pci bus, see virt.c:
> 
>    mc->block_default_type = IF_VIRTIO;
> 
> The backend options might look a bit much, a simpler case with qcow2
> would be:
> 
>    -drive driver=qcow2,file=trixie-x86_64.qcow2

A raw block device can be loopback mounted on the host and mounted 
within qemu in the same way. (Qemu doesn't even mind if they're sparse, 
although loopback still did last I checked.)

The block device isn't even always a filesystem, sometimes it's a 
tarball. And when it is a filesystem, half the time it's squashfs. Why 
wrap it in a ubifs-style block remapping layer? Extra step, more tools, 
can't look at the data in hexedit (which comes up)... It may be nice to 
have a more complicated option, but "raw image" is the generic version.

> However if you don't have any default bus for your block devices you
> must use -device/-blockdev pairs. It doesn't add much:
> 
>   -device virtio-scsi-pci \
>   -device scsi-hd,drive=hd \
>   -blockdev driver=raw,node-name=hd,file.driver=host_device,file.filename=/dev/zen-ssd2/trixie-arm64,discard=unmap \

Where "you don't" means "the board doesn't"? So the user of the board 
has to manually tell the board emulation what bus type it has available? 
(And guess right, because I can't add arbitrary types, the board will 
only ACCEPT certain ones...)

> So all I've added is the bus, a device and then linked them with the
> drive/node-name ids.

With hda I'm saying "here's the filename for your block device", and I'm 
usually externally supplying it to an existing board setup. When 
comparing the m68k behavior with the arm behavior of the same piece of 
software, I don't need to tell each system how to be itself. (It already 
knows.)

With your suggested API, specifying the filename as its own argument 
separate from the rest doesn't seem to have occurred to the designers.

If I don't provide an -hda then there isn't one. Cleanly drops out. If I 
specify -device without a filename, qemu tends to get upset and refuse 
to start.

You jumped into the thread to encouraging a move from -hda to -drive. In 
what way is -drive the superior option from a user perspective?

>>>> I checked the x86 machine code and confirm it seems to work like this.  There is
>>>> code in the system setup to look for hd* drives and wire them into IDE.  There
>>>> is no such code in openrisc.
>>> Yeah don't use -hdX as they are legacy options with a lot of default
>>> assumptions. As the docs say: https://qemu.readthedocs.io/en/master/system/invocation.html#hxtool-1
>>>     The QEMU block device handling options have a long history and
>>> have
>>>     gone through several iterations as the feature set and complexity of
>>>     the block layer have grown. Many online guides to QEMU often reference
>>>     older and deprecated options, which can lead to confusion.
>>
>> I want "a block device from this file" in a generic way that works the
>> same across multiple architectures regardless of the board being
>> emulated, where I only have to specify the file not explicitly
>> micromanage bus plumbing details, and which is easy for a human to
>> type from when explained over a voice call.
> 
> You shouldn't need to micro manage bus details, you just need to link
> the device to the backend via an id.

By "shouldn't need to micro manage the bus details", are you saying I 
can link the device to the backend without ever having manually 
specified the bus type on the command line?

Inside the VM, my init script does something like:

   [ -e /dev/?da ] && mount /dev/?da /mnt
   [ -x /mnt/init ] && exec /mnt/init

So it doesn't care if it's sda or vda and "just works" as long as the 
device shows up in /dev. And it just works if it's _not_ there either. 
(And used to work with /dev/hda before they made everything pretend to 
be scsi.)

To trigger it, I go "./run-qemu.sh -hda filename.img" because my run 
script has "$@" in the qemu command line, so I don't need to know how it 
will be used. (This isn't the ONLY thing they can add, another common 
thing is to set up port forwarding so a server in the emulator can 
appear on a host loopback port.) It's a generic tool like a hammer. It 
works the same whether ./run-qemu.sh is in the m68k directory, sh4eb, 
powerpc, etc.

>> What's the alternative to -hda you suggest for that?
>>
>> Can I do "./run-qemu.sh -drive file=blah.img" without the rest?
>> Perhaps specify all the details in the script and then optionally add
>> an extra argument at the end? I couldn't get that to work:
>>
>> $ root/or1k/run-qemu.sh -netdev user,id=net0 -device
>> virtio-net-device,netdev=net0 -drive format=raw,id=hd0 -device
>> virtio-blk-device,drive=hd0 -drive file=README
>> qemu-system-or1k: -drive format=raw,id=hd0: A block device must be
>> specified for "file"
>>
>> Also, if you say -device and -drive but do NOT specify a file, qemu
>> refuses to start. So I can't set the defaults but only optionally use
>> them, the way -hda has defaults built into the image that don't cause
>> a problem if I DON'T add a -hda argument to the command line.
> 
> device and blockdev pairs are required.

Required when -hda doesn't work.

> -drive attempts to do both in one command line option

If I can -drive setup,setup,setup and then -hda filename.img (or not) 
later, then the details can get hidden in run-qemu.sh, which is what I 
was asking how to do earlier in this thread.

Or does -drive still require I know what filename to attach, and whether 
or not to do it? So if I wanted to support -hda command.sqf -hdb 
workspace.ext3 -hdc toolchain.iso that got intercepted and turned into
-drive by a shell script I would need... I dunno, some sort of loop?

Do you want my wrapper script to include a regex to notice one of its 
arguments was "-hda", find the argument after that and convert it into a 
-drive line, with error handling for "there wasn't another argument" or 
, and = in the filename? Plus it needs to know -hdb and -hdc and -hdd as 
well to keep parity with what was previously there...

Or do you want the users to have to externally specify the -drive line 
to the script, with target-specific bus type they didn't need to know 
before in order to attach a disk to a VM?

Which of these is the improvement that makes a move from -hda to -drive 
better for the user?

>> I am attempting to get generic behavior out of multiple architectures,
>> among other reasons so I can cross-test and package up "it fails on X,
>> here's a build and test" to point package maintainers at.
> 
> We support a wide variety of boards some with fixed block device buses
> and some with the ability to add stuff dynamically. While we certainly
> could do better documenting the edge cases (patches welcome ;-) I'm not
> sure its possible to come up with a generic command line that works
> across all boards.

I have one already. The qemu devs deprecated it for some reason, and 
made it spit warnings and break fdisk when using "raw" images. (If you 
can wipe everything _except_ the first few blocks, what exactly have you 
improved? Never understood that part...)

> That said any of the VirtIO enabled platforms (often
> called virt) will have fairly similar command lines for adding devices
> (modulo PCI/MMIO support).

The test I ran upthread said that aarch64, armv7l, riscv32, riscv64, and 
s390x were all using /dev/vda for -hda.

And armv4l, armv5l, i486, i686, m68k, mips, mips64, mipsel, powerpc, 
powerpc64, powerpc64le, sh4, sh4eb, and x86_64 weren't.

And the caller didn't need to know the difference.

>> If I have to explain "-drive virtio-potato-walrus,inkpot=striated
>> -device collect=striated,burbank-potato,ireland" at somebody whose
>> domain expertise is xfce or something, the barrier to getting them to
>> reproduce the issue I'm seeing is noticeably higher. If I have to MAKE
>> a bespoke wrapper shell script for them with every bug report, the
>> likelihood that it works differently for them than when I tried it is
>> noticeably nonzero, and the likelihood of the issue going on my todo
>> heap and never getting pursued upstream is also noticeably higher.
>> Which is why I try to make generic tools...
> 
> Just put it in a script then.

Which is why my question upthread was if there was a way to use the 
micromanaging --longopts to tell qemu what plumbing -hda (and -hdb, etc) 
should attach to, without actually requiring there to _be_ one.

The answer seems to be "no, it needs a patch and rebuild".

If I have to agglutinate a supplied filename into a -drive 
thing,thing,thing list, how do you suggest I do that without #!/bin/bash 
having to be present because I wrote shell plumbing to make elaborate 
command line option parsing decisions that were never tested on "fish" 
or "zsh" or "dash" or busybox ash or...

Keep in mind the current shell plumbing is literally "$@" from posix.

>> (And when I have to set up the long version for a nightly cron job,
>> and then when the test fails 6 months later and I look at it and go
>> "huh? salad?" that's a bad 3am digression as well. And which is more
>> likely to break from version skew during qemu version upgrades: two
>> lines of micromanaging --longopts or -hda that gets adjusted by the
>> maintainers?)
> 
> QEMU's command line reputation is not undeserved but it is at least
> consistent with the modern composable options. If we can improve the
> documentation then let us know:
> 
>    https://qemu.readthedocs.io/en/master/system/device-emulation.html
> 
> But expanding the use of automagical options is not really a long term
> solution.

Do you plan to replace "-m 256" with a longopt that requires you to 
specify the physical address range of the new memory using multiple 
name=value comma separated arguments?

Or is "-m 256" a useful control knob indicating a thing a user wants to 
have happen without needing to know all the board-specific 
implementation details to invoke it?

>> Rob
>>
>> P.S. For some reason -hda grew an "I'm going to make the first block
>> read-only just like loopback devices do because you can't be trusted"
>> nag a few years back, but it's mostly yet more boot spam. I can tell
>> the kernel to be quiet during boot, but never figured out the
>> equivalent for qemu-system...
> 
> -append passes options to the kernel command line if you are doing a
> direct kernel boot or your firmware supports direct kernel booting.

I asked why append doesn't 8 years ago:

https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg03127.html

I'm not sure how your response relates to "-hda complains about raw 
images". The commit in question that broke it (38f3ef574b48) preserved 
the probe for image types, instead of having -hda just default to "raw" 
always, and requiring the --longopt version to specify the types. That's 
the part I never understood, THAT seemed a really weird decision. "The 
qcow image format is insecure, because qcow can always #include 
arbitrary external files so if you run qemu as root it can read 
/etc/shadow which is apparently a thing people do". There is indeed a 
problem, but it's not raw?

For that matter, why not do what qemu did for:

-m [size=]megs[,slots=n,maxmem=size]

You _can_ have a short option do a sane thing by default but add 
_optional_ compostable csv nonsense to it, where "in order to use an 
image file with an = or , in it you need -hda name=thing,thing and name= 
will always eat the rest of its argument", possibly with an informative 
error message when -hda file,name or -hda file=name doesn't understand 
what it was given.

The UI decisions here struck me as very strange, and deprecating the way 
-hda has worked since 2005 seemed odd to me. (Possibly earlier but 
https://landley.net/notes-2005.html#02-12-2005 is when I first got an 
-hda of a partitioned disk image with LILO installed on it to boot under 
qemu. That was before even aboriginal linux, that was 
https://landley.net/aboriginal/old/ and most of the progress reports 
about _that_ were on livejournal. I switched from User Mode Linux _to_ 
qemu.)

Rob

Re: or1k -M virt -hda and net.

[Rob Landley]

On 1/8/25 10:26, Geert Uytterhoeven wrote:
> Hi Rob,
> 
> On Wed, Jan 8, 2025 at 5:23 PM Rob Landley <rob@landley.net> wrote:
>> On 1/8/25 02:24, Geert Uytterhoeven wrote:
>>> On Tue, Jan 7, 2025 at 11:53 PM Rob Landley <rob@landley.net> wrote:
>>>> Microblaze has no /dev/?da (as in there's no device I could attach it to
>>>> even with the long option, I'd have to use NBD), sh4eb has -hda is
>>>> working but it's having some endianness hiccup with the network card
>>>> (works in sh4 but not sh4eb, I'm trying to track it down, I thought this
>>>> worked at one point), and the two "No kernel" ones legitimately have no
>>>
>>> That's using sh_eth, right?
>>
>> 8139cp: 8139cp: 10/100 PCI Ethernet driver v1.3 (Mar 22, 2004)
>> 8139cp 0000:00:02.0: enabling device (0000 -> 0003)
>> 8139cp 0000:00:02.0 eth0: RTL-8139C+ at 0x(ptrval), 52:54:00:12:34:56,
>> IRQ 128
>>
>> Is there a different driver I should use? That one works on sh4 little
>> endian with almost the same config.
> 
> Oh, you're using an SH4 SoC without internal Ethernet (sh7751 in landisk
> or rts7751r2d, I guess?). That indeed doesn't use sh_eth.

That's the driver used by qemu-system-sh4eb (I.E. -M r2d -cpu sh7751r) 
which has the advantage that it's _really_ easy to test. (Nightly cron 
job even, when I can get a proper server set up to run that...)

The sh2eb turtle board uses JCORE_EMAC which is still an out of tree 
driver. It's not secret, just too ugly to go upstream: done by a 
contractor and never cleaned up, full of half-finished IEEE-1588 support 
and so on. It used to be on 
https://web.archive.org/web/20200812035510/http://git.musl-libc.org/cgit/linux-sh/ 
but alas Rich purged his trees when he stopped being maintainer and 
archive.org didn't crawl that far down.

I have the patch locally if you care, but without the corresponding 
hardware, not much point attaching 1600 lines. (We sent several turtle 
boards to Glaubitz and his assistants last year, and at least one of the 
recipients asked for a copy of the ethernet driver patch, which I sent. 
Still applies and works as of the last kernel I updated on the board, 
6.18 I think? The update is a "pop the sd card, stick in laptop, copy 
files, move sd card back" kinda thing. I should do a linux image that 
can wget and kexec, but haven't yet... Anyway, doesn't help with this 
issue.)

All my other superh boards are A) little endian, B) still in storage 
from the move last may.

Rob

Re: or1k -M virt -hda and net.

[Rob Landley]

On 1/8/25 07:01, BALATON Zoltan wrote:
> On Tue, 7 Jan 2025, Rob Landley wrote:
>> What's the alternative to -hda you suggest for that?
>>
>> Can I do "./run-qemu.sh -drive file=blah.img" without the rest? 
>> Perhaps specify all the details in the script and then optionally add 
>> an extra argument at the end? I couldn't get that to work:
>>
>> $ root/or1k/run-qemu.sh -netdev user,id=net0 -device virtio-net- 
>> device,netdev=net0 -drive format=raw,id=hd0 -device virtio-blk- 
>> device,drive=hd0 -drive file=README
> 
> You need '-drive if=none,id=hd0,format=raw,file=README' as a single 
> option not split into two.

I'm not always specifying an -hda. Sometimes it does, and sometimes it 
runs without it. I would like to have everything EXCEPT the media 
specified, so it can be inserted into a ready drive or run without it.

That's what -hda traditionally does.

> With if=none -drive won't auto-create a  device

$ root/or1k/run-qemu.sh -netdev user,id=net0 -device 
virtio-net-device,netdev=net0 -drive if=none,id=hd0,format=raw
qemu-system-or1k: -drive if=none,id=hd0,format=raw: A block device must 
be specified for "file"

> so you then also need a corresponding -device option for the 
> drive that you seem to have already above.

$ root/or1k/run-qemu.sh -netdev user,id=net0 -device 
virtio-net-device,netdev=net0 -device virtio-blk-device
qemu-system-or1k: -device virtio-blk-device: drive property not set
$ root/or1k/run-qemu.sh -netdev user,id=net0 -device 
virtio-net-device,netdev=net0 -device virtio-blk-device,drive=hd0
qemu-system-or1k: -device virtio-blk-device,drive=hd0: Property 
'virtio-blk-device.drive' can't find value 'hd0'
$ root/or1k/run-qemu.sh -netdev user,id=net0 -device 
virtio-net-device,netdev=net0 -device virtio-blk-device,drive=hd0 -drive 
id=hd0,if=none
qemu-system-or1k: -device virtio-blk-device,drive=hd0: Device needs 
media, but drive is empty

That's as close as I can get. As far as I can tell, it's complaining 
that I got it into the state I wanted, and it doesn't want to be in that 
state. The "if=none" does not appear to help.

I also don't know what drive=/id= pair "-hda" would be trying to 
populate, so dunno what name to use there.

Also, it requires -drive and its argument to be seperate:

   qemu-system-or1k: -device=virtio-blk-device,drive=hda: invalid option

Refusing to let them to be glued together with = (like most --longopt 
parsers do, including the 
https://github.com/landley/toybox/blob/master/lib/args.c I wrote) means 
I can't use bash's brace expansion to set up multiple at once (to 
preload support for -hda, -hdb, -hdc, and -hdd), ala:

   -drive=id=hd{0,1,2,3},if=none

Not that the current script has a bash dependency... :P

> If you want -hda to work you 
> may need something like commit d36b2f4e78 (hw/ppc/sam460ex: Support 
> short options for adding drives) for the machine you use. In particular 
> the MachineClass block_default_type field says what's the default 
> interface that -drive and other short options should use (at least I 
> think so, I'm no expert on this either but searching for it should at 
> least point to where it's handled).

Thanks for the pointer, I'll take a look.

Rob

Re: or1k -M virt -hda and net.

[BALATON Zoltan]

On Wed, 8 Jan 2025, Rob Landley wrote:
> On 1/8/25 07:01, BALATON Zoltan wrote:
>> On Tue, 7 Jan 2025, Rob Landley wrote:
>>> What's the alternative to -hda you suggest for that?
>>> 
>>> Can I do "./run-qemu.sh -drive file=blah.img" without the rest? Perhaps 
>>> specify all the details in the script and then optionally add an extra 
>>> argument at the end? I couldn't get that to work:
>>> 
>>> $ root/or1k/run-qemu.sh -netdev user,id=net0 -device virtio-net- 
>>> device,netdev=net0 -drive format=raw,id=hd0 -device virtio-blk- 
>>> device,drive=hd0 -drive file=README
>> 
>> You need '-drive if=none,id=hd0,format=raw,file=README' as a single option 
>> not split into two.
>
> I'm not always specifying an -hda. Sometimes it does, and sometimes it runs 
> without it. I would like to have everything EXCEPT the media specified, so it 
> can be inserted into a ready drive or run without it.
>
> That's what -hda traditionally does.

I think you can't do that with -drive. You either do

-drive if=none,id=d,format=raw,file=raw.img -device whatever,drive=d

or try to have -drive create the device by specifying the type in if 
and hope it works such as

-drive if=virtio,format=raw,file=raw.img

and then you don't need corresponding device but this may not work for 
every machine. The -hda option should expand to such -drive option but it 
cannot if the machine does not define a default interface for block 
devices. So maybe just setting block_default_type in the MachineClass to 
IF_VIRTIO might be enough, otherwise you may need to implement handling 
for it in the machine. In the sam460ex I had to implement it because it 
uses SATA with only two ports which only takes one device per IDE bus 
while the default in QEMU expects PATA IDE that can have two devices per 
bus and two buses but for virtio maybe it works without additional 
handling of drives and only the default is missing which -hda would use.

>> With if=none -drive won't auto-create a  device
>
> $ root/or1k/run-qemu.sh -netdev user,id=net0 -device 
> virtio-net-device,netdev=net0 -drive if=none,id=hd0,format=raw
> qemu-system-or1k: -drive if=none,id=hd0,format=raw: A block device must be 
> specified for "file"
>
>> so you then also need a corresponding -device option for the drive that you 
>> seem to have already above.
>
> $ root/or1k/run-qemu.sh -netdev user,id=net0 -device 
> virtio-net-device,netdev=net0 -device virtio-blk-device
> qemu-system-or1k: -device virtio-blk-device: drive property not set
> $ root/or1k/run-qemu.sh -netdev user,id=net0 -device 
> virtio-net-device,netdev=net0 -device virtio-blk-device,drive=hd0
> qemu-system-or1k: -device virtio-blk-device,drive=hd0: Property 
> 'virtio-blk-device.drive' can't find value 'hd0'
> $ root/or1k/run-qemu.sh -netdev user,id=net0 -device 
> virtio-net-device,netdev=net0 -device virtio-blk-device,drive=hd0 -drive 
> id=hd0,if=none
> qemu-system-or1k: -device virtio-blk-device,drive=hd0: Device needs media, 
> but drive is empty
>
> That's as close as I can get. As far as I can tell, it's complaining that I 
> got it into the state I wanted, and it doesn't want to be in that state. The 
> "if=none" does not appear to help.

I think you can only have empty cdrom devices which can be created without 
a drive option but not devices that don't have removable media. You also 
can't define a drive without a file or some other backing store.

> I also don't know what drive=/id= pair "-hda" would be trying to populate, so 
> dunno what name to use there.

Maybe you can try 'info block' in QEMU monitor to see what drives are 
defined and 'info qtree' to see what devices are attached to them. But if 
-hda option does not work this won't help, you can only check on other 
machines what -hda defines.

Regards,
BALATON Zoltan

> Also, it requires -drive and its argument to be seperate:
>
>  qemu-system-or1k: -device=virtio-blk-device,drive=hda: invalid option
>
> Refusing to let them to be glued together with = (like most --longopt parsers 
> do, including the https://github.com/landley/toybox/blob/master/lib/args.c I 
> wrote) means I can't use bash's brace expansion to set up multiple at once 
> (to preload support for -hda, -hdb, -hdc, and -hdd), ala:
>
>  -drive=id=hd{0,1,2,3},if=none
>
> Not that the current script has a bash dependency... :P
>
>> If you want -hda to work you may need something like commit d36b2f4e78 
>> (hw/ppc/sam460ex: Support short options for adding drives) for the machine 
>> you use. In particular the MachineClass block_default_type field says 
>> what's the default interface that -drive and other short options should use 
>> (at least I think so, I'm no expert on this either but searching for it 
>> should at least point to where it's handled).
>
> Thanks for the pointer, I'll take a look.
>
> Rob
>
>

Re: or1k -M virt -hda and net.

[BALATON Zoltan]

[-- Attachment #1: Type: text/plain, Size: 17929 bytes --]

On Wed, 8 Jan 2025, Rob Landley wrote:
> On 1/8/25 08:59, Alex Bennée wrote:
>> Rob Landley <rob@landley.net> writes:
>> 
>>> On 1/7/25 12:05, Alex Bennée wrote:
>>>> Stafford Horne <shorne@gmail.com> writes:
>>>>> I have not used -hda before, do you have it working with other targets?
>>>>> 
>>>>> According to the qemu docs in qemu-options.hx. I see:
>>>>>
>>>>>       Use file as hard disk 0, 1, 2 or 3 image on the default bus of the
>>>>>       emulated machine (this is for example the IDE bus on most x86 
>>>>> machines,
>>>>>       but it can also be SCSI, virtio or something else on other target
>>>>>       architectures). See also the :ref:`disk images` chapter in the 
>>>>> System
>>>>>       Emulation Users Guide.
>>>>> 
>>>>> I think, since we don't have a "default" bus in openrisc this doesn't 
>>>>> work so we
>>>>> need to specify the -drive explictly.
>> 
>> Well if you want a simple drive command you need something. For example
>> on -M virt for aarch64:
>>
>>    -drive 
>> driver=raw,file.driver=host_device,file.filename=/dev/zen-ssd2/trixie-arm64,discard=unmap
>
> echo "We've replaced the dilithium they normally use with Folger's Crystals." 
>> file.img
> qemu-system-aarch64 -M virt -cpu cortex-a57 -m 256 -nographic \
>  -no-reboot -kernel linux-kernel -initrd initramfs.cpio.gz \
>  -append "HOST=aarch64 console=ttyAMA0" -hda README
> ...
> Type exit when done.
> $ cat /dev/vda
> We've replaced the dilithium they normally use with Folger's Crystals.
> $ exit
> reboot: Restarting system
>
> Seems like a sane default is already there. (And on most other boards.)
>
> I also note that 90% of the above qemu invocation is the same for all 
> targets. My build system's entire configuration for the m68k architecture is 
> just:
>
>>   elif [ "$CROSS" == m68k ]; then
>>     QEMU_M=q800 KARCH=m68k
>>     KCONF="$(be2csv MMU M68040 M68KFPU_EMU MAC BLK_DEV_SD MACINTOSH_DRIVERS 
>> \
>>       NET_VENDOR_NATSEMI MACSONIC SCSI{,_LOWLEVEL,_MAC_ESP} \
>>       SERIAL_PMACZILOG{,_TTYS,_CONSOLE})"
>
> And here's the or1k target under discussion:
>
>>   elif [ "$CROSS" == or1k ]; then
>>     KARCH=openrisc QEMU_M=virt KARGS=ttyS0
>>     KCONF="$(be2csv ETHOC SERIO SERIAL_OF_PLATFORM SERIAL_8250{,_CONSOLE} \
>>       VIRTIO_{MENU,NET,BLK,PCI,MMIO} POWER_RESET{,_SYSCON{,_POWEROFF}} 
>> SYSCON_REBOOT_MODE)"
>
> There are some targets I have to poke harder, armv5l and armv4tl have
> "qemu-system-arm -M versatilepb -net nic,model=rtl8139 -net user" for some 
> reason... Huh, apparently I've been doing that since 2007?
>
> https://github.com/landley/aboriginal/commit/5a51e551568a
>
> Why did I do that... https://landley.net/notes-2007.html#04-07-2007 says 
> "switch to using the rtl8139 driver because PIO doesn't work on
> the qemu-system-arm PCI controller yet so I need something with mmio." Maybe 
> that's fixed by now and I can go back to the default network card there? 
> Hmmm, hw/arm/versatilepb.c says the default is smc91c111, the kernel driver 
> for that is CONFIG_SMC91X... which needs GPIOLIB for some reason (why is that 
> a depends not a selects?) And yes, the board's default network card works if 
> I flip on the right driver and remove the --micromanagement.
>
> Ok, fixed: https://github.com/landley/toybox/commit/65887c2f3cd8
>
> But the point was it _can_ do that. Just seldom needs to. I was asking for a 
> "that" prefix which let me use -hda on or1k (and still worked when I didn't 
> supply one). I don't mind supplementing or overriding an unworkable default 
> in the wrapper script, I've done it before. But what I was asking for was 
> something to make -hda work when $@ expanded to it.
>
>> only really contains backend options. By default this will attach the
>> block device to the virtio-pci bus, see virt.c:
>>
>>    mc->block_default_type = IF_VIRTIO;
>> 
>> The backend options might look a bit much, a simpler case with qcow2
>> would be:
>>
>>    -drive driver=qcow2,file=trixie-x86_64.qcow2
>
> A raw block device can be loopback mounted on the host and mounted within 
> qemu in the same way. (Qemu doesn't even mind if they're sparse, although 
> loopback still did last I checked.)
>
> The block device isn't even always a filesystem, sometimes it's a tarball. 
> And when it is a filesystem, half the time it's squashfs. Why wrap it in a 
> ubifs-style block remapping layer? Extra step, more tools, can't look at the 
> data in hexedit (which comes up)... It may be nice to have a more complicated 
> option, but "raw image" is the generic version.
>
>> However if you don't have any default bus for your block devices you
>> must use -device/-blockdev pairs. It doesn't add much:
>>
>>   -device virtio-scsi-pci \
>>   -device scsi-hd,drive=hd \
>>   -blockdev 
>> driver=raw,node-name=hd,file.driver=host_device,file.filename=/dev/zen-ssd2/trixie-arm64,discard=unmap 
>> \
>
> Where "you don't" means "the board doesn't"? So the user of the board has to 
> manually tell the board emulation what bus type it has available? (And guess 
> right, because I can't add arbitrary types, the board will only ACCEPT 
> certain ones...)
>
>> So all I've added is the bus, a device and then linked them with the
>> drive/node-name ids.
>
> With hda I'm saying "here's the filename for your block device", and I'm 
> usually externally supplying it to an existing board setup. When comparing 
> the m68k behavior with the arm behavior of the same piece of software, I 
> don't need to tell each system how to be itself. (It already knows.)
>
> With your suggested API, specifying the filename as its own argument separate 
> from the rest doesn't seem to have occurred to the designers.
>
> If I don't provide an -hda then there isn't one. Cleanly drops out. If I 
> specify -device without a filename, qemu tends to get upset and refuse to 
> start.
>
> You jumped into the thread to encouraging a move from -hda to -drive. In what 
> way is -drive the superior option from a user perspective?
>
>>>>> I checked the x86 machine code and confirm it seems to work like this. 
>>>>> There is
>>>>> code in the system setup to look for hd* drives and wire them into IDE. 
>>>>> There
>>>>> is no such code in openrisc.
>>>> Yeah don't use -hdX as they are legacy options with a lot of default
>>>> assumptions. As the docs say: 
>>>> https://qemu.readthedocs.io/en/master/system/invocation.html#hxtool-1
>>>>     The QEMU block device handling options have a long history and
>>>> have
>>>>     gone through several iterations as the feature set and complexity of
>>>>     the block layer have grown. Many online guides to QEMU often 
>>>> reference
>>>>     older and deprecated options, which can lead to confusion.
>>> 
>>> I want "a block device from this file" in a generic way that works the
>>> same across multiple architectures regardless of the board being
>>> emulated, where I only have to specify the file not explicitly
>>> micromanage bus plumbing details, and which is easy for a human to
>>> type from when explained over a voice call.
>> 
>> You shouldn't need to micro manage bus details, you just need to link
>> the device to the backend via an id.
>
> By "shouldn't need to micro manage the bus details", are you saying I can 
> link the device to the backend without ever having manually specified the bus 
> type on the command line?
>
> Inside the VM, my init script does something like:
>
>  [ -e /dev/?da ] && mount /dev/?da /mnt
>  [ -x /mnt/init ] && exec /mnt/init
>
> So it doesn't care if it's sda or vda and "just works" as long as the device 
> shows up in /dev. And it just works if it's _not_ there either. (And used to 
> work with /dev/hda before they made everything pretend to be scsi.)
>
> To trigger it, I go "./run-qemu.sh -hda filename.img" because my run script 
> has "$@" in the qemu command line, so I don't need to know how it will be 
> used. (This isn't the ONLY thing they can add, another common thing is to set 
> up port forwarding so a server in the emulator can appear on a host loopback 
> port.) It's a generic tool like a hammer. It works the same whether 
> ./run-qemu.sh is in the m68k directory, sh4eb, powerpc, etc.
>
>>> What's the alternative to -hda you suggest for that?
>>> 
>>> Can I do "./run-qemu.sh -drive file=blah.img" without the rest?
>>> Perhaps specify all the details in the script and then optionally add
>>> an extra argument at the end? I couldn't get that to work:
>>> 
>>> $ root/or1k/run-qemu.sh -netdev user,id=net0 -device
>>> virtio-net-device,netdev=net0 -drive format=raw,id=hd0 -device
>>> virtio-blk-device,drive=hd0 -drive file=README
>>> qemu-system-or1k: -drive format=raw,id=hd0: A block device must be
>>> specified for "file"
>>> 
>>> Also, if you say -device and -drive but do NOT specify a file, qemu
>>> refuses to start. So I can't set the defaults but only optionally use
>>> them, the way -hda has defaults built into the image that don't cause
>>> a problem if I DON'T add a -hda argument to the command line.
>> 
>> device and blockdev pairs are required.
>
> Required when -hda doesn't work.
>
>> -drive attempts to do both in one command line option
>
> If I can -drive setup,setup,setup and then -hda filename.img (or not) later, 
> then the details can get hidden in run-qemu.sh, which is what I was asking 
> how to do earlier in this thread.
>
> Or does -drive still require I know what filename to attach, and whether or 
> not to do it? So if I wanted to support -hda command.sqf -hdb workspace.ext3 
> -hdc toolchain.iso that got intercepted and turned into
> -drive by a shell script I would need... I dunno, some sort of loop?
>
> Do you want my wrapper script to include a regex to notice one of its 
> arguments was "-hda", find the argument after that and convert it into a 
> -drive line, with error handling for "there wasn't another argument" or , and 
> = in the filename? Plus it needs to know -hdb and -hdc and -hdd as well to 
> keep parity with what was previously there...
>
> Or do you want the users to have to externally specify the -drive line to the 
> script, with target-specific bus type they didn't need to know before in 
> order to attach a disk to a VM?
>
> Which of these is the improvement that makes a move from -hda to -drive 
> better for the user?
>
>>> I am attempting to get generic behavior out of multiple architectures,
>>> among other reasons so I can cross-test and package up "it fails on X,
>>> here's a build and test" to point package maintainers at.
>> 
>> We support a wide variety of boards some with fixed block device buses
>> and some with the ability to add stuff dynamically. While we certainly
>> could do better documenting the edge cases (patches welcome ;-) I'm not
>> sure its possible to come up with a generic command line that works
>> across all boards.
>
> I have one already. The qemu devs deprecated it for some reason, and made it 
> spit warnings and break fdisk when using "raw" images. (If you can wipe 
> everything _except_ the first few blocks, what exactly have you improved? 
> Never understood that part...)
>
>> That said any of the VirtIO enabled platforms (often
>> called virt) will have fairly similar command lines for adding devices
>> (modulo PCI/MMIO support).
>
> The test I ran upthread said that aarch64, armv7l, riscv32, riscv64, and 
> s390x were all using /dev/vda for -hda.
>
> And armv4l, armv5l, i486, i686, m68k, mips, mips64, mipsel, powerpc, 
> powerpc64, powerpc64le, sh4, sh4eb, and x86_64 weren't.
>
> And the caller didn't need to know the difference.
>
>>> If I have to explain "-drive virtio-potato-walrus,inkpot=striated
>>> -device collect=striated,burbank-potato,ireland" at somebody whose
>>> domain expertise is xfce or something, the barrier to getting them to
>>> reproduce the issue I'm seeing is noticeably higher. If I have to MAKE
>>> a bespoke wrapper shell script for them with every bug report, the
>>> likelihood that it works differently for them than when I tried it is
>>> noticeably nonzero, and the likelihood of the issue going on my todo
>>> heap and never getting pursued upstream is also noticeably higher.
>>> Which is why I try to make generic tools...
>> 
>> Just put it in a script then.
>
> Which is why my question upthread was if there was a way to use the 
> micromanaging --longopts to tell qemu what plumbing -hda (and -hdb, etc) 
> should attach to, without actually requiring there to _be_ one.
>
> The answer seems to be "no, it needs a patch and rebuild".
>
> If I have to agglutinate a supplied filename into a -drive thing,thing,thing 
> list, how do you suggest I do that without #!/bin/bash having to be present 
> because I wrote shell plumbing to make elaborate command line option parsing 
> decisions that were never tested on "fish" or "zsh" or "dash" or busybox ash 
> or...

Sometimes sed can do wonders (or break completely) but maybe it can turn a 
-hda filename option into -drive if=virtio,file=filename if you can't fix 
hda for some machines.

> Keep in mind the current shell plumbing is literally "$@" from posix.
>
>>> (And when I have to set up the long version for a nightly cron job,
>>> and then when the test fails 6 months later and I look at it and go
>>> "huh? salad?" that's a bad 3am digression as well. And which is more
>>> likely to break from version skew during qemu version upgrades: two
>>> lines of micromanaging --longopts or -hda that gets adjusted by the
>>> maintainers?)
>> 
>> QEMU's command line reputation is not undeserved but it is at least
>> consistent with the modern composable options. If we can improve the
>> documentation then let us know:
>>
>>    https://qemu.readthedocs.io/en/master/system/device-emulation.html
>> 
>> But expanding the use of automagical options is not really a long term
>> solution.
>
> Do you plan to replace "-m 256" with a longopt that requires you to specify 
> the physical address range of the new memory using multiple name=value comma 
> separated arguments?
>
> Or is "-m 256" a useful control knob indicating a thing a user wants to have 
> happen without needing to know all the board-specific implementation details 
> to invoke it?
>
>>> Rob
>>> 
>>> P.S. For some reason -hda grew an "I'm going to make the first block
>>> read-only just like loopback devices do because you can't be trusted"
>>> nag a few years back, but it's mostly yet more boot spam. I can tell
>>> the kernel to be quiet during boot, but never figured out the
>>> equivalent for qemu-system...
>> 
>> -append passes options to the kernel command line if you are doing a
>> direct kernel boot or your firmware supports direct kernel booting.
>
> I asked why append doesn't 8 years ago:
>
> https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg03127.html

I think this originates from lilo which had an append option to specify 
options to append to kernel command line and then had addappend to append 
more options to that... QEMU only took the append which like in lilo 
overrides earlier instances despite what its name might suggest.

> I'm not sure how your response relates to "-hda complains about raw images". 
> The commit in question that broke it (38f3ef574b48) preserved the probe for 
> image types, instead of having -hda just default to "raw" always, and

The commit explains why this was a security issue. Making -hda only accept 
raw would also less inconvenient and would have changed how it worked 
before so I think this solution was chose to preserve as much 
functionality as possible.

> requiring the --longopt version to specify the types. That's the part I never 
> understood, THAT seemed a really weird decision. "The qcow image format is 
> insecure, because qcow can always #include arbitrary external files so if you 
> run qemu as root it can read /etc/shadow which is apparently a thing people 
> do". There is indeed a problem, but it's not raw?
>
> For that matter, why not do what qemu did for:
>
> -m [size=]megs[,slots=n,maxmem=size]
>
> You _can_ have a short option do a sane thing by default but add _optional_ 
> compostable csv nonsense to it, where "in order to use an image file with an 
> = or , in it you need -hda name=thing,thing and name= will always eat the 
> rest of its argument", possibly with an informative error message when -hda 
> file,name or -hda file=name doesn't understand what it was given.

Patches are welcome, I guess. Originally QEMU had a less options and they 
were simple. If you look at -help it still says:

usage: qemu-system-x86_64 [options] [disk_image]

'disk_image' is a raw hard disk image for IDE hard disk 0

So in the simplest form you can just pass a disk image to boot without any 
options but not sure it still works.

But later QEMU got more complex and this needed more complex options. Then 
a lot of the mess in the command line came from that it needs to serve 
both users and management apps which require low level control for 
everything and these apps took over QEMU for a while, that's how a lot of 
non-user friendly options crept in and then user friendly options were 
changed to be high level wrappers on those low level ones to avoid having 
two independent set of options for the same thing but maybe this was not 
always done and there's still room to improve. Also having those 
management apps meant less people cared about user friendly options and 
just used those apps instead.

Regards,
BALATON Zoltan

> The UI decisions here struck me as very strange, and deprecating the way -hda 
> has worked since 2005 seemed odd to me. (Possibly earlier but 
> https://landley.net/notes-2005.html#02-12-2005 is when I first got an -hda of 
> a partitioned disk image with LILO installed on it to boot under qemu. That 
> was before even aboriginal linux, that was 
> https://landley.net/aboriginal/old/ and most of the progress reports about 
> _that_ were on livejournal. I switched from User Mode Linux _to_ qemu.)
>
> Rob
>
>

Re: or1k -M virt -hda and net.

[Geert Uytterhoeven]

Hi Rob,

CC linux-sh

On Wed, Jan 8, 2025 at 11:40 PM Rob Landley <rob@landley.net> wrote:
> The sh2eb turtle board uses JCORE_EMAC which is still an out of tree
> driver. It's not secret, just too ugly to go upstream: done by a
> contractor and never cleaned up, full of half-finished IEEE-1588 support
> and so on. It used to be on
> https://web.archive.org/web/20200812035510/http://git.musl-libc.org/cgit/linux-sh/
> but alas Rich purged his trees when he stopped being maintainer and
> archive.org didn't crawl that far down.
>
> I have the patch locally if you care, but without the corresponding
> hardware, not much point attaching 1600 lines. (We sent several turtle
> boards to Glaubitz and his assistants last year, and at least one of the
> recipients asked for a copy of the ethernet driver patch, which I sent.

I still have to ask you for that driver, so yes I am interested in
the patch ;-)

> Still applies and works as of the last kernel I updated on the board,
> 6.18 I think? The update is a "pop the sd card, stick in laptop, copy
> files, move sd card back" kinda thing.

... which has been the main obstacle for me to integrate turtleboard
in my regular kernel testing workflow. There are only 24 hours in a day,
so usually there is no time left for juggling SD cards :-(

> I should do a linux image that
> can wget and kexec, but haven't yet... Anyway, doesn't help with this
> issue.)

Does kexec work on MMU-less J2?
On SH4, there was never an upstream kernel that worked with upstream
kexec-tools. The only one that works is the kexec binary from the old
landisk distro, which predates SH support in upstream kexec-tools,
and can only start a new kernel from a system that is running kernel
2.6.22...

BTW, this is seriously off-topic for openrisc and qemu, so please
reduce the CC list when continuing the linux-sh discussion..

Thanks!

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds