[Qemu-devel] qemu-system-sh4 -M r2d serial is broken.

[Qemu-devel] qemu-system-sh4 -M r2d serial is broken.

[Rob Landley]

Serial input hangs after the first character in the 4.11 kernel:

  http://www.spinics.net/lists/linux-sh/msg51183.html

Because they enabled support for a buffer size thing QEMU doesn't
emulate right:

  http://www.spinics.net/lists/linux-sh/msg51189.html

Rob

Re: [Qemu-devel] qemu-system-sh4 -M r2d serial is broken.

[Aurelien Jarno]

On 2017-05-18 11:08, Rob Landley wrote:
> Serial input hangs after the first character in the 4.11 kernel:
> 
>   http://www.spinics.net/lists/linux-sh/msg51183.html
> 
> Because they enabled support for a buffer size thing QEMU doesn't
> emulate right:
> 
>   http://www.spinics.net/lists/linux-sh/msg51189.html

Indeed the SCIF emulation in QEMU is quite limited. The problem is that
it exposes many internal states to the software (and that's the same for
the SH4 CPU in general), and that's not really compatible with quick
emulation. In that case the timer should depend on the baud rate which
we don't really emulate.

I'll try to have a look, that said my test environment is stuck with
kernel 4.8 due to the broken futex support on UP in kernel 4.9 (and
that's not QEMU specific). I'll try to build a more recent kernel with
additional patches.

Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: [Qemu-devel] qemu-system-sh4 -M r2d serial is broken.

[Rob Landley]

On 05/18/2017 02:00 PM, Aurelien Jarno wrote:
> On 2017-05-18 11:08, Rob Landley wrote:
>> Serial input hangs after the first character in the 4.11 kernel:
>>
>>   http://www.spinics.net/lists/linux-sh/msg51183.html
>>
>> Because they enabled support for a buffer size thing QEMU doesn't
>> emulate right:
>>
>>   http://www.spinics.net/lists/linux-sh/msg51189.html
> 
> Indeed the SCIF emulation in QEMU is quite limited. The problem is that
> it exposes many internal states to the software (and that's the same for
> the SH4 CPU in general), and that's not really compatible with quick
> emulation. In that case the timer should depend on the baud rate which
> we don't really emulate.
> 
> I'll try to have a look, that said my test environment is stuck with
> kernel 4.8 due to the broken futex support on UP in kernel 4.9 (and
> that's not QEMU specific). I'll try to build a more recent kernel with
> additional patches.

I thought Rich fixed that. Rich?

I'm doing a new https://github.com/landley/mkroot thing on top of Rich's
musl-cross-make toolchain, but it's still cooking and the kernels it
builds are really simple so far.

Rob

Re: [Qemu-devel] qemu-system-sh4 -M r2d serial is broken.

[Aurelien Jarno]

On 2017-05-18 17:37, Rob Landley wrote:
> 
> 
> On 05/18/2017 02:00 PM, Aurelien Jarno wrote:
> > On 2017-05-18 11:08, Rob Landley wrote:
> >> Serial input hangs after the first character in the 4.11 kernel:
> >>
> >>   http://www.spinics.net/lists/linux-sh/msg51183.html
> >>
> >> Because they enabled support for a buffer size thing QEMU doesn't
> >> emulate right:
> >>
> >>   http://www.spinics.net/lists/linux-sh/msg51189.html
> > 
> > Indeed the SCIF emulation in QEMU is quite limited. The problem is that
> > it exposes many internal states to the software (and that's the same for
> > the SH4 CPU in general), and that's not really compatible with quick
> > emulation. In that case the timer should depend on the baud rate which
> > we don't really emulate.
> > 
> > I'll try to have a look, that said my test environment is stuck with
> > kernel 4.8 due to the broken futex support on UP in kernel 4.9 (and
> > that's not QEMU specific). I'll try to build a more recent kernel with
> > additional patches.
> 
> I thought Rich fixed that. Rich?
> 

I have sent a patch already, but TTBOMK it hasn't been applied yet.

Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: [Qemu-devel] qemu-system-sh4 -M r2d serial is broken.

[Rob Landley]

On 05/18/2017 06:01 PM, Aurelien Jarno wrote:
> On 2017-05-18 17:37, Rob Landley wrote:
>> On 05/18/2017 02:00 PM, Aurelien Jarno wrote:
>>> On 2017-05-18 11:08, Rob Landley wrote:
>>>> Serial input hangs after the first character in the 4.11 kernel:
>>>>
>>>>   http://www.spinics.net/lists/linux-sh/msg51183.html
>>>>
>>>> Because they enabled support for a buffer size thing QEMU doesn't
>>>> emulate right:
>>>>
>>>>   http://www.spinics.net/lists/linux-sh/msg51189.html
>>>
>>> Indeed the SCIF emulation in QEMU is quite limited. The problem is that
>>> it exposes many internal states to the software (and that's the same for
>>> the SH4 CPU in general), and that's not really compatible with quick
>>> emulation. In that case the timer should depend on the baud rate which
>>> we don't really emulate.
>>>
>>> I'll try to have a look, that said my test environment is stuck with
>>> kernel 4.8 due to the broken futex support on UP in kernel 4.9 (and
>>> that's not QEMU specific). I'll try to build a more recent kernel with
>>> additional patches.
>>
>> I thought Rich fixed that. Rich?
> 
> I have sent a patch already, but TTBOMK it hasn't been applied yet.
> 
> Aurelien

I poked Rich about the futex patch again today, he's been buried up to
his neck in work but has to flush his bugfix queue before -rc5 so that
should get sorted this week.

Also, how do I tell the kernel to read the persistent clock on r2d? Both
CONFIG_RTC_DRV_R9701 (from r2d defconfig) and CONFIG_RTC_DRV_SH give
error messages and fail to read anything at boot time.

If you need a new test environment (simple one that doesn't use futexes
that I'm aware of) https://github.com/landley/mkroot is nearing its
first release. You'll need to follow the README instructions to build
musl-cross-make toolchains and set up the mcm symlink, but then:

  ./cross.sh sh4 ./mkroot.sh kernel
  cd output/sh4
  ./qemu-sh4.sh

Should boot you to a shell prompt. And given that the root filesystem
builder (mkroot.sh) is ~300 lines of bash and module/kernel is another
300 lines (mostly a big target-specific if/else staircase), it shouldn't
be too hard to pull apart. :)

Right now sh4 is the only target in the release list that hasn't got the
full "boots to a shell prompt and exits when you type exit, clock is set
to correct time, block device works, network card works" functionality
list. (That's all working on arm64 armv5l armv7l i486 i686 mips mipsel
powerpc s390x x86-64.)

Rob

Re: [Qemu-devel] qemu-system-sh4 -M r2d serial is broken.

[Aurelien Jarno]

On 2017-06-05 17:29, Rob Landley wrote:
> On 05/18/2017 06:01 PM, Aurelien Jarno wrote:
> > On 2017-05-18 17:37, Rob Landley wrote:
> >> On 05/18/2017 02:00 PM, Aurelien Jarno wrote:
> >>> On 2017-05-18 11:08, Rob Landley wrote:
> >>>> Serial input hangs after the first character in the 4.11 kernel:
> >>>>
> >>>>   http://www.spinics.net/lists/linux-sh/msg51183.html
> >>>>
> >>>> Because they enabled support for a buffer size thing QEMU doesn't
> >>>> emulate right:
> >>>>
> >>>>   http://www.spinics.net/lists/linux-sh/msg51189.html
> >>>
> >>> Indeed the SCIF emulation in QEMU is quite limited. The problem is that
> >>> it exposes many internal states to the software (and that's the same for
> >>> the SH4 CPU in general), and that's not really compatible with quick
> >>> emulation. In that case the timer should depend on the baud rate which
> >>> we don't really emulate.
> >>>
> >>> I'll try to have a look, that said my test environment is stuck with
> >>> kernel 4.8 due to the broken futex support on UP in kernel 4.9 (and
> >>> that's not QEMU specific). I'll try to build a more recent kernel with
> >>> additional patches.
> >>
> >> I thought Rich fixed that. Rich?
> > 
> > I have sent a patch already, but TTBOMK it hasn't been applied yet.
> > 
> > Aurelien
> 
> I poked Rich about the futex patch again today, he's been buried up to
> his neck in work but has to flush his bugfix queue before -rc5 so that
> should get sorted this week.

Thanks!

> Also, how do I tell the kernel to read the persistent clock on r2d? Both
> CONFIG_RTC_DRV_R9701 (from r2d defconfig) and CONFIG_RTC_DRV_SH give
> error messages and fail to read anything at boot time.

The R2D has such a RTC chip, but it is not emulated in QEMU. Someone has
to write it.

> If you need a new test environment (simple one that doesn't use futexes
> that I'm aware of) https://github.com/landley/mkroot is nearing its
> first release. You'll need to follow the README instructions to build
> musl-cross-make toolchains and set up the mcm symlink, but then:
> 
>   ./cross.sh sh4 ./mkroot.sh kernel
>   cd output/sh4
>   ./qemu-sh4.sh
>
> Should boot you to a shell prompt. And given that the root filesystem
> builder (mkroot.sh) is ~300 lines of bash and module/kernel is another
> 300 lines (mostly a big target-specific if/else staircase), it shouldn't
> be too hard to pull apart. :)

Ok, thanks.

> Right now sh4 is the only target in the release list that hasn't got the
> full "boots to a shell prompt and exits when you type exit, clock is set
> to correct time, block device works, network card works" functionality
> list. (That's all working on arm64 armv5l armv7l i486 i686 mips mipsel
> powerpc s390x x86-64.)

Patches to fix that are welcome.

Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: [Qemu-devel] qemu-system-sh4 -M r2d serial is broken.

[Paolo Bonzini]

On 18/05/2017 18:08, Rob Landley wrote:
> Serial input hangs after the first character in the 4.11 kernel:
> 
>   http://www.spinics.net/lists/linux-sh/msg51183.html
> 
> Because they enabled support for a buffer size thing QEMU doesn't
> emulate right:
> 
>   http://www.spinics.net/lists/linux-sh/msg51189.html

Hey,

regarding this:

> The really _fun_ thing about this is qemu broke it a couple years back
> and now if you hit ctrl-c it kills the _emulator_ rather than passing it
> through to the Linux console. (Just sh4, the rest do it right.)

Try replacing "-serial stdio" with "-serial mon:stdio".  You'll also be
able to use the QEMU monitor with "Ctrl-a c", so you can quit with
"Ctrl-a c quit <enter>".

> Very likely given how crappy the rest of its serial emulation is for
> this architecture, but ever since qemu added glib support I've stopped
> trying to understand their developers' thought processes.

I'm sure this is going to help you get support, right?

Anyway, Uli Hecht is saying SCIF "asserts DR even if the FIFO threshold
has not been reached if no data is received for 1.5 frames".  If that's
just a register and doesn't trigger an interrupt, you can compute the
bit's value dynamically based on the current clock.

This is for example how the x86 RTC chip computes the "update in
progress" bit, which is set for 220 us before the RC updates.  Doing
that with a timer would be really imprecise.

Paolo

Re: [Qemu-devel] qemu-system-sh4 -M r2d serial is broken.

[Aurelien Jarno]

On 2017-05-18 21:57, Paolo Bonzini wrote:
> Anyway, Uli Hecht is saying SCIF "asserts DR even if the FIFO threshold
> has not been reached if no data is received for 1.5 frames".  If that's
> just a register and doesn't trigger an interrupt, you can compute the
> bit's value dynamically based on the current clock.
> 
> This is for example how the x86 RTC chip computes the "update in
> progress" bit, which is set for 220 us before the RC updates.  Doing
> that with a timer would be really imprecise.

Unfortunately, the DR bit is set to 1 after 1.5 frames (so the exact
timing depends on the actual baud rate), and that also triggers an
interrupt if the RIE bit is set to 1. I haven't checked yet if the
kernel relies on the bit or the interrupt or both.

Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: [Qemu-devel] qemu-system-sh4 -M r2d serial is broken.

[Paolo Bonzini]

On 18/05/2017 22:10, Aurelien Jarno wrote:
> On 2017-05-18 21:57, Paolo Bonzini wrote:
>> Anyway, Uli Hecht is saying SCIF "asserts DR even if the FIFO threshold
>> has not been reached if no data is received for 1.5 frames".  If that's
>> just a register and doesn't trigger an interrupt, you can compute the
>> bit's value dynamically based on the current clock.
>>
>> This is for example how the x86 RTC chip computes the "update in
>> progress" bit, which is set for 220 us before the RC updates.  Doing
>> that with a timer would be really imprecise.
> 
> Unfortunately, the DR bit is set to 1 after 1.5 frames (so the exact
> timing depends on the actual baud rate), and that also triggers an
> interrupt if the RIE bit is set to 1. I haven't checked yet if the
> kernel relies on the bit or the interrupt or both.

DR only generates interrupts in asynchronous mode according to the data
sheet I found
(http://datasheet.octopart.com/DF72115D160FPV-Renesas-datasheet-11797591.pdf).

Paolo

Re: [Qemu-devel] qemu-system-sh4 -M r2d serial is broken.

[Rob Landley]

On 05/18/2017 02:57 PM, Paolo Bonzini wrote:
> 
> 
> On 18/05/2017 18:08, Rob Landley wrote:
>> Serial input hangs after the first character in the 4.11 kernel:
>>
>>   http://www.spinics.net/lists/linux-sh/msg51183.html
>>
>> Because they enabled support for a buffer size thing QEMU doesn't
>> emulate right:
>>
>>   http://www.spinics.net/lists/linux-sh/msg51189.html
> 
> Hey,
> 
> regarding this:
> 
>> The really _fun_ thing about this is qemu broke it a couple years back
>> and now if you hit ctrl-c it kills the _emulator_ rather than passing it
>> through to the Linux console. (Just sh4, the rest do it right.)
> 
> Try replacing "-serial stdio" with "-serial mon:stdio".  You'll also be
> able to use the QEMU monitor with "Ctrl-a c", so you can quit with
> "Ctrl-a c quit <enter>".

Ctrl-c now goes through. Thanks!

Exiting the linux instance says "reboot: Restarting system\nUnauthorized
access" and then hangs eating 100% cpu, but maybe I can fix that with
kernel configuration changes.

(I'm still doing a variant of
https://landley.net/aboriginal/control-images so the virtual system
needs to be able to indicate when it's done. But
https://landley.net/aboriginal/about.html is getting replaced with
https://github.com/landley/mkroot so there's some plumbing to do before
that's blocking anything...)

Thanks again,

Rob

Re: [Qemu-devel] qemu-system-sh4 -M r2d serial is broken.

[Aurelien Jarno]

On 2017-05-18 17:35, Rob Landley wrote:
> 
> 
> On 05/18/2017 02:57 PM, Paolo Bonzini wrote:
> > 
> > 
> > On 18/05/2017 18:08, Rob Landley wrote:
> >> Serial input hangs after the first character in the 4.11 kernel:
> >>
> >>   http://www.spinics.net/lists/linux-sh/msg51183.html
> >>
> >> Because they enabled support for a buffer size thing QEMU doesn't
> >> emulate right:
> >>
> >>   http://www.spinics.net/lists/linux-sh/msg51189.html
> > 
> > Hey,
> > 
> > regarding this:
> > 
> >> The really _fun_ thing about this is qemu broke it a couple years back
> >> and now if you hit ctrl-c it kills the _emulator_ rather than passing it
> >> through to the Linux console. (Just sh4, the rest do it right.)
> > 
> > Try replacing "-serial stdio" with "-serial mon:stdio".  You'll also be
> > able to use the QEMU monitor with "Ctrl-a c", so you can quit with
> > "Ctrl-a c quit <enter>".
> 
> Ctrl-c now goes through. Thanks!
> 
> Exiting the linux instance says "reboot: Restarting system\nUnauthorized
> access" and then hangs eating 100% cpu, but maybe I can fix that with
> kernel configuration changes.

I guess you mean rebooting the linux instance. This is because you do
not use a bootloader to load the kernel and the initrd. Then you might
be interested in:

  http://lists.nongnu.org/archive/html/qemu-devel/2017-05/msg03881.html

Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: [Qemu-devel] qemu-system-sh4 -M r2d serial is broken.

[Rob Landley]

On 05/18/2017 06:00 PM, Aurelien Jarno wrote:
>> Exiting the linux instance says "reboot: Restarting system\nUnauthorized
>> access" and then hangs eating 100% cpu, but maybe I can fix that with
>> kernel configuration changes.
> 
> I guess you mean rebooting the linux instance. This is because you do
> not use a bootloader to load the kernel and the initrd. Then you might
> be interested in:
> 
>   http://lists.nongnu.org/archive/html/qemu-devel/2017-05/msg03881.html

That patch worked for me.

Rob