[Qemu-devel] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Qemu-devel] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Mark Cave-Ayland]

Hi all,

Whilst trying to debug a regression, I've found that I can't
successfully execute a savevm/restart/loadvm sequence in qemu-system-ppc
-M g3beige. The loadvm command succeeds after the restart, however the
keyboard remains unresponsive making it impossible to continue from
where I left off.

Steps to reproduce:

1) Launch qemu-system-ppc as follows:

qemu-system-ppc -M g3beige -hda tmp.qcow2 -prom-env 'auto-boot?=false'

2) Switch to the monitor in order to pause the VM and save

(qemu) stop
(qemu) savevm test
(qemu) c

3) Switch back to the VGA display and confirm the keyboard works

4) Close qemu-system-ppc

5) Restart qemu-system-ppc with -loadvm options

qemu-system-ppc -M g3beige -prom-env 'auto-boot?=false' -loadvm test


After the restart the keyboard is frozen and unresponsive. Some
preliminary poking with gdb before and after seems to indicate that the
cuda_receive_packet() functions aren't being called after the restart
indicating that likely some state is missing from the savevm image and
not being restored correctly. Anyone have any ideas as to what might be
going wrong?


ATB,

Mark.

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Alexander Graf]

On 17.12.14 11:17, Mark Cave-Ayland wrote:
> Hi all,
> 
> Whilst trying to debug a regression, I've found that I can't
> successfully execute a savevm/restart/loadvm sequence in qemu-system-ppc
> -M g3beige. The loadvm command succeeds after the restart, however the
> keyboard remains unresponsive making it impossible to continue from
> where I left off.
> 
> Steps to reproduce:
> 
> 1) Launch qemu-system-ppc as follows:
> 
> qemu-system-ppc -M g3beige -hda tmp.qcow2 -prom-env 'auto-boot?=false'
> 
> 2) Switch to the monitor in order to pause the VM and save
> 
> (qemu) stop
> (qemu) savevm test
> (qemu) c
> 
> 3) Switch back to the VGA display and confirm the keyboard works
> 
> 4) Close qemu-system-ppc
> 
> 5) Restart qemu-system-ppc with -loadvm options
> 
> qemu-system-ppc -M g3beige -prom-env 'auto-boot?=false' -loadvm test
> 
> 
> After the restart the keyboard is frozen and unresponsive. Some
> preliminary poking with gdb before and after seems to indicate that the
> cuda_receive_packet() functions aren't being called after the restart
> indicating that likely some state is missing from the savevm image and
> not being restored correctly. Anyone have any ideas as to what might be
> going wrong?

I don't think anyone has test the migration code on non-papr ppc for
quite a while. In fact, I'm surprised it's only the keyboard not working :).

You're more than invited to send patches to fix it! :)


Alex

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Mark Cave-Ayland]

On 17/12/14 10:23, Alexander Graf wrote:

>> After the restart the keyboard is frozen and unresponsive. Some
>> preliminary poking with gdb before and after seems to indicate that the
>> cuda_receive_packet() functions aren't being called after the restart
>> indicating that likely some state is missing from the savevm image and
>> not being restored correctly. Anyone have any ideas as to what might be
>> going wrong?
> 
> I don't think anyone has test the migration code on non-papr ppc for
> quite a while. In fact, I'm surprised it's only the keyboard not working :).
> 
> You're more than invited to send patches to fix it! :)

How did I know that would be the response ;) I don't suppose anyone has
a vague memory of the timeframe/QEMU version when a savevm/loadvm was
last working for -M g3beige as a starting point?!


ATB,

Mark.

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Alexander Graf]

On 17.12.14 11:47, Mark Cave-Ayland wrote:
> On 17/12/14 10:23, Alexander Graf wrote:
> 
>>> After the restart the keyboard is frozen and unresponsive. Some
>>> preliminary poking with gdb before and after seems to indicate that the
>>> cuda_receive_packet() functions aren't being called after the restart
>>> indicating that likely some state is missing from the savevm image and
>>> not being restored correctly. Anyone have any ideas as to what might be
>>> going wrong?
>>
>> I don't think anyone has test the migration code on non-papr ppc for
>> quite a while. In fact, I'm surprised it's only the keyboard not working :).
>>
>> You're more than invited to send patches to fix it! :)
> 
> How did I know that would be the response ;) I don't suppose anyone has
> a vague memory of the timeframe/QEMU version when a savevm/loadvm was
> last working for -M g3beige as a starting point?!

Phew - 0.7.0 maybe? If you say that only CUDA is broken, I don't think
it'd be too hard to fix :). Check that there are VMSTATEs for everything
and maybe retrigger interrupts after migration.


Alex

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Peter Maydell]

On 17 December 2014 at 11:11, Alexander Graf <agraf@suse.de> wrote:
> Phew - 0.7.0 maybe? If you say that only CUDA is broken, I don't think
> it'd be too hard to fix :). Check that there are VMSTATEs for everything
> and maybe retrigger interrupts after migration.

It shouldn't be necessary to retrigger anything post-migration:
the devices on both ends should both believe the interrupt is
asserted once they've loaded their state.

Given how long it's been since this was known-working, it's
probably easiest just to compare the VMState structs against
the device structs for every device that might be vaguely
relevant, looking for missing fields.

-- PMM

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Mark Cave-Ayland]

On 17/12/14 11:23, Peter Maydell wrote:

> On 17 December 2014 at 11:11, Alexander Graf <agraf@suse.de> wrote:
>> Phew - 0.7.0 maybe? If you say that only CUDA is broken, I don't think
>> it'd be too hard to fix :). Check that there are VMSTATEs for everything
>> and maybe retrigger interrupts after migration.
> 
> It shouldn't be necessary to retrigger anything post-migration:
> the devices on both ends should both believe the interrupt is
> asserted once they've loaded their state.
> 
> Given how long it's been since this was known-working, it's
> probably easiest just to compare the VMState structs against
> the device structs for every device that might be vaguely
> relevant, looking for missing fields.

Trying to figure out why my CUDA breakpoints weren't being hit, it turns
out that the problem is even more fundamental than this. Check out the
output of "info mtree" before and after a "loadvm" command:

BEFORE:
(qemu) info mtree
memory
0000000000000000-ffffffffffffffff (prio 0, RW): system
  0000000000000000-0000000007ffffff (prio 0, RW): ppc_heathrow.ram
  0000000080000000-00000000fdffffff (prio 0, RW): alias pci-hole
@pci-mmio 0000000080000000-00000000fdffffff
  00000000f0000510-00000000f0000511 (prio 0, RW): fwcfg.ctl
  00000000f0000512-00000000f0000512 (prio 0, RW): fwcfg.data
  00000000fe000000-00000000fe1fffff (prio 0, RW): alias isa_mmio @io
0000000000000000-00000000001fffff
  00000000fec00000-00000000fec00fff (prio 0, RW): pci-conf-idx
  00000000fee00000-00000000fee00fff (prio 0, RW): pci-data-idx
  00000000fff00000-00000000ffffffff (prio 0, R-): ppc_heathrow.bios
I/O
0000000000000000-000000000000ffff (prio 0, RW): io
  00000000000001ce-00000000000001ce (prio 0, RW): vbe
  00000000000001d0-00000000000001d0 (prio 0, RW): vbe
  00000000000003b4-00000000000003b5 (prio 0, RW): vga
  00000000000003ba-00000000000003ba (prio 0, RW): vga
  00000000000003c0-00000000000003cf (prio 0, RW): vga
  00000000000003d4-00000000000003d5 (prio 0, RW): vga
  00000000000003da-00000000000003da (prio 0, RW): vga
  0000000000000400-00000000000004ff (prio 1, RW): ne2000
grackle
VGA
ne2k_pci
macio-oldworld
aliases
pci-mmio
0000000000000000-00000000ffffffff (prio 0, RW): pci-mmio
  00000000000a0000-00000000000affff (prio 2, RW): alias vga.chain4
@vga.vram 0000000000000000-000000000000ffff
  00000000000a0000-00000000000bffff (prio 1, RW): vga-lowmem
  0000000080000000-0000000080ffffff (prio 1, RW): vga.vram
  0000000081000000-0000000081000fff (prio 1, RW): vga.mmio
    0000000081000400-000000008100041f (prio 0, RW): vga ioports remapped
    0000000081000500-0000000081000515 (prio 0, RW): bochs dispi interface
    0000000081000600-0000000081000607 (prio 0, RW): qemu extended regs
  0000000081010000-000000008101ffff (prio 1, RW): vga.rom
  0000000081040000-000000008107ffff (prio 1, RW): ne2000.rom
  0000000081080000-00000000810fffff (prio 1, RW): macio
    0000000081080000-0000000081080fff (prio 0, RW): heathrow-pic
    0000000081088000-0000000081088fff (prio 0, RW): dbdma
    0000000081092000-00000000810920ff (prio 0, RW): escc-legacy
      0000000081092000-0000000081092001 (prio 0, RW): alias
escc-legacy-port @escc-bar 0000000000000000-0000000000000001
      0000000081092002-0000000081092003 (prio 0, RW): alias
escc-legacy-port @escc-bar 0000000000000020-0000000000000021
      0000000081092004-0000000081092005 (prio 0, RW): alias
escc-legacy-port @escc-bar 0000000000000010-0000000000000011
      0000000081092006-0000000081092007 (prio 0, RW): alias
escc-legacy-port @escc-bar 0000000000000030-0000000000000031
      0000000081092008-0000000081092009 (prio 0, RW): alias
escc-legacy-port @escc-bar 0000000000000040-0000000000000041
      000000008109200a-000000008109200b (prio 0, RW): alias
escc-legacy-port @escc-bar 0000000000000050-0000000000000051
      0000000081092060-0000000081092061 (prio 0, RW): alias
escc-legacy-port @escc-bar 0000000000000060-0000000000000061
      0000000081092070-0000000081092071 (prio 0, RW): alias
escc-legacy-port @escc-bar 0000000000000070-0000000000000071
      0000000081092080-0000000081092081 (prio 0, RW): alias
escc-legacy-port @escc-bar 0000000000000070-0000000000000071
      0000000081092090-0000000081092091 (prio 0, RW): alias
escc-legacy-port @escc-bar 0000000000000080-0000000000000081
      00000000810920a0-00000000810920a1 (prio 0, RW): alias
escc-legacy-port @escc-bar 0000000000000090-0000000000000091
      00000000810920b0-00000000810920b1 (prio 0, RW): alias
escc-legacy-port @escc-bar 00000000000000a0-00000000000000a1
      00000000810920c0-00000000810920c1 (prio 0, RW): alias
escc-legacy-port @escc-bar 00000000000000b0-00000000000000b1
      00000000810920d0-00000000810920d1 (prio 0, RW): alias
escc-legacy-port @escc-bar 00000000000000c0-00000000000000c1
      00000000810920e0-00000000810920e1 (prio 0, RW): alias
escc-legacy-port @escc-bar 00000000000000d0-00000000000000d1
      00000000810920f0-00000000810920f1 (prio 0, RW): alias
escc-legacy-port @escc-bar 00000000000000e0-00000000000000e1
    0000000081093000-000000008109303f (prio 0, RW): alias escc-bar @escc
0000000000000000-000000000000003f
    0000000081096000-0000000081097fff (prio 0, RW): cuda
    00000000810a0000-00000000810a0fff (prio 0, RW): pmac-ide
    00000000810a1000-00000000810a1fff (prio 0, RW): pmac-ide
    00000000810e0000-00000000810fffff (prio 0, RW): macio-nvram
io
0000000000000000-000000000000ffff (prio 0, RW): io
  00000000000001ce-00000000000001ce (prio 0, RW): vbe
  00000000000001d0-00000000000001d0 (prio 0, RW): vbe
  00000000000003b4-00000000000003b5 (prio 0, RW): vga
  00000000000003ba-00000000000003ba (prio 0, RW): vga
  00000000000003c0-00000000000003cf (prio 0, RW): vga
  00000000000003d4-00000000000003d5 (prio 0, RW): vga
  00000000000003da-00000000000003da (prio 0, RW): vga
  0000000000000400-00000000000004ff (prio 1, RW): ne2000
vga.vram
0000000080000000-0000000080ffffff (prio 1, RW): vga.vram
escc-bar
0000000000013000-000000000001303f (prio 0, RW): alias escc-bar @escc
0000000000000000-000000000000003f
escc
0000000000000000-000000000000003f (prio 0, RW): escc


AFTER:
(qemu) info mtree
memory
0000000000000000-ffffffffffffffff (prio 0, RW): system
  0000000000000000-0000000007ffffff (prio 0, RW): ppc_heathrow.ram
  0000000080000000-00000000fdffffff (prio 0, RW): alias pci-hole
@pci-mmio 0000000080000000-00000000fdffffff
  00000000f0000510-00000000f0000511 (prio 0, RW): fwcfg.ctl
  00000000f0000512-00000000f0000512 (prio 0, RW): fwcfg.data
  00000000fe000000-00000000fe1fffff (prio 0, RW): alias isa_mmio @io
0000000000000000-00000000001fffff
  00000000fec00000-00000000fec00fff (prio 0, RW): pci-conf-idx
  00000000fee00000-00000000fee00fff (prio 0, RW): pci-data-idx
  00000000fff00000-00000000ffffffff (prio 0, R-): ppc_heathrow.bios
I/O
0000000000000000-000000000000ffff (prio 0, RW): io
  00000000000001ce-00000000000001ce (prio 0, RW): vbe
  00000000000001d0-00000000000001d0 (prio 0, RW): vbe
  00000000000003b4-00000000000003b5 (prio 0, RW): vga
  00000000000003ba-00000000000003ba (prio 0, RW): vga
  00000000000003c0-00000000000003cf (prio 0, RW): vga
  00000000000003d4-00000000000003d5 (prio 0, RW): vga
  00000000000003da-00000000000003da (prio 0, RW): vga
  0000000000000400-00000000000004ff (prio 1, RW): ne2000
grackle
VGA
ne2k_pci
macio-oldworld
aliases
pci-mmio
0000000000000000-00000000ffffffff (prio 0, RW): pci-mmio
  00000000000a0000-00000000000bffff (prio 1, RW): vga-lowmem
  0000000080000000-0000000080ffffff (prio 1, RW): vga.vram
  0000000081000000-0000000081000fff (prio 1, RW): vga.mmio
    0000000081000400-000000008100041f (prio 0, RW): vga ioports remapped
    0000000081000500-0000000081000515 (prio 0, RW): bochs dispi interface
    0000000081000600-0000000081000607 (prio 0, RW): qemu extended regs
  0000000081010000-000000008101ffff (prio 1, RW): vga.rom
  0000000081040000-000000008107ffff (prio 1, RW): ne2000.rom
io
0000000000000000-000000000000ffff (prio 0, RW): io
  00000000000001ce-00000000000001ce (prio 0, RW): vbe
  00000000000001d0-00000000000001d0 (prio 0, RW): vbe
  00000000000003b4-00000000000003b5 (prio 0, RW): vga
  00000000000003ba-00000000000003ba (prio 0, RW): vga
  00000000000003c0-00000000000003cf (prio 0, RW): vga
  00000000000003d4-00000000000003d5 (prio 0, RW): vga
  00000000000003da-00000000000003da (prio 0, RW): vga
  0000000000000400-00000000000004ff (prio 1, RW): ne2000


So it looks like several of the device MemoryRegions aren't being added
after the "loadvm". Does this mean there is an object lifecycle issue
here in that some of the initialisation needs to be done in realizefn
rather than initfn or vice-versa?


ATB,

Mark.

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Alexander Graf]

On 18.12.14 14:54, Mark Cave-Ayland wrote:
> On 17/12/14 11:23, Peter Maydell wrote:
> 
>> On 17 December 2014 at 11:11, Alexander Graf <agraf@suse.de> wrote:
>>> Phew - 0.7.0 maybe? If you say that only CUDA is broken, I don't think
>>> it'd be too hard to fix :). Check that there are VMSTATEs for everything
>>> and maybe retrigger interrupts after migration.
>>
>> It shouldn't be necessary to retrigger anything post-migration:
>> the devices on both ends should both believe the interrupt is
>> asserted once they've loaded their state.
>>
>> Given how long it's been since this was known-working, it's
>> probably easiest just to compare the VMState structs against
>> the device structs for every device that might be vaguely
>> relevant, looking for missing fields.
> 
> Trying to figure out why my CUDA breakpoints weren't being hit, it turns
> out that the problem is even more fundamental than this. Check out the
> output of "info mtree" before and after a "loadvm" command:
> 
> BEFORE:
> (qemu) info mtree
> memory
> 0000000000000000-ffffffffffffffff (prio 0, RW): system
>   0000000000000000-0000000007ffffff (prio 0, RW): ppc_heathrow.ram
>   0000000080000000-00000000fdffffff (prio 0, RW): alias pci-hole
> @pci-mmio 0000000080000000-00000000fdffffff
>   00000000f0000510-00000000f0000511 (prio 0, RW): fwcfg.ctl
>   00000000f0000512-00000000f0000512 (prio 0, RW): fwcfg.data
>   00000000fe000000-00000000fe1fffff (prio 0, RW): alias isa_mmio @io
> 0000000000000000-00000000001fffff
>   00000000fec00000-00000000fec00fff (prio 0, RW): pci-conf-idx
>   00000000fee00000-00000000fee00fff (prio 0, RW): pci-data-idx
>   00000000fff00000-00000000ffffffff (prio 0, R-): ppc_heathrow.bios
> I/O
> 0000000000000000-000000000000ffff (prio 0, RW): io
>   00000000000001ce-00000000000001ce (prio 0, RW): vbe
>   00000000000001d0-00000000000001d0 (prio 0, RW): vbe
>   00000000000003b4-00000000000003b5 (prio 0, RW): vga
>   00000000000003ba-00000000000003ba (prio 0, RW): vga
>   00000000000003c0-00000000000003cf (prio 0, RW): vga
>   00000000000003d4-00000000000003d5 (prio 0, RW): vga
>   00000000000003da-00000000000003da (prio 0, RW): vga
>   0000000000000400-00000000000004ff (prio 1, RW): ne2000
> grackle
> VGA
> ne2k_pci
> macio-oldworld
> aliases
> pci-mmio
> 0000000000000000-00000000ffffffff (prio 0, RW): pci-mmio
>   00000000000a0000-00000000000affff (prio 2, RW): alias vga.chain4
> @vga.vram 0000000000000000-000000000000ffff
>   00000000000a0000-00000000000bffff (prio 1, RW): vga-lowmem
>   0000000080000000-0000000080ffffff (prio 1, RW): vga.vram
>   0000000081000000-0000000081000fff (prio 1, RW): vga.mmio
>     0000000081000400-000000008100041f (prio 0, RW): vga ioports remapped
>     0000000081000500-0000000081000515 (prio 0, RW): bochs dispi interface
>     0000000081000600-0000000081000607 (prio 0, RW): qemu extended regs
>   0000000081010000-000000008101ffff (prio 1, RW): vga.rom
>   0000000081040000-000000008107ffff (prio 1, RW): ne2000.rom
>   0000000081080000-00000000810fffff (prio 1, RW): macio
>     0000000081080000-0000000081080fff (prio 0, RW): heathrow-pic
>     0000000081088000-0000000081088fff (prio 0, RW): dbdma
>     0000000081092000-00000000810920ff (prio 0, RW): escc-legacy
>       0000000081092000-0000000081092001 (prio 0, RW): alias
> escc-legacy-port @escc-bar 0000000000000000-0000000000000001
>       0000000081092002-0000000081092003 (prio 0, RW): alias
> escc-legacy-port @escc-bar 0000000000000020-0000000000000021
>       0000000081092004-0000000081092005 (prio 0, RW): alias
> escc-legacy-port @escc-bar 0000000000000010-0000000000000011
>       0000000081092006-0000000081092007 (prio 0, RW): alias
> escc-legacy-port @escc-bar 0000000000000030-0000000000000031
>       0000000081092008-0000000081092009 (prio 0, RW): alias
> escc-legacy-port @escc-bar 0000000000000040-0000000000000041
>       000000008109200a-000000008109200b (prio 0, RW): alias
> escc-legacy-port @escc-bar 0000000000000050-0000000000000051
>       0000000081092060-0000000081092061 (prio 0, RW): alias
> escc-legacy-port @escc-bar 0000000000000060-0000000000000061
>       0000000081092070-0000000081092071 (prio 0, RW): alias
> escc-legacy-port @escc-bar 0000000000000070-0000000000000071
>       0000000081092080-0000000081092081 (prio 0, RW): alias
> escc-legacy-port @escc-bar 0000000000000070-0000000000000071
>       0000000081092090-0000000081092091 (prio 0, RW): alias
> escc-legacy-port @escc-bar 0000000000000080-0000000000000081
>       00000000810920a0-00000000810920a1 (prio 0, RW): alias
> escc-legacy-port @escc-bar 0000000000000090-0000000000000091
>       00000000810920b0-00000000810920b1 (prio 0, RW): alias
> escc-legacy-port @escc-bar 00000000000000a0-00000000000000a1
>       00000000810920c0-00000000810920c1 (prio 0, RW): alias
> escc-legacy-port @escc-bar 00000000000000b0-00000000000000b1
>       00000000810920d0-00000000810920d1 (prio 0, RW): alias
> escc-legacy-port @escc-bar 00000000000000c0-00000000000000c1
>       00000000810920e0-00000000810920e1 (prio 0, RW): alias
> escc-legacy-port @escc-bar 00000000000000d0-00000000000000d1
>       00000000810920f0-00000000810920f1 (prio 0, RW): alias
> escc-legacy-port @escc-bar 00000000000000e0-00000000000000e1
>     0000000081093000-000000008109303f (prio 0, RW): alias escc-bar @escc
> 0000000000000000-000000000000003f
>     0000000081096000-0000000081097fff (prio 0, RW): cuda
>     00000000810a0000-00000000810a0fff (prio 0, RW): pmac-ide
>     00000000810a1000-00000000810a1fff (prio 0, RW): pmac-ide
>     00000000810e0000-00000000810fffff (prio 0, RW): macio-nvram
> io
> 0000000000000000-000000000000ffff (prio 0, RW): io
>   00000000000001ce-00000000000001ce (prio 0, RW): vbe
>   00000000000001d0-00000000000001d0 (prio 0, RW): vbe
>   00000000000003b4-00000000000003b5 (prio 0, RW): vga
>   00000000000003ba-00000000000003ba (prio 0, RW): vga
>   00000000000003c0-00000000000003cf (prio 0, RW): vga
>   00000000000003d4-00000000000003d5 (prio 0, RW): vga
>   00000000000003da-00000000000003da (prio 0, RW): vga
>   0000000000000400-00000000000004ff (prio 1, RW): ne2000
> vga.vram
> 0000000080000000-0000000080ffffff (prio 1, RW): vga.vram
> escc-bar
> 0000000000013000-000000000001303f (prio 0, RW): alias escc-bar @escc
> 0000000000000000-000000000000003f
> escc
> 0000000000000000-000000000000003f (prio 0, RW): escc
> 
> 
> AFTER:
> (qemu) info mtree
> memory
> 0000000000000000-ffffffffffffffff (prio 0, RW): system
>   0000000000000000-0000000007ffffff (prio 0, RW): ppc_heathrow.ram
>   0000000080000000-00000000fdffffff (prio 0, RW): alias pci-hole
> @pci-mmio 0000000080000000-00000000fdffffff
>   00000000f0000510-00000000f0000511 (prio 0, RW): fwcfg.ctl
>   00000000f0000512-00000000f0000512 (prio 0, RW): fwcfg.data
>   00000000fe000000-00000000fe1fffff (prio 0, RW): alias isa_mmio @io
> 0000000000000000-00000000001fffff
>   00000000fec00000-00000000fec00fff (prio 0, RW): pci-conf-idx
>   00000000fee00000-00000000fee00fff (prio 0, RW): pci-data-idx
>   00000000fff00000-00000000ffffffff (prio 0, R-): ppc_heathrow.bios
> I/O
> 0000000000000000-000000000000ffff (prio 0, RW): io
>   00000000000001ce-00000000000001ce (prio 0, RW): vbe
>   00000000000001d0-00000000000001d0 (prio 0, RW): vbe
>   00000000000003b4-00000000000003b5 (prio 0, RW): vga
>   00000000000003ba-00000000000003ba (prio 0, RW): vga
>   00000000000003c0-00000000000003cf (prio 0, RW): vga
>   00000000000003d4-00000000000003d5 (prio 0, RW): vga
>   00000000000003da-00000000000003da (prio 0, RW): vga
>   0000000000000400-00000000000004ff (prio 1, RW): ne2000
> grackle
> VGA
> ne2k_pci
> macio-oldworld
> aliases
> pci-mmio
> 0000000000000000-00000000ffffffff (prio 0, RW): pci-mmio
>   00000000000a0000-00000000000bffff (prio 1, RW): vga-lowmem
>   0000000080000000-0000000080ffffff (prio 1, RW): vga.vram
>   0000000081000000-0000000081000fff (prio 1, RW): vga.mmio
>     0000000081000400-000000008100041f (prio 0, RW): vga ioports remapped
>     0000000081000500-0000000081000515 (prio 0, RW): bochs dispi interface
>     0000000081000600-0000000081000607 (prio 0, RW): qemu extended regs
>   0000000081010000-000000008101ffff (prio 1, RW): vga.rom
>   0000000081040000-000000008107ffff (prio 1, RW): ne2000.rom
> io
> 0000000000000000-000000000000ffff (prio 0, RW): io
>   00000000000001ce-00000000000001ce (prio 0, RW): vbe
>   00000000000001d0-00000000000001d0 (prio 0, RW): vbe
>   00000000000003b4-00000000000003b5 (prio 0, RW): vga
>   00000000000003ba-00000000000003ba (prio 0, RW): vga
>   00000000000003c0-00000000000003cf (prio 0, RW): vga
>   00000000000003d4-00000000000003d5 (prio 0, RW): vga
>   00000000000003da-00000000000003da (prio 0, RW): vga
>   0000000000000400-00000000000004ff (prio 1, RW): ne2000
> 
> 
> So it looks like several of the device MemoryRegions aren't being added
> after the "loadvm". Does this mean there is an object lifecycle issue
> here in that some of the initialisation needs to be done in realizefn
> rather than initfn or vice-versa?

I always thought we're going through both - initfn and realizefn with
normal system boot as well as vmstate load.

So that means that the additional mappings above must be runtime
creations that aren't saved and restored properly.


Alex

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Peter Maydell]

On 18 December 2014 at 14:46, Alexander Graf <agraf@suse.de> wrote:
> On 18.12.14 14:54, Mark Cave-Ayland wrote:
>> So it looks like several of the device MemoryRegions aren't being added
>> after the "loadvm". Does this mean there is an object lifecycle issue
>> here in that some of the initialisation needs to be done in realizefn
>> rather than initfn or vice-versa?
>
> I always thought we're going through both - initfn and realizefn with
> normal system boot as well as vmstate load.

Yes. Migration incoming and vmstate load both work as "create,
initialize, realize and reset system as normal, then do state
load before running it".

> So that means that the additional mappings above must be runtime
> creations that aren't saved and restored properly.

Looks likely. Memory regions themselves don't have any saved or
reloaded state, so it's the responsibility of the devices that
create and control them to ensure that they're set up correctly
again on state load. This is trivial for most devices which
just have an unchanging set, but controller chip equivalents
that allow the guest to map and unmap stuff need to be cleverer.

-- PMM

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Mark Cave-Ayland]

On 18/12/14 15:13, Peter Maydell wrote:

> On 18 December 2014 at 14:46, Alexander Graf <agraf@suse.de> wrote:
>> On 18.12.14 14:54, Mark Cave-Ayland wrote:
>>> So it looks like several of the device MemoryRegions aren't being added
>>> after the "loadvm". Does this mean there is an object lifecycle issue
>>> here in that some of the initialisation needs to be done in realizefn
>>> rather than initfn or vice-versa?
>>
>> I always thought we're going through both - initfn and realizefn with
>> normal system boot as well as vmstate load.
> 
> Yes. Migration incoming and vmstate load both work as "create,
> initialize, realize and reset system as normal, then do state
> load before running it".
> 
>> So that means that the additional mappings above must be runtime
>> creations that aren't saved and restored properly.
> 
> Looks likely. Memory regions themselves don't have any saved or
> reloaded state, so it's the responsibility of the devices that
> create and control them to ensure that they're set up correctly
> again on state load. This is trivial for most devices which
> just have an unchanging set, but controller chip equivalents
> that allow the guest to map and unmap stuff need to be cleverer.

I think that the problem is that the macio device doesn't have any
declared vmstate - presumably since no vmstate is saved for that device
then it doesn't appear in the loadvm restore list and so the object is
never re-initialised.

I can probably have a go at trying to sort out the VMStateDescription
(and maybe see how easy it is to convert some of these things to QOM)
but it seems that MACIOState has an array of qemu_irqs embedded directly
in it which I'm not sure how to handle.

Can anyone point me towards an example device the best current practice
for either wiring up qemu_irqs via QOM or how to represent them within a
VMStateDescription? In general it seems that SPARC and PPC mostly use
old APIs so there is a distinct lack of good reference examples for
devices I am familiar with.


ATB,

Mark.

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Alexander Graf]

On 18.12.14 22:36, Mark Cave-Ayland wrote:
> On 18/12/14 15:13, Peter Maydell wrote:
> 
>> On 18 December 2014 at 14:46, Alexander Graf <agraf@suse.de> wrote:
>>> On 18.12.14 14:54, Mark Cave-Ayland wrote:
>>>> So it looks like several of the device MemoryRegions aren't being added
>>>> after the "loadvm". Does this mean there is an object lifecycle issue
>>>> here in that some of the initialisation needs to be done in realizefn
>>>> rather than initfn or vice-versa?
>>>
>>> I always thought we're going through both - initfn and realizefn with
>>> normal system boot as well as vmstate load.
>>
>> Yes. Migration incoming and vmstate load both work as "create,
>> initialize, realize and reset system as normal, then do state
>> load before running it".
>>
>>> So that means that the additional mappings above must be runtime
>>> creations that aren't saved and restored properly.
>>
>> Looks likely. Memory regions themselves don't have any saved or
>> reloaded state, so it's the responsibility of the devices that
>> create and control them to ensure that they're set up correctly
>> again on state load. This is trivial for most devices which
>> just have an unchanging set, but controller chip equivalents
>> that allow the guest to map and unmap stuff need to be cleverer.
> 
> I think that the problem is that the macio device doesn't have any
> declared vmstate - presumably since no vmstate is saved for that device
> then it doesn't appear in the loadvm restore list and so the object is
> never re-initialised.
> 
> I can probably have a go at trying to sort out the VMStateDescription
> (and maybe see how easy it is to convert some of these things to QOM)
> but it seems that MACIOState has an array of qemu_irqs embedded directly
> in it which I'm not sure how to handle.
> 
> Can anyone point me towards an example device the best current practice
> for either wiring up qemu_irqs via QOM or how to represent them within a
> VMStateDescription? In general it seems that SPARC and PPC mostly use
> old APIs so there is a distinct lack of good reference examples for
> devices I am familiar with.

Why exactly would you need to wire up the qemu_irqs? If the lines are
asserted at the point of migration, the MPIC model should migrate over
the fact that a line is pending, no?


Alex

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Mark Cave-Ayland]

On 18/12/14 23:01, Alexander Graf wrote:

> On 18.12.14 22:36, Mark Cave-Ayland wrote:
>> On 18/12/14 15:13, Peter Maydell wrote:
>>
>>> On 18 December 2014 at 14:46, Alexander Graf <agraf@suse.de> wrote:
>>>> On 18.12.14 14:54, Mark Cave-Ayland wrote:
>>>>> So it looks like several of the device MemoryRegions aren't being added
>>>>> after the "loadvm". Does this mean there is an object lifecycle issue
>>>>> here in that some of the initialisation needs to be done in realizefn
>>>>> rather than initfn or vice-versa?
>>>>
>>>> I always thought we're going through both - initfn and realizefn with
>>>> normal system boot as well as vmstate load.
>>>
>>> Yes. Migration incoming and vmstate load both work as "create,
>>> initialize, realize and reset system as normal, then do state
>>> load before running it".
>>>
>>>> So that means that the additional mappings above must be runtime
>>>> creations that aren't saved and restored properly.
>>>
>>> Looks likely. Memory regions themselves don't have any saved or
>>> reloaded state, so it's the responsibility of the devices that
>>> create and control them to ensure that they're set up correctly
>>> again on state load. This is trivial for most devices which
>>> just have an unchanging set, but controller chip equivalents
>>> that allow the guest to map and unmap stuff need to be cleverer.
>>
>> I think that the problem is that the macio device doesn't have any
>> declared vmstate - presumably since no vmstate is saved for that device
>> then it doesn't appear in the loadvm restore list and so the object is
>> never re-initialised.
>>
>> I can probably have a go at trying to sort out the VMStateDescription
>> (and maybe see how easy it is to convert some of these things to QOM)
>> but it seems that MACIOState has an array of qemu_irqs embedded directly
>> in it which I'm not sure how to handle.
>>
>> Can anyone point me towards an example device the best current practice
>> for either wiring up qemu_irqs via QOM or how to represent them within a
>> VMStateDescription? In general it seems that SPARC and PPC mostly use
>> old APIs so there is a distinct lack of good reference examples for
>> devices I am familiar with.
> 
> Why exactly would you need to wire up the qemu_irqs? If the lines are
> asserted at the point of migration, the MPIC model should migrate over
> the fact that a line is pending, no?

It seems that I've misunderstood something mentioned earlier in the
thread, i.e. that loadvm also does the create, initialize, realize and
reset cycle. At least issuing "loadvm" in the monitor doesn't seem to do
that as far as I can see here? Otherwise I could see how recreating
qemu_irqs via the old-style init() functions every time "loadvm" is
issued would cause quite a bit of chaos.

Taking a different approach with a debugger this morning I think I may
have just figured out what's going on with the macio device - more to
follow soon I hope.


ATB,

Mark.

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Peter Maydell]

On 19 December 2014 at 10:53, Mark Cave-Ayland
<mark.cave-ayland@ilande.co.uk> wrote:
> It seems that I've misunderstood something mentioned earlier in the
> thread, i.e. that loadvm also does the create, initialize, realize and
> reset cycle. At least issuing "loadvm" in the monitor doesn't seem to do
> that as far as I can see here?

For monitor "loadvm", we've already done create/initialize/realize
as part of our initial setup of the VM when we started QEMU.
reset is done by load_vmstate() calling qemu_system_reset().

For command line "-loadvm", we do the load_vmstate() after the
initial reset in vl.c.

Either way, VM reloading doesn't need to deal with anything that's
set up in device init/realize as a fixed config, and it doesn't
need to assert IRQ lines either.

thanks
-- PMM

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Mark Cave-Ayland]

On 19/12/14 12:29, Peter Maydell wrote:

> On 19 December 2014 at 10:53, Mark Cave-Ayland
> <mark.cave-ayland@ilande.co.uk> wrote:
>> It seems that I've misunderstood something mentioned earlier in the
>> thread, i.e. that loadvm also does the create, initialize, realize and
>> reset cycle. At least issuing "loadvm" in the monitor doesn't seem to do
>> that as far as I can see here?
> 
> For monitor "loadvm", we've already done create/initialize/realize
> as part of our initial setup of the VM when we started QEMU.
> reset is done by load_vmstate() calling qemu_system_reset().
> 
> For command line "-loadvm", we do the load_vmstate() after the
> initial reset in vl.c.
> 
> Either way, VM reloading doesn't need to deal with anything that's
> set up in device init/realize as a fixed config, and it doesn't
> need to assert IRQ lines either.

I now have something that nearly works in that I can issue a "savevm"
followed by a "loadvm" in the monitor and be able to resume where I left
off - but only in OpenBIOS. If I boot into an OS and try this then it
doesn't work and the console just sits there frozen.

Also there seems to be a difference between issuing "loadvm" in the
monitor which works, and adding -loadvm to the command line which
completes successfully but still doesn't work as the console remains
frozen, even just in OpenBIOS. This looks similar to trying to load an
image created within an OS as above.

I poked around with gdbstub and the difference with -loadvm on the
command line is that as soon as I step into the next instruction I get a
DSI at the PC address within OpenBIOS space which can't be resolved.

According to "info mtree" the BIOS is present at the physical address,
but it appears that the 1:1 physical to virtual mapping for the BIOS
isn't there (or at least both gdbstub and trying to access the virtual
address where the BIOS is supposed to be located gives a "Cannot access
memory" error).

Could it be that some of the CPU TLB state doesn't get serialized to
disk as part of "savevm" which is causing these immediate faults when
launching with -loadvm?


ATB,

Mark.

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Peter Maydell]

On 19 December 2014 at 14:12, Mark Cave-Ayland
<mark.cave-ayland@ilande.co.uk> wrote:
> I now have something that nearly works in that I can issue a "savevm"
> followed by a "loadvm" in the monitor and be able to resume where I left
> off - but only in OpenBIOS. If I boot into an OS and try this then it
> doesn't work and the console just sits there frozen.

I'd check whether the interrupt controller is correctly restoring
its state...

-- PMM

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Mark Cave-Ayland]

On 19/12/14 14:29, Peter Maydell wrote:

> On 19 December 2014 at 14:12, Mark Cave-Ayland
> <mark.cave-ayland@ilande.co.uk> wrote:
>> I now have something that nearly works in that I can issue a "savevm"
>> followed by a "loadvm" in the monitor and be able to resume where I left
>> off - but only in OpenBIOS. If I boot into an OS and try this then it
>> doesn't work and the console just sits there frozen.
> 
> I'd check whether the interrupt controller is correctly restoring
> its state...

It's not even getting that far - if I single-step through OpenBIOS then
I can see the first access raising an ISI exception, the entry being
added to the MMU hashtable and then we retry which immediately faults again.

(goes and digs further...)

I think this is a CPUState problem. If I step into
ppc_hash32_htab_lookup() after trying to load an image with -loadvm then
I get this:

Breakpoint 1, ppc_hash32_htab_lookup (env=0x7ffff7fdf260, sr=536871951,
eaddr=4294083148, pte=0x7fffe57ee810) at
/home/build/src/qemu/git/qemu/target-ppc/mmu-hash32.c:338
338     {
(gdb) n
343         vsid = sr & SR32_VSID;
(gdb)
344         pgidx = (eaddr & ~SEGMENT_MASK_256M) >> TARGET_PAGE_BITS;
(gdb)
345         hash = vsid ^ pgidx;
(gdb)
346         ptem = (vsid << 7) | (pgidx >> 10);
(gdb)
352                 env->htab_base, env->htab_mask, hash);
(gdb)
349         qemu_log_mask(CPU_LOG_MMU, "htab_base " TARGET_FMT_plx
(gdb) p/x env->htab_base
$1 = 0x0


It looks like after restoring the CPU state with loadvm env->htab_base
is still at 0x0 rather than its pre-migration value of 0x7e000000.


ATB,

Mark.

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Peter Maydell]

On 19 December 2014 at 16:15, Mark Cave-Ayland
<mark.cave-ayland@ilande.co.uk> wrote:
> It looks like after restoring the CPU state with loadvm env->htab_base
> is still at 0x0 rather than its pre-migration value of 0x7e000000.

Odd -- there is code that is at least trying to do that:
machine.c calls ppc_store_sdr1() which should set htab_base/mask
according to the migrated value of the register...

-- PMM

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Mark Cave-Ayland]

On 19/12/14 16:35, Peter Maydell wrote:

> On 19 December 2014 at 16:15, Mark Cave-Ayland
> <mark.cave-ayland@ilande.co.uk> wrote:
>> It looks like after restoring the CPU state with loadvm env->htab_base
>> is still at 0x0 rather than its pre-migration value of 0x7e000000.
> 
> Odd -- there is code that is at least trying to do that:
> machine.c calls ppc_store_sdr1() which should set htab_base/mask
> according to the migrated value of the register...

Stepping through ppc_store_sdr1() I see the problem is the if() statement:

if (env->spr[SPR_SDR1] != value) {
   ....
}

It looks like env->spr[SPR_SDR1] has already been set to 0x7e000000
before the call to ppc_store_sdr1() and so as the values are already
equal then the code to setup env->htab_mask and env->htab_base is bypassed.

The quick fix is to comment out the above if() statement which allows me
to restore my OpenBIOS image successfully with -loadvm but sadly not my
OS image (I guess there are probably a few more state bugs still lying
around). Does this seem the right thing to do or is there a better way?


ATB,

Mark.

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Alexander Graf]

On 19.12.14 17:51, Mark Cave-Ayland wrote:
> On 19/12/14 16:35, Peter Maydell wrote:
> 
>> On 19 December 2014 at 16:15, Mark Cave-Ayland
>> <mark.cave-ayland@ilande.co.uk> wrote:
>>> It looks like after restoring the CPU state with loadvm env->htab_base
>>> is still at 0x0 rather than its pre-migration value of 0x7e000000.
>>
>> Odd -- there is code that is at least trying to do that:
>> machine.c calls ppc_store_sdr1() which should set htab_base/mask
>> according to the migrated value of the register...
> 
> Stepping through ppc_store_sdr1() I see the problem is the if() statement:
> 
> if (env->spr[SPR_SDR1] != value) {
>    ....
> }
> 
> It looks like env->spr[SPR_SDR1] has already been set to 0x7e000000
> before the call to ppc_store_sdr1() and so as the values are already
> equal then the code to setup env->htab_mask and env->htab_base is bypassed.
> 
> The quick fix is to comment out the above if() statement which allows me
> to restore my OpenBIOS image successfully with -loadvm but sadly not my
> OS image (I guess there are probably a few more state bugs still lying
> around). Does this seem the right thing to do or is there a better way?

How about something like this instead? Then we at least don't lose the
acceleration we get from not invalidating the TLB on identical SDR1 writes.


Alex


diff --git a/target-ppc/machine.c b/target-ppc/machine.c
index c801b82..6f347bf 100644
--- a/target-ppc/machine.c
+++ b/target-ppc/machine.c
@@ -71,6 +71,7 @@ static int cpu_load_old(QEMUFile *f, void *opaque, int
version_id)
     for (i = 0; i < 1024; i++)
         qemu_get_betls(f, &env->spr[i]);
     if (!env->external_htab) {
+        env->spr[SPR_SDR1] = 0;
         ppc_store_sdr1(env, sdr1);
     }
     qemu_get_be32s(f, &env->vscr);

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Paolo Bonzini]

On 19/12/2014 18:16, Alexander Graf wrote:
> 
> 
> On 19.12.14 17:51, Mark Cave-Ayland wrote:
>> On 19/12/14 16:35, Peter Maydell wrote:
>>
>>> On 19 December 2014 at 16:15, Mark Cave-Ayland
>>> <mark.cave-ayland@ilande.co.uk> wrote:
>>>> It looks like after restoring the CPU state with loadvm env->htab_base
>>>> is still at 0x0 rather than its pre-migration value of 0x7e000000.
>>>
>>> Odd -- there is code that is at least trying to do that:
>>> machine.c calls ppc_store_sdr1() which should set htab_base/mask
>>> according to the migrated value of the register...
>>
>> Stepping through ppc_store_sdr1() I see the problem is the if() statement:
>>
>> if (env->spr[SPR_SDR1] != value) {
>>    ....
>> }
>>
>> It looks like env->spr[SPR_SDR1] has already been set to 0x7e000000
>> before the call to ppc_store_sdr1() and so as the values are already
>> equal then the code to setup env->htab_mask and env->htab_base is bypassed.
>>
>> The quick fix is to comment out the above if() statement which allows me
>> to restore my OpenBIOS image successfully with -loadvm but sadly not my
>> OS image (I guess there are probably a few more state bugs still lying
>> around). Does this seem the right thing to do or is there a better way?
> 
> How about something like this instead? Then we at least don't lose the
> acceleration we get from not invalidating the TLB on identical SDR1 writes.
> 
> 
> Alex
> 
> 
> diff --git a/target-ppc/machine.c b/target-ppc/machine.c
> index c801b82..6f347bf 100644
> --- a/target-ppc/machine.c
> +++ b/target-ppc/machine.c
> @@ -71,6 +71,7 @@ static int cpu_load_old(QEMUFile *f, void *opaque, int
> version_id)
>      for (i = 0; i < 1024; i++)
>          qemu_get_betls(f, &env->spr[i]);
>      if (!env->external_htab) {
> +        env->spr[SPR_SDR1] = 0;
>          ppc_store_sdr1(env, sdr1);
>      }
>      qemu_get_be32s(f, &env->vscr);

Or even move the tlb_flush and if to the caller, helper_store_sdr1.  It
is only needed there, and would get in the way if one day PPC were to
support --disable-tcg (which removes cputlb.c from the executable).

Paolo

Re: [Qemu-devel] [Qemu-ppc] Unable to loadvm on qemu-system-ppc -M g3beige (keyboard freeze)

[Alexander Graf]

On 19.12.14 18:29, Paolo Bonzini wrote:
> 
> 
> On 19/12/2014 18:16, Alexander Graf wrote:
>>
>>
>> On 19.12.14 17:51, Mark Cave-Ayland wrote:
>>> On 19/12/14 16:35, Peter Maydell wrote:
>>>
>>>> On 19 December 2014 at 16:15, Mark Cave-Ayland
>>>> <mark.cave-ayland@ilande.co.uk> wrote:
>>>>> It looks like after restoring the CPU state with loadvm env->htab_base
>>>>> is still at 0x0 rather than its pre-migration value of 0x7e000000.
>>>>
>>>> Odd -- there is code that is at least trying to do that:
>>>> machine.c calls ppc_store_sdr1() which should set htab_base/mask
>>>> according to the migrated value of the register...
>>>
>>> Stepping through ppc_store_sdr1() I see the problem is the if() statement:
>>>
>>> if (env->spr[SPR_SDR1] != value) {
>>>    ....
>>> }
>>>
>>> It looks like env->spr[SPR_SDR1] has already been set to 0x7e000000
>>> before the call to ppc_store_sdr1() and so as the values are already
>>> equal then the code to setup env->htab_mask and env->htab_base is bypassed.
>>>
>>> The quick fix is to comment out the above if() statement which allows me
>>> to restore my OpenBIOS image successfully with -loadvm but sadly not my
>>> OS image (I guess there are probably a few more state bugs still lying
>>> around). Does this seem the right thing to do or is there a better way?
>>
>> How about something like this instead? Then we at least don't lose the
>> acceleration we get from not invalidating the TLB on identical SDR1 writes.
>>
>>
>> Alex
>>
>>
>> diff --git a/target-ppc/machine.c b/target-ppc/machine.c
>> index c801b82..6f347bf 100644
>> --- a/target-ppc/machine.c
>> +++ b/target-ppc/machine.c
>> @@ -71,6 +71,7 @@ static int cpu_load_old(QEMUFile *f, void *opaque, int
>> version_id)
>>      for (i = 0; i < 1024; i++)
>>          qemu_get_betls(f, &env->spr[i]);
>>      if (!env->external_htab) {
>> +        env->spr[SPR_SDR1] = 0;
>>          ppc_store_sdr1(env, sdr1);
>>      }
>>      qemu_get_be32s(f, &env->vscr);
> 
> Or even move the tlb_flush and if to the caller, g.  It
> is only needed there, and would get in the way if one day PPC were to
> support --disable-tcg (which removes cputlb.c from the executable).

Even better, yes.


Alex