Re: [Qemu-devel] vm state save/restore question

[Benjamin Herrenschmidt <benh@kernel.crashing.org>]

On Sat, 2012-06-09 at 20:53 +1000, Benjamin Herrenschmidt wrote:
> Hi folks !

(After some discussion with Andreas ...)

> I'm looking at sorting out the state save/restore of target-ppc (which
> means understanding in general how it works in qemu :-)
> 
> So far I've somewhat figured out that there's the "old way" where we
> just provide a "bulk" save/restore function pair, and the "new way"
> where we have nicely tagged field lists etc...
> 
> x86 seems to use the later for the CPU state, ppc is a mess and uses the
> former with interesting incompatible format change depending on how qemu
> is build :-) So I think that's one area I need to fix.

Ok, so I'm told there are patches to convert ppc, I haven't seen them in
my list archives, so if somebody has a pointer, please shoot, that will
save me some work :-)

  .../...

> What I'd need is something in spapr that can be used to "resync" bits of
> the cpu state with the external htab that gets run after everything is
> loaded and before emulation restarts.
> 
> Any idea how to do that properly ? I suppose I could also try to iterate
> all the vcpu's after loading the hash table & update the fields but not
> only that's gross ... I also don't know how to do it :-)

So I did an experiment using the "old style" save/restore (bad boy !)
and got that part to work by just iterating the vcpu's.

It's a bit nasty but it's the right way I think, ie, what we have here
(the external hash table) is a global object under control/ownership of
the platform code for which a pointer is cached in the CPU state (so the
mmu emulation gets to it easily), so those cached pointers need to be
updated in all CPUs when a new hash table is loaded/allocated.

That leads to another question however... I need to add save/restore to
a bunch more stuff such as the xics (interrupt controller), the various
spapr devices, etc...

So far the VMState stuff is all nice if you have fixed sized arrays.
However I haven't quite found out the right way to use it for things
like:

 - The hash table (mentioned above). This is just a big chunk of memory
(it will routinely be 16M), so I really don't want to start iterating
all elements, just a bulk load will do, and the size might actually be
variable.

 - The XICS (interrupt controller). The actual size of the interrupt
state array can vary (the number of interrupt sources can vary, it's
fixed today by the machine code but I wouldn't rely too much on that and
in any case, from the XICS driver perspective, it's not a constant, it's
a variable it gets passed when initializing).

So in both these cases, I need either code to control the save/load
process (old style ? hard to hook into vmstate as far as I can tell) or
maybe a way to describe the array so that the array size itself is a
pointer to a variable (Andreas mentioned something along those lines).
Is there any doco for that stuff btw ? I haven't seen anything
detailed...

Suggestions ?

Cheers,
Ben.