[Qemu-devel] KVM call agenda for tuesday 31

[Qemu-devel] KVM call agenda for tuesday 31

[Juan Quintela]

hi

Please send in any agenda items you are interested in covering.

Cheers,

Juan.

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Andreas Färber]

Am 30.01.2012 19:55, schrieb Juan Quintela:
> Please send in any agenda items you are interested in covering.

QOM roadmap update:
* Series 3/4 is on the list.
-> Please officially designate a merge date (Friday?).
-> To make review sensible, I ask for a hard device freeze until merged.
   I.e., no new devices and no conflicting changes to DeviceInfo.

* Further roadmap: i440FX, CPU, MemoryRegion (when, who, where, how)

VMState:
Anthony specifically said that VMState were not affected by QOM and that
patches should not be deferred until the merge. Yet there's no review
and/or decision-making for a month now. Ping^2 for AHCI+SDHC.

Mascot contest:
Any update? Were there too few entries? Has it been forgotten?
If the contest is still open then that should be stated on the list.

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Anthony Liguori]

On 01/30/2012 05:41 PM, Andreas Färber wrote:
> Am 30.01.2012 19:55, schrieb Juan Quintela:
>> Please send in any agenda items you are interested in covering.
>
> QOM roadmap update:
> * Series 3/4 is on the list.
> ->  Please officially designate a merge date (Friday?).
> ->  To make review sensible, I ask for a hard device freeze until merged.
>     I.e., no new devices and no conflicting changes to DeviceInfo.
>
> * Further roadmap: i440FX, CPU, MemoryRegion (when, who, where, how)
>
> VMState:
> Anthony specifically said that VMState were not affected by QOM and that
> patches should not be deferred until the merge. Yet there's no review
> and/or decision-making for a month now. Ping^2 for AHCI+SDHC.

Do you have pointers (to pending VMState patches)?

>
> Mascot contest:
> Any update? Were there too few entries? Has it been forgotten?
> If the contest is still open then that should be stated on the list.

Yes, I'm falling victim to perfectionism here attempting to find a clever online 
voting mechanism.  I'm just going to hold a vote over the mailing list.  I'll 
post an email tomorrow with details.

Regards,

Anthony Liguori

>
> Andreas
>

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Alexander Graf]

On 31.01.2012, at 00:53, Anthony Liguori wrote:

> On 01/30/2012 05:41 PM, Andreas Färber wrote:
>> Am 30.01.2012 19:55, schrieb Juan Quintela:
>>> Please send in any agenda items you are interested in covering.
>> 
>> QOM roadmap update:
>> * Series 3/4 is on the list.
>> ->  Please officially designate a merge date (Friday?).
>> ->  To make review sensible, I ask for a hard device freeze until merged.
>>    I.e., no new devices and no conflicting changes to DeviceInfo.
>> 
>> * Further roadmap: i440FX, CPU, MemoryRegion (when, who, where, how)
>> 
>> VMState:
>> Anthony specifically said that VMState were not affected by QOM and that
>> patches should not be deferred until the merge. Yet there's no review
>> and/or decision-making for a month now. Ping^2 for AHCI+SDHC.
> 
> Do you have pointers (to pending VMState patches)?
> 
>> 
>> Mascot contest:
>> Any update? Were there too few entries? Has it been forgotten?
>> If the contest is still open then that should be stated on the list.
> 
> Yes, I'm falling victim to perfectionism here attempting to find a clever online voting mechanism.  I'm just going to hold a vote over the mailing list.  I'll post an email tomorrow with details.

I hear google docs has some sort of online form thing in its docs bits. Might be easier to parse than mails on the ML.


Alex

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Lluís Vilanova]

Anthony Liguori writes:

> On 01/30/2012 05:41 PM, Andreas Färber wrote:
>> 
>> Mascot contest:
>> Any update? Were there too few entries? Has it been forgotten?
>> If the contest is still open then that should be stated on the list.

> Yes, I'm falling victim to perfectionism here attempting to find a clever online
> voting mechanism.  I'm just going to hold a vote over the mailing list.  I'll
> post an email tomorrow with details.

http://www.cs.cornell.edu/andru/civs.html


Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Andreas Färber]

Am 31.01.2012 00:53, schrieb Anthony Liguori:
> On 01/30/2012 05:41 PM, Andreas Färber wrote:
>> Am 30.01.2012 19:55, schrieb Juan Quintela:
>>> Please send in any agenda items you are interested in covering.

>> VMState:
>> Anthony specifically said that VMState were not affected by QOM and that
>> patches should not be deferred until the merge. Yet there's no review
>> and/or decision-making for a month now. Ping^2 for AHCI+SDHC.
> 
> Do you have pointers (to pending VMState patches)?

http://patchwork.ozlabs.org/patch/137732/ (PATCH v4)

It's basically about how to deal with variable-sized arrays. (Alex
mentioned it on one call around November.) I found ways to deal with
subsets of arrays embedded within the struct and variable-sized list of
pointers to structs but no solution for a malloc()'ed array of structs.
Maybe I'm just too stupid to see. Anyway, no one commented since Xmas.

Igor posted (and refined for v2) a patch with a callback-based approach
that I find promising. From my view, unofficially Juan is the VMState
guy, he's been cc'ed. Are we lacking an official maintainer that cares?
Or is Juan the official, undocumented maintainer but simply busy?

SUSE's interest is making AHCI migratable, and my VMState workaround for
that is simply ugly:

http://patchwork.ozlabs.org/patch/133066/ (RFC)

Therefore I'm waiting for some resolution.

Regards,
Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Anthony Liguori]

On 01/30/2012 05:41 PM, Andreas Färber wrote:
> Am 30.01.2012 19:55, schrieb Juan Quintela:
>> Please send in any agenda items you are interested in covering.
>
> QOM roadmap update:
> * Series 3/4 is on the list.
> ->  Please officially designate a merge date (Friday?).
> ->  To make review sensible, I ask for a hard device freeze until merged.
>     I.e., no new devices and no conflicting changes to DeviceInfo.

I put together a few slides to help this discussion:

http://www.codemonkey.ws/files/qom-overview.pdf

Regards,

Anthony Liguori

>
> * Further roadmap: i440FX, CPU, MemoryRegion (when, who, where, how)
>
> VMState:
> Anthony specifically said that VMState were not affected by QOM and that
> patches should not be deferred until the merge. Yet there's no review
> and/or decision-making for a month now. Ping^2 for AHCI+SDHC.
>
> Mascot contest:
> Any update? Were there too few entries? Has it been forgotten?
> If the contest is still open then that should be stated on the list.
>
> Andreas
>

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Mitsyanko Igor]

On 01/31/2012 05:15 PM, Andreas Färber wrote:
> Am 31.01.2012 00:53, schrieb Anthony Liguori:
>> On 01/30/2012 05:41 PM, Andreas Färber wrote:
>>> Am 30.01.2012 19:55, schrieb Juan Quintela:
>>>> Please send in any agenda items you are interested in covering.
>
>>> VMState:
>>> Anthony specifically said that VMState were not affected by QOM and that
>>> patches should not be deferred until the merge. Yet there's no review
>>> and/or decision-making for a month now. Ping^2 for AHCI+SDHC.
>>
>> Do you have pointers (to pending VMState patches)?
>
> http://patchwork.ozlabs.org/patch/137732/ (PATCH v4)
>
> It's basically about how to deal with variable-sized arrays. (Alex
> mentioned it on one call around November.) I found ways to deal with
> subsets of arrays embedded within the struct and variable-sized list of
> pointers to structs but no solution for a malloc()'ed array of structs.
> Maybe I'm just too stupid to see. Anyway, no one commented since Xmas.
>
> Igor posted (and refined for v2) a patch with a callback-based approach
> that I find promising. From my view, unofficially Juan is the VMState
> guy, he's been cc'ed. Are we lacking an official maintainer that cares?
> Or is Juan the official, undocumented maintainer but simply busy?
>
> SUSE's interest is making AHCI migratable, and my VMState workaround for
> that is simply ugly:
>
> http://patchwork.ozlabs.org/patch/133066/ (RFC)
>

If I'm not mistaken, if you change AHCIState's ".ports" type to uint32_t 
you can use existing VMSTATE_BUFFER_MULTIPLY macro like this:

VMSTATE_BUFFER_MULTIPLY(dev, AHCIState, 0, NULL, 0, ports, 
sizeof(AHCIDevice))

VMSTATE_BUFFER_MULTIPLY currently lacks VMS_POINTER flag and therefore 
doesn't make any use of _start field (you don't need it anyway)

Nevertheless, VMSTATE_BUFFER_MULTIPLY is just a partial solution to a 
bigger set of possible problems. SD card's vmstate implementation 
requires shift operation, SDHC gets size from switch {} statement, 
something else later may require division or addition e.t.c., 
get_bufsize callback will cover all possible cases.



-- 
Mitsyanko Igor
ASWG, Moscow R&D center, Samsung Electronics
email: i.mitsyanko@samsung.com

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Avi Kivity]

On 01/31/2012 03:59 PM, Anthony Liguori wrote:
> On 01/30/2012 05:41 PM, Andreas Färber wrote:
>> Am 30.01.2012 19:55, schrieb Juan Quintela:
>>> Please send in any agenda items you are interested in covering.
>>
>> QOM roadmap update:
>> * Series 3/4 is on the list.
>> ->  Please officially designate a merge date (Friday?).
>> ->  To make review sensible, I ask for a hard device freeze until
>> merged.
>>     I.e., no new devices and no conflicting changes to DeviceInfo.
>
> I put together a few slides to help this discussion:
>
> http://www.codemonkey.ws/files/qom-overview.pdf
>

That was helpful, thanks.

Can you clarify

- Types and their properties will be ABI  compatible
- Types and properties will not be backwards  compatible
– We can re-examine this as the device model  matures and stabilizes

the first two seem very similar, except for the "not".


-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Anthony Liguori]

On 01/31/2012 07:15 AM, Andreas Färber wrote:
> Am 31.01.2012 00:53, schrieb Anthony Liguori:
>> On 01/30/2012 05:41 PM, Andreas Färber wrote:
>>> Am 30.01.2012 19:55, schrieb Juan Quintela:
>>>> Please send in any agenda items you are interested in covering.
>
>>> VMState:
>>> Anthony specifically said that VMState were not affected by QOM and that
>>> patches should not be deferred until the merge. Yet there's no review
>>> and/or decision-making for a month now. Ping^2 for AHCI+SDHC.
>>
>> Do you have pointers (to pending VMState patches)?
>
> http://patchwork.ozlabs.org/patch/137732/ (PATCH v4)
>
> It's basically about how to deal with variable-sized arrays. (Alex
> mentioned it on one call around November.) I found ways to deal with
> subsets of arrays embedded within the struct and variable-sized list of
> pointers to structs but no solution for a malloc()'ed array of structs.
> Maybe I'm just too stupid to see. Anyway, no one commented since Xmas.

/me puts on his flame proof suit

Don't use VMState.  Just open code a save/restore function.  VMState is too 
limited in how it handles complex data structures.

I really believe the only long term solution we're going to get to here is 
something that uses a builder interface (like Visitors).

Regards,

Anthony Liguori

>
> Igor posted (and refined for v2) a patch with a callback-based approach
> that I find promising. From my view, unofficially Juan is the VMState
> guy, he's been cc'ed. Are we lacking an official maintainer that cares?
> Or is Juan the official, undocumented maintainer but simply busy?
>
> SUSE's interest is making AHCI migratable, and my VMState workaround for
> that is simply ugly:
>
> http://patchwork.ozlabs.org/patch/133066/ (RFC)
>
> Therefore I'm waiting for some resolution.
>
> Regards,
> Andreas
>

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Anthony Liguori]

On 01/31/2012 08:09 AM, Avi Kivity wrote:
> On 01/31/2012 03:59 PM, Anthony Liguori wrote:
>> On 01/30/2012 05:41 PM, Andreas Färber wrote:
>>> Am 30.01.2012 19:55, schrieb Juan Quintela:
>>>> Please send in any agenda items you are interested in covering.
>>>
>>> QOM roadmap update:
>>> * Series 3/4 is on the list.
>>> ->   Please officially designate a merge date (Friday?).
>>> ->   To make review sensible, I ask for a hard device freeze until
>>> merged.
>>>      I.e., no new devices and no conflicting changes to DeviceInfo.
>>
>> I put together a few slides to help this discussion:
>>
>> http://www.codemonkey.ws/files/qom-overview.pdf
>>
>
> That was helpful, thanks.
>
> Can you clarify
>
> - Types and their properties will be ABI  compatible
> - Types and properties will not be backwards  compatible
> – We can re-examine this as the device model  matures and stabilizes
>
> the first two seem very similar, except for the "not".

I guess I really mean QMP compatible.  The expectation is that well written 
management code does:

if (check_for_command("qom-list")) {
    run_command("qom-list", "/");
}

ABI compatible means that if qom-list is there, it's semantics never change. 
Backwards compatibility would mean that once qom-list is introduced, it never 
goes away.

In terms of QOM types, we won't guarantee that a Type will stick around forever, 
but we will guarantee if it's there, it'll behave in a certain way.

When the device model stabilizes over time, we can revisit this, but in the 1.x 
series, I'm sure we're going to go through a few significant refactorings that 
change types significantly.

Regards,

Anthony Liguori

>
>

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Avi Kivity]

On 01/31/2012 04:17 PM, Anthony Liguori wrote:
> On 01/31/2012 08:09 AM, Avi Kivity wrote:
>> On 01/31/2012 03:59 PM, Anthony Liguori wrote:
>>> On 01/30/2012 05:41 PM, Andreas Färber wrote:
>>>> Am 30.01.2012 19:55, schrieb Juan Quintela:
>>>>> Please send in any agenda items you are interested in covering.
>>>>
>>>> QOM roadmap update:
>>>> * Series 3/4 is on the list.
>>>> ->   Please officially designate a merge date (Friday?).
>>>> ->   To make review sensible, I ask for a hard device freeze until
>>>> merged.
>>>>      I.e., no new devices and no conflicting changes to DeviceInfo.
>>>
>>> I put together a few slides to help this discussion:
>>>
>>> http://www.codemonkey.ws/files/qom-overview.pdf
>>>
>>
>> That was helpful, thanks.
>>
>> Can you clarify
>>
>> - Types and their properties will be ABI  compatible
>> - Types and properties will not be backwards  compatible
>> – We can re-examine this as the device model  matures and stabilizes
>>
>> the first two seem very similar, except for the "not".
>
> I guess I really mean QMP compatible.  The expectation is that well
> written management code does:
>
> if (check_for_command("qom-list")) {
>    run_command("qom-list", "/");
> }
>
> ABI compatible means that if qom-list is there, it's semantics never
> change. Backwards compatibility would mean that once qom-list is
> introduced, it never goes away.
>
> In terms of QOM types, we won't guarantee that a Type will stick
> around forever, but we will guarantee if it's there, it'll behave in a
> certain way.
>
> When the device model stabilizes over time, we can revisit this, but
> in the 1.x series, I'm sure we're going to go through a few
> significant refactorings that change types significantly.

Ok.  We have to communicate this very carefully.  If a management tool
uses something that is ABI compatible, but not backwards compatible, and
lacks a known alternative at the time of writing the management tool,
then the "ABI-compatible-but-not-backwards-compatible" property becomes
"not backwards compatible" to the management tool's users.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Michael S. Tsirkin]

On Tue, Jan 31, 2012 at 08:12:29AM -0600, Anthony Liguori wrote:
> On 01/31/2012 07:15 AM, Andreas Färber wrote:
> >Am 31.01.2012 00:53, schrieb Anthony Liguori:
> >>On 01/30/2012 05:41 PM, Andreas Färber wrote:
> >>>Am 30.01.2012 19:55, schrieb Juan Quintela:
> >>>>Please send in any agenda items you are interested in covering.
> >
> >>>VMState:
> >>>Anthony specifically said that VMState were not affected by QOM and that
> >>>patches should not be deferred until the merge. Yet there's no review
> >>>and/or decision-making for a month now. Ping^2 for AHCI+SDHC.
> >>
> >>Do you have pointers (to pending VMState patches)?
> >
> >http://patchwork.ozlabs.org/patch/137732/ (PATCH v4)
> >
> >It's basically about how to deal with variable-sized arrays. (Alex
> >mentioned it on one call around November.) I found ways to deal with
> >subsets of arrays embedded within the struct and variable-sized list of
> >pointers to structs but no solution for a malloc()'ed array of structs.
> >Maybe I'm just too stupid to see. Anyway, no one commented since Xmas.
> 
> /me puts on his flame proof suit
> 
> Don't use VMState.  Just open code a save/restore function.  VMState
> is too limited in how it handles complex data structures.
> 
> I really believe the only long term solution we're going to get to
> here is something that uses a builder interface (like Visitors).
> 
> Regards,
> 
> Anthony Liguori

So the TPM patches started implementing a visitor interface
for BER, they are still blocked, right?

> >
> >Igor posted (and refined for v2) a patch with a callback-based approach
> >that I find promising. From my view, unofficially Juan is the VMState
> >guy, he's been cc'ed. Are we lacking an official maintainer that cares?
> >Or is Juan the official, undocumented maintainer but simply busy?
> >
> >SUSE's interest is making AHCI migratable, and my VMState workaround for
> >that is simply ugly:
> >
> >http://patchwork.ozlabs.org/patch/133066/ (RFC)
> >
> >Therefore I'm waiting for some resolution.
> >
> >Regards,
> >Andreas
> >
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Michael S. Tsirkin]

On Tue, Jan 31, 2012 at 05:04:48PM +0200, Michael S. Tsirkin wrote:
> On Tue, Jan 31, 2012 at 08:12:29AM -0600, Anthony Liguori wrote:
> > On 01/31/2012 07:15 AM, Andreas Färber wrote:
> > >Am 31.01.2012 00:53, schrieb Anthony Liguori:
> > >>On 01/30/2012 05:41 PM, Andreas Färber wrote:
> > >>>Am 30.01.2012 19:55, schrieb Juan Quintela:
> > >>>>Please send in any agenda items you are interested in covering.
> > >
> > >>>VMState:
> > >>>Anthony specifically said that VMState were not affected by QOM and that
> > >>>patches should not be deferred until the merge. Yet there's no review
> > >>>and/or decision-making for a month now. Ping^2 for AHCI+SDHC.
> > >>
> > >>Do you have pointers (to pending VMState patches)?
> > >
> > >http://patchwork.ozlabs.org/patch/137732/ (PATCH v4)
> > >
> > >It's basically about how to deal with variable-sized arrays. (Alex
> > >mentioned it on one call around November.) I found ways to deal with
> > >subsets of arrays embedded within the struct and variable-sized list of
> > >pointers to structs but no solution for a malloc()'ed array of structs.
> > >Maybe I'm just too stupid to see. Anyway, no one commented since Xmas.
> > 
> > /me puts on his flame proof suit
> > 
> > Don't use VMState.  Just open code a save/restore function.  VMState
> > is too limited in how it handles complex data structures.
> > 
> > I really believe the only long term solution we're going to get to
> > here is something that uses a builder interface (like Visitors).
> > 
> > Regards,
> > 
> > Anthony Liguori
> 
> So the TPM patches started implementing a visitor interface
> for BER, they are still blocked, right?

And by the way, I believe we really should switch to
something like BER and just drop the legacy migration
format - being non self delimiting makes it just too
painful for words to abstract, and attempts to
hide that mess behind a visitor interface will just cause
more cruft to accumulate, and cause inefficiencies
such as extra data copies.

> > >
> > >Igor posted (and refined for v2) a patch with a callback-based approach
> > >that I find promising. From my view, unofficially Juan is the VMState
> > >guy, he's been cc'ed. Are we lacking an official maintainer that cares?
> > >Or is Juan the official, undocumented maintainer but simply busy?
> > >
> > >SUSE's interest is making AHCI migratable, and my VMState workaround for
> > >that is simply ugly:
> > >
> > >http://patchwork.ozlabs.org/patch/133066/ (RFC)
> > >
> > >Therefore I'm waiting for some resolution.
> > >
> > >Regards,
> > >Andreas
> > >
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Paolo Bonzini]

On 01/31/2012 03:12 PM, Anthony Liguori wrote:
>
> Don't use VMState.  Just open code a save/restore function.  VMState is
> too limited in how it handles complex data structures.
>
> I really believe the only long term solution we're going to get to here
> is something that uses a builder interface (like Visitors).

Visitors for complex data structures still require you separate get/set 
functions.  VMState was created to avoid having to write things twice 
and avoid getting the two out of sync.

Paolo

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Andreas Färber]

Am 31.01.2012 14:59, schrieb Anthony Liguori:
> On 01/30/2012 05:41 PM, Andreas Färber wrote:
>> Am 30.01.2012 19:55, schrieb Juan Quintela:
>>> Please send in any agenda items you are interested in covering.
>>
>> QOM roadmap update:
>> * Series 3/4 is on the list.
>> ->  Please officially designate a merge date (Friday?).
>> ->  To make review sensible, I ask for a hard device freeze until merged.
>>     I.e., no new devices and no conflicting changes to DeviceInfo.
> 
> I put together a few slides to help this discussion:
> 
> http://www.codemonkey.ws/files/qom-overview.pdf

So basically we barely managed to walk through the slides without
getting into any of the requested agenda points. ;)

Re slide 5 I could use a more detailed explanation.
>From what I understand, ...
...types are registered through device_init() and module_call_init().
...classes are lazily initialized, eventually calling class_init.
...objects are instantiated through object_new[_with_type](), eventually
calling instance_init, by convention named initfn.
Up to here would be step 1.

How does the realize step 2 fit in technically? What types does this
apply to? All? Who calls realize? Where should class authors draw the
line between the two? (DOs and DON'Ts)

Then some conceptual questions:

How is inheritance (virtual methods in C++) supposed to work? (in my
case a reset function overwritten by each subclass) I've seen code in
series 3/4 calling methods the parent class defines in its header,
replacing the parent class' function pointer? Or should derived classes
store their parent's function pointer in a new ObjectClass and invoke
that? (i.e., are such parent functions supposed to be public or static)

When you talk about setting properties, does that refer to using the
property accessor functions moved to Object in series 3/4? Or should
qdev devices that currently expose only arbitrary setup functions
instead move their state to a header file so that it can be written into
by the parent? If so, should such headers reside in hw/ or include/qemu/?

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Peter Maydell]

Ping re the VMState and variable sized arrays issue. I don't
see any consensus in this discussion for a different approach,
so should we just commit Mitsyanko's patchset?

- PMM

On 31 January 2012 13:15, Andreas Färber <afaerber@suse.de> wrote:
> Am 31.01.2012 00:53, schrieb Anthony Liguori:
>> On 01/30/2012 05:41 PM, Andreas Färber wrote:
>>> Am 30.01.2012 19:55, schrieb Juan Quintela:
>>>> Please send in any agenda items you are interested in covering.
>
>>> VMState:
>>> Anthony specifically said that VMState were not affected by QOM and that
>>> patches should not be deferred until the merge. Yet there's no review
>>> and/or decision-making for a month now. Ping^2 for AHCI+SDHC.
>>
>> Do you have pointers (to pending VMState patches)?
>
> http://patchwork.ozlabs.org/patch/137732/ (PATCH v4)
>
> It's basically about how to deal with variable-sized arrays. (Alex
> mentioned it on one call around November.) I found ways to deal with
> subsets of arrays embedded within the struct and variable-sized list of
> pointers to structs but no solution for a malloc()'ed array of structs.
> Maybe I'm just too stupid to see. Anyway, no one commented since Xmas.
>
> Igor posted (and refined for v2) a patch with a callback-based approach
> that I find promising. From my view, unofficially Juan is the VMState
> guy, he's been cc'ed. Are we lacking an official maintainer that cares?
> Or is Juan the official, undocumented maintainer but simply busy?
>
> SUSE's interest is making AHCI migratable, and my VMState workaround for
> that is simply ugly:
>
> http://patchwork.ozlabs.org/patch/133066/ (RFC)
>
> Therefore I'm waiting for some resolution.
>
> Regards,
> Andreas

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Anthony Liguori]

On 02/09/2012 04:23 PM, Peter Maydell wrote:
> Ping re the VMState and variable sized arrays issue. I don't
> see any consensus in this discussion for a different approach,
> so should we just commit Mitsyanko's patchset?

I don't know if I mentioned this, but do we really need variable sizes?

Can we just use a fixed size (pre-allocated) array and then use a VMSTATE_SUB_ARRAY?

If it's truly variable size with no upper bound, then that's actually a security 
problem since it implies a guest can do unbounded memory allocation.

Regards,

Anthony Liguori

>
> - PMM
>
> On 31 January 2012 13:15, Andreas Färber<afaerber@suse.de>  wrote:
>> Am 31.01.2012 00:53, schrieb Anthony Liguori:
>>> On 01/30/2012 05:41 PM, Andreas Färber wrote:
>>>> Am 30.01.2012 19:55, schrieb Juan Quintela:
>>>>> Please send in any agenda items you are interested in covering.
>>
>>>> VMState:
>>>> Anthony specifically said that VMState were not affected by QOM and that
>>>> patches should not be deferred until the merge. Yet there's no review
>>>> and/or decision-making for a month now. Ping^2 for AHCI+SDHC.
>>>
>>> Do you have pointers (to pending VMState patches)?
>>
>> http://patchwork.ozlabs.org/patch/137732/ (PATCH v4)
>>
>> It's basically about how to deal with variable-sized arrays. (Alex
>> mentioned it on one call around November.) I found ways to deal with
>> subsets of arrays embedded within the struct and variable-sized list of
>> pointers to structs but no solution for a malloc()'ed array of structs.
>> Maybe I'm just too stupid to see. Anyway, no one commented since Xmas.
>>
>> Igor posted (and refined for v2) a patch with a callback-based approach
>> that I find promising. From my view, unofficially Juan is the VMState
>> guy, he's been cc'ed. Are we lacking an official maintainer that cares?
>> Or is Juan the official, undocumented maintainer but simply busy?
>>
>> SUSE's interest is making AHCI migratable, and my VMState workaround for
>> that is simply ugly:
>>
>> http://patchwork.ozlabs.org/patch/133066/ (RFC)
>>
>> Therefore I'm waiting for some resolution.
>>
>> Regards,
>> Andreas
>

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Peter Maydell]

On 9 February 2012 22:37, Anthony Liguori <anthony@codemonkey.ws> wrote:
> I don't know if I mentioned this, but do we really need variable sizes?
>
> Can we just use a fixed size (pre-allocated) array and then use a
> VMSTATE_SUB_ARRAY?
>
> If it's truly variable size with no upper bound, then that's actually a
> security problem since it implies a guest can do unbounded memory
> allocation.

The array in question is a property of the number of sectors on
the sd card, so it's only unbounded in the same sense that you
could pass an enormous disk image to qemu...

-- PMM

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Andreas Färber]

Am 09.02.2012 23:37, schrieb Anthony Liguori:
> On 02/09/2012 04:23 PM, Peter Maydell wrote:
>> Ping re the VMState and variable sized arrays issue. I don't
>> see any consensus in this discussion for a different approach,
>> so should we just commit Mitsyanko's patchset?
> 
> I don't know if I mentioned this, but do we really need variable sizes?

You didn't in this context. :)

I didn't write the original code so don't know what use cases beyond
ICH9 it had in mind. Alex?

> Can we just use a fixed size (pre-allocated) array and then use a
> VMSTATE_SUB_ARRAY?

my RFC did something similar. Apparently I forgot the link:

http://patchwork.ozlabs.org/patch/133065/

In the case of AHCI allocating 32 structs when we know we only need six
seems like a waste - although I agree that preallocation can be a good
thing (but that's for a different thread).

> If it's truly variable size with no upper bound, then that's actually a
> security problem since it implies a guest can do unbounded memory
> allocation.

In the cases we're talking about here, I believe the allocation is
always in the init or realize phase based on user parameters, not the guest.

And whatever we decide, we definitely need better documentation on
VMState! There's also a small part that wasn't moved to vmstate.h. I'd
volunteer for the easy macros but honestly don't understand all of them.

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Alexander Graf]

On 10.02.2012, at 00:17, Andreas Färber wrote:

> Am 09.02.2012 23:37, schrieb Anthony Liguori:
>> On 02/09/2012 04:23 PM, Peter Maydell wrote:
>>> Ping re the VMState and variable sized arrays issue. I don't
>>> see any consensus in this discussion for a different approach,
>>> so should we just commit Mitsyanko's patchset?
>> 
>> I don't know if I mentioned this, but do we really need variable sizes?
> 
> You didn't in this context. :)
> 
> I didn't write the original code so don't know what use cases beyond
> ICH9 it had in mind. Alex?

I'm having a hard time to grasp the question. AHCI is basically the spec to specific implementations. ICH9 implements AHCI, just like Nehalem implements x86. There are other chips out there that also implement AHCI.

As long as we don't need any other, I don't think it'd make sense to add more AHCI consumers, but I certainly don't want to block the road in case someone needs it. There are AHCI implementations for ARM for example. Didn't the Calxeda guys hack on something there?


Alex

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Peter Maydell]

On 9 February 2012 22:23, Peter Maydell <peter.maydell@linaro.org> wrote:
> Ping re the VMState and variable sized arrays issue. I don't
> see any consensus in this discussion for a different approach,
> so should we just commit Mitsyanko's patchset?

>From an IRC conversation I just had with Anthony and Juan:
===begin==
14:51 < pm215> quintela: do you have an opinion on the vmstate variable-
length array stuff (needed for sd card) ?
14:51 < quintela> pm havent looked, email title?
14:52 < pm215> "KVM call agenda for tuesday 31" is the most recent
discussion :-)
14:53 < pm215> http://patchwork.ozlabs.org/patch/137732/ and
 http://patchwork.ozlabs.org/patch/137733/ are the relevant patches
14:54 < quintela> pm215: found it, that it is a difficult thing to do (TM)
14:54 < quintela> it should be on the "card" file, or whatever :-(
14:55 < quintela> notice the "should" part.
14:55 < pm215> I'm not sure what you mean, can you elaborate?
14:57 < quintela> pm215: sect is number of sectors, right?
14:58 < pm215> yes
14:59 < quintela> so, a 1GB card would have around 8MB array?
14:59 < quintela> took or left some byties here andthere.
14:59 < quintela> bytes indeed.
14:59 < quintela> I _think_ that we should put that in a save_live
 section, but that is just me (TM)
15:00 < quintela> I guess that at some point, people are going to need
 bigger SD cards (16GB are already on the wild, right)
15:00 < quintela> and that would make live migration just impossible
15:00 < quintela> or very slow, that is completely equivalent.
15:01 < quintela> it is my understanding that AHCI is using similar code,
 or did I missread some of the information?
15:03 < pm215> I think alex would like ahci to use a similar 'variable
 length array' thing, but in that case the array is much smaller
15:03 < aliguori> pm215, it's large but of bounded size, right?
15:03 < aliguori> for SD cards?
15:03 < quintela> aliguori: number of sectors * 4 bytes
15:03 < quintela> aliguori: so hugeeeeeeeeeeeeeeeeee
15:04 < quintela> 8MB array for each 1GB of disk.
15:04 < quintela> but or take some bytes.
15:04 < aliguori> quintela, you cannot save that much data via savevm
15:04 < quintela> this is more than all other devices combined in a
 normal instalaltion.
15:04 < aliguori> that's too much
15:04 < quintela> aliguori: see my answer, we need a save_live section,
 really.
15:04 < aliguori> it will screw up the live migration downtime algorithm
15:04 < aliguori> pm215, ^
15:04 < aliguori> or just treat it as a ram section
15:05 < aliguori> qemu_ram_alloc() it, and call it a day
15:05 < pm215> the spec isn't very clear, but I think technically this
 info should go in the sd card image, except there's no way to tack
 additional info into a raw file
15:05 < aliguori> pm215, yeah, qemu_ram_alloc() is the way to go I think,
 that makes it effectively volatile on-card RAM
15:05 < quintela> pm215: I fully agree that it should go into the card
 image, but ..... no space for it :-(
15:05 < aliguori> which i think makes sense
15:05 < quintela> pm215: another thing, forgetting about migration at all.
15:06 < quintela> how does this work if you stop marchine and restart
 it again?
15:06  * quintela guess that it is stored somewhere?
15:06 < quintela> s/marchine/machine/
15:06 < pm215> no, we just assume that any fresh sd card image has no
 write-protect set up
15:07 < quintela> pm215: what is stored on that image? /me would have
 assumed that wearing information
15:07 < quintela> but that is without reading the whole code.
15:09 < quintela> humm, it looks like only 1 bit is used for each sector,
 why are we storing 32 bits if we only use 1 bit?
15:09 < pm215> it's write-protect : you can set parts of the sd card to
 not be writeable (with a granularity of a "write-protect group size")
15:09 < pm215> we probably don't implement fantastically efficiently
15:10 < quintela> pm215: ok, only 1 bit is needed.
15:10 < quintela> we can move to 1bit/sector (8 times smaller)
15:10 < quintela> but I still think that doing the qemu_ram_alloc()
 trick that aliguori pointed is the easiest way to fix it
15:11 < quintela> you can create a ram_save_live section, but it is
 going to be more complex for almost no gain
15:11 < pm215> ah, so we qemu_ram_alloc() it and then the contents get
 transferred in the same way as main memory ?
15:12 < pm215> ...that is in exec-obsolete.h and marked as "to be
 removed soon"...
15:13 < aliguori> pm215, yeah, so you'll need to create it using
 whatever the new fancy memory api interface is
15:13 < aliguori> pm215, note that whenever you touch that memory, you
 have to set the dirty bits appropriately
15:13 < aliguori> or else live migration won't work
15:14 < quintela> aliguori: if they have to touch the dirty bits, it is
 equivalent to do their own save_live section.
15:14 < quintela> but I agree that this is the only "easy" solution.
15:17 < pm215> doesn't sound too hard...
15:18 < quintela> as usual with vmstate, problem is testing (althought
 shouldn't be very difficult, either)
===endit===

Short summary:
 * switch wp groups to bitfield rather than int array
 * convert sd.c to use memory_region_init_ram() to allocate the wp groups
(being careful to use memory_region_set_dirty() when we touch them)
 * we don't need variable-length fields for sd.c any more
 * rest of the vmstate conversion is straightforward

-- PMM

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Mitsyanko Igor]

On 02/21/2012 07:33 PM, Peter Maydell wrote:
> Short summary:
>   * switch wp groups to bitfield rather than int array
>   * convert sd.c to use memory_region_init_ram() to allocate the wp groups
> (being careful to use memory_region_set_dirty() when we touch them)
>   * we don't need variable-length fields for sd.c any more
>   * rest of the vmstate conversion is straightforward

OK, got it.
But it's not clear to me, have you decided against introducing 
.get_bufsize completely or just for sd.c? I still think it's a good idea 
and I wouldn't mind to use it in SD host controller implementation to 
retrieve buffer size directly from capabilities register, but I can take 
another approach for this of course.
-- 
Mitsyanko Igor
ASWG, Moscow R&D center, Samsung Electronics
email: i.mitsyanko@samsung.com

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Igor Mitsyanko]

On 02/21/2012 07:33 PM, Peter Maydell wrote:
> On 9 February 2012 22:23, Peter Maydell<peter.maydell@linaro.org>  wrote:
>> Ping re the VMState and variable sized arrays issue. I don't
>> see any consensus in this discussion for a different approach,
>> so should we just commit Mitsyanko's patchset?
>
>  From an IRC conversation I just had with Anthony and Juan:
> ===begin==
> 14:51<  pm215>  quintela: do you have an opinion on the vmstate variable-
> length array stuff (needed for sd card) ?
> 14:51<  quintela>  pm havent looked, email title?
> 14:52<  pm215>  "KVM call agenda for tuesday 31" is the most recent
> discussion :-)
> 14:53<  pm215>  http://patchwork.ozlabs.org/patch/137732/ and
>   http://patchwork.ozlabs.org/patch/137733/ are the relevant patches
> 14:54<  quintela>  pm215: found it, that it is a difficult thing to do (TM)
> 14:54<  quintela>  it should be on the "card" file, or whatever :-(
> 14:55<  quintela>  notice the "should" part.
> 14:55<  pm215>  I'm not sure what you mean, can you elaborate?
> 14:57<  quintela>  pm215: sect is number of sectors, right?
> 14:58<  pm215>  yes
> 14:59<  quintela>  so, a 1GB card would have around 8MB array?
> 14:59<  quintela>  took or left some byties here andthere.
> 14:59<  quintela>  bytes indeed.
> 14:59<  quintela>  I _think_ that we should put that in a save_live
>   section, but that is just me (TM)
> 15:00<  quintela>  I guess that at some point, people are going to need
>   bigger SD cards (16GB are already on the wild, right)
> 15:00<  quintela>  and that would make live migration just impossible
> 15:00<  quintela>  or very slow, that is completely equivalent.
> 15:01<  quintela>  it is my understanding that AHCI is using similar code,
>   or did I missread some of the information?
> 15:03<  pm215>  I think alex would like ahci to use a similar 'variable
>   length array' thing, but in that case the array is much smaller
> 15:03<  aliguori>  pm215, it's large but of bounded size, right?
> 15:03<  aliguori>  for SD cards?
> 15:03<  quintela>  aliguori: number of sectors * 4 bytes
> 15:03<  quintela>  aliguori: so hugeeeeeeeeeeeeeeeeee
> 15:04<  quintela>  8MB array for each 1GB of disk.
> 15:04<  quintela>  but or take some bytes.
> 15:04<  aliguori>  quintela, you cannot save that much data via savevm
> 15:04<  quintela>  this is more than all other devices combined in a
>   normal instalaltion.
> 15:04<  aliguori>  that's too much
> 15:04<  quintela>  aliguori: see my answer, we need a save_live section,
>   really.
> 15:04<  aliguori>  it will screw up the live migration downtime algorithm
> 15:04<  aliguori>  pm215, ^
> 15:04<  aliguori>  or just treat it as a ram section
> 15:05<  aliguori>  qemu_ram_alloc() it, and call it a day
> 15:05<  pm215>  the spec isn't very clear, but I think technically this
>   info should go in the sd card image, except there's no way to tack
>   additional info into a raw file
> 15:05<  aliguori>  pm215, yeah, qemu_ram_alloc() is the way to go I think,
>   that makes it effectively volatile on-card RAM
> 15:05<  quintela>  pm215: I fully agree that it should go into the card
>   image, but ..... no space for it :-(
> 15:05<  aliguori>  which i think makes sense
> 15:05<  quintela>  pm215: another thing, forgetting about migration at all.
> 15:06<  quintela>  how does this work if you stop marchine and restart
>   it again?
> 15:06  * quintela guess that it is stored somewhere?
> 15:06<  quintela>  s/marchine/machine/
> 15:06<  pm215>  no, we just assume that any fresh sd card image has no
>   write-protect set up
> 15:07<  quintela>  pm215: what is stored on that image? /me would have
>   assumed that wearing information
> 15:07<  quintela>  but that is without reading the whole code.
> 15:09<  quintela>  humm, it looks like only 1 bit is used for each sector,
>   why are we storing 32 bits if we only use 1 bit?
> 15:09<  pm215>  it's write-protect : you can set parts of the sd card to
>   not be writeable (with a granularity of a "write-protect group size")
> 15:09<  pm215>  we probably don't implement fantastically efficiently
> 15:10<  quintela>  pm215: ok, only 1 bit is needed.
> 15:10<  quintela>  we can move to 1bit/sector (8 times smaller)
> 15:10<  quintela>  but I still think that doing the qemu_ram_alloc()
>   trick that aliguori pointed is the easiest way to fix it
> 15:11<  quintela>  you can create a ram_save_live section, but it is
>   going to be more complex for almost no gain
> 15:11<  pm215>  ah, so we qemu_ram_alloc() it and then the contents get
>   transferred in the same way as main memory ?
> 15:12<  pm215>  ...that is in exec-obsolete.h and marked as "to be
>   removed soon"...
> 15:13<  aliguori>  pm215, yeah, so you'll need to create it using
>   whatever the new fancy memory api interface is
> 15:13<  aliguori>  pm215, note that whenever you touch that memory, you
>   have to set the dirty bits appropriately
> 15:13<  aliguori>  or else live migration won't work
> 15:14<  quintela>  aliguori: if they have to touch the dirty bits, it is
>   equivalent to do their own save_live section.
> 15:14<  quintela>  but I agree that this is the only "easy" solution.
> 15:17<  pm215>  doesn't sound too hard...
> 15:18<  quintela>  as usual with vmstate, problem is testing (althought
>   shouldn't be very difficult, either)
> ===endit===
>
> Short summary:
>   * switch wp groups to bitfield rather than int array
>   * convert sd.c to use memory_region_init_ram() to allocate the wp groups
> (being careful to use memory_region_set_dirty() when we touch them)
>   * we don't need variable-length fields for sd.c any more
>   * rest of the vmstate conversion is straightforward
>
> -- PMM
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

OK, it turned out to be not so simple, we can't use memory API in sd.c 
because TARGET_PHYS_ADDR_BITS value (and, consequently, 
target_phys_addr_t) is not defined for common objects.

-- 
Mitsyanko Igor
ASWG, Moscow R&D center, Samsung Electronics
email: i.mitsyanko@samsung.com

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Avi Kivity]

On 03/05/2012 03:38 PM, Igor Mitsyanko wrote:
>> Short summary:
>>   * switch wp groups to bitfield rather than int array
>>   * convert sd.c to use memory_region_init_ram() to allocate the wp
>> groups
>> (being careful to use memory_region_set_dirty() when we touch them)
>>   * we don't need variable-length fields for sd.c any more
>>   * rest of the vmstate conversion is straightforward
>>
>
> OK, it turned out to be not so simple, we can't use memory API in sd.c
> because TARGET_PHYS_ADDR_BITS value (and, consequently,
> target_phys_addr_t) is not defined for common objects.
>

Well, can't you make sd.c target dependent?  It's not so nice, but it
does solve the problem.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Igor Mitsyanko]

On 03/05/2012 06:13 PM, Avi Kivity wrote:
> On 03/05/2012 03:38 PM, Igor Mitsyanko wrote:
>>> Short summary:
>>>    * switch wp groups to bitfield rather than int array
>>>    * convert sd.c to use memory_region_init_ram() to allocate the wp
>>> groups
>>> (being careful to use memory_region_set_dirty() when we touch them)
>>>    * we don't need variable-length fields for sd.c any more
>>>    * rest of the vmstate conversion is straightforward
>>>
>>
>> OK, it turned out to be not so simple, we can't use memory API in sd.c
>> because TARGET_PHYS_ADDR_BITS value (and, consequently,
>> target_phys_addr_t) is not defined for common objects.
>>
>
> Well, can't you make sd.c target dependent?  It's not so nice, but it
> does solve the problem.
>
OK, but it will turn qemu from it's "long term path to suppress *all* 
target specific code" :)

-- 
Mitsyanko Igor
ASWG, Moscow R&D center, Samsung Electronics
email: i.mitsyanko@samsung.com

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Avi Kivity]

On 03/05/2012 04:37 PM, Igor Mitsyanko wrote:
>> Well, can't you make sd.c target dependent?  It's not so nice, but it
>> does solve the problem.
>>
>
> OK, but it will turn qemu from it's "long term path to suppress *all*
> target specific code" :)
>

The other alternative is to s/target_phys_addr_t/uint64_t/ in the memory
API.  I think 32-on-32 is quite rare these days, so it wouldn't be much
of a performance issue.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Anthony Liguori]

On 03/05/2012 09:10 AM, Avi Kivity wrote:
> On 03/05/2012 04:37 PM, Igor Mitsyanko wrote:
>>> Well, can't you make sd.c target dependent?  It's not so nice, but it
>>> does solve the problem.
>>>
>>
>> OK, but it will turn qemu from it's "long term path to suppress *all*
>> target specific code" :)
>>
>
> The other alternative is to s/target_phys_addr_t/uint64_t/ in the memory
> API.  I think 32-on-32 is quite rare these days, so it wouldn't be much
> of a performance issue.

I think this makes sense independent of other discussions regarding fixing 
target_phys_addr_t size.

Hardware addresses should be independent of the target.  If we wanted to use a 
hw_addr_t that would be okay too.

Regards,

Anthony Liguori

>

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Avi Kivity]

On 03/05/2012 05:15 PM, Anthony Liguori wrote:
>> The other alternative is to s/target_phys_addr_t/uint64_t/ in the memory
>> API.  I think 32-on-32 is quite rare these days, so it wouldn't be much
>> of a performance issue.
>
>
> I think this makes sense independent of other discussions regarding
> fixing target_phys_addr_t size.
>
> Hardware addresses should be independent of the target.  If we wanted
> to use a hw_addr_t that would be okay too.
>

Would this hw_addr (s/_t$//, or you'll be Blued) be fixed at uint64_t
(and thus only documentary), or also subject to multiple compilation?

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Peter Maydell]

On 5 March 2012 15:10, Avi Kivity <avi@redhat.com> wrote:
> I think 32-on-32 is quite rare these days, so it wouldn't be much
> of a performance issue.

32-on-32 will be the standard case for KVM on ARM I think...

-- PMM

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Avi Kivity]

On 03/05/2012 05:20 PM, Peter Maydell wrote:
> On 5 March 2012 15:10, Avi Kivity <avi@redhat.com> wrote:
> > I think 32-on-32 is quite rare these days, so it wouldn't be much
> > of a performance issue.
>
> 32-on-32 will be the standard case for KVM on ARM I think...

Won't we be virtualizing LPAE per default?

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Peter Maydell]

On 5 March 2012 15:21, Avi Kivity <avi@redhat.com> wrote:
> On 03/05/2012 05:20 PM, Peter Maydell wrote:
>> On 5 March 2012 15:10, Avi Kivity <avi@redhat.com> wrote:
>> > I think 32-on-32 is quite rare these days, so it wouldn't be much
>> > of a performance issue.
>>
>> 32-on-32 will be the standard case for KVM on ARM I think...
>
> Won't we be virtualizing LPAE per default?

Mmm, I guess that would give you 64-on-32.

-- PMM

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Andreas Färber]

Am 05.03.2012 16:10, schrieb Avi Kivity:
> On 03/05/2012 04:37 PM, Igor Mitsyanko wrote:
>>> Well, can't you make sd.c target dependent?  It's not so nice, but it
>>> does solve the problem.
>>>
>>
>> OK, but it will turn qemu from it's "long term path to suppress *all*
>> target specific code" :)
>>
> 
> The other alternative is to s/target_phys_addr_t/uint64_t/ in the memory
> API.  I think 32-on-32 is quite rare these days, so it wouldn't be much
> of a performance issue.

Maybe rare, but 32-bit ARM netbooks and tablets are gaining marketshare.

Mid-term also depends on how me want to proceed with LPAE softmmu-wise
(bump "arm" to 64-bit target_phys_addr_t, or do LPAE and AArch64 in a
new "arm64").

i386 is 64-on-32 these days already; most of the embedded targets are
still at most 32-bit though (xtensa, mblaze, ...).

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Avi Kivity]

On 03/05/2012 05:43 PM, Andreas Färber wrote:
> Am 05.03.2012 16:10, schrieb Avi Kivity:
> > On 03/05/2012 04:37 PM, Igor Mitsyanko wrote:
> >>> Well, can't you make sd.c target dependent?  It's not so nice, but it
> >>> does solve the problem.
> >>>
> >>
> >> OK, but it will turn qemu from it's "long term path to suppress *all*
> >> target specific code" :)
> >>
> > 
> > The other alternative is to s/target_phys_addr_t/uint64_t/ in the memory
> > API.  I think 32-on-32 is quite rare these days, so it wouldn't be much
> > of a performance issue.
>
> Maybe rare, but 32-bit ARM netbooks and tablets are gaining marketshare.
>
> Mid-term also depends on how me want to proceed with LPAE softmmu-wise
> (bump "arm" to 64-bit target_phys_addr_t, or do LPAE and AArch64 in a
> new "arm64").

I was counting on LPAE to make 32-on-32 rare.

> i386 is 64-on-32 these days already; most of the embedded targets are
> still at most 32-bit though (xtensa, mblaze, ...).

These would be 32-on-64, since the host would usually be x86.  I guess
it would be even more true when the w64 port is complete.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Peter Maydell]

On 5 March 2012 15:43, Andreas Färber <afaerber@suse.de> wrote:
> Mid-term also depends on how me want to proceed with LPAE softmmu-wise
> (bump "arm" to 64-bit target_phys_addr_t, or do LPAE and AArch64 in a
> new "arm64").

For LPAE I would have thought we want to make "arm" go to a 64 bit
target_phys_addr_t, since that's exactly what it is: same old
ARM architecture with wider physical addresses :-)

I notice that for the architectures we currently have that have
32 and 64 bit versions we have separate {i386,x86_64}-softmmu,
{ppc,ppc64}-softmmu, {mips,mips64}-softmmu. What's the advantage
of separating out the 64 bit flavours that way rather than
having everything be a single binary?

-- PMM

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Avi Kivity]

On 03/05/2012 05:50 PM, Peter Maydell wrote:
> On 5 March 2012 15:43, Andreas Färber <afaerber@suse.de> wrote:
> > Mid-term also depends on how me want to proceed with LPAE softmmu-wise
> > (bump "arm" to 64-bit target_phys_addr_t, or do LPAE and AArch64 in a
> > new "arm64").
>
> For LPAE I would have thought we want to make "arm" go to a 64 bit
> target_phys_addr_t, since that's exactly what it is: same old
> ARM architecture with wider physical addresses :-)
>
> I notice that for the architectures we currently have that have
> 32 and 64 bit versions we have separate {i386,x86_64}-softmmu,
> {ppc,ppc64}-softmmu, {mips,mips64}-softmmu. What's the advantage
> of separating out the 64 bit flavours that way rather than
> having everything be a single binary?

The registers are smaller; if target_ulong fits in a long then
everything is faster.

Although, you could pretend that target_ulong is 32-bit when in 32-bit
mode, and zero the high half when switching modes, if the target allows
it (I believe i386->x86_64 does, but 8086->i386 does not).

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Blue Swirl]

On Mon, Mar 5, 2012 at 15:17, Avi Kivity <avi@redhat.com> wrote:
> On 03/05/2012 05:15 PM, Anthony Liguori wrote:
>>> The other alternative is to s/target_phys_addr_t/uint64_t/ in the memory
>>> API.  I think 32-on-32 is quite rare these days, so it wouldn't be much
>>> of a performance issue.
>>
>>
>> I think this makes sense independent of other discussions regarding
>> fixing target_phys_addr_t size.
>>
>> Hardware addresses should be independent of the target.  If we wanted
>> to use a hw_addr_t that would be okay too.
>>
>
> Would this hw_addr (s/_t$//, or you'll be Blued) be fixed at uint64_t

Malced? Posixed?

> (and thus only documentary), or also subject to multiple compilation?

In real world CPU physical addresses, bus addresses and device
addresses need not have anything in common. The best would be if we
could have devices with 10-bit addresses mixing freely with 32 bit
buses and 36 bit CPU physical addresses. The next best thing probably
is to fix all of them to shortest possible reasonable value, like now.
Fixing all of them to 64 bits would simplify things a lot if we no
longer care about the small performance loss on 32 bit hosts.

> --
> error compiling committee.c: too many arguments to function
>
>

Re: [Qemu-devel] KVM call agenda for tuesday 31

[malc]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 780 bytes --]

On Mon, 5 Mar 2012, Blue Swirl wrote:

> On Mon, Mar 5, 2012 at 15:17, Avi Kivity <avi@redhat.com> wrote:
> > On 03/05/2012 05:15 PM, Anthony Liguori wrote:
> >>> The other alternative is to s/target_phys_addr_t/uint64_t/ in the memory
> >>> API.  I think 32-on-32 is quite rare these days, so it wouldn't be much
> >>> of a performance issue.
> >>
> >>
> >> I think this makes sense independent of other discussions regarding
> >> fixing target_phys_addr_t size.
> >>
> >> Hardware addresses should be independent of the target.  If we wanted
> >> to use a hw_addr_t that would be okay too.
> >>
> >
> > Would this hw_addr (s/_t$//, or you'll be Blued) be fixed at uint64_t
> 
> Malced? Posixed?

Heh, a_moo would be Malced, no _t is Posixed indeed.

-- 
mailto:av1474@comtv.ru

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Igor Mitsyanko]

On 02/21/2012 07:33 PM, Peter Maydell wrote:
>
> Short summary:
>   * switch wp groups to bitfield rather than int array
>   * convert sd.c to use memory_region_init_ram() to allocate the wp groups
> (being careful to use memory_region_set_dirty() when we touch them)
>   * we don't need variable-length fields for sd.c any more
>   * rest of the vmstate conversion is straightforward
>

After a conversion to bitfield wp_groups consumes 2048 bytes for 32 GB 
SD image, do we really need live section for this?

-- 
Mitsyanko Igor
ASWG, Moscow R&D center, Samsung Electronics
email: i.mitsyanko@samsung.com

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Igor Mitsyanko]

On 03/12/2012 01:43 PM, Igor Mitsyanko wrote:
> On 02/21/2012 07:33 PM, Peter Maydell wrote:
>>
>> Short summary:
>> * switch wp groups to bitfield rather than int array
>> * convert sd.c to use memory_region_init_ram() to allocate the wp groups
>> (being careful to use memory_region_set_dirty() when we touch them)
>> * we don't need variable-length fields for sd.c any more
>> * rest of the vmstate conversion is straightforward
>>
>
> After a conversion to bitfield wp_groups consumes 2048 bytes for 32 GB
> SD image, do we really need live section for this?
>

I don't know how large is too large, but currently we have

VMSTATE_UINT32_ARRAY(tx_status_fifo, lan9118_state, 512),
VMSTATE_UINT32_ARRAY(rx_status_fifo, lan9118_state, 896),
VMSTATE_UINT32_ARRAY(rx_fifo, lan9118_state, 3360),
VMSTATE_INT32_ARRAY(rx_packet_size, lan9118_state, 1024),

in hw/lan9118.c
-- 
Mitsyanko Igor
ASWG, Moscow R&D center, Samsung Electronics
email: i.mitsyanko@samsung.com

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Avi Kivity]

On 03/12/2012 11:43 AM, Igor Mitsyanko wrote:
> On 02/21/2012 07:33 PM, Peter Maydell wrote:
>>
>> Short summary:
>>   * switch wp groups to bitfield rather than int array
>>   * convert sd.c to use memory_region_init_ram() to allocate the wp
>> groups
>> (being careful to use memory_region_set_dirty() when we touch them)
>>   * we don't need variable-length fields for sd.c any more
>>   * rest of the vmstate conversion is straightforward
>>
>
> After a conversion to bitfield wp_groups consumes 2048 bytes for 32 GB
> SD image, do we really need live section for this?
>

2K is certainly okay, some processors hold more state.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Andreas Färber]

Am 31.01.2012 15:01, schrieb Mitsyanko Igor:
> On 01/31/2012 05:15 PM, Andreas Färber wrote:
>> Am 31.01.2012 00:53, schrieb Anthony Liguori:
>>> On 01/30/2012 05:41 PM, Andreas Färber wrote:
>>>> Am 30.01.2012 19:55, schrieb Juan Quintela:
>>>>> Please send in any agenda items you are interested in covering.
>>
>>>> VMState:
>>>> Anthony specifically said that VMState were not affected by QOM and
>>>> that
>>>> patches should not be deferred until the merge. Yet there's no review
>>>> and/or decision-making for a month now. Ping^2 for AHCI+SDHC.
>>>
>>> Do you have pointers (to pending VMState patches)?
>>
>> http://patchwork.ozlabs.org/patch/137732/ (PATCH v4)
>>
>> It's basically about how to deal with variable-sized arrays. (Alex
>> mentioned it on one call around November.) I found ways to deal with
>> subsets of arrays embedded within the struct and variable-sized list of
>> pointers to structs but no solution for a malloc()'ed array of structs.
>> Maybe I'm just too stupid to see. Anyway, no one commented since Xmas.
>>
>> Igor posted (and refined for v2) a patch with a callback-based approach
>> that I find promising. From my view, unofficially Juan is the VMState
>> guy, he's been cc'ed. Are we lacking an official maintainer that cares?
>> Or is Juan the official, undocumented maintainer but simply busy?
>>
>> SUSE's interest is making AHCI migratable, and my VMState workaround for
>> that is simply ugly:
>>
>> http://patchwork.ozlabs.org/patch/133066/ (RFC)
>>
> 
> If I'm not mistaken, if you change AHCIState's ".ports" type to uint32_t
> you can use existing VMSTATE_BUFFER_MULTIPLY macro like this:
> 
> VMSTATE_BUFFER_MULTIPLY(dev, AHCIState, 0, NULL, 0, ports,
> sizeof(AHCIDevice))

Igor, I finally got around to rebasing and trying this: Am I seeing
correctly that this tries to serialize the whole of AHCIDevice as an
opaque buffer? The difficulty here was that we were looking for a way to
serialize a variable number of structured elements with their own
&vmstate_ahci_device specifying what fields to serialize.

Juan, how should we proceed there? Do as Igor suggested? Some VMSTATE_
macro I'm overlooking? Or do we need some new macro for this use case?

Regards,
Andreas

> VMSTATE_BUFFER_MULTIPLY currently lacks VMS_POINTER flag and therefore
> doesn't make any use of _start field (you don't need it anyway)
> 
> Nevertheless, VMSTATE_BUFFER_MULTIPLY is just a partial solution to a
> bigger set of possible problems. SD card's vmstate implementation
> requires shift operation, SDHC gets size from switch {} statement,
> something else later may require division or addition e.t.c.,
> get_bufsize callback will cover all possible cases.

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] KVM call agenda for tuesday 31

[Igor Mitsyanko]

On 08/08/2012 08:25 PM, Andreas Färber wrote:
> Am 31.01.2012 15:01, schrieb Mitsyanko Igor:
>> On 01/31/2012 05:15 PM, Andreas Färber wrote:
>>> Am 31.01.2012 00:53, schrieb Anthony Liguori:
>>>> On 01/30/2012 05:41 PM, Andreas Färber wrote:
>>>>> Am 30.01.2012 19:55, schrieb Juan Quintela:
>>>>>> Please send in any agenda items you are interested in covering.
>>>>> VMState:
>>>>> Anthony specifically said that VMState were not affected by QOM and
>>>>> that
>>>>> patches should not be deferred until the merge. Yet there's no review
>>>>> and/or decision-making for a month now. Ping^2 for AHCI+SDHC.
>>>> Do you have pointers (to pending VMState patches)?
>>> http://patchwork.ozlabs.org/patch/137732/ (PATCH v4)
>>>
>>> It's basically about how to deal with variable-sized arrays. (Alex
>>> mentioned it on one call around November.) I found ways to deal with
>>> subsets of arrays embedded within the struct and variable-sized list of
>>> pointers to structs but no solution for a malloc()'ed array of structs.
>>> Maybe I'm just too stupid to see. Anyway, no one commented since Xmas.
>>>
>>> Igor posted (and refined for v2) a patch with a callback-based approach
>>> that I find promising. From my view, unofficially Juan is the VMState
>>> guy, he's been cc'ed. Are we lacking an official maintainer that cares?
>>> Or is Juan the official, undocumented maintainer but simply busy?
>>>
>>> SUSE's interest is making AHCI migratable, and my VMState workaround for
>>> that is simply ugly:
>>>
>>> http://patchwork.ozlabs.org/patch/133066/ (RFC)
>>>
>> If I'm not mistaken, if you change AHCIState's ".ports" type to uint32_t
>> you can use existing VMSTATE_BUFFER_MULTIPLY macro like this:
>>
>> VMSTATE_BUFFER_MULTIPLY(dev, AHCIState, 0, NULL, 0, ports,
>> sizeof(AHCIDevice))
> Igor, I finally got around to rebasing and trying this: Am I seeing
> correctly that this tries to serialize the whole of AHCIDevice as an
> opaque buffer? The difficulty here was that we were looking for a way to
> serialize a variable number of structured elements with their own
> &vmstate_ahci_device specifying what fields to serialize.
>
> Juan, how should we proceed there? Do as Igor suggested? Some VMSTATE_
> macro I'm overlooking? Or do we need some new macro for this use case?
>
> Regards,
> Andreas

Hi Andreas, that was a bad suggestion, sorry for that, we can't treat 
structures as opaque buffers because that wouldn't work when migrating 
between hosts with different alignment/endianess.
Perhaps you can use VMSTATE_STRUCT_VARRAY_UINT32 for this, if I'm not 
again mistaken.

>> VMSTATE_BUFFER_MULTIPLY currently lacks VMS_POINTER flag and therefore
>> doesn't make any use of _start field (you don't need it anyway)
>>
>> Nevertheless, VMSTATE_BUFFER_MULTIPLY is just a partial solution to a
>> bigger set of possible problems. SD card's vmstate implementation
>> requires shift operation, SDHC gets size from switch {} statement,
>> something else later may require division or addition e.t.c.,
>> get_bufsize callback will cover all possible cases.