[Qemu-devel] steps towards deprecation of old boards and devices

[Qemu-devel] steps towards deprecation of old boards and devices

[Peter Maydell]

If we're going to aim for deprecating and eventually removing
some of our unmaintained device and board models, it seems to
me that a good first step would be to come up with a definition
of what our baseline "needs to be at least this good" level is.
I'm guessing that ought to include at least "devices are QOM"
and "uses vmstate rather than save/load functions".

So
(a) are there any other things we want to include?
(b) does anybody feel like writing up a helpful wiki page
on how to update old devices, that we can point prospective
maintainers at?

(In particular I would appreciate the documentation on how to
write a state-of-the-art QOMified device as I don't really have
a good idea myself...)

thanks
-- PMM

Re: [Qemu-devel] steps towards deprecation of old boards and devices

[Markus Armbruster]

Peter Maydell <peter.maydell@linaro.org> writes:

> If we're going to aim for deprecating and eventually removing
> some of our unmaintained device and board models, it seems to
> me that a good first step would be to come up with a definition
> of what our baseline "needs to be at least this good" level is.
> I'm guessing that ought to include at least "devices are QOM"
> and "uses vmstate rather than save/load functions".

Sounds like a start.  We can always refine.

Qdevified devices that aren't fully QOMified are reasonably easy to
find: search for init() and exit() methods.

Non-qdevified devices are harder to find.  Anything that maps memory or
wires up interrupts might be a device.  If it's done outside QOM
realize(), chances are it's either wrong or legacy crap.

In my opinion, legacy crap is much more tolerable when it doesn't have
any configuration knobs.  See also below.

> So
> (a) are there any other things we want to include?

A few ideas:

* Anything configurable needs to be configurable with non-legacy means:
  -machine, -device.

  Counter-example: a board has serial devices that can only be
  configured with -serial.  Hmm, almost certainly covered by "devices
  are QOM" already, but it may still be a useful approach to finding
  problematic stuff that is actually relevant.

* A smoke test exists: can boot at least into firmware with generally
  available bits.  Ideally, the bits are in tree, and the smoke test is
  run by "make check".  Perhaps too ambitious for the first round, but I
  think it makes sense.

* A maintainer exists (d'oh): the machine initialization function is
  covered by MAINTAINERS.

> (b) does anybody feel like writing up a helpful wiki page
> on how to update old devices, that we can point prospective
> maintainers at?
>
> (In particular I would appreciate the documentation on how to
> write a state-of-the-art QOMified device as I don't really have
> a good idea myself...)

I guess the first step is identifying good examples, and examples of
stuff that needs work.

Paolo, Andreas, can you point us to some reasonably QOMified devices?

Re: [Qemu-devel] steps towards deprecation of old boards and devices

[Paolo Bonzini]

On 20/09/2016 10:08, Markus Armbruster wrote:
> Peter Maydell <peter.maydell@linaro.org> writes:
> 
>> If we're going to aim for deprecating and eventually removing
>> some of our unmaintained device and board models, it seems to
>> me that a good first step would be to come up with a definition
>> of what our baseline "needs to be at least this good" level is.
>> I'm guessing that ought to include at least "devices are QOM"
>> and "uses vmstate rather than save/load functions".
> 
> Sounds like a start.  We can always refine.
> 
> Qdevified devices that aren't fully QOMified are reasonably easy to
> find: search for init() and exit() methods.

This is not a big deal.  Devices need to be classes, that's enough.

>> So (a) are there any other things we want to include?
> 
> A few ideas:
> 
> * Anything configurable needs to be configurable with non-legacy means:
>   -machine, -device.
> 
>   Counter-example: a board has serial devices that can only be
>   configured with -serial.  Hmm, almost certainly covered by "devices
>   are QOM" already, but it may still be a useful approach to finding
>   problematic stuff that is actually relevant.

I think -serial and -net configuration is so pervasive that we have to
pass on this requirement.

>> (b) does anybody feel like writing up a helpful wiki page
>> on how to update old devices, that we can point prospective
>> maintainers at?
>>
>> (In particular I would appreciate the documentation on how to
>> write a state-of-the-art QOMified device as I don't really have
>> a good idea myself...)
> 
> I guess the first step is identifying good examples, and examples of
> stuff that needs work.
> 
> Paolo, Andreas, can you point us to some reasonably QOMified devices?

The Raspberry Pi board is probably one of the best examples.  Its only
snag is that one of the devices (hw/arm/bcm2835_peripherals.c) uses
serial_hds[].

Paolo

Re: [Qemu-devel] steps towards deprecation of old boards and devices

[Markus Armbruster]

Paolo Bonzini <pbonzini@redhat.com> writes:

> On 20/09/2016 10:08, Markus Armbruster wrote:
>> Peter Maydell <peter.maydell@linaro.org> writes:
>> 
>>> If we're going to aim for deprecating and eventually removing
>>> some of our unmaintained device and board models, it seems to
>>> me that a good first step would be to come up with a definition
>>> of what our baseline "needs to be at least this good" level is.
>>> I'm guessing that ought to include at least "devices are QOM"
>>> and "uses vmstate rather than save/load functions".
>> 
>> Sounds like a start.  We can always refine.
>> 
>> Qdevified devices that aren't fully QOMified are reasonably easy to
>> find: search for init() and exit() methods.
>
> This is not a big deal.  Devices need to be classes, that's enough.

The trouble with init() methods is how they fail: error_report() &
return -1.  Sucks for QMP: the specific error message from
error_report() goes to stderr, and QMP is left with a rather useless
"Device initialization failed" error.

That's still not a big deal when init() can't fail, but then init() is
trivial to convert to realize(), and I think we got most of those by
now.

>>> So (a) are there any other things we want to include?
>> 
>> A few ideas:
>> 
>> * Anything configurable needs to be configurable with non-legacy means:
>>   -machine, -device.
>> 
>>   Counter-example: a board has serial devices that can only be
>>   configured with -serial.  Hmm, almost certainly covered by "devices
>>   are QOM" already, but it may still be a useful approach to finding
>>   problematic stuff that is actually relevant.
>
> I think -serial and -net configuration is so pervasive that we have to
> pass on this requirement.

I'm not proposing to get rid of -serial.  I'm proposing to use it as
indicator of old code in need of modernization.  A properly QOMified
serial device should be "configurable with non-legacy means".  Devices
that aren't are probably not QOMified.

Of course, we'll want to be pragmatic.  If half of the boards have a
certain kind of problem, making QEMU threaten deprecation isn't
credible.  It's not even useful.  We'll have to work with active
maintainers to reduce the scope of the problem first.  And for that, we
need to gauge the scope.

>>> (b) does anybody feel like writing up a helpful wiki page
>>> on how to update old devices, that we can point prospective
>>> maintainers at?
>>>
>>> (In particular I would appreciate the documentation on how to
>>> write a state-of-the-art QOMified device as I don't really have
>>> a good idea myself...)
>> 
>> I guess the first step is identifying good examples, and examples of
>> stuff that needs work.
>> 
>> Paolo, Andreas, can you point us to some reasonably QOMified devices?
>
> The Raspberry Pi board is probably one of the best examples.  Its only
> snag is that one of the devices (hw/arm/bcm2835_peripherals.c) uses
> serial_hds[].

Yes, but it uses it to configure QOMified devices, doesn't it?

It should be possible to configure these with non-legacy means, at least
in theory.

Re: [Qemu-devel] steps towards deprecation of old boards and devices

[Andreas Färber]

Am 20.09.2016 um 10:08 schrieb Markus Armbruster:
> Peter Maydell <peter.maydell@linaro.org> writes:
> 
>> If we're going to aim for deprecating and eventually removing
>> some of our unmaintained device and board models, it seems to
>> me that a good first step would be to come up with a definition
>> of what our baseline "needs to be at least this good" level is.
>> I'm guessing that ought to include at least "devices are QOM"
>> and "uses vmstate rather than save/load functions".
> 
> Sounds like a start.  We can always refine.
> 
> Qdevified devices that aren't fully QOMified are reasonably easy to
> find: search for init() and exit() methods.
> 
> Non-qdevified devices are harder to find.  Anything that maps memory or
> wires up interrupts might be a device.  If it's done outside QOM
> realize(), chances are it's either wrong or legacy crap.
> 
> In my opinion, legacy crap is much more tolerable when it doesn't have
> any configuration knobs.  See also below.
> 
>> So
>> (a) are there any other things we want to include?
> 
> A few ideas:
> 
> * Anything configurable needs to be configurable with non-legacy means:
>   -machine, -device.
> 
>   Counter-example: a board has serial devices that can only be
>   configured with -serial.  Hmm, almost certainly covered by "devices
>   are QOM" already, but it may still be a useful approach to finding
>   problematic stuff that is actually relevant.
> 
> * A smoke test exists: can boot at least into firmware with generally
>   available bits.  Ideally, the bits are in tree, and the smoke test is
>   run by "make check".  Perhaps too ambitious for the first round, but I
>   think it makes sense.
> 
> * A maintainer exists (d'oh): the machine initialization function is
>   covered by MAINTAINERS.
> 
>> (b) does anybody feel like writing up a helpful wiki page
>> on how to update old devices, that we can point prospective
>> maintainers at?
>>
>> (In particular I would appreciate the documentation on how to
>> write a state-of-the-art QOMified device as I don't really have
>> a good idea myself...)
> 
> I guess the first step is identifying good examples, and examples of
> stuff that needs work.
> 
> Paolo, Andreas, can you point us to some reasonably QOMified devices?

I see Paolo already replied, so just a few more comments.

(Reminds me that I still have some ColdFire QOM conversions from a KVM
Forum session...)

If you want to replace some of the legacy command line options, we need
to finish the work of defining named paths from /machine. Only then (and
when giving all user devices IDs for /machine/peripheral) can you
realistically use qom-set operations for tweaking things in a new way.

Another aspect is that most properties can't be changed any more after
the device is realized. So I would need to finish the deferred
(recursive) realization patchset, for which ordering concerns remained -
we wanted to generate a list of to-be-realized devices and sort them
before starting the realization.

Don't assume PC behavior where -device serial-pci magically replaces the
default device with your customized one, the serial device may be hidden
beneath a Super I/O chipset like on PReP or inside a SoC device.

Similarly we used to have ARM machines where the new -netdev stuff
couldn't be used because the device doesn't get user-added.

One idea once was to extend the by-ID-reference semantics to allow a QOM
path as transparent means of conversion. I don't think we ever
implemented that?

Regards,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

Re: [Qemu-devel] steps towards deprecation of old boards and devices

[Paolo Bonzini]

On 20/09/2016 13:04, Markus Armbruster wrote:
> I'm not proposing to get rid of -serial.  I'm proposing to use it as
> indicator of old code in need of modernization.  A properly QOMified
> serial device should be "configurable with non-legacy means".  Devices
> that aren't are probably not QOMified. [...]
> 
> > The Raspberry Pi board is probably one of the best examples.  Its only
> > snag is that one of the devices (hw/arm/bcm2835_peripherals.c) uses
> > serial_hds[].
> 
> Yes, but it uses it to configure QOMified devices, doesn't it?
> It should be possible to configure these with non-legacy means, at least
> in theory.

The devices are QOMified, but because there is more than one you cannot
hack their configuration with -global.

Paolo

Re: [Qemu-devel] steps towards deprecation of old boards and devices

[Markus Armbruster]

Andreas Färber <afaerber@suse.de> writes:

> Am 20.09.2016 um 10:08 schrieb Markus Armbruster:
>> Peter Maydell <peter.maydell@linaro.org> writes:
>> 
>>> If we're going to aim for deprecating and eventually removing
>>> some of our unmaintained device and board models, it seems to
>>> me that a good first step would be to come up with a definition
>>> of what our baseline "needs to be at least this good" level is.
>>> I'm guessing that ought to include at least "devices are QOM"
>>> and "uses vmstate rather than save/load functions".
>> 
>> Sounds like a start.  We can always refine.
>> 
>> Qdevified devices that aren't fully QOMified are reasonably easy to
>> find: search for init() and exit() methods.
>> 
>> Non-qdevified devices are harder to find.  Anything that maps memory or
>> wires up interrupts might be a device.  If it's done outside QOM
>> realize(), chances are it's either wrong or legacy crap.
>> 
>> In my opinion, legacy crap is much more tolerable when it doesn't have
>> any configuration knobs.  See also below.
>> 
>>> So
>>> (a) are there any other things we want to include?
>> 
>> A few ideas:
>> 
>> * Anything configurable needs to be configurable with non-legacy means:
>>   -machine, -device.
>> 
>>   Counter-example: a board has serial devices that can only be
>>   configured with -serial.  Hmm, almost certainly covered by "devices
>>   are QOM" already, but it may still be a useful approach to finding
>>   problematic stuff that is actually relevant.
>> 
>> * A smoke test exists: can boot at least into firmware with generally
>>   available bits.  Ideally, the bits are in tree, and the smoke test is
>>   run by "make check".  Perhaps too ambitious for the first round, but I
>>   think it makes sense.
>> 
>> * A maintainer exists (d'oh): the machine initialization function is
>>   covered by MAINTAINERS.
>> 
>>> (b) does anybody feel like writing up a helpful wiki page
>>> on how to update old devices, that we can point prospective
>>> maintainers at?
>>>
>>> (In particular I would appreciate the documentation on how to
>>> write a state-of-the-art QOMified device as I don't really have
>>> a good idea myself...)
>> 
>> I guess the first step is identifying good examples, and examples of
>> stuff that needs work.
>> 
>> Paolo, Andreas, can you point us to some reasonably QOMified devices?
>
> I see Paolo already replied, so just a few more comments.
>
> (Reminds me that I still have some ColdFire QOM conversions from a KVM
> Forum session...)
>
> If you want to replace some of the legacy command line options, we need
> to finish the work of defining named paths from /machine. Only then (and
> when giving all user devices IDs for /machine/peripheral) can you
> realistically use qom-set operations for tweaking things in a new way.
>
> Another aspect is that most properties can't be changed any more after
> the device is realized. So I would need to finish the deferred
> (recursive) realization patchset, for which ordering concerns remained -
> we wanted to generate a list of to-be-realized devices and sort them
> before starting the realization.

Do you think you can work on this?

> Don't assume PC behavior where -device serial-pci magically replaces the
> default device with your customized one, the serial device may be hidden
> beneath a Super I/O chipset like on PReP or inside a SoC device.
>
> Similarly we used to have ARM machines where the new -netdev stuff
> couldn't be used because the device doesn't get user-added.

We need to find a way to name onboard devices, so we can configure them.
QOM paths might do.

Our traditional way to configure onboard devices is to deposit
configuration in global variables, where the board can pick it up if it
feels like it.  That's how -serial, -net and so forth work.  Nice when
you don't care about the anatomy of your board.  Not so nice when your
board silently ignores your configuration.  Also not nice: the
connection between configuration and device is rather unobvious.  You
need to find the spot in the code where configuration is picked out of
global variables.  Finally, it doesn't let you configure "unusual" stuff
at all.

If we had means to configure specific onboard devices, we could then
reimplement the legacy options on top: the board describes how they
apply to (named) devices, in one place.  I think that would be easier to
understand.  It would also make rejecting non-sensical configuration
easy.

> One idea once was to extend the by-ID-reference semantics to allow a QOM
> path as transparent means of conversion. I don't think we ever
> implemented that?

What exactly do you mean by "transparent means of conversion"?

Re: [Qemu-devel] steps towards deprecation of old boards and devices

[Markus Armbruster]

Paolo Bonzini <pbonzini@redhat.com> writes:

> On 20/09/2016 13:04, Markus Armbruster wrote:
>> I'm not proposing to get rid of -serial.  I'm proposing to use it as
>> indicator of old code in need of modernization.  A properly QOMified
>> serial device should be "configurable with non-legacy means".  Devices
>> that aren't are probably not QOMified. [...]
>> 
>> > The Raspberry Pi board is probably one of the best examples.  Its only
>> > snag is that one of the devices (hw/arm/bcm2835_peripherals.c) uses
>> > serial_hds[].
>> 
>> Yes, but it uses it to configure QOMified devices, doesn't it?
>> It should be possible to configure these with non-legacy means, at least
>> in theory.
>
> The devices are QOMified, but because there is more than one you cannot
> hack their configuration with -global.

Yes.  It's a pity, and not trivial to fix.  See also Andreas's reply.

So let me rephrase: following the -serial etc. clues will lead us to
some modern devices and some that need work.  I believe it's a promising
way to find devices that need work.