Re: [Qemu-devel] QEMU and Kconfig

Re: [Qemu-devel] QEMU and Kconfig

[Samuel Ortiz]

Hi Paolo,

On Thu, Sep 27, 2018 at 10:55:59AM +0200, Paolo Bonzini wrote:
> > What is the syntactic thing in this example which distinguishes
> > "user can toggle this" (ESP_PCI, ARM_VIRT, SUN4M) from "user
> > can't toggle this, it's just an internal thing selected by
> > other nodes" (the rest) ? I'm assuming we'd have some sort
> > of UI thingy that presents the user only with the user-settable
> > options.
> 
> There's no UI, the configuration is still done with default-configs/;
> however the defaults are specified in the Kconfig files, so the
> default-configs/ files are basically empty (they are not yet empty in
> the branch I posted, but that's not by design; it's just one of the
> reasons why the code was never sent for inclusion upstream).
Reviving this thread as we're starting to work on your minkconf
patches. Let me try to summarize what I understand you have in mind:

- Use Kconfig as a language, not the Kconfig tools from the kernel.
- For each folder, have a Kconfig file describing dependencies and
  selections for any machine/feature/device.
- The Kconfig parser would be used to generate the equivalent of what we
  currently have under default-configs/
- From a user's build perspective, there would be no noticeable
  difference, ./configure && make. Internally, both steps will consume
  the *.mak files generated by minikconf.

Does that sound accurate to you?

Cheers,
Samuel.

Re: [Qemu-devel] QEMU and Kconfig

[Paolo Bonzini]

On 07/11/2018 16:41, Samuel Ortiz wrote:
> - The Kconfig parser would be used to generate the equivalent of what we
>   currently have under default-configs/

It would be used to generate config-devices.mak, instead of
scripts/make_device_config.sh.  My branch already had some Makefile
integration.

> - From a user's build perspective, there would be no noticeable
>   difference, ./configure && make. Internally, both steps will consume
>   the *.mak files generated by minikconf.

Right.  The only difference is that a user could do "./configure && make
randconfig && make" and similar.

Paolo

Re: [Qemu-devel] QEMU and Kconfig

[Eduardo Habkost]

On Wed, Nov 07, 2018 at 06:39:54PM +0100, Paolo Bonzini wrote:
> On 07/11/2018 16:41, Samuel Ortiz wrote:
> > - The Kconfig parser would be used to generate the equivalent of what we
> >   currently have under default-configs/
> 
> It would be used to generate config-devices.mak, instead of
> scripts/make_device_config.sh.  My branch already had some Makefile
> integration.
> 
> > - From a user's build perspective, there would be no noticeable
> >   difference, ./configure && make. Internally, both steps will consume
> >   the *.mak files generated by minikconf.
> 
> Right.  The only difference is that a user could do "./configure && make
> randconfig && make" and similar.

Also, I would like to eventually replace many ./configure options
with options read from a build configuration file.

Distributions often have huge ./configure command lines in their
QEMU packages, and they could be replaced by simple build
configuration files.

Having a mode that requires all build options to be specified
explicitly (instead of silently picking a default) would be
useful for distributions, too.

-- 
Eduardo

Re: [Qemu-devel] QEMU and Kconfig

[Thomas Huth]

On 2018-11-07 20:24, Eduardo Habkost wrote:
> On Wed, Nov 07, 2018 at 06:39:54PM +0100, Paolo Bonzini wrote:
>> On 07/11/2018 16:41, Samuel Ortiz wrote:
>>> - The Kconfig parser would be used to generate the equivalent of what we
>>>   currently have under default-configs/

I think we would still have something like default-configs - but there
would only be the bare minimum config switches in there, the rest would
be pulled in by dependencies.

We could then also even have multiple config directories:

./configs
 +-------/default-softmmu
 +-------/default-linux-user
 +-------/nemu (or lean-kvm or something similar)
 +...

... just my 0.02 €, feel free to ignore that idea ;-)

>> It would be used to generate config-devices.mak, instead of
>> scripts/make_device_config.sh.  My branch already had some Makefile
>> integration.
>>
>>> - From a user's build perspective, there would be no noticeable
>>>   difference, ./configure && make. Internally, both steps will consume
>>>   the *.mak files generated by minikconf.
>>
>> Right.  The only difference is that a user could do "./configure && make
>> randconfig && make" and similar.
> 
> Also, I would like to eventually replace many ./configure options
> with options read from a build configuration file.
> 
> Distributions often have huge ./configure command lines in their
> QEMU packages, and they could be replaced by simple build
> configuration files.
> 
> Having a mode that requires all build options to be specified
> explicitly (instead of silently picking a default) would be
> useful for distributions, too.

I think we should maybe not mix host configuration (via ./configure) and
the target configuration (via kconfig), should we?

 Thomas

Re: [Qemu-devel] QEMU and Kconfig

[Paolo Bonzini]

On 07/11/2018 20:30, Thomas Huth wrote:
> On 2018-11-07 20:24, Eduardo Habkost wrote:
>> On Wed, Nov 07, 2018 at 06:39:54PM +0100, Paolo Bonzini wrote:
>>> On 07/11/2018 16:41, Samuel Ortiz wrote:
>>>> - The Kconfig parser would be used to generate the equivalent of what we
>>>>   currently have under default-configs/
> 
> I think we would still have something like default-configs - but there
> would only be the bare minimum config switches in there, the rest would
> be pulled in by dependencies.

Yes, in theory default-configs would end up empty, except for possibly
some commented lines to show the "default y" symbols for the target.

> We could then also even have multiple config directories:
> 
> ./configs
>  +-------/default-softmmu
>  +-------/default-linux-user
>  +-------/nemu (or lean-kvm or something similar)
>  +...
> 
> ... just my 0.02 €, feel free to ignore that idea ;-)

Yup, one can also think of a configure option like "./configure
--with-device-config=configs/nemu/" to pick up the alternative
configurations.

>> Also, I would like to eventually replace many ./configure options
>> with options read from a build configuration file.
>>
>> Distributions often have huge ./configure command lines in their
>> QEMU packages, and they could be replaced by simple build
>> configuration files.
>>
>> Having a mode that requires all build options to be specified
>> explicitly (instead of silently picking a default) would be
>> useful for distributions, too.
> 
> I think we should maybe not mix host configuration (via ./configure) and
> the target configuration (via kconfig), should we?

Yeah, the configure command line is a different story.  If there are
suggestion on how to improve it, great, but let's not conflate it with
Kconfig.

Paolo

Re: [Qemu-devel] QEMU and Kconfig

[Thomas Huth]

On 2018-11-08 10:55, Paolo Bonzini wrote:
> On 07/11/2018 20:30, Thomas Huth wrote:
>> On 2018-11-07 20:24, Eduardo Habkost wrote:
>>> On Wed, Nov 07, 2018 at 06:39:54PM +0100, Paolo Bonzini wrote:
>>>> On 07/11/2018 16:41, Samuel Ortiz wrote:
>>>>> - The Kconfig parser would be used to generate the equivalent of what we
>>>>>   currently have under default-configs/
>>
>> I think we would still have something like default-configs - but there
>> would only be the bare minimum config switches in there, the rest would
>> be pulled in by dependencies.
> 
> Yes, in theory default-configs would end up empty, except for possibly
> some commented lines to show the "default y" symbols for the target.

I think we should enable the machines in the default configs (and maybe
optional devices that are not automatically selected by machines), e.g.:

CONFIG_I440FX=y
CONFIG_Q35=y
CONFIG_VIRTPC=y
CONFIG_ISAPC=y
CONFIG_VIRTIO=y
CONFIG_PARALLEL=y

In configs/nemu (or configs/lean-kvm or however we'll call it), you will
then only have:

CONFIG_Q35=y
CONFIG_VIRTPC=y
CONFIG_VIRTIO=y

Would that make sense?

 Thomas

Re: [Qemu-devel] QEMU and Kconfig

[Philippe Mathieu-Daudé]

Le jeu. 8 nov. 2018 11:30, Thomas Huth <thuth@redhat.com> a écrit :

> On 2018-11-08 10:55, Paolo Bonzini wrote:
> > On 07/11/2018 20:30, Thomas Huth wrote:
> >> On 2018-11-07 20:24, Eduardo Habkost wrote:
> >>> On Wed, Nov 07, 2018 at 06:39:54PM +0100, Paolo Bonzini wrote:
> >>>> On 07/11/2018 16:41, Samuel Ortiz wrote:
> >>>>> - The Kconfig parser would be used to generate the equivalent of
> what we
> >>>>>   currently have under default-configs/
> >>
> >> I think we would still have something like default-configs - but there
> >> would only be the bare minimum config switches in there, the rest would
> >> be pulled in by dependencies.
> >
> > Yes, in theory default-configs would end up empty, except for possibly
> > some commented lines to show the "default y" symbols for the target.
>
> I think we should enable the machines in the default configs (and maybe
> optional devices that are not automatically selected by machines), e.g.:
>
> CONFIG_I440FX=y
> CONFIG_Q35=y
> CONFIG_VIRTPC=y
> CONFIG_ISAPC=y
> CONFIG_VIRTIO=y
> CONFIG_PARALLEL=y
>
> In configs/nemu (or configs/lean-kvm or however we'll call it), you will
> then only have:
>
> CONFIG_Q35=y
> CONFIG_VIRTPC=y
>

CONFIG_VIRTX86 seems more XXI century.

CONFIG_VIRTIO=y
>
> Would that make sense?
>
>  Thomas
>
>

Re: [Qemu-devel] QEMU and Kconfig

[Paolo Bonzini]

On 08/11/2018 11:14, Thomas Huth wrote:
> On 2018-11-08 10:55, Paolo Bonzini wrote:
>> On 07/11/2018 20:30, Thomas Huth wrote:
>>> On 2018-11-07 20:24, Eduardo Habkost wrote:
>>>> On Wed, Nov 07, 2018 at 06:39:54PM +0100, Paolo Bonzini wrote:
>>>>> On 07/11/2018 16:41, Samuel Ortiz wrote:
>>>>>> - The Kconfig parser would be used to generate the equivalent of what we
>>>>>>   currently have under default-configs/
>>>
>>> I think we would still have something like default-configs - but there
>>> would only be the bare minimum config switches in there, the rest would
>>> be pulled in by dependencies.
>>
>> Yes, in theory default-configs would end up empty, except for possibly
>> some commented lines to show the "default y" symbols for the target.
> 
> I think we should enable the machines in the default configs (and maybe
> optional devices that are not automatically selected by machines), e.g.:
> 
> CONFIG_I440FX=y
> CONFIG_Q35=y
> CONFIG_VIRTPC=y
> CONFIG_ISAPC=y
> CONFIG_VIRTIO=y
> CONFIG_PARALLEL=y
> 
> In configs/nemu (or configs/lean-kvm or however we'll call it), you will
> then only have:
> 
> CONFIG_Q35=y
> CONFIG_VIRTPC=y
> CONFIG_VIRTIO=y
> 
> Would that make sense?

It would, however it would be hard to draw a line between which devices
are enabled by default or not.  I think in particular all devices that
currently are in default-configs/{hyperv,pci,sound,usb,virtio}.mak
should be "default y".  CONFIG_PARALLEL would be in a hypothetical
isa.mak and also "default y".

One example that we discussed of devices that should not be "default y"
is stuff like various I2C sensors (CONFIG_TMP105 for example); these
should not be included in x86 targets just because x86 has an I2C/SMBus
controller in it.  So for example tmp105 should probably be select-ed by
CONFIG_ASPEED and CONFIG_NSERIES.

Let me reiterate that your proposal _also_ makes sense, in fact it's
exactly it is done in the 2015 branch.  However, if everything were
"default y" and we write a script to write out the selected
configuration symbols to default-configs/ file, the difference would be
mostly that the lines in default-configs/ will look like
"#CONFIG_I440FX=y" instead of "CONFIG_I440FX=y".

And in fact, the latter would be nicer for people who want to modify
default-configs/ to specify a few boards and devices explicitly[1].  For
such people (which include both the RHEL QEMU and the NEMU maintainers)
after a rebase you would not get any new board or device.  So I'm
inclined more towards empty default-configs/ file, with comments
mentioning what can be enabled.

Paolo

Re: [Qemu-devel] QEMU and Kconfig

[Eduardo Habkost]

On Thu, Nov 08, 2018 at 10:55:21AM +0100, Paolo Bonzini wrote:
> On 07/11/2018 20:30, Thomas Huth wrote:
> > On 2018-11-07 20:24, Eduardo Habkost wrote:
> >> On Wed, Nov 07, 2018 at 06:39:54PM +0100, Paolo Bonzini wrote:
> >>> On 07/11/2018 16:41, Samuel Ortiz wrote:
> >>>> - The Kconfig parser would be used to generate the equivalent of what we
> >>>>   currently have under default-configs/
> > 
> > I think we would still have something like default-configs - but there
> > would only be the bare minimum config switches in there, the rest would
> > be pulled in by dependencies.
> 
> Yes, in theory default-configs would end up empty, except for possibly
> some commented lines to show the "default y" symbols for the target.
> 
> > We could then also even have multiple config directories:
> > 
> > ./configs
> >  +-------/default-softmmu
> >  +-------/default-linux-user
> >  +-------/nemu (or lean-kvm or something similar)
> >  +...
> > 
> > ... just my 0.02 €, feel free to ignore that idea ;-)
> 
> Yup, one can also think of a configure option like "./configure
> --with-device-config=configs/nemu/" to pick up the alternative
> configurations.
> 
> >> Also, I would like to eventually replace many ./configure options
> >> with options read from a build configuration file.
> >>
> >> Distributions often have huge ./configure command lines in their
> >> QEMU packages, and they could be replaced by simple build
> >> configuration files.
> >>
> >> Having a mode that requires all build options to be specified
> >> explicitly (instead of silently picking a default) would be
> >> useful for distributions, too.
> > 
> > I think we should maybe not mix host configuration (via ./configure) and
> > the target configuration (via kconfig), should we?
> 
> Yeah, the configure command line is a different story.  If there are
> suggestion on how to improve it, great, but let's not conflate it with
> Kconfig.

I believe we have many ./configure options that are supposed to
be target configuration.  e.g.: --enable-slirp, --eanble-kvm,
--enable-xen, etc.

-- 
Eduardo

Re: [Qemu-devel] QEMU and Kconfig

[Paolo Bonzini]

On 08/11/2018 14:06, Eduardo Habkost wrote:
> On Thu, Nov 08, 2018 at 10:55:21AM +0100, Paolo Bonzini wrote:
>> On 07/11/2018 20:30, Thomas Huth wrote:
>>> On 2018-11-07 20:24, Eduardo Habkost wrote:
>>>> On Wed, Nov 07, 2018 at 06:39:54PM +0100, Paolo Bonzini wrote:
>>>>> On 07/11/2018 16:41, Samuel Ortiz wrote:
>>>>>> - The Kconfig parser would be used to generate the equivalent of what we
>>>>>>   currently have under default-configs/
>>>
>>> I think we would still have something like default-configs - but there
>>> would only be the bare minimum config switches in there, the rest would
>>> be pulled in by dependencies.
>>
>> Yes, in theory default-configs would end up empty, except for possibly
>> some commented lines to show the "default y" symbols for the target.
>>
>>> We could then also even have multiple config directories:
>>>
>>> ./configs
>>>  +-------/default-softmmu
>>>  +-------/default-linux-user
>>>  +-------/nemu (or lean-kvm or something similar)
>>>  +...
>>>
>>> ... just my 0.02 €, feel free to ignore that idea ;-)
>>
>> Yup, one can also think of a configure option like "./configure
>> --with-device-config=configs/nemu/" to pick up the alternative
>> configurations.
>>
>>>> Also, I would like to eventually replace many ./configure options
>>>> with options read from a build configuration file.
>>>>
>>>> Distributions often have huge ./configure command lines in their
>>>> QEMU packages, and they could be replaced by simple build
>>>> configuration files.
>>>>
>>>> Having a mode that requires all build options to be specified
>>>> explicitly (instead of silently picking a default) would be
>>>> useful for distributions, too.
>>>
>>> I think we should maybe not mix host configuration (via ./configure) and
>>> the target configuration (via kconfig), should we?
>>
>> Yeah, the configure command line is a different story.  If there are
>> suggestion on how to improve it, great, but let's not conflate it with
>> Kconfig.
> 
> I believe we have many ./configure options that are supposed to
> be target configuration.  e.g.: --enable-slirp, --eanble-kvm,
> --enable-xen, etc.

SLIRP is not a target configuration, it's a backend like most other
configure command line options.  The accelerators are but (except for
TCG of course) they also depend on the host OS and architecture, which
makes them a bit more complicated than default-configs/ symbols.  There
are also things like --enable-vhost-user which affect the creation of
both devices (e.g. vhost-user-scsi-pci) and backends (e.g. the
vhost-user-net backend, which uses a "regular" virtio-net-pci device).

It would surely be possible for configure to call into minikconf to
parse a configuration file and apply dependencies (do we actually have
dependencies across configure options?) or something like that, but
let's not put the cart before the horse...

Paolo

Re: [Qemu-devel] QEMU and Kconfig

[Eduardo Habkost]

On Thu, Nov 08, 2018 at 02:42:19PM +0100, Paolo Bonzini wrote:
> On 08/11/2018 14:06, Eduardo Habkost wrote:
> > On Thu, Nov 08, 2018 at 10:55:21AM +0100, Paolo Bonzini wrote:
> >> On 07/11/2018 20:30, Thomas Huth wrote:
> >>> On 2018-11-07 20:24, Eduardo Habkost wrote:
> >>>> On Wed, Nov 07, 2018 at 06:39:54PM +0100, Paolo Bonzini wrote:
> >>>>> On 07/11/2018 16:41, Samuel Ortiz wrote:
> >>>>>> - The Kconfig parser would be used to generate the equivalent of what we
> >>>>>>   currently have under default-configs/
> >>>
> >>> I think we would still have something like default-configs - but there
> >>> would only be the bare minimum config switches in there, the rest would
> >>> be pulled in by dependencies.
> >>
> >> Yes, in theory default-configs would end up empty, except for possibly
> >> some commented lines to show the "default y" symbols for the target.
> >>
> >>> We could then also even have multiple config directories:
> >>>
> >>> ./configs
> >>>  +-------/default-softmmu
> >>>  +-------/default-linux-user
> >>>  +-------/nemu (or lean-kvm or something similar)
> >>>  +...
> >>>
> >>> ... just my 0.02 €, feel free to ignore that idea ;-)
> >>
> >> Yup, one can also think of a configure option like "./configure
> >> --with-device-config=configs/nemu/" to pick up the alternative
> >> configurations.
> >>
> >>>> Also, I would like to eventually replace many ./configure options
> >>>> with options read from a build configuration file.
> >>>>
> >>>> Distributions often have huge ./configure command lines in their
> >>>> QEMU packages, and they could be replaced by simple build
> >>>> configuration files.
> >>>>
> >>>> Having a mode that requires all build options to be specified
> >>>> explicitly (instead of silently picking a default) would be
> >>>> useful for distributions, too.
> >>>
> >>> I think we should maybe not mix host configuration (via ./configure) and
> >>> the target configuration (via kconfig), should we?
> >>
> >> Yeah, the configure command line is a different story.  If there are
> >> suggestion on how to improve it, great, but let's not conflate it with
> >> Kconfig.
> > 
> > I believe we have many ./configure options that are supposed to
> > be target configuration.  e.g.: --enable-slirp, --eanble-kvm,
> > --enable-xen, etc.
> 
> SLIRP is not a target configuration, it's a backend like most other
> configure command line options.  The accelerators are but (except for
> TCG of course) they also depend on the host OS and architecture, which
> makes them a bit more complicated than default-configs/ symbols.  There
> are also things like --enable-vhost-user which affect the creation of
> both devices (e.g. vhost-user-scsi-pci) and backends (e.g. the
> vhost-user-net backend, which uses a "regular" virtio-net-pci device).
> 

Keeping in mind that I might be talking about extra challenges we
won't address right now (no cart before the horse), I have new
questions:

Why you say backends are not a target configuration and
accelerators are?  What's the definition of "target
configuration"?

Are we explicitly restricting the scope of this work to
enabling/disabling device emulation code right now?  Why?  Why
wouldn't we use kconfig to enable/disable simple backends with no
host dependency like SLIRP?

Don't we want to make backends configurable per binary, too?
e.g.: I would expect the default configuration for a NEMU-like
binary to disable many backends.


> It would surely be possible for configure to call into minikconf to
> parse a configuration file and apply dependencies (do we actually have
> dependencies across configure options?) or something like that, but
> let's not put the cart before the horse...

Agreed.

-- 
Eduardo

Re: [Qemu-devel] QEMU and Kconfig

[Paolo Bonzini]

On 08/11/2018 18:14, Eduardo Habkost wrote:
> Keeping in mind that I might be talking about extra challenges we
> won't address right now (no cart before the horse), I have new
> questions:
> 
> Why you say backends are not a target configuration and
> accelerators are?  What's the definition of "target
> configuration"?

Something that affects the way

> Are we explicitly restricting the scope of this work to
> enabling/disabling device emulation code right now?  Why?  Why
> wouldn't we use kconfig to enable/disable simple backends with no
> host dependency like SLIRP?

I think it would be more confusing if some backends were to use kconfig
and some wouldn't.  We could certainly add something like

config VHOST_NET
    depends on HOST_LINUX
    default y

config SPICE
    depends on HAVE_SPICE_SERVER
    default Y

etc. but I think we agree it's more of a long term idea.

> Don't we want to make backends configurable per binary, too?
> e.g.: I would expect the default configuration for a NEMU-like
> binary to disable many backends.

Sure, we could do that.  However, right now you cannot have multiple
binaries for a single target, so you couldn't have one single build
include both a "full-blown" and a "reduced" x86 target.  Therefore,
including e.g. SLIRP in qemu-system-arm but not in qemu-system-x86_64
does not seem too interesting to me.  It would be different if you could
build qemu-system-arm, qemu-system-x86_64, qemu-system-x86_64-lite, etc.

Paolo

> 
>> It would surely be possible for configure to call into minikconf to
>> parse a configuration file and apply dependencies (do we actually have
>> dependencies across configure options?) or something like that, but
>> let's not put the cart before the horse...
> 
> Agreed.
> 

Re: [Qemu-devel] QEMU and Kconfig

[Eduardo Habkost]

On Thu, Nov 08, 2018 at 06:58:11PM +0100, Paolo Bonzini wrote:
> On 08/11/2018 18:14, Eduardo Habkost wrote:
> > Keeping in mind that I might be talking about extra challenges we
> > won't address right now (no cart before the horse), I have new
> > questions:
> > 
> > Why you say backends are not a target configuration and
> > accelerators are?  What's the definition of "target
> > configuration"?
> 
> Something that affects the way

?

> 
> > Are we explicitly restricting the scope of this work to
> > enabling/disabling device emulation code right now?  Why?  Why
> > wouldn't we use kconfig to enable/disable simple backends with no
> > host dependency like SLIRP?
> 
> I think it would be more confusing if some backends were to use kconfig
> and some wouldn't.  We could certainly add something like
> 
> config VHOST_NET
>     depends on HOST_LINUX
>     default y
> 
> config SPICE
>     depends on HAVE_SPICE_SERVER
>     default Y
> 
> etc. but I think we agree it's more of a long term idea.

Agreed.

> 
> > Don't we want to make backends configurable per binary, too?
> > e.g.: I would expect the default configuration for a NEMU-like
> > binary to disable many backends.
> 
> Sure, we could do that.  However, right now you cannot have multiple
> binaries for a single target, so you couldn't have one single build
> include both a "full-blown" and a "reduced" x86 target.  Therefore,
> including e.g. SLIRP in qemu-system-arm but not in qemu-system-x86_64
> does not seem too interesting to me.  It would be different if you could
> build qemu-system-arm, qemu-system-x86_64, qemu-system-x86_64-lite, etc.

Understood.  I assumed this would be one of the short-term goals.
We can work on that later, then.


> 
> Paolo
> 
> > 
> >> It would surely be possible for configure to call into minikconf to
> >> parse a configuration file and apply dependencies (do we actually have
> >> dependencies across configure options?) or something like that, but
> >> let's not put the cart before the horse...
> > 
> > Agreed.
> > 
> 

-- 
Eduardo

Re: [Qemu-devel] QEMU and Kconfig

[Paolo Bonzini]

Oops. :)

On 08/11/2018 19:42, Eduardo Habkost wrote:
>>> Keeping in mind that I might be talking about extra challenges we
>>> won't address right now (no cart before the horse), I have new
>>> questions:
>>>
>>> Why you say backends are not a target configuration and
>>> accelerators are?  What's the definition of "target
>>> configuration"?

Something that affects the hardware seen by the guest is target
configuration.

Backends do not affect what hardware the guest sees.  Boards and devices
do; accelerators do, but that's more of a side-effect than something
intended.

Paolo

Re: [Qemu-devel] QEMU and Kconfig

[Eduardo Habkost]

On Thu, Nov 08, 2018 at 09:28:06PM +0100, Paolo Bonzini wrote:
> Oops. :)
> 
> On 08/11/2018 19:42, Eduardo Habkost wrote:
> >>> Keeping in mind that I might be talking about extra challenges we
> >>> won't address right now (no cart before the horse), I have new
> >>> questions:
> >>>
> >>> Why you say backends are not a target configuration and
> >>> accelerators are?  What's the definition of "target
> >>> configuration"?
> 
> Something that affects the hardware seen by the guest is target
> configuration.
> 
> Backends do not affect what hardware the guest sees.  Boards and devices
> do; accelerators do, but that's more of a side-effect than something
> intended.

Understood.  My interpretation of "target" was just "a QEMU
binary".  In other words, I thought we were talking about
anything that could be compiled in/out from a specific QEMU
binary.

Do you have a specific reason to restrict the scope to only
guest-visible effects?  Is this just a way to reduce the effort
required for the task, or there are other caveats I'm missing?

-- 
Eduardo

Re: [Qemu-devel] QEMU and Kconfig

[Paolo Bonzini]

On 08/11/2018 22:00, Eduardo Habkost wrote:
> Understood.  My interpretation of "target" was just "a QEMU
> binary".  In other words, I thought we were talking about
> anything that could be compiled in/out from a specific QEMU
> binary.

The idea is "target" as opposed to "host".

> Do you have a specific reason to restrict the scope to only
> guest-visible effects?  Is this just a way to reduce the effort
> required for the task, or there are other caveats I'm missing?

Because that's what default-configs/ is for---producing
config-devices.mak.  IOW it's mostly to reduce the scope, but also
because there are differences between config-devices.mak (produced from
default-configs/) and config-{host,target}.h (produced by configure).

In particular, config-host.h and config-target.h are header files, but
config-devices.mak is not because the same file is linked into multiple
QEMU binaries that can and will enable different devices.  The only way
to use a hypothetical config-devices.h would be to move its users from
common-obj-y to obj-y.

Paolo

Re: [Qemu-devel] QEMU and Kconfig

[Eduardo Habkost]

On Fri, Nov 09, 2018 at 11:10:21AM +0100, Paolo Bonzini wrote:
> On 08/11/2018 22:00, Eduardo Habkost wrote:
> > Understood.  My interpretation of "target" was just "a QEMU
> > binary".  In other words, I thought we were talking about
> > anything that could be compiled in/out from a specific QEMU
> > binary.
> 
> The idea is "target" as opposed to "host".
> 
> > Do you have a specific reason to restrict the scope to only
> > guest-visible effects?  Is this just a way to reduce the effort
> > required for the task, or there are other caveats I'm missing?
> 
> Because that's what default-configs/ is for---producing
> config-devices.mak.  IOW it's mostly to reduce the scope, but also
> because there are differences between config-devices.mak (produced from
> default-configs/) and config-{host,target}.h (produced by configure).

I have the impression that the build system has an implicit
assumption: that any build option that affects only one QEMU
binary is always guest-visible, and that any build option that is
not guest visible must affect all built QEMU binaries.  Is this
going to be always true?


> 
> In particular, config-host.h and config-target.h are header files, but
> config-devices.mak is not because the same file is linked into multiple
> QEMU binaries that can and will enable different devices.  The only way
> to use a hypothetical config-devices.h would be to move its users from
> common-obj-y to obj-y.

Yeah, I wouldn't want to move things to obj-y.  I would prefer to
make kconfig affect only makefile variables (not header files and
macros).

-- 
Eduardo

Re: [Qemu-devel] QEMU and Kconfig

[Paolo Bonzini]

On 09/11/2018 20:16, Eduardo Habkost wrote:
> On Fri, Nov 09, 2018 at 11:10:21AM +0100, Paolo Bonzini wrote:
>> On 08/11/2018 22:00, Eduardo Habkost wrote:
>>> Understood.  My interpretation of "target" was just "a QEMU
>>> binary".  In other words, I thought we were talking about
>>> anything that could be compiled in/out from a specific QEMU
>>> binary.
>>
>> The idea is "target" as opposed to "host".
>>
>>> Do you have a specific reason to restrict the scope to only
>>> guest-visible effects?  Is this just a way to reduce the effort
>>> required for the task, or there are other caveats I'm missing?
>>
>> Because that's what default-configs/ is for---producing
>> config-devices.mak.  IOW it's mostly to reduce the scope, but also
>> because there are differences between config-devices.mak (produced from
>> default-configs/) and config-{host,target}.h (produced by configure).
> 
> I have the impression that the build system has an implicit
> assumption: that any build option that affects only one QEMU
> binary is always guest-visible, and that any build option that is
> not guest visible must affect all built QEMU binaries.  Is this
> going to be always true?

I don't think it's an assumption.  It's more a side effect of avoiding
obj-y unless needed.  Because any build option that affects only one
QEMU binary must use obj-y, and non-guest-visible code generally doesn't
use obj-y, the result is what you say.

Paolo

Re: [Qemu-devel] QEMU and Kconfig

[Samuel Ortiz]

On Wed, Nov 07, 2018 at 05:24:14PM -0200, Eduardo Habkost wrote:
> On Wed, Nov 07, 2018 at 06:39:54PM +0100, Paolo Bonzini wrote:
> > On 07/11/2018 16:41, Samuel Ortiz wrote:
> > > - The Kconfig parser would be used to generate the equivalent of what we
> > >   currently have under default-configs/
> > 
> > It would be used to generate config-devices.mak, instead of
> > scripts/make_device_config.sh.  My branch already had some Makefile
> > integration.
> > 
> > > - From a user's build perspective, there would be no noticeable
> > >   difference, ./configure && make. Internally, both steps will consume
> > >   the *.mak files generated by minikconf.
> > 
> > Right.  The only difference is that a user could do "./configure && make
> > randconfig && make" and similar.
> 
> Also, I would like to eventually replace many ./configure options
> with options read from a build configuration file.
> 
> Distributions often have huge ./configure command lines in their
> QEMU packages, and they could be replaced by simple build
> configuration files.
> 
> Having a mode that requires all build options to be specified
> explicitly (instead of silently picking a default)
I agree. This has been a problem for us several times, we saw build
errors because on some host build options were silently selected while
on others they were not.
But to emphasize on what Paolo said, I think using kconfig for
describing the target/device/platform dependencies is almost orthogonal
to improving the configuration step.

Cheers,
Samuel.

Re: [Qemu-devel] QEMU and Kconfig

[Yang Zhong]

Hello Paolo and all,

The Kconfig has been completed the porting from your repo and
now i can sucessfully build x86_64-softmmu binary. But there
are still some issues:
   1) Only support x86_64 platform.
      Other platform, especially for ARM platforms, it's hard for me to define 
      detailed boards like pc or q35 in x86.

   2) Only support "defconfig". 
      "randconfig" build has some issues, which are mostly related with CONFIG* in Kconfig.host, 
      some CONFIG* has different setting value in config-host.mak and %/config-device.mak. like
      CONFIG_OPENGL=y in %/config-device.mak and CONFIG_OPENGL=n in config-host.mak, which result
      in build failure.

The current configure and build command are almost same with before and if we want to disable or enable
some features, like "tcg", we still need add "--enable/--disable-tcg" in configure command line. If we want
to disable one emulation device, we can disable this in related Kconfig file in hw/ directory.

There are still some left issues
   a) How to replace the CONFIG* in configure, Some CONFIG* need some logic to generate, 
      those are hard to input this CONFIG* in Kconfig.host or Kconfig* file.

   b) The CONFIG* in %/config-target.mak file, this is still related with configure. 
        
   c) If a) and b) can be fixed, randconfig is not critical issue.

I will send RFC patches to QEMU community and please all give some comments. many thanks!

Regards,

Yang

Re: [Qemu-devel] QEMU and Kconfig

[Paolo Bonzini]

On 13/12/18 12:50, Yang Zhong wrote:
> Hello Paolo and all,
> 
> The Kconfig has been completed the porting from your repo and
> now i can sucessfully build x86_64-softmmu binary. But there
> are still some issues:
>    1) Only support x86_64 platform.
>       Other platform, especially for ARM platforms, it's hard for me to define 
>       detailed boards like pc or q35 in x86.

No problem, Philippe can look at least at ARM and MIPS.

>    2) Only support "defconfig". 
>       "randconfig" build has some issues, which are mostly related with CONFIG* in Kconfig.host, 
>       some CONFIG* has different setting value in config-host.mak and %/config-device.mak. like
>       CONFIG_OPENGL=y in %/config-device.mak and CONFIG_OPENGL=n in config-host.mak, which result
>       in build failure.

This is probably a bug or missing features in minikconf; values from
config-host.mak and config-target.mak should not be included in
config-devices.mak.

> The current configure and build command are almost same with before and if we want to disable or enable
> some features, like "tcg", we still need add "--enable/--disable-tcg" in configure command line. If we want
> to disable one emulation device, we can disable this in related Kconfig file in hw/ directory.

Yes, this is expected.

> There are still some left issues
>    a) How to replace the CONFIG* in configure, Some CONFIG* need some logic to generate, 
>       those are hard to input this CONFIG* in Kconfig.host or Kconfig* file.
> 
>    b) The CONFIG* in %/config-target.mak file, this is still related with configure. 
>         
>    c) If a) and b) can be fixed, randconfig is not critical issue.
> 
> I will send RFC patches to QEMU community and please all give some comments. many thanks!

Great, this way people can experiment it and we can finalize the design
before continuing the work.

Thanks,

Paolo

Re: [Qemu-devel] QEMU and Kconfig

[Philippe Mathieu-Daudé]

Le 26 sept. 2018 6:58 PM, "Paolo Bonzini" <pbonzini@redhat.com> a écrit :

On 26/09/2018 16:15, Samuel Ortiz wrote:
> On Wed, Sep 26, 2018 at 03:36:07PM +0200, Paolo Bonzini wrote:
>> On 26/09/2018 10:32, Peter Maydell wrote:
>>> On 25 September 2018 at 10:26, Paolo Bonzini <pbonzini@redhat.com>
wrote:
>>>> However, the Kconfig language is a good idea.  The idea is that
>>>> dependencies can be expressed:
>>>>
>>>> - as "select" whenever a board requires a device, or whenever a device
>>>> requires generic subsystem code in order to implement a controller for
>>>> that subsystem (e.g.: PC selects MC146818RTC, VIRTIO_SCSI selects SCSI)
>>>>
>>>> - as "depends on" whenever a device requires a bus (e.g.: SERIAL_PCI
>>>> depends on ISA)
>>>>
>>>> Putting the two things together, AHCI depends on PCI but it selects
IDE.
>>>
>>> Could you clarify the distinction you're making here between
>>> these two kinds of dependency and what the top level user
>>> interface would be? I'm getting confused about why we need
>>> two -- I have a vague impression it depends on what the
>>> top level thing the user is selecting enablement of is,
>>> but maybe not ?
>>
>> Yes, exactly.  The user selects enablement of 1) boards 2) non-embedded
>> devices.  This mimics the level at which configuration is done today in
>> default-configs (it worked like this in 2013 during GSoC, and it is
>> still pretty much the same).  The idea is that everything else is
>> selected automatically by the configuration tool, but two different
>> paths are available depending on the desired defaults:
>>
>> - users of something (that is often "default n") use "select" to force
>> that configuration symbol to y
>>
>> - "depends on" is used for something that is (usually) default y, but
>> becomes n if the dependencies are missing
>>
>
> So "depends on" will go through the dependency graph and turn the config
> off if they're not entirely filled. But "select" just turns the config on
> regardless of the dependencies. Is that correct?

Almost; if there's a conflict between the decision from "depends on" and
"select" says, it's an error.  (Likewise if there's a conflict between
default-configs/ on one side, and "depends on"/"select" on the other).

This is different between kernel Kconfig and minikconf in my git repo.
The difference is because the Kconfig documentation warns against the
surprising effects of "select" blindly enabling the target symbol;
making minikconf stricter avoids those surprises.


So can we rename your tool Qconfig?



Paolo

Re: [Qemu-devel] QEMU and Kconfig

[Paolo Bonzini]

On 08/11/2018 09:46, Philippe Mathieu-Daudé wrote:
> 
>     Almost; if there's a conflict between the decision from "depends on" and
>     "select" says, it's an error.  (Likewise if there's a conflict between
>     default-configs/ on one side, and "depends on"/"select" on the other).
> 
>     This is different between kernel Kconfig and minikconf in my git repo.
>     The difference is because the Kconfig documentation warns against the
>     surprising effects of "select" blindly enabling the target symbol;
>     making minikconf stricter avoids those surprises.
> 
> 
> So can we rename your tool Qconfig?

Haha, that's up to whoever finishes the work. :)

Paolo

Re: [Qemu-devel] QEMU and Kconfig

[Markus Armbruster]

Paolo Bonzini <pbonzini@redhat.com> writes:

> On 08/11/2018 09:46, Philippe Mathieu-Daudé wrote:
>> 
>>     Almost; if there's a conflict between the decision from "depends on" and
>>     "select" says, it's an error.  (Likewise if there's a conflict between
>>     default-configs/ on one side, and "depends on"/"select" on the other).
>> 
>>     This is different between kernel Kconfig and minikconf in my git repo.
>>     The difference is because the Kconfig documentation warns against the
>>     surprising effects of "select" blindly enabling the target symbol;
>>     making minikconf stricter avoids those surprises.
>> 
>> 
>> So can we rename your tool Qconfig?
>
> Haha, that's up to whoever finishes the work. :)

There's your chance to leave your mark on QEMU!  SamConfig has a nice
ring to it, doesn't it?  ;-P