Re: [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

Re: [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Stefano Stabellini]

On Fri, 6 Jan 2012, Andrew Jones wrote:
> XEN_DOM0 is a silent option that has been automatically selected when
> CONFIG_XEN is selected since 6b0661a5e6fbf. If this option was changed
> to a menu configurable option then it would only give users the ability
> to compile out about 100 kbytes of code.

I think that it is less than that because backends are not tied to dom0
in any way, even though currently they cannot be compiled without
XEN_DOM0.
Running a network or block backend in domU is a well known
configuration.
Therefore the code compiled out only amounts to about 10K.


> diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
> index 8795480..0725228 100644
> --- a/drivers/xen/Kconfig
> +++ b/drivers/xen/Kconfig
> @@ -78,8 +78,9 @@ config XEN_DEV_EVTCHN
>  
>  config XEN_BACKEND
>  	bool "Backend driver support"
> -	depends on XEN_DOM0
>  	default y
> +	depends on XEN && PCI_XEN && SWIOTLB_XEN
> +	depends on X86_LOCAL_APIC && X86_IO_APIC && ACPI && PCI
>  	help
>  	  Support for backend device drivers that provide I/O services
>  	  to other virtual machines.

I think we can trimmer the dependency list here: after all backends can
be run in unpriviledged guests as well (see driver domains).
So I think it should depend only on XEN.

Re: [Xen-devel] [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Andrew Jones]

----- Original Message -----
> On Fri, 6 Jan 2012, Andrew Jones wrote:
> > XEN_DOM0 is a silent option that has been automatically selected
> > when
> > CONFIG_XEN is selected since 6b0661a5e6fbf. If this option was
> > changed
> > to a menu configurable option then it would only give users the
> > ability
> > to compile out about 100 kbytes of code.
> 
> I think that it is less than that because backends are not tied to
> dom0
> in any way, even though currently they cannot be compiled without
> XEN_DOM0.
> Running a network or block backend in domU is a well known
> configuration.
> Therefore the code compiled out only amounts to about 10K.
> 
> 
> > diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
> > index 8795480..0725228 100644
> > --- a/drivers/xen/Kconfig
> > +++ b/drivers/xen/Kconfig
> > @@ -78,8 +78,9 @@ config XEN_DEV_EVTCHN
> >  
> >  config XEN_BACKEND
> >  	bool "Backend driver support"
> > -	depends on XEN_DOM0
> >  	default y
> > +	depends on XEN && PCI_XEN && SWIOTLB_XEN
> > +	depends on X86_LOCAL_APIC && X86_IO_APIC && ACPI && PCI
> >  	help
> >  	  Support for backend device drivers that provide I/O services
> >  	  to other virtual machines.
> 
> I think we can trimmer the dependency list here: after all backends
> can
> be run in unpriviledged guests as well (see driver domains).
> So I think it should depend only on XEN.

Hmm.. Actually, with dependency lists in mind, I think a really messed
this patch up. I should have either had CONFIG_XEN inherit these deps,
or I should have spread them around to be required at only the specific
places they're needed, and then left the stub functions in. Neither of
those options seems like a good idea to me. We either force all XEN
guests to always have all the deps needed for an initial domain, which
is exactly the opposite of the suggestion above (trimming XEN_BACKEND
deps for driver domains), or we actually make the code messier and
more complicated with more #ifdefs, which are now neatly packaged with
XEN_DOM0.

The driver domain concept may actually be the technical need for
making XEN_DOM0 optional. If we want the ability to build a minimally
configed XEN kernel to be used as a driver domain, then it doesn't
seem like we'd want it to also be capable of running as an initial
domain, or to be forced to have all the dependencies and code of an
initial domain.

With that in mind, I propose we go back to my initial patch, relax
deps for XEN_BACKEND, and double check that each individual backend
has the appropriate minimal deps. In general, it should be a goal to
make sure most XEN options are menu selectable and that all dep lists
are minimized in order to make sure driver domains can be built with
minimal configs.

Drew

> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 

Re: [Xen-devel] [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Stefano Stabellini]

On Mon, 9 Jan 2012, Andrew Jones wrote:
> ----- Original Message -----
> > On Fri, 6 Jan 2012, Andrew Jones wrote:
> > > XEN_DOM0 is a silent option that has been automatically selected
> > > when
> > > CONFIG_XEN is selected since 6b0661a5e6fbf. If this option was
> > > changed
> > > to a menu configurable option then it would only give users the
> > > ability
> > > to compile out about 100 kbytes of code.
> > 
> > I think that it is less than that because backends are not tied to
> > dom0
> > in any way, even though currently they cannot be compiled without
> > XEN_DOM0.
> > Running a network or block backend in domU is a well known
> > configuration.
> > Therefore the code compiled out only amounts to about 10K.
> > 
> > 
> > > diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
> > > index 8795480..0725228 100644
> > > --- a/drivers/xen/Kconfig
> > > +++ b/drivers/xen/Kconfig
> > > @@ -78,8 +78,9 @@ config XEN_DEV_EVTCHN
> > >  
> > >  config XEN_BACKEND
> > >  	bool "Backend driver support"
> > > -	depends on XEN_DOM0
> > >  	default y
> > > +	depends on XEN && PCI_XEN && SWIOTLB_XEN
> > > +	depends on X86_LOCAL_APIC && X86_IO_APIC && ACPI && PCI
> > >  	help
> > >  	  Support for backend device drivers that provide I/O services
> > >  	  to other virtual machines.
> > 
> > I think we can trimmer the dependency list here: after all backends
> > can
> > be run in unpriviledged guests as well (see driver domains).
> > So I think it should depend only on XEN.
> 
> Hmm.. Actually, with dependency lists in mind, I think a really messed
> this patch up. I should have either had CONFIG_XEN inherit these deps,
> or I should have spread them around to be required at only the specific
> places they're needed, and then left the stub functions in. Neither of
> those options seems like a good idea to me. We either force all XEN
> guests to always have all the deps needed for an initial domain, which
> is exactly the opposite of the suggestion above (trimming XEN_BACKEND
> deps for driver domains), or we actually make the code messier and
> more complicated with more #ifdefs, which are now neatly packaged with
> XEN_DOM0.

I don't think we should add "PCI_XEN && SWIOTLB_XEN && X86_LOCAL_APIC &&
X86_IO_APIC && ACPI && PCI" to XEN either.
However it should be possible to add only the right dependencies to the
right places.
In practice it should be sufficient to:

- make PCI_XEN depend on X86_LOCAL_APIC && X86_IO_APIC && ACPI;

- make XEN_PCIDEV_FRONTEND and XEN_PCIDEV_BACKEND depend on PCI_XEN;

- remove the 'ifdef CONFIG_ACPI' from arch/x86/pci/xen.c.



> The driver domain concept may actually be the technical need for
> making XEN_DOM0 optional. If we want the ability to build a minimally
> configed XEN kernel to be used as a driver domain, then it doesn't
> seem like we'd want it to also be capable of running as an initial
> domain, or to be forced to have all the dependencies and code of an
> initial domain.

In practice any decent driver domain needs PCI and ACPI support, so
basically it has the same config requirement of XEN_DOM0. At that point
we have already discussed that having XEN_DOM0 enabled or disabled
hardly makes any differences, so we can just get rid of it.

Re: [Xen-devel] [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Konrad Rzeszutek Wilk]

On Mon, Jan 09, 2012 at 11:39:44AM +0000, Stefano Stabellini wrote:
> On Mon, 9 Jan 2012, Andrew Jones wrote:
> > ----- Original Message -----
> > > On Fri, 6 Jan 2012, Andrew Jones wrote:
> > > > XEN_DOM0 is a silent option that has been automatically selected
> > > > when
> > > > CONFIG_XEN is selected since 6b0661a5e6fbf. If this option was
> > > > changed
> > > > to a menu configurable option then it would only give users the
> > > > ability
> > > > to compile out about 100 kbytes of code.
> > > 
> > > I think that it is less than that because backends are not tied to
> > > dom0
> > > in any way, even though currently they cannot be compiled without
> > > XEN_DOM0.
> > > Running a network or block backend in domU is a well known
> > > configuration.
> > > Therefore the code compiled out only amounts to about 10K.
> > > 
> > > 
> > > > diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
> > > > index 8795480..0725228 100644
> > > > --- a/drivers/xen/Kconfig
> > > > +++ b/drivers/xen/Kconfig
> > > > @@ -78,8 +78,9 @@ config XEN_DEV_EVTCHN
> > > >  
> > > >  config XEN_BACKEND
> > > >  	bool "Backend driver support"
> > > > -	depends on XEN_DOM0
> > > >  	default y
> > > > +	depends on XEN && PCI_XEN && SWIOTLB_XEN
> > > > +	depends on X86_LOCAL_APIC && X86_IO_APIC && ACPI && PCI
> > > >  	help
> > > >  	  Support for backend device drivers that provide I/O services
> > > >  	  to other virtual machines.
> > > 
> > > I think we can trimmer the dependency list here: after all backends
> > > can
> > > be run in unpriviledged guests as well (see driver domains).
> > > So I think it should depend only on XEN.
> > 
> > Hmm.. Actually, with dependency lists in mind, I think a really messed
> > this patch up. I should have either had CONFIG_XEN inherit these deps,
> > or I should have spread them around to be required at only the specific
> > places they're needed, and then left the stub functions in. Neither of
> > those options seems like a good idea to me. We either force all XEN
> > guests to always have all the deps needed for an initial domain, which
> > is exactly the opposite of the suggestion above (trimming XEN_BACKEND
> > deps for driver domains), or we actually make the code messier and
> > more complicated with more #ifdefs, which are now neatly packaged with
> > XEN_DOM0.
> 
> I don't think we should add "PCI_XEN && SWIOTLB_XEN && X86_LOCAL_APIC &&
> X86_IO_APIC && ACPI && PCI" to XEN either.
> However it should be possible to add only the right dependencies to the
> right places.
> In practice it should be sufficient to:
> 
> - make PCI_XEN depend on X86_LOCAL_APIC && X86_IO_APIC && ACPI;

Not good. You can make non-ACPI builds with PCI.

.. and you are missing CONFIG_PCI in there.
> 
> - make XEN_PCIDEV_FRONTEND and XEN_PCIDEV_BACKEND depend on PCI_XEN;

OK. That sounds good.
> 
> - remove the 'ifdef CONFIG_ACPI' from arch/x86/pci/xen.c.

No.. same issue - non-ACPI builds can be with PCI.
> 
> 
> 
> > The driver domain concept may actually be the technical need for
> > making XEN_DOM0 optional. If we want the ability to build a minimally
> > configed XEN kernel to be used as a driver domain, then it doesn't
> > seem like we'd want it to also be capable of running as an initial
> > domain, or to be forced to have all the dependencies and code of an
> > initial domain.
> 
> In practice any decent driver domain needs PCI and ACPI support, so
> basically it has the same config requirement of XEN_DOM0. At that point

Sure.. for backends. But for frontends that requirement is not always true.

> we have already discussed that having XEN_DOM0 enabled or disabled
> hardly makes any differences, so we can just get rid of it.

Or in other words, substitute it as CONFIG_PCI_XEN.

But this brings a question about MCE, ACPI cpu freq and ACPI S3.
Those all belong to the dom0 - not to any driver domain. So getting rid
of CONFIG_XEN_DOM0 would present a problem - what would those depend on?

Re: [Xen-devel] [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Stefano Stabellini]

On Mon, 9 Jan 2012, Konrad Rzeszutek Wilk wrote:
> On Mon, Jan 09, 2012 at 11:39:44AM +0000, Stefano Stabellini wrote:
> > I don't think we should add "PCI_XEN && SWIOTLB_XEN && X86_LOCAL_APIC &&
> > X86_IO_APIC && ACPI && PCI" to XEN either.
> > However it should be possible to add only the right dependencies to the
> > right places.
> > In practice it should be sufficient to:
> > 
> > - make PCI_XEN depend on X86_LOCAL_APIC && X86_IO_APIC && ACPI;
> 
> Not good. You can make non-ACPI builds with PCI.
> 
> .. and you are missing CONFIG_PCI in there.
> > 
> > - make XEN_PCIDEV_FRONTEND and XEN_PCIDEV_BACKEND depend on PCI_XEN;
> 
> OK. That sounds good.
> > 
> > - remove the 'ifdef CONFIG_ACPI' from arch/x86/pci/xen.c.
> 
> No.. same issue - non-ACPI builds can be with PCI.
> > 

Right.. in that case we should carefully replace the 'ifdef
CONFIG_XEN_DOM0' with 'ifdef CONFIG_ACPI' in arch/x86/pci/xen.c.
It would effectively keep things as they are now: if ACPI and XEN are
enabled together in the config file, your kernel is able to setup
interrupts when running as DOM0.
Regarding X86_LOCAL_APIC and X86_IO_APIC I cannot recall anymore where
these dependencies come from exactly.


> > > The driver domain concept may actually be the technical need for
> > > making XEN_DOM0 optional. If we want the ability to build a minimally
> > > configed XEN kernel to be used as a driver domain, then it doesn't
> > > seem like we'd want it to also be capable of running as an initial
> > > domain, or to be forced to have all the dependencies and code of an
> > > initial domain.
> > 
> > In practice any decent driver domain needs PCI and ACPI support, so
> > basically it has the same config requirement of XEN_DOM0. At that point
> 
> Sure.. for backends. But for frontends that requirement is not always true.

Right but a driver domain is bound to have at least the pci backend.


> > we have already discussed that having XEN_DOM0 enabled or disabled
> > hardly makes any differences, so we can just get rid of it.
> 
> Or in other words, substitute it as CONFIG_PCI_XEN.

I was just trying to assign the dependencies where they really belong
and I proposed to remove the 'ifdef CONFIG_ACPI' because I didn't
realize that one can build working PCI configurations on XEN without it.

The fact that PCI_XEN would be used then almost as XEN_DOM0 is used now
is just a side effect, due to the fact that PCI and device interrupts
initialization is what mainly differentiates dom0 from other
configurations.


> But this brings a question about MCE, ACPI cpu freq and ACPI S3.
> Those all belong to the dom0 - not to any driver domain. So getting rid
> of CONFIG_XEN_DOM0 would present a problem - what would those depend on?

They would depend on XEN and whatever else they really depend on, for
example ACPI.

Re: [Xen-devel] [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Konrad Rzeszutek Wilk]

On Mon, Jan 09, 2012 at 04:12:10PM +0000, Stefano Stabellini wrote:
> On Mon, 9 Jan 2012, Konrad Rzeszutek Wilk wrote:
> > On Mon, Jan 09, 2012 at 11:39:44AM +0000, Stefano Stabellini wrote:
> > > I don't think we should add "PCI_XEN && SWIOTLB_XEN && X86_LOCAL_APIC &&
> > > X86_IO_APIC && ACPI && PCI" to XEN either.
> > > However it should be possible to add only the right dependencies to the
> > > right places.
> > > In practice it should be sufficient to:
> > > 
> > > - make PCI_XEN depend on X86_LOCAL_APIC && X86_IO_APIC && ACPI;
> > 
> > Not good. You can make non-ACPI builds with PCI.
> > 
> > .. and you are missing CONFIG_PCI in there.
> > > 
> > > - make XEN_PCIDEV_FRONTEND and XEN_PCIDEV_BACKEND depend on PCI_XEN;
> > 
> > OK. That sounds good.
> > > 
> > > - remove the 'ifdef CONFIG_ACPI' from arch/x86/pci/xen.c.
> > 
> > No.. same issue - non-ACPI builds can be with PCI.
> > > 
> 
> Right.. in that case we should carefully replace the 'ifdef
> CONFIG_XEN_DOM0' with 'ifdef CONFIG_ACPI' in arch/x86/pci/xen.c.

.. which I think I did at some point as part of cleanup and then
removed them since it peppered the xen.c with tons of #ifdef CONFIG_ACPI.

What about PVonHVM. There is this nagging feeling in the back of my
head that turning CONFIG_XEN_DOM0 disables the CONFIG_PVHVM option?
Or something like that? Maybe it is the other way around?

It would seem we need to seperate the PVHVM  from DOM0. But the extra
complexity comes with Mukesh's PVonHVM Hybrid which can do dom0 stuff
with PVonHVM (should be searchable on xen-devel to find the patches).

Ian also mentioned that we MUST have the XEN_PRIVILIGED option, otherwise
we will cause a regression with the GRUB tools. So that must stay or we need
provide a deprecation schedule for the next 3 years to remove it and
have patches in the GRUB toolchains.

Either way, there is also the question of making sure that no regressions
are introduced - so a lot of the CONFIG_* interdependencies will have
to be checked. I think that means checking CONFIG_XEN, CONFIG_PRIVILIGED,
CONFIG_PVHVM, CONFIG_PCI, CONFIG_ACPI, CONFIG_XEN_BACKEND in various combinations.

Oh, I forgot CONFIG_XEN_FRONTEND.. And I think that one can also become
a module (ditto for XEN_BACKEND)

And everytime we did something to those we managed to mess them up so
we should be dilligient.

Re: [Xen-devel] [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Andrew Jones]

----- Original Message -----
> On Mon, Jan 09, 2012 at 04:12:10PM +0000, Stefano Stabellini wrote:
> > On Mon, 9 Jan 2012, Konrad Rzeszutek Wilk wrote:
> > > On Mon, Jan 09, 2012 at 11:39:44AM +0000, Stefano Stabellini
> > > wrote:
> > > > I don't think we should add "PCI_XEN && SWIOTLB_XEN &&
> > > > X86_LOCAL_APIC &&
> > > > X86_IO_APIC && ACPI && PCI" to XEN either.
> > > > However it should be possible to add only the right
> > > > dependencies to the
> > > > right places.
> > > > In practice it should be sufficient to:
> > > > 
> > > > - make PCI_XEN depend on X86_LOCAL_APIC && X86_IO_APIC && ACPI;
> > > 
> > > Not good. You can make non-ACPI builds with PCI.
> > > 
> > > .. and you are missing CONFIG_PCI in there.
> > > > 
> > > > - make XEN_PCIDEV_FRONTEND and XEN_PCIDEV_BACKEND depend on
> > > > PCI_XEN;
> > > 
> > > OK. That sounds good.
> > > > 
> > > > - remove the 'ifdef CONFIG_ACPI' from arch/x86/pci/xen.c.
> > > 
> > > No.. same issue - non-ACPI builds can be with PCI.
> > > > 
> > 
> > Right.. in that case we should carefully replace the 'ifdef
> > CONFIG_XEN_DOM0' with 'ifdef CONFIG_ACPI' in arch/x86/pci/xen.c.
> 
> .. which I think I did at some point as part of cleanup and then
> removed them since it peppered the xen.c with tons of #ifdef
> CONFIG_ACPI.
> 
> What about PVonHVM. There is this nagging feeling in the back of my
> head that turning CONFIG_XEN_DOM0 disables the CONFIG_PVHVM option?
> Or something like that? Maybe it is the other way around?
> 
> It would seem we need to seperate the PVHVM  from DOM0. But the extra
> complexity comes with Mukesh's PVonHVM Hybrid which can do dom0 stuff
> with PVonHVM (should be searchable on xen-devel to find the patches).

They're currently separate, i.e. no problem with DOM0 off and PVHVM on,
or vice-versa afaict. I don't know anything about the hybrid though.

> 
> Ian also mentioned that we MUST have the XEN_PRIVILIGED option,
> otherwise
> we will cause a regression with the GRUB tools. So that must stay or
> we need
> provide a deprecation schedule for the next 3 years to remove it and
> have patches in the GRUB toolchains.

Yeah, that's a bummer.

> 
> Either way, there is also the question of making sure that no
> regressions
> are introduced - so a lot of the CONFIG_* interdependencies will have
> to be checked. I think that means checking CONFIG_XEN,
> CONFIG_PRIVILIGED,
> CONFIG_PVHVM, CONFIG_PCI, CONFIG_ACPI, CONFIG_XEN_BACKEND in various
> combinations.
> 

I guess if we did the s/XEN_DOM0/LOCAL_APIC && IO_APIC && ACPI/ in
arch/x86/pci/xen.c it would be pretty easy to review for equivalence.
Then keep CONFIG_PRIVILIGED, but drop XEN_DOM0 from everywhere else and
compile in the 3 files. I don't think it makes much sense to do it
though. XEN_DOM0 keeps things tidier now and might be useful later.

> Oh, I forgot CONFIG_XEN_FRONTEND.. And I think that one can also
> become
> a module (ditto for XEN_BACKEND)
> 
> And everytime we did something to those we managed to mess them up so
> we should be dilligient.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 

Re: [Xen-devel] [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Stefano Stabellini]

On Mon, 9 Jan 2012, Andrew Jones wrote:
> I guess if we did the s/XEN_DOM0/LOCAL_APIC && IO_APIC && ACPI/ in
> arch/x86/pci/xen.c it would be pretty easy to review for equivalence.
> Then keep CONFIG_PRIVILIGED, but drop XEN_DOM0 from everywhere else and
> compile in the 3 files. I don't think it makes much sense to do it
> though. XEN_DOM0 keeps things tidier now and might be useful later.

we can keep things clean with the following:

#ifdef CONFIG_LOCAL_APIC && CONFIG_IO_APIC && CONFIG_ACPI && CONFIG_PCI_XEN

#define XEN_DOM0

#endif

in include/xen/xen.h.

So in the source files we can still '#ifdef XEN_DOM0', but at the same
time we can get rid of the build symbol: everybody wins.

Re: [Xen-devel] [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Jeremy Fitzhardinge]

On 01/10/2012 05:44 AM, Stefano Stabellini wrote:
> On Mon, 9 Jan 2012, Andrew Jones wrote:
>> I guess if we did the s/XEN_DOM0/LOCAL_APIC && IO_APIC && ACPI/ in
>> arch/x86/pci/xen.c it would be pretty easy to review for equivalence.
>> Then keep CONFIG_PRIVILIGED, but drop XEN_DOM0 from everywhere else and
>> compile in the 3 files. I don't think it makes much sense to do it
>> though. XEN_DOM0 keeps things tidier now and might be useful later.
> we can keep things clean with the following:
>
> #ifdef CONFIG_LOCAL_APIC && CONFIG_IO_APIC && CONFIG_ACPI && CONFIG_PCI_XEN
>
> #define XEN_DOM0
>
> #endif
>
> in include/xen/xen.h.
>
> So in the source files we can still '#ifdef XEN_DOM0', but at the same
> time we can get rid of the build symbol: everybody wins.

No, really, I think this is silly.  This kind of dependency information
should be encoded in the Kconfig system, and not reimplemented in an
ad-hoc way.

If there were a clean way to do what Andrew wants then I'd support it,
but this thread has descended into a spiral of madness, which makes me
think its a lost cause.

If the root complaint is that "customers think that anything set in
.config is a supported feature", then the solutions are to support all
the features in .config, re-educate the customers that they're wrong, or
maintain a local patch to do this stuff.

    J

Re: [Xen-devel] [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Stefano Stabellini]

On Mon, 9 Jan 2012, Konrad Rzeszutek Wilk wrote:
> On Mon, Jan 09, 2012 at 04:12:10PM +0000, Stefano Stabellini wrote:
> > On Mon, 9 Jan 2012, Konrad Rzeszutek Wilk wrote:
> > > On Mon, Jan 09, 2012 at 11:39:44AM +0000, Stefano Stabellini wrote:
> > > > I don't think we should add "PCI_XEN && SWIOTLB_XEN && X86_LOCAL_APIC &&
> > > > X86_IO_APIC && ACPI && PCI" to XEN either.
> > > > However it should be possible to add only the right dependencies to the
> > > > right places.
> > > > In practice it should be sufficient to:
> > > > 
> > > > - make PCI_XEN depend on X86_LOCAL_APIC && X86_IO_APIC && ACPI;
> > > 
> > > Not good. You can make non-ACPI builds with PCI.
> > > 
> > > .. and you are missing CONFIG_PCI in there.
> > > > 
> > > > - make XEN_PCIDEV_FRONTEND and XEN_PCIDEV_BACKEND depend on PCI_XEN;
> > > 
> > > OK. That sounds good.
> > > > 
> > > > - remove the 'ifdef CONFIG_ACPI' from arch/x86/pci/xen.c.
> > > 
> > > No.. same issue - non-ACPI builds can be with PCI.
> > > > 
> > 
> > Right.. in that case we should carefully replace the 'ifdef
> > CONFIG_XEN_DOM0' with 'ifdef CONFIG_ACPI' in arch/x86/pci/xen.c.
> 
> .. which I think I did at some point as part of cleanup and then
> removed them since it peppered the xen.c with tons of #ifdef CONFIG_ACPI.

that's unfortunate


> What about PVonHVM. There is this nagging feeling in the back of my
> head that turning CONFIG_XEN_DOM0 disables the CONFIG_PVHVM option?
> Or something like that? Maybe it is the other way around?

nope, nothing I can think of.


> It would seem we need to seperate the PVHVM  from DOM0. But the extra
> complexity comes with Mukesh's PVonHVM Hybrid which can do dom0 stuff
> with PVonHVM (should be searchable on xen-devel to find the patches).

I think that at the moment PVHVM and DOM0 aren't tie to one another in
any way, apart from the fact that they use the same generic
infrastructure to remap interrupts and MSIs into event channels.

Re: [Xen-devel] [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Andrew Jones]

----- Original Message -----
> On Mon, 9 Jan 2012, Konrad Rzeszutek Wilk wrote:
> > On Mon, Jan 09, 2012 at 11:39:44AM +0000, Stefano Stabellini wrote:
> > > I don't think we should add "PCI_XEN && SWIOTLB_XEN &&
> > > X86_LOCAL_APIC &&
> > > X86_IO_APIC && ACPI && PCI" to XEN either.
> > > However it should be possible to add only the right dependencies
> > > to the
> > > right places.
> > > In practice it should be sufficient to:
> > > 
> > > - make PCI_XEN depend on X86_LOCAL_APIC && X86_IO_APIC && ACPI;
> > 
> > Not good. You can make non-ACPI builds with PCI.
> > 
> > .. and you are missing CONFIG_PCI in there.
> > > 
> > > - make XEN_PCIDEV_FRONTEND and XEN_PCIDEV_BACKEND depend on
> > > PCI_XEN;
> > 
> > OK. That sounds good.
> > > 
> > > - remove the 'ifdef CONFIG_ACPI' from arch/x86/pci/xen.c.
> > 
> > No.. same issue - non-ACPI builds can be with PCI.
> > > 
> 
> Right.. in that case we should carefully replace the 'ifdef
> CONFIG_XEN_DOM0' with 'ifdef CONFIG_ACPI' in arch/x86/pci/xen.c.
> It would effectively keep things as they are now: if ACPI and XEN are
> enabled together in the config file, your kernel is able to setup
> interrupts when running as DOM0.
> Regarding X86_LOCAL_APIC and X86_IO_APIC I cannot recall anymore
> where
> these dependencies come from exactly.
> 

Here's one path

pci_xen_initial_domain [arch/x86/pci/xen.c] (#ifdef XEN_DOM0)
   xen_setup_acpi_sci [arch/x86/pci/xen.c]  (#ifdef XEN_DOM0)
       acpi_get_override_irq [arch/x86/kernel/apic/io_apic.c]
       (#ifdef X86_IO_APIC)

The messiness I stated before comes in when we try to find all these
paths and make sure they're appropriately #ifdef'ed with the minimal
set of symbols from the set that XEN_DOM0 used to cover for us.
However, without alternative implementations in the absence of
particular config options, then we still need to stub the functions out
at the top level. So we could simply s/r XEN_DOM0 with X86_LOCAL_APIC
&& X86_IO_APIC && ACPI in arch/x86/pci/xen.c, but that doesn't make
much sense either, because XEN_DOM0 does a nice job keeping things neat
and well documented, i.e. we can quickly tell what functions are
dom0-only.

> 
> > > > The driver domain concept may actually be the technical need
> > > > for
> > > > making XEN_DOM0 optional. If we want the ability to build a
> > > > minimally
> > > > configed XEN kernel to be used as a driver domain, then it
> > > > doesn't
> > > > seem like we'd want it to also be capable of running as an
> > > > initial
> > > > domain, or to be forced to have all the dependencies and code
> > > > of an
> > > > initial domain.
> > > 
> > > In practice any decent driver domain needs PCI and ACPI support,
> > > so
> > > basically it has the same config requirement of XEN_DOM0. At that
> > > point
> > 
> > Sure.. for backends. But for frontends that requirement is not
> > always true.
> 
> Right but a driver domain is bound to have at least the pci backend.
> 
> 
> > > we have already discussed that having XEN_DOM0 enabled or
> > > disabled
> > > hardly makes any differences, so we can just get rid of it.
> > 
> > Or in other words, substitute it as CONFIG_PCI_XEN.
> 
> I was just trying to assign the dependencies where they really belong
> and I proposed to remove the 'ifdef CONFIG_ACPI' because I didn't
> realize that one can build working PCI configurations on XEN without
> it.
> 
> The fact that PCI_XEN would be used then almost as XEN_DOM0 is used
> now
> is just a side effect, due to the fact that PCI and device interrupts
> initialization is what mainly differentiates dom0 from other
> configurations.
> 
> 
> > But this brings a question about MCE, ACPI cpu freq and ACPI S3.
> > Those all belong to the dom0 - not to any driver domain. So getting
> > rid
> > of CONFIG_XEN_DOM0 would present a problem - what would those
> > depend on?
> 
> They would depend on XEN and whatever else they really depend on, for
> example ACPI.

This brings up my comment before about me not being the best person
to suggest removing XEN_DOM0. It appears this symbol is fairly useful
even now for the reasons I mentioned above, and future work may once
again find it the cleanest way to manage dependencies. Possibly
dependencies will even expand to a point where it sufficiently
diverges from the requirements of driver domains. Then users may like
to be able to configure it out. I don't know.

Drew

Re: [Xen-devel] [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Ian Campbell]

On Fri, 2012-01-06 at 16:39 +0000, Andrew Jones wrote:
> remove XEN_PRIVILEGED_GUEST as it's just an alias for XEN_DOM0.

Hmm, this one is used by tools like update-grub to know when it is ok to
create xen+kernel entries so I think it needs to stay, or we at least
need to give lengthly warning to distros etc that it is going away.

Perhaps you can just patch it out locally?

Ian.

> 
> I compile tested this on a latest pull using an F16 config. The compile
> succeeded and 'make oldconfig' only removed these two options as
> expected.
> 
> CONFIG_XEN_DOM0=y
> CONFIG_XEN_PRIVILEGED_GUEST=y
> 
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> ---
>  arch/x86/include/asm/xen/pci.h |   21 +--------------------
>  arch/x86/pci/xen.c             |    6 ------
>  arch/x86/xen/Kconfig           |   10 ----------
>  arch/x86/xen/Makefile          |    3 +--
>  arch/x86/xen/xen-ops.h         |    7 -------
>  drivers/xen/Kconfig            |    3 ++-
>  drivers/xen/Makefile           |    3 +--
>  drivers/xen/xenfs/Makefile     |    3 +--
>  include/xen/xen.h              |   11 +++--------
>  9 files changed, 9 insertions(+), 58 deletions(-)
> 
> diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
> index 968d57d..b423889 100644
> --- a/arch/x86/include/asm/xen/pci.h
> +++ b/arch/x86/include/asm/xen/pci.h
> @@ -13,30 +13,11 @@ static inline int pci_xen_hvm_init(void)
>  	return -1;
>  }
>  #endif
> -#if defined(CONFIG_XEN_DOM0)
> +
>  int __init pci_xen_initial_domain(void);
>  int xen_find_device_domain_owner(struct pci_dev *dev);
>  int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
>  int xen_unregister_device_domain_owner(struct pci_dev *dev);
> -#else
> -static inline int __init pci_xen_initial_domain(void)
> -{
> -	return -1;
> -}
> -static inline int xen_find_device_domain_owner(struct pci_dev *dev)
> -{
> -	return -1;
> -}
> -static inline int xen_register_device_domain_owner(struct pci_dev *dev,
> -						   uint16_t domain)
> -{
> -	return -1;
> -}
> -static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
> -{
> -	return -1;
> -}
> -#endif
>  
>  #if defined(CONFIG_PCI_MSI)
>  #if defined(CONFIG_PCI_XEN)
> diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
> index 492ade8..e298726 100644
> --- a/arch/x86/pci/xen.c
> +++ b/arch/x86/pci/xen.c
> @@ -108,7 +108,6 @@ static int acpi_register_gsi_xen_hvm(struct device *dev, u32 gsi,
>  				 false /* no mapping of GSI to PIRQ */);
>  }
>  
> -#ifdef CONFIG_XEN_DOM0
>  static int xen_register_gsi(u32 gsi, int gsi_override, int triggering, int polarity)
>  {
>  	int rc, irq;
> @@ -143,7 +142,6 @@ static int acpi_register_gsi_xen(struct device *dev, u32 gsi,
>  	return xen_register_gsi(gsi, -1 /* no GSI override */, trigger, polarity);
>  }
>  #endif
> -#endif
>  
>  #if defined(CONFIG_PCI_MSI)
>  #include <linux/msi.h>
> @@ -251,7 +249,6 @@ error:
>  	return irq;
>  }
>  
> -#ifdef CONFIG_XEN_DOM0
>  static bool __read_mostly pci_seg_supported = true;
>  
>  static int xen_initdom_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
> @@ -324,7 +321,6 @@ static int xen_initdom_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
>  out:
>  	return ret;
>  }
> -#endif
>  
>  static void xen_teardown_msi_irqs(struct pci_dev *dev)
>  {
> @@ -392,7 +388,6 @@ int __init pci_xen_hvm_init(void)
>  	return 0;
>  }
>  
> -#ifdef CONFIG_XEN_DOM0
>  static __init void xen_setup_acpi_sci(void)
>  {
>  	int rc;
> @@ -539,4 +534,3 @@ int xen_unregister_device_domain_owner(struct pci_dev *dev)
>  	return 0;
>  }
>  EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
> -#endif
> diff --git a/arch/x86/xen/Kconfig b/arch/x86/xen/Kconfig
> index 26c731a..3c7e89a 100644
> --- a/arch/x86/xen/Kconfig
> +++ b/arch/x86/xen/Kconfig
> @@ -13,16 +13,6 @@ config XEN
>  	  kernel to boot in a paravirtualized environment under the
>  	  Xen hypervisor.
>  
> -config XEN_DOM0
> -	def_bool y
> -	depends on XEN && PCI_XEN && SWIOTLB_XEN
> -	depends on X86_LOCAL_APIC && X86_IO_APIC && ACPI && PCI
> -
> -# Dummy symbol since people have come to rely on the PRIVILEGED_GUEST
> -# name in tools.
> -config XEN_PRIVILEGED_GUEST
> -	def_bool XEN_DOM0
> -
>  config XEN_PVHVM
>  	def_bool y
>  	depends on XEN && PCI && X86_LOCAL_APIC
> diff --git a/arch/x86/xen/Makefile b/arch/x86/xen/Makefile
> index add2c2d..b2d4c4b 100644
> --- a/arch/x86/xen/Makefile
> +++ b/arch/x86/xen/Makefile
> @@ -13,12 +13,11 @@ CFLAGS_mmu.o			:= $(nostackp)
>  obj-y		:= enlighten.o setup.o multicalls.o mmu.o irq.o \
>  			time.o xen-asm.o xen-asm_$(BITS).o \
>  			grant-table.o suspend.o platform-pci-unplug.o \
> -			p2m.o
> +			p2m.o vga.o
>  
>  obj-$(CONFIG_EVENT_TRACING) += trace.o
>  
>  obj-$(CONFIG_SMP)		+= smp.o
>  obj-$(CONFIG_PARAVIRT_SPINLOCKS)+= spinlock.o
>  obj-$(CONFIG_XEN_DEBUG_FS)	+= debugfs.o
> -obj-$(CONFIG_XEN_DOM0)		+= vga.o
>  obj-$(CONFIG_SWIOTLB_XEN)	+= pci-swiotlb-xen.o
> diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h
> index b095739..d9dbbca 100644
> --- a/arch/x86/xen/xen-ops.h
> +++ b/arch/x86/xen/xen-ops.h
> @@ -90,14 +90,7 @@ static inline void xen_uninit_lock_cpu(int cpu)
>  
>  struct dom0_vga_console_info;
>  
> -#ifdef CONFIG_XEN_DOM0
>  void __init xen_init_vga(const struct dom0_vga_console_info *, size_t size);
> -#else
> -static inline void __init xen_init_vga(const struct dom0_vga_console_info *info,
> -				       size_t size)
> -{
> -}
> -#endif
>  
>  /* Declare an asm function, along with symbols needed to make it
>     inlineable */
> diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
> index 8795480..0725228 100644
> --- a/drivers/xen/Kconfig
> +++ b/drivers/xen/Kconfig
> @@ -78,8 +78,9 @@ config XEN_DEV_EVTCHN
>  
>  config XEN_BACKEND
>  	bool "Backend driver support"
> -	depends on XEN_DOM0
>  	default y
> +	depends on XEN && PCI_XEN && SWIOTLB_XEN
> +	depends on X86_LOCAL_APIC && X86_IO_APIC && ACPI && PCI
>  	help
>  	  Support for backend device drivers that provide I/O services
>  	  to other virtual machines.
> diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
> index 974fffd..7e61662 100644
> --- a/drivers/xen/Makefile
> +++ b/drivers/xen/Makefile
> @@ -1,4 +1,4 @@
> -obj-y	+= grant-table.o features.o events.o manage.o balloon.o
> +obj-y	+= grant-table.o features.o events.o manage.o balloon.o pci.o
>  obj-y	+= xenbus/
>  
>  nostackp := $(call cc-option, -fno-stack-protector)
> @@ -17,7 +17,6 @@ obj-$(CONFIG_XEN_SYS_HYPERVISOR)	+= sys-hypervisor.o
>  obj-$(CONFIG_XEN_PVHVM)			+= platform-pci.o
>  obj-$(CONFIG_XEN_TMEM)			+= tmem.o
>  obj-$(CONFIG_SWIOTLB_XEN)		+= swiotlb-xen.o
> -obj-$(CONFIG_XEN_DOM0)			+= pci.o
>  obj-$(CONFIG_XEN_PCIDEV_BACKEND)	+= xen-pciback/
>  
>  xen-evtchn-y				:= evtchn.o
> diff --git a/drivers/xen/xenfs/Makefile b/drivers/xen/xenfs/Makefile
> index 4fde944..d20c060 100644
> --- a/drivers/xen/xenfs/Makefile
> +++ b/drivers/xen/xenfs/Makefile
> @@ -1,4 +1,3 @@
>  obj-$(CONFIG_XENFS) += xenfs.o
>  
> -xenfs-y			  = super.o xenbus.o privcmd.o
> -xenfs-$(CONFIG_XEN_DOM0) += xenstored.o
> +xenfs-y			  = super.o xenbus.o privcmd.o xenstored.o
> diff --git a/include/xen/xen.h b/include/xen/xen.h
> index a164024..d814ef7 100644
> --- a/include/xen/xen.h
> +++ b/include/xen/xen.h
> @@ -1,6 +1,9 @@
>  #ifndef _XEN_XEN_H
>  #define _XEN_XEN_H
>  
> +#include <xen/interface/xen.h>
> +#include <asm/xen/hypervisor.h>
> +
>  enum xen_domain_type {
>  	XEN_NATIVE,		/* running on bare hardware    */
>  	XEN_PV_DOMAIN,		/* running in a PV domain      */
> @@ -18,15 +21,7 @@ extern enum xen_domain_type xen_domain_type;
>  				 xen_domain_type == XEN_PV_DOMAIN)
>  #define xen_hvm_domain()	(xen_domain() &&			\
>  				 xen_domain_type == XEN_HVM_DOMAIN)
> -
> -#ifdef CONFIG_XEN_DOM0
> -#include <xen/interface/xen.h>
> -#include <asm/xen/hypervisor.h>
> -
>  #define xen_initial_domain()	(xen_pv_domain() && \
>  				 xen_start_info->flags & SIF_INITDOMAIN)
> -#else  /* !CONFIG_XEN_DOM0 */
> -#define xen_initial_domain()	(0)
> -#endif	/* CONFIG_XEN_DOM0 */
>  
>  #endif	/* _XEN_XEN_H */

Re: [Xen-devel] [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Andrew Jones]

----- Original Message -----
> On 01/10/2012 05:44 AM, Stefano Stabellini wrote:
> > On Mon, 9 Jan 2012, Andrew Jones wrote:
> >> I guess if we did the s/XEN_DOM0/LOCAL_APIC && IO_APIC && ACPI/ in
> >> arch/x86/pci/xen.c it would be pretty easy to review for
> >> equivalence.
> >> Then keep CONFIG_PRIVILIGED, but drop XEN_DOM0 from everywhere
> >> else and
> >> compile in the 3 files. I don't think it makes much sense to do it
> >> though. XEN_DOM0 keeps things tidier now and might be useful
> >> later.
> > we can keep things clean with the following:
> >
> > #ifdef CONFIG_LOCAL_APIC && CONFIG_IO_APIC && CONFIG_ACPI &&
> > CONFIG_PCI_XEN
> >
> > #define XEN_DOM0
> >
> > #endif
> >
> > in include/xen/xen.h.
> >
> > So in the source files we can still '#ifdef XEN_DOM0', but at the
> > same
> > time we can get rid of the build symbol: everybody wins.
> 
> No, really, I think this is silly.  This kind of dependency
> information
> should be encoded in the Kconfig system, and not reimplemented in an
> ad-hoc way.
> 
> If there were a clean way to do what Andrew wants then I'd support
> it,
> but this thread has descended into a spiral of madness, which makes
> me
> think its a lost cause.
> 
> If the root complaint is that "customers think that anything set in
> .config is a supported feature", then the solutions are to support
> all
> the features in .config, re-educate the customers that they're wrong,
> or
> maintain a local patch to do this stuff.

If only re-educating people was free, like preempting questions is.
Local patches are of course always an option, and perhaps in this
case it's the best one. However, I think we already made a case for
better xen configurability for the driver domains, so I'm not 100%
convinced my initial patch (making dom0 configurable) isn't worthy
of upstream. Also, I didn't see any comments on my v2[*] of that
patch, which I believe satisfies the menu complexity issue and
brings in more configurability. That said, I'm about to reply to
that patch myself, since there's an issue with it.

Drew

[*] http://article.gmane.org/gmane.linux.kernel.virtualization/14635

Re: [Xen-devel] [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Konrad Rzeszutek Wilk]

> > If the root complaint is that "customers think that anything set in
> > .config is a supported feature", then the solutions are to support
> > all
> > the features in .config, re-educate the customers that they're wrong,
> > or
> > maintain a local patch to do this stuff.
> 
> If only re-educating people was free, like preempting questions is.
> Local patches are of course always an option, and perhaps in this
> case it's the best one. However, I think we already made a case for
> better xen configurability for the driver domains, so I'm not 100%

Could you repost those backend patches please? At this point I am not
sure which one we have discarded?

> convinced my initial patch (making dom0 configurable) isn't worthy
> of upstream. Also, I didn't see any comments on my v2[*] of that
> patch, which I believe satisfies the menu complexity issue and
> brings in more configurability. That said, I'm about to reply to
> that patch myself, since there's an issue with it.
> 
> Drew
> 
> [*] http://article.gmane.org/gmane.linux.kernel.virtualization/14635
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: [Xen-devel] [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Konrad Rzeszutek Wilk]

On Wed, Jan 11, 2012 at 12:19:11PM -0400, Konrad Rzeszutek Wilk wrote:
> > > If the root complaint is that "customers think that anything set in
> > > .config is a supported feature", then the solutions are to support
> > > all
> > > the features in .config, re-educate the customers that they're wrong,
> > > or
> > > maintain a local patch to do this stuff.
> > 
> > If only re-educating people was free, like preempting questions is.
> > Local patches are of course always an option, and perhaps in this
> > case it's the best one. However, I think we already made a case for
> > better xen configurability for the driver domains, so I'm not 100%
> 
> Could you repost those backend patches please? At this point I am not
> sure which one we have discarded?

hm, I was thinking in terms of the XenBus ones. We had somewhere in this
converstion something about seperating the backend's from depending on
CONFIG_XEN_DOM0 as they can be run in any domain nowadays.

> 
> > convinced my initial patch (making dom0 configurable) isn't worthy
> > of upstream. Also, I didn't see any comments on my v2[*] of that
> > patch, which I believe satisfies the menu complexity issue and
> > brings in more configurability. That said, I'm about to reply to
> > that patch myself, since there's an issue with it.
> > 
> > Drew
> > 
> > [*] http://article.gmane.org/gmane.linux.kernel.virtualization/14635
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel

Re: [Xen-devel] [PATCH] xen: remove CONFIG_XEN_DOM0 compile option

[Andrew Jones]

----- Original Message -----
> On Wed, Jan 11, 2012 at 12:19:11PM -0400, Konrad Rzeszutek Wilk
> wrote:
> > > > If the root complaint is that "customers think that anything
> > > > set in
> > > > .config is a supported feature", then the solutions are to
> > > > support
> > > > all
> > > > the features in .config, re-educate the customers that they're
> > > > wrong,
> > > > or
> > > > maintain a local patch to do this stuff.
> > > 
> > > If only re-educating people was free, like preempting questions
> > > is.
> > > Local patches are of course always an option, and perhaps in this
> > > case it's the best one. However, I think we already made a case
> > > for
> > > better xen configurability for the driver domains, so I'm not
> > > 100%
> > 
> > Could you repost those backend patches please? At this point I am
> > not
> > sure which one we have discarded?
> 
> hm, I was thinking in terms of the XenBus ones. We had somewhere in
> this
> converstion something about seperating the backend's from depending
> on
> CONFIG_XEN_DOM0 as they can be run in any domain nowadays.
> 

Ah, I still haven't posted anything like that. So nothing got lost :-)
It's a good idea, but it's probably best done and tested by someone
working with driver domains, or even with the backends at all, which
I'm not.

Drew

> > 
> > > convinced my initial patch (making dom0 configurable) isn't
> > > worthy
> > > of upstream. Also, I didn't see any comments on my v2[*] of that
> > > patch, which I believe satisfies the menu complexity issue and
> > > brings in more configurability. That said, I'm about to reply to
> > > that patch myself, since there's an issue with it.
> > > 
> > > Drew
> > > 
> > > [*]
> > > http://article.gmane.org/gmane.linux.kernel.virtualization/14635
> > > 
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@lists.xensource.com
> > > http://lists.xensource.com/xen-devel
>