[PATCH v3 0/5] Allow schedulers to be selectable through Kconfig

[PATCH v3 0/5] Allow schedulers to be selectable through Kconfig

[Jonathan Creekmore]

Add machinery to allow the schedulers to be individually selectable
through the Kconfig interface. The first patch in the series sets up
the CONFIG_EXPERT Kconfig variable that is only enabled by passing an
environment variable to the build. The second patch in the series sets up
the Kconfig options for the schedulers and places the appropriate CONFIG_
options around the scheduler list. The third, fourth, and fifth patches
rework the scheduler list from being manually curated into a model
where just compiling any schedulers into the hypervisor causes the
scheduler list to be built up.

---
Changed since v2:
 * Added a predecessor patch that introduces a environment
   variable for the build to enable expert configuration options
   (1/5)
 * Hide the scheduler menu behind the expert option (2/5)
 * Provide static inlines for credit functions that are assumed to be
   present if it is compiled out (2/5)
 * Provide an absolute default of the credit scheduler if the 
   scheduler menu is not visible (2/5)

Changed since v1:
 * Marked credit2 as EXPERIMENTAL
 * Removed confusing language from the commit message
 * Alphabetize the schedulers
 * rename the __start and __end symbols to better match
   the rest of the file
 * Simplify the calculation of the number of schedulers
 * Make the scheduler ops structures static to their files

Jonathan Creekmore (5):
  build: Env var to enable expert config options
  build: Hook the schedulers into Kconfig
  build: Alloc space for sched list in the link file
  sched: Register the schedulers into the list
  sched: Use the auto-generated list of schedulers

 xen/Kconfig                 |  4 +++
 xen/Makefile                |  1 +
 xen/arch/arm/xen.lds.S      |  4 +++
 xen/arch/x86/xen.lds.S      |  4 +++
 xen/common/Kconfig          | 67 +++++++++++++++++++++++++++++++++++++++++++++
 xen/common/Makefile         |  8 +++---
 xen/common/sched_arinc653.c |  4 ++-
 xen/common/sched_credit.c   |  4 ++-
 xen/common/sched_credit2.c  |  4 ++-
 xen/common/sched_rt.c       |  4 ++-
 xen/common/schedule.c       | 20 ++++++--------
 xen/include/xen/sched-if.h  |  7 ++---
 xen/include/xen/sched.h     |  5 ++++
 13 files changed, 112 insertions(+), 24 deletions(-)

-- 
2.6.4

[PATCH v3 1/5] build: Env var to enable expert config options

[Jonathan Creekmore]

Add an additional environment variable, defaulting to disabled,
that enables the CONFIG_EXPERT configuration option. The purpose
of the CONFIG_EXPERT configuration option is to make non-standard
Kconfig options visible during the configuration process. The
CONFIG_EXPERT option is not, itself, visible during the Kconfig
configuration process, so typical users will never see it nor
any of the non-standard configuration options.

CC: Ian Campbell <ian.campbell@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Jan Beulich <jbeulich@suse.com>
CC: Keir Fraser <keir@xen.org>
CC: Tim Deegan <tim@xen.org>
Signed-off-by: Jonathan Creekmore <jonathan.creekmore@gmail.com>
Reviewed-by: Doug Goldstein <cardoe@cardoe.com>

---
 xen/Kconfig  | 4 ++++
 xen/Makefile | 1 +
 2 files changed, 5 insertions(+)

diff --git a/xen/Kconfig b/xen/Kconfig
index ffe3f45..fa8b27c 100644
--- a/xen/Kconfig
+++ b/xen/Kconfig
@@ -22,3 +22,7 @@ config DEFCONFIG_LIST
 	string
 	option defconfig_list
 	default "$ARCH_DEFCONFIG"
+
+config EXPERT
+	string
+	option env="XEN_CONFIG_EXPERT"
diff --git a/xen/Makefile b/xen/Makefile
index 9023863..4950afb 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -11,6 +11,7 @@ export XEN_DOMAIN	?= $(shell ([ -x /bin/dnsdomainname ] && /bin/dnsdomainname) |
 export XEN_BUILD_DATE	?= $(shell LC_ALL=C date)
 export XEN_BUILD_TIME	?= $(shell LC_ALL=C date +%T)
 export XEN_BUILD_HOST	?= $(shell hostname)
+export XEN_CONFIG_EXPERT ?= n
 
 export BASEDIR := $(CURDIR)
 export XEN_ROOT := $(BASEDIR)/..
-- 
2.6.4

Re: [PATCH v3 1/5] build: Env var to enable expert config options

[Konrad Rzeszutek Wilk]

On Thu, Jan 07, 2016 at 11:29:17AM -0600, Jonathan Creekmore wrote:
> Add an additional environment variable, defaulting to disabled,
> that enables the CONFIG_EXPERT configuration option. The purpose
> of the CONFIG_EXPERT configuration option is to make non-standard
> Kconfig options visible during the configuration process. The
> CONFIG_EXPERT option is not, itself, visible during the Kconfig
> configuration process, so typical users will never see it nor
> any of the non-standard configuration options.
> 
> CC: Ian Campbell <ian.campbell@citrix.com>
> CC: Ian Jackson <ian.jackson@eu.citrix.com>
> CC: Jan Beulich <jbeulich@suse.com>
> CC: Keir Fraser <keir@xen.org>
> CC: Tim Deegan <tim@xen.org>
> Signed-off-by: Jonathan Creekmore <jonathan.creekmore@gmail.com>
> Reviewed-by: Doug Goldstein <cardoe@cardoe.com>

Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> 
> ---
>  xen/Kconfig  | 4 ++++
>  xen/Makefile | 1 +
>  2 files changed, 5 insertions(+)
> 
> diff --git a/xen/Kconfig b/xen/Kconfig
> index ffe3f45..fa8b27c 100644
> --- a/xen/Kconfig
> +++ b/xen/Kconfig
> @@ -22,3 +22,7 @@ config DEFCONFIG_LIST
>  	string
>  	option defconfig_list
>  	default "$ARCH_DEFCONFIG"
> +
> +config EXPERT
> +	string
> +	option env="XEN_CONFIG_EXPERT"
> diff --git a/xen/Makefile b/xen/Makefile
> index 9023863..4950afb 100644
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -11,6 +11,7 @@ export XEN_DOMAIN	?= $(shell ([ -x /bin/dnsdomainname ] && /bin/dnsdomainname) |
>  export XEN_BUILD_DATE	?= $(shell LC_ALL=C date)
>  export XEN_BUILD_TIME	?= $(shell LC_ALL=C date +%T)
>  export XEN_BUILD_HOST	?= $(shell hostname)
> +export XEN_CONFIG_EXPERT ?= n
>  
>  export BASEDIR := $(CURDIR)
>  export XEN_ROOT := $(BASEDIR)/..
> -- 
> 2.6.4
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

[PATCH v3 2/5] build: Hook the schedulers into Kconfig

[Jonathan Creekmore]

Allow the schedulers to be independently enabled or disabled at
compile-time. To match existing behavior, all four schedulers are
compiled in by default, although the Credit2, RTDS, and ARINC653 are
marked EXPERIMENTAL to match their not currently supported status.

CC: George Dunlap <george.dunlap@eu.citrix.com>
CC: Dario Faggioli <dario.faggioli@citrix.com>
Signed-off-by: Jonathan Creekmore <jonathan.creekmore@gmail.com>
Reviewed-by: Doug Goldstein <cardoe@cardoe.com>

---
Changed since v2:

  * Hid the scheduler menu behind the EXPERT option
  * Provide static inlines for credit functions that are assumed to be
    always available
  * Provide an absolute default of the credit scheduler if the
    scheduler menu is not visible

Changed since v1:

  * Marked credit2 as EXPERIMENTAL
  * Removed confusing language from the commit message
  * Alphabetize the schedulers
---
 xen/common/Kconfig      | 67 +++++++++++++++++++++++++++++++++++++++++++++++++
 xen/common/Makefile     |  8 +++---
 xen/common/schedule.c   | 12 +++++++--
 xen/include/xen/sched.h |  5 ++++
 4 files changed, 86 insertions(+), 6 deletions(-)

diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 046e257..db7411b 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -51,4 +51,71 @@ config KEXEC
 
 	  If unsure, say Y.
 
+# Enable schedulers
+menu "Schedulers"
+	visible if EXPERT = "y"
+
+config SCHED_CREDIT
+	bool "Credit scheduler support"
+	default y
+	---help---
+	  The traditional credit scheduler is a general purpose scheduler.
+
+	  If unsure, say Y.
+
+config SCHED_CREDIT2
+	bool "Credit2 scheduler support (EXPERIMENTAL)"
+	default y
+	---help---
+	  The credit2 scheduler is a general purpose scheduler that is
+	  optimized for lower latency and higher VM density.
+
+	  If unsure, say Y.
+
+config SCHED_RTDS
+	bool "RTDS scheduler support (EXPERIMENTAL)"
+	default y
+	---help---
+	  The RTDS scheduler is a soft and firm real-time scheduler for
+	  multicore, targeted for embedded, automotive, graphics and gaming
+	  in the cloud, and general low-latency workloads.
+
+	  If unsure, say N.
+
+config SCHED_ARINC653
+	bool "ARINC653 scheduler support (EXPERIMENTAL)"
+	default y
+	---help---
+	  The ARINC653 scheduler is a hard real-time scheduler for single
+	  cores, targeted for avionics, drones, and medical devices.
+
+	  If unsure, say N.
+
+choice
+	prompt "Default Scheduler?"
+	default SCHED_CREDIT_DEFAULT if SCHED_CREDIT
+	default SCHED_CREDIT2_DEFAULT if SCHED_CREDIT2
+	default SCHED_RTDS_DEFAULT if SCHED_RTDS
+	default SCHED_ARINC653_DEFAULT if SCHED_ARINC653
+
+	config SCHED_CREDIT_DEFAULT
+		bool "Credit Scheduler" if SCHED_CREDIT
+	config SCHED_CREDIT2_DEFAULT
+		bool "Credit2 Scheduler" if SCHED_CREDIT2
+	config SCHED_RTDS_DEFAULT
+		bool "RT Scheduler" if SCHED_RTDS
+	config SCHED_ARINC653_DEFAULT
+		bool "ARINC653 Scheduler" if SCHED_ARINC653
+endchoice
+
+config SCHED_DEFAULT
+	string
+	default "credit" if SCHED_CREDIT_DEFAULT
+	default "credit2" if SCHED_CREDIT2_DEFAULT
+	default "rtds" if SCHED_RTDS_DEFAULT
+	default "arinc653" if SCHED_ARINC653_DEFAULT
+	default "credit"
+
+endmenu
+
 endmenu
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 8ab15ba..29a5916 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -30,10 +30,10 @@ obj-y += rangeset.o
 obj-y += radix-tree.o
 obj-y += rbtree.o
 obj-y += rcupdate.o
-obj-y += sched_credit.o
-obj-y += sched_credit2.o
-obj-y += sched_arinc653.o
-obj-y += sched_rt.o
+obj-$(CONFIG_SCHED_ARINC653) += sched_arinc653.o
+obj-$(CONFIG_SCHED_CREDIT) += sched_credit.o
+obj-$(CONFIG_SCHED_CREDIT2) += sched_credit2.o
+obj-$(CONFIG_SCHED_RTDS) += sched_rt.o
 obj-y += schedule.o
 obj-y += shutdown.o
 obj-y += softirq.o
diff --git a/xen/common/schedule.c b/xen/common/schedule.c
index d121896..2f98a48 100644
--- a/xen/common/schedule.c
+++ b/xen/common/schedule.c
@@ -38,8 +38,8 @@
 #include <public/sched.h>
 #include <xsm/xsm.h>
 
-/* opt_sched: scheduler - default to credit */
-static char __initdata opt_sched[10] = "credit";
+/* opt_sched: scheduler - default to configured value */
+static char __initdata opt_sched[10] = CONFIG_SCHED_DEFAULT;
 string_param("sched", opt_sched);
 
 /* if sched_smt_power_savings is set,
@@ -65,10 +65,18 @@ DEFINE_PER_CPU(struct schedule_data, schedule_data);
 DEFINE_PER_CPU(struct scheduler *, scheduler);
 
 static const struct scheduler *schedulers[] = {
+#ifdef CONFIG_SCHED_CREDIT
     &sched_credit_def,
+#endif
+#ifdef CONFIG_SCHED_CREDIT2
     &sched_credit2_def,
+#endif
+#ifdef CONFIG_SCHED_ARINC653
     &sched_arinc653_def,
+#endif
+#ifdef CONFIG_SCHED_RTDS
     &sched_rtds_def,
+#endif
 };
 
 static struct scheduler __read_mostly ops;
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index fc61fc3..246338e 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -850,8 +850,13 @@ static inline bool_t is_vcpu_online(const struct vcpu *v)
     return !test_bit(_VPF_down, &v->pause_flags);
 }
 
+#ifdef CONFIG_SCHED_CREDIT
 void set_vcpu_migration_delay(unsigned int delay);
 unsigned int get_vcpu_migration_delay(void);
+#else
+static inline void set_vcpu_migration_delay(unsigned int delay) { }
+static inline unsigned int get_vcpu_migration_delay(void) { return 0; }
+#endif
 
 extern bool_t sched_smt_power_savings;
 
-- 
2.6.4

Re: [PATCH v3 2/5] build: Hook the schedulers into Kconfig

[Konrad Rzeszutek Wilk]

On Thu, Jan 07, 2016 at 11:29:18AM -0600, Jonathan Creekmore wrote:
> Allow the schedulers to be independently enabled or disabled at
> compile-time. To match existing behavior, all four schedulers are
> compiled in by default, although the Credit2, RTDS, and ARINC653 are
> marked EXPERIMENTAL to match their not currently supported status.

By unmarking all of them in sequence I was able to get this:

#
# Schedulers
#
# CONFIG_SCHED_CREDIT is not set
# CONFIG_SCHED_CREDIT2 is not set
# CONFIG_SCHED_RTDS is not set
# CONFIG_SCHED_ARINC653 is not set
# CONFIG_SCHED_CREDIT_DEFAULT is not set
# CONFIG_SCHED_CREDIT2_DEFAULT is not set
# CONFIG_SCHED_RTDS_DEFAULT is not set
# CONFIG_SCHED_ARINC653_DEFAULT is not set
CONFIG_SCHED_DEFAULT="credit"


And the hypervisor did build with:

[konrad@char xen]$ nm --defined xen-syms |grep schedulers
ffff82d080290d58 D __end_schedulers_array
ffff82d080290d58 D __start_schedulers_array

:-)

Not exactly sure if there is some way to make us _not_ shoot
ourselves in the foot by mistake.

Perhaps the build should complain if the size of the
__schedulers_array is zero?

Re: [PATCH v3 2/5] build: Hook the schedulers into Kconfig

[Andrew Cooper]

On 08/01/16 15:47, Konrad Rzeszutek Wilk wrote:
> On Thu, Jan 07, 2016 at 11:29:18AM -0600, Jonathan Creekmore wrote:
>> Allow the schedulers to be independently enabled or disabled at
>> compile-time. To match existing behavior, all four schedulers are
>> compiled in by default, although the Credit2, RTDS, and ARINC653 are
>> marked EXPERIMENTAL to match their not currently supported status.
> By unmarking all of them in sequence I was able to get this:
>
> #
> # Schedulers
> #
> # CONFIG_SCHED_CREDIT is not set
> # CONFIG_SCHED_CREDIT2 is not set
> # CONFIG_SCHED_RTDS is not set
> # CONFIG_SCHED_ARINC653 is not set
> # CONFIG_SCHED_CREDIT_DEFAULT is not set
> # CONFIG_SCHED_CREDIT2_DEFAULT is not set
> # CONFIG_SCHED_RTDS_DEFAULT is not set
> # CONFIG_SCHED_ARINC653_DEFAULT is not set
> CONFIG_SCHED_DEFAULT="credit"
>
>
> And the hypervisor did build with:
>
> [konrad@char xen]$ nm --defined xen-syms |grep schedulers
> ffff82d080290d58 D __end_schedulers_array
> ffff82d080290d58 D __start_schedulers_array
>
> :-)
>
> Not exactly sure if there is some way to make us _not_ shoot
> ourselves in the foot by mistake.
>
> Perhaps the build should complain if the size of the
> __schedulers_array is zero?

Hmm yes - an ASSERT() at the bottom of the linker file would be a very
good defensive measure.

A hypervisor without any schedulers compiled in will be rather sad.

~Andrew

Re: [PATCH v3 2/5] build: Hook the schedulers into Kconfig

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1311 bytes --]

On Fri, 2016-01-08 at 15:54 +0000, Andrew Cooper wrote:
> On 08/01/16 15:47, Konrad Rzeszutek Wilk wrote:
> > 
> > And the hypervisor did build with:
> > 
> > [konrad@char xen]$ nm --defined xen-syms |grep schedulers
> > ffff82d080290d58 D __end_schedulers_array
> > ffff82d080290d58 D __start_schedulers_array
> > 
> > :-)
> > 
> > Not exactly sure if there is some way to make us _not_ shoot
> > ourselves in the foot by mistake.
> > 
> > Perhaps the build should complain if the size of the
> > __schedulers_array is zero?
> 
> Hmm yes - an ASSERT() at the bottom of the linker file would be a
> very
> good defensive measure.
> 
+1

> A hypervisor without any schedulers compiled in will be rather sad.
> 
Well, it depends on how many pCPUs the host you want to run the
hypervisor on has: if it has 0 of them, no scheduler at all is the
perfect solution... not to mention that the combination of these things
(0 CPUs and no scheduling), would make a lot of issues just instantly
disappear! :-D

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 2/5] build: Hook the schedulers into Kconfig

[Jonathan Creekmore]

Andrew Cooper writes:

> On 08/01/16 15:47, Konrad Rzeszutek Wilk wrote:
>> On Thu, Jan 07, 2016 at 11:29:18AM -0600, Jonathan Creekmore wrote:
>>> Allow the schedulers to be independently enabled or disabled at
>>> compile-time. To match existing behavior, all four schedulers are
>>> compiled in by default, although the Credit2, RTDS, and ARINC653 are
>>> marked EXPERIMENTAL to match their not currently supported status.
>> By unmarking all of them in sequence I was able to get this:
>>
>> #
>> # Schedulers
>> #
>> # CONFIG_SCHED_CREDIT is not set
>> # CONFIG_SCHED_CREDIT2 is not set
>> # CONFIG_SCHED_RTDS is not set
>> # CONFIG_SCHED_ARINC653 is not set
>> # CONFIG_SCHED_CREDIT_DEFAULT is not set
>> # CONFIG_SCHED_CREDIT2_DEFAULT is not set
>> # CONFIG_SCHED_RTDS_DEFAULT is not set
>> # CONFIG_SCHED_ARINC653_DEFAULT is not set
>> CONFIG_SCHED_DEFAULT="credit"
>>
>>
>> And the hypervisor did build with:
>>
>> [konrad@char xen]$ nm --defined xen-syms |grep schedulers
>> ffff82d080290d58 D __end_schedulers_array
>> ffff82d080290d58 D __start_schedulers_array
>>
>> :-)
>>
>> Not exactly sure if there is some way to make us _not_ shoot
>> ourselves in the foot by mistake.
>>
>> Perhaps the build should complain if the size of the
>> __schedulers_array is zero?
>
> Hmm yes - an ASSERT() at the bottom of the linker file would be a very
> good defensive measure.
>
> A hypervisor without any schedulers compiled in will be rather sad.

I can definitely add that in.

Re: [PATCH v3 2/5] build: Hook the schedulers into Kconfig

[Juergen Gross]

On 08/01/16 17:02, Jonathan Creekmore wrote:
> 
> Andrew Cooper writes:
> 
>> On 08/01/16 15:47, Konrad Rzeszutek Wilk wrote:
>>> On Thu, Jan 07, 2016 at 11:29:18AM -0600, Jonathan Creekmore wrote:
>>>> Allow the schedulers to be independently enabled or disabled at
>>>> compile-time. To match existing behavior, all four schedulers are
>>>> compiled in by default, although the Credit2, RTDS, and ARINC653 are
>>>> marked EXPERIMENTAL to match their not currently supported status.
>>> By unmarking all of them in sequence I was able to get this:
>>>
>>> #
>>> # Schedulers
>>> #
>>> # CONFIG_SCHED_CREDIT is not set
>>> # CONFIG_SCHED_CREDIT2 is not set
>>> # CONFIG_SCHED_RTDS is not set
>>> # CONFIG_SCHED_ARINC653 is not set
>>> # CONFIG_SCHED_CREDIT_DEFAULT is not set
>>> # CONFIG_SCHED_CREDIT2_DEFAULT is not set
>>> # CONFIG_SCHED_RTDS_DEFAULT is not set
>>> # CONFIG_SCHED_ARINC653_DEFAULT is not set
>>> CONFIG_SCHED_DEFAULT="credit"
>>>
>>>
>>> And the hypervisor did build with:
>>>
>>> [konrad@char xen]$ nm --defined xen-syms |grep schedulers
>>> ffff82d080290d58 D __end_schedulers_array
>>> ffff82d080290d58 D __start_schedulers_array
>>>
>>> :-)
>>>
>>> Not exactly sure if there is some way to make us _not_ shoot
>>> ourselves in the foot by mistake.
>>>
>>> Perhaps the build should complain if the size of the
>>> __schedulers_array is zero?
>>
>> Hmm yes - an ASSERT() at the bottom of the linker file would be a very
>> good defensive measure.
>>
>> A hypervisor without any schedulers compiled in will be rather sad.
> 
> I can definitely add that in.

I think there should be an ASSERT (or some other measure) to
ensure the default scheduler is available.


Juergen

Re: [PATCH v3 2/5] build: Hook the schedulers into Kconfig

[Jonathan Creekmore]

Juergen Gross writes:

> On 08/01/16 17:02, Jonathan Creekmore wrote:
>>
>> Andrew Cooper writes:
>>
>>> On 08/01/16 15:47, Konrad Rzeszutek Wilk wrote:
>>>> On Thu, Jan 07, 2016 at 11:29:18AM -0600, Jonathan Creekmore wrote:
>>>>> Allow the schedulers to be independently enabled or disabled at
>>>>> compile-time. To match existing behavior, all four schedulers are
>>>>> compiled in by default, although the Credit2, RTDS, and ARINC653 are
>>>>> marked EXPERIMENTAL to match their not currently supported status.
>>>> By unmarking all of them in sequence I was able to get this:
>>>>
>>>> #
>>>> # Schedulers
>>>> #
>>>> # CONFIG_SCHED_CREDIT is not set
>>>> # CONFIG_SCHED_CREDIT2 is not set
>>>> # CONFIG_SCHED_RTDS is not set
>>>> # CONFIG_SCHED_ARINC653 is not set
>>>> # CONFIG_SCHED_CREDIT_DEFAULT is not set
>>>> # CONFIG_SCHED_CREDIT2_DEFAULT is not set
>>>> # CONFIG_SCHED_RTDS_DEFAULT is not set
>>>> # CONFIG_SCHED_ARINC653_DEFAULT is not set
>>>> CONFIG_SCHED_DEFAULT="credit"
>>>>
>>>>
>>>> And the hypervisor did build with:
>>>>
>>>> [konrad@char xen]$ nm --defined xen-syms |grep schedulers
>>>> ffff82d080290d58 D __end_schedulers_array
>>>> ffff82d080290d58 D __start_schedulers_array
>>>>
>>>> :-)
>>>>
>>>> Not exactly sure if there is some way to make us _not_ shoot
>>>> ourselves in the foot by mistake.
>>>>
>>>> Perhaps the build should complain if the size of the
>>>> __schedulers_array is zero?
>>>
>>> Hmm yes - an ASSERT() at the bottom of the linker file would be a very
>>> good defensive measure.
>>>
>>> A hypervisor without any schedulers compiled in will be rather sad.
>>
>> I can definitely add that in.
>
> I think there should be an ASSERT (or some other measure) to
> ensure the default scheduler is available.

So, in a normal configuration case, that should not be able to
happen. The way the Kconfig is set up, the default will only give you an
option to choose schedulers that are enabled in the build.

1) "normal" (non-expert) configuration --- the user never sees that
schedulers are available for configuration, so all of them are compiled
in and the credit scheduler is chosen by the default.

2) "expert" configuration --- the user can select which schedulers are
available in the build and can choose which one is the default. If a
scheduler is not selected to be part of the build, it is not available
in the default selection dialog.

3) "pathological" configuration --- the user disabled all of the
schedulers, but due to a quirk with hiding the scheduler menu for the
non-expert case, it fell back to the "credit" scheduler for the
default.

So, based on the Kconfig setup and the linker ASSERT, there should be no
way to have a default scheduler that is not in the build. I wish Kconfig
allowed you to state that you must have at least one option selected or
up to all of the options selected (so I could require 1-4 schedulers
compiled in, but not 0). Unfortunately, the only way that seems allowed
is if you compile code in as modules, which Xen does not (with a choice
block, you can select multiple items as M, but only 1 as Y, but at least
1 must be present --- since we only support Y, the choice block wouldn't
work for the schedulers).

Re: [PATCH v3 2/5] build: Hook the schedulers into Kconfig

[Jan Beulich]

>>> On 08.01.16 at 17:30, <jonathan.creekmore@gmail.com> wrote:
> So, based on the Kconfig setup and the linker ASSERT, there should be no
> way to have a default scheduler that is not in the build. I wish Kconfig
> allowed you to state that you must have at least one option selected or
> up to all of the options selected (so I could require 1-4 schedulers
> compiled in, but not 0). Unfortunately, the only way that seems allowed
> is if you compile code in as modules, which Xen does not (with a choice
> block, you can select multiple items as M, but only 1 as Y, but at least
> 1 must be present --- since we only support Y, the choice block wouldn't
> work for the schedulers).

Perhaps credit should just not be configurable then?

Jan

Re: [PATCH v3 2/5] build: Hook the schedulers into Kconfig

[Doug Goldstein]

[-- Attachment #1.1: Type: text/plain, Size: 1408 bytes --]

On 1/8/16 10:49 AM, Jan Beulich wrote:
>>>> On 08.01.16 at 17:30, <jonathan.creekmore@gmail.com> wrote:
>> So, based on the Kconfig setup and the linker ASSERT, there should be no
>> way to have a default scheduler that is not in the build. I wish Kconfig
>> allowed you to state that you must have at least one option selected or
>> up to all of the options selected (so I could require 1-4 schedulers
>> compiled in, but not 0). Unfortunately, the only way that seems allowed
>> is if you compile code in as modules, which Xen does not (with a choice
>> block, you can select multiple items as M, but only 1 as Y, but at least
>> 1 must be present --- since we only support Y, the choice block wouldn't
>> work for the schedulers).
> 
> Perhaps credit should just not be configurable then?
> 
> Jan
> 

How much effort are we willing to put in saving someone? We've already
got these options completely hidden away. We've already given the user a
warning that what they're doing isn't supported and they shouldn't be
touching this. Then they have to disable the scheduler that they pick as
the default. How many people are we expecting to really do this? I'm all
for idiot proofing things but at some point we've got to say there's
enough barriers to prevent this from happening. Otherwise we need to put
a Xen developer in every Xen users home or office.

-- 
Doug Goldstein


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 959 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 2/5] build: Hook the schedulers into Kconfig

[Jonathan Creekmore]

Doug Goldstein writes:

> On 1/8/16 10:49 AM, Jan Beulich wrote:
>>>>> On 08.01.16 at 17:30, <jonathan.creekmore@gmail.com> wrote:
>>> So, based on the Kconfig setup and the linker ASSERT, there should be no
>>> way to have a default scheduler that is not in the build. I wish Kconfig
>>> allowed you to state that you must have at least one option selected or
>>> up to all of the options selected (so I could require 1-4 schedulers
>>> compiled in, but not 0). Unfortunately, the only way that seems allowed
>>> is if you compile code in as modules, which Xen does not (with a choice
>>> block, you can select multiple items as M, but only 1 as Y, but at least
>>> 1 must be present --- since we only support Y, the choice block wouldn't
>>> work for the schedulers).
>>
>> Perhaps credit should just not be configurable then?
>>
>> Jan
>>
>
> How much effort are we willing to put in saving someone? We've already
> got these options completely hidden away. We've already given the user a
> warning that what they're doing isn't supported and they shouldn't be
> touching this. Then they have to disable the scheduler that they pick as
> the default. How many people are we expecting to really do this? I'm all
> for idiot proofing things but at some point we've got to say there's
> enough barriers to prevent this from happening. Otherwise we need to put
> a Xen developer in every Xen users home or office.

At this point, with the (not submitted yet) ASSERT in the linker file,
it is a compile error to not have any schedulers linked in and Kconfig
enforces that only schedulers compiled in can be a default.

Re: [PATCH v3 2/5] build: Hook the schedulers into Kconfig

[Andrew Cooper]

On 08/01/16 17:04, Jonathan Creekmore wrote:
> Doug Goldstein writes:
>
>> On 1/8/16 10:49 AM, Jan Beulich wrote:
>>>>>> On 08.01.16 at 17:30, <jonathan.creekmore@gmail.com> wrote:
>>>> So, based on the Kconfig setup and the linker ASSERT, there should be no
>>>> way to have a default scheduler that is not in the build. I wish Kconfig
>>>> allowed you to state that you must have at least one option selected or
>>>> up to all of the options selected (so I could require 1-4 schedulers
>>>> compiled in, but not 0). Unfortunately, the only way that seems allowed
>>>> is if you compile code in as modules, which Xen does not (with a choice
>>>> block, you can select multiple items as M, but only 1 as Y, but at least
>>>> 1 must be present --- since we only support Y, the choice block wouldn't
>>>> work for the schedulers).
>>> Perhaps credit should just not be configurable then?
>>>
>>> Jan
>>>
>> How much effort are we willing to put in saving someone? We've already
>> got these options completely hidden away. We've already given the user a
>> warning that what they're doing isn't supported and they shouldn't be
>> touching this. Then they have to disable the scheduler that they pick as
>> the default. How many people are we expecting to really do this? I'm all
>> for idiot proofing things but at some point we've got to say there's
>> enough barriers to prevent this from happening. Otherwise we need to put
>> a Xen developer in every Xen users home or office.
> At this point, with the (not submitted yet) ASSERT in the linker file,
> it is a compile error to not have any schedulers linked in and Kconfig
> enforces that only schedulers compiled in can be a default.

I think that should be sufficient.  A user has to go a long way out of
their way to get into this situation in the first place.


If there is a nice way for checking for an empty string in the
preprocessor, then something like

BUILD_BUG_ON(ARRAY_SIZE(CONFIG_SCHED_DEFAULT) == 1);

would also be a good sanity check, but I doubt this specific example
compiles.

~Andrew

Re: [PATCH v3 2/5] build: Hook the schedulers into Kconfig

[Konrad Rzeszutek Wilk]

On Fri, Jan 08, 2016 at 11:04:56AM -0600, Jonathan Creekmore wrote:
> 
> Doug Goldstein writes:
> 
> > On 1/8/16 10:49 AM, Jan Beulich wrote:
> >>>>> On 08.01.16 at 17:30, <jonathan.creekmore@gmail.com> wrote:
> >>> So, based on the Kconfig setup and the linker ASSERT, there should be no
> >>> way to have a default scheduler that is not in the build. I wish Kconfig
> >>> allowed you to state that you must have at least one option selected or
> >>> up to all of the options selected (so I could require 1-4 schedulers
> >>> compiled in, but not 0). Unfortunately, the only way that seems allowed
> >>> is if you compile code in as modules, which Xen does not (with a choice
> >>> block, you can select multiple items as M, but only 1 as Y, but at least
> >>> 1 must be present --- since we only support Y, the choice block wouldn't
> >>> work for the schedulers).
> >>
> >> Perhaps credit should just not be configurable then?
> >>
> >> Jan
> >>
> >
> > How much effort are we willing to put in saving someone? We've already
> > got these options completely hidden away. We've already given the user a
> > warning that what they're doing isn't supported and they shouldn't be
> > touching this. Then they have to disable the scheduler that they pick as
> > the default. How many people are we expecting to really do this? I'm all
> > for idiot proofing things but at some point we've got to say there's
> > enough barriers to prevent this from happening. Otherwise we need to put
> > a Xen developer in every Xen users home or office.
> 
> At this point, with the (not submitted yet) ASSERT in the linker file,
> it is a compile error to not have any schedulers linked in and Kconfig
> enforces that only schedulers compiled in can be a default.

That would be good!

Re: [PATCH v3 2/5] build: Hook the schedulers into Kconfig

[Jan Beulich]

>>> On 08.01.16 at 17:58, <cardoe@cardoe.com> wrote:
> On 1/8/16 10:49 AM, Jan Beulich wrote:
>>>>> On 08.01.16 at 17:30, <jonathan.creekmore@gmail.com> wrote:
>>> So, based on the Kconfig setup and the linker ASSERT, there should be no
>>> way to have a default scheduler that is not in the build. I wish Kconfig
>>> allowed you to state that you must have at least one option selected or
>>> up to all of the options selected (so I could require 1-4 schedulers
>>> compiled in, but not 0). Unfortunately, the only way that seems allowed
>>> is if you compile code in as modules, which Xen does not (with a choice
>>> block, you can select multiple items as M, but only 1 as Y, but at least
>>> 1 must be present --- since we only support Y, the choice block wouldn't
>>> work for the schedulers).
>> 
>> Perhaps credit should just not be configurable then?
> 
> How much effort are we willing to put in saving someone? We've already
> got these options completely hidden away. We've already given the user a
> warning that what they're doing isn't supported and they shouldn't be
> touching this. Then they have to disable the scheduler that they pick as
> the default. How many people are we expecting to really do this? I'm all
> for idiot proofing things but at some point we've got to say there's
> enough barriers to prevent this from happening. Otherwise we need to put
> a Xen developer in every Xen users home or office.

I personally think that we shouldn't allow .config-s to be produced
that will yield a build failure. No matter whether supported, we'd
expect (as discussed) people to take care of problems themselves,
but we'd also hope for them to contribute back the adjustments
they had to make. I.e. there's need to be a way for someone to
fix the build issue here, and since we're already aware of the issue
I can't see why we should allow in the brokenness. The precautions
are only against issues we didn't happen to notice.

Jan

Re: [PATCH v3 2/5] build: Hook the schedulers into Kconfig

[Jan Beulich]

>>> On 08.01.16 at 17:13, <JGross@suse.com> wrote:
> I think there should be an ASSERT (or some other measure) to
> ensure the default scheduler is available.

Indeed I was about to say exactly this.

Jan

[PATCH v3 3/5] build: Alloc space for sched list in the link file

[Jonathan Creekmore]

Creates a section to contain scheduler entry pointers that are gathered
together into an array. This will allow, in a follow-on patch, scheduler
entries to be automatically gathered together into the array for
automatic parsing.

CC: Ian Campbell <ian.campbell@citrix.com>
CC: Stefano Stabellini <stefano.stabellini@citrix.com>
CC: Keir Fraser <keir@xen.org>
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jonathan Creekmore <jonathan.creekmore@gmail.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Doug Goldstein <cardoe@cardoe.com>

---
Changed since v1:
  * rename the __start and __end symbols to better match
    the rest of the file
---
 xen/arch/arm/xen.lds.S | 4 ++++
 xen/arch/x86/xen.lds.S | 4 ++++
 2 files changed, 8 insertions(+)

diff --git a/xen/arch/arm/xen.lds.S b/xen/arch/arm/xen.lds.S
index 0488f37..f501a2f 100644
--- a/xen/arch/arm/xen.lds.S
+++ b/xen/arch/arm/xen.lds.S
@@ -57,6 +57,10 @@ SECTIONS
        . = ALIGN(PAGE_SIZE);
        *(.data.page_aligned)
        *(.data)
+       . = ALIGN(8);
+       __start_schedulers_array = .;
+       *(.data.schedulers)
+       __end_schedulers_array = .;
        *(.data.rel)
        *(.data.rel.*)
        CONSTRUCTORS
diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
index e18e08f..c1ce027 100644
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -80,6 +80,10 @@ SECTIONS
        __stop___pre_ex_table = .;
 
        *(.data.read_mostly)
+       . = ALIGN(8);
+       __start_schedulers_array = .;
+       *(.data.schedulers)
+       __end_schedulers_array = .;
        *(.data.rel.ro)
        *(.data.rel.ro.*)
   } :text
-- 
2.6.4

Re: [PATCH v3 3/5] build: Alloc space for sched list in the link file

[Konrad Rzeszutek Wilk]

On Thu, Jan 07, 2016 at 11:29:19AM -0600, Jonathan Creekmore wrote:
> Creates a section to contain scheduler entry pointers that are gathered
> together into an array. This will allow, in a follow-on patch, scheduler
> entries to be automatically gathered together into the array for
> automatic parsing.
> 
> CC: Ian Campbell <ian.campbell@citrix.com>
> CC: Stefano Stabellini <stefano.stabellini@citrix.com>
> CC: Keir Fraser <keir@xen.org>
> CC: Jan Beulich <jbeulich@suse.com>
> CC: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Jonathan Creekmore <jonathan.creekmore@gmail.com>
> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Doug Goldstein <cardoe@cardoe.com>

Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> 
> ---
> Changed since v1:
>   * rename the __start and __end symbols to better match
>     the rest of the file
> ---
>  xen/arch/arm/xen.lds.S | 4 ++++
>  xen/arch/x86/xen.lds.S | 4 ++++
>  2 files changed, 8 insertions(+)
> 
> diff --git a/xen/arch/arm/xen.lds.S b/xen/arch/arm/xen.lds.S
> index 0488f37..f501a2f 100644
> --- a/xen/arch/arm/xen.lds.S
> +++ b/xen/arch/arm/xen.lds.S
> @@ -57,6 +57,10 @@ SECTIONS
>         . = ALIGN(PAGE_SIZE);
>         *(.data.page_aligned)
>         *(.data)
> +       . = ALIGN(8);
> +       __start_schedulers_array = .;
> +       *(.data.schedulers)
> +       __end_schedulers_array = .;
>         *(.data.rel)
>         *(.data.rel.*)
>         CONSTRUCTORS
> diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
> index e18e08f..c1ce027 100644
> --- a/xen/arch/x86/xen.lds.S
> +++ b/xen/arch/x86/xen.lds.S
> @@ -80,6 +80,10 @@ SECTIONS
>         __stop___pre_ex_table = .;
>  
>         *(.data.read_mostly)
> +       . = ALIGN(8);
> +       __start_schedulers_array = .;
> +       *(.data.schedulers)
> +       __end_schedulers_array = .;
>         *(.data.rel.ro)
>         *(.data.rel.ro.*)
>    } :text
> -- 
> 2.6.4
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

[PATCH v3 4/5] sched: Register the schedulers into the list

[Jonathan Creekmore]

Adds a simple macro to place a pointer to a scheduler into an array
section at compile time. Also, goes ahead and generates the array
entries with each of the schedulers.

CC: George Dunlap <george.dunlap@eu.citrix.com>
CC: Dario Faggioli <dario.faggioli@citrix.com>
CC: Josh Whitehead <josh.whitehead@dornerworks.com>
CC: Robert VanVossen <robert.vanvossen@dornerworks.com>
Signed-off-by: Jonathan Creekmore <jonathan.creekmore@gmail.com>
Acked-by: Dario Faggioli <dario.faggioli@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Doug Goldstein <cardoe@cardoe.com>
---
 xen/common/sched_arinc653.c | 2 ++
 xen/common/sched_credit.c   | 2 ++
 xen/common/sched_credit2.c  | 2 ++
 xen/common/sched_rt.c       | 2 ++
 xen/include/xen/sched-if.h  | 2 ++
 5 files changed, 10 insertions(+)

diff --git a/xen/common/sched_arinc653.c b/xen/common/sched_arinc653.c
index dbe02ed..3b59514 100644
--- a/xen/common/sched_arinc653.c
+++ b/xen/common/sched_arinc653.c
@@ -767,6 +767,8 @@ const struct scheduler sched_arinc653_def = {
     .tick_resume    = NULL,
 };
 
+REGISTER_SCHEDULER(sched_arinc653_def);
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c
index 0dce790..e586248 100644
--- a/xen/common/sched_credit.c
+++ b/xen/common/sched_credit.c
@@ -2027,3 +2027,5 @@ const struct scheduler sched_credit_def = {
     .tick_suspend   = csched_tick_suspend,
     .tick_resume    = csched_tick_resume,
 };
+
+REGISTER_SCHEDULER(sched_credit_def);
diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 3c49ffa..38b02d0 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -2228,3 +2228,5 @@ const struct scheduler sched_credit2_def = {
     .alloc_domdata  = csched2_alloc_domdata,
     .free_domdata   = csched2_free_domdata,
 };
+
+REGISTER_SCHEDULER(sched_credit2_def);
diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
index 3f1d047..7640cd0 100644
--- a/xen/common/sched_rt.c
+++ b/xen/common/sched_rt.c
@@ -1199,3 +1199,5 @@ const struct scheduler sched_rtds_def = {
     .wake           = rt_vcpu_wake,
     .context_saved  = rt_context_saved,
 };
+
+REGISTER_SCHEDULER(sched_rtds_def);
diff --git a/xen/include/xen/sched-if.h b/xen/include/xen/sched-if.h
index 493d43f..9c6e0f5 100644
--- a/xen/include/xen/sched-if.h
+++ b/xen/include/xen/sched-if.h
@@ -170,6 +170,8 @@ extern const struct scheduler sched_credit2_def;
 extern const struct scheduler sched_arinc653_def;
 extern const struct scheduler sched_rtds_def;
 
+#define REGISTER_SCHEDULER(x) static const struct scheduler *x##_entry \
+  __used_section(".data.schedulers") = &x;
 
 struct cpupool
 {
-- 
2.6.4

Re: [PATCH v3 4/5] sched: Register the schedulers into the list

[Konrad Rzeszutek Wilk]

On Thu, Jan 07, 2016 at 11:29:20AM -0600, Jonathan Creekmore wrote:
> Adds a simple macro to place a pointer to a scheduler into an array
> section at compile time. Also, goes ahead and generates the array
> entries with each of the schedulers.
> 
> CC: George Dunlap <george.dunlap@eu.citrix.com>
> CC: Dario Faggioli <dario.faggioli@citrix.com>
> CC: Josh Whitehead <josh.whitehead@dornerworks.com>
> CC: Robert VanVossen <robert.vanvossen@dornerworks.com>
> Signed-off-by: Jonathan Creekmore <jonathan.creekmore@gmail.com>
> Acked-by: Dario Faggioli <dario.faggioli@citrix.com>
> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Doug Goldstein <cardoe@cardoe.com>

Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  xen/common/sched_arinc653.c | 2 ++
>  xen/common/sched_credit.c   | 2 ++
>  xen/common/sched_credit2.c  | 2 ++
>  xen/common/sched_rt.c       | 2 ++
>  xen/include/xen/sched-if.h  | 2 ++
>  5 files changed, 10 insertions(+)
> 
> diff --git a/xen/common/sched_arinc653.c b/xen/common/sched_arinc653.c
> index dbe02ed..3b59514 100644
> --- a/xen/common/sched_arinc653.c
> +++ b/xen/common/sched_arinc653.c
> @@ -767,6 +767,8 @@ const struct scheduler sched_arinc653_def = {
>      .tick_resume    = NULL,
>  };
>  
> +REGISTER_SCHEDULER(sched_arinc653_def);
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c
> index 0dce790..e586248 100644
> --- a/xen/common/sched_credit.c
> +++ b/xen/common/sched_credit.c
> @@ -2027,3 +2027,5 @@ const struct scheduler sched_credit_def = {
>      .tick_suspend   = csched_tick_suspend,
>      .tick_resume    = csched_tick_resume,
>  };
> +
> +REGISTER_SCHEDULER(sched_credit_def);
> diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
> index 3c49ffa..38b02d0 100644
> --- a/xen/common/sched_credit2.c
> +++ b/xen/common/sched_credit2.c
> @@ -2228,3 +2228,5 @@ const struct scheduler sched_credit2_def = {
>      .alloc_domdata  = csched2_alloc_domdata,
>      .free_domdata   = csched2_free_domdata,
>  };
> +
> +REGISTER_SCHEDULER(sched_credit2_def);
> diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
> index 3f1d047..7640cd0 100644
> --- a/xen/common/sched_rt.c
> +++ b/xen/common/sched_rt.c
> @@ -1199,3 +1199,5 @@ const struct scheduler sched_rtds_def = {
>      .wake           = rt_vcpu_wake,
>      .context_saved  = rt_context_saved,
>  };
> +
> +REGISTER_SCHEDULER(sched_rtds_def);
> diff --git a/xen/include/xen/sched-if.h b/xen/include/xen/sched-if.h
> index 493d43f..9c6e0f5 100644
> --- a/xen/include/xen/sched-if.h
> +++ b/xen/include/xen/sched-if.h
> @@ -170,6 +170,8 @@ extern const struct scheduler sched_credit2_def;
>  extern const struct scheduler sched_arinc653_def;
>  extern const struct scheduler sched_rtds_def;
>  
> +#define REGISTER_SCHEDULER(x) static const struct scheduler *x##_entry \
> +  __used_section(".data.schedulers") = &x;
>  
>  struct cpupool
>  {
> -- 
> 2.6.4
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

[PATCH v3 5/5] sched: Use the auto-generated list of schedulers

[Jonathan Creekmore]

Instead of having a manually-curated list of schedulers, use the array
that was auto-generated simply by compiling in the scheduler files as
the sole source of truth of the available schedulers.

CC: George Dunlap <george.dunlap@eu.citrix.com>
CC: Dario Faggioli <dario.faggioli@citrix.com>
Signed-off-by: Jonathan Creekmore <jonathan.creekmore@gmail.com>
Acked-by: Dario Faggioli <dario.faggioli@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Doug Goldstein <cardoe@cardoe.com>

---
Changed since v1:
  * Simplify the calculation of the number of schedulers
  * Make the scheduler ops structures static to their files
---
 xen/common/sched_arinc653.c |  2 +-
 xen/common/sched_credit.c   |  2 +-
 xen/common/sched_credit2.c  |  2 +-
 xen/common/sched_rt.c       |  2 +-
 xen/common/schedule.c       | 24 +++++++-----------------
 xen/include/xen/sched-if.h  |  5 -----
 6 files changed, 11 insertions(+), 26 deletions(-)

diff --git a/xen/common/sched_arinc653.c b/xen/common/sched_arinc653.c
index 3b59514..0606988 100644
--- a/xen/common/sched_arinc653.c
+++ b/xen/common/sched_arinc653.c
@@ -724,7 +724,7 @@ a653sched_adjust_global(const struct scheduler *ops,
  * callback functions.
  * The symbol must be visible to the rest of Xen at link time.
  */
-const struct scheduler sched_arinc653_def = {
+static const struct scheduler sched_arinc653_def = {
     .name           = "ARINC 653 Scheduler",
     .opt_name       = "arinc653",
     .sched_id       = XEN_SCHEDULER_ARINC653,
diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c
index e586248..028e41b 100644
--- a/xen/common/sched_credit.c
+++ b/xen/common/sched_credit.c
@@ -1991,7 +1991,7 @@ static void csched_tick_resume(const struct scheduler *ops, unsigned int cpu)
 
 static struct csched_private _csched_priv;
 
-const struct scheduler sched_credit_def = {
+static const struct scheduler sched_credit_def = {
     .name           = "SMP Credit Scheduler",
     .opt_name       = "credit",
     .sched_id       = XEN_SCHEDULER_CREDIT,
diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 38b02d0..78220a7 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -2194,7 +2194,7 @@ csched2_deinit(const struct scheduler *ops)
 
 static struct csched2_private _csched2_priv;
 
-const struct scheduler sched_credit2_def = {
+static const struct scheduler sched_credit2_def = {
     .name           = "SMP Credit Scheduler rev2",
     .opt_name       = "credit2",
     .sched_id       = XEN_SCHEDULER_CREDIT2,
diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
index 7640cd0..2e5430f 100644
--- a/xen/common/sched_rt.c
+++ b/xen/common/sched_rt.c
@@ -1170,7 +1170,7 @@ rt_dom_cntl(
 
 static struct rt_private _rt_priv;
 
-const struct scheduler sched_rtds_def = {
+static const struct scheduler sched_rtds_def = {
     .name           = "SMP RTDS Scheduler",
     .opt_name       = "rtds",
     .sched_id       = XEN_SCHEDULER_RTDS,
diff --git a/xen/common/schedule.c b/xen/common/schedule.c
index 2f98a48..91e53c1 100644
--- a/xen/common/schedule.c
+++ b/xen/common/schedule.c
@@ -64,20 +64,10 @@ static void poll_timer_fn(void *data);
 DEFINE_PER_CPU(struct schedule_data, schedule_data);
 DEFINE_PER_CPU(struct scheduler *, scheduler);
 
-static const struct scheduler *schedulers[] = {
-#ifdef CONFIG_SCHED_CREDIT
-    &sched_credit_def,
-#endif
-#ifdef CONFIG_SCHED_CREDIT2
-    &sched_credit2_def,
-#endif
-#ifdef CONFIG_SCHED_ARINC653
-    &sched_arinc653_def,
-#endif
-#ifdef CONFIG_SCHED_RTDS
-    &sched_rtds_def,
-#endif
-};
+extern const struct scheduler *__start_schedulers_array[], *__end_schedulers_array[];
+extern const size_t schedulers_array_size;
+#define NUM_SCHEDULERS (__end_schedulers_array - __start_schedulers_array)
+static const struct scheduler **schedulers = __start_schedulers_array;
 
 static struct scheduler __read_mostly ops;
 
@@ -1468,7 +1458,7 @@ void __init scheduler_init(void)
 
     open_softirq(SCHEDULE_SOFTIRQ, schedule);
 
-    for ( i = 0; i < ARRAY_SIZE(schedulers); i++ )
+    for ( i = 0; i < NUM_SCHEDULERS; i++)
     {
         if ( schedulers[i]->global_init && schedulers[i]->global_init() < 0 )
             schedulers[i] = NULL;
@@ -1479,7 +1469,7 @@ void __init scheduler_init(void)
     if ( !ops.name )
     {
         printk("Could not find scheduler: %s\n", opt_sched);
-        for ( i = 0; i < ARRAY_SIZE(schedulers); i++ )
+        for ( i = 0; i < NUM_SCHEDULERS; i++ )
             if ( schedulers[i] )
             {
                 ops = *schedulers[i];
@@ -1599,7 +1589,7 @@ struct scheduler *scheduler_alloc(unsigned int sched_id, int *perr)
     int i;
     struct scheduler *sched;
 
-    for ( i = 0; i < ARRAY_SIZE(schedulers); i++ )
+    for ( i = 0; i < NUM_SCHEDULERS; i++ )
         if ( schedulers[i] && schedulers[i]->sched_id == sched_id )
             goto found;
     *perr = -ENOENT;
diff --git a/xen/include/xen/sched-if.h b/xen/include/xen/sched-if.h
index 9c6e0f5..66dc9c8 100644
--- a/xen/include/xen/sched-if.h
+++ b/xen/include/xen/sched-if.h
@@ -165,11 +165,6 @@ struct scheduler {
     void         (*tick_resume)     (const struct scheduler *, unsigned int);
 };
 
-extern const struct scheduler sched_credit_def;
-extern const struct scheduler sched_credit2_def;
-extern const struct scheduler sched_arinc653_def;
-extern const struct scheduler sched_rtds_def;
-
 #define REGISTER_SCHEDULER(x) static const struct scheduler *x##_entry \
   __used_section(".data.schedulers") = &x;
 
-- 
2.6.4

[PATCH v3 0/5] Allow schedulers to be selectable through Kconfig

[Jonathan Creekmore]

Add machinery to allow the schedulers to be individually selectable
through the Kconfig interface. The first patch in the series sets up
the CONFIG_EXPERT Kconfig variable that is only enabled by passing an
environment variable to the build. The second patch in the series sets up
the Kconfig options for the schedulers and places the appropriate CONFIG_
options around the scheduler list. The third, fourth, and fifth patches
rework the scheduler list from being manually curated into a model
where just compiling any schedulers into the hypervisor causes the
scheduler list to be built up.

---
Changed since v2:
  * Added a predecessor patch that introduces a environment
    variable for the build to enable expert configuration options
    (1/5)
  * Hide the scheduler menu behind the expert option (2/5)
  * Provide static inlines for credit functions that are assumed to be
    present if it is compiled out (2/5)
  * Provide an absolute default of the credit scheduler if the 
    scheduler menu is not visible (2/5)

Changed since v1:
  * Marked credit2 as EXPERIMENTAL
  * Removed confusing language from the commit message
  * Alphabetize the schedulers
  * rename the __start and __end symbols to better match
    the rest of the file
  * Simplify the calculation of the number of schedulers
  * Make the scheduler ops structures static to their files

Jonathan Creekmore (5):
  build: Env var to enable expert config options
  build: Hook the schedulers into Kconfig
  build: Alloc space for sched list in the link file
  sched: Register the schedulers into the list
  sched: Use the auto-generated list of schedulers

 xen/Kconfig                 |  4 +++
 xen/Makefile                |  1 +
 xen/arch/arm/xen.lds.S      |  4 +++
 xen/arch/x86/xen.lds.S      |  4 +++
 xen/common/Kconfig          | 67 +++++++++++++++++++++++++++++++++++++++++++++
 xen/common/Makefile         |  8 +++---
 xen/common/sched_arinc653.c |  4 ++-
 xen/common/sched_credit.c   |  4 ++-
 xen/common/sched_credit2.c  |  4 ++-
 xen/common/sched_rt.c       |  4 ++-
 xen/common/schedule.c       | 20 ++++++--------
 xen/include/xen/sched-if.h  |  7 ++---
 xen/include/xen/sched.h     |  5 ++++
 13 files changed, 112 insertions(+), 24 deletions(-)

-- 
2.6.4