From mboxrd@z Thu Jan 1 00:00:00 1970 From: Juergen Gross Subject: Re: [PATCH] Xen sched: Fix multiple runqueues in credit2 Date: Thu, 06 Feb 2014 10:13:28 +0100 Message-ID: <52F35238.90806@ts.fujitsu.com> References: <1391677118-3071-1-git-send-email-jtweaver@hawaii.edu> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1391677118-3071-1-git-send-email-jtweaver@hawaii.edu> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Justin Weaver Cc: Marcus.Granado@eu.citrix.com, george.dunlap@eu.citrix.com, dario.faggioli@citrix.com, esb@ics.hawaii.edu, xen-devel@lists.xen.org, henric@hawaii.edu List-Id: xen-devel@lists.xenproject.org On 06.02.2014 09:58, Justin Weaver wrote: > This patch attempts to address the issue of the Xen Credit 2 > Scheduler only creating one vCPU run queue on multiple physical > processor systems. It should be creating one run queue per > physical processor. > > CPU 0 does not get a starting callback, so it is hard coded to run > queue 0. At the time this happens, socket information is not > available for CPU 0. > > Socket information is available for each individual CPU when each > gets the STARTING callback (I believe socket information is also > available for CPU 0 by that time). This patch adds the following > algorithm... > > IF cpu is on the same socket as CPU 0, add it to run queue 0 You should check whether cpu and CPU0 are in the same cpupool. BTW: CPU0 is allowed to be moved to another cpupool, too. Juergen > ELSE, IF cpu is on socket 0, add it to a run queue based on the > socket CPU 0 is actually on > ELSE add it to a run queue based on the socket it is on > --- > xen/common/sched_credit2.c | 37 +++++++++++++++++++++++++++---------- > 1 file changed, 27 insertions(+), 10 deletions(-) > > diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c > index 4e68375..c0ecb50 100644 > --- a/xen/common/sched_credit2.c > +++ b/xen/common/sched_credit2.c > @@ -85,8 +85,7 @@ > * to a small value, and a fixed credit is added to everyone. > * > * The plan is for all cores that share an L2 will share the same > - * runqueue. At the moment, there is one global runqueue for all > - * cores. > + * runqueue. > */ > > /* > @@ -1945,6 +1944,8 @@ static void deactivate_runqueue(struct csched_private *prv, int rqi) > static void init_pcpu(const struct scheduler *ops, int cpu) > { > int rqi; > + int cpu0_socket; > + int cpu_socket; > unsigned long flags; > struct csched_private *prv = CSCHED_PRIV(ops); > struct csched_runqueue_data *rqd; > @@ -1962,12 +1963,28 @@ static void init_pcpu(const struct scheduler *ops, int cpu) > /* Figure out which runqueue to put it in */ > rqi = 0; > > - /* Figure out which runqueue to put it in */ > /* NB: cpu 0 doesn't get a STARTING callback, so we hard-code it to runqueue 0. */ > if ( cpu == 0 ) > rqi = 0; > else > - rqi = cpu_to_socket(cpu); > + { > + cpu_socket = cpu_to_socket(cpu); > + cpu0_socket = cpu_to_socket(0); > + > + /* If cpu is on the same socket as CPU 0, put it with CPU 0 on run queue 0 */ > + if ( cpu_socket == cpu0_socket ) > + rqi = 0; > + else > + /* If cpu is on socket 0, assign it to a run queue based on the > + * socket CPU 0 is actually on */ > + if ( cpu_socket == 0 ) > + rqi = cpu0_socket; > + > + /* If cpu is NOT on socket 0, just assign it to a run queue based on > + * its own socket */ > + else > + rqi = cpu_socket; > + } > > if ( rqi < 0 ) > { > @@ -2010,13 +2027,11 @@ static void init_pcpu(const struct scheduler *ops, int cpu) > static void * > csched_alloc_pdata(const struct scheduler *ops, int cpu) > { > - /* Check to see if the cpu is online yet */ > - /* Note: cpu 0 doesn't get a STARTING callback */ > - if ( cpu == 0 || cpu_to_socket(cpu) >= 0 ) > + /* This function is only for calling init_pcpu on CPU 0 > + * because it does not get a STARTING callback */ > + > + if ( cpu == 0 ) > init_pcpu(ops, cpu); > - else > - printk("%s: cpu %d not online yet, deferring initializatgion\n", > - __func__, cpu); > > return (void *)1; > } > @@ -2072,6 +2087,8 @@ csched_free_pdata(const struct scheduler *ops, void *pcpu, int cpu) > static int > csched_cpu_starting(int cpu) > { > + // This function is for calling init_pcpu on every CPU, except for CPU 0 */ > + > struct scheduler *ops; > > /* Hope this is safe from cpupools switching things around. :-) */ > -- Juergen Gross Principal Developer Operating Systems PBG PDG ES&S SWE OS6 Telephone: +49 (0) 89 62060 2932 Fujitsu e-mail: juergen.gross@ts.fujitsu.com Mies-van-der-Rohe-Str. 8 Internet: ts.fujitsu.com D-80807 Muenchen Company details: ts.fujitsu.com/imprint.html