From mboxrd@z Thu Jan 1 00:00:00 1970 From: Juergen Gross Subject: Re: credit2 data structures Date: Fri, 14 Oct 2011 06:35:46 +0200 Message-ID: <4E97BC22.5060003@ts.fujitsu.com> References: <4E96CEBD020000780005B151@nat28.tlf.novell.com> <4E96F48A020000780005B2A3@nat28.tlf.novell.com> <4E96DF7B.5060502@ts.fujitsu.com> <4E970F2E020000780005B2F8@nat28.tlf.novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <4E970F2E020000780005B2F8@nat28.tlf.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Jan Beulich Cc: George Dunlap , "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org On 10/13/2011 04:17 PM, Jan Beulich wrote: >>>> On 13.10.11 at 14:54, Juergen Gross wrote: >> On 10/13/2011 02:24 PM, Jan Beulich wrote: >>>>>> On 13.10.11 at 12:11, George Dunlap wrote: >>>> On Thu, Oct 13, 2011 at 10:42 AM, Jan Beulich wrote: >>>>> Apart from the possibility of allocating the arrays (and maybe also the >>>>> cpumask_t-s) separately (for which I can come up with a patch on top >>>>> of what I' currently putting together) - is it really necessary to have >>>>> all these, the more that there can be multiple instances of the structure >>>>> with CPU pools? >>>> I'm not quite sure what it is that you're asking. Do you mean, are >>>> all of the things in each runqueue structure necessary? Specifically, >>>> I guess, the cpumask_t structures (because the rest of the structure >>>> isn't significantly larger than the per-cpu structure for credit1)? >>> No, it's really the NR_CPUS-sized array of struct csched_runqueue_data. >>> Credit1 otoh has *no* NR_CPUS sized arrays at all. >>> >>>> At first blush, all of those cpu masks are necessary. The assignment >>>> of cpus to runqueues may be arbitrary, so we need a cpu mask for that. >>>> In theory, "idle" and "tickled" only need bits for the cpus actually >>>> assigned to this runqueue (which should be 2-8 under normal >>>> circumstances). But then we would need some kind of mechanism to >>>> translate "mask just for these cpus" to "mask of all cpus" in order to >>>> use the normal cpumask mechanisms, which seems like a lot of extra >>>> complexity just to save a few bytes. Surely a system with 4096 >>>> logical cpus can afford 6 megabytes of memory for scheduling? >>> I'm not concerned about the total amount if run on a system that >>> large. I'm more concerned about this being a single chunk (possibly >>> allocated post-boot, where we're really aiming at having no >>> allocations larger than a page at all) and this size being allocated >>> even when running on a much smaller system (i.e. depending only >>> on compile time parameters). >>> >>>> For one thing, the number of runqueues in credit2 is actually meant to >>>> be smaller than the number of logical cpus -- it's meant to be one per >>>> L2 cache, which should have between 2 and 8 logical cpus, depending on >>>> the architecture. I just put NR_CPUS because it was easier to get >>>> working. Making that an array of pointers, which is allocated on an >>>> as-needed basis, should reduce that requirement a great deal. >>> That would help, but would probably not suffice (since a NR_CPUS >>> sized array of pointers is still going to be larger than a page). We >>> may need to introduce dynamic per-CPU data allocation for this... >> Couldn't the run-queue data be dynamically allocated and the pcpu-data of >> credit2 contain a pointer to it? > Not if the per-CPU data is also per scheduler instance (which I can't > easily tell whether it is). Each cpu has only one dynamically allocated scheduler pcpu-data area which is anchored in the per_cpu area of that cpu. Juergen -- Juergen Gross Principal Developer Operating Systems PDG ES&S SWE OS6 Telephone: +49 (0) 89 3222 2967 Fujitsu Technology Solutions e-mail: juergen.gross@ts.fujitsu.com Domagkstr. 28 Internet: ts.fujitsu.com D-80807 Muenchen Company details: ts.fujitsu.com/imprint.html