From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S964988AbcJQXhk (ORCPT ); Mon, 17 Oct 2016 19:37:40 -0400 Received: from mga07.intel.com ([134.134.136.100]:6954 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934184AbcJQXhb (ORCPT ); Mon, 17 Oct 2016 19:37:31 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.31,359,1473145200"; d="scan'208";a="20560451" Date: Mon, 17 Oct 2016 16:37:30 -0700 From: "Luck, Tony" To: Thomas Gleixner Cc: Fenghua Yu , "H. Peter Anvin" , Ingo Molnar , Peter Zijlstra , Stephane Eranian , Borislav Petkov , Dave Hansen , Nilay Vaish , Shaohua Li , David Carrillo-Cisneros , Ravi V Shankar , Sai Prakhya , Vikas Shivappa , linux-kernel , x86 Subject: Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system Message-ID: <20161017233729.GA6386@intel.com> References: <1476497548-11169-1-git-send-email-fenghua.yu@intel.com> <1476497548-11169-14-git-send-email-fenghua.yu@intel.com> <20161018011845.GD8999@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Oct 18, 2016 at 01:20:36AM +0200, Thomas Gleixner wrote: > On Mon, 17 Oct 2016, Fenghua Yu wrote: > > part0: L3:0=1;1=1 closid0/cbm=1 on cache0 and closid0/cbm=1 on cache1 > > (closid 15 on cache0 combined with 16 different closids on cache1) > > ... > > part254: L3:0=ffff;1=7fff closid15/cbm=ffff on cache0 and closid14/cbm=7fff on cache1 > > part255: L3:0=ffff;1=ffff closid15/cbm=ffff on cache0 and closid15/cbm=ffff on cache1 > > > > To utilize as much combinations as possbile, we may implement a > > more complex allocation than current one. > > > > Does this make sense? > > Thanks for the explanation. I knew that I'm missing something. > > But how is that supposed to work? The schemata files have no idea of > closids simply because the closids are assigned automatically. And that > makes the whole thing exponentially complex. You must allow to create ALL > rdt groups (initialy as a copy of the root group) and then when the > schemata file is written you have to look whether the particular CBM value > for a particular domain is already used and assign the same cosid for this > domain. That of course makes the whole L2 business completely diffuse > because you might end up with: > > Dom0 = COSID1 and DOM1 = COSID9 > > So you can set the L2 for Dom0, but not for DOM1 and then if you set L2 for > Dom0 you must find a new COSID for Dom0. If there is none, then you must > reject the write and leave the admin puzzled. > > There is a reason why I suggested: > > https://lkml.kernel.org/r/alpine.DEB.2.11.1511181534450.3761@nanos > > It's certainly not perfect (missing L2 etc.), but clearly avoids exactly > the above issues. And it would allow you to utilize the 256 groups in an > understandable way. If you head down that path someone with a 4-socket system will try to make 16x16x16x16 = 65536 groups and "understandable" takes a bit of a beating. The eight socket system with 16^8 = 4G groups defies any rationale hope. Best not to think about 16 sockets. The L2 + L3 configuration space gets unbelievably messy too. There's a reason why I ripped out the allocation code and went with a simple global allocator in this version. If we decide we need something fancier we can adapt later. Some solutions might be transparent to applications, others might add a "closid" file into each directory to give 2nd generation applications hooks to view (and maybe control) which closid is used by each group. -Tony