From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756717AbbKRTi0 (ORCPT ); Wed, 18 Nov 2015 14:38:26 -0500 Received: from mx1.redhat.com ([209.132.183.28]:46550 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754512AbbKRTiZ (ORCPT ); Wed, 18 Nov 2015 14:38:25 -0500 Date: Wed, 18 Nov 2015 14:38:22 -0500 From: Luiz Capitulino To: Thomas Gleixner Cc: LKML , Peter Zijlstra , x86@kernel.org, Marcelo Tosatti , Vikas Shivappa , Tejun Heo , Yu Fenghua , will.auld@intel.com, donald.d.dugger@intel.com, riel@redhat.com Subject: Re: [RFD] CAT user space interface revisited Message-ID: <20151118143822.3900dfff@redhat.com> In-Reply-To: References: Organization: Red Hat MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 18 Nov 2015 19:25:03 +0100 (CET) Thomas Gleixner wrote: > We really need to make this as configurable as possible from userspace > without imposing random restrictions to it. I played around with it on > my new intel toy and the restriction to 16 COS ids (that's 8 with CDP > enabled) makes it really useless if we force the ids to have the same > meaning on all sockets and restrict it to per task partitioning. > > Even if next generation systems will have more COS ids available, > there are not going to be enough to have a system wide consistent > view unless we have COS ids > nr_cpus. > > Aside of that I don't think that a system wide consistent view is > useful at all. This is a great writeup! I agree with everything you said. > So now to the interface part. Unfortunately we need to expose this > very close to the hardware implementation as there are really no > abstractions which allow us to express the various bitmap > combinations. Any abstraction I tried to come up with renders that > thing completely useless. > > I was not able to identify any existing infrastructure where this > really fits in. I chose a directory/file based representation. We > certainly could do the same with a syscall, but that's just an > implementation detail. > > At top level: > > xxxxxxx/cat/max_cosids <- Assume that all CPUs are the same > xxxxxxx/cat/max_maskbits <- Assume that all CPUs are the same > xxxxxxx/cat/cdp_enable <- Depends on CDP availability > > Per socket data: > > xxxxxxx/cat/socket-0/ > ... > xxxxxxx/cat/socket-N/l3_size > xxxxxxx/cat/socket-N/hwsharedbits > > Per socket mask data: > > xxxxxxx/cat/socket-N/cos-id-0/ > ... > xxxxxxx/cat/socket-N/cos-id-N/inuse > /cat_mask > /cdp_mask <- Data mask if CDP enabled > > Per cpu default cos id for the cpus on that socket: > > xxxxxxx/cat/socket-N/cpu-x/default_cosid > ... > xxxxxxx/cat/socket-N/cpu-N/default_cosid > > The above allows a simple cpu based partitioning. All tasks which do > not have a cache partition assigned on a particular socket use the > default one of the cpu they are running on. > > Now for the task(s) partitioning: > > xxxxxxx/cat/partitions/ > > Under that directory one can create partitions > > xxxxxxx/cat/partitions/p1/tasks > /socket-0/cosid > ... > /socket-n/cosid > > The default value for the per socket cosid is COSID_DEFAULT, which > causes the task(s) to use the per cpu default id. I hope I've got all the details right, but this proposal looks awesome. There's more people who seem to agree with something like this. Btw, I think it should be possible to implement this with cgroups. But I too don't care that much on cgroups vs. syscalls.