From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753845AbcGDR6j (ORCPT ); Mon, 4 Jul 2016 13:58:39 -0400 Received: from mx2.suse.de ([195.135.220.15]:53968 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753804AbcGDR6h (ORCPT ); Mon, 4 Jul 2016 13:58:37 -0400 Date: Mon, 4 Jul 2016 19:58:26 +0200 From: Borislav Petkov To: "Yu, Fenghua" Cc: "Luck, Tony" , Thomas Gleixner , Ingo Molnar , "Anvin, H Peter" , Peter Zijlstra , Stephane Eranian , "Shankar, Ravi V" , Vikas Shivappa , linux-kernel , x86 Subject: Re: [PATCH] cacheinfo: Introduce cache id Message-ID: <20160704175826.GC6349@pd.tnic> References: <1467251771-55501-1-git-send-email-fenghua.yu@intel.com> <20160701102143.GB4749@pd.tnic> <20160701165041.GA18186@intel.com> <20160701172743.GC4749@pd.tnic> <3E5A0FA7E9CA944F9D5414FEC6C712205DFD172A@ORSMSX106.amr.corp.intel.com> <20160701183536.GE4749@pd.tnic> <3E5A0FA7E9CA944F9D5414FEC6C712205DFD182F@ORSMSX106.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <3E5A0FA7E9CA944F9D5414FEC6C712205DFD182F@ORSMSX106.amr.corp.intel.com> User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Jul 01, 2016 at 07:24:27PM +0000, Yu, Fenghua wrote: > I haven't tested on AMD. But I think AMD should have the same code. Again, bear in mind, this is a qemu+kvm guest. > Could you please check if > /sys/device/system/cpu/cpu#/cache/index#/shared_cpu_map contains only > the cpu itself? Yes it does: $ grep . -EriIn /sys/devices/system/cpu/cpu*/cache/index*/shared_cpu_map /sys/devices/system/cpu/cpu0/cache/index0/shared_cpu_map:1:01 /sys/devices/system/cpu/cpu0/cache/index1/shared_cpu_map:1:01 /sys/devices/system/cpu/cpu0/cache/index2/shared_cpu_map:1:01 /sys/devices/system/cpu/cpu1/cache/index0/shared_cpu_map:1:02 /sys/devices/system/cpu/cpu1/cache/index1/shared_cpu_map:1:02 /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map:1:02 /sys/devices/system/cpu/cpu2/cache/index0/shared_cpu_map:1:04 /sys/devices/system/cpu/cpu2/cache/index1/shared_cpu_map:1:04 /sys/devices/system/cpu/cpu2/cache/index2/shared_cpu_map:1:04 /sys/devices/system/cpu/cpu3/cache/index0/shared_cpu_map:1:08 /sys/devices/system/cpu/cpu3/cache/index1/shared_cpu_map:1:08 /sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map:1:08 /sys/devices/system/cpu/cpu4/cache/index0/shared_cpu_map:1:10 /sys/devices/system/cpu/cpu4/cache/index1/shared_cpu_map:1:10 /sys/devices/system/cpu/cpu4/cache/index2/shared_cpu_map:1:10 /sys/devices/system/cpu/cpu5/cache/index0/shared_cpu_map:1:20 /sys/devices/system/cpu/cpu5/cache/index1/shared_cpu_map:1:20 /sys/devices/system/cpu/cpu5/cache/index2/shared_cpu_map:1:20 /sys/devices/system/cpu/cpu6/cache/index0/shared_cpu_map:1:40 /sys/devices/system/cpu/cpu6/cache/index1/shared_cpu_map:1:40 /sys/devices/system/cpu/cpu6/cache/index2/shared_cpu_map:1:40 /sys/devices/system/cpu/cpu7/cache/index0/shared_cpu_map:1:80 /sys/devices/system/cpu/cpu7/cache/index1/shared_cpu_map:1:80 /sys/devices/system/cpu/cpu7/cache/index2/shared_cpu_map:1:80 Here's the same from a real AMD system: grep . -EriIn /sys/devices/system/cpu/cpu*/cache/index*/shared_cpu_map /sys/devices/system/cpu/cpu0/cache/index0/shared_cpu_map:1:01 /sys/devices/system/cpu/cpu0/cache/index1/shared_cpu_map:1:01 /sys/devices/system/cpu/cpu0/cache/index2/shared_cpu_map:1:01 /sys/devices/system/cpu/cpu0/cache/index3/shared_cpu_map:1:3f /sys/devices/system/cpu/cpu1/cache/index0/shared_cpu_map:1:02 /sys/devices/system/cpu/cpu1/cache/index1/shared_cpu_map:1:02 /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map:1:02 /sys/devices/system/cpu/cpu1/cache/index3/shared_cpu_map:1:3f /sys/devices/system/cpu/cpu2/cache/index0/shared_cpu_map:1:04 /sys/devices/system/cpu/cpu2/cache/index1/shared_cpu_map:1:04 /sys/devices/system/cpu/cpu2/cache/index2/shared_cpu_map:1:04 /sys/devices/system/cpu/cpu2/cache/index3/shared_cpu_map:1:3f /sys/devices/system/cpu/cpu3/cache/index0/shared_cpu_map:1:08 /sys/devices/system/cpu/cpu3/cache/index1/shared_cpu_map:1:08 /sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map:1:08 /sys/devices/system/cpu/cpu3/cache/index3/shared_cpu_map:1:3f /sys/devices/system/cpu/cpu4/cache/index0/shared_cpu_map:1:10 /sys/devices/system/cpu/cpu4/cache/index1/shared_cpu_map:1:10 /sys/devices/system/cpu/cpu4/cache/index2/shared_cpu_map:1:10 /sys/devices/system/cpu/cpu4/cache/index3/shared_cpu_map:1:3f /sys/devices/system/cpu/cpu5/cache/index0/shared_cpu_map:1:20 /sys/devices/system/cpu/cpu5/cache/index1/shared_cpu_map:1:20 /sys/devices/system/cpu/cpu5/cache/index2/shared_cpu_map:1:20 /sys/devices/system/cpu/cpu5/cache/index3/shared_cpu_map:1:3f L3 is correctly shared between all cores. > From the cache id you dump on KVM, it tells there are 8 cache leaf 0, > 8 cache leaf 1, and 8 cache leaf 2. That means each CPU has its own > 3 caches (I can't tell from the cache id which level they are. The > cache/index#/level will tell that). > > If this is not the case, maybe cache id doesn't work on AMD. Maybe I > don't enable cache id for AMD? Nah, leave it as it is. Who knows what we might use it for on AMD. > From the info, cache id on one level is unique on that level across the board. Same on the AMD box: $ grep . -EriIn /sys/devices/system/cpu/cpu*/cache/index*/id /sys/devices/system/cpu/cpu0/cache/index0/id:1:0 /sys/devices/system/cpu/cpu0/cache/index1/id:1:0 /sys/devices/system/cpu/cpu0/cache/index2/id:1:0 /sys/devices/system/cpu/cpu0/cache/index3/id:1:0 /sys/devices/system/cpu/cpu1/cache/index0/id:1:1 /sys/devices/system/cpu/cpu1/cache/index1/id:1:1 /sys/devices/system/cpu/cpu1/cache/index2/id:1:1 /sys/devices/system/cpu/cpu1/cache/index3/id:1:1 /sys/devices/system/cpu/cpu2/cache/index0/id:1:2 /sys/devices/system/cpu/cpu2/cache/index1/id:1:2 /sys/devices/system/cpu/cpu2/cache/index2/id:1:2 /sys/devices/system/cpu/cpu2/cache/index3/id:1:2 /sys/devices/system/cpu/cpu3/cache/index0/id:1:3 /sys/devices/system/cpu/cpu3/cache/index1/id:1:3 /sys/devices/system/cpu/cpu3/cache/index2/id:1:3 /sys/devices/system/cpu/cpu3/cache/index3/id:1:3 /sys/devices/system/cpu/cpu4/cache/index0/id:1:4 /sys/devices/system/cpu/cpu4/cache/index1/id:1:4 /sys/devices/system/cpu/cpu4/cache/index2/id:1:4 /sys/devices/system/cpu/cpu4/cache/index3/id:1:4 /sys/devices/system/cpu/cpu5/cache/index0/id:1:5 /sys/devices/system/cpu/cpu5/cache/index1/id:1:5 /sys/devices/system/cpu/cpu5/cache/index2/id:1:5 /sys/devices/system/cpu/cpu5/cache/index3/id:1:5 > /sys/devices/system/cpu/cpu0/cache/index0/id:0 > /sys/devices/system/cpu/cpu0/cache/index1/id:0 > /sys/devices/system/cpu/cpu0/cache/index2/id:0 > /sys/devices/system/cpu/cpu0/cache/index3/id:0 > /sys/devices/system/cpu/cpu10/cache/index0/id:17 > /sys/devices/system/cpu/cpu10/cache/index1/id:17 > /sys/devices/system/cpu/cpu10/cache/index2/id:17 Hmm, so non-L3 caches, i.e., the unshared ones, have different IDs from the CPU numbers. That's probably because we use the APIC IDs to generate the cache IDs. Not that it matters too much... ... > /sys/devices/system/cpu/cpu0/cache/index0/shared_cpu_list:0,36 > /sys/devices/system/cpu/cpu0/cache/index1/shared_cpu_list:0,36 > /sys/devices/system/cpu/cpu0/cache/index2/shared_cpu_list:0,36 > /sys/devices/system/cpu/cpu0/cache/index3/shared_cpu_list:0-17,36-53 Right, and this shows what is shared by what. -- Regards/Gruss, Boris. SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) --