From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752644AbdCFDAT (ORCPT ); Sun, 5 Mar 2017 22:00:19 -0500 Received: from szxga01-in.huawei.com ([45.249.212.187]:3818 "EHLO dggrg01-dlp.huawei.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1752556AbdCFDAS (ORCPT ); Sun, 5 Mar 2017 22:00:18 -0500 To: , , , CC: From: linhaifeng Subject: LLC cache miss increases with the increases of hugepages Message-ID: <58BCCF12.40804@huawei.com> Date: Mon, 6 Mar 2017 10:53:06 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset="gbk" Content-Transfer-Encoding: 8bit X-Originating-IP: [10.177.20.223] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A010201.58BCCF16.00D8,ss=1,re=0.000,recu=0.000,reip=0.000,cl=1,cld=1,fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 067417f73e126e7c91621b331f05f088 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org HiŁ¬ I did a network forward test with ovs-dpdk, and found that LLC cache miss increases with the increases of hugepages. ovs-dpdk only use 4 hugepages, hugepage size is 1G.Why LLC cache miss increases ? I just echo more hugepages but haven't been used yet. some informations: CPU: Intel(R) Xeon(R) CPU E5-2620 v3@ 2.40GHz L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 15360K echo 10 > /proc/sys/vm/nr_hugepages ovs-dpdk's LLC-load-misses:3.88% ovs-dpdk's iTLB-load-misses: 4.37% echo 100 > /proc/sys/vm/nr_hugepages reboot VM restart test. ovs-dpdk's LLC-load-misses:31.55% ovs-dpdk's iTLB-load-misses: 36.36% I use perf to watch LLC-loads when not run test: Samples: 1M of event 'LLC-loads', Event count (approx.): 57240206 Overhead Shared Object Symbol 9.19% [kernel] [k] copy_page_rep 5.03% [kernel] [k] ktime_get 4.06% [unknown] [.] 0x000000000043e39d 2.65% bash [.] 0x000000000003e39d 2.13% [kernel] [k] rcu_check_callbacks The function copy_page_rep seems like would prefetch all hugepages?