All of lore.kernel.org
 help / color / mirror / Atom feed
* Re: [another NUMA code dump] -3.9% netperf throughput drop
       [not found] <20130828140240.GA28272@localhost>
@ 2013-08-30 10:07 ` Fengguang Wu
  0 siblings, 0 replies; only message in thread
From: Fengguang Wu @ 2013-08-30 10:07 UTC (permalink / raw)
  To: lkp

[-- Attachment #1: Type: text/plain, Size: 28475 bytes --]

Hi Peter,

On Wed, Aug 28, 2013 at 10:02:40PM +0800, Fengguang Wu wrote:
> Hi Peter,
> 
> FYI, we find a little netperf performance drop, together with much
> reduced lock contentions.  Please let me know if you would like us do
> more tests on it. 
> 
> 126d9c20bc99ccf1d62581d6e7fd63fd12e222bf is the first bad commit
> commit 126d9c20bc99ccf1d62581d6e7fd63fd12e222bf
> Author: Peter Zijlstra <peterz@infradead.org>
> Date:   Tue Aug 20 15:36:48 2013 +0200
> 
>     another NUMA code dump

Now we got more results: there are both ups and downs.

                  661.10        +2.3%       676.20  TOTAL netperf.Throughput_Mbps
               165733.95        -6.7%    154709.48  TOTAL aim7.2000.jobs-per-min
                 2203.20        -1.7%      2165.50  TOTAL netperf.Throughput_tps
                  569.20       -68.0%       181.94  TOTAL slabinfo.skbuff_fclone_cache.num_objs
                  542.84       -66.5%       181.94  TOTAL slabinfo.skbuff_fclone_cache.active_objs
                  376.92       -69.2%       116.27  TOTAL sched_debug.cpu#2.nr_uninterruptible
                  441.02       +89.6%       836.36  TOTAL sched_debug.cfs_rq[7]:/.nr_spread_over
                62464.94       -36.5%     39676.33  TOTAL interrupts.CPU3.RES
                53942.34       -46.8%     28701.16  TOTAL interrupts.CPU1.RES
                   85.95      +164.3%       227.20  TOTAL sched_debug.cpu#5.nr_uninterruptible
                73764.87       -32.5%     49819.94  TOTAL interrupts.CPU2.RES
                 7698.32      +118.7%     16836.46  TOTAL sched_debug.cfs_rq[7]:/.spread0
                89601.90       -31.3%     61544.49  TOTAL interrupts.CPU0.RES
               496290.10       -25.8%    368250.18  TOTAL interrupts.RES
                11166.54       +82.7%     20397.97  TOTAL sched_debug.cfs_rq[7]:/.min_vruntime
               158066.00       -34.2%    103970.26  TOTAL softirqs.CPU0.NET_RX
               630988.15       -29.8%    442956.09  TOTAL softirqs.NET_RX
               156872.34       -29.2%    110987.49  TOTAL softirqs.CPU2.NET_RX
               157622.69       -30.0%    110326.06  TOTAL softirqs.CPU3.NET_RX
                 7690.77       -15.2%      6523.75  TOTAL sched_debug.cpu#2.ttwu_local
                33587.66       -16.5%     28048.84  TOTAL sched_debug.cpu#1.sched_goidle
               158427.11       -25.7%    117672.28  TOTAL softirqs.CPU1.NET_RX
                 8975.45       -21.6%      7038.36  TOTAL sched_debug.cpu#3.ttwu_local
                21103.12       -21.6%     16553.45  TOTAL softirqs.CPU3.RCU
                21376.43       -20.8%     16934.73  TOTAL softirqs.CPU2.RCU
                 1663.42        -1.9%      1631.68  TOTAL slabinfo.kmalloc-2048.num_objs
                 1554.94        +3.8%      1613.65  TOTAL slabinfo.kmalloc-2048.active_objs
              3526131.15       +20.5%   4248631.25  TOTAL proc-vmstat.pgalloc_normal
                56884.60       +20.3%     68432.74  TOTAL slabinfo.kmalloc-64.num_objs
                55951.48       +19.8%     67030.33  TOTAL slabinfo.kmalloc-64.active_objs
                16903.28       +18.2%     19976.14  TOTAL softirqs.CPU0.SCHED
                24396.71       -16.6%     20337.48  TOTAL interrupts.CPU6.RES
                24424.39       -15.4%     20660.17  TOTAL interrupts.CPU5.RES
                68269.93       -14.8%     58135.54  TOTAL sched_debug.cpu#6.nr_switches
                68301.75       -14.8%     58169.82  TOTAL sched_debug.cpu#6.sched_count
                36349.50       -14.3%     31146.73  TOTAL sched_debug.cpu#0.sched_goidle
               169088.39       -14.3%    144828.56  TOTAL softirqs.RCU
                29181.61       -13.3%     25309.49  TOTAL sched_debug.cpu#5.sched_goidle
                29244.19       -14.7%     24941.43  TOTAL sched_debug.cpu#6.sched_goidle
                46361.62        +7.7%     49935.54  TOTAL softirqs.SCHED
                 2254.81       -11.6%      1993.26  TOTAL sched_debug.cfs_rq[2]:/.min_vruntime
                84214.17       -12.9%     73322.31  TOTAL sched_debug.cpu#3.nr_switches
                21465.75       -15.9%     18061.93  TOTAL softirqs.CPU0.RCU
                84259.37       -12.9%     73360.78  TOTAL sched_debug.cpu#3.sched_count
                 2717.34       +13.4%      3081.63  TOTAL slabinfo.kmalloc-128.num_objs
                 2717.34       +12.7%      3061.40  TOTAL slabinfo.kmalloc-128.active_objs
                44008.82       -13.0%     38280.08  TOTAL sched_debug.cpu#3.ttwu_count
                20872.34       -11.8%     18400.51  TOTAL softirqs.CPU4.RCU
                 1164.51       +13.5%      1321.68  TOTAL buddyinfo.Node.0.zone.Normal.0
                 1687.56       +12.3%      1895.26  TOTAL slabinfo.filp.active_objs
                67219.90       -12.5%     58843.19  TOTAL sched_debug.cpu#5.nr_switches
                67253.78       -12.5%     58880.24  TOTAL sched_debug.cpu#5.sched_count
              5312453.48       +13.5%   6029326.46  TOTAL proc-vmstat.pgfree
                21554.90       -13.8%     18579.77  TOTAL softirqs.CPU6.RCU
                36507.99       -14.9%     31078.42  TOTAL sched_debug.cpu#2.sched_goidle
                43167.98       -13.2%     37457.64  TOTAL sched_debug.cpu#2.ttwu_count
                24019.62       -11.5%     21267.02  TOTAL interrupts.CPU7.RES
                82664.49       -15.0%     70263.14  TOTAL sched_debug.cpu#2.sched_count
                82620.08       -15.0%     70223.42  TOTAL sched_debug.cpu#2.nr_switches
                37659.16       -13.7%     32493.72  TOTAL sched_debug.cpu#6.ttwu_count
                20774.12       -10.7%     18544.93  TOTAL softirqs.CPU1.RCU
                28506.05       -10.2%     25592.08  TOTAL sched_debug.cpu#7.sched_goidle
                35939.20       -11.5%     31822.25  TOTAL sched_debug.cpu#3.sched_goidle
                21152.73       -11.4%     18737.90  TOTAL softirqs.CPU5.RCU
                  861.68       +12.4%       968.22  TOTAL buddyinfo.Node.0.zone.Normal.1
                37078.89       -10.4%     33230.12  TOTAL sched_debug.cpu#5.ttwu_count
               860788.53       +10.6%    952024.03  TOTAL proc-vmstat.numa_hit
               860788.53       +10.6%    952023.96  TOTAL proc-vmstat.numa_local
                 7802.01       +11.1%      8670.11  TOTAL sched_debug.cpu#5.nr_load_updates
                65555.21        -9.3%     59426.89  TOTAL sched_debug.cpu#7.nr_switches
                65585.05        -9.3%     59462.10  TOTAL sched_debug.cpu#7.sched_count
                 8846.95       -14.5%      7563.70  TOTAL sched_debug.cpu#6.ttwu_local
                22195.37        -9.0%     20188.74  TOTAL softirqs.CPU3.SCHED
                  227.63        +8.2%       246.30  TOTAL uptime.idle
                23712.01        +9.2%     25893.32  TOTAL interrupts.LOC
                 4307.40       -99.3%        31.00  TOTAL lock_stat.((&icsk->icsk_retransmit_timer)).acquisitions
                 7139.90       -96.3%       265.40  TOTAL lock_stat.((&icsk->icsk_delack_timer)).acquisitions
               566879.90       -99.9%       657.00  TOTAL lock_stat.&wq->wait.acq-bounces
                13827.10       -99.6%        56.80  TOTAL lock_stat.&(&base->lock)->rlock.con-bounces
                40546.40       -99.6%       161.40  TOTAL lock_stat.&(&base->lock)->rlock.contentions.lock_timer_base
                13827.30       -99.6%        57.00  TOTAL lock_stat.&(&base->lock)->rlock.contentions
                74164.23       -99.2%       628.83  TOTAL lock_stat.&(&base->lock)->rlock.waittime-total
                 4249.11      -100.0%         0.00  TOTAL lock_stat.&(&list->lock)->rlock#2.waittime-total
               215936.60       -99.8%       324.60  TOTAL lock_stat.&(&list->lock)->rlock#2.acq-bounces
                 2975.23      -100.0%         0.00  TOTAL lock_stat.&wq->wait.waittime-total
                12327.20       -99.9%         8.30  TOTAL lock_stat.slock-AF_INET.contentions.release_sock
                23594.90       -99.9%        13.80  TOTAL lock_stat.slock-AF_INET/1.con-bounces
                23594.90       -99.9%        13.80  TOTAL lock_stat.slock-AF_INET/1.contentions
                23594.90       -99.9%        13.80  TOTAL lock_stat.slock-AF_INET/1.contentions.tcp_v4_rcv
                11909.80       -99.9%         7.40  TOTAL lock_stat.slock-AF_INET/1.contentions.release_sock
                11682.50       -99.9%         7.11  TOTAL lock_stat.slock-AF_INET/1.contentions.lock_sock_nested
              2891155.00      +793.9%  25844317.50  TOTAL lock_stat.&pipe->mutex/1.acquisitions
                  584.68      +770.8%      5091.37  TOTAL lock_stat.&pipe->wait.waittime-total
              2870693.90      +799.0%  25808748.90  TOTAL lock_stat.&pipe->wait.acquisitions
               653854.40       -99.9%       361.90  TOTAL lock_stat.slock-AF_INET/1.acq-bounces
               132668.89      -100.0%        58.82  TOTAL lock_stat.slock-AF_INET/1.waittime-total
               135481.90       -99.9%        86.50  TOTAL lock_stat.slock-AF_INET.contentions.tcp_v4_rcv
               113885.70       -99.9%        75.00  TOTAL lock_stat.slock-AF_INET.contentions.lock_sock_nested
               834535.10       -99.9%       695.50  TOTAL lock_stat.slock-AF_INET.acq-bounces
               135509.30       -99.9%        86.50  TOTAL lock_stat.slock-AF_INET.con-bounces
               115477.01       -96.7%      3808.53  TOTAL lock_stat.((&icsk->icsk_delack_timer)).holdtime-total
               135511.30       -99.9%        86.50  TOTAL lock_stat.slock-AF_INET.contentions
               916330.86       -93.8%     56991.34  TOTAL lock_stat.&rq->lock.waittime-total
              6994048.67       -99.9%      4638.26  TOTAL lock_stat.slock-AF_INET.waittime-total
                72918.76       -93.8%      4501.73  TOTAL lock_stat.((&icsk->icsk_retransmit_timer)).holdtime-total
                  561.79      +366.8%      2622.34  TOTAL lock_stat.&pipe->wait.waittime-max
                37855.16      +392.2%    186310.37  TOTAL lock_stat.(&task->u.tk_work).holdtime-max
                37856.44      +392.2%    186311.75  TOTAL lock_stat.rpciod-R.holdtime-max
              1882726.00       -30.9%   1301055.80  TOTAL lock_stat.&rq->lock.acq-bounces
                 1192.74      +349.8%      5364.45  TOTAL lock_stat.slock-AF_INET-RPC.waittime-total
              1057131.87       -68.5%    332629.78  TOTAL lock_stat.&rq->lock/1.holdtime-total
               230850.10       -47.2%    121845.20  TOTAL lock_stat.&rq->lock.contentions.try_to_wake_up
              2054554.10      +337.8%   8995698.10  TOTAL lock_stat.&p->pi_lock.acquisitions
               958070.10       -86.1%    133493.10  TOTAL lock_stat.&(&base->lock)->rlock.acq-bounces
                 3001.10       -83.1%       508.26  TOTAL lock_stat.&mm->mmap_sem-R.holdtime-max
                41448.45      +277.4%    156411.45  TOTAL lock_stat.zone->wait_table+i.waittime-total
               206432.67      +247.1%    716507.42  TOTAL lock_stat.(&task->u.tk_work).holdtime-total
               207795.52      +246.0%    718916.47  TOTAL lock_stat.rpciod-R.holdtime-total
                42415.30       +13.1%     47964.02  TOTAL lock_stat.&rq->lock/1.waittime-total
              1598626.30      +671.8%  12338701.15  TOTAL lock_stat.&pipe->mutex/1.holdtime-total
               240154.70       -24.1%    182204.10  TOTAL lock_stat.&rq->lock.contentions
               239875.20       -24.1%    182101.50  TOTAL lock_stat.&rq->lock.con-bounces
             10334983.54      +260.8%  37290852.98  TOTAL lock_stat.all_cpu_access_lock.holdtime-total
             10382840.38      +260.4%  37424524.18  TOTAL lock_stat.trace_event_sem-R.holdtime-total
             10682456.18      +257.6%  38195309.13  TOTAL lock_stat.&iter->mutex.holdtime-total
                11622.60      +254.2%     41166.90  TOTAL lock_stat.slock-AF_INET-RPC/1.acquisitions
                11716.00      +252.2%     41260.90  TOTAL lock_stat._xmit_ETHER#2.acquisitions
                23432.00      +252.2%     82521.90  TOTAL lock_stat.&(&list->lock)->rlock.acquisitions
             15159239.80      +253.4%  53574107.00  TOTAL lock_stat.buffer->reader_lock_key.acquisitions
             13357123.60      +261.4%  48273636.60  TOTAL lock_stat.&rq->lock.acquisitions
              3250919.71      +225.8%  10592024.15  TOTAL lock_stat.buffer->reader_lock_key.holdtime-total
                15000.00      +219.6%     47943.30  TOTAL lock_stat.slock-AF_INET-RPC.acquisitions
              1821865.51      +799.3%  16383840.49  TOTAL lock_stat.&pipe->wait.holdtime-total
                 6306.35      +216.5%     19962.11  TOTAL lock_stat.&(&list->lock)->rlock.holdtime-total
               104745.20      +220.5%    335684.60  TOTAL lock_stat.trace_types_lock.acquisitions
               104745.20      +220.5%    335684.00  TOTAL lock_stat.trace_event_sem-R.acquisitions
               104745.20      +220.5%    335683.80  TOTAL lock_stat.all_cpu_access_lock.acquisitions
                 1411.65      +126.4%      3196.43  TOTAL lock_stat.&vfsmount_lock-R.holdtime-max
              1669574.80       -60.4%    661408.70  TOTAL lock_stat.&(&base->lock)->rlock.acquisitions
               116585.00      +197.0%    346231.10  TOTAL lock_stat.&iter->mutex.acquisitions
                39046.00      +188.4%    112601.00  TOTAL lock_stat.trace_types_lock.holdtime-total
              2356238.10       -20.5%   1872483.90  TOTAL lock_stat.rcu_read_lock_bh-R.acquisitions
               712340.80       -58.5%    295574.20  TOTAL lock_stat.&wq->wait.acquisitions
               106482.08      +160.0%    276823.43  TOTAL lock_stat.slock-AF_INET-RPC/1.holdtime-total
               161126.72      +183.9%    457435.26  TOTAL lock_stat._xmit_ETHER#2.holdtime-total
               134393.55      +183.8%    381435.33  TOTAL lock_stat.sk_lock-AF_INET-RPC.holdtime-total
                23948.20      +127.0%     54351.40  TOTAL lock_stat.&sb->s_type->i_lock_key#1.acquisitions
               509327.41       -54.9%    229766.53  TOTAL lock_stat.&sig->cred_guard_mutex.holdtime-max
                27084.80      +118.9%     59298.00  TOTAL lock_stat.sb_writers#4-R.acquisitions
                 6221.93       -51.1%      3045.54  TOTAL lock_stat.sk_lock-AF_INET.holdtime-max
                 4656.87      +181.9%     13126.12  TOTAL lock_stat.&pipe->mutex/1.holdtime-max
                10765.33      +123.3%     24037.30  TOTAL lock_stat.(&task->u.tk_work)#2.holdtime-total
             43141327.41       -13.1%  37507805.71  TOTAL lock_stat.rcu_read_lock_bh-R.holdtime-total
                 4100.50       -51.8%      1975.29  TOTAL lock_stat.rcu_read_lock-R.holdtime-max
               739556.16       -17.4%    610769.01  TOTAL lock_stat.&mm->mmap_sem-W.holdtime-max
                11290.17      +117.2%     24520.74  TOTAL lock_stat.nfsiod-R.holdtime-total
                42400.54       -47.1%     22416.67  TOTAL lock_stat.&(&info->si_lock)->rlock.holdtime-total
                48336.50       +80.9%     87451.90  TOTAL lock_stat.&(&nf->lru_lock)->rlock.contentions.ip_defrag
              1168351.69       +10.8%   1294640.42  TOTAL lock_stat.&pipe->mutex/1.waittime-total
               102707.56       -48.3%     53101.39  TOTAL lock_stat.&sb->s_type->i_mutex_key#2.waittime-total
             91318694.78       +26.6% 115632734.76  TOTAL lock_stat.&rq->lock.holdtime-total
             50189004.65       +18.5%  59471866.33  TOTAL lock_stat.&p->pi_lock.holdtime-total
               133163.30       -44.5%     73942.90  TOTAL lock_stat.&anon_vma->rwsem.acq-bounces
                66818.00       -45.6%     36333.20  TOTAL lock_stat.&p->pi_lock.acq-bounces
               361874.50       +82.2%    659237.20  TOTAL lock_stat.&(&zone->lock)->rlock.contentions.__free_pages_ok
                 3957.60       -33.3%      2638.84  TOTAL lock_stat.tasklist_lock-W.waittime-total
               244764.50       +72.1%    421131.00  TOTAL lock_stat.&(&zone->lock)->rlock.contentions.get_page_from_freelist
              6540563.51       -53.2%   3058553.07  TOTAL lock_stat.&(&base->lock)->rlock.holdtime-total
               214843.60       +74.5%    374915.00  TOTAL lock_stat.&(&nf->lru_lock)->rlock.con-bounces
               214846.50       +74.5%    374916.00  TOTAL lock_stat.&(&nf->lru_lock)->rlock.contentions
               641418.67       +75.6%   1126321.06  TOTAL lock_stat.&(&nf->lru_lock)->rlock.waittime-total
               192855.90       +76.0%    339378.40  TOTAL lock_stat.&(&nf->lru_lock)->rlock.contentions.inet_frag_find
              2322889.38      +111.6%   4915093.78  TOTAL lock_stat.sb_writers#7-R.holdtime-total
                63744.63       -42.3%     36809.55  TOTAL lock_stat.&(&n->list_lock)->rlock.holdtime-total
               188500.60       +71.4%    323001.70  TOTAL lock_stat.&(&nf->lru_lock)->rlock.contentions.inet_frag_kill
                14078.22       +71.4%     24129.54  TOTAL lock_stat.&(&queue->lock)->rlock.holdtime-total
               350330.40       +68.4%    589915.00  TOTAL lock_stat.&(&zone->lock)->rlock.contentions
               350276.20       +68.2%    589108.30  TOTAL lock_stat.&(&zone->lock)->rlock.con-bounces
                 2435.44       +68.8%      4109.92  TOTAL lock_stat.&(&hb->chain_lock)->rlock.waittime-total
                11039.58       +38.2%     15255.39  TOTAL lock_stat.(&(&wb->dwork)->work).holdtime-total
             45302081.74        -8.2%  41581942.88  TOTAL lock_stat.&wq->wait.holdtime-total
               182856.27       +59.9%    292446.19  TOTAL lock_stat.zone->wait_table+i.holdtime-total
              2182608.45       +60.5%   3502508.79  TOTAL lock_stat.&(&zone->lock)->rlock.waittime-total
                21148.30       +57.5%     33298.65  TOTAL lock_stat.&(&xprt->transport_lock)->rlock.holdtime-total
                11841.06       +49.7%     17725.30  TOTAL lock_stat.&sb->s_type->i_lock_key#1.holdtime-total
                20350.70       +48.1%     30148.50  TOTAL lock_stat.&pipe->mutex/1.acq-bounces
                19903.70       +48.3%     29522.80  TOTAL lock_stat.&pipe->wait.acq-bounces
                12025.44       +88.2%     22631.20  TOTAL lock_stat.&type->i_mutex_dir_key#2.holdtime-total
                12025.44       +88.2%     22631.20  TOTAL lock_stat.&type->i_mutex_dir_key#2.holdtime-min
                12025.44       +88.2%     22631.20  TOTAL lock_stat.&type->i_mutex_dir_key#2.holdtime-max
               459953.53        +9.5%    503614.41  TOTAL lock_stat.&mapping->i_mmap_mutex.waittime-max
                36732.44       +38.4%     50824.74  TOTAL lock_stat.sb_writers#4-R.holdtime-total
               439874.42        -5.3%    416534.11  TOTAL lock_stat.clock-AF_INET-R.holdtime-total
               277796.51       +44.6%    401667.68  TOTAL lock_stat.&p->pi_lock.waittime-total
              2313283.70       -26.8%   1693267.70  TOTAL lock_stat.&(&list->lock)->rlock#3.acquisitions
                74032.10       -28.6%     52864.10  TOTAL lock_stat.&rq->lock/1.acquisitions
               931324.30       +34.7%   1254788.40  TOTAL lock_stat.&(&zone->lock)->rlock.acq-bounces
                14328.30       +25.0%     17906.30  TOTAL lock_stat.hrtimer_bases.lock#7.holdtime-total
                37493.70       -26.3%     27624.80  TOTAL lock_stat.&rq->lock/1.acq-bounces
             15085692.76       +36.5%  20597805.11  TOTAL lock_stat.rcu_read_lock-R.holdtime-total
                31239.40       +36.1%     42514.40  TOTAL lock_stat.zone->wait_table+i.contentions.__wake_up
               159447.37       +32.3%    211019.95  TOTAL lock_stat.rcu_node_0.waittime-total
               100236.97       -33.9%     66247.10  TOTAL lock_stat.&pipe->mutex/1.waittime-max
              1166309.60       -23.3%    894892.70  TOTAL lock_stat.slock-AF_INET/1.acquisitions
                 4053.60       +29.4%      5245.03  TOTAL lock_stat.sk_lock-AF_INET6.holdtime-total
                 5036.72       -23.9%      3834.90  TOTAL lock_stat.&(&tsk->delays->lock)->rlock.holdtime-total
                10019.40       +45.8%     14613.28  TOTAL lock_stat.slock-AF_INET-RPC.holdtime-total
              1417553.50       +23.9%   1755833.90  TOTAL lock_stat.&(&zone->lock)->rlock.acquisitions
              6946024.55       -19.3%   5602189.45  TOTAL lock_stat.&(&list->lock)->rlock#3.holdtime-total
                 5172.44       -19.2%      4177.08  TOTAL lock_stat.&q->lock.holdtime-total
                18369.95       +23.1%     22618.18  TOTAL lock_stat.(&(work)->timer).holdtime-total
               834643.80       +22.7%   1024368.10  TOTAL lock_stat.&(&nf->lru_lock)->rlock.acq-bounces
                25251.78       +22.4%     30899.04  TOTAL lock_stat.&(&pool->lock)->rlock.holdtime-total
              3931866.39       +23.0%   4834661.56  TOTAL lock_stat.((&timer)).holdtime-total
               400630.70       -19.6%    322076.80  TOTAL lock_stat.&(&dentry->d_lock)->rlock.contentions.path_get
               729459.97       -23.7%    556449.31  TOTAL lock_stat.&mapping->i_mmap_mutex.waittime-total
              1205124.95       +17.2%   1412040.97  TOTAL lock_stat.&rsp->gp_wq.holdtime-total
               778969.60       -18.0%    639110.20  TOTAL lock_stat.&(&dentry->d_lock)->rlock.contentions.dput
               203249.44       -17.2%    168359.45  TOTAL lock_stat.&(&dentry->d_lock)->rlock.waittime-total
               787586.90       -16.9%    654513.10  TOTAL lock_stat.&(&dentry->d_lock)->rlock.con-bounces
               787737.60       -16.9%    654670.40  TOTAL lock_stat.&(&dentry->d_lock)->rlock.contentions
               982029.40       +20.0%   1178821.80  TOTAL lock_stat.&f->lock-R.acq-bounces
             11377298.70       -15.7%   9591520.20  TOTAL lock_stat.rcu_read_lock-R.acquisitions
                21232.88       +19.0%     25274.64  TOTAL lock_stat.&rnp->nocb_gp_wq[1].holdtime-total
                22050.24       +18.9%     26217.07  TOTAL lock_stat.&rnp->nocb_gp_wq[0].holdtime-total
               175592.50       +16.0%    203669.97  TOTAL lock_stat.clockevents_lock.waittime-total
               182770.97       +18.0%    215662.07  TOTAL lock_stat.rcu_sched_state.onoff_mutex.holdtime-total
               169134.80       -14.9%    143897.00  TOTAL lock_stat.zone->wait_table+i.acquisitions
             11269453.16       +12.6%  12691439.20  TOTAL lock_stat.&(&zone->lock)->rlock.holdtime-total
                13525.95       +17.0%     15830.27  TOTAL lock_stat.(&(work)->work).holdtime-total
                 8975.59       +16.8%     10485.40  TOTAL lock_stat.proc_subdir_lock.holdtime-total
                99222.00       +16.0%    115055.75  TOTAL lock_stat.sb_writers#2-R.holdtime-total
               124847.90       -13.7%    107753.10  TOTAL lock_stat.zone->wait_table+i.acq-bounces
              1518193.46       +10.4%   1676339.59  TOTAL lock_stat.rcu_node_0.holdtime-total
                95007.69       +15.2%    109413.23  TOTAL lock_stat.&sig->wait_chldexit.holdtime-total
               256302.50       -13.1%    222725.30  TOTAL lock_stat.&mm->mmap_sem-W.acq-bounces
                 6934.41       +14.6%      7944.90  TOTAL lock_stat.sb_writers#3-R.holdtime-total
                15463.56       +13.7%     17577.66  TOTAL lock_stat.entries_lock-R.holdtime-total
               263989.70       -13.1%    229316.00  TOTAL lock_stat.&(&mm->page_table_lock)->rlock.acq-bounces
                13700.25       +87.2%     25651.38  TOTAL lock_stat.hrtimer_bases.lock#4.holdtime-total
                10082.81       +14.1%     11507.23  TOTAL lock_stat.bdev_lock.holdtime-total
               258345.90       -12.6%    225740.10  TOTAL lock_stat.&(&newf->file_lock)->rlock.acq-bounces
                28512.94       +12.8%     32154.48  TOTAL lock_stat.events-R.holdtime-total
                47912.04       -12.3%     41996.01  TOTAL lock_stat.rename_lock.waittime-total
               224803.20       -12.6%    196381.70  TOTAL lock_stat.&(&fs->lock)->rlock.acq-bounces
               103370.20       -12.7%     90277.20  TOTAL lock_stat.rename_lock.contentions.SyS_getcwd
                13222.56       +18.3%     15637.30  TOTAL lock_stat.hrtimer_bases.lock#5.holdtime-total
               702867.20       -12.0%    618778.40  TOTAL lock_stat.&mapping->i_mmap_mutex.contentions.vma_link
               238640.40       -11.8%    210514.70  TOTAL lock_stat.&(&sighand->siglock)->rlock.acq-bounces
               144491.30       +12.8%    163038.30  TOTAL lock_stat.clockevents_lock.con-bounces
               289138.00       +12.8%    326170.80  TOTAL lock_stat.clockevents_lock.contentions.clockevents_notify
               144569.00       +12.8%    163085.40  TOTAL lock_stat.clockevents_lock.contentions
               585553.00       -11.5%    518161.60  TOTAL lock_stat.&mapping->i_mmap_mutex.con-bounces
               585553.00       -11.5%    518161.60  TOTAL lock_stat.&mapping->i_mmap_mutex.contentions
               174036.80       +11.7%    194446.50  TOTAL lock_stat.((&timer)).acquisitions
               198920.50       +12.5%    223756.40  TOTAL lock_stat.rcu_node_0.acquisitions
               159105.97       +12.2%    178524.65  TOTAL lock_stat.&(&p->alloc_lock)->rlock.holdtime-total
                12962.55       +11.3%     14431.29  TOTAL lock_stat.&child->perf_event_mutex.holdtime-total
               467032.10       -10.9%    416276.60  TOTAL lock_stat.&mapping->i_mmap_mutex.contentions.unlink_file_vma
              2639031.80       -10.6%   2360489.00  TOTAL lock_stat.sk_lock-AF_INET.acquisitions
                23964.02        +9.5%     26236.37  TOTAL lock_stat.&fsnotify_mark_srcu-R.holdtime-total
               706694.52        -9.6%    638876.20  TOTAL lock_stat.jiffies_lock.holdtime-total
                14111.25       +36.4%     19245.96  TOTAL lock_stat.hrtimer_bases.lock.holdtime-total
                47545.73       +11.3%     52915.50  TOTAL lock_stat.pgd_lock.holdtime-total
               319799.80       +10.4%    353118.34  TOTAL lock_stat.tasklist_lock-W.holdtime-total
             13131948.50       -10.6%  11739786.10  TOTAL lock_stat.&(&dentry->d_lock)->rlock.acq-bounces
              5748518.20       -10.3%   5156548.00  TOTAL lock_stat.&mapping->i_mmap_mutex.acq-bounces
             10293967.02        -6.7%   9599300.67  TOTAL lock_stat.&mm->mmap_sem-R.holdtime-total
              2486744.60      +590.1%  17161466.60  TOTAL perf-stat.context-switches
                91101.74       +19.8%    109116.17  TOTAL vmstat.system.cs
                   15.96       +17.8%        18.79  TOTAL iostat.cpu.user
                   19.81       +16.5%        23.07  TOTAL vmstat.cpu.us
                 1818.70       +29.6%      2356.40  TOTAL time.involuntary_context_switches
               313376.90       -20.3%    249842.70  TOTAL perf-stat.cpu-migrations
                17606.67       -17.4%     14540.72  TOTAL vmstat.system.in
               669432.20       -12.7%    584367.90  TOTAL time.voluntary_context_switches
                  306.12        +0.1%       306.52  TOTAL iostat.cpu.system
                   72.62        +7.1%        77.79  TOTAL time.elapsed_time
                  400.28        +0.0%       400.32  TOTAL vmstat.cpu.sy
                  188.10        -6.8%       175.40  TOTAL time.percent_of_cpu_this_job_got
                    1.35        -7.9%         1.24  TOTAL iostat.sda.rkB/s
                    0.35        -7.9%         0.32  TOTAL vmstat.io.bi
                  178.19        -4.7%       169.90  TOTAL vmstat.cpu.wa
                  178.08        -4.6%       169.91  TOTAL iostat.cpu.iowait
                  311.74        +2.6%       319.92  TOTAL boottime.dhcp
                  269.22        +2.7%       276.37  TOTAL boottime.boot
                  568.45        +3.1%       586.18  TOTAL boottime.idle
                 2699.20        +0.6%      2714.10  TOTAL time.minor_page_faults
                   82.14        +0.3%        82.40  TOTAL iostat.cpu.idle
                   82.12        +0.3%        82.39  TOTAL vmstat.cpu.id

Details attached.

Thanks,
Fengguang


[-- Attachment #2: 126d9c20bc99ccf1d62581d6e7fd63fd12e222bf.ksh --]
[-- Type: text/plain, Size: 853266 bytes --]

\f
                              netperf.Throughput_Mbps

   239 ++-------------*-----------------------------------------------------+
       |           ..       ..                                              |
   238 *+......  ..           *.......*......*.              *......    ....*
   237 ++      *.             O                ..          ..       *...    |
       |                                         .        .                 |
   236 ++                                         .      .                  |
   235 ++                                          ..  ..                   |
       |                                              .                     |
   234 ++                                            *                      |
   233 O+                                                           O       |
       |                                                                    |
   232 ++      O                                                            |
   231 ++                             O      O               O              O
       |                                                                    |
   230 ++-------------O------------------------------O----------------------+

\f
                              netperf.Throughput_Mbps

   192 ++-------------------------------------------------------------------+
       |                  ....*.......                                      |
   190 *+......    ...*...            *......*.......*.......*......*.......*
       |       *...                                                         |
       |                                                                    |
   188 ++                                                                   |
       |                                                                    |
   186 ++                                                                   |
       |                                                                    |
   184 ++                                                                   |
       |       O              O                                             O
       |                                                                    |
   182 O+                             O      O       O       O      O       |
       |              O                                                     |
   180 ++-------------------------------------------------------------------+

\f
                              netperf.Throughput_Mbps

   270 ++------O------------------------------------------------------------+
       |                                                            O       |
   265 O+             O               O                      O              |
   260 ++                                    O                              |
       |                      O                                             |
   255 ++                                                                   O
       |                                                                    |
   250 ++                                            O                      |
       |                                                                    |
   245 ++                                                                   |
   240 ++                                                                   |
       |                                                                    |
   235 ++          ...*.......                   ....*.....             ....*
       *.......*...           *.......*......*...          ..    ...*...    |
   230 ++----------------------------------------------------*--------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  237.50        -2.4%       231.90  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                  190.00        -4.1%       182.20  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                  233.60       +12.2%       262.10  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                  661.10        +2.3%       676.20  TOTAL netperf.Throughput_Mbps
\f
                                aim7.2000.jobs-per-min

   168000 ++--------------------*--------------*----------------------------+
          |                    .      .  ....         *......*.......    ...*
   166000 *+.....*..          .        *.                            *...   |
   164000 ++        ..      ..                                              |
          |           ..   .                                                |
   162000 ++            . .                                                 |
   160000 ++             *                                                  |
          |                                                                 |
   158000 ++             O                                                  |
   156000 O+                                          O      O       O      |
          |      O                                                          |
   154000 ++                    O                                           |
   152000 ++                           O                                    O
          |                                    O                            |
   150000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               165733.95        -6.7%    154709.48  nhm-white/micro/aim7/misc_rtns_1
               165733.95        -6.7%    154709.48  TOTAL aim7.2000.jobs-per-min
\f
                               netperf.Throughput_tps

     167 *+-----*-------*------*-------*------O-------*--------------*------*
         |                              +            . +            .       |
         |                               +          .   +          .        |
   166.5 ++                               +       ..     +       ..         |
         |                                 +     .        +     .           |
         |                                  +   .          +   .            |
         |                                   + .            + .             |
     166 ++                                   *       O      *              |
         |                                                                  |
         |                                                                  |
   165.5 ++                                                                 |
         |                                                                  |
         |                                                                  |
         |                                                                  |
     165 O+-----O-------O------O-------O-----------------------------O------O

\f
                              netperf.Throughput_tps

   910 ++-------------------------------------------------------------------+
       |                                                                    |
   905 ++                                 ...*.....              ...*.......*
       |           ...*.....        ..*...         ..    ....*...           |
   900 *+......*...         ..  ....                 *...                   |
       |                      *.                                            |
   895 ++                                                                   |
       |                                                                    |
   890 ++             O                      O                              |
       O       O                      O                                     |
   885 ++                                            O              O       |
       |                                                                    |
   880 ++                     O                              O              |
       |                                                                    O
   875 ++-------------------------------------------------------------------+

\f
                               netperf.Throughput_tps

   1140 *+------------------------------------------------------------------+
        |     ..                             .*...                 .*.......|
   1135 ++      *......    ....*.....     ...     ..           ....         *
        |              *...          .  ..          .    ....*.             |
   1130 ++                            *.             *...                   |
        |                                                                   |
   1125 ++                                                                  |
        |                                     O                             |
   1120 ++                                                                  O
        |                                                                   |
   1115 O+             O              O                      O      O       |
        |                                            O                      |
   1110 ++                                                                  |
        |                      O                                            |
   1105 ++------O-----------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  166.80        -0.8%       165.40  lkp-a04/micro/netperf/120s-200%-TCP_CRR
                  902.00        -1.8%       885.50  lkp-a04/micro/netperf/120s-200%-TCP_RR
                 1134.40        -1.7%      1114.60  lkp-a04/micro/netperf/120s-200%-UDP_RR
                 2203.20        -1.7%      2165.50  TOTAL netperf.Throughput_tps
\f
                       slabinfo.skbuff_fclone_cache.num_objs

   650 ++-------------------------------------------------------------------+
   600 ++                                                          .*.....  |
       *.......*......*.......*.......*......*.......          ....       ..|
   550 ++                                            *.......*.             *
   500 ++                                                                   |
       |                                                                    |
   450 ++                                                                   |
   400 ++                                                                   |
   350 ++                                                                   |
       |                                                                    |
   300 ++                                                                   |
   250 ++                     O                                             |
       |                                                                    |
   200 ++             O                                      O      O       O
   150 O+------O----------------------O------O-------O----------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  569.20       -68.0%       181.94  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                  569.20       -68.0%       181.94  TOTAL slabinfo.skbuff_fclone_cache.num_objs
\f
                     slabinfo.skbuff_fclone_cache.active_objs

   600 ++-------------------------------------------------------------------+
       |                  ....*.......                           ...*.......|
   550 *+......*......*...            *......*.......    ....*...           *
   500 ++                                            *...                   |
       |                                                                    |
   450 ++                                                                   |
   400 ++                                                                   |
       |                                                                    |
   350 ++                                                                   |
   300 ++                                                                   |
       |                      O                                             |
   250 ++                                                                   |
   200 ++                                                                   |
       |              O                                      O      O       O
   150 O+------O----------------------O------O-------O----------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  542.84       -66.5%       181.94  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                  542.84       -66.5%       181.94  TOTAL slabinfo.skbuff_fclone_cache.active_objs
\f
                       sched_debug.cpu#2.nr_uninterruptible

   600 ++-------------------------------------------------------------------+
       |                                            .*.                     *
   500 ++                                        ...   ..                 ..|
   400 ++  ....*......              ..*......  ..        .               .  |
       *...           *.......  ....         *.           .             .   |
   300 ++                     *.                     O     ..         ..    |
       |       O                                                     .      |
   200 ++                                                    *......*       |
       |                              O                      O      O       |
   100 O+                                    O                              |
     0 ++             O       O                                             |
       |                                                                    |
  -100 ++                                                                   |
       |                                                                    O
  -200 ++-------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  376.92       -69.2%       116.27  nhm-white/micro/aim7/misc_rtns_1
                  376.92       -69.2%       116.27  TOTAL sched_debug.cpu#2.nr_uninterruptible
\f
                       sched_debug.cfs_rq[7]:/.nr_spread_over

   1200 ++------------------------------------------------------------------+
   1100 O+                     O                                            |
        |                                                                   |
   1000 ++                            O       *                             |
    900 ++                                   .O+                    O       O
    800 ++             O                    .   +    O                      |
    700 ++                                ..     +                          |
        |       *......                  .        +                         |
    600 ++    ..       *...             .          +                        |
    500 ++  ..  O          ...         .            +                       |
    400 ++ .                  .    ...*              *...               ....*
    300 ++.                    *...                      ... O     .*...    |
        *                                                   .  ....         |
    200 ++                                                   *.             |
    100 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  441.02       +89.6%       836.36  nhm-white/micro/aim7/misc_rtns_1
                  441.02       +89.6%       836.36  TOTAL sched_debug.cfs_rq[7]:/.nr_spread_over
\f
                                 interrupts.CPU3.RES

   13000 ++-----------------------------------------------------------------+
   12000 ++                                       ....*......               |
         *......*.......*......*.......*......*...           *.......*......*
   11000 ++                                                                 |
   10000 ++                                                                 |
         |                                                                  |
    9000 ++                                                                 |
    8000 ++                                                                 |
    7000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
    5000 O+                                                                 |
         |                     O                                            |
    4000 ++     O       O              O      O       O      O       O      O
    3000 ++-----------------------------------------------------------------+

\f
                                 interrupts.CPU3.RES

   11000 ++-----------------------------------------------------------------+
         |                    .*.....                       .*.......       |
   10000 *+.....*.......  ....       ..    ...*.....      ..         *..... |
    9000 ++             *.             *...         ..  ..                 .|
         |                                            *.                    *
    8000 ++                                                                 |
         |                                                                  |
    7000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
    5000 ++     O                                                    O      |
         |                                                                  |
    4000 O+                            O                     O              |
         |              O      O              O       O                     O
    3000 ++-----------------------------------------------------------------+

\f
                                 interrupts.CPU3.RES

   11000 ++-----------------------------------------------------------------*
         *......*.......*......*.......*......*.......           ....*...   |
   10000 ++                                           *......*...           |
    9000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
         |                                                                  |
    7000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
    5000 O+                                           O                     |
         |                                                                  |
    4000 ++     O              O              O              O       O      O
         |              O              O                                    |
    3000 ++-----------------------------------------------------------------+

\f
                                 interrupts.CPU3.RES

   31500 ++-----*-----------------------------------------------------------+
   31000 ++....         *.....                                              |
         *.                   .                           ...*.....        .*
   30500 ++                    *.......*......*.......*...         ..  .... |
   30000 ++                                                          *.     |
         |                                                                  |
   29500 ++                                                                 |
   29000 ++                                                                 |
   28500 ++                                                                 |
         O                                                                  |
   28000 ++             O      O                                            |
   27500 ++                                           O      O       O      |
         |                             O      O                             |
   27000 ++                                                                 O
   26500 ++-----O-----------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                11640.97       -66.7%      3878.02  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                 9704.37       -58.4%      4038.38  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                10455.61       -58.8%      4310.34  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                30664.00       -10.5%     27449.58  nhm-white/micro/aim7/misc_rtns_1
                62464.94       -36.5%     39676.33  TOTAL interrupts.CPU3.RES
\f
                                 interrupts.CPU1.RES

   16000 ++-----------------------------------------------------------------+
         |                                                                  |
   15000 ++                    O                                            O
         |                                    O                             |
   14000 ++                                                                 |
         |      O                                                    O      |
   13000 O+                                           O                     |
         *..                       ....*.....                O              |
   12000 ++ ..          O      *...    O     .              .*.....        .*
         |    ..             ..               *.......  ....       ..  .... |
   11000 ++                ..                         *.             *.     |
         |      *....... ..                                                 |
   10000 ++             *                                                   |
         |                                                                  |
    9000 ++-----------------------------------------------------------------+

\f
                                 interrupts.CPU1.RES

   11000 ++-----------------------------------------------------------------+
         *......    ....*......                           ...*.......    ...*
   10000 ++     *...           *.......    ...*.......*...           *...   |
    9000 ++                            *...                                 |
         |                                                                  |
    8000 ++                                                                 |
    7000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
    5000 ++                                                  O              |
         |                             O      O                             |
    4000 ++             O                             O              O      |
    3000 ++                                                                 O
         O      O                                                           |
    2000 ++--------------------O--------------------------------------------+

\f
                                 interrupts.CPU1.RES

   14000 ++-----------------------------------------------------------------+
         |     .*.......    ...*...                                         |
   12000 ++....         *...       ...                                      |
         *.                           .    ...*.......*......*.......    ...*
         |                             *...                          *...   |
   10000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
         |                                                                  O
         |              O      O       O                                    |
    4000 ++     O                             O                             |
         O                                            O      O       O      |
    2000 ++-----------------------------------------------------------------+

\f
                                 interrupts.CPU1.RES

   11000 ++-----------------------------------------------------------------+
         *......*.......                                                 ...*
   10000 ++             *......*.......*......*.......*......*.......*...   |
    9000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
    7000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
    5000 ++                    O                                            |
         |      O       O                                                   |
    4000 ++                            O              O              O      O
    3000 O+                                   O                             |
         |                                                   O              |
    2000 ++-----------------------------------------------------------------+

\f
                                 interrupts.CPU1.RES

   12000 ++-----------------------------------*-----------------------------+
   11000 ++                        ....*...           *......*.......*......|
         *......*.......*......*...                                         *
   10000 ++                                                                 |
    9000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
    7000 ++                                                                 |
    6000 ++                                                                 |
         |                                                                  |
    5000 ++     O                             O                             |
    4000 O+             O      O       O              O                     |
         |                                                                  O
    3000 ++                                                  O       O      |
    2000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                11521.24       +15.8%     13335.96  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                 9871.39       -61.5%      3799.49  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                11620.73       -67.5%      3781.70  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                 9895.82       -60.8%      3883.62  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                11033.15       -64.6%      3900.39  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                53942.34       -46.8%     28701.16  TOTAL interrupts.CPU1.RES
\f
                       sched_debug.cpu#5.nr_uninterruptible

   450 ++-------------------------------------------------------------------+
   400 ++                             O                                     |
       |                                     O                              |
   350 ++                                                                   |
   300 ++                                            O                      O
       |                                                                    |
   250 ++                                                          .*.      |
   200 ++      O      O       O      .*...                     ....   ..    |
   150 ++                         ...     ..                .*.         .   |
       O                        ..          .             .. O           .  |
   100 ++                    .*.             *.....     ..                ..|
    50 ++  ....*.....     ...                      .. ..                    |
       *...          .  ..                           *              O       *
     0 ++             *.                                                    |
   -50 ++-------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                   85.95      +164.3%       227.20  nhm-white/micro/aim7/misc_rtns_1
                   85.95      +164.3%       227.20  TOTAL sched_debug.cpu#5.nr_uninterruptible
\f
                                 interrupts.CPU2.RES

   13500 ++-----------------------------------------------------------------+
         |                     O              O                             |
   13000 ++                                                                 |
   12500 ++                                                                 |
         |      O                                                           O
   12000 ++                                                          O      |
   11500 *+.                                                                |
         O  ..                 *.....                                       |
   11000 ++   ..              +      ..                      *...           |
   10500 ++             O    +         *......             ..    ...       .*
         |      *..         +                 *...        .         .  .... |
   10000 ++        ..      +                      ... O  .           *.     |
    9500 ++          ..   +                          . ..                   |
         |             . +                            *                     |
    9000 ++-------------*---------------------------------------------------+

\f
                                 interrupts.CPU2.RES

   13000 ++----------------------------------------------------------*------+
   12000 *+.....                                               ....        .|
         |      *.....             ....*......*.......*......*.             *
   11000 ++           ..    ...*...                                         |
   10000 ++             *...                                                |
         |                                                                  |
    9000 ++                                                                 |
    8000 ++                                                                 |
    7000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
    5000 ++             O                                                   |
         |                     O                             O       O      |
    4000 O+     O                      O      O       O                     O
    3000 ++-----------------------------------------------------------------+

\f
                                 interrupts.CPU2.RES

   11000 ++-----------------------------------------------------------------+
         *......*.......                     .*.......    ...*.......       |
   10000 ++             *......*.......  ....         *...           *......*
    9000 ++                            *.                                   |
         |                                                                  |
    8000 ++                                                                 |
    7000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
    5000 ++                                                                 |
         |                                                                  O
    4000 O+     O              O              O              O       O      |
    3000 ++                            O                                    |
         |              O                             O                     |
    2000 ++-----------------------------------------------------------------+

\f
                                 interrupts.CPU2.RES

   13000 ++-----------------------------------------------------------------+
   12000 ++                                         ..*...          .*..... |
         |                    .*.....           ....      ..     ...       .|
   11000 *+.....*.......  ....       ..    ...*.            .  ..           *
   10000 ++             *.             *...                  *.             |
    9000 ++                                                                 |
    8000 ++                                                                 |
         |                                                                  |
    7000 ++                                                                 |
    6000 ++                                                                 |
    5000 ++     O                                                           O
    4000 ++             O      O                                            |
         |                             O      O              O       O      |
    3000 O+                                           O                     |
    2000 ++-----------------------------------------------------------------+

\f
                                 interrupts.CPU2.RES

   34000 ++-----------------------------------------------------------------+
         |              *..                                                 |
   33000 *+.          ..   ..                                               |
   32000 ++ ..      ..       ..                                     .*..... |
         |    ..   .                                             ...       .|
   31000 ++      ..            *.......    ...*.......         ..           *
   30000 ++     *                      *...           *......*.             |
         |                                                                  |
   29000 ++                                                                 |
   28000 ++                                                                 |
         O      O                                                           |
   27000 ++                    O                             O       O      |
   26000 ++             O              O      O                             |
         |                                            O                     O
   25000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                10305.83       +12.4%     11587.75  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                11522.57       -64.0%      4142.69  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                 9968.00       -63.8%      3606.15  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                10840.78       -64.2%      3877.98  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                31127.69       -14.5%     26605.38  nhm-white/micro/aim7/misc_rtns_1
                73764.87       -32.5%     49819.94  TOTAL interrupts.CPU2.RES
\f
                           sched_debug.cfs_rq[7]:/.spread0

   40000 ++-----------------------------------------------------------------+
         |                     O                                            |
   35000 ++                                                                 |
   30000 ++                                                                 |
         |                                                                  |
   25000 ++     O                                                           |
         |                                                                  |
   20000 ++                                                                 |
         O                             O                                    |
   15000 ++             O                    .*...                   O      |
   10000 ++    .*.......                 .... O   ... O                     O
         | ....         *......*.......*.            .       O              |
    5000 *+                                           *......*.......*......*
         |                                                                  |
       0 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 7698.32      +118.7%     16836.46  nhm-white/micro/aim7/misc_rtns_1
                 7698.32      +118.7%     16836.46  TOTAL sched_debug.cfs_rq[7]:/.spread0
\f
                                 interrupts.CPU0.RES

   18000 ++-----------------------------------------------------------------+
         |                                                                  O
   17000 ++                                                                 |
   16000 ++                    O              O                             |
         |                                                           O      |
   15000 O+     O                                                           |
         |                                                                  |
   14000 ++             O                             O      O              |
         |                                                                  |
   13000 *+.                  .*.......*.....                              .*
   12000 ++ ..            ....               .              .*.....      .. |
         |    ..      ..*.                    *.....      ..       ..  ..   |
   11000 ++       ....                              ..  ..           *.     |
         |      *.                                    *.                    |
   10000 ++-----------------------------------------------------------------+

\f
                                 interrupts.CPU0.RES

   12000 ++-----------------------------------------------------------------+
         |     .*...                                ..*......               |
   11000 ++....     ...       .*.......*......  ....         *.......       |
   10000 *+            .  ....                *.                     *......|
         |              *.                                                  *
    9000 ++                                                                 |
    8000 ++                                                                 |
         |                                                                  |
    7000 ++                                                                 |
    6000 ++                    O                                            |
         O      O                                                    O      |
    5000 ++                                                                 O
    4000 ++                                                                 |
         |              O              O              O                     |
    3000 ++-----------------------------------O--------------O--------------+

\f
                                 interrupts.CPU0.RES

   16000 ++-----------------------------------------------------------------+
         |                                                                  |
   14000 ++           ..*......                                             |
         |        ....         *.......*......            ...*.......    ...*
   12000 *+.....*.                            *.......*...           *...   |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
         |                                                   O              |
    6000 ++                                           O              O      |
         O      O                             O                             |
    4000 ++             O      O       O                                    O
         |                                                                  |
    2000 ++-----------------------------------------------------------------+

\f
                                 interrupts.CPU0.RES

   13000 ++-----------------------------------------------------------------+
   12000 ++                                                        ..*......|
         |                  ...*.......                        ....         *
   11000 ++  ...*.......*...           *......*.......*......*.             |
   10000 *+..                                                               |
         |                                                                  |
    9000 ++                                                                 |
    8000 ++                                                                 |
    7000 ++                                                                 |
         |                                    O       O      O              |
    6000 O+                                                                 |
    5000 ++             O                                                   |
         |                     O       O                                    O
    4000 ++     O                                                    O      |
    3000 ++-----------------------------------------------------------------+

\f
                                 interrupts.CPU0.RES

   14000 ++-----------------------------------------------------------------+
   13000 ++                                                                .*
         *......            ...*.......    ...*.......    ...*.......  .... |
   12000 ++     *.......*...           *...           *...           *.     |
   11000 ++                                                                 |
   10000 ++                                                                 |
    9000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
    7000 ++                                                                 |
    6000 ++                                                  O       O      |
    5000 O+             O              O              O                     |
         |                                                                  |
    4000 ++     O              O              O                             O
    3000 ++-----------------------------------------------------------------+

\f
                                 interrupts.CPU0.RES

   33000 ++-----------------------------------------------------------------+
         |            ..*.....                                              |
   32000 *+.....  ....        .                                             |
   31000 ++     *.             *.......*......                           ...*
         |                                    *.......*......*.......*...   |
   30000 ++                                                                 |
         |                                                                  |
   29000 ++                                                                 |
         |                                                   O              |
   28000 ++     O                                                           |
   27000 O+             O                             O              O      |
         |                                                                  |
   26000 ++                    O       O      O                             |
         |                                                                  O
   25000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                11923.66       +27.3%     15174.21  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                10412.42       -57.1%      4470.51  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                12732.32       -61.5%      4907.33  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                11063.89       -54.2%      5063.58  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                12185.71       -59.1%      4978.52  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                31283.90       -13.9%     26950.34  nhm-white/micro/aim7/misc_rtns_1
                89601.90       -31.3%     61544.49  TOTAL interrupts.CPU0.RES
\f
                                   interrupts.RES

   25000 ++-----------------------------------O-----------------------------O
         |                                                   O              |
   24000 O+     O              O                                            |
         |                             O                             O      |
   23000 ++                                                                 |
         |              O                             O                     |
   22000 ++                                                                 |
         |                                                                  |
   21000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         |                                   .*.......*......               |
   19000 ++         ....*......          ....                *.......*......|
         *......*...           *.......*.                                   *
   18000 ++-----------------------------------------------------------------+

\f
                                   interrupts.RES

   58000 ++-----------------------------------------------------------------+
   56000 ++                    O              O                             O
         |                                                                  |
   54000 ++                                                                 |
   52000 ++     O                                                    O      |
         O                                                                  |
   50000 ++                                                                 |
   48000 *+.                                                 O              |
   46000 ++ .           O      *.......*.....         O      *..            |
         |   .               ..        O     .             ..   ..         .*
   44000 ++   ..            .                 *...        .       ..     .. |
   42000 ++                .                      ...    .          .  ..   |
         |      *....... ..                          . ..            *.     |
   40000 ++             *                             *                     |
   38000 ++-----------------------------------------------------------------+

\f
                                   interrupts.RES

   45000 ++-----------------------------------------------------------------+
         |                                                                  |
   40000 *+.....                                  ....*......            ...*
         |      *.......*......*.......*......*...           *.......*...   |
         |                                                                  |
   35000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
         |                                                                  |
   25000 ++                                                                 |
         |                                                                  |
         |                                                                  |
   20000 ++                                                                 |
         O      O       O      O       O      O       O      O       O      O
   15000 ++-----------------------------------------------------------------+

\f
                                   interrupts.RES

   20500 ++-----------------------------------------------------------------+
         |                                                                  |
   20000 *+....                                                             |
         |     .               *.......                                     |
   19500 ++     *...         ..        *...          .*......               |
         |          ...    ..              ..     ...        *.......    ...*
   19000 ++            . ..                  .  ..                   *...   |
         |              *                     *.                            |
   18500 ++                                                                 |
         |                                                                  |
   18000 ++                                   O                             |
         |                                                                  |
   17500 ++     O                      O                     O              O
         O              O                             O                     |
   17000 ++--------------------O-------------------------------------O------+

\f
                                   interrupts.RES

   50000 ++--------------------*--------------------------------------------+
         *......*.......*...           *......*.......*......*.......*......*
   45000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
         |                                                                  |
   35000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
         |                                                                  |
   25000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         O      O       O      O       O      O              O       O      O
   15000 ++-------------------------------------------O---------------------+

\f
                                   interrupts.RES

   45000 ++-----------------------------------------------------------------+
         |                                                                  |
   40000 *+.....*.......*......*.......    ...*.......    ...*.......*......*
         |                             *...           *...                  |
         |                                                                  |
   35000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
         |                                                                  |
   25000 ++                                                                 |
         |                                                                  |
         |                                                                  |
   20000 ++                                                                 |
         O      O       O      O       O      O       O      O       O      O
   15000 ++-----------------------------------------------------------------+

\f
                                   interrupts.RES

   20500 ++-----------------------------------------------------------------+
         |            ..*......    ....*..                                  |
   20000 ++       ....         *...       ..         .*......    ....*..... |
   19500 ++    .*.                          ..    ...        *...          .|
         | ....                                 ..                          *
   19000 *+                                   *.                            |
         |                                                                  |
   18500 ++                                                                 |
         |                                                                  |
   18000 O+     O                                                           |
   17500 ++                                   O       O              O      O
         |              O              O                     O              |
   17000 ++                                                                 |
         |                     O                                            |
   16500 ++-----------------------------------------------------------------+

\f
                                   interrupts.RES

   50000 ++-----------------------------------------------------------------+
         |                                                                  |
   45000 *+.....            ...*.......*......*.......*......    ....*......*
         |      *.......*...                                 *...           |
   40000 ++                                                                 |
         |                                                                  |
   35000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
         |                                                                  |
   25000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         O      O       O      O       O      O       O      O       O      O
   15000 ++-----------------------------------------------------------------+

\f
                                    interrupts.RES

   230000 ++-------------*--------------------------------------------------+
   225000 *+.....  ....      ..                                             |
          |      *.            .    ...*.......           ...*.......       |
   220000 ++                    *...           *......*...           *......*
   215000 ++                                                                |
          |                                                                 |
   210000 ++                                                                |
   205000 ++                                                                |
   200000 ++                                                                |
          |      O                                                          |
   195000 O+             O                                   O       O      |
   190000 ++                    O      O              O                     |
          |                                    O                            |
   185000 ++                                                                O
   180000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                18914.24       +25.8%     23795.18  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                43735.54       +16.4%     50924.41  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                39314.43       -57.8%     16574.48  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                19312.73       -10.0%     17382.42  lkp-a04/micro/netperf/120s-200%-TCP_RR
                47516.60       -64.8%     16709.74  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                40632.07       -59.2%     16591.72  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                19733.15       -11.2%     17530.44  lkp-a04/micro/netperf/120s-200%-UDP_RR
                44515.25       -61.7%     17067.23  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               222616.08       -13.9%    191674.54  nhm-white/micro/aim7/misc_rtns_1
               496290.10       -25.8%    368250.18  TOTAL interrupts.RES
\f
                        sched_debug.cfs_rq[7]:/.min_vruntime

   45000 ++-----------------------------------------------------------------+
         |                     O                                            |
   40000 ++                                                                 |
   35000 ++                                                                 |
         |                                                                  |
   30000 ++     O                                                           |
         |                                                                  |
   25000 ++                                                                 |
         |                                                                  |
   20000 O+                            O                                    |
   15000 ++             O                    .*.....                 O      O
         |     .*.......*......          ....       ..O                     |
   10000 ++....                *.......*.             *......O           ...*
         *.                                                  *.......*...   |
    5000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                11166.54       +82.7%     20397.97  nhm-white/micro/aim7/misc_rtns_1
                11166.54       +82.7%     20397.97  TOTAL sched_debug.cfs_rq[7]:/.min_vruntime
\f
                                 softirqs.CPU0.NET_RX

   170000 ++----------------------------------------------------------------+
   160000 ++                 ...*......                     .*.......*......*
          *......*.......*...          *.......*......  ....                |
   150000 ++                                          *.                    |
   140000 ++                                                                |
          |                                                                 |
   130000 ++             O      O                     O              O      |
   120000 ++                                                                |
   110000 ++                                                                |
          O                                                                 |
   100000 ++     O                                                          |
    90000 ++                                                                O
          |                            O                                    |
    80000 ++                                   O             O              |
    70000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               158066.00       -34.2%    103970.26  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               158066.00       -34.2%    103970.26  TOTAL softirqs.CPU0.NET_RX
\f
                                    softirqs.NET_RX

   650000 ++--------------------*-------------------------------------------*
          *......*.......*...          *.......*......    ...*.......*...   |
   600000 ++                                          *...                  |
          |                                                                 |
          |                                                                 |
   550000 ++                                          O                     |
          |                                                                 |
   500000 ++                                                                |
          |                     O                                           |
   450000 ++                                   O                            O
          O                                                                 |
          |              O             O                     O       O      |
   400000 ++     O                                                          |
          |                                                                 |
   350000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               630988.15       -29.8%    442956.09  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               630988.15       -29.8%    442956.09  TOTAL softirqs.NET_RX
\f
                                 softirqs.CPU2.NET_RX

   170000 ++----------------------------------------------------------------+
          |                                                                 |
   160000 ++  ...*.......    ...*......*.......*......    ...*.......*......*
   150000 *+..           *...                         *...                  |
          |                                                                 |
   140000 ++                                                                |
          |                                                                 |
   130000 ++                           O                                    |
          |                                           O      O              |
   120000 ++                                   O                            |
   110000 ++                    O                                           |
          |                                                                 |
   100000 O+     O       O                                           O      |
          |                                                                 |
    90000 ++----------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               156872.34       -29.2%    110987.49  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               156872.34       -29.2%    110987.49  TOTAL softirqs.CPU2.NET_RX
\f
                                 softirqs.CPU3.NET_RX

   170000 ++----------------------------------------------------------------+
   160000 *+.....*.......    ...*......                     .*.......    ...*
          |              *...          *.......*......  ....         *...   |
   150000 ++                                          *.                    |
   140000 ++                                                                |
          |                                           O                     O
   130000 O+                                   O                            |
   120000 ++                                                                |
   110000 ++             O      O                                           |
          |                                                  O              |
   100000 ++                                                                |
    90000 ++                                                         O      |
          |                            O                                    |
    80000 ++     O                                                          |
    70000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               157622.69       -30.0%    110326.06  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               157622.69       -30.0%    110326.06  TOTAL softirqs.CPU3.NET_RX
\f
                            sched_debug.cpu#2.ttwu_local

   13000 ++-----------------------------------------------------------------+
         |                                    O                             |
   12000 ++                                                                 |
   11000 ++                                                                 |
         *..                                                                |
   10000 ++ .                                                               |
    9000 ++  .         .*......*.....                                       |
         |    ..    ...              ..                             .*...   |
    8000 ++       ..                   *.....                     ..     .. |
    7000 ++     *.                           .        O         ..         .|
         |                             O      *.......*......O..            *
    6000 ++                                                  *       O      |
    5000 O+     O       O      O                                            |
         |                                                                  O
    4000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 7690.77       -15.2%      6523.75  nhm-white/micro/aim7/misc_rtns_1
                 7690.77       -15.2%      6523.75  TOTAL sched_debug.cpu#2.ttwu_local
\f
                           sched_debug.cpu#1.sched_goidle

   40000 ++-----------------------------------------------------------------+
         |                                                                  |
   38000 ++                            *              *......*.             *
   36000 ++     *..                   + +            .         .           +|
         |     .   .                 +   +          .           .         + |
   34000 ++   .     ..              +     +       ..             ..      +  |
         |  ..  O     ..           +       +     .                 .    +   |
   32000 ++.                      +         +   .                   .O +    |
         |.             *.....   +     O     + .      O               +     |
   30000 *+                   . +             *                      *      |
   28000 ++                    *                                            |
         |                                                                  |
   26000 O+             O                     O                             |
         |                     O                             O              O
   24000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                33587.66       -16.5%     28048.84  nhm-white/micro/aim7/misc_rtns_1
                33587.66       -16.5%     28048.84  TOTAL sched_debug.cpu#1.sched_goidle
\f
                                 softirqs.CPU1.NET_RX

   170000 ++----------------------------------------------------------------+
          |                  ...*......    ....*......                      |
   160000 *+.....*.......*...          *...           *......*.......*......*
   150000 ++                                                                |
          |                                                                 O
   140000 ++                                                                |
   130000 ++                                   O                            |
          |                                                                 |
   120000 ++                    O                            O              |
   110000 ++     O                     O                                    |
          |                                                                 |
   100000 O+                                                         O      |
    90000 ++                                                                |
          |                                                                 |
    80000 ++-------------O--------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               158427.11       -25.7%    117672.28  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               158427.11       -25.7%    117672.28  TOTAL softirqs.CPU1.NET_RX
\f
                            sched_debug.cpu#3.ttwu_local

   22000 ++-----------------------------------------------------------------+
         |                                                           O      |
   20000 ++                                                                 |
   18000 ++                                                                 |
         |                                                                  |
   16000 ++                    *.                                           |
   14000 ++    .*..           .  .                                          |
         |   ..    .         .    .                                         |
   12000 ++..       ..     ..      ..                                       |
   10000 *+           ..  .          .                                      |
         |               .            .                                     |
    8000 ++             *                                 ...*.......       |
    6000 ++     O                      *......*.......*...           *......*
         O              O      O       O      O       O      O              O
    4000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 8975.45       -21.6%      7038.36  nhm-white/micro/aim7/misc_rtns_1
                 8975.45       -21.6%      7038.36  TOTAL sched_debug.cpu#3.ttwu_local
\f
                                  softirqs.CPU3.RCU

   24000 ++-----------------------------------------------------------------+
   23000 ++  ...*..            *.                                           |
         *...      .         ..  ..                                         |
   22000 ++         ..      .      .                        .*..     O      |
   21000 ++           ..   .        .                   ....    ..         .*
         |               ..          ..      .*.......*.          ..     .. |
   20000 ++             *                ....                       .  ..   |
   19000 ++                            *.                            *.     |
   18000 ++                                                                 |
         O      O                                     O                     |
   17000 ++                                                                 |
   16000 ++             O                                    O              |
         |                                                                  |
   15000 ++                    O       O      O                             O
   14000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                21103.12       -21.6%     16553.45  nhm-white/micro/aim7/misc_rtns_1
                21103.12       -21.6%     16553.45  TOTAL softirqs.CPU3.RCU
\f
                                  softirqs.CPU2.RCU

   24000 ++-----------------------------------------------------------------+
         |     .*...                                                        |
   23000 ++....     ...                             ..*..                   |
   22000 *+            .                        ....     ..                 |
         |              *......            ...*.           ..              .*
   21000 ++                    *.......*...                            .... |
   20000 ++                                                  *.......*.     |
         |                                            O                     |
   19000 ++                                                                 |
   18000 ++                                                                 |
         |                                    O                      O      |
   17000 O+     O       O                                                   |
   16000 ++                                                  O              O
         |                     O       O                                    |
   15000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                21376.43       -20.8%     16934.73  nhm-white/micro/aim7/misc_rtns_1
                21376.43       -20.8%     16934.73  TOTAL softirqs.CPU2.RCU
\f
                          slabinfo.kmalloc-2048.num_objs

   900 ++-------------------------------------------------------------------+
       |                                                                    |
   850 *+....                       ..*......*.....              ...*.......|
       |     ..    ...*.......  ....               ..    ....*...           *
       |       *...           *.                     *...                   |
   800 ++                                                                   |
       |                                                                    |
   750 ++                     O                                             |
       |                                                                    |
   700 ++                                                                   |
       |                                                                    |
       |              O                      O       O                      O
   650 ++      O                      O                             O       |
       O                                                     O              |
   600 ++-------------------------------------------------------------------+

\f
                           slabinfo.kmalloc-2048.num_objs

   1050 ++------------------------------------------------------------------+
        |       O                                                           |
        |              O                                                    |
   1000 ++                            O                      O              |
        O                      O                                    O       |
        |                                     O                             |
    950 ++                                                                  |
        |                                                                   |
    900 ++                                                                  |
        |                                            O                      O
        |                                                                   |
    850 ++            .*.....                       .*...                   |
        *.....      ..       ..    ...*.......  ....     ...                |
        |     ..  ..           *...           *.            .           ....*
    800 ++------*--------------------------------------------*------*-------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  839.33       -21.3%       660.74  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                  824.09       +17.8%       970.94  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                 1663.42        -1.9%      1631.68  TOTAL slabinfo.kmalloc-2048.num_objs
\f
                         slabinfo.kmalloc-2048.active_objs

   820 ++-------------------------------------------------------------------+
   800 ++                           ..*......*..                   .*.....  |
       *.....        .*.......  ....            .              ....       ..|
   780 ++    ..  ....         *.                 ..         .*.             *
   760 ++      *.                                  ..    ...                |
       |                                               ..                   |
   740 ++                     O                      *.                     |
   720 ++                                                                   |
   700 ++                                                                   |
       |                                                                    |
   680 ++                                                                   |
   660 ++             O                      O       O                      O
       |       O                      O                             O       |
   640 O+                                                    O              |
   620 ++-------------------------------------------------------------------+

\f
                          slabinfo.kmalloc-2048.active_objs

   1050 ++------------------------------------------------------------------+
        |       O                                                           |
   1000 ++             O                                                    |
        |                             O                      O      O       |
    950 O+                     O              O                             |
        |                                                                   |
    900 ++                                                                  |
        |                                            O                      O
    850 ++                                                                  |
        |                                           .*...                   |
    800 *+....        .*.....                     ..     ...                |
        |     ..  ....       ..    ...*.......  ..          .           ....*
    750 ++      *.             *...           *.             *......*...    |
        |                                                                   |
    700 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  784.38       -16.0%       658.58  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                  770.56       +23.9%       955.06  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                 1554.94        +3.8%      1613.65  TOTAL slabinfo.kmalloc-2048.active_objs
\f
                              proc-vmstat.pgalloc_normal

   4.5e+06 ++---------------------------------------------------------------+
   4.4e+06 ++     O                                                         |
           O                            O                     O      O      |
   4.3e+06 ++            O                                                  |
   4.2e+06 ++                    O             O                            |
   4.1e+06 ++                                                               O
     4e+06 ++                                                               |
           |                                                                |
   3.9e+06 ++                                         O                     |
   3.8e+06 ++                                                               |
   3.7e+06 ++                                                               |
   3.6e+06 ++                                        .*.....                |
           *......    ...*.......                ....       ..              |
   3.5e+06 ++     *...           *......*......*.             *......*......*
   3.4e+06 ++---------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              3526131.15       +20.5%   4248631.25  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              3526131.15       +20.5%   4248631.25  TOTAL proc-vmstat.pgalloc_normal
\f
                            slabinfo.kmalloc-64.num_objs

   10000 ++-----------------------------------------------------------------+
         |      O                                     O                     |
    9500 O+             O      O                             O       O      |
         |                                                                  |
         |                                    O                             |
    9000 ++                            O                                    |
         |                                                                  O
    8500 ++                                                                 |
         *......                                                            |
    8000 ++     *...                                        .*...           |
         |          ...              ..*......          ....     ...       .*
         |             .         ....         *.......*.            .  .... |
    7500 ++             *......*.                                    *.     |
         |                                                                  |
    7000 ++-----------------------------------------------------------------+

\f
                            slabinfo.kmalloc-64.num_objs

   11000 ++-----------------------------------------------------------------+
         |                                    O              O              |
   10500 ++                                                                 |
         |                                                                  O
   10000 ++                                                                 |
         O              O              O              O                     |
    9500 ++                    O                                            |
         |                                                                  |
    9000 ++     O                                                    O      |
         |                           ..*...          .*...                  |
    8500 *+..                    ....      ..     ...     ..                |
         |   ..       ..*......*.            .  ..          .    ....*......*
    8000 ++    .  ....                        *.             *...           |
         |      *.                                                          |
    7500 ++-----------------------------------------------------------------+

\f
                            slabinfo.kmalloc-64.num_objs

   11500 ++-----------------------------------------------------------------+
         |                     O                                            |
   11000 ++                            O              O                     |
   10500 ++                                                                 |
         |                                                           O      |
   10000 ++                                                  O              |
         |                                                                  O
    9500 ++     O       O                     O                             |
         O                                                                  |
    9000 ++                                                                 |
    8500 *+.....                                                            |
         |      *.....             ....*......              .*.......       |
    8000 ++           ..    ...*...           *.......  ....         *......|
         |              *...                          *.                    *
    7500 ++-----------------------------------------------------------------+

\f
                            slabinfo.kmalloc-64.num_objs

   11000 ++-----------------------------------------------------------------+
         |                                                           O      |
   10500 ++                                   O                             |
         |                                                                  |
   10000 ++                            O                                    |
         O      O       O                                                   |
    9500 ++                                                  O              O
         |                     O                                            |
    9000 ++                                           O                     |
         |                                                                  |
    8500 ++                   .*.....                              ..*......|
         *......*.......  ....       ..                        ....         *
    8000 ++             *.             *......            ...*.             |
         |                                    *.......*...                  |
    7500 ++-----------------------------------------------------------------+

\f
                            slabinfo.kmalloc-64.num_objs

   11000 ++-----------------------------------------------------------------+
         |                                                                  |
   10500 ++                                                  O              |
         |      O                                     O                     |
   10000 ++                    O                                            |
         |              O                     O                             |
    9500 O+                                                          O      O
         |                                                                  |
    9000 ++                            O                                    |
         |                                                                  |
    8500 ++                                ...*.......                      |
         |   ...*.......*......    ....*...           *.....       ..*......*
    8000 *+..                  *...                         .  ....         |
         |                                                   *.             |
    7500 ++-----------------------------------------------------------------+

\f
                            slabinfo.kmalloc-64.num_objs

   11500 ++-----------------------------------------------------------------+
         |                             O                                    |
   11000 ++                                                                 |
   10500 ++                                                                 |
         |                                                   O              |
   10000 ++             O                                                   |
         |                                            O              O      |
    9500 O+                    O              O                             |
         |      O                                                           O
    9000 ++                                                                 |
    8500 ++                                                                 |
         *......*.......    ...*.......             ..*......*.......       |
    8000 ++             *...           *......  ....                 *......*
         |                                    *.                            |
    7500 ++-----------------------------------------------------------------+

\f
                            slabinfo.kmalloc-64.num_objs

   10500 ++-----------------------------------------------------------------+
         |                                                                  |
   10000 ++             O              O              O                     |
         O                                                           O      O
         |      O                                                           |
    9500 ++                    O                                            |
         |                                    O                             |
    9000 ++                                                  O              |
         |                                                                  |
    8500 ++                                                                 |
         |                                                ...*.....        .*
         *......*.......*......    ....*......*.......*...         ..  .... |
    8000 ++                    *...                                  *.     |
         |                                                                  |
    7500 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 7749.98       +21.8%      9441.07  lkp-a04/micro/netperf/120s-200%-TCP_CRR
                 8312.41       +18.4%      9845.58  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                 8156.12       +23.3%     10055.85  lkp-a04/micro/netperf/120s-200%-TCP_RR
                 8154.60       +20.3%      9809.85  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                 8167.66       +19.8%      9787.45  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                 8173.57       +19.8%      9794.01  lkp-a04/micro/netperf/120s-200%-UDP_RR
                 8170.26       +18.7%      9698.93  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                56884.60       +20.3%     68432.74  TOTAL slabinfo.kmalloc-64.num_objs
\f
                           slabinfo.kmalloc-64.active_objs

   10000 ++-----------------------------------------------------------------+
         |      O                                     O                     |
    9500 ++             O                                    O              |
         O                     O                                     O      |
         |                                                                  |
    9000 ++                            O                                    |
         |                                    O                             O
    8500 ++                                                                 |
         |                                                                  |
    8000 *+.....*...                                        .*...           |
         |          ...              ..*......          ....     ...       .*
         |             .         ....         *.......*.            .  .... |
    7500 ++             *......*.                                    *.     |
         |                                                                  |
    7000 ++-----------------------------------------------------------------+

\f
                           slabinfo.kmalloc-64.active_objs

   11000 ++-----------------------------------------------------------------+
         |                                                   O              |
   10500 ++                                                                 |
         |                                    O                             O
   10000 ++                                                                 |
         |                                                                  |
    9500 ++                                           O                     |
         O              O      O       O                                    |
    9000 ++     O                                                    O      |
         |                                                                  |
    8500 *+..                        ..*.....       ..*...                  |
         |   ..       ..*......  ....        .  ....      ..       ..*......*
    8000 ++    .  ....         *.             *.            .  ....         |
         |      *.                                           *.             |
    7500 ++-----------------------------------------------------------------+

\f
                           slabinfo.kmalloc-64.active_objs

   11000 ++-----------------------------------------------------------------+
         |                     O       O              O                     |
   10500 ++                                                                 |
         |                                                                  |
   10000 ++                                                          O      |
         |                                                   O              |
    9500 ++                                                                 O
         |      O       O                     O                             |
    9000 ++                                                                 |
         *.....                                                             |
    8500 ++    .                                                            |
         |      *.....             ....*......*.....        .*.......       |
    8000 ++           ..    ...*...                 ..  ....         *..... |
         |              *...                          *.                   .|
    7500 ++-----------------------------------------------------------------*

\f
                           slabinfo.kmalloc-64.active_objs

   10500 ++-----------------------------------------------------------------+
         |                                    O                      O      |
   10000 ++                                                                 |
         |                                                                  |
         O              O              O                                    |
    9500 ++     O                                                           |
         |                     O                             O              O
    9000 ++                                           O                     |
         |                                                                  |
    8500 ++                                                         .*...   |
         |                                                       ...     .. |
         *......*.......    ...*.......                        ..          .|
    8000 ++             *...           *.....             ...*.             *
         |                                   .    ....*...                  |
    7500 ++-----------------------------------*-----------------------------+

\f
                           slabinfo.kmalloc-64.active_objs

   10500 ++-----------------------------------------------------------------+
         |                                                                  |
   10000 ++     O                                     O      O              |
         |                     O                                            |
         O                                                                  |
    9500 ++             O                                                   O
         |                                    O                      O      |
    9000 ++                            O                                    |
         |                                                                  |
    8500 ++                                                                 |
         |                                                                  |
         |          ....*......    ....*......*.......*.....       ..*......*
    8000 *+.....*...           *...                         .  ....         |
         |                                                   *.             |
    7500 ++-----------------------------------------------------------------+

\f
                           slabinfo.kmalloc-64.active_objs

   11000 ++-----------------------------------------------------------------+
         |                             O                                    |
   10500 ++                                                                 |
         |                                                                  |
   10000 ++                                                  O              |
         |              O                     O                             |
    9500 ++                                           O              O      |
         O      O              O                                            |
    9000 ++                                                                 O
         |                                                                  |
    8500 ++                                                                 |
         |                                                ...*.......       |
    8000 *+.....*.......*......*.......*......    ....*...           *......*
         |                                    *...                          |
    7500 ++-----------------------------------------------------------------+

\f
                           slabinfo.kmalloc-64.active_objs

   10000 ++----------------------------O------------------------------------+
         |              O                             O                     |
         O                                                                  |
    9500 ++     O              O                                     O      O
         |                                                                  |
         |                                    O                             |
    9000 ++                                                                 |
         |                                                   O              |
    8500 ++                                                                 |
         |                                                                  |
         |                                                                 .*
    8000 *+.....                   ....*......*.......                 .... |
         |      *.......*......*...                   *......*.......*.     |
         |                                                                  |
    7500 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 7737.89       +20.0%      9286.10  lkp-a04/micro/netperf/120s-200%-TCP_CRR
                 8188.56       +17.5%      9618.42  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                 8045.23       +21.9%      9808.12  lkp-a04/micro/netperf/120s-200%-TCP_RR
                 7981.94       +19.8%      9565.98  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                 8045.71       +19.6%      9623.72  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                 8003.80       +20.2%      9622.53  lkp-a04/micro/netperf/120s-200%-UDP_RR
                 7948.34       +19.6%      9505.45  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                55951.48       +19.8%     67030.33  TOTAL slabinfo.kmalloc-64.active_objs
\f
                                 softirqs.CPU0.SCHED

   23000 ++-----------------------------------------------------------------+
         |                                                                  O
   22000 ++                                                                 |
   21000 O+                                                                 |
         |                     O                                            |
   20000 ++     O       O                                                   |
         |                                            O      O              |
   19000 ++                            *..    O                      O      |
         |                           ..   ..                .*...           |
   18000 ++           ..*..         .  O    ..          ....     ...        |
   17000 ++       ....     ..      .              ....*.            .       |
         |     .*.           ..  ..           *...                   *...   |
   16000 ++....                 .                                        .. |
         *.                    *                                           .|
   15000 ++-----------------------------------------------------------------*

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                16903.28       +18.2%     19976.14  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                16903.28       +18.2%     19976.14  TOTAL softirqs.CPU0.SCHED
\f
                                 interrupts.CPU6.RES

   26000 ++----------------------------*------------------------------------+
         |            ..*...        ..     ..                               |
   25000 *+.....  ....      ..    ..         .                              |
   24000 ++     *.            . ..            *.......           ....*......|
         |                     *                      *......*...           *
   23000 ++                                                                 |
         |                                                                  |
   22000 ++                                                  O              |
         |      O                                                           |
   21000 ++                            O                                    |
   20000 O+             O                             O              O      |
         |                     O                                            |
   19000 ++                                   O                             |
         |                                                                  O
   18000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                24396.71       -16.6%     20337.48  nhm-white/micro/aim7/misc_rtns_1
                24396.71       -16.6%     20337.48  TOTAL interrupts.CPU6.RES
\f
                                 interrupts.CPU5.RES

   26000 ++-----------------------------------------------------------------+
         |                    .*.....                                       |
   25000 *+.....*.......  ....       ..                                     |
         |              *.             *...          .*......*.....         |
   24000 ++                                ..     ...              ..    ...*
         |                                   .  ..                   *...   |
   23000 ++                                   *.                            |
         |                                                                  |
   22000 ++                                                                 |
         |      O                                                           |
   21000 O+             O                             O      O       O      |
         |                     O       O                                    |
   20000 ++                                   O                             |
         |                                                                  O
   19000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                24424.39       -15.4%     20660.17  nhm-white/micro/aim7/misc_rtns_1
                24424.39       -15.4%     20660.17  TOTAL interrupts.CPU5.RES
\f
                            sched_debug.cpu#6.nr_switches

   80000 ++-----------------------------------------------------------------+
         |                                                                  |
   75000 ++                                                  *..            |
         |                                                 ..   ..          |
         |                                               ..       ..        |
   70000 *+.....    ....*.....       ..*.....          ..           .       |
         |      *...          .  ....        .    ....*              *......*
   65000 ++                    *.             *...                          |
         |                                                                  |
   60000 ++     O       O                                    O              |
         O                             O              O              O      |
         |                     O              O                             |
   55000 ++                                                                 O
         |                                                                  |
   50000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                68269.93       -14.8%     58135.54  nhm-white/micro/aim7/misc_rtns_1
                68269.93       -14.8%     58135.54  TOTAL sched_debug.cpu#6.nr_switches
\f
                            sched_debug.cpu#6.sched_count

   80000 ++-----------------------------------------------------------------+
         |                                                                  |
   75000 ++                                                  *..            |
         |                                                 ..   ..          |
         |                                               ..       ..        |
   70000 *+.....    ....*.....       ..*.....          ..           .       |
         |      *...          .  ....        .    ....*              *......*
   65000 ++                    *.             *...                          |
         |                                                                  |
   60000 ++     O       O                                    O              |
         O                             O              O              O      |
         |                     O              O                             |
   55000 ++                                                                 O
         |                                                                  |
   50000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                68301.75       -14.8%     58169.82  nhm-white/micro/aim7/misc_rtns_1
                68301.75       -14.8%     58169.82  TOTAL sched_debug.cpu#6.sched_count
\f
                           sched_debug.cpu#0.sched_goidle

   38000 ++-----------------------------------------------------------------+
         |          ....*.....                                              |
   37000 *+.....*...          .    ....*......            ...*.....         |
   36000 ++                    *...           *.......*...         ..    ...*
         |                                                           *...   |
   35000 ++                                                                 |
   34000 ++                                                                 |
         |                                                   O              |
   33000 ++                                                                 |
   32000 ++             O                                                   |
         O      O                                                           |
   31000 ++                    O                      O              O      |
   30000 ++                            O      O                             |
         |                                                                  O
   29000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                36349.50       -14.3%     31146.73  nhm-white/micro/aim7/misc_rtns_1
                36349.50       -14.3%     31146.73  TOTAL sched_debug.cpu#0.sched_goidle
\f
                                     softirqs.RCU

   175000 ++----------------------------------------------------------------+
          *......*.....        .*.....                                      |
   170000 ++           ..  ....       .    ....*......*.....               .*
          |              *.            *...                 .            .. |
   165000 ++                                                 *.......  ..   |
          |                                                          *.     |
   160000 ++                                                                |
          |                                                                 |
   155000 ++                                                                |
          |                                                                 |
   150000 ++                                                                |
          |      O                                    O              O      |
   145000 O+             O                                   O              |
          |                     O      O       O                            |
   140000 ++----------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               169088.39       -14.3%    144828.56  nhm-white/micro/aim7/misc_rtns_1
               169088.39       -14.3%    144828.56  TOTAL softirqs.RCU
\f
                           sched_debug.cpu#5.sched_goidle

   30000 ++-----*--------------*--------------------------------------------+
         | ....       ..  ....         *..           .*......              .*
   29000 *+             *.                ..       ..        *.....      .. |
         |                                  ..   ..                ..  ..   |
         |                                     ..                    *.     |
   28000 ++                                   *                             |
         |                                                                  |
   27000 ++                                                                 |
         |      O                                                           |
   26000 ++                                                                 |
         |                             O                                    |
         O              O      O                      O              O      |
   25000 ++                                   O              O              O
         |                                                                  |
   24000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                29181.61       -13.3%     25309.49  nhm-white/micro/aim7/misc_rtns_1
                29181.61       -13.3%     25309.49  TOTAL sched_debug.cpu#5.sched_goidle
\f
                           sched_debug.cpu#6.sched_goidle

   31000 ++----------------------------*------------------------------------+
         |            ..*...        ..    ..                                |
   30000 *+.....  ....      ..    ..        ..                              |
   29000 ++     *.            . ..                                          |
         |                     *              *.......*......*.......*......*
   28000 ++                                                                 |
         |                                                                  |
   27000 ++                                                                 |
         |                                                                  |
   26000 ++     O       O                                    O              |
   25000 ++                            O                                    |
         O                                            O              O      |
   24000 ++                    O              O                             |
         |                                                                  O
   23000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                29244.19       -14.7%     24941.43  nhm-white/micro/aim7/misc_rtns_1
                29244.19       -14.7%     24941.43  TOTAL sched_debug.cpu#6.sched_goidle
\f
                                   softirqs.SCHED

   24000 ++-----------------------------------------------------------------+
   23500 ++             *...                                                |
         |             .    ..                                              |
   23000 ++           .       .                                             |
   22500 ++         ..         *..                                          |
   22000 *+..      .              .                       ...*.......*...   |
   21500 ++  ..   .                ..             ....*...               .. |
         |     . .                   ..      .*...                         .|
   21000 ++     *                        ....                               *
   20500 ++                            *.     O                             |
   20000 ++                                                                 |
   19500 ++     O                      O                             O      O
         |              O                             O      O              |
   19000 O+                    O                                            |
   18500 ++-----------------------------------------------------------------+

\f
                                   softirqs.SCHED

   32000 ++-------------------------------------------O---------------------+
   31000 O+     O                                                    O      |
         |              O                                    O              |
   30000 ++                    O              O                             |
   29000 ++                            O                                    O
         |                                                                  |
   28000 ++                                                                 |
   27000 ++                                                                 |
   26000 *+.....                                                            |
         |      *.....                                                      *
   25000 ++           ..    ...*.....             ....*......*...         ..|
   24000 ++             *...         ..    ...*...               ...    ..  |
         |                             *...                         . ..    |
   23000 ++                                                          *      |
   22000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                21699.13       -10.0%     19519.87  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                24662.49       +23.3%     30415.67  nhm-white/micro/aim7/misc_rtns_1
                46361.62        +7.7%     49935.54  TOTAL softirqs.SCHED
\f
                        sched_debug.cfs_rq[2]:/.min_vruntime

   3400 ++------------------------------------------------------------------+
   3200 ++             *                      O                             |
        |             : +                                                   |
   3000 ++           :   +                                                  |
   2800 ++           :    +                                                 |
        *..         :      +                                                |
   2600 ++ .       :        +                                               |
   2400 ++  ..    :          +       .*.....                                |
   2200 ++    ..  :           +  ....       ..                      *.....  |
        |        :             *.     O       *......             ..      ..|
   2000 ++      *                                    *.....     ..          *
   1800 O+                                                 ..O..            O
        |              O                                     *      O       |
   1600 ++                                           O                      |
   1400 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 2254.81       -11.6%      1993.26  nhm-white/micro/aim7/misc_rtns_1
                 2254.81       -11.6%      1993.26  TOTAL sched_debug.cfs_rq[2]:/.min_vruntime
\f
                             sched_debug.cpu#3.nr_switches

   100000 ++---------------------------------------------------------O------+
          |      *.                                                         |
    95000 ++   ..  ..           *.                                          |
          |   .      .        ..  .                                         |
    90000 ++ .        .      .     ..                                       |
          |..          ..   .        .                                      |
    85000 *+              ..          .                                     |
          |              *                                ...*.....         |
    80000 ++                           *.......    ...*...         ..    ...*
          |                                    *...                  *...   |
    75000 ++                                                                |
          O                     O                                           |
    70000 ++     O       O             O       O      O      O              |
          |                                                                 O
    65000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                84214.17       -12.9%     73322.31  nhm-white/micro/aim7/misc_rtns_1
                84214.17       -12.9%     73322.31  TOTAL sched_debug.cpu#3.nr_switches
\f
                                  softirqs.CPU0.RCU

   24000 ++-----------------------------------------------------------------+
         |                            .*..           .*...                  |
   23000 ++                         ..    ..       ..     ..                |
   22000 ++     O                 ..        ..   ..         .               |
         |          ....*...... ..             ..            *.......       |
   21000 *+.....*...           *              *                      *......*
   20000 ++                    O                                            |
         |                                                                  |
   19000 ++                                                                 O
   18000 ++             O                     O                             |
         |                                                   O              |
   17000 O+                            O                                    |
   16000 ++                                                          O      |
         |                                            O                     |
   15000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                21465.75       -15.9%     18061.93  nhm-white/micro/aim7/misc_rtns_1
                21465.75       -15.9%     18061.93  TOTAL softirqs.CPU0.RCU
\f
                             sched_debug.cpu#3.sched_count

   100000 ++---------------------------------------------------------O------+
          |      *.                                                         |
    95000 ++   ..  ..           *.                                          |
          |  ..      .        ..  .                                         |
    90000 ++.         .      .     ..                                       |
          *            ..   .        .                                      |
    85000 ++              ..          .                                     |
          |              *                                ...*.......       |
    80000 ++                           *.......    ...*...           *......*
          |                                    *...                         |
    75000 ++                                                                |
          O                     O                                           |
    70000 ++     O       O             O       O      O      O              |
          |                                                                 O
    65000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                84259.37       -12.9%     73360.78  nhm-white/micro/aim7/misc_rtns_1
                84259.37       -12.9%     73360.78  TOTAL sched_debug.cpu#3.sched_count
\f
                            slabinfo.kmalloc-128.num_objs

   3300 ++----------------------------O-------------------------------------+
        |                                                                   |
   3200 O+                     O                                            |
   3100 ++                                    O                     O       O
        |              *                                                    |
   3000 ++      O    .. +                                                   |
        |           .    +                           O       O              |
   2900 ++         .   O  +           *..                                   |
        |        ..        +        ..   .                                  |
   2800 ++    ..*           +      .      ..                                |
   2700 ++....               +    .         ..                             .*
        *.                    + ..                                      ... |
   2600 ++                     *              *......*.......*......  ..    |
        |                                                           *.      |
   2500 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 2717.34       +13.4%      3081.63  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                 2717.34       +13.4%      3081.63  TOTAL slabinfo.kmalloc-128.num_objs
\f
                          slabinfo.kmalloc-128.active_objs

   3300 ++----------------------------O-------------------------------------+
        |                                                                   |
   3200 O+                                                                  |
   3100 ++                     O                                    O       O
        |              *                                                    |
   3000 ++      O    .. +                     O                             |
        |           .    +                                   O              |
   2900 ++         .      +           *..            O                      |
        |        ..    O   +        ..   .                                  |
   2800 ++    ..*           +      .      ..                                |
   2700 ++....               +    .         ..                             .*
        *.                    + ..                                      ... |
   2600 ++                     *              *......*.......*......  ..    |
        |                                                           *.      |
   2500 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 2717.34       +12.7%      3061.40  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                 2717.34       +12.7%      3061.40  TOTAL slabinfo.kmalloc-128.active_objs
\f
                            sched_debug.cpu#3.ttwu_count

   52000 ++----------------------------------------------------------O------+
         |      *.                                                          |
   50000 ++   ..  ..           *.                                           |
   48000 ++ ..      .        ..  ..                                         |
         |..         .      .      .                                        |
   46000 *+           ..   .        .                                       |
   44000 ++              ..          ..                     .*.....         |
         |              *                               ....       ..       |
   42000 ++                            *......    ....*.             *......*
   40000 ++                                   *...                          |
         |                                                                  |
   38000 O+             O                             O      O              |
   36000 ++     O              O              O                             |
         |                             O                                    O
   34000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                44008.82       -13.0%     38280.08  nhm-white/micro/aim7/misc_rtns_1
                44008.82       -13.0%     38280.08  TOTAL sched_debug.cpu#3.ttwu_count
\f
                                  softirqs.CPU4.RCU

   24000 ++-----------------------------------------------------------------+
         |                                    *                             |
   23000 ++                 ...*..          .. +                            |
   22000 ++             *...      ..       .    +                          .*
         *..          ..            ..    .      +           O         .... |
   21000 O+ ..      ..                . ..        +                 .*.     |
   20000 ++   ..   .                   *           +             ...        |
         |       ..     O                           +          ..           |
   19000 ++     *                                    +O   ...*.      O      |
   18000 ++     O                                     *...                  |
         |                             O                                    |
   17000 ++                                                                 |
   16000 ++                                   O                             |
         |                     O                                            |
   15000 ++-----------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                20872.34       -11.8%     18400.51  nhm-white/micro/aim7/misc_rtns_1
                20872.34       -11.8%     18400.51  TOTAL softirqs.CPU4.RCU
\f
                           buddyinfo.Node.0.zone.Normal.0

   1400 ++------------------------------------------------------------------+
        |                                                                   O
   1350 ++                            O                                     |
        |       O                             O      O       O              |
        |                      O                                            |
   1300 ++                                                          O       |
        |              O                                                    |
   1250 ++                                                                  |
        O                                                                   |
   1200 ++                                                 ..*......*..     |
        |                          ...*.......         ....            .    |
        *.....        .*.......*...           *......*.                 ..  |
   1150 ++    ..  ....                                                    ..|
        |       *.                                                          |
   1100 ++------------------------------------------------------------------*

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 1164.51       +13.5%      1321.68  nhm-white/micro/aim7/misc_rtns_1
                 1164.51       +13.5%      1321.68  TOTAL buddyinfo.Node.0.zone.Normal.0
\f
                              slabinfo.filp.active_objs

   1100 ++------------------------------------------------------------------+
        |                                                                   |
   1050 ++             O                                                    |
        |                                                                   |
        |                             O                                     |
   1000 ++                                           O       O              O
        |                                                                   |
    950 O+                                                                  |
        |                                                                   |
    900 ++                     O                                    *.......|
        |       O     .*...          .*...    O                   ..        *
        |           ..     ...     ..     ...       .*.....     ..          |
    850 *+......  ..          .  ..          .  ....       .. ..            |
        |       *.             *.             *.             *              |
    800 ++------------------------------------------------------------------+

\f
                              slabinfo.filp.active_objs

   1100 ++------------------------------------------------------------------+
        |                                                           O       |
   1050 ++                                                                  |
        |                                                                   |
   1000 ++                                                                  |
        O                                                                   |
    950 ++                     O                             O              O
        |       O                     O              O                      |
    900 ++     .*..                                                         |
        |   ...    ..  O                      O                             |
    850 ++..         ..            ...*.......*.....       ..*......        |
        *.                 ....*...                 .  ....         *.......*
    800 ++             *...                          *.                     |
        |                                                                   |
    750 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  854.95       +11.8%       956.18  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                  832.61       +12.8%       939.08  lkp-a04/micro/netperf/120s-200%-UDP_RR
                 1687.56       +12.3%      1895.26  TOTAL slabinfo.filp.active_objs
\f
                            sched_debug.cpu#5.nr_switches

   70000 ++-----------------------------------------------------------------+
         |   ...*.......*......                                             |
   68000 *+..                  *.......                                     |
         |                             *...          .*......            ...*
   66000 ++                                ..     ...        *.......*...   |
         |                                   .  ..                          |
   64000 ++                                   *.                            |
         |                                                                  |
   62000 ++     O                                                           |
         |                                                                  |
   60000 ++                            O                                    |
         |              O      O                      O              O      |
   58000 O+                                   O              O              O
         |                                                                  |
   56000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                67219.90       -12.5%     58843.19  nhm-white/micro/aim7/misc_rtns_1
                67219.90       -12.5%     58843.19  TOTAL sched_debug.cpu#5.nr_switches
\f
                            sched_debug.cpu#5.sched_count

   70000 ++-----------------------------------------------------------------+
         |   ...*.......*......                                             |
   68000 *+..                  *.......                                     |
         |                             *...          .*......            ...*
   66000 ++                                ..     ...        *.......*...   |
         |                                   .  ..                          |
   64000 ++                                   *.                            |
         |                                                                  |
   62000 ++     O                                                           |
         |                                                                  |
   60000 ++                            O                                    |
         |              O      O                      O              O      |
   58000 O+                                   O              O              O
         |                                                                  |
   56000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                67253.78       -12.5%     58880.24  nhm-white/micro/aim7/misc_rtns_1
                67253.78       -12.5%     58880.24  TOTAL sched_debug.cpu#5.sched_count
\f
                                  proc-vmstat.pgfree

   6.3e+06 ++---------------------------------------------------------------+
   6.2e+06 ++     O                                                         |
           O                            O                            O      |
   6.1e+06 ++            O                                    O             |
     6e+06 ++                    O             O                            |
   5.9e+06 ++                                                               O
   5.8e+06 ++                                                               |
           |                                                                |
   5.7e+06 ++                                         O                     |
   5.6e+06 ++                                                               |
   5.5e+06 ++                                                               |
   5.4e+06 ++                                        .*.....                |
           *......    ...*.......                ....       ..              |
   5.3e+06 ++     *...           *......*......*.             *......*......*
   5.2e+06 ++---------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              5312453.48       +13.5%   6029326.46  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              5312453.48       +13.5%   6029326.46  TOTAL proc-vmstat.pgfree
\f
                                  softirqs.CPU6.RCU

   22500 ++-----------------------------------------------------------------+
   22000 *+....                                                          ...*
         |     .                                            .*.......*...   |
   21500 ++     *.......*......*.......*......          ....                |
   21000 ++                                   *.......*.                    |
         |                                                                  |
   20500 ++                                                                 |
   20000 ++                                                                 |
   19500 ++                                                                 |
         |                                                                  |
   19000 ++             O      O       O      O              O              |
   18500 ++                                           O              O      |
         |      O                                                           O
   18000 O+                                                                 |
   17500 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                21554.90       -13.8%     18579.77  nhm-white/micro/aim7/misc_rtns_1
                21554.90       -13.8%     18579.77  TOTAL softirqs.CPU6.RCU
\f
                           sched_debug.cpu#2.sched_goidle

   39000 ++-----------------------------------------------------------------+
   38000 *+.           .*...                                                |
         |  ..       ..     ..                                     ..*..... |
   37000 ++   ..   ..         .                                ....        .|
   36000 ++      ..            *.....        .*.......*......*.             *
         |      *                    ..  ....                               |
   35000 ++                            *.                                   |
   34000 ++                                                                 |
   33000 ++                                                                 |
         |                                                                  |
   32000 O+                    O              O       O              O      |
   31000 ++     O                                                           |
         |              O              O                     O              |
   30000 ++                                                                 |
   29000 ++-----------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                36507.99       -14.9%     31078.42  nhm-white/micro/aim7/misc_rtns_1
                36507.99       -14.9%     31078.42  TOTAL sched_debug.cpu#2.sched_goidle
\f
                            sched_debug.cpu#2.ttwu_count

   48000 ++-----------------------------------------------------------------+
         *..                                                                |
   46000 ++ .          .*.....                                              |
         |   .       ..       .                                             |
   44000 ++   ..   ..          *.......       O                             |
         |       ..                    *.....                      ..*......|
   42000 ++     *                            .    ....*......  ....         *
         |                                    *...           *.             |
   40000 ++                                                                 |
         |                                                                  |
   38000 ++                                                                 |
         O      O              O       O              O      O       O      |
   36000 ++             O                                                   |
         |                                                                  O
   34000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                43167.98       -13.2%     37457.64  nhm-white/micro/aim7/misc_rtns_1
                43167.98       -13.2%     37457.64  TOTAL sched_debug.cpu#2.ttwu_count
\f
                                 interrupts.CPU7.RES

   25000 ++-------------*---------------------------------------------------+
   24500 *+....     ...       .                                             |
         |     .  ..           *.....        .*.......                      |
   24000 ++     *.                   ..  ....         *......*.......    ...*
   23500 ++                            *.                            *...   |
         |                                                                  |
   23000 ++                                                                 |
   22500 ++     O              O                                            |
   22000 ++                                                                 |
         O                                                                  |
   21500 ++                                                          O      |
   21000 ++             O                             O      O              |
         |                             O      O                             |
   20500 ++                                                                 O
   20000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                24019.62       -11.5%     21267.02  nhm-white/micro/aim7/misc_rtns_1
                24019.62       -11.5%     21267.02  TOTAL interrupts.CPU7.RES
\f
                            sched_debug.cpu#2.sched_count

   95000 ++-----------------------------------------------------------------+
         *..                                                                |
   90000 ++ .          .*...                                                |
         |   .       ..     ..                                      .*...   |
   85000 ++   ..   ..         .                                   ..     .. |
         |       ..            *.....         O                 ..         .|
   80000 ++     *                    ..    ...*.......*...... ..            *
         |                             *...                  *              |
   75000 ++                                                                 |
         |                                                                  |
   70000 O+     O              O                      O              O      |
         |              O              O                     O              |
   65000 ++                                                                 O
         |                                                                  |
   60000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                82664.49       -15.0%     70263.14  nhm-white/micro/aim7/misc_rtns_1
                82664.49       -15.0%     70263.14  TOTAL sched_debug.cpu#2.sched_count
\f
                            sched_debug.cpu#2.nr_switches

   95000 ++-----------------------------------------------------------------+
         *..                                                                |
   90000 ++ .          .*..                                                 |
         |   .       ..    ..                                       .*...   |
   85000 ++   ..   ..        ..                                   ..     .. |
         |       ..                           O                 ..         .|
   80000 ++     *              *.......    ...*.......*...... ..            *
         |                             *...                  *              |
   75000 ++                                                                 |
         |                                                                  |
   70000 O+     O              O                      O              O      |
         |              O              O                     O              |
   65000 ++                                                                 O
         |                                                                  |
   60000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                82620.08       -15.0%     70223.42  nhm-white/micro/aim7/misc_rtns_1
                82620.08       -15.0%     70223.42  TOTAL sched_debug.cpu#2.nr_switches
\f
                            sched_debug.cpu#6.ttwu_count

   42000 ++-----------------------------------------------------------------+
         |                                                   *..            |
   40000 ++                                                ..   ..          |
         |                                                .       ..        |
         *.....       ..*...                             .          .       |
   38000 ++    .  ....      ..                         ..            *......|
         |      *.            .    ....*......    ....*                     *
   36000 ++                    *...           *...                          |
         |                                                                  |
   34000 ++                                                                 |
         |      O       O                                                   |
         O                     O              O       O              O      |
   32000 ++                            O                     O              |
         |                                                                  O
   30000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                37659.16       -13.7%     32493.72  nhm-white/micro/aim7/misc_rtns_1
                37659.16       -13.7%     32493.72  TOTAL sched_debug.cpu#6.ttwu_count
\f
                                  softirqs.CPU1.RCU

   23000 ++-----------------------------------------------------------------+
         |                                                                  |
   22000 ++                            *..           .*......               |
         |     .*...                 ..   ..       ..        *.....         |
   21000 ++....     ...            ..  O    ..   ..                ..       |
         *.            .          .            ..                    *......|
   20000 ++             *...... ..            *       O                     *
         |                     *                                            |
   19000 ++                                                                 O
         |              O      O              O                             |
   18000 O+                                                  O       O      |
         |                                                                  |
   17000 ++                                                                 |
         |      O                                                           |
   16000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                20774.12       -10.7%     18544.93  nhm-white/micro/aim7/misc_rtns_1
                20774.12       -10.7%     18544.93  TOTAL softirqs.CPU1.RCU
\f
                           sched_debug.cpu#7.sched_goidle

   29500 ++-----------------------------------------------------------------+
   29000 *+....       ..*......                                             |
         |     .  ....         *.....             ....*......              .*
   28500 ++     *.             O     ..    ...*...           *.......  .... |
   28000 ++                            *...                          *.     |
   27500 ++                                                                 |
   27000 ++                                                                 |
         |                                                                  |
   26500 ++     O                                                           |
   26000 O+                                                                 |
   25500 ++                                   O                             |
   25000 ++                                                  O       O      |
         |              O              O              O                     |
   24500 ++                                                                 O
   24000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                28506.05       -10.2%     25592.08  nhm-white/micro/aim7/misc_rtns_1
                28506.05       -10.2%     25592.08  TOTAL sched_debug.cpu#7.sched_goidle
\f
                           sched_debug.cpu#3.sched_goidle

   37000 ++-------------*---------------------------------------------------+
         |   ...*...        ..                                              |
   36000 *+..                 .                   ....*......*.......    ...*
         |                     *.......*......*...                   *...   |
   35000 ++                                                                 |
         |                                                                  |
   34000 ++                                                                 |
         |                                                                  |
   33000 ++                    O                                            |
         O                                                                  |
   32000 ++             O                     O       O              O      |
         |                                                   O              |
   31000 ++     O                      O                                    |
         |                                                                  O
   30000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                35939.20       -11.5%     31822.25  nhm-white/micro/aim7/misc_rtns_1
                35939.20       -11.5%     31822.25  TOTAL sched_debug.cpu#3.sched_goidle
\f
                                  softirqs.CPU5.RCU

   22000 ++-----*-----------------------------------------------------------+
         | ....         *......              .*.....                        |
   21500 *+                    *.....      ..       ..                      |
   21000 ++                          ..  ..           *......              .*
         |                             *.                    *.......  .... |
   20500 ++                                                          *.     |
   20000 ++                                                                 |
         |                                                                  |
   19500 ++                            O      O                             |
   19000 ++                    O                                            |
         |      O                                     O              O      |
   18500 ++                                                  O              O
   18000 O+                                                                 |
         |              O                                                   |
   17500 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                21152.73       -11.4%     18737.90  nhm-white/micro/aim7/misc_rtns_1
                21152.73       -11.4%     18737.90  TOTAL softirqs.CPU5.RCU
\f
                           buddyinfo.Node.0.zone.Normal.1

   1100 ++------------------------------------------------------------------+
        |                                            O                      |
   1050 ++                                                                  |
        |                                                                   |
   1000 ++                            O                                     O
        |                      O      *..     O              O              |
    950 ++      O                    .   .                                  |
        |                           .     ..               ..*..... O       |
    900 ++             O          ..        ..         ....        .        |
        O             .*.....    .                ...*.             *..     |
    850 ++        ....       .. .             *...                     ..   |
        *.......*.             *                                         .. |
    800 ++                                                                 .|
        |                                                                   *
    750 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  861.68       +12.4%       968.22  nhm-white/micro/aim7/misc_rtns_1
                  861.68       +12.4%       968.22  TOTAL buddyinfo.Node.0.zone.Normal.1
\f
                            sched_debug.cpu#5.ttwu_count

   39000 ++-----------------------------------------------------------------+
         |          ....*..                                                 |
   38000 ++  ...*...       .                                                |
         *...               .         .*...                                 |
   37000 ++                  ..    ...     ..       ..*......*.......    ...*
         |                       ..          .  ....                 *...   |
   36000 ++                    *.             *.                            |
         |                                                                  |
   35000 ++     O                                                           |
         |                                                                  |
   34000 ++                                                                 |
         |              O              O                                    |
   33000 O+                    O              O       O              O      |
         |                                                   O              O
   32000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                37078.89       -10.4%     33230.12  nhm-white/micro/aim7/misc_rtns_1
                37078.89       -10.4%     33230.12  TOTAL sched_debug.cpu#5.ttwu_count
\f
                                 proc-vmstat.numa_hit

   980000 ++-----O----------------------------------------------------------+
          |                            O                             O      |
   960000 O+                                                 O              |
          |              O                     O                            |
   940000 ++                    O                                           O
          |                                                                 |
   920000 ++                                                                |
          |                                                                 |
   900000 ++                                          O                     |
          |                                                                 |
   880000 ++                                                                |
          *......                                  ...*......               |
   860000 ++     *.......*......*......*.......*...          *.......*......*
          |                                                                 |
   840000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               860788.53       +10.6%    952024.03  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               860788.53       +10.6%    952024.03  TOTAL proc-vmstat.numa_hit
\f
                                proc-vmstat.numa_local

   980000 ++-----O----------------------------------------------------------+
          |                            O                             O      |
   960000 O+                                                 O              |
          |              O                     O                            |
   940000 ++                    O                                           O
          |                                                                 |
   920000 ++                                                                |
          |                                                                 |
   900000 ++                                          O                     |
          |                                                                 |
   880000 ++                                                                |
          *......                                  ...*......               |
   860000 ++     *.......*......*......*.......*...          *.......*......*
          |                                                                 |
   840000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               860788.53       +10.6%    952023.96  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               860788.53       +10.6%    952023.96  TOTAL proc-vmstat.numa_local
\f
                          sched_debug.cpu#5.nr_load_updates

   9400 ++------O-----------------------------------------------------------+
        |                             O                                     O
   9200 ++                                                                  |
   9000 ++                                                                  |
        |                                                                   |
   8800 ++                                    O                             |
   8600 ++                     O                                            |
        |                                                                   |
   8400 ++             O                             O              O       |
   8200 ++                                                                  |
        O                                                    O     .*...    |
   8000 ++          ...*...                                      ..     ... |
   7800 *+......*...       ...                                 ..          .|
        |                     .    ...*.......    ...*.......*.             *
   7600 ++---------------------*--------------*-----------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 7802.01       +11.1%      8670.11  nhm-white/micro/aim7/misc_rtns_1
                 7802.01       +11.1%      8670.11  TOTAL sched_debug.cpu#5.nr_load_updates
\f
                            sched_debug.cpu#7.nr_switches

   68000 ++-------------*---------------------------------------------------+
         |           ..        *..                                          |
   66000 *+....    ..             ..                                        |
         |     . ..            O    ..       .*.......*......*.......    ...*
         |      *                     .  ....                        *...   |
   64000 ++                            *.                                   |
         |                                                                  |
   62000 ++                                                                 |
         |      O                                                           |
   60000 ++                                                                 |
         O                                    O                             |
         |                                                   O       O      |
   58000 ++             O              O              O                     |
         |                                                                  O
   56000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                65555.21        -9.3%     59426.89  nhm-white/micro/aim7/misc_rtns_1
                65555.21        -9.3%     59426.89  TOTAL sched_debug.cpu#7.nr_switches
\f
                            sched_debug.cpu#7.sched_count

   68000 ++-------------*---------------------------------------------------+
         |           ..        *..                                          |
   66000 *+....    ..             ..                                        |
         |     . ..            O    ..       .*.......*......*.......    ...*
         |      *                     .  ....                        *...   |
   64000 ++                            *.                                   |
         |                                                                  |
   62000 ++                                                                 |
         |      O                                                           |
   60000 ++                                                                 |
         O                                    O                             |
         |                                                   O       O      |
   58000 ++             O              O              O                     |
         |                                                                  O
   56000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                65585.05        -9.3%     59462.10  nhm-white/micro/aim7/misc_rtns_1
                65585.05        -9.3%     59462.10  TOTAL sched_debug.cpu#7.sched_count
\f
                            sched_debug.cpu#6.ttwu_local

   14000 ++-----------------------------------------------------------------+
         |                                                                  |
   13000 ++                                                  *              |
         |                                                  : +             |
   12000 ++                                                 :  +            |
         |                                                 :    +           |
   11000 ++                                               :      +          |
         |                                                :       +         |
   10000 ++                                              :         +        |
         |                                              :           +       |
    9000 *+.....    ....*.....                         :             *..... |
         |      *...    O     .                        :                   .|
    8000 ++                    *.......*......*.......*                     *
         O      O              O       O      O       O      O       O      |
    7000 ++-----------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 8846.95       -14.5%      7563.70  nhm-white/micro/aim7/misc_rtns_1
                 8846.95       -14.5%      7563.70  TOTAL sched_debug.cpu#6.ttwu_local
\f
                                 softirqs.CPU3.SCHED

   25000 ++-----------------------------------------------------------------+
         |                                    *.....                 *...   |
   24000 ++                                  +      ..              +    .. |
         |                                  +         *.           +       .|
   23000 ++                                +            .         +         *
         |             .*..            O  +              ..      +          |
   22000 *+..        ..    .             +                 .    +           |
         |   ..    ..       .           +                   .O +            |
   21000 ++    . ..          ..      ..*                      +             O
         |      *                ....         O              *       O      |
   20000 ++     O              *.                                           |
         |                                                                  |
   19000 O+                                                                 |
         |              O                             O                     |
   18000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                22195.37        -9.0%     20188.74  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                22195.37        -9.0%     20188.74  TOTAL softirqs.CPU3.SCHED
\f
                                    uptime.idle

   265 ++---------------------O---------------------------------------------+
   260 ++                                                                   |
       |                                                                    O
   255 O+                                                    O      O       |
   250 ++                                                                   |
       |                                                                    |
   245 ++             O                                                     |
   240 ++                                                                   |
   235 ++      O                             O                              |
       |                              O            ..*..                    |
   230 *+......    ...*.......*.......*......  ....     .           *.......|
   225 ++      *...                          *.          ..       ..        *
       |                                                   ..   ..          |
   220 ++                                                     ..            |
   215 ++----------------------------------------------------*--------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  227.63        +8.2%       246.30  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                  227.63        +8.2%       246.30  TOTAL uptime.idle
\f
                                   interrupts.LOC

   27000 ++-----------------------------------------------------------------+
         |                             O                                    O
   26500 ++     O              O                                            |
   26000 ++             O                                                   |
         |                                    O                             |
   25500 ++                                                                 |
         O                                            O      O              |
   25000 ++                                                          O      |
         |                                                                  |
   24500 ++           ..*..                                         .*...   |
   24000 ++       ....     ..                                    ...     .. |
         |   ...*.           ..                                ..          .|
   23500 *+..                      ....*......            ...*.             *
         |                     *...           *.......*...                  |
   23000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                23712.01        +9.2%     25893.32  nhm-white/micro/aim7/misc_rtns_1
                23712.01        +9.2%     25893.32  TOTAL interrupts.LOC
\f
               lock_stat.((&icsk->icsk_retransmit_timer)).acquisitions

   2500 ++------------------------------------------------------------------+
        |                                                                   |
        *.......*......*.......*......*.......*......*.......*......*.......*
   2000 ++                                                                  |
        |                                                                   |
        |                                                                   |
   1500 ++                                                                  |
        |                                                                   |
   1000 ++                                                                  |
        |                                                                   |
        |                                                                   |
    500 ++                                                                  |
        |                                                                   |
        |                                                                   |
      0 O+------O------O-------O------O-------O------O-------O------O-------O

\f
               lock_stat.((&icsk->icsk_retransmit_timer)).acquisitions

   2500 ++------------------------------------------------------------------+
        |   ....*......                                          ...*.....  |
        *...           *.......*......*.......*......*.......*...         ..|
   2000 ++                                                                  *
        |                                                                   |
        |                                                                   |
   1500 ++                                                                  |
        |                                                                   |
   1000 ++                                                                  |
        |                                                                   |
        |                                                                   |
    500 ++                                                                  |
        |                                                                   |
        |                                                                   |
      0 O+------O------O-------O------O-------O------O-------O------O-------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 2142.30       -98.9%        22.80  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                 2165.10       -99.6%         8.20  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                 4307.40       -99.3%        31.00  TOTAL lock_stat.((&icsk->icsk_retransmit_timer)).acquisitions
\f
                 lock_stat.((&icsk->icsk_delack_timer)).acquisitions

   3000 ++------------------------------------------------------------------+
        |           ...*.......    ...*.......*......    ....*......*.......*
   2500 *+......*...           *...                  *...                   |
        |                                                                   |
        |                                                                   |
   2000 ++                                                                  |
        |                                                                   |
   1500 ++                                                                  |
        |                                                                   |
   1000 ++                                                                  |
        |                      O                                            |
        |                                                                   |
    500 ++                                                                  |
        |                                                                   |
      0 O+------O------O--------------O-------O------O-------O------O-------O

\f
                 lock_stat.((&icsk->icsk_delack_timer)).acquisitions

   2000 ++------*------*----------------------------------------------------+
   1800 *+..                   *......    ....*......*.......    ...*.......*
        |                             *...                   *...           |
   1600 ++                                                                  |
   1400 ++                                                                  |
        |                                                                   |
   1200 ++                                                                  |
   1000 ++                                                                  |
    800 ++                                                                  |
        |                                                                   |
    600 ++                     O                                            |
    400 ++                                                                  |
        |                                                                   |
    200 ++                                                   O              O
      0 O+------O------O--------------O-------O------O--------------O-------+

\f
                 lock_stat.((&icsk->icsk_delack_timer)).acquisitions

   3000 ++------------------------------------------------------------------+
        |                                                                   |
   2500 *+......*......*.......*......*.......*......*.......*......*.......*
        |                                                                   |
        |                                                                   |
   2000 ++                                                                  |
        |                                                                   |
   1500 ++                                                                  |
        |                                                                   |
   1000 ++                                                                  |
        |                                                                   |
        |                                                                   |
    500 ++                                                                  |
        |                      O                                    O       |
      0 O+------O------O--------------O-------O------O-------O--------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 2693.20       -95.2%       128.70  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                 1875.60       -95.8%        77.90  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                 2571.10       -97.7%        58.80  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                 7139.90       -96.3%       265.40  TOTAL lock_stat.((&icsk->icsk_delack_timer)).acquisitions
\f
                            lock_stat.&wq->wait.acq-bounces

   120000 ++----------------------------------------------------------------+
          |                                                                 |
   100000 ++  ...*.......*......*......            ...*......*.......    ...*
          *...                         *.......*...                  *...   |
          |                                                                 |
    80000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
          |                                                                 |
    40000 ++                                                                |
          |                                                                 |
          |                                                                 |
    20000 ++                                                                |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

\f
                           lock_stat.&wq->wait.acq-bounces

   70000 ++-----------------------------------------------------------------+
         |     .*..                                                         |
   60000 ++  ..    .                                                        |
         | ..       ..                                                      |
   50000 *+           ..      .*.....                                       |
         |                ....       ..    ...*.......                      |
   40000 ++             *.             *...           *......*.......       |
         |                                                           *......|
   30000 ++                                                                 *
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                            lock_stat.&wq->wait.acq-bounces

   160000 ++----------------------------------------------------------------+
          |          ....*......                                            |
   140000 *+.....*...           *......*.......*......*......*.......*......*
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
    80000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
    40000 ++                                                                |
          |                                                                 |
    20000 ++                                                                |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

\f
                            lock_stat.&wq->wait.acq-bounces

   120000 ++----------------------------------------------------------------+
          |   ...*.......                                 ...*.......       |
   100000 *+..           *......*......*.......*......*...           *......*
          |                                                                 |
          |                                                                 |
    80000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
          |                                                                 |
    40000 ++                                                                |
          |                                                                 |
          |                                                                 |
    20000 ++                                                                |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

\f
                           lock_stat.&wq->wait.acq-bounces

   70000 ++-----------------------------------------------------------------+
         |                                                                  |
   60000 ++            .*.....       ..*..           .*..                   |
         |          ...       .  ....     ..       ..    ..                 |
   50000 ++       ..           *.           ..   ..        ..               |
         |     .*.                             ..                        ...*
   40000 ++....                               *              *.......*...   |
         *.                                                                 |
   30000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                            lock_stat.&wq->wait.acq-bounces

   140000 ++-----------------------------------*----------------------------*
          *......*.......*......*......*...           *......*.......*...   |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
    80000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
          |                                                                 |
    40000 ++                                                                |
          |                                                                 |
    20000 ++                                                                |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                98792.00       -99.9%        73.40  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                44294.20       -99.7%       137.90  lkp-a04/micro/netperf/120s-200%-TCP_RR
               138942.60       -99.9%        91.30  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               104643.10      -100.0%        24.60  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                47139.30       -99.9%        69.30  lkp-a04/micro/netperf/120s-200%-UDP_RR
               133068.70       -99.8%       260.50  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               566879.90       -99.9%       657.00  TOTAL lock_stat.&wq->wait.acq-bounces
\f
                     lock_stat.&(&base->lock)->rlock.con-bounces

   25000 ++-----------------------------------------------------------------+
         |      *.                                                          |
         |    ..  ..                                                        |
   20000 ++ ..      .                                                       |
         |..         .                                                      |
         *            ..      .*.....        .*.....                        |
   15000 ++               ....       ..  ....       ..                      |
         |              *.             *.             *......*.......       |
   10000 ++                                                          *......|
         |                                                                  *
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                13827.10       -99.6%        56.80  lkp-a04/micro/netperf/120s-200%-TCP_RR
                13827.10       -99.6%        56.80  TOTAL lock_stat.&(&base->lock)->rlock.con-bounces
\f
             lock_stat.&(&base->lock)->rlock.contentions.lock_timer_base

   2000 ++------------------------------------------------------------------+
   1800 ++                 ....*......                                      |
        |           ...*...           *.......*......*.......*......        |
   1600 ++  ....*...                                                *.......*
   1400 *+..                                                                |
        |                                                                   |
   1200 ++                                                                  |
   1000 ++                                                                  |
    800 ++                                                                  |
        |                                                                   |
    600 ++                                                                  |
    400 ++                                                                  |
        |                                                                   |
    200 ++                                                                  |
      0 O+------O------O-------O------O-------O------O-------O------O-------O

\f
             lock_stat.&(&base->lock)->rlock.contentions.lock_timer_base

   4500 ++------------------------------------------------------------------+
        |   ....*......*.......*......            ...*.......*......*.......|
   4000 *+..                          *.......*...                          *
   3500 ++                                                                  |
        |                                                                   |
   3000 ++                                                                  |
   2500 ++                                                                  |
        |                                                                   |
   2000 ++                                                                  |
   1500 ++                                                                  |
        |                                                                   |
   1000 ++                                                                  |
    500 ++                                                                  |
        |                                                                   |
      0 O+------O------O-------O------O-------O------O-------O------O-------O

\f
             lock_stat.&(&base->lock)->rlock.contentions.lock_timer_base

   45000 ++-----*-----------------------------------------------------------+
         |    ..  ..                                                        |
   40000 ++  .      .                                                       |
   35000 ++ .        .                                                      |
         |..          ..      .*.....                                       |
   30000 *+               ....       ..    ...*.....                        |
   25000 ++             *.             *...         ..                      |
         |                                            *......*.......*......|
   20000 ++                                                                 *
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
    5000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
             lock_stat.&(&base->lock)->rlock.contentions.lock_timer_base

   3500 ++------------------------------------------------------------------+
        *.......*......    ....*......*.......*......*.......    ...*.......*
   3000 ++             *...                                  *...           |
        |                                                                   |
   2500 ++                                                                  |
        |                                                                   |
   2000 ++                                                                  |
        |                                                                   |
   1500 ++                                                                  |
        |                                                                   |
   1000 ++                                                                  |
        |                                                                   |
    500 ++                                                                  |
        |                                                                   |
      0 O+------O------O-------O------O-------O------O-------O------O-------O

\f
             lock_stat.&(&base->lock)->rlock.contentions.lock_timer_base

   4500 ++---------------------------------------------------*--------------+
        *.......*......*.......*......*.......*......  ....         *.......*
   4000 ++                                           *.                     |
   3500 ++                                                                  |
        |                                                                   |
   3000 ++                                                                  |
   2500 ++                                                                  |
        |                                                                   |
   2000 ++                                                                  |
   1500 ++                                                                  |
        |                                                                   |
   1000 ++                                                                  |
    500 ++                                                                  |
        |                                                                   |
      0 O+------O------O-------O------O-------O------O-------O------O-------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 1646.20       -98.0%        33.00  lkp-a04/micro/netperf/120s-200%-TCP_CRR
                 4088.70       -99.0%        39.60  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                27415.10       -99.8%        59.10  lkp-a04/micro/netperf/120s-200%-TCP_RR
                 3187.60       -99.5%        16.90  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                 4208.80       -99.7%        12.80  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                40546.40       -99.6%       161.40  TOTAL lock_stat.&(&base->lock)->rlock.contentions.lock_timer_base
\f
                     lock_stat.&(&base->lock)->rlock.contentions

   25000 ++-----------------------------------------------------------------+
         |      *.                                                          |
         |    ..  ..                                                        |
   20000 ++ ..      .                                                       |
         |..         .                                                      |
         *            ..      .*.....        .*.....                        |
   15000 ++               ....       ..  ....       ..                      |
         |              *.             *.             *......*.......       |
   10000 ++                                                          *......|
         |                                                                  *
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                13827.30       -99.6%        57.00  lkp-a04/micro/netperf/120s-200%-TCP_RR
                13827.30       -99.6%        57.00  TOTAL lock_stat.&(&base->lock)->rlock.contentions
\f
                   lock_stat.&(&base->lock)->rlock.waittime-total

   3500 ++------------------------------------------------------------------+
        |                    ..*.....                                       |
   3000 ++               ....        .    ....*......*.......*.....         |
        |   ....*......*.             *...                         .    ....*
   2500 *+..                                                        *...    |
        |                                                                   |
   2000 ++                                                                  |
        |                                                                   |
   1500 ++                                                                  |
        |                                                                   |
   1000 ++                                                                  |
        |                                                                   |
    500 ++                                                                  |
        |                                     O      O                      O
      0 O+------O------O-------O------O----------------------O------O-------+

\f
                   lock_stat.&(&base->lock)->rlock.waittime-total

   10000 ++-----------------------------------------------------------------+
    9000 ++         ....*......*.......*......    ....*......*.......*......|
         *......*...                          *...                          *
    8000 ++                                                                 |
    7000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
    5000 ++                                                                 |
    4000 ++                                                                 |
         |                                                                  |
    3000 ++                                                                 |
    2000 ++                                                                 |
         |                                                                  |
    1000 ++                                                                 |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                   lock_stat.&(&base->lock)->rlock.waittime-total

   80000 ++-----------------------------------------------------------------+
         |      *..                                                         |
   70000 ++   ..   .                                                        |
   60000 ++ ..      ..                                                      |
         |..          ..                                                    |
   50000 *+                 ...*.....        .*.....                        |
         |              *...         ..  ....       ..                      |
   40000 ++                            *.             *......*.......*..... |
         |                                                                 .|
   30000 ++                                                                 *
   20000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                   lock_stat.&(&base->lock)->rlock.waittime-total

   7000 ++------------------------------------------------------------------+
        *.......                          ....*......*.......           ....*
   6000 ++      *......*.......*......*...                   *......*...    |
        |                                                                   |
   5000 ++                                                                  |
        |                                                                   |
   4000 ++                                                                  |
        |                                                                   |
   3000 ++                                                                  |
        |                                                                   |
   2000 ++                                                                  |
        |                                                                   |
   1000 ++                                                                  |
        |                                                                   |
      0 O+------O------O-------O------O-------O------O-------O------O-------O

\f
                   lock_stat.&(&base->lock)->rlock.waittime-total

   10000 ++-----------------------------------------------------------------+
    9000 *+.....*.......*......*.......*......*.......    ...*.......*......*
         |                                            *...                  |
    8000 ++                                                                 |
    7000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
    5000 ++                                                                 |
    4000 ++                                                                 |
         |                                                                  |
    3000 ++                                                                 |
    2000 ++                                                                 |
         |                                                                  |
    1000 ++                                                                 |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 2830.05       -96.2%       106.19  lkp-a04/micro/netperf/120s-200%-TCP_CRR
                 9049.50       -98.3%       158.23  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                46864.82       -99.5%       212.30  lkp-a04/micro/netperf/120s-200%-TCP_RR
                 6225.57       -98.8%        77.38  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                 9194.30       -99.2%        74.72  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                74164.23       -99.2%       628.83  TOTAL lock_stat.&(&base->lock)->rlock.waittime-total
\f
                  lock_stat.&(&list->lock)->rlock#2.waittime-total

   5000 ++------------------------------------------------------------------+
   4500 *+....                                    ...*.....        .*.......|
        |     ..    ...*.......    ...*.......*...         ..  ....         *
   4000 ++      *...           *...                          *.             |
   3500 ++                                                                  |
        |                                                                   |
   3000 ++                                                                  |
   2500 ++                                                                  |
   2000 ++                                                                  |
        |                                                                   |
   1500 ++                                                                  |
   1000 ++                                                                  |
        |                                                                   |
    500 ++                                                                  |
      0 O+------O------O-------O------O-------O------O-------O------O-------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 4249.11      -100.0%         0.00  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                 4249.11      -100.0%         0.00  TOTAL lock_stat.&(&list->lock)->rlock#2.waittime-total
\f
                    lock_stat.&(&list->lock)->rlock#2.acq-bounces

   70000 ++-----------------------------------------------------------------+
         |                                                                  |
   60000 ++            .*......    ....*..           .*..                   |
         |          ...        *...       ..       ..    ..                 |
   50000 ++       ..                        ..   ..        ..               |
         |     .*.                             ..                        ...*
   40000 ++....                               *              *.......*...   |
         *.                                                                 |
   30000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                     lock_stat.&(&list->lock)->rlock#2.acq-bounces

   180000 ++----------------------------------------------------------------+
          *......*.......*......*......*.......*......*......*.......*......*
   160000 ++                                                                |
   140000 ++                                                                |
          |                                                                 |
   120000 ++                                                                |
   100000 ++                                                                |
          |                                                                 |
    80000 ++                                                                |
    60000 ++                                                                |
          |                                                                 |
    40000 ++                                                                |
    20000 ++                                                                |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                47702.90       -99.9%        65.20  lkp-a04/micro/netperf/120s-200%-UDP_RR
               168233.70       -99.8%       259.40  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               215936.60       -99.8%       324.60  TOTAL lock_stat.&(&list->lock)->rlock#2.acq-bounces
\f
                         lock_stat.&wq->wait.waittime-total

   3500 ++-------------------------------------------*----------------------+
        *.......                                ....    ..                  |
   3000 ++      *......*.......*......    ....*.          ..       .*.......*
        |                             *...                  .  ....         |
   2500 ++                                                   *.             |
        |                                                                   |
   2000 ++                                                                  |
        |                                                                   |
   1500 ++                                                                  |
        |                                                                   |
   1000 ++                                                                  |
        |                                                                   |
    500 ++                                                                  |
        |                                                                   |
      0 O+------O------O-------O------O-------O------O-------O------O-------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 2975.23      -100.0%         0.00  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                 2975.23      -100.0%         0.00  TOTAL lock_stat.&wq->wait.waittime-total
\f
                  lock_stat.slock-AF_INET.contentions.release_sock

   14000 ++-----------------------------------------------------------------+
         *......    ....*......                                  ....*......|
   12000 ++     *...           *.......*......*.......*......*...           *
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
         |                                                                  |
    4000 ++                                                                 |
         |                                                                  |
    2000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                12327.20       -99.9%         8.30  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                12327.20       -99.9%         8.30  TOTAL lock_stat.slock-AF_INET.contentions.release_sock
\f
                        lock_stat.slock-AF_INET/1.con-bounces

   25000 ++-------------*---------------------------------------------------+
         *......*...           *.......*......*.......*......*.......*......*
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                23594.90       -99.9%        13.80  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                23594.90       -99.9%        13.80  TOTAL lock_stat.slock-AF_INET/1.con-bounces
\f
                        lock_stat.slock-AF_INET/1.contentions

   25000 ++-------------*---------------------------------------------------+
         *......*...           *.......*......*.......*......*.......*......*
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                23594.90       -99.9%        13.80  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                23594.90       -99.9%        13.80  TOTAL lock_stat.slock-AF_INET/1.contentions
\f
                  lock_stat.slock-AF_INET/1.contentions.tcp_v4_rcv

   25000 ++-------------*---------------------------------------------------+
         *......*...           *.......*......*.......*......*.......*......*
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                23594.90       -99.9%        13.80  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                23594.90       -99.9%        13.80  TOTAL lock_stat.slock-AF_INET/1.contentions.tcp_v4_rcv
\f
                 lock_stat.slock-AF_INET/1.contentions.release_sock

   14000 ++-----------------------------------------------------------------+
         |                                                                  |
   12000 *+.....*.......*......*.......*......*.......*......*.......*......*
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
         |                                                                  |
    4000 ++                                                                 |
         |                                                                  |
    2000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                11909.80       -99.9%         7.40  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                11909.80       -99.9%         7.40  TOTAL lock_stat.slock-AF_INET/1.contentions.release_sock
\f
               lock_stat.slock-AF_INET/1.contentions.lock_sock_nested

   14000 ++-----------------------------------------------------------------+
         |                                                                  |
   12000 *+.....*.......*......*.......*......    ....*......            ...*
         |                                    *...           *.......*...   |
   10000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
         |                                                                  |
    4000 ++                                                                 |
         |                                                                  |
    2000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                11682.50       -99.9%         7.11  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                11682.50       -99.9%         7.11  TOTAL lock_stat.slock-AF_INET/1.contentions.lock_sock_nested
\f
                         lock_stat.&pipe->mutex/1.acquisitions

   1.4e+07 ++---------------------------O------O--------------O-------------+
           O      O      O       O                    O              O      O
   1.2e+07 ++                                                               |
           |                                                                |
     1e+07 ++                                                               |
           |                                                                |
     8e+06 ++                                                               |
           |                                                                |
     6e+06 ++                                                               |
           |                                                                |
     4e+06 ++                                                               |
           |                                                                |
     2e+06 *+.....    ...*.......*......*......*......*.......*......    ...*
           |      *...                                               *...   |
         0 ++---------------------------------------------------------------+

\f
                         lock_stat.&pipe->mutex/1.acquisitions

   1.4e+07 ++-----O---------------------------------------------------------+
           O             O       O             O                            |
   1.2e+07 ++                           O             O       O      O      O
           |                                                                |
     1e+07 ++                                                               |
           |                                                                |
     8e+06 ++                                                               |
           |                                                                |
     6e+06 ++                                                               |
           |                                                                |
     4e+06 ++                                                               |
           |                                                                |
     2e+06 ++         ...*.......           ...*......                      |
           *......*...           *......*...          *.......*......*......*
         0 ++---------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              1602361.00      +723.0%  13187117.60  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
              1288794.00      +882.1%  12657199.90  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
              2891155.00      +793.9%  25844317.50  TOTAL lock_stat.&pipe->mutex/1.acquisitions
\f
                        lock_stat.&pipe->wait.waittime-total

   25000 ++--------------------------------------------------O--------------+
         |                                                                  |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  O
         |                                                                  |
    5000 ++                                                                 |
         O                     O       O                                    |
         |      O   ....*......               O       O   ...*.......*......|
       0 *+-----*--------------*-------*------*-------*--------------O------*

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  584.68      +770.8%      5091.37  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                  584.68      +770.8%      5091.37  TOTAL lock_stat.&pipe->wait.waittime-total
\f
                          lock_stat.&pipe->wait.acquisitions

   1.4e+07 ++---------------------------O------O--------------O-------------+
           O      O      O       O                    O              O      O
   1.2e+07 ++                                                               |
           |                                                                |
     1e+07 ++                                                               |
           |                                                                |
     8e+06 ++                                                               |
           |                                                                |
     6e+06 ++                                                               |
           |                                                                |
     4e+06 ++                                                               |
           |                                                                |
     2e+06 *+.....    ...*.......*......*......*......*.......*......    ...*
           |      *...                                               *...   |
         0 ++---------------------------------------------------------------+

\f
                          lock_stat.&pipe->wait.acquisitions

   1.4e+07 ++-----O---------------------------------------------------------+
           O             O       O             O                            |
   1.2e+07 ++                           O             O       O      O      O
           |                                                                |
     1e+07 ++                                                               |
           |                                                                |
     8e+06 ++                                                               |
           |                                                                |
     6e+06 ++                                                               |
           |                                                                |
     4e+06 ++                                                               |
           |                                                                |
     2e+06 ++         ...*.......           ...*......                      |
           *......*...           *......*...          *.......*......*......*
         0 ++---------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              1591860.90      +727.3%  13168894.70  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
              1278833.00      +888.4%  12639854.20  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
              2870693.90      +799.0%  25808748.90  TOTAL lock_stat.&pipe->wait.acquisitions
\f
                         lock_stat.slock-AF_INET/1.acq-bounces

   250000 ++----------------------------------------------------------------+
          |                                                                 |
          |   ...*.......*......*......            ...*......*.......*......*
   200000 *+..                         *.......*...                         |
          |                                                                 |
          |                                                                 |
   150000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
          |                                                                 |
    50000 ++                                                                |
          |                                                                 |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

\f
                        lock_stat.slock-AF_INET/1.acq-bounces

   35000 ++-----------------------------------------------------------------+
         |     .*..                                                         |
   30000 ++  ..    .                                                        |
         | ..       ..                                                      |
   25000 *+           ..      .*.....                                       |
         |                ....       ..    ...*.......                      |
   20000 ++             *.             *...           *......*.......*..... |
         |                                                                 .|
   15000 ++                                                                 *
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                         lock_stat.slock-AF_INET/1.acq-bounces

   250000 ++----------------------------------------------------------------+
          |                                                                 |
          *......*.......*......*......*.......*......*......*.......*......*
   200000 ++                                                                |
          |                                                                 |
          |                                                                 |
   150000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
          |                                                                 |
    50000 ++                                                                |
          |                                                                 |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

\f
                         lock_stat.slock-AF_INET/1.acq-bounces

   250000 ++----------------------------------------------------------------+
          |                                                                 |
          *......*.......*......*......*.......*......    ...*.......*......*
   200000 ++                                          *...                  |
          |                                                                 |
          |                                                                 |
   150000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
          |                                                                 |
    50000 ++                                                                |
          |                                                                 |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               208959.60       -99.9%       116.60  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                22408.80       -99.7%        69.10  lkp-a04/micro/netperf/120s-200%-TCP_RR
               210497.10       -99.9%       127.20  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               211988.90      -100.0%        49.00  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               653854.40       -99.9%       361.90  TOTAL lock_stat.slock-AF_INET/1.acq-bounces
\f
                      lock_stat.slock-AF_INET/1.waittime-total

   35000 ++-----------------------------------------------------------------+
         |   ...*.......    ...*.......                   ...*.......    ...*
   30000 *+..           *...           *......*.......*...           *...   |
         |                                                                  |
   25000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                      lock_stat.slock-AF_INET/1.waittime-total

   80000 ++-----------------------------------------------------------------+
         |          ....*......                                             |
   70000 *+.....*...           *.......*......*.......*......*.......*......*
   60000 ++                                                                 |
         |                                                                  |
   50000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
   20000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                      lock_stat.slock-AF_INET/1.waittime-total

   35000 ++-------------*---------------------------------------------------+
         |   ...*...          .            ...*.......    ...*.......       |
   30000 *+..                  *.......*...           *...           *......*
         |                                                                  |
   25000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                30746.35       -99.9%        17.12  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                70695.11       -99.9%        35.40  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                31227.43      -100.0%         6.30  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               132668.89      -100.0%        58.82  TOTAL lock_stat.slock-AF_INET/1.waittime-total
\f
                   lock_stat.slock-AF_INET.contentions.tcp_v4_rcv

   30000 ++-----------------------------------------------------------------+
         |                                                                  |
   25000 ++  ...*.......*......*.......*......    ....*......*.......*......*
         *...                                 *...                          |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                   lock_stat.slock-AF_INET.contentions.tcp_v4_rcv

   90000 ++-----------------------------------------------------------------+
         *......*.......*......*.......*......*.......*......*.......*......*
   80000 ++                                                                 |
   70000 ++                                                                 |
         |                                                                  |
   60000 ++                                                                 |
   50000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
   30000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
   10000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                   lock_stat.slock-AF_INET.contentions.tcp_v4_rcv

   30000 ++-----------------------------------------------------------------+
         |   ...*.......                                                    |
   25000 *+..           *......*.......*......*.......*......*.......*......*
         |                                                                  |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                25079.50       -99.9%        21.20  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                84262.50       -99.9%        57.20  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                26139.90      -100.0%         8.10  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               135481.90       -99.9%        86.50  TOTAL lock_stat.slock-AF_INET.contentions.tcp_v4_rcv
\f
                lock_stat.slock-AF_INET.contentions.lock_sock_nested

   25000 ++-----------------------------------------------------------------+
         |                                                                  |
         |   ...*.......*......*.......           ....*......*.......    ...*
   20000 *+..                          *......*...                   *...   |
         |                                                                  |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                lock_stat.slock-AF_INET.contentions.lock_sock_nested

   80000 ++-----------------------------------------------------------------+
         *......*.......*......*.......           ....*......*.......*......*
   70000 ++                            *......*...                          |
   60000 ++                                                                 |
         |                                                                  |
   50000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
   20000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                lock_stat.slock-AF_INET.contentions.lock_sock_nested

   25000 ++-----------------------------------------------------------------+
         |                                                                  |
         *......*.......*......*.......*......*.......    ...*.......*......*
   20000 ++                                           *...                  |
         |                                                                  |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                20479.50       -99.9%        18.70  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                71941.30       -99.9%        48.90  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                21464.90      -100.0%         7.40  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               113885.70       -99.9%        75.00  TOTAL lock_stat.slock-AF_INET.contentions.lock_sock_nested
\f
                          lock_stat.slock-AF_INET.acq-bounces

   250000 ++----------------------------------------------------------------+
          |                                                                 |
          |   ...*.......*......*......            ...*......            ...*
   200000 *+..                         *.......*...          *.......*...   |
          |                                                                 |
          |                                                                 |
   150000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
          |                                                                 |
    50000 ++                                                                |
          |                                                                 |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

\f
                         lock_stat.slock-AF_INET.acq-bounces

   35000 ++-----------------------------------------------------------------+
         |     .*..                                                         |
   30000 ++  ..    .                                                        |
         | ..       ..                                                      |
   25000 *+           ..      .*.....                                       |
         |                ....       ..    ...*.......                      |
   20000 ++             *.             *...           *......*.......*..... |
         |                                                                 .|
   15000 ++                                                                 *
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                          lock_stat.slock-AF_INET.acq-bounces

   250000 ++----------------------------------------------------------------+
          |                                                                 |
          *......*.......*......*......            ...*......*.......*......*
   200000 ++                           *.......*...                         |
          |                                                                 |
          |                                                                 |
   150000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
          |                                                                 |
    50000 ++                                                                |
          |                                                                 |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

\f
                          lock_stat.slock-AF_INET.acq-bounces

   250000 ++----------------------------------------------------------------+
          |                                                                 |
          *......*.......    ...*......    ....*......    ...*.......*......|
   200000 ++             *...          *...           *...                  *
          |                                                                 |
          |                                                                 |
   150000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
          |                                                                 |
    50000 ++                                                                |
          |                                                                 |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

\f
                         lock_stat.slock-AF_INET.acq-bounces

   70000 ++-----------------------------------------------------------------+
         |                                                                  |
   60000 ++            .*......    ....*..           .*..                   |
         |          ...        *...       ..       ..    ..                 |
   50000 ++       ..                        ..   ..        ..               |
         |     .*.                             ..                        ...*
   40000 ++....                               *              *.......*...   |
         *.                                                                 |
   30000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                          lock_stat.slock-AF_INET.acq-bounces

   160000 ++----------------------------------------------------------------+
          *......*.......*......*......*.......*......*......*.......*......*
   140000 ++                                                                |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
    80000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
    40000 ++                                                                |
          |                                                                 |
    20000 ++                                                                |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               204210.00       -99.9%       117.50  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                22440.80       -99.7%        71.30  lkp-a04/micro/netperf/120s-200%-TCP_RR
               206935.50       -99.9%       127.40  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               207040.80      -100.0%        51.70  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                47692.20       -99.9%        67.80  lkp-a04/micro/netperf/120s-200%-UDP_RR
               146215.80       -99.8%       259.80  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               834535.10       -99.9%       695.50  TOTAL lock_stat.slock-AF_INET.acq-bounces
\f
                         lock_stat.slock-AF_INET.con-bounces

   30000 ++-----------------------------------------------------------------+
         |                                                                  |
   25000 ++  ...*.......*......*.......*......    ....*......*.......*......*
         *...                                 *...                          |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                         lock_stat.slock-AF_INET.con-bounces

   90000 ++-----------------------------------------------------------------+
         *......*.......*......*.......*......*.......*......*.......*......*
   80000 ++                                                                 |
   70000 ++                                                                 |
         |                                                                  |
   60000 ++                                                                 |
   50000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
   30000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
   10000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                         lock_stat.slock-AF_INET.con-bounces

   30000 ++-----------------------------------------------------------------+
         |   ...*.......                                                    |
   25000 *+..           *......*.......*......*.......*......*.......*......*
         |                                                                  |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                25091.80       -99.9%        21.20  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                84265.90       -99.9%        57.20  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                26151.60      -100.0%         8.10  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               135509.30       -99.9%        86.50  TOTAL lock_stat.slock-AF_INET.con-bounces
\f
                lock_stat.((&icsk->icsk_delack_timer)).holdtime-total

   45000 ++-----------------------------------*-----------------------------+
         *......*.......    ...*.......*...         ..    ...*.......*......*
   40000 ++             *...                          *...                  |
   35000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
   25000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                    O                                            |
    5000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O--------------O------O-------O------O-------O------O

\f
                lock_stat.((&icsk->icsk_delack_timer)).holdtime-total

   40000 ++-----------------------------------------------------------------+
         |   ...*.......*......            ...*.......*.....                |
   35000 *+..                  *.......*...                 .    ....*......*
   30000 ++                                                  *...           |
         |                                                                  |
   25000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
   10000 ++                                                                 |
         |                                                                  |
    5000 ++                    O                                            |
         |                                                                  |
       0 O+-----O-------O--------------O------O-------O------O-------O------O

\f
                lock_stat.((&icsk->icsk_delack_timer)).holdtime-total

   45000 ++-----------------------------------------------------------------+
         |                                          ..*......*.......       |
   40000 *+.....*.......*......*.......*......  ....                 *......|
   35000 ++                                   *.                            *
         |                                                                  |
   30000 ++                                                                 |
   25000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
    5000 ++                                                                 |
         |                     O                                            |
       0 O+-----O-------O--------------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                41632.92       -95.9%      1721.45  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                35084.77       -96.9%      1104.23  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                38759.33       -97.5%       982.84  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               115477.01       -96.7%      3808.53  TOTAL lock_stat.((&icsk->icsk_delack_timer)).holdtime-total
\f
                         lock_stat.slock-AF_INET.contentions

   30000 ++-----------------------------------------------------------------+
         |                                                                  |
   25000 *+.....*.......*......*.......*......    ....*......*.......*......*
         |                                    *...                          |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                         lock_stat.slock-AF_INET.contentions

   90000 ++-----------------------------------------------------------------+
         *......*.......*......*.......*......*.......*......*.......*......*
   80000 ++                                                                 |
   70000 ++                                                                 |
         |                                                                  |
   60000 ++                                                                 |
   50000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
   30000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
   10000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                         lock_stat.slock-AF_INET.contentions

   30000 ++-----------------------------------------------------------------+
         |   ...*.......                                                    |
   25000 *+..           *......*.......*......*.......*......*.......*......*
         |                                                                  |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                25092.60       -99.9%        21.20  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                84266.80       -99.9%        57.20  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                26151.90      -100.0%         8.10  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               135511.30       -99.9%        86.50  TOTAL lock_stat.slock-AF_INET.contentions
\f
                         lock_stat.&rq->lock.waittime-total

   30000 ++-----------------------------------------------------------------+
         |                    .*.....             ....*......*.......       |
         |   ...*.......  ....       ..    ...*...                   *......*
   25000 *+..           *.             *...                                 |
         |                                                                  |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
         |                                                                  |
   10000 ++                                                                 |
         |              O                     O       O                     O
         O      O              O       O                     O       O      |
    5000 ++-----------------------------------------------------------------+

\f
                          lock_stat.&rq->lock.waittime-total

   160000 ++----------------------------------------------------------------+
          |   ...*.......*......*......    ....*......                      |
   140000 *+..                         *...           *......*.......*......*
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
    80000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
    40000 ++                                                                |
          |                                                                 |
    20000 ++                                                                |
          O      O       O      O      O       O      O      O       O      O
        0 ++----------------------------------------------------------------+

\f
                          lock_stat.&rq->lock.waittime-total

   250000 ++----------------------------------------------------------------+
          |      *..                                                        |
          |    ..   .                                                       |
   200000 ++ ..      ..                                                     |
          |..          ..      .*.....       ..*.....                       |
          *                ....       .  ....        .                      |
   150000 ++             *.            *.             *......*.......       |
          |                                                          *......|
   100000 ++                                                                *
          |                                                                 |
          |                                                                 |
    50000 ++                                                                |
          |                                                                 |
          O      O       O             O       O             O              O
        0 ++--------------------O---------------------O--------------O------+

\f
                          lock_stat.&rq->lock.waittime-total

   110000 ++----------------------------------------------------------------+
   100000 ++         ....*......*......*.......    ...*......    ....*......|
          *......*...                          *...          *...           *
    90000 ++                                                                |
    80000 ++                                                                |
    70000 ++                                                                |
    60000 ++                                                                |
          |                                                                 |
    50000 ++                                                                |
    40000 ++                                                                |
    30000 ++                                                                |
    20000 ++                                                                |
          |                                                                 |
    10000 O+     O       O      O      O       O      O      O       O      O
        0 ++----------------------------------------------------------------+

\f
                          lock_stat.&rq->lock.waittime-total

   160000 ++----------------------------------------------------------------+
          |   ...*.......    ...*......*.......*......    ...*.......*..... |
   140000 *+..           *...                         *...                 .|
   120000 ++                                                                *
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
    80000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
    40000 ++                                                                |
          |                                                                 |
    20000 ++                                                                |
          O                     O              O      O      O       O      |
        0 ++-----O-------O-------------O------------------------------------O

\f
                          lock_stat.&rq->lock.waittime-total

   250000 ++----------------------------------------------------------------+
          |                                                                 |
          |             .*.....       .*..            *..                   |
   200000 ++         ...       .  ....    ..        ..   ..                 |
          |        ..           *.          ..    ..       ..               |
          |     .*.                           . ..                         .*
   150000 ++....                               *             *.......  .... |
          *.                                                         *.     |
   100000 ++                                                                |
          |                                                                 |
          |                                                                 |
    50000 ++                                                                |
          |                                                                 |
          O      O       O             O       O      O      O       O      O
        0 ++--------------------O-------------------------------------------+

\f
                          lock_stat.&rq->lock.waittime-total

   200000 ++----------------------------------------------------------------+
   180000 *+.....*.......*......*......*.......*......    ...*.......*......*
          |                                           *...                  |
   160000 ++                                                                |
   140000 ++                                                                |
          |                                                                 |
   120000 ++                                                                |
   100000 ++                                                                |
    80000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
    40000 ++                                                                |
          |                                                                 |
    20000 ++     O                     O       O                     O      |
        0 O+-------------O------O---------------------O------O--------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                26354.46       -70.3%      7833.85  lkp-a04/micro/netperf/120s-200%-TCP_CRR
               142837.18       -94.4%      8005.81  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
               154375.19       -93.8%      9600.79  lkp-a04/micro/netperf/120s-200%-TCP_RR
                97226.01       -93.0%      6824.98  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               142269.17       -95.6%      6218.05  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               170848.76       -93.3%     11496.83  lkp-a04/micro/netperf/120s-200%-UDP_RR
               182420.08       -96.2%      7011.02  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               916330.86       -93.8%     56991.34  TOTAL lock_stat.&rq->lock.waittime-total
\f
                       lock_stat.slock-AF_INET.waittime-total

   2500 ++------------------------------------------------------------------+
        |                                                    *..            |
        |                                                  ..   .           |
   2000 ++                                               ..      .          |
        |                  ....*......                  .         ..        |
        |             .*...           *.......        ..                    |
   1500 ++        ....                        *......*              *.......|
        |   ....*.                                                          *
   1000 *+..                                                                |
        |                                     O                             |
        |                                                                   |
    500 ++      O                                    O                      |
        |              O       O      O                      O              O
        O                                                           O       |
      0 ++------------------------------------------------------------------+

\f
                        lock_stat.slock-AF_INET.waittime-total

   1.6e+06 ++---------------------------------------------------------------+
           |                                                                |
   1.4e+06 *+.....*......*.......*......*......*......*.......*......*......*
           |                                                                |
   1.2e+06 ++                                                               |
     1e+06 ++                                                               |
           |                                                                |
    800000 ++                                                               |
           |                                                                |
    600000 ++                                                               |
    400000 ++                                                               |
           |                                                                |
    200000 ++                                                               |
           |                                                                |
         0 O+-----O------O-------O------O------O------O-------O------O------O

\f
                        lock_stat.slock-AF_INET.waittime-total

   4.5e+06 ++---------------------------------------------------------------+
           *......*......*.......*......*......    ...*.......*......*......*
     4e+06 ++                                  *...                         |
   3.5e+06 ++                                                               |
           |                                                                |
     3e+06 ++                                                               |
   2.5e+06 ++                                                               |
           |                                                                |
     2e+06 ++                                                               |
   1.5e+06 ++                                                               |
           |                                                                |
     1e+06 ++                                                               |
    500000 ++                                                               |
           |                                                                |
         0 O+-----O------O-------O------O------O------O-------O------O------O

\f
                        lock_stat.slock-AF_INET.waittime-total

   1.6e+06 ++---------------------------------------------------------------+
           *......*......*.......*......*......*......    ....*......*......*
   1.4e+06 ++                                         *...                  |
           |                                                                |
   1.2e+06 ++                                                               |
     1e+06 ++                                                               |
           |                                                                |
    800000 ++                                                               |
           |                                                                |
    600000 ++                                                               |
    400000 ++                                                               |
           |                                                                |
    200000 ++                                                               |
           |                                                                |
         0 O+-----O------O-------O------O------O------O-------O------O------O

\f
                       lock_stat.slock-AF_INET.waittime-total

   20000 ++-----------------------------------------------------------------+
   18000 *+.....                           ...*.......*......    ....*......|
         |      *.......*......*.......*...                  *...           *
   16000 ++                                                                 |
   14000 ++                                                                 |
         |                                                                  |
   12000 ++                                                                 |
   10000 ++                                                                 |
    8000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
    4000 ++                                                                 |
         |                                                                  |
    2000 ++                                                                 |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 1520.14       -73.2%       408.10  lkp-a04/micro/netperf/120s-200%-TCP_CRR
              1394498.03       -99.9%      1117.05  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
              4113384.81       -99.9%      2526.88  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
              1466930.10      -100.0%       508.92  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                17715.59       -99.6%        77.31  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
              6994048.67       -99.9%      4638.26  TOTAL lock_stat.slock-AF_INET.waittime-total
\f
              lock_stat.((&icsk->icsk_retransmit_timer)).holdtime-total

   8000 ++------------------------------------------------------------------+
        |                                                                   O
   7000 ++                                                                  |
   6000 ++                                                                  |
        |                                                                   |
   5000 ++                                                                  |
        |                                                                   |
   4000 ++                                                                  |
        |                                                                   |
   3000 ++                                                                  |
   2000 ++      O                    .*...           O                      |
        O              O       O   ..     ...                               |
   1000 ++                       ..   O      .               O      O       |
        *.......*......*.......*.             *......*.......*......*.......*
      0 ++------------------------------------------------------------------+

\f
              lock_stat.((&icsk->icsk_retransmit_timer)).holdtime-total

   45000 ++-----------------------------------------------------------------+
         |                                   .*...                          |
   40000 ++                 ...*.......  ....     ...            ....*......|
   35000 *+.....*.......*...           *.            .    ...*...           *
         |                                            *...                  |
   30000 ++                                                                 |
   25000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
    5000 ++                                                                 |
         |                                                   O              O
       0 O+-----O-------O------O-------O------O-------O--------------O------+

\f
              lock_stat.((&icsk->icsk_retransmit_timer)).holdtime-total

   2500 ++------------------------------------------------------------------+
        *.....                                             ..*..            |
        |     ..    ...*.......*.....                  ....     ..          |
   2000 ++      *...                 .              .*.           ..        |
        |                             *.....      ..                    ....*
        |                                   ..  ..                  *...    |
   1500 ++                                    *.                            |
        |                                                                   |
   1000 ++                                                                  |
        |                                                                   |
        |                                     O                             |
    500 ++      O              O                     O              O       |
        O              O              O                      O              O
        |                                                                   |
      0 ++------------------------------------------------------------------+

\f
              lock_stat.((&icsk->icsk_retransmit_timer)).holdtime-total

   45000 ++-----------------------------------------------------------------+
         |                                           .*.....                |
   40000 ++                                       ...       .    ....*...   |
   35000 *+.....    ....*......*.......*......  ..           *...        .. |
         |      *...                          *.                           .|
   30000 ++                                                                 *
   25000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
    5000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  580.45      +277.4%      2190.71  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                36070.55       -95.8%      1516.35  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                 1938.98       -77.1%       444.98  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                34328.78       -99.0%       349.70  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                72918.76       -93.8%      4501.73  TOTAL lock_stat.((&icsk->icsk_retransmit_timer)).holdtime-total
\f
                         lock_stat.&pipe->wait.waittime-max

   12000 ++-----------------------------------------------------------------+
         |                                                   O              |
   10000 ++                                                                 |
         |                                                                  |
         |                                                                  |
    8000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
         |                                                                  |
    4000 ++                                                                 O
         |                                                                  |
         O              O              O                                    |
    2000 ++           ..*..... O                                 ....*..... |
         |      O ....        .            ...*.......*......*...          .|
       0 *+-----*--------------*-------*-----------------------------O------*

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  561.79      +366.8%      2622.34  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                  561.79      +366.8%      2622.34  TOTAL lock_stat.&pipe->wait.waittime-max
\f
                       lock_stat.(&task->u.tk_work).holdtime-max

   500000 ++-------------O--------------------------------------------------+
   450000 ++                                                                |
          |                                                                 O
   400000 ++                                                                |
   350000 ++                                                                |
          |                                                                 |
   300000 ++                                   *      O                     |
   250000 ++                                  + :                           |
   200000 ++                                 +   :                          |
          |                                 +     :                         |
   150000 O+                               +       :                        |
   100000 ++                              +    O   :                        |
          |                              +          :                       |
    50000 ++                 ...*......O+            :       O       O      |
        0 *+-----*-------*------O------*--------------*------*-------*------*

\f
                       lock_stat.(&task->u.tk_work).holdtime-max

   100000 ++----------------------------------------------------------------+
    90000 O+                                                                |
          |                                                                 |
    80000 ++                                                                |
    70000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
    50000 ++                                                                |
    40000 ++                                                                |
          |                                                                 |
    30000 ++                                   O                            |
    20000 ++                    O                            O       O      |
          |      O       O             O                    .*.......       O
    10000 *+.....    ....*......*......    ....*......O ....         *......|
        0 ++-----*---------------------*--------------*---------------------*

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                33202.91      +385.8%    161283.94  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                 4652.25      +437.9%     25026.43  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                37855.16      +392.2%    186310.37  TOTAL lock_stat.(&task->u.tk_work).holdtime-max
\f
                            lock_stat.rpciod-R.holdtime-max

   500000 ++-------------O--------------------------------------------------+
   450000 ++                                                                |
          |                                                                 O
   400000 ++                                                                |
   350000 ++                                                                |
          |                                                                 |
   300000 ++                                   *      O                     |
   250000 ++                                  + :                           |
   200000 ++                                 +   :                          |
          |                                 +     :                         |
   150000 O+                               +       :                        |
   100000 ++                              +    O   :                        |
          |                              +          :                       |
    50000 ++                 ...*......O+            :       O       O      |
        0 *+-----*-------*------O------*--------------*------*-------*------*

\f
                            lock_stat.rpciod-R.holdtime-max

   100000 ++----------------------------------------------------------------+
    90000 O+                                                                |
          |                                                                 |
    80000 ++                                                                |
    70000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
    50000 ++                                                                |
    40000 ++                                                                |
          |                                                                 |
    30000 ++                                   O                            |
    20000 ++                    O                            O       O      |
          |      O       O             O                    .*.......       O
    10000 *+.....    ....*......*......    ....*......O ....         *......|
        0 ++-----*---------------------*--------------*---------------------*

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                33203.58      +385.7%    161284.69  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                 4652.87      +437.9%     25027.06  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                37856.44      +392.2%    186311.75  TOTAL lock_stat.rpciod-R.holdtime-max
\f
                           lock_stat.&rq->lock.acq-bounces

   80000 ++-----------------------------------------------------------------+
         |                                                                  |
   70000 ++  ...*.......*......*.......           ....*......*.......*......*
   60000 *+..                          *......*...                          |
         |                                                                  |
   50000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
   20000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         O      O       O      O       O      O       O      O       O      O
       0 ++-----------------------------------------------------------------+

\f
                           lock_stat.&rq->lock.acq-bounces

   80000 ++-----------------------------------------------------------------+
         |     .*..                                                         |
   70000 ++  ..    .                                                        |
   60000 ++..       ..                                                      |
         *.           ..      .*.....                                       |
   50000 ++               ....       ..    ...*.......                      |
         |              *.             *...           *......*.......*......|
   40000 ++                                                                 *
         |                                                                  |
   30000 ++                                                                 |
   20000 ++                                                                 |
         |                                                                  |
   10000 ++     O       O              O      O                             |
         O                     O                      O      O       O      O
       0 ++-----------------------------------------------------------------+

\f
                           lock_stat.&rq->lock.acq-bounces

   90000 ++-------------*--------------*---------------------*-------*------+
         *......*...           *...           *.......*...                  *
   80000 ++                                                                 |
   70000 ++                                                                 |
         |                                                                  |
   60000 ++                                                                 |
   50000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
   30000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
   10000 ++                                                                 |
         O      O       O      O       O      O       O      O       O      O
       0 ++-----------------------------------------------------------------+

\f
                           lock_stat.&rq->lock.acq-bounces

   80000 ++-----------------------------------------------------------------+
         *......            ...*.......*......            ...*.......       |
   70000 ++     *.......*...                  *.......*...           *......*
   60000 ++                                                                 |
         |                                                                  |
   50000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
   20000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         O      O       O      O       O      O       O      O       O      O
       0 ++-----------------------------------------------------------------+

\f
                           lock_stat.&rq->lock.acq-bounces

   70000 ++-------------*-----------------------------*---------------------+
         |          ...       .    ....*...        ..    .                  |
   60000 ++       ..           *...        ..    ..       .                 |
         |     .*.                           . ..          ..               |
   50000 ++  ..                               *                          ...*
         | ..                                                *.......*...   |
   40000 *+                                                                 |
         |                                                                  |
   30000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   10000 O+     O       O              O      O       O      O       O      O
         |                     O                                            |
       0 ++-----------------------------------------------------------------+

\f
                           lock_stat.&rq->lock.acq-bounces

   90000 ++-----------------------------------------------------------------+
         *......*.......*......                           ...*.......       |
   80000 ++                    *.......*......*.......*...           *......*
   70000 ++                                                                 |
         |                                                                  |
   60000 ++                                                                 |
   50000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
   30000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
   10000 ++                                                                 |
         O      O       O      O       O      O       O      O       O      O
       0 ++-----------------------------------------------------------------+

\f
                             lock_stat.&rq->lock.acq-bounces

   1.55e+06 ++--------------------------------------------------------------+
            |                                                               |
    1.5e+06 *+.....    ...*.....                                            |
            |      *...         .    ...*.......    ...*......*......    ...*
   1.45e+06 ++                   *...           *...                 *...   |
            |                                                               |
    1.4e+06 ++                                                              |
            |                                                               |
   1.35e+06 ++                                                              |
            |                                                               |
    1.3e+06 ++                                                              |
            O      O      O      O                            O             |
   1.25e+06 ++                                  O      O             O      |
            |                           O                                   O
    1.2e+06 ++--------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                66958.10       -90.4%      6423.30  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                50836.70       -83.8%      8238.00  lkp-a04/micro/netperf/120s-200%-TCP_RR
                86072.10       -92.7%      6295.30  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                70723.30       -92.0%      5683.70  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                54857.30       -82.9%      9361.30  lkp-a04/micro/netperf/120s-200%-UDP_RR
                79910.70       -90.7%      7401.20  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
              1473367.80       -14.6%   1257653.00  nhm-white/micro/aim7/misc_rtns_1
              1882726.00       -30.9%   1301055.80  TOTAL lock_stat.&rq->lock.acq-bounces
\f
                     lock_stat.slock-AF_INET-RPC.waittime-total

   9000 ++------------------------------------------------------------------+
        |                                                                   O
   8000 ++                            O                                     |
   7000 ++                                                                  |
        |                                                                   |
   6000 ++                                                                  |
   5000 ++                                                                  |
        |                                                                   |
   4000 ++                                                                  |
   3000 ++                                                                  |
        |                      O     .*...                   O      O       |
   2000 O+      O                  ..     ... O      O                      |
   1000 ++             O         ..          .                              |
        *.......*......*.......*.             *......*.......*......*.......*
      0 ++------------------------------------------------------------------+

\f
                     lock_stat.slock-AF_INET-RPC.waittime-total

   3000 ++------------------------------------------------------------------+
        |              O                                                    |
   2500 ++                                                                  O
        |                      O                                            |
        O                                            O                      |
   2000 ++      O                     O                                     |
        |                                     O              O              |
   1500 ++                                                                  |
        |                                    .*..                           |
   1000 ++                                 ..    ..                 O       |
        |                                ..        ..                       |
        *.....                         ..                                   |
    500 ++    ..    ...*.......*......*              *.......*......*.......*
        |       *...                                                        |
      0 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  682.48      +392.3%      3359.61  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                  510.26      +292.9%      2004.85  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                 1192.74      +349.8%      5364.45  TOTAL lock_stat.slock-AF_INET-RPC.waittime-total
\f
                         lock_stat.&rq->lock/1.holdtime-total

   280000 ++----------------------------------------------------------------+
          |                                                                 |
   260000 O+                                                 O              O
          |                                                                 |
   240000 ++                                                                |
          |              O      O             .*.                           |
   220000 ++     O              *.....     ...   .                   O      |
          |                   ..      .  ..       ..                        |
   200000 ++                ..         *.      O    . O                     |
          |               ..                         .      .*.......*...   |
   180000 *+..          .*                              ....             .. |
          |   ..     ...                              *.                   .|
   160000 ++    .  ..                                                       *
          |      *.                                                         |
   140000 ++----------------------------------------------------------------+

\f
                        lock_stat.&rq->lock/1.holdtime-total

   55000 ++-----------------------------------------------------------------+
   50000 ++                   .*.......           ....*......*.....         |
         |                ....         *......*...                 ..    ...*
   45000 *+.....*.......*.                                           *...   |
   40000 ++                                                                 |
   35000 ++                                                                 |
   30000 ++                                                                 |
         |                                                                  |
   25000 ++                                                                 |
   20000 ++                                                                 |
   15000 ++                                                                 |
   10000 ++                                                                 |
         |                                    O                             |
    5000 O+     O       O      O       O              O      O       O      O
       0 ++-----------------------------------------------------------------+

\f
                         lock_stat.&rq->lock/1.holdtime-total

   250000 ++----------------------------------------------------------------+
          |                                                                 |
          |   ...*.......*......           ....*......                      |
   200000 *+..                  *......*...           *......*.......       |
          |                                                          *......*
          |                                                                 |
   150000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
          |                                                                 |
    50000 ++                                                                |
          |                                                                 |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

\f
                        lock_stat.&rq->lock/1.holdtime-total

   50000 ++--------------------------------------------------*--------------+
   45000 ++  ...*.......    ...*.......*......          ....       ..       |
         *...           *...                  *.......*.             *......|
   40000 ++                                                                 *
   35000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
   25000 ++                                                                 |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
   10000 ++                                                                 |
         |                                                                  |
    5000 O+     O       O      O       O      O       O      O       O      O
       0 ++-----------------------------------------------------------------+

\f
                         lock_stat.&rq->lock/1.holdtime-total

   160000 ++----------------------------------------------------------------+
          |                                          .*.....       ..*..... |
   140000 *+.....    ....*......*......          ....       .  ....        .|
   120000 ++     *...                  *.......*.            *.             *
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
    80000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
    40000 ++                                                                |
          |                                                                 |
    20000 ++                                                                |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

\f
                         lock_stat.&rq->lock/1.holdtime-total

   250000 ++----------------------------------------------------------------+
          |                                                                 |
          |                                                                 |
   200000 ++  ...*.......    ...*......*.......    ...*......*.......       |
          *...           *...                  *...                  *......|
          |                                                                 *
   150000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
          |                                                                 |
    50000 ++                                                                |
          |                                                                 |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

\f
                        lock_stat.&rq->lock/1.holdtime-total

   45000 ++-----------------------------------------------------------------+
         |                                                                  |
   40000 ++         ....*......*.......             ..*.....               .*
   35000 ++    .*...                   *......  ....        .            .. |
         | ....                               *.             *.......  ..   |
   30000 *+                                                          *.     |
   25000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
    5000 ++                                                                 |
         O      O       O      O       O      O       O      O       O      O
       0 ++-----------------------------------------------------------------+

\f
                         lock_stat.&rq->lock/1.holdtime-total

   200000 ++----------------------------------------------------------------+
   180000 *+.....                            ..*.....                    ...*
          |      *.....        .*......  ....        .           ....*...   |
   160000 ++           ..  ....        *.             *......*...           |
   140000 ++             *.                                                 |
          |                                                                 |
   120000 ++                                                                |
   100000 ++                                                                |
    80000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
    40000 ++                                                                |
          |                                                                 |
    20000 ++                                                                |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

\f
                        lock_stat.&rq->lock/1.holdtime-total

   90000 ++--------------------O--------------------------------------------+
         |                                                                  O
   85000 O+     O       O              O      O       O      O              |
   80000 ++                                                          O      |
         |                                                                  |
   75000 ++                                                                 |
         |                                                                  |
   70000 ++                                                                 |
         |                                                                  |
   65000 ++                                                                 |
   60000 ++                                                                 |
         |                                                                  |
   55000 *+.....*.......*......*.......                                  ...*
         |                             *......*.......*......*.......*...   |
   50000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               188309.55       +20.6%    227184.30  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                46756.77       -92.1%      3706.54  lkp-a04/micro/netperf/120s-200%-TCP_CRR
               198118.01       -98.8%      2449.77  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                44517.06       -92.7%      3267.74  lkp-a04/micro/netperf/120s-200%-TCP_RR
               135375.74       -98.5%      2097.69  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               185664.22       -98.7%      2354.57  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                35297.09       -90.0%      3527.14  lkp-a04/micro/netperf/120s-200%-UDP_RR
               168513.62       -98.3%      2910.05  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                54579.81       +56.0%     85131.96  nhm-white/micro/aim7/misc_rtns_1
              1057131.87       -68.5%    332629.78  TOTAL lock_stat.&rq->lock/1.holdtime-total
\f
                   lock_stat.&rq->lock.contentions.try_to_wake_up

   45000 ++-----*-----------------------------------------------------------+
         |    ..  ..                                                        |
   40000 ++  .      .                                                       |
   35000 ++ .        .                                                      |
         |..          ..                                                    |
   30000 *+                 ...*.....        .*.....                        |
   25000 ++             *...         ..  ....       ..                      |
         |                             *.             *......*.......*......|
   20000 ++                                                                 *
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
    5000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                   lock_stat.&rq->lock.contentions.try_to_wake_up

   45000 ++-------------------------------------------*---------------------+
         |             .*.....       ..*..          ..  .                   |
   40000 ++         ...       .  ....     ..      ..     ..                 |
   35000 ++       ..           *.           ..   .         .                |
         |     .*.                             ..           .               |
   30000 ++  ..                               *                          ...*
   25000 ++..                                                *.......*...   |
         *.                                                                 |
   20000 ++                                                                 |
   15000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
    5000 ++                                                                 |
         |                                                                  |
       0 O+-----O-------O------O-------O------O-------O------O-------O------O

\f
                   lock_stat.&rq->lock.contentions.try_to_wake_up

   14000 ++-----------------------------------------------------------------+
         |                                                                  |
   12000 ++  ...*.......*......                           ...*.......*......|
         *...                  *.......*......*.......*...                  *
   10000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
         |                                                                  |
    4000 ++                                                                 |
         |                                                                  |
    2000 ++                                                                 |
         O      O                      O                                    |
       0 ++-------------O------O--------------O-------O------O-------O------O

\f
                    lock_stat.&rq->lock.contentions.try_to_wake_up

   200000 *+----------------------------------------------------------------+
          |   ..                      .*.....                               |
   180000 O+    .    ....*......  ....       ..    ...*              *......|
   160000 ++     *...           *.     O       *...    :            :       *
          |                     O                      :            :       O
   140000 ++                                            :          :        |
   120000 ++                                            :         :         |
          |      O                                       :        :         |
   100000 ++                                   O          :      :          |
    80000 ++             O                                :     :           |
          |                                           O    :    :           |
    60000 ++                                               :   :            |
    40000 ++                                                : :             |
          |                                                 :O:             |
    20000 ++-------------------------------------------------*--------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                26962.80       -97.2%       765.80  lkp-a04/micro/netperf/120s-200%-TCP_RR
                32851.00       -96.7%      1086.10  lkp-a04/micro/netperf/120s-200%-UDP_RR
                11478.00       -96.7%       383.80  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               159558.30       -25.0%    119609.50  nhm-white/micro/aim7/misc_rtns_1
               230850.10       -47.2%    121845.20  TOTAL lock_stat.&rq->lock.contentions.try_to_wake_up
\f
                          lock_stat.&p->pi_lock.acquisitions

     5e+06 ++---------------------------------------------------------------+
           |             O              O      O              O             |
   4.5e+06 O+     O              O                    O              O      O
     4e+06 ++                                                               |
           |                                                                |
   3.5e+06 ++                                                               |
     3e+06 ++                                                               |
           |                                                                |
   2.5e+06 ++                                                               |
     2e+06 ++                                                               |
           |                                                                |
   1.5e+06 ++                                                               |
     1e+06 ++                                                               |
           *......*......*.......*......*......*......*.......*......*......*
    500000 ++---------------------------------------------------------------+

\f
                          lock_stat.&p->pi_lock.acquisitions

     5e+06 ++---------------------------------------------------------------+
           O      O                                                         |
   4.5e+06 ++            O       O             O              O      O      |
     4e+06 ++                           O             O                     O
           |                                                                |
   3.5e+06 ++                                                               |
     3e+06 ++                                                               |
           |                                                                |
   2.5e+06 ++                                                               |
     2e+06 ++                                                               |
           |                                                                |
   1.5e+06 ++                                                               |
     1e+06 ++                                                               |
           *......*......*.......*......*......*......*.......*......*......*
    500000 ++---------------------------------------------------------------+

\f
                          lock_stat.&p->pi_lock.acquisitions

   450000 ++----------------------------------------------------------------+
          *......*.......*......*......*.......*......    ...*.......*......*
   400000 ++                                          *...                  |
          |                                                                 |
          |                                                                 |
   350000 ++                                          O                     |
          |                                                                 |
   300000 ++                                                                |
          |                                                                 |
   250000 ++                                                                O
          |                     O              O                            |
          O                                                  O              |
   200000 ++             O             O                             O      |
          |      O                                                          |
   150000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               844661.80      +427.7%   4457102.10  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
               780146.30      +453.1%   4314785.00  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
               429746.00       -47.9%    223811.00  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              2054554.10      +337.8%   8995698.10  TOTAL lock_stat.&p->pi_lock.acquisitions
\f
                     lock_stat.&(&base->lock)->rlock.acq-bounces

   80000 ++-----------------------------------------------------------------+
         |          ....*......    ....*......                      .*..... |
   75000 ++  ...*...           *...           *.......            ..       .|
         *...                                         *.....    ..          *
   70000 ++                                                 . ..            |
         |                                                   *              |
   65000 ++     O                                                           O
         |                             O      O              O              |
   60000 O+                                           O                     |
         |                                                           O      |
   55000 ++                                                                 |
         |                     O                                            |
   50000 ++                                                                 |
         |              O                                                   |
   45000 ++-----------------------------------------------------------------+

\f
                     lock_stat.&(&base->lock)->rlock.acq-bounces

   90000 ++-----------------------------------------------------------------+
         |                             *.....                               |
   85000 ++            .*..          ..      .              .*.......       |
         |           ..    ..      ..         *.....      ..         *..    |
         *.....    ..        ..   .                 ..  ..              .   |
   80000 ++    . ..             ..                    *.                 .  |
         O      *              *                                          ..|
   75000 ++                                                                 |
         |      O                                                           *
   70000 ++             O                                    O              |
         |                     O                                            O
         |                             O              O                     |
   65000 ++                                                          O      |
         |                                    O                             |
   60000 ++-----------------------------------------------------------------+

\f
                     lock_stat.&(&base->lock)->rlock.acq-bounces

   18000 ++-----------------------------------------------------------------+
         |                                                ...*.....         |
   16000 ++                 ...*.......*......    ....*...         ..    ...*
   14000 ++  ...*.......*...                  *...                   *...   |
         *...                                                               |
   12000 ++                                                                 |
   10000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
    6000 ++                                                                 |
         |                                                                  |
    4000 ++                                                                 |
    2000 ++                                   O                             |
         O      O       O      O       O              O      O       O      O
       0 ++-----------------------------------------------------------------+

\f
                      lock_stat.&(&base->lock)->rlock.acq-bounces

   250000 ++----------------------------------------------------------------+
          |                                                                 |
          |   ...*.......*......*......*.......    ...*......*.......*......*
   200000 *+..                                 *...                         |
          |                                                                 |
          |                                                                 |
   150000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
          |                                                                 |
    50000 ++                                                                |
          |                                                                 |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

\f
                      lock_stat.&(&base->lock)->rlock.acq-bounces

   140000 ++-----*----------------------------------------------------------+
          |    ..  ..                                                       |
   120000 ++ ..      .                                                      |
          |..         .                                                     |
   100000 *+           ..      .*......    ....*.....                       |
          |                ....        *...          .                      |
    80000 ++             *.                           *......*.......*......|
          |                                                                 *
    60000 ++                                                                |
          |                                                                 |
    40000 ++                                                                |
          |                                                                 |
    20000 ++                                                                |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

\f
                      lock_stat.&(&base->lock)->rlock.acq-bounces

   300000 ++----------------------------------------------------------------+
          |          ....*......                                            |
   250000 *+.....*...           *......*.......*......*......*.......*......*
          |                                                                 |
          |                                                                 |
   200000 ++                                                                |
          |                                                                 |
   150000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
          |                                                                 |
    50000 ++                                                                |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

\f
                      lock_stat.&(&base->lock)->rlock.acq-bounces

   250000 ++----------------------------------------------------------------+
          *......*.......                  ....*......    ...*.......*......|
          |              *......*......*...           *...                  *
   200000 ++                                                                |
          |                                                                 |
          |                                                                 |
   150000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
          |                                                                 |
    50000 ++                                                                |
          |                                                                 |
          |                                                                 |
        0 O+-----O-------O------O------O-------O------O------O-------O------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                74212.80       -19.7%     59575.80  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                81567.00       -15.7%     68787.50  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                14777.20       -92.2%      1158.70  lkp-a04/micro/netperf/120s-200%-TCP_CRR
               211623.40       -99.5%      1063.20  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                91628.70       -98.7%      1162.00  lkp-a04/micro/netperf/120s-200%-TCP_RR
               262885.30       -99.7%       870.60  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               221375.70       -99.6%       875.30  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               958070.10       -86.1%    133493.10  TOTAL lock_stat.&(&base->lock)->rlock.acq-bounces
\f
                       lock_stat.&mm->mmap_sem-R.holdtime-max

   8000 ++------------------------------------------------------------------+
        |                      *                                    *       |
   7000 ++                    : +                                  : :      |
   6000 ++                   :   +                                 :  :     |
        |                    :    +                               :   :     |
   5000 *+                  :      +                             :     :    |
        | .                :        +                            :      :   |
   4000 ++ .              :          +                          :        :  |
        |   ..           :            *..                       :         : |
   3000 ++    .          :               .                     :          : |
   2000 ++     .        :                 ..                  :            :|
        |           ...*                    ..                :             *
   1000 O+      *...                                     ....*              |
        |              O       O      O       *......*...    O      O       O
      0 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 3001.10       -83.1%       508.26  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                 3001.10       -83.1%       508.26  TOTAL lock_stat.&mm->mmap_sem-R.holdtime-max
\f
                      lock_stat.zone->wait_table+i.waittime-total

   180000 ++----------------------------------------------------------------+
          |              O      O                                           |
   160000 O+     O                             O      O      O       O      |
   140000 ++                           O                                    O
          |                                                                 |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
    80000 ++                                                                |
    60000 ++                                                                |
          |                                                                 |
    40000 *+.....*.......*......*......*.......*......*......*.......*......*
          |                                                                 |
    20000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                41448.45      +277.4%    156411.45  nhm-white/micro/aim7/misc_rtns_1
                41448.45      +277.4%    156411.45  TOTAL lock_stat.zone->wait_table+i.waittime-total
\f
                      lock_stat.(&task->u.tk_work).holdtime-total

   700000 ++----------------------------------------------------------------+
          |              O                                                  O
   600000 ++                                          O                     |
          |                                                                 |
   500000 ++                                                                |
          |                                    O                            |
   400000 O+     O                                                          |
          |                                    *..                          |
   300000 ++                                 ..   .                  O      |
          |                            O   ..      .                        |
   200000 ++                              .         ..       O              |
          |                  ...*...... ..                                  |
   100000 ++  ...*.......*...          *              *......    ....*......*
          *...                                               *...           |
        0 ++----------------------------------------------------------------+

\f
                      lock_stat.(&task->u.tk_work).holdtime-total

   550000 ++-------------------------------------------------O--------------+
   500000 ++                                                                |
          |                                                                 |
   450000 O+                                                                |
   400000 ++                                                         O      |
          |                                                                 |
   350000 ++                                                                |
   300000 ++                                                                |
   250000 ++             O      O              O                            O
          |      O                                                          |
   200000 ++                           O              O                     |
   150000 ++                                                                |
          *.....                                                            |
   100000 ++    .    ....*......*......*.......*......*......*.......*......|
    50000 ++-----*----------------------------------------------------------*

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               121903.88      +235.8%    409344.82  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                84528.79      +263.4%    307162.60  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
               206432.67      +247.1%    716507.42  TOTAL lock_stat.(&task->u.tk_work).holdtime-total
\f
                           lock_stat.rpciod-R.holdtime-total

   700000 ++----------------------------------------------------------------+
          |              O                                                  O
   600000 ++                                          O                     |
          |                                                                 |
   500000 ++                                                                |
          |                                    O                            |
   400000 O+     O                             *.                           |
          |                                  ..  .                          |
   300000 ++                                .     ..                 O      |
          |                            O   .        .                       |
   200000 ++                             ..          .       O              |
          |                  ...*...... .                                   |
   100000 *+.....*.......*...          *              *......    ....*......*
          |                                                  *...           |
        0 ++----------------------------------------------------------------+

\f
                           lock_stat.rpciod-R.holdtime-total

   550000 ++-------------------------------------------------O--------------+
   500000 ++                                                                |
          |                                                                 |
   450000 O+                                                                |
   400000 ++                                                         O      |
          |                                                                 |
   350000 ++                                                                |
   300000 ++                                                                |
   250000 ++             O      O              O                            O
          |      O                                                          |
   200000 ++                           O              O                     |
   150000 ++                                                                |
          *.....                                                            |
   100000 ++    .    ....*......*......*.......*......*......*.......*......|
    50000 ++-----*----------------------------------------------------------*

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               122644.23      +234.8%    410586.88  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                85151.29      +262.1%    308329.59  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
               207795.52      +246.0%    718916.47  TOTAL lock_stat.rpciod-R.holdtime-total
\f
                         lock_stat.&rq->lock/1.waittime-total

   100000 ++----------------------------------------------------------------+
    90000 ++                                                 O              |
          |                                                                 |
    80000 ++                                                                |
    70000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
    50000 ++                                                                O
    40000 O+                                                                |
          |      O              O              O                     O      |
    30000 ++             O   ...*...   O              O                     |
    20000 *+....       ..*...       ..       ..*.....              ..*......|
          |     .  ....               .  ....        .         ....         *
    10000 ++     *.                    *.             *......*.             |
        0 ++----------------------------------------------------------------+

\f
                        lock_stat.&rq->lock/1.waittime-total

   6000 ++------------------------------------------------------------------+
        |      .*......                     ..*..                  .*.......|
   5000 ++   ..        *.......         ....     ..            ....         *
        |  ..                  *......*.           ..      ..*.             |
        |..                                            ....                 |
   4000 *+                                           *.                     |
        |                                                                   |
   3000 ++                                                                  |
        |                                                                   |
   2000 ++                                                                  |
        |                                                                   |
        |                                                                   |
   1000 ++                                                          O       |
        |       O      O       O      O       O      O       O              |
      0 O+------------------------------------------------------------------O

\f
                        lock_stat.&rq->lock/1.waittime-total

   2400 ++----------------------------------------------------------*-------+
   2200 *+......                             .*.....             ..     ... |
        |       *...          .*.....     ...       .          ..          .|
   2000 ++          ..     ...       .  ..           *.......*.             *
   1800 ++            .  ..           *.                                    |
        |              *.                                                   |
   1600 ++                                                                  |
   1400 ++                                                                  |
   1200 ++                                                                  |
        |                                                                   |
   1000 ++                                                                  |
    800 ++                                                          O       |
        |       O                             O                             |
    600 ++             O       O                     O                      O
    400 O+----------------------------O----------------------O--------------+

\f
                        lock_stat.&rq->lock/1.waittime-total

   4500 ++------------------------------------------------------------------+
        |           ...*.......*......                             .*.....  |
   4000 ++     .*...                  *.......                 ....       ..|
   3500 ++   ..                               *...          .*.             *
        |  ..                                     ..     ...                |
   3000 ++.                                         .  ..                   |
   2500 *+                                           *.                     |
        |                                                                   |
   2000 ++                                                                  |
   1500 ++                                                                  |
        |                                                                   |
   1000 ++                                                                  |
    500 ++                                                                  |
        O       O              O      O                      O      O       O
      0 ++-------------O----------------------O------O----------------------+

\f
                        lock_stat.&rq->lock/1.waittime-total

   7000 ++------------------------------------------------------------------+
        |                                                          .*..     |
   6000 ++                                                       ..    ..   |
        |                                                      ..        .. |
   5000 ++          ...*.....               ..*......*.......*.            .|
        *.......*...         ..         ....                                *
   4000 ++                     *......*.                                    |
        |                                                                   |
   3000 ++                                                                  |
        |                                                                   |
   2000 ++                                                                  |
        |                                                                   |
   1000 ++                                                                  |
        O                                                                   |
      0 ++------O------O-------O------O-------O------O-------O------O-------O

\f
                        lock_stat.&rq->lock/1.waittime-total

   3000 ++------------------------------------------------------------------+
        |                     .*......                                      |
        |                  ...        *.......           ....*...           |
   2500 ++               ..                   *......*...        ..         |
        *...          .*.                                          .    ....*
        |   ...     ..                                              *...    |
   2000 ++     .  ..                                                        |
        |       *.                                                          |
   1500 ++                                                                  |
        |                                                                   |
        |       O                                                           |
   1000 O+                                    O                             |
        |              O                                                    O
        |                      O                     O       O      O       |
    500 ++----------------------------O-------------------------------------+

\f
                        lock_stat.&rq->lock/1.waittime-total

   4500 ++------------------------------------------------------------------+
        *...                 ..*..                                          |
   4000 ++  ...          ....     .                                .*.......*
   3500 ++     .    ...*.          .                           ....         |
        |       *...                ..      ..*......    ....*.             |
   3000 ++                              ....         *...                   |
   2500 ++                            *.                                    |
        |                                                                   |
   2000 ++                                                                  |
   1500 ++                                                                  |
        |                                                                   |
   1000 ++                                                                  |
    500 O+      O      O                      O      O                      |
        |                      O      O                      O      O       O
      0 ++------------------------------------------------------------------+

\f
                        lock_stat.&rq->lock/1.waittime-total

   6500 ++------*-----------------------------------------------------------+
        |   ...    .                                                        |
   6000 ++..        .                             ...*.....                 *
        *.           ..                     ..*...         ..             ..|
        |                  ....*......  ....                 *...       ..  |
   5500 ++             *...           *.                         ..    .    |
        |                                                          . ..     |
   5000 ++                                                          *       |
        |                                                                   |
   4500 ++                                                                  |
        |                                                                   |
        |                                            O                      |
   4000 ++                                                                  O
        O       O      O       O              O                     O       |
   3500 ++----------------------------O----------------------O--------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                15510.69      +168.8%     41687.83  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                 4870.06       -93.2%       329.71  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                 2061.14       -71.1%       595.04  lkp-a04/micro/netperf/120s-200%-TCP_RR
                 3562.07       -92.6%       263.30  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                 4838.49       -97.7%       112.63  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                 2357.36       -65.6%       809.82  lkp-a04/micro/netperf/120s-200%-UDP_RR
                 3476.65       -87.9%       420.07  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                 5738.85       -34.7%      3745.61  nhm-white/micro/aim7/misc_rtns_1
                42415.30       +13.1%     47964.02  TOTAL lock_stat.&rq->lock/1.waittime-total
\f
                       lock_stat.&pipe->mutex/1.holdtime-total

   8e+06 ++-----------------------------------------------------------------+
         |                                                                  |
   7e+06 ++     O                             O                             O
   6e+06 ++                            O                     O       O      |
         O              O      O                      O                     |
   5e+06 ++                                                                 |
         |                                                                  |
   4e+06 ++                                                                 |
         |                                                                  |
   3e+06 ++                                                                 |
   2e+06 ++                                                                 |
         |                                                                  |
   1e+06 *+.....            ...*.......*......    ....*......            ...*
         |      *.......*...                  *...           *.......*...   |
       0 ++-----------------------------------------------------------------+

\f
                       lock_stat.&pipe->mutex/1.holdtime-total

   7e+06 ++-----------------------------------------------------------------+
         O              O      O                             O       O      |
   6e+06 ++     O                                                           O
         |                                    O       O                     |
   5e+06 ++                            O                                    |
         |                                                                  |
   4e+06 ++                                                                 |
         |                                                                  |
   3e+06 ++                                                                 |
         |                                                                  |
   2e+06 ++                                                                 |
         |                                                                  |
   1e+06 *+.....    ....*......            ...*.......           ....*......|
         |      *...           *.......*...           *......*...           *
       0 ++-----------------------------------------------------------------+

\f
                       lock_stat.&pipe->mutex/1.holdtime-total

   25500 ++-----------------------------------------------------------------+
         |      O       O              O      O                             |
   25000 ++                    O                      O                     |
   24500 O+                                                  O              |
         |                                                           O      O
   24000 ++                                                                 |
   23500 ++                                                                 |
         |                                                                  |
   23000 ++                                                                 |
   22500 ++                                                                 |
         |                                                                  |
   22000 ++         ....*.....       ..*.....                    ....*......*
   21500 ++  ...*...          .  ....        .    ....*......*...           |
         *...                  *.             *...                          |
   21000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               867755.90      +612.9%   6186197.65  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
               709185.05      +764.0%   6127699.72  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                21685.36       +14.4%     24803.77  nhm-white/micro/aim7/misc_rtns_1
              1598626.30      +671.8%  12338701.15  TOTAL lock_stat.&pipe->mutex/1.holdtime-total
\f
                           lock_stat.&rq->lock.contentions

   25000 ++-----*-----------------------------------------------------------+
         |    ..  ..                                                        |
         |  ..      .                                                       |
   20000 ++.         .                                                      |
         *            ..      .*.....                                       |
         |                ....       ..    ...*.......                      |
   15000 ++             *.             *...           *......               |
         |                                                   *.......*......|
   10000 ++                                                                 *
         |                                                                  |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
         |      O       O              O      O                             |
       0 O+--------------------O----------------------O------O-------O------O

\f
                           lock_stat.&rq->lock.contentions

   25000 ++-----------------------------------------------------------------+
         |             .*.....       ..*..           .*..                   |
         |          ...       .  ....     ..       ..    .                  |
   20000 ++       ..           *.           ..   ..       .                 |
         |     .*.                             ..          ..               |
         |   ..                               *                          ...*
   15000 ++..                                                *.......*...   |
         *.                                                                 |
   10000 ++                                                                 |
         |                                                                  |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
         O      O       O              O      O       O      O       O      O
       0 ++--------------------O--------------------------------------------+

\f
                            lock_stat.&rq->lock.contentions

   215000 ++----------------------------------------------------------------+
          *...          .*......                                            |
   210000 ++  ..     ...        *......    ....*......                   ...*
   205000 ++    .  ..                  *...           *......    ....*...   |
          |      *.                                          *...           |
   200000 ++                                                                |
   195000 ++                                                                |
          |                                                                 |
   190000 ++                                                                |
   185000 O+                                                                |
          |      O       O      O                            O              |
   180000 ++                                   O      O              O      |
   175000 ++                                                                |
          |                            O                                    O
   170000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                15310.40       -94.1%       899.50  lkp-a04/micro/netperf/120s-200%-TCP_RR
                18140.70       -93.4%      1190.70  lkp-a04/micro/netperf/120s-200%-UDP_RR
               206703.60       -12.9%    180113.90  nhm-white/micro/aim7/misc_rtns_1
               240154.70       -24.1%    182204.10  TOTAL lock_stat.&rq->lock.contentions
\f
                           lock_stat.&rq->lock.con-bounces

   25000 ++-----*-----------------------------------------------------------+
         |    ..  ..                                                        |
         |  ..      .                                                       |
   20000 ++.         .                                                      |
         *            ..      .*.....                                       |
         |                ....       ..    ...*.......                      |
   15000 ++             *.             *...           *......               |
         |                                                   *.......*......|
   10000 ++                                                                 *
         |                                                                  |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
         |      O       O              O      O                             |
       0 O+--------------------O----------------------O------O-------O------O

\f
                           lock_stat.&rq->lock.con-bounces

   25000 ++-----------------------------------------------------------------+
         |             .*.....       ..*..           .*..                   |
         |          ...       .  ....     ..       ..    .                  |
   20000 ++       ..           *.           ..   ..       .                 |
         |     .*.                             ..          ..               |
         |   ..                               *                          ...*
   15000 ++..                                                *.......*...   |
         *.                                                                 |
   10000 ++                                                                 |
         |                                                                  |
         |                                                                  |
    5000 ++                                                                 |
         |                                                                  |
         O      O       O              O      O       O      O       O      O
       0 ++--------------------O--------------------------------------------+

\f
                            lock_stat.&rq->lock.con-bounces

   215000 ++----------------------------------------------------------------+
          *...          .*......                                            |
   210000 ++  ..     ...        *......    ....*......                   ...*
   205000 ++    .  ..                  *...           *......    ....*...   |
          |      *.                                          *...           |
   200000 ++                                                                |
   195000 ++                                                                |
          |                                                                 |
   190000 ++                                                                |
   185000 O+                                                                |
          |      O       O      O                            O              |
   180000 ++                                   O      O              O      |
   175000 ++                                                                |
          |                            O                                    O
   170000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                15310.40       -94.1%       899.40  lkp-a04/micro/netperf/120s-200%-TCP_RR
                18140.70       -93.4%      1190.60  lkp-a04/micro/netperf/120s-200%-UDP_RR
               206424.10       -12.8%    180011.50  nhm-white/micro/aim7/misc_rtns_1
               239875.20       -24.1%    182101.50  TOTAL lock_stat.&rq->lock.con-bounces
\f
                     lock_stat.all_cpu_access_lock.holdtime-total

   2.2e+07 ++---------------------------------------------------------------+
           |                                   O                            O
     2e+07 ++     O                                                         |
   1.8e+07 ++                    O      O                     O      O      |
           O             O                            O                     |
   1.6e+07 ++                                                               |
   1.4e+07 ++                                                               |
           |                                                                |
   1.2e+07 ++                                                               |
     1e+07 ++                                                               |
           |                                                                |
     8e+06 ++                                                               |
     6e+06 ++                                                               |
           *......*......*.......*......*......*......*.......*......*......*
     4e+06 ++---------------------------------------------------------------+

\f
                     lock_stat.all_cpu_access_lock.holdtime-total

     2e+07 O+------------O-------O----------------------------O------O------+
           |      O                                   O                     |
   1.8e+07 ++                                  O                            O
   1.6e+07 ++                                                               |
           |                            O                                   |
   1.4e+07 ++                                                               |
           |                                                                |
   1.2e+07 ++                                                               |
           |                                                                |
     1e+07 ++                                                               |
     8e+06 ++                                                               |
           |                                                                |
     6e+06 ++                                 .*......                      |
           *......*......*.......*......  ....        *.......    ...*......*
     4e+06 ++---------------------------*---------------------*-------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              5351021.81      +247.6%  18598404.06  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
              4983961.74      +275.1%  18692448.91  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
             10334983.54      +260.8%  37290852.98  TOTAL lock_stat.all_cpu_access_lock.holdtime-total
\f
                      lock_stat.trace_event_sem-R.holdtime-total

   2.2e+07 ++---------------------------------------------------------------+
           |                                   O                            O
     2e+07 ++     O                                                  O      |
   1.8e+07 ++                    O      O                     O             |
           O             O                            O                     |
   1.6e+07 ++                                                               |
   1.4e+07 ++                                                               |
           |                                                                |
   1.2e+07 ++                                                               |
     1e+07 ++                                                               |
           |                                                                |
     8e+06 ++                                                               |
     6e+06 ++                                                               |
           *......*......*.......*......*......*......*.......*......*......*
     4e+06 ++---------------------------------------------------------------+

\f
                      lock_stat.trace_event_sem-R.holdtime-total

     2e+07 O+------------O-------O----------------------------O------O------+
           |      O                                   O                     |
   1.8e+07 ++                                  O                            O
   1.6e+07 ++                                                               |
           |                            O                                   |
   1.4e+07 ++                                                               |
           |                                                                |
   1.2e+07 ++                                                               |
           |                                                                |
     1e+07 ++                                                               |
     8e+06 ++                                                               |
           |                                                                |
     6e+06 ++                                 .*......                      |
           *......*......*.......*......  ....        *.......    ...*......*
     4e+06 ++---------------------------*---------------------*-------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              5375816.52      +247.2%  18666508.28  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
              5007023.86      +274.6%  18758015.91  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
             10382840.38      +260.4%  37424524.18  TOTAL lock_stat.trace_event_sem-R.holdtime-total
\f
                         lock_stat.&iter->mutex.holdtime-total

   2.2e+07 ++----------------------------------O----------------------------O
           |                                                                |
     2e+07 ++     O                     O                     O      O      |
   1.8e+07 O+                    O                                          |
           |             O                            O                     |
   1.6e+07 ++                                                               |
   1.4e+07 ++                                                               |
           |                                                                |
   1.2e+07 ++                                                               |
     1e+07 ++                                                               |
           |                                                                |
     8e+06 ++                                                               |
     6e+06 *+.....                                                          |
           |      *......*.......*......*......*......*.......*......*......*
     4e+06 ++---------------------------------------------------------------+

\f
                         lock_stat.&iter->mutex.holdtime-total

   2.2e+07 ++---------------------------------------------------------------+
           |             O       O                            O      O      |
     2e+07 O+     O                                   O                     |
   1.8e+07 ++                                  O                            O
           |                            O                                   |
   1.6e+07 ++                                                               |
   1.4e+07 ++                                                               |
           |                                                                |
   1.2e+07 ++                                                               |
     1e+07 ++                                                               |
           |                                                                |
     8e+06 ++                                                               |
     6e+06 ++                                 .*......                      |
           *......*......*.......*......  ....        *.......    ...*......*
     4e+06 ++---------------------------*---------------------*-------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              5529096.71      +244.6%  19051541.30  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
              5153359.47      +271.5%  19143767.84  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
             10682456.18      +257.6%  38195309.13  TOTAL lock_stat.&iter->mutex.holdtime-total
\f
                     lock_stat.slock-AF_INET-RPC/1.acquisitions

   22000 ++----------------------------O------O--------------O--------------O
         O      O              O                                     O      |
   20000 ++             O                             O                     |
   18000 ++                                                                 |
         |                                                                  |
   16000 ++                                                                 |
   14000 ++                                                                 |
         |                                                                  |
   12000 ++                                                                 |
   10000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
    6000 *+.....*.......*......*.......*......*.......*......*.......*......*
         |                                                                  |
    4000 ++-----------------------------------------------------------------+

\f
                     lock_stat.slock-AF_INET-RPC/1.acquisitions

   24000 ++-----------------------------------------------------------------+
   22000 O+                                                                 |
         |      O              O                                            |
   20000 ++             O                     O       O      O       O      O
   18000 ++                            O                                    |
         |                                                                  |
   16000 ++                                                                 |
   14000 ++                                                                 |
   12000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
    8000 ++                                                                 |
         |                                                                  |
    6000 *+.....*.......*......*.......*......*.......*......*.......*......*
    4000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 6126.20      +239.6%     20806.30  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                 5496.40      +270.4%     20360.60  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                11622.60      +254.2%     41166.90  TOTAL lock_stat.slock-AF_INET-RPC/1.acquisitions
\f
                        lock_stat._xmit_ETHER#2.acquisitions

   24000 ++-----------------------------------------------------------------+
   22000 ++                            O                                    |
         |      O                             O              O       O      O
   20000 O+             O      O                      O                     |
   18000 ++                                                                 |
         |                                                                  |
   16000 ++                                                                 |
   14000 ++                                                                 |
   12000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
    8000 ++                                                                 |
         *......*.......*......*.......*......*.......                      |
    6000 ++                                           *......*.......*......*
    4000 ++-----------------------------------------------------------------+

\f
                        lock_stat._xmit_ETHER#2.acquisitions

   24000 ++-----------------------------------------------------------------+
   22000 O+                                                                 |
         |      O              O                                            O
   20000 ++             O                     O       O      O       O      |
   18000 ++                            O                                    |
         |                                                                  |
   16000 ++                                                                 |
   14000 ++                                                                 |
   12000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
    8000 ++                                                                 |
         |                                                                  |
    6000 *+.....*.......*......*.......*......*.......*......*.......*......*
    4000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 6176.10      +237.5%     20846.90  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                 5539.90      +268.5%     20414.00  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                11716.00      +252.2%     41260.90  TOTAL lock_stat._xmit_ETHER#2.acquisitions
\f
                    lock_stat.&(&list->lock)->rlock.acquisitions

   45000 ++----------------------------O------------------------------------+
         |      O                             O              O       O      O
   40000 O+                    O                      O                     |
         |              O                                                   |
   35000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
         |                                                                  |
   25000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
         *......*.......*......*.......*......*.......*......*.......*......*
   10000 ++-----------------------------------------------------------------+

\f
                    lock_stat.&(&list->lock)->rlock.acquisitions

   45000 O+-----------------------------------------------------------------+
         |      O              O                                            |
   40000 ++             O                     O       O      O       O      O
         |                             O                                    |
   35000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
         |                                                                  |
   25000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
         *......*.......                   ...*.......                      |
   10000 ++-------------*------*-------*--------------*------*-------*------*

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                12352.20      +237.5%     41693.90  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                11079.80      +268.5%     40828.00  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                23432.00      +252.2%     82521.90  TOTAL lock_stat.&(&list->lock)->rlock.acquisitions
\f
                    lock_stat.buffer->reader_lock_key.acquisitions

     3e+07 ++---------------------------------------------------------------+
           |                            O      O              O             O
           O      O      O       O                    O              O      |
   2.5e+07 ++                                                               |
           |                                                                |
           |                                                                |
     2e+07 ++                                                               |
           |                                                                |
   1.5e+07 ++                                                               |
           |                                                                |
           |                                                                |
     1e+07 ++                                                               |
           *......*......*.......*......*......*......*.......           ...*
           |                                                  *......*...   |
     5e+06 ++---------------------------------------------------------------+

\f
                    lock_stat.buffer->reader_lock_key.acquisitions

     3e+07 ++---------------------------------------------------------------+
           O      O                                                         |
           |             O       O                    O       O      O      O
   2.5e+07 ++                           O      O                            |
           |                                                                |
           |                                                                |
     2e+07 ++                                                               |
           |                                                                |
   1.5e+07 ++                                                               |
           |                                                                |
           |                                                                |
     1e+07 ++                                                               |
           |   ...*......*.......           ...*......                      |
           *...                  *......*...          *.......*......*......*
     5e+06 ++---------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              7979467.90      +243.4%  27405368.50  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
              7179771.90      +264.5%  26168738.50  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
             15159239.80      +253.4%  53574107.00  TOTAL lock_stat.buffer->reader_lock_key.acquisitions
\f
                           lock_stat.&rq->lock.acquisitions

   2.4e+07 ++---------------------------O------O--------------O-------------+
   2.2e+07 ++     O      O       O                    O              O      O
           O                                                                |
     2e+07 ++                                                               |
   1.8e+07 ++                                                               |
           |                                                                |
   1.6e+07 ++                                                               |
   1.4e+07 ++                                                               |
   1.2e+07 ++                                                               |
           |                                                                |
     1e+07 ++                                                               |
     8e+06 ++                                                               |
           |                                                                |
     6e+06 *+.....    ...*.......*......*......*......*.......*......    ...*
     4e+06 ++-----*--------------------------------------------------*------+

\f
                           lock_stat.&rq->lock.acquisitions

   2.4e+07 ++-----O---------------------------------------------------------+
   2.2e+07 O+            O       O             O                            |
           |                                          O       O      O      O
     2e+07 ++                           O                                   |
   1.8e+07 ++                                                               |
   1.6e+07 ++                                                               |
   1.4e+07 ++                                                               |
           |                                                                |
   1.2e+07 ++                                                               |
     1e+07 ++                                                               |
     8e+06 ++                                                               |
     6e+06 ++                                                               |
           *......    ...*.......           ...*......                      |
     4e+06 ++     *...           *......*...          *.......*......*......*
     2e+06 ++---------------------------------------------------------------+

\f
                           lock_stat.&rq->lock.acquisitions

   2.2e+06 ++---------------------------------------------------------------+
           *......*......*.......*......*......*......    ....*......*......*
     2e+06 ++                                         *...                  |
           |                                                                |
   1.8e+06 ++                                                               |
   1.6e+06 ++                                         O                     |
           |                                                                |
   1.4e+06 ++                                                               |
           |                                                                |
   1.2e+06 ++                    O             O                            O
     1e+06 ++                                                               |
           O             O              O                     O             |
    800000 ++                                                        O      |
           |      O                                                         |
    600000 ++---------------------------------------------------------------+

\f
                            lock_stat.&rq->lock.acquisitions

    2.6e+06 ++--------------------------------------------------------------+
            |                                                 O             |
   2.55e+06 ++                                                              |
    2.5e+06 ++     O      O             O                                   |
            O                    O                     O                    |
   2.45e+06 ++                                                              |
    2.4e+06 ++                                                              O
            |                                                        O      |
   2.35e+06 ++                                  O                           |
    2.3e+06 ++                                                              |
            |                                                              .*
   2.25e+06 *+....                          ....*......*......         .... |
    2.2e+06 ++    .    ...*......*......*...                  *......*.     |
            |      *...                                                     |
   2.15e+06 ++--------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              4850370.80      +371.7%  22877390.80  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
              4192613.60      +422.3%  21897029.50  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
              2087959.20       -50.1%   1041781.30  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              2226180.00       +10.4%   2457435.00  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
             13357123.60      +261.4%  48273636.60  TOTAL lock_stat.&rq->lock.acquisitions
\f
                   lock_stat.buffer->reader_lock_key.holdtime-total

     6e+06 ++----------------------------------O----------------------------O
   5.5e+06 ++                                                               |
           |      O              O      O                     O      O      |
     5e+06 O+            O                            O                     |
   4.5e+06 ++                                                               |
           |                                                                |
     4e+06 ++                                                               |
   3.5e+06 ++                                                               |
     3e+06 ++                                                               |
           |                                                                |
   2.5e+06 ++                                                               |
     2e+06 ++                                                               |
           *......*......    ....*......*......*......*.......*......*......*
   1.5e+06 ++            *...                                               |
     1e+06 ++---------------------------------------------------------------+

\f
                   lock_stat.buffer->reader_lock_key.holdtime-total

     6e+06 ++---------------------------------------------------------------+
   5.5e+06 O+     O      O       O                                   O      |
           |                                          O       O             |
     5e+06 ++                                  O                            O
   4.5e+06 ++                           O                                   |
           |                                                                |
     4e+06 ++                                                               |
   3.5e+06 ++                                                               |
     3e+06 ++                                                               |
           |                                                                |
   2.5e+06 ++                                                               |
     2e+06 ++                                                               |
           *......*......*.......           ...*......*.......    ...*......*
   1.5e+06 ++                    *......*...                  *...          |
     1e+06 ++---------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              1680573.60      +215.7%   5306077.80  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
              1570346.10      +236.6%   5285946.35  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
              3250919.71      +225.8%  10592024.15  TOTAL lock_stat.buffer->reader_lock_key.holdtime-total
\f
                      lock_stat.slock-AF_INET-RPC.acquisitions

   26000 ++----------------------------O------------------------------------+
   24000 ++     O                             O              O       O      O
         O                     O                      O                     |
   22000 ++             O                                                   |
   20000 ++                                                                 |
         |                                                                  |
   18000 ++                                                                 |
   16000 ++                                                                 |
   14000 ++                                                                 |
         |                                                                  |
   12000 ++                                                                 |
   10000 ++                                                                 |
         |                                 ...*.......                      |
    8000 *+.....*.......*......*.......*...           *......*.......*......*
    6000 ++-----------------------------------------------------------------+

\f
                      lock_stat.slock-AF_INET-RPC.acquisitions

   26000 O+-----------------------------------------------------------------+
   24000 ++     O              O                             O              |
         |              O                     O       O              O      O
   22000 ++                            O                                    |
   20000 ++                                                                 |
         |                                                                  |
   18000 ++                                                                 |
   16000 ++                                                                 |
   14000 ++                                                                 |
         |                                                                  |
   12000 ++                                                                 |
   10000 ++                                                                 |
         |                                                                  |
    8000 *+.....*.......*......*.......*......*.......*......*.......*......*
    6000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 7907.60      +206.5%     24239.60  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                 7092.40      +234.2%     23703.70  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                15000.00      +219.6%     47943.30  TOTAL lock_stat.slock-AF_INET-RPC.acquisitions
\f
                        lock_stat.&pipe->wait.holdtime-total

   1e+07 ++-----------------------------------------------------------------+
   9e+06 ++                                   O                             O
         |      O                                                    O      |
   8e+06 ++             O      O       O              O      O              |
   7e+06 O+                                                                 |
         |                                                                  |
   6e+06 ++                                                                 |
   5e+06 ++                                                                 |
   4e+06 ++                                                                 |
         |                                                                  |
   3e+06 ++                                                                 |
   2e+06 ++                                                                 |
         |                                                                  |
   1e+06 *+.....*.......*......*.......*......*.......*......*.......*......*
       0 ++-----------------------------------------------------------------+

\f
                        lock_stat.&pipe->wait.holdtime-total

   1e+07 ++-----------------------------------------------------------------+
   9e+06 ++             O                                                   |
         O      O              O                             O       O      |
   8e+06 ++                                   O                             |
   7e+06 ++                                           O                     O
         |                             O                                    |
   6e+06 ++                                                                 |
   5e+06 ++                                                                 |
   4e+06 ++                                                                 |
         |                                                                  |
   3e+06 ++                                                                 |
   2e+06 ++                                                                 |
         |                                                                  |
   1e+06 *+.....*.......*......*.......*......*.......*......*.......*......*
       0 ++-----------------------------------------------------------------+

\f
                         lock_stat.&pipe->wait.holdtime-total

   330000 ++----------------------------------------------------------------+
          |                            O                                    |
   320000 ++                                                 O              |
   310000 ++     O                                                          O
          |              O                                           O      |
   300000 O+                    O              O                            |
   290000 ++                                                                |
          |                                                                 |
   280000 ++                                          O                     |
   270000 ++                                                                |
          |                                                                 |
   260000 ++                                                                |
   250000 *+.....*.......    ...*......*.......    ...*......*.......*......*
          |              *...                  *...                         |
   240000 ++----------------------------------------------------------------+

\f
                        lock_stat.&pipe->wait.holdtime-total

   19000 ++-----------------------------------------------------------------+
         |                                                                  |
   18000 ++     O              O       O      O                             |
         O              O                             O                     |
   17000 ++                                                  O       O      O
         |                                                                  |
   16000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   14000 ++                                                                 |
         |                                                                  |
   13000 ++         ....*......    ....*......    ....*......               |
         *......*...           *...           *...           *.......*......*
   12000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               905867.22      +793.2%   8091078.58  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
               652073.89     +1122.4%   7970926.87  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
               251241.57       +21.1%    304249.35  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                12682.84       +38.7%     17585.69  nhm-white/micro/aim7/misc_rtns_1
              1821865.51      +799.3%  16383840.49  TOTAL lock_stat.&pipe->wait.holdtime-total
\f
                   lock_stat.&(&list->lock)->rlock.holdtime-total

   12000 ++-----------------------------------------------------------------+
         |                                    O                             |
   11000 ++     O                                                           O
   10000 ++                                                  O       O      |
         O                             O              O                     |
    9000 ++                    O                                            |
    8000 ++             O                                                   |
         |                                                                  |
    7000 ++                                                                 |
    6000 ++                                                                 |
         |                                                                  |
    5000 ++                                                                 |
    4000 ++                                                                 |
         |                  ...*.......*......                           ...*
    3000 *+-----*-------*---------------------*-------*------*-------*------+

\f
                   lock_stat.&(&list->lock)->rlock.holdtime-total

   12000 ++-----------------------------------------------------------------+
   11000 ++                    O                             O              |
         O              O                     O                             |
   10000 ++     O                                                    O      O
    9000 ++                                           O                     |
         |                             O                                    |
    8000 ++                                                                 |
    7000 ++                                                                 |
    6000 ++                                                                 |
         |                                                                  |
    5000 ++                                                                 |
    4000 ++                                                                 |
         *......    ....*......*.......    ...*.......*......               |
    3000 ++     *...                   *...                  *.......*......*
    2000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 3272.82      +200.1%      9823.11  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                 3033.54      +234.2%     10139.00  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                 6306.35      +216.5%     19962.11  TOTAL lock_stat.&(&list->lock)->rlock.holdtime-total
\f
                        lock_stat.trace_types_lock.acquisitions

   200000 ++----------------------------------------------------------------+
          |                                                                 |
   180000 ++                           O       O             O              O
   160000 O+     O       O      O                     O              O      |
          |                                                                 |
   140000 ++                                                                |
          |                                                                 |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
    80000 ++                                                                |
          |                                                                 |
    60000 *+.....    ....*......                                            |
          |      *...           *......*.......*......*......*.......*......*
    40000 ++----------------------------------------------------------------+

\f
                        lock_stat.trace_types_lock.acquisitions

   180000 ++----------------------------------------------------------------+
          O      O              O                                           |
   160000 ++             O                     O      O      O       O      O
          |                            O                                    |
   140000 ++                                                                |
          |                                                                 |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
    80000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
          *......*.......*......*......    ....*......*......*.......*......*
    40000 ++---------------------------*------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                55123.10      +211.9%    171905.20  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                49622.10      +230.1%    163779.40  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
               104745.20      +220.5%    335684.60  TOTAL lock_stat.trace_types_lock.acquisitions
\f
                       lock_stat.trace_event_sem-R.acquisitions

   200000 ++----------------------------------------------------------------+
          |                                                                 |
   180000 ++                           O       O             O              O
   160000 O+     O       O      O                     O              O      |
          |                                                                 |
   140000 ++                                                                |
          |                                                                 |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
    80000 ++                                                                |
          |                                                                 |
    60000 *+.....    ....*......                                            |
          |      *...           *......*.......*......*......*.......*......*
    40000 ++----------------------------------------------------------------+

\f
                       lock_stat.trace_event_sem-R.acquisitions

   180000 ++----------------------------------------------------------------+
          O      O              O                                           |
   160000 ++             O                     O      O      O       O      O
          |                            O                                    |
   140000 ++                                                                |
          |                                                                 |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
    80000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
          *......*.......*......*......    ....*......*......*.......*......*
    40000 ++---------------------------*------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                55123.10      +211.9%    171904.90  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                49622.10      +230.1%    163779.10  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
               104745.20      +220.5%    335684.00  TOTAL lock_stat.trace_event_sem-R.acquisitions
\f
                      lock_stat.all_cpu_access_lock.acquisitions

   200000 ++----------------------------------------------------------------+
          |                                                                 |
   180000 ++                           O       O             O              O
   160000 O+     O       O      O                     O              O      |
          |                                                                 |
   140000 ++                                                                |
          |                                                                 |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
    80000 ++                                                                |
          |                                                                 |
    60000 *+.....    ....*......                                            |
          |      *...           *......*.......*......*......*.......*......*
    40000 ++----------------------------------------------------------------+

\f
                      lock_stat.all_cpu_access_lock.acquisitions

   180000 ++----------------------------------------------------------------+
          O      O              O                                           |
   160000 ++             O                     O      O      O       O      O
          |                            O                                    |
   140000 ++                                                                |
          |                                                                 |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
    80000 ++                                                                |
          |                                                                 |
    60000 ++                                                                |
          *......*.......*......*......    ....*......*......*.......*......*
    40000 ++---------------------------*------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                55123.10      +211.9%    171904.70  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                49622.10      +230.1%    163779.10  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
               104745.20      +220.5%    335683.80  TOTAL lock_stat.all_cpu_access_lock.acquisitions
\f
                       lock_stat.&vfsmount_lock-R.holdtime-max

   5500 ++------------------------------------------------------------------+
   5000 ++                                    O              O              |
        O                                                                   |
   4500 ++                                                                  |
   4000 ++                                                          O       |
        |                                                                   |
   3500 ++                                                                  *
   3000 ++      O                     *.             O                     +|
   2500 ++                           .  ..                                + |
        |                           .     .                              +  |
   2000 ++                        ..       .                            +   O
   1500 ++            .*.....    .          ..      .*.....            +    |
        |           ..       .. .               ....       ..         +     |
   1000 *+......  ..           *              *.             *...... +      |
    500 ++------*--------------O------------------------------------*-------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 1411.65      +126.4%      3196.43  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                 1411.65      +126.4%      3196.43  TOTAL lock_stat.&vfsmount_lock-R.holdtime-max
\f
                     lock_stat.&(&base->lock)->rlock.acquisitions

   1.8e+06 ++---------------------------------------------------------------+
           *......*......    ....*......*......*......    ....*......*......*
   1.6e+06 ++            *...                         *...                  |
           |                                                                |
   1.4e+06 ++                                                               |
           |                                                                |
   1.2e+06 ++                                                               |
           |                                          O                     |
     1e+06 ++                                                               |
           |                                                                |
    800000 ++                    O                                          O
           |                                   O                            |
    600000 O+            O                                    O             |
           |                            O                            O      |
    400000 ++-----O---------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              1669574.80       -60.4%    661408.70  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              1669574.80       -60.4%    661408.70  TOTAL lock_stat.&(&base->lock)->rlock.acquisitions
\f
                          lock_stat.&iter->mutex.acquisitions

   200000 ++----------------------------------------------------------------+
          |                            O                                    O
   180000 O+     O              O              O             O       O      |
   160000 ++             O                            O                     |
          |                                                                 |
   140000 ++                                                                |
          |                                                                 |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
    80000 ++                                                                |
          |                                                                 |
    60000 *+.....*.......*......*......*.......*......*......*.......*......*
          |                                                                 |
    40000 ++----------------------------------------------------------------+

\f
                          lock_stat.&iter->mutex.acquisitions

   180000 O+-----O----------------------------------------------------------+
          |              O      O                     O      O       O      O
   160000 ++                           O       O                            |
          |                                                                 |
   140000 ++                                                                |
          |                                                                 |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
    80000 ++                                                                |
          |                                                                 |
    60000 *+.....*.......*......           ....*......           ....*......*
          |                     *......*...           *......*...           |
    40000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                61050.30      +190.2%    177165.20  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                55534.70      +204.4%    169065.90  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
               116585.00      +197.0%    346231.10  TOTAL lock_stat.&iter->mutex.acquisitions
\f
                      lock_stat.trace_types_lock.holdtime-total

   65000 ++-----------------------------------O-----------------------------O
   60000 ++                                                                 |
         |      O                                                    O      |
   55000 O+                    O       O                     O              |
   50000 ++             O                             O                     |
         |                                                                  |
   45000 ++                                                                 |
   40000 ++                                                                 |
   35000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
   25000 ++                                                                 |
         *......*.......                          ....*......            ...*
   20000 ++             *......*.......*......*...           *.......*...   |
   15000 ++-----------------------------------------------------------------+

\f
                      lock_stat.trace_types_lock.holdtime-total

   65000 ++-----------------------------------------------------------------+
   60000 ++             O      O                             O              |
         O      O                                                    O      |
   55000 ++                                   O       O                     O
   50000 ++                                                                 |
         |                             O                                    |
   45000 ++                                                                 |
   40000 ++                                                                 |
   35000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
   25000 ++                                                                 |
         |                                 ...*.......*.....                |
   20000 *+.....*.......*......*.......*...                 .    ....*......*
   15000 ++--------------------------------------------------*--------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                20236.40      +177.4%     56129.32  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                18809.60      +200.2%     56471.68  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                39046.00      +188.4%    112601.00  TOTAL lock_stat.trace_types_lock.holdtime-total
\f
                      lock_stat.rcu_read_lock_bh-R.acquisitions

   45000 ++----------------------------O------------------------------------+
         |      O                             O              O       O      O
   40000 O+                    O                      O                     |
         |              O                                                   |
   35000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
         |                                                                  |
   25000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
         *......*.......*......*.......*......*.......*......*.......*......*
   10000 ++-----------------------------------------------------------------+

\f
                      lock_stat.rcu_read_lock_bh-R.acquisitions

   45000 O+-----------------------------------------------------------------+
         |      O              O                                            |
   40000 ++             O                     O       O      O       O      O
         |                             O                                    |
   35000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
         |                                                                  |
   25000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
         *......*.......                   ...*.......                      |
   10000 ++-------------*------*-------*--------------*------*-------*------*

\f
                       lock_stat.rcu_read_lock_bh-R.acquisitions

   2.4e+06 ++---------------------------------------------------------------+
           *......*......*.......*......*......*......    ....*......*......*
   2.3e+06 ++                                         *...                  |
           |                                                                |
   2.2e+06 ++                                                               |
   2.1e+06 ++                                                               |
           |                                          O                     |
     2e+06 ++                                                               |
           |                                                                |
   1.9e+06 ++                                                               |
   1.8e+06 ++                    O             O                            O
           |             O                                    O             |
   1.7e+06 O+                           O                            O      |
           |      O                                                         |
   1.6e+06 ++---------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                12357.20      +237.4%     41698.60  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                11084.80      +268.4%     40833.00  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
              2332796.10       -23.3%   1789952.30  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              2356238.10       -20.5%   1872483.90  TOTAL lock_stat.rcu_read_lock_bh-R.acquisitions
\f
                           lock_stat.&wq->wait.acquisitions

   800000 ++----------------------------------------------------------------+
          |                                ....*.....       .*.......       |
   700000 *+.....*.......*......*......*...          .  ....         *......*
          |                                           *.                    |
   600000 ++                                                                |
          |                                                                 |
   500000 ++                                          O                     |
          |                                                                 |
   400000 ++                                                                |
          |                     O              O                            O
   300000 ++                                                                |
          O              O             O                     O       O      |
   200000 ++     O                                                          |
          |                                                                 |
   100000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               712340.80       -58.5%    295574.20  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               712340.80       -58.5%    295574.20  TOTAL lock_stat.&wq->wait.acquisitions
\f
                     lock_stat.slock-AF_INET-RPC/1.holdtime-total

   180000 ++----------------------------------------------------------------+
          |                                                                 |
   160000 ++                                   O                            O
          |                                                  O              |
   140000 O+                                                         O      |
          |      O              O                     O                     |
   120000 ++             O             O                                    |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
    80000 ++                                                                |
          |                                                                 |
    60000 ++         ....*......*......*.......*......*......    ....*......*
          *......*...                                        *...           |
    40000 ++----------------------------------------------------------------+

\f
                     lock_stat.slock-AF_INET-RPC/1.holdtime-total

   160000 ++----------------------------------------------------------------+
          |                                    O                            |
   140000 O+             O      O                                           O
          |      O                     O              O      O              |
          |                                                          O      |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
    80000 ++                                                                |
          |                                                                 |
          |                                                                 |
    60000 *+.....            ...*......    ....*......                      |
          |      *.......*...          *...           *......*.......*......|
    40000 ++----------------------------------------------------------------*

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                56228.59      +146.8%    138799.27  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                50253.49      +174.7%    138024.15  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
               106482.08      +160.0%    276823.43  TOTAL lock_stat.slock-AF_INET-RPC/1.holdtime-total
\f
                        lock_stat._xmit_ETHER#2.holdtime-total

   400000 ++----------------------------------------------------------------+
          |                                                                 |
   350000 O+                                                                |
          |      O                                                          |
   300000 ++                                                                O
          |                                                                 |
   250000 ++                                   O                            |
          |                                                                 |
   200000 ++                                          O              O      |
          |                            O                     O              |
   150000 ++                                                                |
          |              O   ...*......                                     |
   100000 ++         ....*...          *.......                             |
          *......*...                          *......*......    ....*......*
    50000 ++-------------------------------------------------*--------------+

\f
                        lock_stat._xmit_ETHER#2.holdtime-total

   450000 ++----------------------------------------------------------------+
          |                                                  O              |
   400000 ++                                                                |
          |                                                                 |
   350000 ++                                                                |
   300000 ++                                                         O      |
          O                                                                 |
   250000 ++                                                                |
          |                     O              O                            O
   200000 ++             O             O                                    |
   150000 ++     O                                                          |
          |                                           O                     |
   100000 *+....                                                            |
          |     .    ....*......*......*.......*......*......*.......*......|
    50000 ++-----*----------------------------------------------------------*

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                84128.22      +165.0%    222979.03  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                76998.50      +204.5%    234456.23  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
               161126.72      +183.9%    457435.26  TOTAL lock_stat._xmit_ETHER#2.holdtime-total
\f
                     lock_stat.sk_lock-AF_INET-RPC.holdtime-total

   350000 ++----------------------------------------------------------------+
          O                                                                 |
   300000 ++                                                                |
          |      O                                                          |
   250000 ++                                                                O
          |                                                                 |
   200000 ++                                   O                            |
          |                                           O              O      |
   150000 ++                           O                     O              |
          |                     O                                           |
   100000 ++             O   ...*......*.......                             |
          |   ...*.......*...                  *......*......    ....*......*
    50000 *+..                                               *...           |
          |                                                                 |
        0 ++----------------------------------------------------------------+

\f
                     lock_stat.sk_lock-AF_INET-RPC.holdtime-total

   350000 ++----------------------------------------------------------------+
          |                                                  O              |
   300000 ++                                                                |
          |                                                                 |
   250000 ++                                                         O      |
          O                                                                 |
   200000 ++                                                                |
          |              O      O              O                            O
   150000 ++     O                     O                                    |
          |                                           O                     |
   100000 ++                                                                |
          *......            ...*......*.......*......    ...*.......*......|
    50000 ++     *.......*...                         *...                  *
          |                                                                 |
        0 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                70397.88      +169.5%    189745.16  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                63995.67      +199.5%    191690.16  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
               134393.55      +183.8%    381435.33  TOTAL lock_stat.sk_lock-AF_INET-RPC.holdtime-total
\f
                  lock_stat.&sb->s_type->i_lock_key#1.acquisitions

   60000 ++-----------------------------------------------------------------+
         |                             O      O              O              O
   55000 ++     O              O                                     O      |
   50000 O+             O                             O                     |
         |                                                                  |
   45000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
         |                                                                  |
   35000 ++                                                                 |
   30000 ++                                                                 |
         |                                                                  |
   25000 *+.....    ....*......                                             |
         |      *...           *.......*......*.......*......*.......*......*
   20000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                23948.20      +127.0%     54351.40  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                23948.20      +127.0%     54351.40  TOTAL lock_stat.&sb->s_type->i_lock_key#1.acquisitions
\f
                     lock_stat.&sig->cred_guard_mutex.holdtime-max

   120000 ++----------------------------------------------------------------+
          |                                                  *...           |
   110000 *+.                                ..*...        ..    ...        |
   100000 ++ .                           ....      ..    ..         .       |
          |   .                        *.            . ..            *......|
    90000 ++   ..                    ..               *                     *
          |                         .                                       |
    80000 ++     *.......*.....    .                                        |
          |                    . ..                                         |
    70000 ++                    *                                           |
    60000 ++                                                                |
          |                            O                     O              |
    50000 ++     O       O      O              O      O              O      O
          O                                                                 |
    40000 ++----------------------------------------------------------------+

\f
                     lock_stat.&sig->cred_guard_mutex.holdtime-max

   160000 ++--------------------*-------------------------------------------+
          |                   ..  .                                         |
   140000 ++                 .     ..                                       |
          *.....            .        .                                      |
   120000 ++    .         ..          .                             .*..... |
          |      *.......*                                        ..       .|
   100000 ++                           *.......*...             ..          *
          |                                        ..         ..            |
    80000 ++                                         .    ...*              |
          |                                           *...                  |
    60000 ++                                                                |
          |      O                             O      O                     |
    40000 O+             O             O                     O       O      O
          |                     O                                           |
    20000 ++----------------------------------------------------------------+

\f
                     lock_stat.&sig->cred_guard_mutex.holdtime-max

   120000 *+----------------------------------------------------------------+
          |   ..                              .*                            |
   110000 ++    .                          ...  +                           |
   100000 ++     *.......*.....          ..      +                   *..... |
          |                    .    ...*.         +                ..      .|
    90000 ++                    *...               +              .         *
          |                                         +            .          |
    80000 ++                                         +         ..    O      |
          |                                           *...... .             |
    70000 ++                                                 *              |
    60000 ++                                                                |
          O                                                                 |
    50000 ++     O       O      O      O              O      O              |
          |                                    O                            O
    40000 ++----------------------------------------------------------------+

\f
                    lock_stat.&sig->cred_guard_mutex.holdtime-max

   58000 ++-----------------------------------------------------------------+
   56000 *+..                                                               |
         |   ..                                                             |
   54000 ++    .                                                            |
   52000 ++     *                                                           |
         |       +             *.......                                     |
   50000 ++       +           +        *......*.......                   ...*
   48000 ++        +         +                        *......    ....*...   |
   46000 O+         +       +                                *...           |
         |           +     +                                                |
   44000 ++           +   +                                                 |
   42000 ++     O      + +     O                                            O
         |              *                     O       O              O      |
   40000 ++             O              O                     O              |
   38000 ++-----------------------------------------------------------------+

\f
                     lock_stat.&sig->cred_guard_mutex.holdtime-max

   250000 ++----------------------------------------------------------------+
          |                            *..                          .*..    |
          |                           .   .                       ..    .   |
   200000 ++                         .     ..                   ..       .  |
          |                        ..        ..               ..          ..|
          |                       .                       ...*              |
   150000 *+.....    ....*...... .             *......*...                  *
          |      *...           *                                           |
   100000 ++                                                                |
          |                                                                 |
          |                                                                 |
    50000 ++                                   O      O                     O
          O      O       O      O      O                     O       O      |
          |                                                                 |
        0 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                94632.87       -44.6%     52417.08  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
               108540.04       -60.6%     42776.44  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                97185.85       -44.9%     53514.42  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                49154.31       -15.9%     41333.62  lkp-a04/micro/netperf/120s-200%-UDP_RR
               159814.34       -75.1%     39724.97  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               509327.41       -54.9%    229766.53  TOTAL lock_stat.&sig->cred_guard_mutex.holdtime-max
\f
                        lock_stat.sb_writers#4-R.acquisitions

   34000 ++-----------------------------------------------------------------+
   32000 ++                                   O                             |
         |                                                           O      O
   30000 ++     O                      O                     O              |
   28000 O+             O      O                      O                     |
   26000 ++                                                                 |
   24000 ++                                                                 |
         |                                                                  |
   22000 ++                                                                 |
   20000 ++                                                                 |
   18000 ++                                                                 |
   16000 ++                                                                 |
         *......                                                         ...*
   14000 ++     *.......*......*.......*......*.......*......*.......*...   |
   12000 ++-----------------------------------------------------------------+

\f
                        lock_stat.sb_writers#4-R.acquisitions

   32000 ++-------------O---------------------------------------------------+
   30000 O+     O              O                             O       O      |
         |                                    O                             O
   28000 ++                                           O                     |
   26000 ++                            O                                    |
         |                                                                  |
   24000 ++                                                                 |
   22000 ++                                                                 |
   20000 ++                                                                 |
         |                                                                  |
   18000 ++                                                                 |
   16000 ++                                                                 |
         |                                                                  |
   14000 *+.....*.......*......*.......    ...*.......*......*.......*......*
   12000 ++----------------------------*------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                14070.00      +109.7%     29499.90  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                13014.80      +129.0%     29798.10  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                27084.80      +118.9%     59298.00  TOTAL lock_stat.sb_writers#4-R.acquisitions
\f
                       lock_stat.sk_lock-AF_INET.holdtime-max

   4500 ++------------------------------------------------------------------+
        |       *..                                  *.......               |
   4000 ++     .   .                                .        *......        |
        |     .     .                .*..          .                *.......*
   3500 ++  ..       ..            ..    ..      ..                         |
        |  .                     ..        ..   .                           |
   3000 ++.            *.......*.            . .                            |
        |.                                    *                             |
   2500 *+                                                                  |
        |                                                                   |
   2000 ++                                                                  |
        |                                                                   |
   1500 O+             O       O      O       O      O       O              |
        |       O                                                   O       O
   1000 ++------------------------------------------------------------------+

\f
                       lock_stat.sk_lock-AF_INET.holdtime-max

   3200 ++------------------------------------------------------------------+
        |                                                          .*.      |
   3000 ++    ..*.....                                           ..   ..    |
   2800 ++....        .    ....*......*...          .*.......  ..       .   |
        *.             *...               ...     ..         *.          .  |
   2600 ++                                   .  ..                        ..|
   2400 ++      O                             *.                            |
        |                                                                   *
   2200 ++                                                                  |
   2000 ++                                                                  |
        |                                                                   |
   1800 ++                                                   O              |
   1600 ++                                    O                             O
        O                      O      O              O              O       |
   1400 ++-------------O----------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 3505.11       -60.2%      1395.79  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                 2716.82       -39.3%      1649.75  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                 6221.93       -51.1%      3045.54  TOTAL lock_stat.sk_lock-AF_INET.holdtime-max
\f
                        lock_stat.&pipe->mutex/1.holdtime-max

   20000 ++-----------------------------------------------------------------+
   18000 O+     O                                                           |
         |                                                                  |
   16000 ++                                                                 |
   14000 ++                                                                 |
         |                                                                  |
   12000 ++                                                                 |
   10000 ++                            O                                    |
    8000 ++                                                                 |
         |                                                                  |
    6000 ++             O                                            O      |
    4000 ++    .*.......    ...*.......*......O   ....*..... O              O
         | ....         *...                  *...    O     .               |
    2000 *+                                                  *.......*......*
       0 ++-----------------------------------------------------------------+

\f
                        lock_stat.&pipe->mutex/1.holdtime-max

   12000 ++-----------------------------------------------------------------+
         |                                                   O              |
   10000 ++                    O                                            |
         |                                                                  |
         |                                                                  O
    8000 ++                                                                 |
         |                                                                  |
    6000 ++     O                                                           |
         |                                                                  |
    4000 O+                                                                 |
         |                             O     .*.....        .*.....  O      |
         |              O                ....       ..  ....       ..       |
    2000 *+.....*.......*......*.......*.     O       *.             *......*
         |                                                                  |
       0 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 2739.97      +183.8%      7777.33  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                 1916.89      +179.0%      5348.78  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                 4656.87      +181.9%     13126.12  TOTAL lock_stat.&pipe->mutex/1.holdtime-max
\f
                    lock_stat.(&task->u.tk_work)#2.holdtime-total

   24000 ++-----------------------------------O-----------------------------+
   22000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
   18000 ++                                                                 |
         |                                                                  |
   16000 ++                                                                 |
   14000 O+                                                                 |
   12000 ++                                                                 O
         |      O              O       O              O      O       O      |
   10000 ++             O                                                   |
    8000 ++                                                                 |
         |                                 ...*.......                      |
    6000 *+.....*.......*......*.......*...           *......    ....*......*
    4000 ++--------------------------------------------------*--------------+

\f
                    lock_stat.(&task->u.tk_work)#2.holdtime-total

   13000 ++-----------------------------------------------------------------+
         O              O                                                   |
   12000 ++     O              O                             O              |
   11000 ++                                   O                             |
         |                                            O              O      O
   10000 ++                            O                                    |
    9000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
    7000 ++                                                                 |
         |                                                                  |
    6000 *+.....                           ...*.......                      |
    5000 ++     *.......*......*.......*...           *......               |
         |                                                   *.......*......*
    4000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 5565.27      +130.3%     12815.79  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                 5200.06      +115.8%     11221.51  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                10765.33      +123.3%     24037.30  TOTAL lock_stat.(&task->u.tk_work)#2.holdtime-total
\f
                      lock_stat.rcu_read_lock_bh-R.holdtime-total

   800000 ++----------------------------------------------------------------+
          O      O                                                          |
   700000 ++                                                                |
          |                                                                 O
   600000 ++                                                                |
          |                                    O                            |
   500000 ++                                                                |
          |                                           O              O      |
   400000 ++                           O                     O              |
          |                                                                 |
   300000 ++             O      O                                           |
          |                  ...*......*.......                             |
   200000 ++  ...*.......*...                  *......           ....*......*
          *...                                        *......*...           |
   100000 ++----------------------------------------------------------------+

\f
                      lock_stat.rcu_read_lock_bh-R.holdtime-total

    1e+06 ++----------------------------------------------------------------+
          |                                                  O              |
   900000 ++                                                                |
   800000 ++                                                                |
          |                                                                 |
   700000 ++                                                         O      |
   600000 O+                                                                |
          |                                                                 |
   500000 ++             O      O              O                            O
   400000 ++     O                     O                                    |
          |                                           O                     |
   300000 ++                                                                |
   200000 *+.....                          ....*......    ...*.......       |
          |      *.......*......*......*...           *...           *......*
   100000 ++----------------------------------------------------------------+

\f
                      lock_stat.rcu_read_lock_bh-R.holdtime-total

   4.4e+07 ++---------------------------------------------------------------+
   4.3e+07 *+.....*.....                                  ....*......       |
           |            .    ....*......*......*......*...           *......*
   4.2e+07 ++            *...                                               |
   4.1e+07 ++                                                               |
           |                                                                |
     4e+07 ++                                         O                     |
   3.9e+07 ++                                                               |
   3.8e+07 ++                                                               |
           |                     O                                          O
   3.7e+07 ++                                  O                            |
   3.6e+07 ++            O                                                  |
           O                            O                     O      O      |
   3.5e+07 ++     O                                                         |
   3.4e+07 ++---------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               189373.52      +167.1%    505746.56  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
               172636.66      +206.5%    529063.93  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
             42779317.23       -14.7%  36472995.22  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
             43141327.41       -13.1%  37507805.71  TOTAL lock_stat.rcu_read_lock_bh-R.holdtime-total
\f
                       lock_stat.rcu_read_lock-R.holdtime-max

   2200 ++----------------------------*-------------------------------------*
        |           ...*.....      ..       ..           ....*......  ....  |
   2000 ++    ..*...         ..  ..           *......*...           *.      |
        | ....                 *.                                           |
   1800 *+                                                                  |
        |                                                                   |
   1600 ++                                                                  |
        |                                                                   |
   1400 ++                                                                  |
        |                                                                   |
   1200 ++                                                                  |
        |                             O                                     |
   1000 ++                                    O      O                      |
        O       O      O       O                             O      O       O
    800 ++------------------------------------------------------------------+

\f
                       lock_stat.rcu_read_lock-R.holdtime-max

   2400 ++-------------*----------------------------------------------------+
        |         ....     ...                      .*.....                 |
   2200 *+......*.            .                 ....       ..               |
   2000 ++                     *......    ....*.             *......*.......*
        |                             *...                                  |
   1800 ++                                                                  |
   1600 ++                                                                  |
        |       O                                                           |
   1400 ++                                                                  |
   1200 ++                     O                                            |
        |                                            O       O              |
   1000 ++                            O       O                     O       O
    800 O+                                                                  |
        |              O                                                    |
    600 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 2024.11       -54.0%       930.20  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                 2076.39       -49.7%      1045.08  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                 4100.50       -51.8%      1975.29  TOTAL lock_stat.rcu_read_lock-R.holdtime-max
\f
                        lock_stat.&mm->mmap_sem-W.holdtime-max

   130000 ++-----------------------------------*----------------------------+
          |                                   . :                           |
   120000 ++                                 .  :                           |
   110000 ++                               ..    :                          |
          *..                             .       :                         |
   100000 ++ ..                          .         :                        |
    90000 ++   ..                       .          :         *.....        .*
          |                            *            :      ..      ..  .... |
    80000 ++     *.......*..         ..              :   ..          *.     |
    70000 ++                ..      .                : ..                   |
          |                   ..   .                  *      O              |
    60000 ++                     ..                                         |
    50000 ++             O      *                                    O      |
          O      O                     O       O      O                     O
    40000 ++----------------------------------------------------------------+

\f
                        lock_stat.&mm->mmap_sem-W.holdtime-max

   120000 ++----------------------------------------------------------------+
          *.....                                                            |
   110000 ++    .    ....*..           *.....                              .*
   100000 ++     *...       .        ..      ..                          .. |
          |                  .      .          *..                     ..   |
    90000 ++                  ..   .              .                 .*.     |
    80000 ++                     ..                .             ...        |
          |                     *                   ..         ..           |
    70000 ++                                              ...*.             |
    60000 ++                                          *...                  |
          |                                                                 |
    50000 ++     O                                                   O      |
    40000 ++                                                                O
          O              O             O       O      O      O              |
    30000 ++--------------------O-------------------------------------------+

\f
                        lock_stat.&mm->mmap_sem-W.holdtime-max

   110000 ++----------------------------------------------------------------+
          |                                  ..*.                           |
   100000 ++                             ....    .                          |
    90000 ++            .*......    ...*.         ..                        |
          |          ...        *...                .       .*...           |
    80000 *+.....  ..                                .    ..     ...        |
          |      *.                                     ..          .       |
    70000 ++                                          *.             *......*
          |                                                                 |
    60000 ++                                                                |
    50000 ++                                                 O       O      |
          O      O                             O      O                     |
    40000 ++             O      O      O                                    |
          |                                                                 O
    30000 ++----------------------------------------------------------------+

\f
                        lock_stat.&mm->mmap_sem-W.holdtime-max

   250000 ++----------------------------------------------------------------+
          |              *                                                  |
          |             : +                                          *..    |
   200000 ++           :   +                                        +   ..  |
          |            :    +          *..                         +      ..|
          |           :      +       ..   ..                      +         |
   150000 ++         :        +    ..       ..                   +          *
          |         :          + ..           .                 +           |
   100000 ++       :            *              *......         +            |
          |        :                                  *...... +             |
          *...... :                                          *              |
    50000 ++     *                                                          |
          O      O       O      O      O       O      O      O       O      O
          |                                                                 |
        0 ++----------------------------------------------------------------+

\f
                        lock_stat.&mm->mmap_sem-W.holdtime-max

   480000 ++----------------------------------------------------------------+
          |                                                                 |
   460000 ++             O             O                                    |
   440000 ++                    O              O             O              O
          |                                                                 |
   420000 ++                                          O              O      |
          O      O                                                          |
   400000 ++                                                                |
          |                                                                 |
   380000 ++                                                                |
   360000 ++                                                     ....*......|
          |     .*.......*......*......    ....*.....       .*...           *
   340000 ++....                       *...          .  ....                |
          *.                                          *.                    |
   320000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                86138.49       -41.9%     50080.04  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                92765.86       -56.4%     40469.94  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                82836.83       -45.1%     45498.45  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               128868.66       -71.4%     36843.48  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               348946.32       +25.5%    437877.10  nhm-white/micro/aim7/misc_rtns_1
               739556.16       -17.4%    610769.01  TOTAL lock_stat.&mm->mmap_sem-W.holdtime-max
\f
                          lock_stat.nfsiod-R.holdtime-total

   24000 ++-----------------------------------O-----------------------------+
   22000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
   18000 ++                                                                 |
         |                                                                  |
   16000 ++                                                                 |
   14000 O+                                                                 |
   12000 ++                                                  O       O      O
         |      O              O       O              O                     |
   10000 ++             O                                                   |
    8000 ++                                                                 |
         |                         ....*......*.......*......            ...*
    6000 *+.....*.......*......*...                          *.......*...   |
    4000 ++-----------------------------------------------------------------+

\f
                          lock_stat.nfsiod-R.holdtime-total

   13000 ++-------------O---------------------------------------------------+
         O      O                                                           |
   12000 ++                    O                             O              |
   11000 ++                                   O       O              O      O
         |                                                                  |
   10000 ++                            O                                    |
    9000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
    7000 ++                                                                 |
         *.....                              .*.....                        |
    6000 ++    .                         ....       ..                      |
    5000 ++     *.......*......*.......*.             *......*.......*......|
         |                                                                  *
    4000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 5840.60      +123.6%     13058.00  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                 5449.57      +110.3%     11462.74  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                11290.17      +117.2%     24520.74  TOTAL lock_stat.nfsiod-R.holdtime-total
\f
                  lock_stat.&(&info->si_lock)->rlock.holdtime-total

   80000 ++-----------------------------------------------------------------+
         |                                                                  |
   70000 ++                                   *                             |
         |                                  .. :                            |
         |                                 .   :                            |
   60000 ++                               .     :                           |
         |                              ..       :           *              |
   50000 *+.                          .*          :         + +             |
         |  .                       ..            :        +   +            *
   40000 ++  .         .*.....    ..               :      +     +          .|
         |    ..    ...       . ..                  :    +       +        . |
         |        ..           *                     :  +         +     ..  |
   30000 ++     *.                                   : +           +   .    |
         O      O       O      O                      *             +O.     |
   20000 ++----------------------------O------O-------O------O-------*------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                42400.54       -47.1%     22416.67  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                42400.54       -47.1%     22416.67  TOTAL lock_stat.&(&info->si_lock)->rlock.holdtime-total
\f
                lock_stat.&(&nf->lru_lock)->rlock.contentions.ip_defrag

   110000 ++----------------------------------------------------------------+
          |                                                          O      |
   100000 ++                    O                     O                     O
          |                                    O                            |
    90000 ++                                                                |
          |                            O                                    |
    80000 ++                                                                |
          O      O                                           O              |
    70000 ++             O                                                  |
          |                                                                 |
    60000 ++                                                                |
          |            ..*......                                            |
    50000 *+.....  ....         *......    ....*......*......    ....*......|
          |      *.                    *...                  *...           *
    40000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                48336.50       +80.9%     87451.90  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                48336.50       +80.9%     87451.90  TOTAL lock_stat.&(&nf->lru_lock)->rlock.contentions.ip_defrag
\f
                        lock_stat.&pipe->mutex/1.waittime-total

   800000 ++----------------------------------------------------------------+
          |                                                                 |
   700000 ++                                   O                            |
          O                                                                 |
   600000 ++                                                                O
          |                                                                 |
   500000 ++                                                                |
          |                                                                 |
   400000 ++                    O                            O              |
          |      O       O                            O              O      |
   300000 ++                                                                |
          |                            O   ....*.....                       |
   200000 ++                   .*......*...          .           ....*..... |
          |   ...*.......  ....                       *......*...          .|
   100000 *+-------------*--------------------------------------------------*

\f
                        lock_stat.&pipe->mutex/1.waittime-total

    1e+06 ++------------------------------------------O---------------------+
   900000 ++                                                                |
          |             .*..                                                |
   800000 ++          ..    ..                       .*.....                |
   700000 ++        ..        ..      .*.......  ....       .    ....*......|
          |       ..            O ....         *.            *...           *
   600000 *+.....*              *.                                          |
   500000 ++                                                                |
   400000 ++     O                             O                            |
          O                                                          O      |
   300000 ++                                                                |
   200000 ++             O             O                     O              |
          |                                                                 |
   100000 ++                                                                O
        0 ++----------------------------------------------------------------+

\f
                        lock_stat.&pipe->mutex/1.waittime-total

   650000 ++----------------------------------------------------------------+
   600000 ++     O                                                          |
          |                            O              O                     |
   550000 ++                                                                |
   500000 ++                                   O                            O
          |                                                                 |
   450000 ++                                                .*.             |
   400000 ++                               ....*......  ....   ..    O      |
   350000 O+             O            .*...           *.         .          |
          *......*.......*.....     ..                            .        .*
   300000 ++                   .  ..                               ..    .. |
   250000 ++                    *.                                     ..   |
          |                                                          *.     |
   200000 ++                    O                                           |
   150000 ++----------------------------------------------------------------+

\f
                       lock_stat.&pipe->mutex/1.waittime-total

   22000 ++-----------------------------------------------------------------+
         O                                                                  |
   20000 ++                                                                 |
   18000 ++                                                                 |
         |      O                                     O                     |
   16000 ++             O                                    O              |
   14000 ++                    O       O      O                             O
         |                                                           O      |
   12000 ++                                                                 |
   10000 ++                                                                 |
         |                                                                  |
    8000 *+.....*.......*......*.......*.....                               |
    6000 ++                                  .    ....*......*.......*......*
         |                                    *...                          |
    4000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               160764.81      +172.0%    437281.51  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
               665119.31       -41.8%    386994.98  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               335214.35       +35.7%    454771.03  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                 7253.22      +115.0%     15592.90  nhm-white/micro/aim7/misc_rtns_1
              1168351.69       +10.8%   1294640.42  TOTAL lock_stat.&pipe->mutex/1.waittime-total
\f
                  lock_stat.&sb->s_type->i_mutex_key#2.waittime-total

   180000 ++----------------------------------------------------------------+
          |                                                                 |
   160000 ++                                   *.....                *      |
   140000 ++                                  +      .             .. :     |
          |                                  +        *..         .    :    |
   120000 *+....                            +            ..      .     :    |
          |     .                          +               ..  ..       :   |
   100000 ++     *.......                 +                   .          :  |
          |              *..             +                   *            : |
    80000 ++                ..          +             O      O            : |
    60000 O+                  ..      .*                                   :|
          |                     O ....                                      *
    40000 ++                    *.             O                            O
          |      O                                                          |
    20000 ++---------------------------O-----------------------------O------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               102707.56       -48.3%     53101.39  lkp-a04/micro/netperf/120s-200%-TCP_RR
               102707.56       -48.3%     53101.39  TOTAL lock_stat.&sb->s_type->i_mutex_key#2.waittime-total
\f
                          lock_stat.&rq->lock.holdtime-total

   1.6e+07 ++----------------------------------O----------------------------+
           |                                                                O
   1.4e+07 ++     O                                           O      O      |
           |                            O                                   |
           O             O       O                    O                     |
   1.2e+07 ++                                                               |
           |                                                                |
     1e+07 ++                                                               |
           |                                                                |
     8e+06 ++                                                               |
           |                                                                |
           |                                                                |
     6e+06 ++                                                               |
           *......*......    ....*......*......*......*.......    ...*......*
     4e+06 ++------------*------------------------------------*-------------+

\f
                          lock_stat.&rq->lock.holdtime-total

   1.6e+07 ++------------O--------------------------------------------------+
           |                                                  O             |
   1.4e+07 O+     O              O                                   O      |
           |                                   O                            O
   1.2e+07 ++                                         O                     |
           |                            O                                   |
     1e+07 ++                                                               |
           |                                                                |
     8e+06 ++                                                               |
           |                                                                |
     6e+06 ++                                                               |
           *......    ...*.......           ...*......                      |
     4e+06 ++     *...           *......*...          *.......*......*......*
           |                                                                |
     2e+06 ++---------------------------------------------------------------+

\f
                           lock_stat.&rq->lock.holdtime-total

    2.3e+07 ++--------------------------------------------------------------+
   2.25e+07 ++            O                                                 |
            |                           O       O      O      O             O
    2.2e+07 O+     O                                                 O      |
   2.15e+07 ++                                                              |
            |                    O                                          |
    2.1e+07 ++                                                              |
   2.05e+07 ++                                                              |
      2e+07 ++                                                              |
            |                                                               |
   1.95e+07 ++                                                              |
    1.9e+07 ++                                        .*.....               |
            |   ...*......    ...*.....           ....       .              |
   1.85e+07 *+..          *...         .    ....*.            *......    ...*
    1.8e+07 ++--------------------------*----------------------------*------+

\f
                           lock_stat.&rq->lock.holdtime-total

   2.65e+07 ++------------------------------------------------O-------------+
    2.6e+07 ++     O      O      O                     O             O      O
            O                           O       O                           |
   2.55e+07 ++                                                              |
    2.5e+07 ++                                                              |
            |                                                               |
   2.45e+07 ++                                                              |
    2.4e+07 ++                                                              |
   2.35e+07 ++                                                              |
            |                                                               |
    2.3e+07 ++                                                              |
   2.25e+07 ++                                                              |
            *......                  ...*.......*......*......    ...*......|
    2.2e+07 ++     *......*......*...                         *...          *
   2.15e+07 ++--------------------------------------------------------------+

\f
                          lock_stat.&rq->lock.holdtime-total

     2e+07 ++-------------------------------------------------*-------------+
   1.9e+07 *+.....*......    ....*......*......*......  ....         *......*
           |             *...                         *.                    |
   1.8e+07 ++                                         O                     |
   1.7e+07 ++                                                               |
           |                                                                |
   1.6e+07 ++                                                               |
   1.5e+07 ++                                                               |
   1.4e+07 ++                    O                                          O
           |                                   O                            |
   1.3e+07 ++                                                               |
   1.2e+07 O+                                                 O             |
           |             O              O                            O      |
   1.1e+07 ++                                                               |
     1e+07 ++-----O---------------------------------------------------------+

\f
                          lock_stat.&rq->lock.holdtime-total

   2.8e+07 O+-----O-------------------------------------------O-------------+
           |                            O                                   |
   2.7e+07 ++            O       O                                          |
           |                                          O                     |
           |                                                         O      O
   2.6e+07 ++                                  O                            |
           |                                                                |
   2.5e+07 ++                                                               |
           |                                                                |
   2.4e+07 ++                                                               |
           |                                                                |
           |                                  .*......                     .*
   2.3e+07 *+.....                        ....        *.......         .... |
           |      *......*.......*......*.                    *......*.     |
   2.2e+07 ++---------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              4686248.48      +193.2%  13741421.79  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
              4321965.80      +219.0%  13786069.16  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
             18476736.24       +19.4%  22068859.06  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
             22034876.10       +18.1%  26016846.30  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
             19149179.45       -32.5%  12934745.26  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
             22649688.71       +19.6%  27084793.20  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
             91318694.78       +26.6% 115632734.76  TOTAL lock_stat.&rq->lock.holdtime-total
\f
                        lock_stat.&p->pi_lock.holdtime-total

   8e+06 ++-----------------------------------------------------------------+
         |                                    O                             O
   7e+06 ++                                                          O      |
         |      O                                            O              |
         O              O      O       O              O                     |
   6e+06 ++                                                                 |
         |                                                                  |
   5e+06 ++                                                                 |
         |                                                                  |
   4e+06 ++                                                                 |
         |                                                                  |
         |                                                                  |
   3e+06 ++                                                                 |
         *......*.......           ....*......*.......*......            ...*
   2e+06 ++-------------*------*-----------------------------*-------*------+

\f
                        lock_stat.&p->pi_lock.holdtime-total

   9e+06 ++-----------------------------------------------------------------+
         |                                                   O              |
   8e+06 ++     O       O      O                                     O      |
         O                                                                  O
   7e+06 ++                                   O       O                     |
         |                                                                  |
   6e+06 ++                            O                                    |
         |                                                                  |
   5e+06 ++                                                                 |
         |                                                                  |
   4e+06 ++                                                                 |
         |                                                                  |
   3e+06 *+.....            ...*.......*......*.......           ....*......|
         |      *.......*...                          *......*...           *
   2e+06 ++-----------------------------------------------------------------+

\f
                          lock_stat.&p->pi_lock.holdtime-total

    1.4e+07 ++--------------------------------------------------------------+
            |                                                               |
   1.35e+07 ++            O                     O      O      O             O
            |                           O                            O      |
            O      O                                                        |
    1.3e+07 ++                   O                                          |
            |                                                               |
   1.25e+07 ++                                                              |
            |                                                               |
    1.2e+07 ++                                        .*.....               |
            |   ...*......    ...*.....           ....       .              |
            *...          *...         .    ....*.            *......    ...*
   1.15e+07 ++                          *...                         *...   |
            |                                                               |
    1.1e+07 ++--------------------------------------------------------------+

\f
                         lock_stat.&p->pi_lock.holdtime-total

   1.3e+07 ++---------------------------------------------------------------+
           *......*......    ....*......*......*......    ....*......*......*
   1.2e+07 ++            *...                         *...                  |
   1.1e+07 ++                                                               |
           |                                          O                     |
     1e+07 ++                                                               |
           |                                                                |
     9e+06 ++                                                               |
           |                     O                                          O
     8e+06 ++                                  O                            |
     7e+06 ++                                                 O             |
           O             O              O                                   |
     6e+06 ++                                                        O      |
           |      O                                                         |
     5e+06 ++---------------------------------------------------------------+

\f
                          lock_stat.&p->pi_lock.holdtime-total

    1.9e+07 ++--------------------------------------------------------------+
            |      O                                          O             |
   1.85e+07 ++                                                              |
            O             O             O                                   |
    1.8e+07 ++                   O                     O                    |
            |                                                               O
   1.75e+07 ++                                  O                           |
            |                                                        O      |
    1.7e+07 ++                                                              |
            |                                                               |
   1.65e+07 ++                                                              |
            |                                                               |
    1.6e+07 ++                              ....*......                    .*
            *......    ...*......*......*...           *......*......  .... |
   1.55e+07 ++-----*-------------------------------------------------*------+

\f
                         lock_stat.&p->pi_lock.holdtime-total

   6.5e+06 ++---------------------------------------------------------------+
   6.4e+06 O+     O                                                         |
           |             O       O                    O       O      O      |
   6.3e+06 ++                           O      O                            |
   6.2e+06 ++                                                               O
           |                                                                |
   6.1e+06 ++                                                               |
     6e+06 ++                                                               |
   5.9e+06 ++                                                               |
           |                                                                |
   5.8e+06 ++                                                               |
   5.7e+06 ++                                                               |
           |          ...*.....                                             |
   5.6e+06 *+.....*...         ..    ...*......           ....*......    ...*
   5.5e+06 ++--------------------*-------------*------*--------------*------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              2278609.42      +196.4%   6753476.95  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
              2712770.22      +178.2%   7547372.01  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
             11691535.47       +14.2%  13354970.63  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
             12191166.14       -39.1%   7420345.79  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
             15765175.80       +14.5%  18051845.03  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
              5549747.60       +14.3%   6343855.91  nhm-white/micro/aim7/misc_rtns_1
             50189004.65       +18.5%  59471866.33  TOTAL lock_stat.&p->pi_lock.holdtime-total
\f
                       lock_stat.&anon_vma->rwsem.acq-bounces

   34000 *+-------------------------------------------*------*-------*------+
   32000 ++     *.......*......*.......*......*...                         .|
         |                                                                  *
   30000 ++                                                                 |
   28000 ++                                                                 |
         |                                                                  |
   26000 ++                                                                 |
   24000 ++                                                                 |
   22000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
   18000 ++                                                                 |
         |      O       O                                                   |
   16000 O+                    O       O      O              O       O      O
   14000 ++-------------------------------------------O---------------------+

\f
                       lock_stat.&anon_vma->rwsem.acq-bounces

   35000 ++-----------------------------------------------------------------+
         *......*.......*......*.......*......    ....*......*.......*......*
         |                                    *...                          |
   30000 ++                                                                 |
         |                                                                  |
         |                                                                  |
   25000 ++                                                                 |
         |                                                                  |
   20000 O+                                                                 |
         |      O                                                           |
         |              O      O       O      O                      O      O
   15000 ++                                                  O              |
         |                                                                  |
         |                                            O                     |
   10000 ++-----------------------------------------------------------------+

\f
                       lock_stat.&anon_vma->rwsem.acq-bounces

   35000 ++-------------*---------------------------------------------------+
         *......*...          .    ....*......*.......*......*.......*......|
         |                     *...                                         *
   30000 ++                                                                 |
         |                                                                  |
         |                                                                  |
   25000 ++                                                                 |
         |                                                                  |
   20000 O+                                                                 |
         |      O              O                                     O      |
         |              O                     O              O              |
   15000 ++                            O              O                     |
         |                                                                  O
         |                                                                  |
   10000 ++-----------------------------------------------------------------+

\f
                       lock_stat.&anon_vma->rwsem.acq-bounces

   36000 *+-----------------------------------------------------------------+
   34000 ++    .    ....*......    ....*......*.......*......*.......*......*
         |      *...           *...                                         |
   32000 ++                                                                 |
   30000 ++                                                                 |
         |                                                                  |
   28000 ++     O                             O                             |
   26000 ++                                                                 |
   24000 ++                            O                             O      O
         O              O                             O                     |
   22000 ++                                                  O              |
   20000 ++                                                                 |
         |                                                                  |
   18000 ++                    O                                            |
   16000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                32893.70       -51.8%     15861.50  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                32892.30       -48.0%     17089.40  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                33182.30       -47.2%     17511.00  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                34195.00       -31.3%     23481.00  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               133163.30       -44.5%     73942.90  TOTAL lock_stat.&anon_vma->rwsem.acq-bounces
\f
                          lock_stat.&p->pi_lock.acq-bounces

   36000 ++-----------------------------------------------------------------+
   34000 ++                 ...*.......           ....*......               |
         |   ...*.......*...           *......*...           *.......*......*
   32000 *+..                                                               |
   30000 ++                                                                 |
         |                                                                  |
   28000 ++                                                                 |
   26000 ++                                                                 |
   24000 ++                                                                 |
         |                                                                  |
   22000 ++                                                                 |
   20000 ++                                                                 |
         O              O              O      O                             |
   18000 ++     O              O                      O      O       O      O
   16000 ++-----------------------------------------------------------------+

\f
                          lock_stat.&p->pi_lock.acq-bounces

   36000 ++-----------------------------------------------------------------+
   34000 ++                                               ...*.......*......|
         *......*.......*......*.......*......*.......*...                  *
   32000 ++                                                                 |
   30000 ++                                                                 |
         |                                                                  |
   28000 ++                                                                 |
   26000 ++                                                                 |
   24000 ++                                                                 |
         |                                                                  |
   22000 ++                                                                 |
   20000 ++                                                                 |
         O      O              O              O       O      O       O      |
   18000 ++             O                                                   |
   16000 ++----------------------------O------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                33338.70       -46.2%     17937.80  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                33479.30       -45.1%     18395.40  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                66818.00       -45.6%     36333.20  TOTAL lock_stat.&p->pi_lock.acq-bounces
\f
              lock_stat.&(&zone->lock)->rlock.contentions.__free_pages_ok

   145000 ++----------------------------------------------------------------+
   140000 O+                           O                                    |
          |      O                             O             O       O      O
   135000 ++             O      O                     O                     |
   130000 ++                                                                |
   125000 ++                                                                |
   120000 ++                                                                |
          |                                                                 |
   115000 ++                                                                |
   110000 ++                                                                |
   105000 ++                                                                |
   100000 ++                                                                |
          *......    ....*......*......*.......*......    ...*.......*......|
    95000 ++     *...                                 *...                  *
    90000 ++----------------------------------------------------------------+

\f
              lock_stat.&(&zone->lock)->rlock.contentions.__free_pages_ok

   220000 ++----------------------------------------------------------------+
          |                                                                 |
   200000 ++     O                                                          |
          |              O                                                  |
   180000 ++                           O                     O       O      |
          O                                                                 |
   160000 ++                                                                |
          |                                    O                            O
   140000 ++                    O                     O                     |
          |                                                                 |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          *......*.......*......*......*.......*......*......*.......*......*
    80000 ++----------------------------------------------------------------+

\f
              lock_stat.&(&zone->lock)->rlock.contentions.__free_pages_ok

   400000 ++----------------------------------------------------------------+
          |              O                                                  |
          O      O                     O                     O              |
   350000 ++                                   O      O              O      |
          |                     O                                           O
          |                                                                 |
   300000 ++                                                                |
          |                                                                 |
   250000 ++                                                                |
          |                                                                 |
          |                                                                 |
   200000 ++                                                                |
          |          ....*......*......*.......                          ...*
          *......*...                          *......*......*.......*...   |
   150000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                96703.00       +41.6%    136899.00  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                90157.60       +86.3%    167980.50  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               175013.90      +102.5%    354357.70  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               361874.50       +82.2%    659237.20  TOTAL lock_stat.&(&zone->lock)->rlock.contentions.__free_pages_ok
\f
                      lock_stat.tasklist_lock-W.waittime-total

   5500 ++------------------------------------------------------------------+
        |                                     O                             |
   5000 ++     .*                         ....*.                            |
   4500 ++  ...  +                   .*...      .                         ..*
        | ..      +                ..            ..                   ....  |
   4000 *+         +             ..                .        .*......*.      |
   3500 ++          +  O      .*.                   .    ... O              |
        |            +     ...                       O ..                   |
   3000 ++            +  ..                          *.                     |
   2500 ++             *.                                                   |
        |       O              O                                    O       |
   2000 ++                                                                  |
   1500 O+                            O                                     |
        |                                                                   |
   1000 ++------------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 3957.60       -33.3%      2638.84  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                 3957.60       -33.3%      2638.84  TOTAL lock_stat.tasklist_lock-W.waittime-total
\f
          lock_stat.&(&zone->lock)->rlock.contentions.get_page_from_freelist

   170000 ++-----O----------------------------------------------------------+
          |                                                                 |
   160000 ++                                                         O      |
   150000 O+             O             O                     O              |
          |                                                                 |
   140000 ++                                                                |
   130000 ++                    O              O                            O
          |                                                                 |
   120000 ++                                                                |
   110000 ++                                          O                     |
          |                                                                 |
   100000 ++                                                                |
    90000 ++                                                                |
          *......*.......*......*......*.......*......*......*.......*......*
    80000 ++----------------------------------------------------------------+

\f
          lock_stat.&(&zone->lock)->rlock.contentions.get_page_from_freelist

   320000 ++----------------------------------------------------------------+
          |                                                  O              |
   300000 O+                                                                |
   280000 ++     O                     O                             O      |
          |                     O              O      O                     |
   260000 ++             O                                                  O
   240000 ++                                                                |
          |                                                                 |
   220000 ++                                                                |
   200000 ++                                                                |
          |                                                                 |
   180000 ++                                                                |
   160000 ++         ....*......    ...*.......*......                   ...*
          *......*...           *...                  *......*.......*...   |
   140000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                86242.90       +66.3%    143438.60  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               158521.60       +75.2%    277692.40  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               244764.50       +72.1%    421131.00  TOTAL lock_stat.&(&zone->lock)->rlock.contentions.get_page_from_freelist
\f
                    lock_stat.&(&base->lock)->rlock.holdtime-total

     7e+06 ++---------------------------------------------------------------+
   6.5e+06 *+.....*......    ....*......*......*......    ....*......*......*
           |             *...                         *...                  |
     6e+06 ++                                                               |
   5.5e+06 ++                                                               |
           |                                                                |
     5e+06 ++                                         O                     |
   4.5e+06 ++                                                               |
     4e+06 ++                                                               |
           |                                                                |
   3.5e+06 ++                    O             O                            O
     3e+06 ++                                                               |
           O             O                                    O             |
   2.5e+06 ++                           O                            O      |
     2e+06 ++-----O---------------------------------------------------------+

\f
                   lock_stat.&(&base->lock)->rlock.holdtime-total

   48000 ++-----------------------------------------------------------------+
   47000 ++                                   O                             O
         |                             O                                    |
   46000 ++                                                                 |
   45000 O+     O                                                           |
         |                     O                                     O      |
   44000 ++             O                                                   |
   43000 ++                                           O      O              |
   42000 ++                                                                 |
         |                                                                  |
   41000 ++                                                                 |
   40000 ++                                                                 |
         *......*.......*......                                            .*
   39000 ++                    *.......*......*.......*......*.......  .... |
   38000 ++----------------------------------------------------------*------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              6501677.06       -53.6%   3013664.80  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                38886.45       +15.4%     44888.27  nhm-white/micro/aim7/misc_rtns_1
              6540563.51       -53.2%   3058553.07  TOTAL lock_stat.&(&base->lock)->rlock.holdtime-total
\f
                     lock_stat.&(&nf->lru_lock)->rlock.con-bounces

   400000 ++-------------------------------------------------O--------------+
   380000 O+                           O                                    |
          |      O       O      O              O      O              O      |
   360000 ++                                                                O
   340000 ++                                                                |
          |                                                                 |
   320000 ++                                                                |
   300000 ++                                                                |
   280000 ++                                                                |
          |                                                                 |
   260000 ++                                                                |
   240000 ++                                                                |
          |          ....*......                                            |
   220000 *+.....*...           *......*.......*......*......*.......*......*
   200000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               214843.60       +74.5%    374915.00  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               214843.60       +74.5%    374915.00  TOTAL lock_stat.&(&nf->lru_lock)->rlock.con-bounces
\f
                     lock_stat.&(&nf->lru_lock)->rlock.contentions

   400000 ++-------------------------------------------------O--------------+
   380000 O+                           O                                    |
          |      O       O      O              O      O              O      |
   360000 ++                                                                O
   340000 ++                                                                |
          |                                                                 |
   320000 ++                                                                |
   300000 ++                                                                |
   280000 ++                                                                |
          |                                                                 |
   260000 ++                                                                |
   240000 ++                                                                |
          |          ....*......                                            |
   220000 *+.....*...           *......*.......*......*......*.......*......*
   200000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               214846.50       +74.5%    374916.00  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               214846.50       +74.5%    374916.00  TOTAL lock_stat.&(&nf->lru_lock)->rlock.contentions
\f
                   lock_stat.&(&nf->lru_lock)->rlock.waittime-total

   1.2e+06 ++--------------------------------------------------------O------+
           |                                   O              O             |
   1.1e+06 O+     O              O      O             O                     O
           |             O                                                  |
           |                                                                |
     1e+06 ++                                                               |
           |                                                                |
    900000 ++                                                               |
           |                                                                |
    800000 ++                                                               |
           |                                                                |
           |                                                                |
    700000 ++                                                               |
           *......*......*.......*......*......*......    ....*......*......*
    600000 ++-----------------------------------------*---------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               641418.67       +75.6%   1126321.06  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               641418.67       +75.6%   1126321.06  TOTAL lock_stat.&(&nf->lru_lock)->rlock.waittime-total
\f
             lock_stat.&(&nf->lru_lock)->rlock.contentions.inet_frag_find

   380000 ++----------------------------------------------------------------+
   360000 ++                                                 O              |
          O      O                     O                                    |
   340000 ++             O                     O      O              O      |
   320000 ++                    O                                           O
          |                                                                 |
   300000 ++                                                                |
   280000 ++                                                                |
   260000 ++                                                                |
          |                                                                 |
   240000 ++                                                                |
   220000 ++                                                                |
          |                                                                 |
   200000 *+.....*.......*......*......*.......*......*......*.......*......*
   180000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               192855.90       +76.0%    339378.40  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               192855.90       +76.0%    339378.40  TOTAL lock_stat.&(&nf->lru_lock)->rlock.contentions.inet_frag_find
\f
                       lock_stat.sb_writers#7-R.holdtime-total

   1e+07 ++-----------------------------------------------------------------+
         |              O                                                   |
   9e+06 ++                                                                 |
   8e+06 ++                                           O                     |
         |                                                                  |
   7e+06 ++                                                                 |
   6e+06 O+                                                                 |
         |                             O                                    |
   5e+06 ++                                                                 |
   4e+06 ++     O      .*...                                                O
         |          ...     ..                O             .*.....         |
   3e+06 ++       ..          .O   ....*...               ..       ..O      |
   2e+06 ++    .*.             *...        ..           ..   O       *......|
         | ....                              .    ....*.                    *
   1e+06 *+-----------------------------------*-----------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              2322889.38      +111.6%   4915093.78  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
              2322889.38      +111.6%   4915093.78  TOTAL lock_stat.sb_writers#7-R.holdtime-total
\f
                  lock_stat.&(&n->list_lock)->rlock.holdtime-total

   70000 ++-----------------------------------------------------------------+
         |                                 ...*..                           |
   65000 *+.....*.......*......*.......*...      ..         .*.......    ...*
   60000 ++                                        ..     ..         *...   |
         |                                           .  ..                  |
   55000 ++                                           *.                    |
         |                                                                  |
   50000 ++                                                                 |
         |                                                                  |
   45000 ++                                                                 |
   40000 ++             O                                                   |
         O                     O       O      O       O                     O
   35000 ++                                                  O              |
         |      O                                                    O      |
   30000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                63744.63       -42.3%     36809.55  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                63744.63       -42.3%     36809.55  TOTAL lock_stat.&(&n->list_lock)->rlock.holdtime-total
\f
             lock_stat.&(&nf->lru_lock)->rlock.contentions.inet_frag_kill

   360000 ++-------------------------------------------------O--------------+
          |                                                                 |
   340000 O+     O                     O                                    |
   320000 ++             O                            O                     |
          |                     O              O                     O      |
   300000 ++                                                                O
   280000 ++                                                                |
          |                                                                 |
   260000 ++                                                                |
   240000 ++                                                                |
          |                                                                 |
   220000 ++                                                                |
   200000 ++                                                                |
          |          ....*......*......*.......*......*......*.......    ...*
   180000 *+-----*---------------------------------------------------*------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               188500.60       +71.4%    323001.70  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               188500.60       +71.4%    323001.70  TOTAL lock_stat.&(&nf->lru_lock)->rlock.contentions.inet_frag_kill
\f
                   lock_stat.&(&queue->lock)->rlock.holdtime-total

   18000 ++-----------------------------------------------------------------+
         O                                                                  |
   16000 ++                                                                 |
         |                                                                  |
         |                                    O                             |
   14000 ++                                                                 O
         |                                                   O              |
   12000 ++                                                          O      |
         |              O      O       O              O                     |
   10000 ++     O                                                           |
         |                                                                  |
         |                                                                  |
    8000 ++  ...*.......           ....*......*.......*......            ...*
         *...           *......*...                          *.......*...   |
    6000 ++-----------------------------------------------------------------+

\f
                   lock_stat.&(&queue->lock)->rlock.holdtime-total

   14000 ++-----------------------------------------------------------------+
         |                                                                  |
   13000 ++             O                                                   |
   12000 ++                                   O                             |
         O      O              O                                            |
   11000 ++                                           O              O      O
         |                             O                     O              |
   10000 ++                                                                 |
         |                                                                  |
    9000 ++                                                                 |
    8000 ++                                                                 |
         |                    .*.......                                     |
    7000 *+.....          ....         *......                              |
         |      *.......*.                    *.......*......*.......       |
    6000 ++----------------------------------------------------------*------*

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 7403.45       +69.5%     12552.28  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                 6674.77       +73.4%     11577.26  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                14078.22       +71.4%     24129.54  TOTAL lock_stat.&(&queue->lock)->rlock.holdtime-total
\f
                      lock_stat.&(&zone->lock)->rlock.contentions

   125000 ++----------------------------------------------------------------+
          |                                                                 |
   120000 O+                           O                                    |
          |      O              O              O             O       O      O
   115000 ++             O                            O                     |
          |                                                                 |
   110000 ++                                                                |
          |                                                                 |
   105000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |          ....*......    ...*.......                             |
    95000 *+.....*...           *...           *.....       .*.......*......*
          |                                          .  ....                |
    90000 ++------------------------------------------*---------------------+

\f
                      lock_stat.&(&zone->lock)->rlock.contentions

   190000 ++----------------------------------------------------------------+
   180000 ++     O                                                          |
          |                                                          O      |
   170000 ++             O             O                     O              |
   160000 O+                                                                |
   150000 ++                                                                |
   140000 ++                                   O                            |
          |                     O                                           O
   130000 ++                                          O                     |
   120000 ++                                                                |
   110000 ++                                                                |
   100000 ++                                                                |
          |                                                                 |
    90000 *+.....*.......*......*......*.......*......*......*.......*......*
    80000 ++----------------------------------------------------------------+

\f
                      lock_stat.&(&zone->lock)->rlock.contentions

   340000 ++-------------------------------------------------O--------------+
          O      O                     O                                    |
   320000 ++             O                            O              O      |
   300000 ++                    O              O                            |
          |                                                                 O
   280000 ++                                                                |
   260000 ++                                                                |
          |                                                                 |
   240000 ++                                                                |
   220000 ++                                                                |
          |                                                                 |
   200000 ++                                                                |
   180000 ++                                                                |
          |          ....*......    ...*.......*......                   ...*
   160000 *+-----*--------------*---------------------*------*-------*------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                94783.20       +23.9%    117455.50  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                88482.40       +76.4%    156071.70  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               167064.80       +89.4%    316387.80  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               350330.40       +68.4%    589915.00  TOTAL lock_stat.&(&zone->lock)->rlock.contentions
\f
                      lock_stat.&(&zone->lock)->rlock.con-bounces

   125000 ++----------------------------------------------------------------+
          |                                                                 |
   120000 O+                           O                                    |
          |      O              O              O             O       O      O
   115000 ++             O                            O                     |
          |                                                                 |
   110000 ++                                                                |
          |                                                                 |
   105000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |          ....*......    ...*.......                             |
    95000 *+.....*...           *...           *.....       .*.......*......*
          |                                          .  ....                |
    90000 ++------------------------------------------*---------------------+

\f
                      lock_stat.&(&zone->lock)->rlock.con-bounces

   190000 ++----------------------------------------------------------------+
   180000 ++     O                                                          |
          |                                                          O      |
   170000 ++             O             O                     O              |
   160000 O+                                                                |
   150000 ++                                                                |
   140000 ++                                   O                            |
          |                     O                                           O
   130000 ++                                          O                     |
   120000 ++                                                                |
   110000 ++                                                                |
   100000 ++                                                                |
          |                                                                 |
    90000 *+.....*.......*......*......*.......*......*......*.......*......*
    80000 ++----------------------------------------------------------------+

\f
                      lock_stat.&(&zone->lock)->rlock.con-bounces

   340000 ++-------------------------------------------------O--------------+
          O      O                     O                                    |
   320000 ++             O                            O              O      |
   300000 ++                    O              O                            |
          |                                                                 O
   280000 ++                                                                |
   260000 ++                                                                |
          |                                                                 |
   240000 ++                                                                |
   220000 ++                                                                |
          |                                                                 |
   200000 ++                                                                |
   180000 ++                                                                |
          |          ....*......    ...*.......*......                   ...*
   160000 *+-----*--------------*---------------------*------*-------*------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                94770.00       +23.9%    117437.60  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                88470.60       +76.3%    156013.30  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               167035.60       +89.0%    315657.40  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               350276.20       +68.2%    589108.30  TOTAL lock_stat.&(&zone->lock)->rlock.con-bounces
\f
                 lock_stat.&(&hb->chain_lock)->rlock.waittime-total

   5000 ++------O-----------------------------------------------------------+
        |                                                    O              |
   4500 ++                                    O                             |
        |                                            O              O       |
   4000 ++                                                                  |
        O                      O                                            O
   3500 ++             O              O                                     |
        |                                                                   |
   3000 ++                                                                  |
        |                    ..*...          .*.....                        |
   2500 *+......*......  ....      ..     ...       .    ....*......    ....*
        |              *.            .  ..           *...           *...    |
   2000 ++                            *.                                    |
        |                                                                   |
   1500 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 2435.44       +68.8%      4109.92  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                 2435.44       +68.8%      4109.92  TOTAL lock_stat.&(&hb->chain_lock)->rlock.waittime-total
\f
                   lock_stat.(&(&wb->dwork)->work).holdtime-total

   30000 ++-----------------------------------------------------------------+
         |                                                   *              |
         |                                                  : :             |
   25000 ++                                                 :O:             |
         |                                                 :   :            |
         |                                                :     :           |
   20000 ++                                               :      :          |
         |              O                                :       :          |
   15000 O+                                              :        :         |
         |      O             .*...           O       O :          : O      O
         |                  ..     ... O               :            :       |
   10000 *+....           ..          .                :            :       |
         |     .    ....*.             *......*.......*              *......|
         |      *...                                                        *
    5000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                11039.58       +38.2%     15255.39  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                11039.58       +38.2%     15255.39  TOTAL lock_stat.(&(&wb->dwork)->work).holdtime-total
\f
                           lock_stat.&wq->wait.holdtime-total

   1.55e+07 ++--------------------------------------------------------------+
            |             O                            O      O             O
    1.5e+07 ++                          O       O                    O      |
            O      O                                                        |
            |                                                               |
   1.45e+07 ++                   O                                          |
            |                                                               |
    1.4e+07 ++                                                              |
            |                                          *...                 |
   1.35e+07 ++                                       ..    ..               |
            |   ...*......                         ..        .              |
            *...          *......*......         ..           *......    ...*
    1.3e+07 ++                          *.......*                    *...   |
            |                                                               |
   1.25e+07 ++--------------------------------------------------------------+

\f
                          lock_stat.&wq->wait.holdtime-total

   1.6e+07 ++---------------------------------------------------------------+
           |                                                                |
   1.4e+07 *+.....*......*.......*......*......*......    ....*......*......*
           |                                          *...                  |
   1.2e+07 ++                                                               |
           |                                          O                     |
     1e+07 ++                                                               |
           |                                                                |
     8e+06 ++                                                               |
           |                     O             O                            O
     6e+06 ++                                                               |
           O             O              O                     O      O      |
     4e+06 ++                                                               |
           |      O                                                         |
     2e+06 ++---------------------------------------------------------------+

\f
                           lock_stat.&wq->wait.holdtime-total

   2.15e+07 ++--------------------------------------------------------------+
            |      O                                          O             |
    2.1e+07 ++                                                              |
            O             O             O                                   |
   2.05e+07 ++                                         O                    |
            |                    O                                          |
      2e+07 ++                                                              O
            |                                   O                           |
   1.95e+07 ++                                                       O      |
            |                                                               |
    1.9e+07 ++                                                              |
            |                                                              .*
   1.85e+07 ++                       ...*.......*......                .... |
            *......    ...*......*...                  *......*......*.     |
    1.8e+07 ++-----*--------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
             13153773.59       +14.6%  15073649.01  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
             13866421.93       -56.7%   6004955.18  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
             18281886.21       +12.2%  20503338.70  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
             45302081.74        -8.2%  41581942.88  TOTAL lock_stat.&wq->wait.holdtime-total
\f
                      lock_stat.zone->wait_table+i.holdtime-total

   320000 ++----------------------------------------------------------------+
          O              O      O                                           |
   300000 ++     O                                           O       O      |
   280000 ++                                   O      O                     |
          |                            O                                    |
   260000 ++                                                                O
          |                                                                 |
   240000 ++                                                                |
          |                                                                 |
   220000 ++                                                                |
   200000 ++                                                                |
          *.....                                                            |
   180000 ++    .    ....*......*......*.......*......*......*.......*......|
          |      *...                                                       *
   160000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               182856.27       +59.9%    292446.19  nhm-white/micro/aim7/misc_rtns_1
               182856.27       +59.9%    292446.19  TOTAL lock_stat.zone->wait_table+i.holdtime-total
\f
                    lock_stat.&(&zone->lock)->rlock.waittime-total

   700000 O+----------------------------------------------------------------+
          |                            O       O                            O
   680000 ++     O              O                                           |
   660000 ++             O                            O      O       O      |
          |                                                                 |
   640000 ++                                                                |
          |                                                                 |
   620000 ++                                                                |
          |                                                                 |
   600000 ++                        ...*.......*..                          |
   580000 *+.....*.......*......*...              ..        .*.......*......*
          |                                         ..    ..                |
   560000 ++                                            ..                  |
          |                                           *.                    |
   540000 ++----------------------------------------------------------------+

\f
                    lock_stat.&(&zone->lock)->rlock.waittime-total

   1.1e+06 ++---------------------------------------------------------------+
           |      O                                                         |
     1e+06 ++                                                               |
           |             O              O                     O      O      |
           O                                                                |
    900000 ++                                                               |
           |                     O             O                            |
    800000 ++                                                               O
           |                                          O                     |
    700000 ++                                                               |
           |                                                                |
           |                                                                |
    600000 ++                                      ...*.......           ...*
           *......*......*.......*......*......*...           *......*...   |
    500000 ++---------------------------------------------------------------+

\f
                    lock_stat.&(&zone->lock)->rlock.waittime-total

   2.1e+06 ++---------------------------------------------------------------+
     2e+06 O+                                                               |
           |      O      O              O                     O      O      |
   1.9e+06 ++                    O             O      O                     |
   1.8e+06 ++                                                               O
   1.7e+06 ++                                                               |
   1.6e+06 ++                                                               |
           |                                                                |
   1.5e+06 ++                                                               |
   1.4e+06 ++                                                               |
   1.3e+06 ++                                                               |
   1.2e+06 ++                                                               |
           |                                                                |
   1.1e+06 *+.....    ...*.......*......*......*......    ....*......    ...*
     1e+06 ++-----*-----------------------------------*--------------*------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               583446.68       +16.1%    677135.11  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
               549820.04       +64.9%    906871.50  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              1049341.73       +82.8%   1918502.18  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
              2182608.45       +60.5%   3502508.79  TOTAL lock_stat.&(&zone->lock)->rlock.waittime-total
\f
              lock_stat.&(&xprt->transport_lock)->rlock.holdtime-total

   22000 O+-----------------------------------------------------------------+
         |                                                                  |
   20000 ++                                                                 |
         |                                    O                             |
   18000 ++                                                  O              O
         |                                                           O      |
   16000 ++             O                                                   |
         |      O              O       O              O                     |
   14000 ++                           .*...                                 |
         |                          ..     ..                               |
   12000 ++  ...*.......          ..         .                              |
         *...           *...... ..            *.......*......            ...*
   10000 ++                    *                             *.......*...   |
         |                                                                  |
    8000 ++-----------------------------------------------------------------+

\f
              lock_stat.&(&xprt->transport_lock)->rlock.holdtime-total

   18000 ++-----------------------------------------------------------------+
         |      O       O                                                   |
   17000 ++                                   O                             |
   16000 O+                    O       O                                    |
         |                                            O      O       O      O
   15000 ++                                                                 |
   14000 ++                                                                 |
         |                                                                  |
   13000 ++                                                                 |
   12000 ++                                                                 |
         |                                                                  |
   11000 *+..               ...*.......                                     |
   10000 ++  ..       ..*...           *......                   ....*......|
         |     .  ....                        *.......*......*...           *
    9000 ++-----*-----------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                11107.93       +53.7%     17076.97  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                10040.37       +61.6%     16221.68  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                21148.30       +57.5%     33298.65  TOTAL lock_stat.&(&xprt->transport_lock)->rlock.holdtime-total
\f
                 lock_stat.&sb->s_type->i_lock_key#1.holdtime-total

   24000 ++-----------------------------------------------------------------+
         |                                                                  O
   22000 O+                                   O                             |
   20000 ++                                                                 |
         |                                                                  |
   18000 ++                                           *                     |
         |      O                                    . :     O       O      |
   16000 ++                    O                    .   :                   |
         |                             *..        ..  O  :                  |
   14000 *+..           O            ..   ..     .        :                 |
   12000 ++  ..                    ..       ..  .         :                 |
         |     .    ....*.....    .            .           :                |
   10000 ++     *...          . ..            *             :            ...*
         |                     *                             *.......*...   |
    8000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                11841.06       +49.7%     17725.30  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                11841.06       +49.7%     17725.30  TOTAL lock_stat.&sb->s_type->i_lock_key#1.holdtime-total
\f
                        lock_stat.&pipe->mutex/1.acq-bounces

   34000 ++-----------------------------------------------------------------+
         |                                                                  |
   32000 O+             O      O                                            O
   30000 ++     O                             O                             |
         |                                            O              O      |
   28000 ++                                                  O              |
         |                             O                                    |
   26000 ++                                                                 |
         |                                                                  |
   24000 ++                                                                 |
   22000 ++                                                                 |
         |                         ....*......                              |
   20000 *+.....*.......*......*...           *.......*......*.......*......*
         |                                                                  |
   18000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                20350.70       +48.1%     30148.50  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                20350.70       +48.1%     30148.50  TOTAL lock_stat.&pipe->mutex/1.acq-bounces
\f
                          lock_stat.&pipe->wait.acq-bounces

   32000 ++-----------------------------------------------------------------+
         |              O      O                                            O
   30000 O+     O                                                           |
         |                                    O                      O      |
   28000 ++                                           O      O              |
         |                             O                                    |
   26000 ++                                                                 |
         |                                                                  |
   24000 ++                                                                 |
         |                                                                  |
   22000 ++                                                                 |
         |                           ..*.....                               |
   20000 *+.....*.......*......  ....        .    ....*......*.......*......*
         |                     *.             *...                          |
   18000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                19903.70       +48.3%     29522.80  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                19903.70       +48.3%     29522.80  TOTAL lock_stat.&pipe->wait.acq-bounces
\f
                  lock_stat.&type->i_mutex_dir_key#2.holdtime-total

   35000 ++-----------------------------------------------------------------+
         |                                                   O              |
   30000 ++     *.             O                                            |
         |     +  ..                                                        |
   25000 ++   +     .                                                       |
         |   +       .                                               O      |
   20000 O+ +   O     ..O              O      O      .*                     O
         | +                                       ..  :                    |
   15000 ++             *...                     ..     :                   |
         *                  ..                 ..        :           *......*
   10000 ++                   .               *           :        ..       |
         |                     *...         ..            :      ..         |
    5000 ++                        ...    ..               :    .           |
         |                            . ..                  : ..            |
       0 ++----------------------------*---------------------*--------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                12025.44       +88.2%     22631.20  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                12025.44       +88.2%     22631.20  TOTAL lock_stat.&type->i_mutex_dir_key#2.holdtime-total
\f
                   lock_stat.&type->i_mutex_dir_key#2.holdtime-min

   35000 ++-----------------------------------------------------------------+
         |                                                   O              |
   30000 ++     *.             O                                            |
         |     +  ..                                                        |
   25000 ++   +     .                                                       |
         |   +       .                                               O      |
   20000 O+ +   O     ..O              O      O      .*                     O
         | +                                       ..  :                    |
   15000 ++             *...                     ..     :                   |
         *                  ..                 ..        :           *......*
   10000 ++                   .               *           :        ..       |
         |                     *...         ..            :      ..         |
    5000 ++                        ...    ..               :    .           |
         |                            . ..                  : ..            |
       0 ++----------------------------*---------------------*--------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                12025.44       +88.2%     22631.20  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                12025.44       +88.2%     22631.20  TOTAL lock_stat.&type->i_mutex_dir_key#2.holdtime-min
\f
                   lock_stat.&type->i_mutex_dir_key#2.holdtime-max

   35000 ++-----------------------------------------------------------------+
         |                                                   O              |
   30000 ++     *.             O                                            |
         |     +  ..                                                        |
   25000 ++   +     .                                                       |
         |   +       .                                               O      |
   20000 O+ +   O     ..O              O      O      .*                     O
         | +                                       ..  :                    |
   15000 ++             *...                     ..     :                   |
         *                  ..                 ..        :           *......*
   10000 ++                   .               *           :        ..       |
         |                     *...         ..            :      ..         |
    5000 ++                        ...    ..               :    .           |
         |                            . ..                  : ..            |
       0 ++----------------------------*---------------------*--------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                12025.44       +88.2%     22631.20  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                12025.44       +88.2%     22631.20  TOTAL lock_stat.&type->i_mutex_dir_key#2.holdtime-max
\f
                    lock_stat.&mapping->i_mmap_mutex.waittime-max

   70000 *+-----------------------------------------------------------------+
         |+                                          .*..                   |
   60000 +++                                       ..    ..                 |
         |  +                 .*.......          ..        ..               |
         |   +              ..         *...... ..                           |
   50000 ++   +           ..                  *              *.......O      |
         |     +    ....*.                                           *..... |
   40000 ++     *...    O                     O                            .|
         O      O                      O              O                     *
   30000 ++                    O                                            |
         |                                                                  |
         |                                                                  |
   20000 ++                                                                 O
         |                                                                  |
   10000 ++-----------------------------------------------------------------+

\f
                     lock_stat.&mapping->i_mmap_mutex.waittime-max

   120000 ++----------------------------------------------------------------+
   110000 ++                                 ..*                            |
          |                              ....   :                           |
   100000 ++                           *.        :                          |
    90000 ++                          .          :           *.             |
          |                          .            :         .  .            |
    80000 ++                       ..              :       .    .           |
    70000 ++                      .                 :    ..      ..         |
    60000 ++                     .                  :   .          .       .*
          |                     *                    : .            .    .. |
    50000 ++                  ..                      *                ..   |
    40000 *+.....            .                                       *.     |
          |      *.....     .                         O      O              |
    30000 O+           ..O..    O      O       O                     O      O
    20000 ++-------------*--------------------------------------------------+

\f
                     lock_stat.&mapping->i_mmap_mutex.waittime-max

   480000 ++----------------------------------------------------------------+
          |                                                                 |
   460000 ++             O             O                                    |
   440000 ++                    O              O             O              O
          |                                                                 |
   420000 ++                                          O              O      |
          O      O                                                          |
   400000 ++                                                                |
          |                                                                 |
   380000 ++                                                                |
   360000 ++                                                     ....*......|
          |     .*.......*......*......    ....*.....       .*...           *
   340000 ++....                       *...          .  ....                |
          *.                                          *.                    |
   320000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                50154.16       -25.3%     37448.79  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                60870.33       -53.5%     28299.40  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               348929.03       +25.5%    437866.22  nhm-white/micro/aim7/misc_rtns_1
               459953.53        +9.5%    503614.41  TOTAL lock_stat.&mapping->i_mmap_mutex.waittime-max
\f
                       lock_stat.sb_writers#4-R.holdtime-total

   22000 ++-----------------------------------------------------------------+
         |                                    O                             |
   20000 ++                                                                 O
         |                                                           O      |
   18000 ++                                                                 |
         |                                                                  |
   16000 O+     O                                                           |
         |                             O                     O              |
   14000 ++             O      O                      O                     |
         |                                        ....*......               |
   12000 ++  ...*.......*.....       ..*......*...           *.......    ...*
         *...                 .  ....                                *...   |
   10000 ++                    *.                                           |
         |                                                                  |
    8000 ++-----------------------------------------------------------------+

\f
                       lock_stat.sb_writers#4-R.holdtime-total

   24000 ++-----------------------------------------------------------------+
         |                                                                  O
   22000 ++                                                                 |
   20000 ++             O                                                   |
         O      O                                                           |
   18000 ++                                   O                             |
         |                     O                             O              |
   16000 ++                                                          O      |
         |                                            O                     |
   14000 ++                                                                 |
   12000 ++                            O                            .*..... |
         |          ....*......                                  ...       .|
   10000 *+.....*...           *.......*......    ....*......  ..           *
         |                                    *...           *.             |
    8000 ++-----------------------------------------------------------------+

\f
                       lock_stat.sb_writers#4-R.holdtime-total

   17500 ++-----------------------------------------------------------------+
         |                             O                             O      |
   17000 ++                                                                 |
   16500 ++     O              O                                            |
         |                                                   O              O
   16000 ++                                                                 |
         O              O                                                   |
   15500 ++                                   O                             |
         |                           ..*.....       ..*.....                |
   15000 ++                      ....        .  ....  O     .               |
   14500 *+.....    ....*......*.             *.             *.....        .*
         |      *...                                               ..  .... |
   14000 ++                                                          *.     |
         |                                                                  |
   13500 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                11701.64       +41.6%     16570.58  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                10397.73       +73.2%     18011.17  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                14633.07       +11.0%     16242.98  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                36732.44       +38.4%     50824.74  TOTAL lock_stat.sb_writers#4-R.holdtime-total
\f
                      lock_stat.clock-AF_INET-R.holdtime-total

   28000 ++-----------------------------------------------------------------+
   26000 O+                                                                 |
         |                                    O                             |
   24000 ++                                                                 |
   22000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 O
   18000 ++                                                  O       O      |
   16000 ++     O       O      O       O              O                     |
         |                                                                  |
   14000 ++                                                                 |
   12000 ++  ...*.......*......    ....*......*.......                   ...*
         *...                  *...                   *......    ....*...   |
   10000 ++                                                  *...           |
    8000 ++-----------------------------------------------------------------+

\f
                      lock_stat.clock-AF_INET-R.holdtime-total

   20000 ++-----------------------------------------------------------------+
         |                                    O                             |
   18000 O+     O       O      O                             O              |
         |                             O                                    |
         |                                                           O      O
   16000 ++                                           O                     |
         |                                                                  |
   14000 ++                                                                 |
         *......                                                            |
   12000 ++     *.....                                                      |
         |            ..                   ...*.....             ....*...   |
         |              *......*.......*...         ..    ...*...        .. |
   10000 ++                                           *...                 .|
         |                                                                  *
    8000 ++-----------------------------------------------------------------+

\f
                       lock_stat.clock-AF_INET-R.holdtime-total

   430000 ++----------------------------------------------------------------+
          |                                               ...*.......*..    |
   420000 ++           ..*.....              ..*......*...              ..  |
          |        ....        .         ....                             ..|
          *......*.             *......*.                                   |
   410000 ++                                                                *
          |                                                                 |
   400000 ++                                                                |
          |                                                                 |
   390000 ++                                   O                            |
          |                                                                 |
          |                     O      O                     O              |
   380000 ++     O       O                            O                     O
          O                                                          O      |
   370000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                11180.86       +68.4%     18824.74  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                10973.31       +62.5%     17836.52  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
               417720.26        -9.1%    379872.85  lkp-a04/micro/netperf/120s-200%-TCP_CRR
               439874.42        -5.3%    416534.11  TOTAL lock_stat.clock-AF_INET-R.holdtime-total
\f
                        lock_stat.&p->pi_lock.waittime-total

   11000 ++----------------------------O------------------------------------+
         |                                                                  |
   10000 ++                                                                 |
    9000 ++                                                                 |
         |      O       O                                                   |
    8000 ++                                                  O              |
         |                             *..                                  |
    7000 O+                    O     ..   .                          O      O
         |                          .      .                .*.......       |
    6000 ++                        .        ..O       O ....         *...   |
    5000 *+....                  ..               ....*.                 .. |
         |     .    ....*...... .             *...                         .|
    4000 ++     *...           *                                            *
         |                                                                  |
    3000 ++-----------------------------------------------------------------+

\f
                        lock_stat.&p->pi_lock.waittime-total

   56000 ++----------------------------------------------------------O------+
         |                                                                  |
   54000 ++                                                                 |
   52000 ++                                                                 |
         |                                                                  |
   50000 ++             O                     O                             |
         |                     O                                            |
   48000 ++                            O                                    |
         |      O                                     O                     |
   46000 ++                                                                 O
   44000 ++                                                                 |
         |                                                                  |
   42000 ++                 ...*.......                     .*.....         |
         *......    ....*...           *......*.......  ....       ..       |
   40000 ++-----*-------------------------------------*--------------*------*

\f
                        lock_stat.&p->pi_lock.waittime-total

   80000 ++-----------------------------------------------------------------+
         |                                                                  |
   75000 ++                                   O                             |
   70000 ++                            O                                    |
         O              O                                    O              |
   65000 ++     O              O                      O                     O
         |                                                           O      |
   60000 ++                                                                 |
         |                                                                  |
   55000 ++                                                                 |
   50000 ++                 ...*.......                                  ...*
         |     .*.......*...           *......    ....*......*.......*...   |
   45000 ++....                               *...                          |
         *.                                                                 |
   40000 ++-----------------------------------------------------------------+

\f
                        lock_stat.&p->pi_lock.waittime-total

   50000 ++----------------------------O------------------------------------+
         |                                                                  |
         |                                                   O              |
   45000 ++                                                                 |
         |      O                                                           O
         |                                    O                             |
   40000 ++             O                                                   |
         |     .*..            O                      O              O      |
   35000 ++  ..    ..                                                       |
         | ..        ..       .*.......                   ...*.......*..... |
         *.            .  ....         *.....       ..*...                 .|
   30000 ++             *.                   .  ....                        *
         |                                    *.                            |
         |                                                                  |
   25000 ++-----------------------------------------------------------------+

\f
                        lock_stat.&p->pi_lock.waittime-total

   52000 ++-----------------------------------------------------------------+
   50000 ++             O                                                   |
         |                                                                  |
   48000 ++                                                          O      |
   46000 O+     O                      O                                    |
         |                                                                  |
   44000 ++                                           O                     |
   42000 ++                    O              O                             |
   40000 ++                                                  O              |
         |                                                                  |
   38000 ++                                                                 O
   36000 ++                                                 .*.....         |
         |            ..*......              .*.......  ....       ..       |
   34000 *+.....  ....         *.......  ....         *.             *......*
   32000 ++-----*----------------------*------------------------------------+

\f
                         lock_stat.&p->pi_lock.waittime-total

   120000 ++----------------------------------------------------------------+
          |                                                                 |
   110000 ++     O                                           O              |
   100000 ++                                                                |
          O                                                          O      |
    90000 ++                                   O                            |
          |              O      O      O                                    |
    80000 ++                                                                O
          |                                                                 |
    70000 ++                                                                |
    60000 ++                                          O                     |
          |                                                                 |
    50000 ++         ....*......                   ...*......               |
          *......*...           *......*.......*...          *.......*......*
    40000 ++----------------------------------------------------------------+

\f
                        lock_stat.&p->pi_lock.waittime-total

   44000 ++----------------------------O------------------------------------+
   42000 ++                                                          O      |
         |                                                                  |
   40000 ++                                                                 |
   38000 ++                                                                 |
         O      O                                                           |
   36000 ++                                   O       O      O              O
   34000 ++             *...                          *..                   |
   32000 ++           ..    ..                       .   ..                 |
         |          ..        .    ....*..          .      ..               |
   30000 ++        .           *...       .       ..                     ...*
   28000 ++      ..                        .     .           *.......*...   |
         |     .*                           ..  .                           |
   26000 ++....                                .                            |
   24000 *+-----------------------------------*-----------------------------+

\f
                        lock_stat.&p->pi_lock.waittime-total

   80000 ++-----------------------------------------------------------------+
         |                                                                  O
   75000 ++                                                                 |
   70000 O+                    O              O                             |
         |                             O                                    |
   65000 ++     O                                     O      O       O      |
   60000 ++                                                                 |
         |              O                                                   |
   55000 ++                                                                 |
   50000 ++                                                                 |
         |                                                                  |
   45000 *+.....*.......                          ....*......*.......       |
   40000 ++             *......            ...*...                   *......|
         |                     *.......*...                                 *
   35000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 5242.08       +46.0%      7652.72  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                41147.79       +15.5%     47517.01  lkp-a04/micro/netperf/120s-200%-TCP_CRR
                47979.02       +41.3%     67803.02  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                32307.00       +26.1%     40736.78  lkp-a04/micro/netperf/120s-200%-TCP_RR
                33806.18       +31.0%     44274.76  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                46768.49       +93.3%     90390.11  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                29230.31       +25.4%     36656.86  lkp-a04/micro/netperf/120s-200%-UDP_RR
                41315.62       +61.3%     66636.42  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               277796.51       +44.6%    401667.68  TOTAL lock_stat.&p->pi_lock.waittime-total
\f
                    lock_stat.&(&list->lock)->rlock#3.acquisitions

   2.4e+06 ++---------------------------------------------------------------+
           *......*......    ....*......*......*......    ....*......    ...*
   2.3e+06 ++            *...                         *...           *...   |
   2.2e+06 ++                                                               |
           |                                                                |
   2.1e+06 ++                                                               |
     2e+06 ++                                         O                     |
           |                                                                |
   1.9e+06 ++                                                               |
   1.8e+06 ++                                                               |
           |                     O             O                            O
   1.7e+06 ++                                                               |
   1.6e+06 O+            O              O                     O      O      |
           |                                                                |
   1.5e+06 ++-----O---------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              2313283.70       -26.8%   1693267.70  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              2313283.70       -26.8%   1693267.70  TOTAL lock_stat.&(&list->lock)->rlock#3.acquisitions
\f
                         lock_stat.&rq->lock/1.acquisitions

   80000 ++-----------------------------------------------------------------+
         |                                                                  |
   75000 *+.....*.......    ...*.......                                    .*
         |              *...           *......*.......*......*.......  .... |
         |                                                           *.     |
   70000 ++                                                                 |
         |                                                                  |
   65000 ++                                                                 |
         |                                                                  |
   60000 ++                                                                 |
         |                                                                  |
         |                                                                  |
   55000 ++     O              O                      O                     |
         O              O              O      O              O       O      O
   50000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                74032.10       -28.6%     52864.10  nhm-white/micro/aim7/misc_rtns_1
                74032.10       -28.6%     52864.10  TOTAL lock_stat.&rq->lock/1.acquisitions
\f
                       lock_stat.&(&zone->lock)->rlock.acq-bounces

    1.4e+06 ++--------------------------------------------------------------+
   1.35e+06 ++     O                                                        |
            |             O                                          O      |
    1.3e+06 ++                          O                     O             |
   1.25e+06 O+                                                              |
            |                    O              O                           O
    1.2e+06 ++                                                              |
   1.15e+06 ++                                                              |
    1.1e+06 ++                                                              |
            |                                          O                    |
   1.05e+06 ++                                                              |
      1e+06 ++                                                              |
            |                                       ...*.....               |
     950000 *+.....*......*......*......*.......*...         .    ...*......*
     900000 ++------------------------------------------------*-------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               931324.30       +34.7%   1254788.40  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               931324.30       +34.7%   1254788.40  TOTAL lock_stat.&(&zone->lock)->rlock.acq-bounces
\f
                    lock_stat.hrtimer_bases.lock#7.holdtime-total

   26000 ++-----------------------------------------------------------------+
         |                                                                  |
   24000 ++                                                                 O
         |                                    *                             |
   22000 O+                                  : :                            |
         |                     O             :  :                           |
   20000 ++                                 :   :                           |
         |              O                  :     :           O              |
   18000 ++                                :      :                         |
         |                                :        :                        |
   16000 ++                              :          :                       |
         |      O                      O:     O     : O              O      |
   14000 ++         ....*......         :            :    ...*.......       |
         *......*...           *.......*              *...           *......*
   12000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                14328.30       +25.0%     17906.30  nhm-white/micro/aim7/misc_rtns_1
                14328.30       +25.0%     17906.30  TOTAL lock_stat.hrtimer_bases.lock#7.holdtime-total
\f
                          lock_stat.&rq->lock/1.acq-bounces

   40000 ++-----------------------------------------------------------------+
         |   ...*.....                                                      |
   38000 *+..         ..    ...*.......           ....*......*...          .*
         |              *...           *......*...               ...     .. |
   36000 ++                                                         .  ..   |
         |                                                           *.     |
   34000 ++                                                                 |
         |                                                                  |
   32000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
         |      O                                     O              O      |
   28000 ++             O                                    O              |
         O                     O       O                                    O
   26000 ++-----------------------------------O-----------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                37493.70       -26.3%     27624.80  nhm-white/micro/aim7/misc_rtns_1
                37493.70       -26.3%     27624.80  TOTAL lock_stat.&rq->lock/1.acq-bounces
\f
                       lock_stat.rcu_read_lock-R.holdtime-total

   2.1e+07 O+-----O---------------------------------------------------------+
           |             O       O      O      O      O       O      O      |
     2e+07 ++                                                               O
           |                                                                |
   1.9e+07 ++                                                               |
           |                                                                |
   1.8e+07 ++                                                               |
           |                                                                |
   1.7e+07 ++                                                               |
           |                                                                |
   1.6e+07 ++                                                               |
           *......    ...*.......                                           |
   1.5e+07 ++     *...           *......*......*......*.......*......*......*
           |                                                                |
   1.4e+07 ++---------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
             15085692.76       +36.5%  20597805.11  nhm-white/micro/aim7/misc_rtns_1
             15085692.76       +36.5%  20597805.11  TOTAL lock_stat.rcu_read_lock-R.holdtime-total
\f
                 lock_stat.zone->wait_table+i.contentions.__wake_up

   46000 ++-----------------------------------------------------------------+
         O              O      O                                            |
   44000 ++     O                                                    O      |
   42000 ++                                           O      O              |
         |                             O      O                             |
   40000 ++                                                                 |
   38000 ++                                                                 O
         |                                                                  |
   36000 ++                                                                 |
   34000 ++                                                                 |
         |                                                                  |
   32000 *+.....            ...*.......*......*.......*......*.......*..... |
   30000 ++     *.......*...                                               .|
         |                                                                  *
   28000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                31239.40       +36.1%     42514.40  nhm-white/micro/aim7/misc_rtns_1
                31239.40       +36.1%     42514.40  TOTAL lock_stat.zone->wait_table+i.contentions.__wake_up
\f
                         lock_stat.rcu_node_0.waittime-total

   40000 ++-----------------------------------------------------------------+
         |                                    O                             |
   38000 ++                            O                                    |
   36000 O+     O                                            O       O      |
         |                     O                                            |
   34000 ++             O                             O                     O
   32000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
   28000 ++                                                                 |
         |                                                                  |
   26000 ++                 ...*.....             ....*......               |
   24000 ++  ...*.......*...         ..    ...*...           *.......*......*
         *...                          *...                                 |
   22000 ++-----------------------------------------------------------------+

\f
                         lock_stat.rcu_node_0.waittime-total

   32000 ++-----------------------------------------------------------------+
         |                                                                  O
   30000 ++                                   O                             |
         |      O                                                           |
         |                     O       O                     O              |
   28000 ++                                                                 |
         |      *..                                          *.....         |
   26000 ++   ..   ..                                      ..      ..       |
         O   .       ..                                  ..          *..... |
   24000 ++ .          .O   ...*.......               O..            O     .|
         |..            *...           *...          .*                     *
         *                                 ..     ...                       |
   22000 ++                                  .  ..                          |
         |                                    *.                            |
   20000 ++-----------------------------------------------------------------+

\f
                         lock_stat.rcu_node_0.waittime-total

   42000 ++-------------O---------------------------------------------------+
         |                                                                  |
   40000 O+     O                                                    O      |
   38000 ++                            O                                    |
         |                                                   O              |
   36000 ++                                   O                             |
   34000 ++                                           O                     |
         |                                                                  O
   32000 ++                    O                                            |
   30000 ++                                                                 |
         |                                                                  |
   28000 ++                                  .*.......    ...*.....         |
   26000 *+....       ..*......*.......  ....         *...         ..    ...*
         |     .  ....                 *.                            *...   |
   24000 ++-----*-----------------------------------------------------------+

\f
                         lock_stat.rcu_node_0.waittime-total

   45000 ++----------------------------------------------------------O------+
         |                                                                  |
         O      O                                                           |
   40000 ++                                                  O              |
         |                     O              O                             |
         |              O                                                   |
   35000 ++                            O              O                     O
         |                                                                  |
   30000 ++                                                                 |
         |                                                                  |
         |            ..*......                                             |
   25000 ++       ....         *.......*......    ....*......*.......*......*
         *......*.                            *...                          |
         |                                                                  |
   20000 ++-----------------------------------------------------------------+

\f
                         lock_stat.rcu_node_0.waittime-total

   32000 ++-----------------------------------------------------------------+
         O                                                                  |
   30000 ++                                                                 |
         |      O                                                           |
         |                             O                                    O
   28000 ++                                   O                      O      |
         |                                            *..                   |
   26000 ++            .*...                         +   .                  |
         |          ... O   ..                      +     .                .*
   24000 ++       ..          .    ....*..         +  O    ..O           .. |
         |     .*.             *...       .       +                    ..   |
         | ....                            .     +           *.......*.     |
   22000 *+                    O            ..  +                           |
         |                                     +                            |
   20000 ++-----------------------------------*-----------------------------+

\f
                         lock_stat.rcu_node_0.waittime-total

   50000 ++-----------------------------------------------------------------+
   48000 ++                                                                 O
         |                                                                  |
   46000 ++                                                                 |
   44000 ++     O                             O                             |
         |                                                                  |
   42000 O+                            O                                    |
   40000 ++                    O                      O      O              |
   38000 ++                                                                 |
         |              O                                            O      |
   36000 *+....                                                             |
   34000 ++    .                                  ....*......*.......       |
         |      *.......                     .*...                   *......*
   32000 ++             *......*.......  ....                               |
   30000 ++----------------------------*------------------------------------+

\f
                         lock_stat.rcu_node_0.waittime-total

   5500 ++------------------------------------------------------------------+
        |                                                                   |
   5000 O+                                                          O       |
        |              O                                     O              |
        |       O                                    O                      |
   4500 ++                                                                  |
        |                                                                   |
   4000 ++                                    O                             O
        |                      O      O                                     |
   3500 *+......                                         ....*...          .*
        |       *......*.......*......            ...*...        ..     ... |
        |                             *.......*...                 .  ..    |
   3000 ++                                                          *.      |
        |                                                                   |
   2500 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                24474.10       +46.6%     35878.51  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                24013.98       +13.8%     27335.71  lkp-a04/micro/netperf/120s-200%-TCP_RR
                26426.20       +39.5%     36853.27  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                24714.99       +54.8%     38251.24  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                23507.03       +14.3%     26877.02  lkp-a04/micro/netperf/120s-200%-UDP_RR
                32978.47       +25.5%     41377.68  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                 3332.61       +33.4%      4446.51  nhm-white/micro/aim7/misc_rtns_1
               159447.37       +32.3%    211019.95  TOTAL lock_stat.rcu_node_0.waittime-total
\f
                        lock_stat.&pipe->mutex/1.waittime-max

   90000 ++-------------*---------------------------------------------------+
         |             :   ..                                               |
   80000 ++           :      ..                                             |
         |            :                                                     |
   70000 ++          :         *.                                           |
         |          :            ..                                         |
   60000 ++        :               .                                        |
         |        :                 .                               .*..... |
   50000 ++       :                  ..      .*.......*...        ..       .|
         |       :                       ....             ..    ..          *
   40000 ++  ...*                      *.                   . ..            |
         *...   O       O              O                     *              O
   30000 O+                    O              O                      O      |
         |                                            O                     |
   20000 ++-----------------------------------------------------------------+

\f
                        lock_stat.&pipe->mutex/1.waittime-max

   80000 ++-----------------------------------------------------------------+
         |                                                                  |
   70000 ++                                                          *.     |
         |                                                          +  .    |
         |                                   .*..                  +    ..  |
   60000 *+.                             ....    ..               +       . |
         |  ..                        .*.          ..            +         .|
   50000 ++   ..                    ..               .          +           |
         |                        ..                  *.....   +            *
   40000 ++     *.......*...... ..                    O     . +             |
         O      O       O      *              O              *       O      |
         |                     O       O                                    |
   30000 ++                                                  O              |
         |                                                                  |
   20000 ++-----------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                50724.81       -36.1%     32431.17  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                49512.17       -31.7%     33815.93  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               100236.97       -33.9%     66247.10  TOTAL lock_stat.&pipe->mutex/1.waittime-max
\f
                         lock_stat.slock-AF_INET/1.acquisitions

    1.2e+06 ++--------------------------------------------------------------+
            *......*......*......*......*.......*......    ...*......*......*
   1.15e+06 ++                                         *...                 |
            |                                                               |
    1.1e+06 ++                                                              |
   1.05e+06 ++                                                              |
            |                                          O                    |
      1e+06 ++                                                              |
            |                                                               |
     950000 ++                                                              |
     900000 ++                   O              O                           O
            |             O                                   O             |
     850000 O+                          O                            O      |
            |      O                                                        |
     800000 ++--------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              1166309.60       -23.3%    894892.70  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              1166309.60       -23.3%    894892.70  TOTAL lock_stat.slock-AF_INET/1.acquisitions
\f
                      lock_stat.sk_lock-AF_INET6.holdtime-total

   6000 ++------------------------------------------------------------------+
   5800 ++      O                                                           |
        |                                                           O       |
   5600 ++                                                   O              |
   5400 ++             O                                                    |
   5200 ++                            O                                     |
   5000 O+                                                                  O
        |                      O              O                             |
   4800 ++                                           O                      |
   4600 ++                                                                  |
   4400 ++     .*.....                                                      |
   4200 ++  ...       .                                    ..*...           |
        | ..           *.......                        ....      ..         |
   4000 *+                     *......*.......*......*.            .    ....*
   3800 ++----------------------------------------------------------*-------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 4053.60       +29.4%      5245.03  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                 4053.60       +29.4%      5245.03  TOTAL lock_stat.sk_lock-AF_INET6.holdtime-total
\f
                lock_stat.&(&tsk->delays->lock)->rlock.holdtime-total

   5600 ++------------------------------------------------------------------+
   5400 ++                                        ...*..                    |
        *..                                 ..*...      ..                  |
   5200 ++ ..                ..*......  ....              ..       .*.......|
   5000 ++   ..          ....         *.                    .  ....         *
   4800 ++     .    ...*.                                    *.             |
   4600 ++      *...                                                        |
        |                                                                   |
   4400 ++                                                                  |
   4200 ++                                                                  |
   4000 ++      O      O       O                             O              |
   3800 O+                                    O                             |
        |                                            O              O       |
   3600 ++                            O                                     O
   3400 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 5036.72       -23.9%      3834.90  nhm-white/micro/aim7/misc_rtns_1
                 5036.72       -23.9%      3834.90  TOTAL lock_stat.&(&tsk->delays->lock)->rlock.holdtime-total
\f
                     lock_stat.slock-AF_INET-RPC.holdtime-total

   35000 ++-----------------------------------------------------------------+
         |                                                                  |
   30000 ++                                                          O      |
         |                                                                  |
         |                                                                  |
   25000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                  O              |
         O      O                  ....*..... O                             |
         *......        O   ...*...          .        O                     O
   10000 ++     *.......*...                  *.......*......*.......*......*
         |                                                                  |
    5000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                10019.40       +45.8%     14613.28  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                10019.40       +45.8%     14613.28  TOTAL lock_stat.slock-AF_INET-RPC.holdtime-total
\f
                      lock_stat.&(&zone->lock)->rlock.acquisitions

   1.85e+06 ++-----O--------------------------------------------------------+
    1.8e+06 ++            O             O                            O      |
            O                                                 O             |
   1.75e+06 ++                                  O                           |
    1.7e+06 ++                   O                                          O
            |                                                               |
   1.65e+06 ++                                                              |
    1.6e+06 ++                                                              |
   1.55e+06 ++                                         O                    |
            |                                                               |
    1.5e+06 ++                                                              |
   1.45e+06 ++                                        .*.....               |
            *......*......*......*......*.......  ....       .           ...*
    1.4e+06 ++                                  *.            *......*...   |
   1.35e+06 ++--------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              1417553.50       +23.9%   1755833.90  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              1417553.50       +23.9%   1755833.90  TOTAL lock_stat.&(&zone->lock)->rlock.acquisitions
\f
                   lock_stat.&(&list->lock)->rlock#3.holdtime-total

   7.2e+06 ++---------------------------------------------------------------+
     7e+06 ++  ...*......                   ...*......    ....*......*......|
           *...          *.......*......*...          *...                  *
   6.8e+06 ++                                                               |
   6.6e+06 ++                                                               |
   6.4e+06 ++                                         O                     |
   6.2e+06 ++                                                               |
           |                                                                |
     6e+06 ++                                                               |
   5.8e+06 ++                    O             O                            O
   5.6e+06 ++                                                               |
   5.4e+06 O+            O              O                     O      O      |
           |                                                                |
   5.2e+06 ++     O                                                         |
     5e+06 ++---------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              6946024.55       -19.3%   5602189.45  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              6946024.55       -19.3%   5602189.45  TOTAL lock_stat.&(&list->lock)->rlock#3.holdtime-total
\f
                          lock_stat.&q->lock.holdtime-total

   6500 ++------------------------------------------------------------------+
        |                                 ....*.                            |
   6000 ++            .*..            *...      .                           |
        |         ....    ..        ..           ..                         |
   5500 ++     .*.          ..    ..               .                *...    |
        |   ...               . ..                  .              :    ... |
   5000 ++..                   *              O              O    :        .|
        *.                                           *..         :          *
   4500 O+                                              .        :          O
        |                                                ..     :   O       |
   4000 ++      O      O                                   ..  :            |
        |                      O                              :             |
   3500 ++                                           O       *              |
        |                             O                                     |
   3000 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 5172.44       -19.2%      4177.08  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                 5172.44       -19.2%      4177.08  TOTAL lock_stat.&q->lock.holdtime-total
\f
                      lock_stat.(&(work)->timer).holdtime-total

   24000 ++-----O-----------------------------------------------------------+
         |                                                           O      |
   23000 ++             O                                    O              |
         O                             O                                    |
         |                     O              O                             O
   22000 ++                                                                 |
         |                                                                  |
   21000 ++                                                                 |
         |                                            O                     |
   20000 ++                                                                 |
         |                                                                  |
         |                                                                  |
   19000 ++         ....*.....                    ....*......               |
         *......*...          .    ....*......*...           *.......*......*
   18000 ++--------------------*--------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                18369.95       +23.1%     22618.18  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                18369.95       +23.1%     22618.18  TOTAL lock_stat.(&(work)->timer).holdtime-total
\f
                      lock_stat.&(&nf->lru_lock)->rlock.acq-bounces

    1.1e+06 ++------------------------------------------------O-------------+
            |                                                               |
   1.05e+06 ++                                                              |
            O      O      O             O              O                    |
            |                    O                                          |
      1e+06 ++                                  O                    O      |
            |                                                               O
     950000 ++                                                              |
            |                                                               |
     900000 ++                                                              |
            |                                                               |
            |                                                               |
     850000 *+.....    ...*......*......*.......*......*......*......    ...*
            |      *...                                              *...   |
     800000 ++--------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               834643.80       +22.7%   1024368.10  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               834643.80       +22.7%   1024368.10  TOTAL lock_stat.&(&nf->lru_lock)->rlock.acq-bounces
\f
                   lock_stat.&(&pool->lock)->rlock.holdtime-total

   33000 ++-----O-----------------------------------------------------------+
         |                                                                  |
   32000 ++                            O                     O       O      |
   31000 O+             O                                                   |
         |                     O              O                             O
   30000 ++                                                                 |
   29000 ++                                                                 |
         |                                                                  |
   28000 ++                                           O                     |
   27000 ++                                                                 |
         |                                                                  |
   26000 ++                                       ....*......               |
   25000 *+.....*.......*......    ....*......*...           *.......*......*
         |                     *...                                         |
   24000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                25251.78       +22.4%     30899.04  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                25251.78       +22.4%     30899.04  TOTAL lock_stat.&(&pool->lock)->rlock.holdtime-total
\f
                          lock_stat.((&timer)).holdtime-total

    1e+06 ++----------------------------------------------------------------+
          |                                                                 |
   950000 ++                                   O                            |
          O              O                                                  |
          |                            O                                    |
   900000 ++                    O                                           O
          |      O                                           O              |
   850000 ++                                          O                     |
          |                                                          O      |
   800000 ++                                                                |
          |                  ...*...                                        |
          |     .*.......*...       ..                    ...*.......    ...*
   750000 ++  ..                      .            ...*...           *...   |
          | ..                         *.......*...                         |
   700000 *+----------------------------------------------------------------+

\f
                          lock_stat.((&timer)).holdtime-total

   700000 ++----------------------------------------------------------------+
   680000 ++                           O       O                            |
          |      O                                                          O
   660000 ++             O                                                  |
   640000 ++                                          O                     |
   620000 ++     *.                                          *.....  O      |
   600000 ++   ..  ..                                      ..      ..       |
          |   .      .          *.....                    .          *..    |
   580000 O+ .        .       ..      .                  .              ..  |
   560000 ++.          ..   ..         *..             ..                 ..|
   540000 *+              ..              ..         .*                     |
   520000 ++             *                  ..     ..                       *
          |                                   .  ..                         |
   500000 ++                                   *.                           |
   480000 ++----------------------------------------------------------------+

\f
                          lock_stat.((&timer)).holdtime-total

   800000 ++----------------------------------------------------------------+
          O                                                                 |
          |              O                                                  |
   750000 ++                           O       O                     O      |
          |                                                  O              |
          |                                                                 |
   700000 ++     O                                                          |
          |                                           O                     |
   650000 ++                                                                O
          |                                                 .*.....         |
          *...                 .*.....       ..*......  ....       ..    ...*
   600000 ++  ..           .... O     .  ....         *.             *...   |
          |     .    ....*.            *.                                   |
          |      *...                                                       |
   550000 ++----------------------------------------------------------------+

\f
                           lock_stat.((&timer)).holdtime-total

    1.2e+06 ++--------------------------------------------------------------+
   1.15e+06 ++     O                                                        |
            |                                   O                           |
    1.1e+06 O+                   O                            O      O      |
   1.05e+06 ++                                                              |
      1e+06 ++            O                                                 |
     950000 ++                          O                                   O
            |                                          O                    |
     900000 ++                                                              |
     850000 ++                                                              |
     800000 ++           .*.....                                            |
     750000 ++         ..       .                     .*......*......*......|
            *......  ..          *......*.......  ....                      *
     700000 ++     *.                           *.                          |
     650000 ++--------------------------------------------------------------+

\f
                          lock_stat.((&timer)).holdtime-total

   700000 ++----------------------------------------------------------------+
          |                                                          O      |
          |                            O                                    |
   650000 O+     O                                                          |
          |              O                     O      O      O              O
          |                                           *..                   |
   600000 ++            .*.....                      +   .                  |
          |          ...       .    ...*..          +     .                 |
   550000 ++       ..           *...      .        +       ..            ...*
          |     .*.                        ..     +              ....*...   |
          |   ..                             ..  +           *...           |
   500000 ++..                  O               +                           |
          *.                                   *                            |
          |                                                                 |
   450000 ++----------------------------------------------------------------+

\f
                          lock_stat.((&timer)).holdtime-total

   950000 ++----------------------------------------------------------------+
          |                            O                                    O
   900000 ++                                                                |
          |                     O              O                     O      |
   850000 O+     O                                    O                     |
          |                                                                 |
   800000 ++             O                                   O              |
          |                                                                 |
   750000 *+....                                          ...*...           |
          |     .                                    .*...       ...        |
   700000 ++     *.......                        ....               .       |
          |              *......*......    ....*.                    *......*
   650000 ++                           *...                                 |
          |                                                                 |
   600000 ++----------------------------------------------------------------+

\f
                         lock_stat.((&timer)).holdtime-total

   50000 ++-----------------------------------------------------------------+
         |                                                                  |
   48000 O+                                                                 |
         |              O                                            O      |
   46000 ++     O                             O                             O
         |                     O                      O                     |
   44000 ++                            O                     O              |
         |                                                                  |
   42000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                .*
         *......*.......*......*.......                   ...*.......  .... |
   38000 ++                            *......    ....*...           *.     |
         |                                    *...                          |
   36000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               749453.70       +19.5%    895420.45  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
               561305.58       +14.4%    642037.56  lkp-a04/micro/netperf/120s-200%-TCP_RR
               608725.40       +17.2%    713283.61  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               735187.26       +42.5%   1047854.58  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               545303.72       +15.4%    629228.91  lkp-a04/micro/netperf/120s-200%-UDP_RR
               693319.76       +24.2%    860914.03  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                38570.97       +19.1%     45922.41  nhm-white/micro/aim7/misc_rtns_1
              3931866.39       +23.0%   4834661.56  TOTAL lock_stat.((&timer)).holdtime-total
\f
              lock_stat.&(&dentry->d_lock)->rlock.contentions.path_get

   55000 *+-----------------------------------------------------------------+
         |     .                                                            |
         |      *.......                                            .*......|
   50000 ++             *......*.......*.....                    ...        *
         |                                   .    ....*......  ..    O      |
         |                     O              *...           *.             |
   45000 ++                                                                 |
         |      O                                                           |
   40000 ++                                                                 O
         |                             O                                    |
         |                                    O              O              |
   35000 O+             O                                                   |
         |                                                                  |
         |                                            O                     |
   30000 ++-----------------------------------------------------------------+

\f
               lock_stat.&(&dentry->d_lock)->rlock.contentions.path_get

   400000 ++----------------------------------------------------------------+
          |     .*.......*.....       .*.......*......*              *......*
   350000 ++....               .  ....                 :            :       |
          *.                    *.                      :    O     :        |
          |              O      O      O       O      O :         :         O
   300000 ++                                             :       :          |
          |                                               :      :          |
   250000 ++                                               :    :           |
          |                                                :   :            |
   200000 ++                                                : :             |
          |                                                  *              |
          |                                                                 |
   150000 ++     O                                                   O      |
          |                                                                 |
   100000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                49453.50       -21.0%     39054.50  lkp-a04/micro/netperf/120s-200%-TCP_CRR
               351177.20       -19.4%    283022.30  nhm-white/micro/aim7/misc_rtns_1
               400630.70       -19.6%    322076.80  TOTAL lock_stat.&(&dentry->d_lock)->rlock.contentions.path_get
\f
                    lock_stat.&mapping->i_mmap_mutex.waittime-total

   1.1e+06 ++---------------------------------------------------------------+
           |                                                                |
     1e+06 ++                                                        *..    |
           |                                                       ..   .   |
    900000 *+..                                                  ..      .  |
    800000 ++  ..                                              ..         ..|
           |     .    ...*...                                .*             |
    700000 ++     *...       ...                          ...               *
           |                    .    ...*......*......O ..    O             O
    600000 ++                    *...          O      *.                    |
    500000 ++                                                               |
           |                            O                                   |
    400000 ++     O                                                         |
           O                                                         O      |
    300000 ++---------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               729459.97       -23.7%    556449.31  lkp-a04/micro/netperf/120s-200%-TCP_CRR
               729459.97       -23.7%    556449.31  TOTAL lock_stat.&mapping->i_mmap_mutex.waittime-total
\f
                         lock_stat.&rsp->gp_wq.holdtime-total

   380000 ++----------------------------------------------------------------+
   370000 ++                                   O                            |
          |                                                                 |
   360000 O+             O                                                  |
   350000 ++     O                     O                                    |
          |                     O                                           O
   340000 ++                                                 O              |
   330000 ++                                          O                     |
   320000 ++                   .*..                                  O      |
          |                ....    ..                                      .*
   310000 ++     *.......*.          ..              .*......*.......  .... |
   300000 ++   ..                                ....                *.     |
          |  ..                        *.......*.                           |
   290000 ++.                                                               |
   280000 *+----------------------------------------------------------------+

\f
                         lock_stat.&rsp->gp_wq.holdtime-total

   330000 ++----------------------------------------------------------------+
          |                                                                 |
   320000 O+             O                                                  |
   310000 ++                                   O                     O      |
          |                            O                     O              |
   300000 ++                                                                |
          |                                                                 |
   290000 ++     O                                          .*..            |
          |                                   .*.....     ..    ..          |
   280000 ++                                ..       .O ..        ..       .*
   270000 *+..                 .*.....    ..          *.            .  .... |
          |   ..           ....       . ..                           *.     |
   260000 ++    .    ....*.            *                                    |
          |      *...           O                                           |
   250000 ++----------------------------------------------------------------+

\f
                         lock_stat.&rsp->gp_wq.holdtime-total

   440000 ++----------------------------------------------------------------+
          |      O                                                          |
   420000 ++                                                                |
   400000 O+                    O              O             O              |
          |                                                          O      |
   380000 ++                                                                |
          |              O                            O                     |
   360000 ++                                                                |
          |                            O                                    O
   340000 ++                                                                |
   320000 ++            .*.....                       *......               |
          |          ...       .                    ..       *.......*..... |
   300000 ++       ..           *......*.....     ..                       .|
          *......*.                          .. ..                          *
   280000 ++-----------------------------------*----------------------------+

\f
                         lock_stat.&rsp->gp_wq.holdtime-total

   370000 ++---------------------------O------------------------------------O
   360000 ++                                                                |
          |                                                                 |
   350000 O+     O              O              O      O                     |
   340000 ++                                                                |
          |                                                          O      |
   330000 *+.                                             ...*..            |
   320000 ++ ..          O                           .*...   O  ..          |
   310000 ++   ..                                  ..             ..        |
          |                                      ..                 .       |
   300000 ++     *.......*......              .*.                    *......|
   290000 ++                    *.....     ...                              *
          |                           .  ..                                 |
   280000 ++                           *.                                   |
   270000 ++----------------------------------------------------------------+

\f
                        lock_stat.&rsp->gp_wq.holdtime-total

   34000 ++-----------------------------------------------------------------O
         |                             O      O                             |
   32000 ++                    O                                            |
         |                                                                  |
   30000 ++     O                                     O                     |
         O                                                           O      |
   28000 ++             O                                    O              |
         |                                                                  |
   26000 ++                                                                 |
         |                                                                  |
   24000 ++                                                        ..*...   |
         |            ..*.....                                 ....      .. |
   22000 *+.....  ....        .                   ....*......*.            .|
         |      *.             *.......*......*...                          *
   20000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               305627.77       +13.8%    347755.23  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
               272162.59        +9.5%    298087.11  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               302460.24       +29.1%    390567.56  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               303210.49       +13.8%    345049.18  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                21663.86       +41.2%     30581.88  nhm-white/micro/aim7/misc_rtns_1
              1205124.95       +17.2%   1412040.97  TOTAL lock_stat.&rsp->gp_wq.holdtime-total
\f
                 lock_stat.&(&dentry->d_lock)->rlock.contentions.dput

   850000 ++----------------------------------------------------------------+
          |                                                 .*.....         |
   800000 ++     *.......                               ....       ..       |
          |    ..        *...         .*.......    ...*.             *......*
          |  ..              ..     ..         *...                         |
   750000 ++.                  .  ..                                        |
          *                     *.                                          |
   700000 ++                                                                |
          O                                                  O              |
   650000 ++                                                         O      |
          |              O      O              O      O                     |
          |                            O                                    O
   600000 ++     O                                                          |
          |                                                                 |
   550000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               778969.60       -18.0%    639110.20  nhm-white/micro/aim7/misc_rtns_1
               778969.60       -18.0%    639110.20  TOTAL lock_stat.&(&dentry->d_lock)->rlock.contentions.dput
\f
                  lock_stat.&(&dentry->d_lock)->rlock.waittime-total

   230000 ++----------------------------------------------------------------+
          |                                                                 |
   220000 ++                                                .*...           |
          |                                               ..     ...        |
   210000 ++                                            ..          .       |
          |     .*.......*...         .*.......    ...*.             *......|
   200000 ++  ..             ..     ..         *...                         *
          | ..                 .  ..                                        |
   190000 *+                    *.                                          |
          |                                                                 |
   180000 ++                                                                |
          |                                                  O       O      |
   170000 O+                    O                     O                     |
          |      O       O             O       O                            O
   160000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               203249.44       -17.2%    168359.45  nhm-white/micro/aim7/misc_rtns_1
               203249.44       -17.2%    168359.45  TOTAL lock_stat.&(&dentry->d_lock)->rlock.waittime-total
\f
                    lock_stat.&(&dentry->d_lock)->rlock.con-bounces

   850000 ++----------------------------------------------------------------+
          |                                                 .*.....         |
          |      *.......                               ....       ..       |
   800000 ++   ..        *...         .*.......    ...*.             *......*
          |  ..              ..     ..         *...                         |
          |..                  .  ..                                        |
   750000 *+                    *.                                          |
          |                                                                 |
   700000 ++                                                                |
          O                                                  O              |
          |                                                          O      |
   650000 ++             O      O              O      O                     |
          |                            O                                    O
          |      O                                                          |
   600000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               787586.90       -16.9%    654513.10  nhm-white/micro/aim7/misc_rtns_1
               787586.90       -16.9%    654513.10  TOTAL lock_stat.&(&dentry->d_lock)->rlock.con-bounces
\f
                    lock_stat.&(&dentry->d_lock)->rlock.contentions

   850000 ++----------------------------------------------------------------+
          |                                                 .*.....         |
          |      *.......                               ....       ..       |
   800000 ++   ..        *...         .*.......    ...*.             *......*
          |  ..              ..     ..         *...                         |
          |..                  .  ..                                        |
   750000 *+                    *.                                          |
          |                                                                 |
   700000 ++                                                                |
          O                                                  O              |
          |                                                          O      |
   650000 ++             O      O              O      O                     |
          |                            O                                    O
          |      O                                                          |
   600000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               787737.60       -16.9%    654670.40  nhm-white/micro/aim7/misc_rtns_1
               787737.60       -16.9%    654670.40  TOTAL lock_stat.&(&dentry->d_lock)->rlock.contentions
\f
                            lock_stat.&f->lock-R.acq-bounces

   1.25e+06 ++--------------------------------------------------------------+
            |      O                                                        |
    1.2e+06 ++                                                O             |
            O             O             O                                   |
            |                    O                     O             O      O
   1.15e+06 ++                                  O                           |
            |                                                               |
    1.1e+06 ++                                                              |
            |                                                               |
   1.05e+06 ++                                                              |
            |                                                               |
            |                                                              .*
      1e+06 ++         ...*......    ...*.......*......                .... |
            *......*...          *...                  *......*......*.     |
     950000 ++--------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               982029.40       +20.0%   1178821.80  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               982029.40       +20.0%   1178821.80  TOTAL lock_stat.&f->lock-R.acq-bounces
\f
                         lock_stat.rcu_read_lock-R.acquisitions

   1.15e+07 *+-----*----------------------------*---------------------------*
            |             *......*......*...           *......*......*...   |
            |                                                               |
    1.1e+07 ++                                                              |
            |                                                               |
            |                                                               |
   1.05e+07 ++                                         O                    |
            |                                                               |
      1e+07 ++                                                              |
            |                                                               O
            |                    O              O                           |
    9.5e+06 ++            O                                                 |
            |                           O                     O      O      |
            O      O                                                        |
      9e+06 ++--------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
             11377298.70       -15.7%   9591520.20  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
             11377298.70       -15.7%   9591520.20  TOTAL lock_stat.rcu_read_lock-R.acquisitions
\f
                    lock_stat.&rnp->nocb_gp_wq[1].holdtime-total

   13500 ++-----------------------------------------------------------------+
         |                                                                  |
   13000 ++                                   O                             |
   12500 O+             O                                                   |
         |                             O                                    |
   12000 ++                                                                 |
         |      O              O                      O      O              O
   11500 ++                                                                 |
         |                    .*...                                  O      |
   11000 ++     *.......  ....     ...                    ...*.......    ...*
   10500 ++   ..        *.            .    ...*.......*...           *...   |
         |  ..                         *...                                 |
   10000 ++.                                                                |
         *                                                                  |
    9500 ++-----------------------------------------------------------------+

\f
                    lock_stat.&rnp->nocb_gp_wq[1].holdtime-total

   14500 ++-----------------------------------------------------------------+
   14000 ++     O                                                           |
         |                     O              O              O              |
   13500 O+                                                                 |
   13000 ++                                           O              O      |
         |              O                                                   |
   12500 ++                                                                 |
   12000 ++                            O                                    O
   11500 ++                                                                 |
         |             .*.....                       .*......               |
   11000 ++         ...       .                    ..        *.......*..... |
   10500 ++       ..           *.......*.....    ..                        .|
         *......*.                           . ..                           *
   10000 ++                                   *                             |
    9500 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                10655.34       +13.4%     12085.54  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                10577.54       +24.7%     13189.10  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                21232.88       +19.0%     25274.64  TOTAL lock_stat.&rnp->nocb_gp_wq[1].holdtime-total
\f
                    lock_stat.&rnp->nocb_gp_wq[0].holdtime-total

   14000 ++-----------------------------------------------------------------+
         |                                                                  |
   13500 ++                                   O                             |
   13000 O+             O                                                   |
         |                             O                                    |
   12500 ++                                                                 O
         |      O              O                             O              |
   12000 ++                                           O                     |
         |                    .*...                                         |
   11500 ++               ....     ...                    ...*.......O   ...*
   11000 ++    .*.......*.            .           ....*...           *...   |
         |   ..                        *......*...                          |
   10500 ++..                                                               |
         *.                                                                 |
   10000 ++-----------------------------------------------------------------+

\f
                    lock_stat.&rnp->nocb_gp_wq[0].holdtime-total

   15000 ++-----------------------------------------------------------------+
   14500 ++     O                                                           |
         |                     O              O              O              |
   14000 ++                                           O                     |
   13500 O+                                                          O      |
         |              O                                                   |
   13000 ++                                                                 |
   12500 ++                            O                                    O
   12000 ++                                                                 |
         |             .*...                                                |
   11500 ++          ..     ..                       .*......*.......       |
   11000 ++        ..         .                   ...                *......|
         *...... ..            *.......*......  ..                          *
   10500 ++     *                             *.                            |
   10000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                11089.10       +13.2%     12552.03  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                10961.14       +24.7%     13665.04  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                22050.24       +18.9%     26217.07  TOTAL lock_stat.&rnp->nocb_gp_wq[0].holdtime-total
\f
                       lock_stat.clockevents_lock.waittime-total

   215000 ++----------------------------------------------------------------O
          |                                                                 |
   210000 ++     O                     O       O                            |
   205000 ++                                                 O              |
          |              O                                                  |
   200000 ++                                          O              O      |
   195000 O+                    O                                           |
          |                                                                 |
   190000 ++             *..                                                |
   185000 ++           ..   .                                               |
          |           .      .                .*..                          |
   180000 ++         .        ..           ...    ..                        |
   175000 *+....   ..                    ..         ..                      |
          |     . .             *......*.                 ...*.......*......*
   170000 ++-----*------------------------------------*---------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               175592.50       +16.0%    203669.97  nhm-white/micro/aim7/misc_rtns_1
               175592.50       +16.0%    203669.97  TOTAL lock_stat.clockevents_lock.waittime-total
\f
                 lock_stat.rcu_sched_state.onoff_mutex.holdtime-total

   115000 ++----------------------------------------------------------------+
          |                                                                 |
   110000 ++                                   O                            |
          O              O                                                  |
          |                            O                                    |
   105000 ++                    O                                           |
          |      O                                           O              O
   100000 ++                                          O                     |
          |                    .*..                                         |
    95000 ++                 ..    ..                                O      |
          |      *.......  ..        ..                   ...*.......    ...*
          |    ..        *.                        ...*...           *...   |
    90000 ++ ..                        *.......*...                         |
          |..                                                               |
    85000 *+----------------------------------------------------------------+

\f
                 lock_stat.rcu_sched_state.onoff_mutex.holdtime-total

   120000 ++----------------------------------------------------------------+
          |      O                             O                            |
   115000 ++                    O                            O              |
          O                                           O              O      |
   110000 ++                                                                |
          |              O                                                  |
   105000 ++                                                                O
          |                            O                                    |
   100000 ++                                                                |
          |             .*...                                               |
    95000 ++          ..     ..                       *......*.......       |
          |         ..         .                    ..               *......|
    90000 *+..... ..            *......*.....     ..                        *
          |      *                           .. ..                          |
    85000 ++-----------------------------------*----------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                91665.09       +13.4%    103917.68  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                91105.89       +22.7%    111744.39  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               182770.97       +18.0%    215662.07  TOTAL lock_stat.rcu_sched_state.onoff_mutex.holdtime-total
\f
                       lock_stat.zone->wait_table+i.acquisitions

   175000 *+----------------------------------------------------------------+
          |   ..               .*......*.......*......*......*.......*..... |
   170000 ++    .          ....                                            .|
   165000 ++     *.......*.                                                 *
          |                                                                 |
   160000 ++                                                                |
   155000 ++                                                                |
          O                                                                 |
   150000 ++                                                                |
   145000 ++     O       O      O                                    O      |
          |                                           O      O              |
   140000 ++                           O       O                            |
   135000 ++                                                                |
          |                                                                 O
   130000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               169134.80       -14.9%    143897.00  nhm-white/micro/aim7/misc_rtns_1
               169134.80       -14.9%    143897.00  TOTAL lock_stat.zone->wait_table+i.acquisitions
\f
                    lock_stat.&(&zone->lock)->rlock.holdtime-total

   245000 ++----------------------------------------------------------------+
          |                            O                                    |
   240000 ++                                                                O
   235000 ++             O                            O                     |
          O                     O                                    O      |
   230000 ++                                                 O              |
   225000 ++                                                                |
          |                                                                 |
   220000 ++                                                                |
   215000 ++    .*.....                                                     |
          | ....       ..                      O          ...*.......*..... |
   210000 *+     O       *......*......*.......*......*...                 .|
   205000 ++                                                                *
          |                                                                 |
   200000 ++----------------------------------------------------------------+

\f
                    lock_stat.&(&zone->lock)->rlock.holdtime-total

     9e+06 ++---------------------------------------------------------------+
           |      O                                                         |
   8.5e+06 ++                                                        O      |
           O             O              O                     O             |
           |                                                                |
     8e+06 ++                    O             O                            O
           |                                                                |
   7.5e+06 ++                                                               |
           |                                                                |
     7e+06 ++                                         O                     |
           |                                                                |
           |                                                                |
   6.5e+06 ++                                      ...*.......              |
           *......*......*.......*......*......*...           *......*......*
     6e+06 ++---------------------------------------------------------------+

\f
                    lock_stat.&(&zone->lock)->rlock.holdtime-total

   255000 ++----------------------------------------------------------------+
   250000 ++                           O                                    |
          |                                                          O      |
   245000 ++                                          O                     |
   240000 ++                                                                |
          |              O                                   O              |
   235000 ++                                                                |
   230000 ++                                                                |
   225000 ++                                                                |
          |                     O                                           |
   220000 ++                                                                |
   215000 ++           ..*......*...                                        |
          O      O ....             ..         O   ...*......    ....*......O
   210000 *+.....*.                   .    ....*...          *...           *
   205000 ++---------------------------*------------------------------------+

\f
                    lock_stat.&(&zone->lock)->rlock.holdtime-total

   4.8e+06 ++---------------------------------------------------------------*
   4.7e+06 ++                                      ...*...               .. |
           *.....              ..*......    ...*...       ...          ..   |
   4.6e+06 ++    .         ....         *...                 .    ...*.     |
   4.5e+06 ++     *......*.                                   *...          |
   4.4e+06 ++                                                               |
   4.3e+06 ++                                                               |
           |                                                                |
   4.2e+06 ++            O                            O       O             |
   4.1e+06 O+                    O      O                                   |
     4e+06 ++     O                                                         O
   3.9e+06 ++                                                        O      |
           |                                                                |
   3.8e+06 ++                                  O                            |
   3.7e+06 ++---------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               210613.39        +9.2%    230072.42  lkp-a04/micro/netperf/120s-200%-TCP_RR
              6235195.34       +31.1%   8172215.89  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               210780.06        +8.5%    228689.02  lkp-a04/micro/netperf/120s-200%-UDP_RR
              4612864.37       -12.0%   4060461.88  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
             11269453.16       +12.6%  12691439.20  TOTAL lock_stat.&(&zone->lock)->rlock.holdtime-total
\f
                      lock_stat.(&(work)->work).holdtime-total

   17000 ++-----------------------------------------------------------------+
         |      O                                                           |
   16500 ++             O                                                   |
   16000 O+                                                          O      |
         |                             O                     O              |
   15500 ++                    O              O                             O
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   14500 ++                                           O                     |
   14000 ++                                                                 |
         |                                        ....*......               |
   13500 *+.....    ....*......*.......    ...*...           *.......    ...*
         |      *...                   *...                          *...   |
   13000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                13525.95       +17.0%     15830.27  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                13525.95       +17.0%     15830.27  TOTAL lock_stat.(&(work)->work).holdtime-total
\f
                      lock_stat.proc_subdir_lock.holdtime-total

   12000 ++-----------------------------------------------------------------+
         |                                                                  |
   11500 ++                            O                                    |
         |                                                                  |
   11000 ++                                   O                             |
         |      O                                                           |
   10500 ++             O                                            O      O
         O                                                   O              |
   10000 ++                                           O                     |
         |                     O                                            |
    9500 *+....                                                             |
         |     .    ....*......                                             |
    9000 ++     *...           *.......    ...*.......*......*.....         |
         |                             *...                        ..    ...*
    8500 ++----------------------------------------------------------*------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 8975.59       +16.8%     10485.40  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                 8975.59       +16.8%     10485.40  TOTAL lock_stat.proc_subdir_lock.holdtime-total
\f
                        lock_stat.sb_writers#2-R.holdtime-total

   125000 ++----------------------------------------------------------------+
          |                                                                 |
   120000 ++     O                     O                                    O
          |                                                                 |
          |                                    O                            |
   115000 ++                                                 O       O      |
          |              O      O                                           |
   110000 ++                                          O                     |
          O                                                                 |
   105000 ++                                                                |
          |                                                                 |
          |          ....*.....                      .*......*.......       |
   100000 ++  ...*...          .                 ....                *......*
          *...                  *......*.......*.                           |
    95000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                99222.00       +16.0%    115055.75  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                99222.00       +16.0%    115055.75  TOTAL lock_stat.sb_writers#2-R.holdtime-total
\f
                       lock_stat.zone->wait_table+i.acq-bounces

   130000 ++----------------------------------------------------------------+
          *...                                     ...*......               |
   125000 ++  ..             ...*......*.......*...          *.......*..... |
          |     .    ....*...                                              .|
   120000 ++     *...                                                       *
          |                                                                 |
   115000 ++                                                                |
          O                                                                 |
   110000 ++     O       O      O                                    O      |
          |                                           O      O              |
   105000 ++                           O       O                            |
          |                                                                 |
   100000 ++                                                                O
          |                                                                 |
    95000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               124847.90       -13.7%    107753.10  nhm-white/micro/aim7/misc_rtns_1
               124847.90       -13.7%    107753.10  TOTAL lock_stat.zone->wait_table+i.acq-bounces
\f
                          lock_stat.rcu_node_0.holdtime-total

   210000 ++----------------------------------------------------------------+
          |                            *.....                               |
   200000 ++                         ..      ..    ...*              *......|
   190000 ++    .*...               .          *...    :            :       *
          | ....     ...           .                    :          :        O
   180000 *+            .        ..                     :         :         |
   170000 ++             *......*                        :   O   :          |
          |                            O                  :      :          |
   160000 O+     O                                         :    :    O      |
   150000 ++                    O                          :   :            |
          |                                                 : :             |
   140000 ++                                                 *              |
   130000 ++                                          O                     |
          |                                                                 |
   120000 ++-------------O--------------------------------------------------+

\f
                          lock_stat.rcu_node_0.holdtime-total

   560000 ++----------------------------------------------------------------+
          |                                    O                            |
   540000 ++                                                                |
          O              O                                                  |
   520000 ++                           O                                    |
          |      O                                           O              O
   500000 ++                    O                                           |
          |                                           O                     |
   480000 ++                   .*..                                  O      |
          |                ....    ..                     ...*.......    ...*
   460000 ++    .*.......*.          ..              .*...           *...   |
          |   ..                                 ....                       |
   440000 ++..                         *.......*.                           |
          *.                                                                |
   420000 ++----------------------------------------------------------------+

\f
                          lock_stat.rcu_node_0.holdtime-total

   520000 ++----------------------------------------------------------------+
          |                                                                 |
   500000 O+                                                                |
          |              O                     O                            |
   480000 ++                                                         O      |
          |                            O                     O              |
   460000 ++                                                                |
          |      O                                          .*...           |
   440000 *+.                                 .*......  ....     ...       .*
          |  ..                .*.....     ...        *.            .  .... O
   420000 ++   ..          ....       .  ..                          *.     |
          |          ....*.            *.                                   |
   400000 ++     *...                                                       |
          |                     O                                           |
   380000 ++----------------------------------------------------------------+

\f
                          lock_stat.rcu_node_0.holdtime-total

   580000 ++----------------------------------------------------------------+
          |      O              O              O                            |
   560000 O+                                                 O              |
   540000 ++                                          O              O      |
          |                                                                 |
   520000 ++             O                                                  |
          |                                                                 |
   500000 ++                           O                                    O
          |                                                                 |
   480000 ++            .*.....                       *......               |
   460000 ++          ..       .                    ..       *.......       |
          |         ..          *......            .                 *......|
   440000 *+..... ..                   *.....     .                         *
          |      *                           .. ..                          |
   420000 ++-----------------------------------*----------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               183426.57       -13.4%    158876.26  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
               455570.43       +12.4%    512179.34  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
               426413.37        +8.0%    460704.73  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               452783.09       +20.3%    544579.27  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              1518193.46       +10.4%   1676339.59  TOTAL lock_stat.rcu_node_0.holdtime-total
\f
                     lock_stat.&sig->wait_chldexit.holdtime-total

   115000 ++-------------------------------------------------O--------------+
          |      O                                                          |
          |                            O                                    |
   110000 ++             O      O              O                     O      |
          O                                                                 O
          |                                                                 |
   105000 ++                                                                |
          |                                                                 |
   100000 ++                                          O                     |
          |                                                                 |
          |                                        ...*......               |
    95000 *+.....*.......*......           ....*...          *.......*......*
          |                     *......*...                                 |
          |                                                                 |
    90000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                95007.69       +15.2%    109413.23  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                95007.69       +15.2%    109413.23  TOTAL lock_stat.&sig->wait_chldexit.holdtime-total
\f
                         lock_stat.&mm->mmap_sem-W.acq-bounces

   265000 ++----------------------------------------------------------------+
   260000 ++         ....*.....                                             |
          *......*...          .    ...*.......                            .*
   255000 ++                    *...           *......*......*.......  .... |
   250000 ++                                                         *.     |
          |                                                                 |
   245000 ++                                                                |
   240000 ++                                                                |
   235000 ++                                                                |
          |                                                                 |
   230000 ++     O                                                          |
   225000 O+                                                 O       O      |
          |              O      O              O      O                     |
   220000 ++                           O                                    |
   215000 ++----------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               256302.50       -13.1%    222725.30  nhm-white/micro/aim7/misc_rtns_1
               256302.50       -13.1%    222725.30  TOTAL lock_stat.&mm->mmap_sem-W.acq-bounces
\f
                       lock_stat.sb_writers#3-R.holdtime-total

   8800 ++------------------------------------------------------------------+
   8600 ++                                                                  O
        |                                                                   |
   8400 ++                            O                             O       |
   8200 ++                                                   O              |
   8000 ++             O                                                    |
   7800 ++                     O                                            |
        O       O                                                           |
   7600 ++                                                          *.....  |
   7400 ++                                    O      O            ..      ..|
   7200 ++                                                       .          *
   7000 ++                         ...*...          .*.....     .           |
        |                  ....*...       ...     ..       .. ..            |
   6800 *+......    ...*...                  .  ..           *              |
   6600 ++------*-----------------------------*-----------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 6934.41       +14.6%      7944.90  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                 6934.41       +14.6%      7944.90  TOTAL lock_stat.sb_writers#3-R.holdtime-total
\f
                       lock_stat.entries_lock-R.holdtime-total

   9200 ++-------------O----------------------------------------------------+
        |       O              O                                            |
   9000 ++                                                          O       |
   8800 ++                                                                  O
        |                             O       O      O                      |
   8600 O+                                                                  |
   8400 ++                                                   O              |
        |                                                                   |
   8200 ++                                                                  |
   8000 ++                                                                  |
        |      .*......                      .*......*.....                 |
   7800 ++  ...        *...               ...              ..           ....*
   7600 ++..               ...          ..                   *......*...    |
        *.                    .    ...*.                                    |
   7400 ++---------------------*--------------------------------------------+

\f
                       lock_stat.entries_lock-R.holdtime-total

   9400 ++------------------------------------------------------------------+
   9200 ++                     O                                    O       |
        |              O                             O                      |
   9000 ++                                                                  O
   8800 O+                                                   O              |
   8600 ++                                                                  |
   8400 ++                                                                  |
        |                             O                                     |
   8200 ++                    .*..            O                             |
   8000 ++      O          ...    .                                .*.......|
   7800 *+....           ..        .         .*......            ..         *
   7600 ++    ..    ...*.           ..    ...        *.......  ..           |
        |       *...                    ..                   *.             |
   7400 ++                            *.                                    |
   7200 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 7727.49       +14.0%      8807.08  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                 7736.07       +13.4%      8770.58  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                15463.56       +13.7%     17577.66  TOTAL lock_stat.entries_lock-R.holdtime-total
\f
                 lock_stat.&(&mm->page_table_lock)->rlock.acq-bounces

   275000 ++----------------------------------------------------------------+
   270000 ++           ..*.....                                             |
          *......  ....        .                                           .*
   265000 ++     *.             *......*.......*......*......*.......  .... |
   260000 ++                                                         *.     |
   255000 ++                                                                |
   250000 ++                                                                |
          |                                                                 |
   245000 ++                                                                |
   240000 ++                                                                |
   235000 ++     O                                                          |
   230000 O+             O      O                            O       O      |
          |                                    O      O                     |
   225000 ++                           O                                    |
   220000 ++----------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               263989.70       -13.1%    229316.00  nhm-white/micro/aim7/misc_rtns_1
               263989.70       -13.1%    229316.00  TOTAL lock_stat.&(&mm->page_table_lock)->rlock.acq-bounces
\f
                    lock_stat.hrtimer_bases.lock#4.holdtime-total

   90000 ++-------------O---------------------------------------------------+
         |                                                                  |
   80000 ++                                                                 |
   70000 ++                                                                 |
         |                                                                  |
   60000 ++                                                                 |
         |                                                                  |
   50000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
   30000 ++                    O                                            |
         |                                                                  |
   20000 O+                                                                 O
         *......*.......*......*.......*......*.......*......*.......*......*
   10000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                13700.25       +87.2%     25651.38  nhm-white/micro/aim7/misc_rtns_1
                13700.25       +87.2%     25651.38  TOTAL lock_stat.hrtimer_bases.lock#4.holdtime-total
\f
                         lock_stat.bdev_lock.holdtime-total

   12500 ++-----------------------------------------------------------------+
         |      O                      O                                    |
   12000 ++                                                  O              |
         |                                                           O      |
         |                                                                  |
   11500 ++             O                     O                             O
         O                                                                  |
   11000 ++                    O                                            |
         |                             *..                                  |
   10500 ++                          ..   ..          O                     |
         |                         ..       ..            ...*.....        .*
         |                        .               ....*...         ..  .... |
   10000 ++  ...*.......*...... ..            *...                   *.     |
         *...                  *                                            |
    9500 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                10082.81       +14.1%     11507.23  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                10082.81       +14.1%     11507.23  TOTAL lock_stat.bdev_lock.holdtime-total
\f
                   lock_stat.&(&newf->file_lock)->rlock.acq-bounces

   265000 ++-------------*--------------------------------------------------+
   260000 *+.....  ....        .                                            |
          |      *.             *......*.......*......*......*.......    ...*
   255000 ++                                                         *...   |
   250000 ++                                                                |
          |                                                                 |
   245000 ++                                                                |
   240000 ++                                                                |
   235000 ++                                                                |
          |                                                                 |
   230000 O+     O                                           O              |
   225000 ++             O      O                     O              O      |
          |                            O       O                            |
   220000 ++                                                                O
   215000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               258345.90       -12.6%    225740.10  nhm-white/micro/aim7/misc_rtns_1
               258345.90       -12.6%    225740.10  TOTAL lock_stat.&(&newf->file_lock)->rlock.acq-bounces
\f
                          lock_stat.events-R.holdtime-total

   33000 ++----------------------------O------------------------------------+
   32500 O+     O                                                           O
         |                                    O                      O      |
   32000 ++             O      O                             O              |
   31500 ++                                                                 |
         |                                                                  |
   31000 ++                                                                 |
   30500 ++                                                                 |
   30000 ++                                           O                     |
         |                                                                  |
   29500 ++                                          .*.....                |
   29000 ++                                       ...       .               |
         *.....       ..*......                 ..           *.......       |
   28500 ++    .  ....         *.......    ...*.                     *......|
   28000 ++-----*----------------------*------------------------------------*

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                28512.94       +12.8%     32154.48  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                28512.94       +12.8%     32154.48  TOTAL lock_stat.events-R.holdtime-total
\f
                        lock_stat.rename_lock.waittime-total

   50000 ++--------------------*-------*------------------------------------+
         |                   ..               *..                           |
   49000 ++                ..                    ..                        .*
   48000 ++              ..                        ..                    .. |
         |          ....*                            .                 ..   |
   47000 ++  ...*...                                  *......*.......*.     |
   46000 *+..                                                               |
         |                                                                  |
   45000 ++                                                                 |
   44000 ++                                                                 |
         |                                                                  |
   43000 O+                                           O                     |
   42000 ++     O                             O                             |
         |              O      O       O                     O       O      |
   41000 ++-----------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                47912.04       -12.3%     41996.01  nhm-white/micro/aim7/misc_rtns_1
                47912.04       -12.3%     41996.01  TOTAL lock_stat.rename_lock.waittime-total
\f
                       lock_stat.&(&fs->lock)->rlock.acq-bounces

   230000 ++-------------*--------------------------------------------------+
          *......  ....        .                                           .*
   225000 ++     *.             *......*.......*......*......*.......  .... |
   220000 ++                                                         *.     |
          |                                                                 |
   215000 ++                                                                |
   210000 ++                                                                |
          |                                                                 |
   205000 ++                                                                |
   200000 O+     O                                                          |
          |              O      O                     O      O       O      |
   195000 ++                                   O                            |
   190000 ++                           O                                    |
          |                                                                 O
   185000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               224803.20       -12.6%    196381.70  nhm-white/micro/aim7/misc_rtns_1
               224803.20       -12.6%    196381.70  TOTAL lock_stat.&(&fs->lock)->rlock.acq-bounces
\f
                     lock_stat.rename_lock.contentions.SyS_getcwd

   108000 ++----------------------------------------------------------------+
   106000 ++                   .*......*.......*...                         |
          |                  ..                    ..                      .*
   104000 *+.....*.......  ..                        .                   .. |
   102000 ++             *.                           *......*.......  ..   |
   100000 ++                                                         *.     |
    98000 ++                                                                |
          |                                                                 |
    96000 ++                                                                |
    94000 ++                                                                |
    92000 O+                                   O      O                     |
    90000 ++     O       O                                   O              |
          |                            O                             O      |
    88000 ++                    O                                           |
    86000 ++----------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               103370.20       -12.7%     90277.20  nhm-white/micro/aim7/misc_rtns_1
               103370.20       -12.7%     90277.20  TOTAL lock_stat.rename_lock.contentions.SyS_getcwd
\f
                    lock_stat.hrtimer_bases.lock#5.holdtime-total

   22000 ++-----------------------------------O-----------------------------+
   21000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
   19000 ++                                                                 |
         |                                                                  |
   18000 ++                                                                 |
   17000 ++                                                                 |
   16000 ++                            O                                    |
         |      O                                                           O
   15000 ++             O      O                                     O      |
   14000 O+                                           O   ...*.......       |
         |   ...*.......*......            ...*.......*...           *......|
   13000 *+..                  *.......*...                                 *
   12000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                13222.56       +18.3%     15637.30  nhm-white/micro/aim7/misc_rtns_1
                13222.56       +18.3%     15637.30  TOTAL lock_stat.hrtimer_bases.lock#5.holdtime-total
\f
                 lock_stat.&mapping->i_mmap_mutex.contentions.vma_link

   740000 ++----------------------------------------------------------------+
          |                                                                 |
   720000 ++         ....*.....                                             |
   700000 *+.....*...          .    ...*.......    ...*......*.....        .*
          |                     *...           *...                ..  .... |
   680000 ++                                                         *.     |
          |                                                                 |
   660000 ++                                                                |
          |                                                                 |
   640000 ++     O                                                          |
   620000 O+             O      O                            O       O      |
          |                            O       O      O                     |
   600000 ++                                                                O
          |                                                                 |
   580000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               702867.20       -12.0%    618778.40  nhm-white/micro/aim7/misc_rtns_1
               702867.20       -12.0%    618778.40  TOTAL lock_stat.&mapping->i_mmap_mutex.contentions.vma_link
\f
                   lock_stat.&(&sighand->siglock)->rlock.acq-bounces

   245000 ++-------------*--------------------------------------------------+
          *......  ....        .                                            |
   240000 ++     *.             *......*.......*......*......*.......    ...*
   235000 ++                                                         *...   |
          |                                                                 |
   230000 ++                                                                |
   225000 ++                                                                |
          |                                                                 |
   220000 ++                                                                |
   215000 ++                                                                |
          O      O       O                                   O       O      |
   210000 ++                    O              O      O                     |
   205000 ++                           O                                    |
          |                                                                 O
   200000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               238640.40       -11.8%    210514.70  nhm-white/micro/aim7/misc_rtns_1
               238640.40       -11.8%    210514.70  TOTAL lock_stat.&(&sighand->siglock)->rlock.acq-bounces
\f
                        lock_stat.clockevents_lock.con-bounces

   170000 ++----------------------------------------------------------------+
          |                                                                 |
   165000 ++                           O       O                            O
          |      O       O                                                  |
          O                                           O      O       O      |
   160000 ++                    O                                           |
          |                                                                 |
   155000 ++                                                                |
          |                                                                 |
   150000 ++            .*..                                                |
          |          ...    ..                                              |
          |        ..         ..                                            |
   145000 *+.....*.                                ...*......*.......*......*
          |                     *......*.......*...                         |
   140000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               144491.30       +12.8%    163038.30  nhm-white/micro/aim7/misc_rtns_1
               144491.30       +12.8%    163038.30  TOTAL lock_stat.clockevents_lock.con-bounces
\f
               lock_stat.clockevents_lock.contentions.clockevents_notify

   335000 ++-----------------------------------O----------------------------+
   330000 ++                           O                                    O
          |      O       O                                                  |
   325000 O+                                          O      O       O      |
   320000 ++                    O                                           |
   315000 ++                                                                |
   310000 ++                                                                |
          |                                                                 |
   305000 ++                                                                |
   300000 ++            .*..                                                |
   295000 ++         ...    ..                                              |
   290000 ++       ..         ..                                            |
          *......*.                 ...*.......*......*......*.......*......*
   285000 ++                    *...                                        |
   280000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               289138.00       +12.8%    326170.80  nhm-white/micro/aim7/misc_rtns_1
               289138.00       +12.8%    326170.80  TOTAL lock_stat.clockevents_lock.contentions.clockevents_notify
\f
                        lock_stat.clockevents_lock.contentions

   170000 ++----------------------------------------------------------------+
          |                                                                 |
   165000 ++                           O       O                            O
          |      O       O                                   O              |
          O                                           O              O      |
   160000 ++                    O                                           |
          |                                                                 |
   155000 ++                                                                |
          |                                                                 |
   150000 ++            .*..                                                |
          |          ...    ..                                              |
          |        ..         ..                                            |
   145000 *+.....*.                 ...*.......    ...*......*.......*......*
          |                     *...           *...                         |
   140000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               144569.00       +12.8%    163085.40  nhm-white/micro/aim7/misc_rtns_1
               144569.00       +12.8%    163085.40  TOTAL lock_stat.clockevents_lock.contentions
\f
                     lock_stat.&mapping->i_mmap_mutex.con-bounces

   600000 ++-------------*--------------------------------------------------+
   590000 *+.....  ....      ..                                            .*
          |      *.            .    ...*.......    ...*......*.....      .. |
   580000 ++                    *...           *...                ..  ..   |
   570000 ++                                                         *.     |
          |                                                                 |
   560000 ++                                                                |
   550000 ++                                                                |
   540000 ++                                                                |
          |                                                                 |
   530000 O+     O                                                          |
   520000 ++             O      O                     O      O       O      |
          |                                    O                            |
   510000 ++                           O                                    |
   500000 ++----------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               585553.00       -11.5%    518161.60  nhm-white/micro/aim7/misc_rtns_1
               585553.00       -11.5%    518161.60  TOTAL lock_stat.&mapping->i_mmap_mutex.con-bounces
\f
                     lock_stat.&mapping->i_mmap_mutex.contentions

   600000 ++-------------*--------------------------------------------------+
   590000 *+.....  ....      ..                                            .*
          |      *.            .    ...*.......    ...*......*.....      .. |
   580000 ++                    *...           *...                ..  ..   |
   570000 ++                                                         *.     |
          |                                                                 |
   560000 ++                                                                |
   550000 ++                                                                |
   540000 ++                                                                |
          |                                                                 |
   530000 O+     O                                                          |
   520000 ++             O      O                     O      O       O      |
          |                                    O                            |
   510000 ++                           O                                    |
   500000 ++----------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               585553.00       -11.5%    518161.60  nhm-white/micro/aim7/misc_rtns_1
               585553.00       -11.5%    518161.60  TOTAL lock_stat.&mapping->i_mmap_mutex.contentions
\f
                          lock_stat.((&timer)).acquisitions

   54000 ++-----------------------------------------------------------------+
         |                                    O                             |
   52000 O+             O                                                   |
         |                             O                                    |
   50000 ++                                                                 O
         |      O              O                             O              |
   48000 ++                                           O                     |
         |                    .*..                                          |
   46000 ++               ....    ..                                 O      |
         |      *.......*.          ..                    ...*.......    ...*
   44000 ++   ..                      .           ....*...           *...   |
         |  ..                         *......*...                          |
   42000 ++.                                                                |
         *                                                                  |
   40000 ++-----------------------------------------------------------------+

\f
                          lock_stat.((&timer)).acquisitions

   52000 O+-----------------------------------------------------------------+
         |              O                                                   |
   50000 ++                                   O                             |
         |                                                   O       O      |
         |                             O                                    |
   48000 ++                                                                 |
         |                                                                  |
   46000 ++     O                                           .*..            O
         |                                   .*.....  O   ..    ..         .*
   44000 *+.                               ..       ..  ..        ..     .. |
         |  ..              ...*.......  ..           *.            .  ..   |
         |    ..      ..*...           *.                            *.     |
   42000 ++       ....         O                                            |
         |      *.                                                          |
   40000 ++-----------------------------------------------------------------+

\f
                          lock_stat.((&timer)).acquisitions

   52000 ++-----------------------------------O-----------------------------+
         |      O              O                                            |
   50000 ++                                           O                     |
         |                                                                  |
         O                                                   O       O      |
   48000 ++                                                                 |
         |              *..                                                 |
   46000 ++           ..   ..                         *......*.......       O
         |           .       ..                     ..               *..... |
   44000 ++         .                  O          ..                       .|
         |        ..           *.......*.....    .                          *
         *...... .                           . ..                           |
   42000 ++     *                             *                             |
         |                                                                  |
   40000 ++-----------------------------------------------------------------+

\f
                          lock_stat.((&timer)).acquisitions

   54000 ++-----------------------------------------------------------------+
         |                                                                  |
   52000 ++                            O                                    |
   50000 ++                                                                 O
         |                                                                  |
   48000 O+     O                                                           |
         |                     O              O       O              O      |
   46000 ++                                               ...*..            |
         *.....         O                            .*...   O  ..          |
   44000 ++    .                                  ...             ..        |
   42000 ++     *.......                        ..                  .       |
         |              *......              .*.                     *......*
   40000 ++                    *.......  ....                               |
         |                             *.                                   |
   38000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                44203.70       +12.7%     49813.00  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                43536.10       +10.3%     48011.80  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                43925.00       +11.1%     48784.80  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                42372.00       +12.9%     47836.90  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               174036.80       +11.7%    194446.50  TOTAL lock_stat.((&timer)).acquisitions
\f
                           lock_stat.rcu_node_0.acquisitions

   125000 ++----------------------------------------------------------------+
          |                                                                 |
   120000 ++                                   O                            |
          O              O                                                  |
   115000 ++                           O                                    |
          |                                                                 O
   110000 ++     O                                           O              |
          |                     O                     O                     |
   105000 ++                   .*...                                        |
          |      *.......  ....     ..                    ...*.......O   ...*
   100000 ++   ..        *.           .            ...*...           *...   |
          |  ..                        *.......*...                         |
    95000 ++.                                                               |
          *                                                                 |
    90000 ++----------------------------------------------------------------+

\f
                           lock_stat.rcu_node_0.acquisitions

   120000 ++-----------------------------------O----------------------------+
          |                     O                     O                     |
   115000 ++     O                                                          |
          |                                                  O              |
          O                                                          O      |
   110000 ++                                                                |
          |              O                                                  |
   105000 ++             *...                         *......               O
          |            ..    ..                     ..       *.......       |
   100000 ++         ..        .       O           .                 *..... |
          |         .           *......*.....     .                        .|
          *...... ..                         .. ..                          *
    95000 ++     *                             *                            |
          |                                                                 |
    90000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                99556.90       +12.4%    111888.40  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                99363.60       +12.6%    111868.00  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               198920.50       +12.5%    223756.40  TOTAL lock_stat.rcu_node_0.acquisitions
\f
                   lock_stat.&(&p->alloc_lock)->rlock.holdtime-total

   190000 ++----------------------------------------------------------------+
          |                            O                                    |
   185000 ++     O                                                          |
          |                                                  O       O      |
   180000 ++                                                                O
          |              O      O              O                            |
   175000 ++                                                                |
          O                                                                 |
   170000 ++                                                                |
          |                                                                 |
   165000 ++                                          O                     |
          |                                                                 |
   160000 *+.....    ....*......    ...*.......    ...*......*.......*......|
          |      *...           *...           *...                         *
   155000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               159105.97       +12.2%    178524.65  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               159105.97       +12.2%    178524.65  TOTAL lock_stat.&(&p->alloc_lock)->rlock.holdtime-total
\f
                  lock_stat.&child->perf_event_mutex.holdtime-total

   15500 ++-----------------------------------------------------------------+
         |                                                   O              |
   15000 ++                    O       O      O                             |
         |                                                                  O
   14500 ++             O                                                   |
         |      O                                                           |
   14000 O+                                                          O      |
         |                                                                  |
   13500 ++                                               ...*...          .*
         |             .*..                         ..*...       ...     .. |
   13000 *+....     ...    ..         .*......  ....                .  ..   |
         |     .  ..         ..    ...        *.      O              *.     |
   12500 ++     *.               ..                                         |
         |                     *.                                           |
   12000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                12962.55       +11.3%     14431.29  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                12962.55       +11.3%     14431.29  TOTAL lock_stat.&child->perf_event_mutex.holdtime-total
\f
             lock_stat.&mapping->i_mmap_mutex.contentions.unlink_file_vma

   480000 ++----------------------------------------------------------------+
          *......    ....*.....                                             |
   470000 ++     *...          .    ...*.......           ...*.......    ...*
   460000 ++                    *...           *......*...           *...   |
          |                                                                 |
   450000 ++                                                                |
          |                                                                 |
   440000 ++                                                                |
          |                                                                 |
   430000 ++                                                                |
   420000 O+     O                                           O              |
          |              O      O                     O              O      |
   410000 ++                                   O                            |
          |                            O                                    |
   400000 ++----------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               467032.10       -10.9%    416276.60  nhm-white/micro/aim7/misc_rtns_1
               467032.10       -10.9%    416276.60  TOTAL lock_stat.&mapping->i_mmap_mutex.contentions.unlink_file_vma
\f
                         lock_stat.sk_lock-AF_INET.acquisitions

    2.7e+06 ++--------------------------------------------------------------+
            |   ...*......                                               ...*
   2.65e+06 *+..          *......*......*.......*......*......*......*...   |
    2.6e+06 ++                                                              |
            |                                                               |
   2.55e+06 ++                                                              |
    2.5e+06 ++                                         O                    |
            |                                                               |
   2.45e+06 ++                                                              |
    2.4e+06 ++                                                              |
            |                    O              O                           O
   2.35e+06 ++            O                                   O             |
    2.3e+06 O+                          O                            O      |
            |      O                                                        |
   2.25e+06 ++--------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              2639031.80       -10.6%   2360489.00  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              2639031.80       -10.6%   2360489.00  TOTAL lock_stat.sk_lock-AF_INET.acquisitions
\f
                   lock_stat.&fsnotify_mark_srcu-R.holdtime-total

   27500 ++-----------------------------------------------------------------+
         |                     O                                            |
   27000 ++                                                                 |
   26500 ++             O                             O              O      O
         O                             O                                    |
   26000 ++                                                                 |
         |                                                   O              |
   25500 ++     O                                                           |
         |                                    O                             |
   25000 *+....                                                             |
   24500 ++    .                                                            |
         |      *.....                                                      |
   24000 ++           ..    ...*.......*......                              |
         |              *...                  *.......*......*.......    ...*
   23500 ++----------------------------------------------------------*------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                23964.02        +9.5%     26236.37  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                23964.02        +9.5%     26236.37  TOTAL lock_stat.&fsnotify_mark_srcu-R.holdtime-total
\f
                         lock_stat.jiffies_lock.holdtime-total

   720000 ++----------------------------------------------------------------+
          |          ....*.....       .*.......*...                         |
   710000 *+.....*...          .  ....             ..            ....*......*
   700000 ++                    *.                   .    ...*...           |
          |                                           *...                  |
   690000 ++                                                                |
   680000 ++                                                                |
          |                                                                 |
   670000 ++                                                                |
   660000 ++                                                                |
          |                                                                 |
   650000 ++                    O                                           |
   640000 ++                                   O             O              O
          |              O             O              O                     |
   630000 O+-----O---------------------------------------------------O------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               706694.52        -9.6%    638876.20  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
               706694.52        -9.6%    638876.20  TOTAL lock_stat.jiffies_lock.holdtime-total
\f
                     lock_stat.hrtimer_bases.lock.holdtime-total

   35000 ++----------------------------------------------------------O------+
         |                                                                  |
         |                                                                  |
   30000 ++                                                                 |
         |                                                                  |
         |                                                                  |
   25000 ++                                                                 |
         |      O                      O              O                     |
   20000 ++                                                                 |
         |                                                                  |
         |                                                                  |
   15000 O+  ...*.......O      O   ....*......O   ....*......*.......       O
         *...           *......*...           *...                   *......*
         |                                                                  |
   10000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                14111.25       +36.4%     19245.96  nhm-white/micro/aim7/misc_rtns_1
                14111.25       +36.4%     19245.96  TOTAL lock_stat.hrtimer_bases.lock.holdtime-total
\f
                          lock_stat.pgd_lock.holdtime-total

   56000 ++-----------------------------------------------------------------+
   55000 ++                            O                                    |
         |      O                                                           |
   54000 ++                                                                 O
   53000 ++                    O                             O       O      |
         |                                    O                             |
   52000 O+             O                                                   |
   51000 ++                                           O                     |
   50000 ++                                                                 |
         |                                                                  |
   49000 ++                                         ..*.....                |
   48000 *+.....    ....*.....                  ....        .               |
         |      *...          .    ....*......*.             *.......       |
   47000 ++                    *...                                  *......*
   46000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                47545.73       +11.3%     52915.50  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                47545.73       +11.3%     52915.50  TOTAL lock_stat.pgd_lock.holdtime-total
\f
                       lock_stat.tasklist_lock-W.holdtime-total

   370000 ++----------------------------------------------------------------+
          |                            O                                    |
   360000 ++     O                                           O              |
          |                                                                 O
          |              O      O              O                     O      |
   350000 ++                                                                |
          |                                                                 |
   340000 O+                                                                |
          |                                                                 |
   330000 ++                                          O                     |
          |                                                                 |
          |          ....*......                   ...*......            ...*
   320000 *+.....*...           *......*.......*...          *.......*...   |
          |                                                                 |
   310000 ++----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               319799.80       +10.4%    353118.34  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               319799.80       +10.4%    353118.34  TOTAL lock_stat.tasklist_lock-W.holdtime-total
\f
                     lock_stat.&(&dentry->d_lock)->rlock.acq-bounces

   1.35e+07 ++--------------------------------------------------------------+
            |                               ....*......                     |
            *......*......*......*......*...           *......*......    ...*
    1.3e+07 ++                                                       *...   |
            |                                                               |
            |                                                               |
   1.25e+07 ++                                                              |
            |                                                               |
    1.2e+07 O+                                                       O      |
            |      O      O                            O      O             |
            |                                                               |
   1.15e+07 ++                                  O                           |
            |                    O      O                                   |
            |                                                               O
    1.1e+07 ++--------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
             13131948.50       -10.6%  11739786.10  nhm-white/micro/aim7/misc_rtns_1
             13131948.50       -10.6%  11739786.10  TOTAL lock_stat.&(&dentry->d_lock)->rlock.acq-bounces
\f
                     lock_stat.&mapping->i_mmap_mutex.acq-bounces

   5.8e+06 ++----------------------------------*----------------------------+
           *......*......*.......*......*...          *.......*......    ...*
   5.7e+06 ++                                                        *...   |
   5.6e+06 ++                                                               |
           |                                                                |
   5.5e+06 ++                                                               |
   5.4e+06 ++                                                               |
           |                                                                |
   5.3e+06 O+                                                        O      |
   5.2e+06 ++     O      O                            O       O             |
           |                                   O                            |
   5.1e+06 ++                                                               |
     5e+06 ++                    O      O                                   |
           |                                                                |
   4.9e+06 ++---------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              5748518.20       -10.3%   5156548.00  nhm-white/micro/aim7/misc_rtns_1
              5748518.20       -10.3%   5156548.00  TOTAL lock_stat.&mapping->i_mmap_mutex.acq-bounces
\f
                        lock_stat.&mm->mmap_sem-R.holdtime-total

   1.12e+07 ++--------------------------------------------------------------+
    1.1e+07 ++            O                                                 |
            |                 ...*..                                        *
   1.08e+07 ++            *...      .                                      .|
   1.06e+07 ++           +           .                     ...*..         . |
   1.04e+07 ++          +             ..              .*...      ..     ..  |
   1.02e+07 ++         +                          ....             ..  .    |
            |         +                 *.......*.                    .     |
      1e+07 ++       +           O                                   *      |
    9.8e+06 ++      +                                         O             |
    9.6e+06 ++  ...*                            O                           |
    9.4e+06 *+..                                       O             O      |
            |                                                               O
    9.2e+06 O+     O                                                        |
      9e+06 ++--------------------------O-----------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
             10293967.02        -6.7%   9599300.67  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
             10293967.02        -6.7%   9599300.67  TOTAL lock_stat.&mm->mmap_sem-R.holdtime-total
\f
                             perf-stat.context-switches

   1e+07 ++-----------------------------------------------------------------+
         |                                                                  |
   9e+06 ++     O       O      O       O      O       O      O       O      O
   8e+06 O+                                                                 |
         |                                                                  |
   7e+06 ++                                                                 |
   6e+06 ++                                                                 |
         |                                                                  |
   5e+06 ++                                                                 |
   4e+06 ++                                                                 |
         |                                                                  |
   3e+06 ++                                                                 |
   2e+06 ++                                                                 |
         *......    ....*......*.......           ....*......*.......    ...*
   1e+06 ++-----*----------------------*------*----------------------*------+

\f
                             perf-stat.context-switches

   1e+07 ++-----------------------------------------------------------------+
   9e+06 ++     O                                                           |
         O              O      O              O              O       O      |
   8e+06 ++                            O              O                     O
   7e+06 ++                                                                 |
         |                                                                  |
   6e+06 ++                                                                 |
   5e+06 ++                                                                 |
   4e+06 ++                                                                 |
         |                                                                  |
   3e+06 ++                                                                 |
   2e+06 ++                                                                 |
         *......*.......*......*.......    ...*.......           ....*......*
   1e+06 ++                            *...           *......*...           |
       0 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
              1346228.90      +550.7%   8759648.50  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
              1140515.70      +636.7%   8401818.10  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
              2486744.60      +590.1%  17161466.60  TOTAL perf-stat.context-switches
\f
                                  vmstat.system.cs

   16000 ++-----------------------------------------------------------------+
         |                             O      O                             |
   14000 O+     O       O      O                      O      O       O      O
   12000 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
    4000 ++                                                                 |
         |                                                                  |
    2000 *+.....*.......*......*.......*......*.......*......*.......*......*
         |                                                                  |
       0 ++-----------------------------------------------------------------+

\f
                                  vmstat.system.cs

   16000 ++-----------------------------------------------------------------+
         |      O                                                           |
   14000 O+             O      O              O                             |
   12000 ++                            O              O      O       O      O
         |                                                                  |
   10000 ++                                                                 |
         |                                                                  |
    8000 ++                                                                 |
         |                                                                  |
    6000 ++                                                                 |
    4000 ++                                                                 |
         |                                                                  |
    2000 *+.....*.......*......*.......    ...*.......           ....*......*
         |                             *...           *......*...           |
       0 ++-----------------------------------------------------------------+

\f
                                  vmstat.system.cs

   6120 ++-------------*----------------------------------------------------+
   6110 ++          ..     ...                                              |
        |         ..          .                                             |
   6100 ++     .*.             *.....                                       |
   6090 ++  ...                      .              .*.....                 |
        | ..                          *.......  ....       ..    ...*.......*
   6080 *+                                    *.             *...           |
   6070 ++                                                   O              |
   6060 ++                                                                  |
        |                                            O                      |
   6050 O+             O                      O                             O
   6040 ++                            O                                     |
        |       O              O                                            |
   6030 ++                                                          O       |
   6020 ++------------------------------------------------------------------+

\f
                                  vmstat.system.cs

   7200 ++------------------------------------------------------------------+
        |              O                      O                             |
   7000 O+                            O              O       O      O       O
        |       O                                                           |
        |                                                                   |
   6800 ++                                                                  |
        |                      O                                            |
   6600 ++                                                                  |
        |                                            *...                   |
   6400 ++                                         ..    ...                |
        |   ....*......    ....*...               .         .               |
        *...           *...        ..            .           *......    ....*
   6200 ++                           .         ..                   *...    |
        |                             *.......*                             |
   6000 ++------------------------------------------------------------------+

\f
                                  vmstat.system.cs

   15200 ++-----------------------------------------------------------------+
   15150 ++                                  .*..            *.......*......*
         |                               ....    .         ..               |
   15100 ++  ...*.......*...          .*.         ..      .                 |
   15050 *+..               ..     ...              ..   .                  |
         |                    .  ..                    ..                   |
   15000 ++                    *.                     *                     |
   14950 ++                                                                 |
   14900 ++             O                                                   |
         |                                    O                             |
   14850 ++     O                      O                                    |
   14800 O+                                                                 |
         |                                            O              O      |
   14750 ++                    O                             O              |
   14700 ++-----------------------------------------------------------------O

\f
                                  vmstat.system.cs

   11000 ++-----------------------------------------------------------------+
         |                     O                                            |
   10800 ++                                                                 |
         |                                                   O              O
   10600 ++                                           O                     |
         O      O       O              O      O                      O      |
   10400 ++                                                                 |
         |                                                                  |
   10200 ++                                                                 |
         |                                                                  |
   10000 ++                                                                 |
         *......    ....*......*.......    ...*.......*......            ...*
    9800 ++     *...                   *...                  *.......*...   |
         |                                                                  |
    9600 ++-----------------------------------------------------------------+

\f
                                  vmstat.system.cs

   7000 ++------------------------------------------------------------------+
   6500 *+......*......    ....*......*.......*......    ....*......*.......*
        |              *...                          *...                   |
   6000 ++                                                                  |
   5500 ++                                                                  |
        |                                            O                      |
   5000 ++                                                                  |
   4500 ++                                                                  |
   4000 ++                                                                  |
        |                                                                   |
   3500 ++                     O              O                             O
   3000 ++                                                                  |
        O              O              O                      O              |
   2500 ++                                                          O       |
   2000 ++------O-----------------------------------------------------------+

\f
                                  vmstat.system.cs

   19000 ++-----------------------------------------------------------------+
         *.....                               *..                   .*..... |
   18900 ++    .              .*...         ..   ..               ..       .|
         |      *.......  ....     ...    ..       ..           ..          *
   18800 ++             *.            . ..           .        ..            |
         |                             *              *......*              |
   18700 ++                                                                 |
         |                                                                  |
   18600 ++                                                                 |
         |                                    O                             O
   18500 O+             O              O                     O       O      |
         |                                            O                     |
   18400 ++                                                                 |
         |      O                                                           |
   18300 ++--------------------O--------------------------------------------+

\f
                                  vmstat.system.cs

   8200 ++---------------------------------------------------O--------------+
        |       O                                                           |
   8000 ++             O              O                                     |
        O                                            O                      |
   7800 ++                     O                                            |
        |                                                                   O
   7600 ++                                                          O       |
        |                                     O                             |
   7400 ++                                                                  |
        |                                                                   |
   7200 ++                                                                  |
        |                            .*.......    ...*.......             ..*
   7000 *+......    ...*.......  ....         *...           *......  ....  |
        |       *...           *.                                   *.      |
   6800 ++------------------------------------------------------------------+

\f
                                  vmstat.system.cs

   18000 ++-----------------------------------------------------------------+
   17500 *+.....                                                            |
         |      *.......*......*.......*......*.......*......*.......    ...*
   17000 ++                                                          *...   |
   16500 ++                                                                 |
         |                                                                  |
   16000 ++                                                                 |
   15500 ++                                                                 |
   15000 ++                                                                 |
         |                                                                  |
   14500 O+     O       O                                    O              |
   14000 ++                                           O              O      |
         |                     O                                            |
   13500 ++                            O      O                             O
   13000 ++-----------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 2188.06      +536.9%     13936.66  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                 1856.67      +619.0%     13348.93  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                 6088.80        -0.7%      6045.24  lkp-a04/micro/netperf/120s-200%-TCP_CRR
                 6244.83       +12.2%      7009.51  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                15097.54        -1.9%     14805.62  lkp-a04/micro/netperf/120s-200%-TCP_RR
                 9852.45        +7.7%     10615.51  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                 6557.39       -53.3%      3060.28  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                18853.10        -2.0%     18473.45  lkp-a04/micro/netperf/120s-200%-UDP_RR
                 7003.45       +12.2%      7859.98  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                17359.45       -19.6%     13961.00  nhm-white/micro/aim7/misc_rtns_1
                91101.74       +19.8%    109116.17  TOTAL vmstat.system.cs
\f
                                  iostat.cpu.user

   1.6 ++-------------------------------------------------------------------+
       |                                     O                              |
   1.4 ++      O                                                            O
       O                                                            O       |
       |                              O              O       O              |
   1.2 ++             O       O                                             |
       |                                                                    |
     1 ++                                                                   |
       |                                                                    |
   0.8 ++                                                                   |
       |                                                                    |
       |                                                                    |
   0.6 ++  ....*......            ....*......*.......*.....        .*.......|
       *...           *.......*...                         ..  ....         *
   0.4 ++----------------------------------------------------*--------------+

\f
                                  iostat.cpu.user

   1.6 ++-------------------------------------------------------------------+
       O                                                                    |
   1.4 ++      O                                             O              |
       |              O       O                                     O       |
       |                                     O       O                      O
   1.2 ++                                                                   |
       |                              O                                     |
     1 ++                                                                   |
       |                                                                    |
   0.8 ++                                                                   |
       |                                                                    |
       |                                                                    |
   0.6 ++                         ....*......*.......    ....*......        |
       *.......    ...*.......*...                   *...           *.......*
   0.4 ++------*------------------------------------------------------------+

\f
                                  iostat.cpu.user

   5.1 ++-------------------------------------------------------------------+
       O                                     O                              |
     5 ++                                                                   |
       |                              O              O                      O
   4.9 ++             O                                      O              |
   4.8 ++                                                                   |
       |       O                                                            |
   4.7 ++                                                                   |
       |                      O                                     O       |
   4.6 ++                                                                   |
       |                             .*......                               |
   4.5 *+....        .*.....      ...        *.......    ....*.....         |
   4.4 ++    ..  ....       ..  ..                   *...          .    ....*
       |       *.             *.                                    *...    |
   4.3 ++-------------------------------------------------------------------+

\f
                                  iostat.cpu.user

   3.6 ++-------------------------------------------------------------------+
       |              O                      O                              |
   3.5 O+      O                      O                                     |
       |                                             O              O       |
   3.4 ++                                                                   |
       |                                                                    |
   3.3 ++                                                                   |
       |                                                     O              |
   3.2 ++                                                                   O
       |                                                                    |
   3.1 ++                                                                   |
       |           ...*.....  O                  ....*.......               |
     3 *+......*...         ..            ...*...            *......*.......*
       |                      *.......*...                                  |
   2.9 ++-------------------------------------------------------------------+

\f
                                  iostat.cpu.user

   5.2 ++-------------------------------------------------------------------+
       |                      O                                             |
     5 ++                                                    O              |
       O              O                                             O       O
       |       O                             O                              |
   4.8 ++                                            O                      |
       |                                             *..                    |
   4.6 ++                             *..          ..   .                   |
       |                            ..   ..      ..      ..                 |
   4.4 *+....                     ..       ..   .          ..               |
       |     ..                  .            ..                            |
       |       *......*....... ..            *               *.....       ..*
   4.2 ++                     *                                    .  ....  |
       |                                                            *.      |
     4 ++-------------------------------------------------------------------+

\f
                                   iostat.cpu.user

    3.3 ++------------------------------------------------------------------+
        |                             *..                                   |
   3.25 ++      *...                 +   ..                                 |
        |      :    ..              +      ..                *.....         |
    3.2 ++    :       .            +         .             ..      .        |
        |    :         *...       +           *...       ..         *.......*
   3.15 ++  :              ...   +                ..    .                   |
        |   :                 . +                   . ..                    |
    3.1 ++ :           O       *                     *                      |
        | :     O                                                           |
   3.05 ++                                                          O       |
        *                      O              O                             |
      3 O+                                           O                      |
        |                                                    O              |
   2.95 ++----------------------------O-------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                    0.52      +151.4%         1.30  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                    0.52      +156.3%         1.32  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                    4.43       +10.0%         4.87  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                    3.00       +13.4%         3.40  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                    4.33       +12.7%         4.88  lkp-a04/micro/netperf/120s-200%-UDP_RR
                    3.17        -4.7%         3.02  nhm-white/micro/aim7/misc_rtns_1
                   15.96       +17.8%        18.79  TOTAL iostat.cpu.user
\f
                                   vmstat.cpu.us

   1.6 ++-------------------------------------------------------------------+
       |                                     O                              |
   1.4 O+      O                                                            O
       |                              O              O       O      O       |
   1.2 ++             O       O                                             |
       |                                                                    |
     1 ++                                                                   |
       |                                                                    |
   0.8 ++                                                                   |
       |                                                                    |
   0.6 ++                                                                   |
       *.......*......    ....*.......*......*.......*.......    ...*.......*
   0.4 ++             *...                                   *...           |
       |                                                                    |
   0.2 ++-------------------------------------------------------------------+

\f
                                   vmstat.cpu.us

   1.6 ++-------------------------------------------------------------------+
       O                                                     O              |
   1.4 ++      O                                                            |
       |              O       O                      O              O       |
       |                                     O                              O
   1.2 ++                             O                                     |
       |                                                                    |
     1 ++                                                                   |
       |                                                                    |
   0.8 ++                                                                   |
       |                                                                    |
       |                            ..*......            ....*......        |
   0.6 ++                       ....         *.......*...           *.......|
       *.......    ...*.......*.                                            *
   0.4 ++------*------------------------------------------------------------+

\f
                                   vmstat.cpu.us

     5 ++-------------------------------------------------------------------+
       O                                     O                              |
   4.9 ++                             O                                     |
       |                                             O                      O
   4.8 ++             O                                      O              |
   4.7 ++                                                                   |
       |       O                                                            |
   4.6 ++                                                                   |
       |                      O                                     O       |
   4.5 ++                                                                   |
       |                            ..*......                               |
   4.4 *+....        .*.......  ....         *.......*.......*.....         |
   4.3 ++    ..  ....         *.                                   .    ....*
       |       *.                                                   *...    |
   4.2 ++-------------------------------------------------------------------+

\f
                                   vmstat.cpu.us

   4.8 ++-------------------------------------------------------------------O
   4.7 ++                                                                   |
       |                      *.                     O                      |
   4.6 ++                    :  ..                                          |
   4.5 O+      O            : O   .          O       *.      O              |
       |                   :       .  O             :  ..                   |
   4.4 ++     .*..        :         ..             :     .          O       |
   4.3 ++  ...    .       :                        :      .                 |
   4.2 ++..        .  O  :            *.          :        ..               |
       *.           ..  :               .        :                          |
   4.1 ++              :                 ..     :            *......        |
     4 ++             *                    .   :                    *.......|
       |                                    .  :                            *
   3.9 ++                                     :                             |
   3.8 ++------------------------------------*------------------------------+

\f
                                   vmstat.cpu.us

   3.5 ++-------------O----------------------O------------------------------+
       |       O                                                            |
   3.4 ++                             O                                     |
       O                                             O              O       |
   3.3 ++                                                                   |
       |                                                                    |
   3.2 ++                                                    O              |
       |                                                                    |
   3.1 ++                                                                   O
       |                      O                                             |
     3 ++                                                                   |
       |   ....*......*.......                   ....*.......               |
   2.9 *+..                   *.......    ...*...            *......    ....*
       |                              *...                          *...    |
   2.8 ++-------------------------------------------------------------------+

\f
                                   vmstat.cpu.us

   5.2 ++-------------------------------------------------------------------+
       |                                                                    |
     5 ++                     O                                             |
       |                                                     O              |
       |              O                                             O       |
   4.8 O+      O                             O       O                      O
       |                                                                    |
   4.6 ++                             *..                                   |
       |                             .O  ..         .*..                    |
   4.4 *+..                         .      ..    ...    ..                  |
       |   ...                    ..           ..         ..                |
       |      .                  .           *.             .               |
   4.2 ++      *......*.....    .                            *.....       ..*
       |                    .. .                                   .  ....  |
     4 ++---------------------*-------------------------------------*-------+

\f
                                    vmstat.cpu.us

    3.3 ++------------------------------------------------------------------+
        |                             *..                                   |
   3.25 ++                           +   .                                  |
    3.2 ++                          +     ..                                |
        |       *.....             +        ..                            ..*
   3.15 ++     .      .           +                        ..*......  ....  |
    3.1 ++    .        *.....    +            *......  ....         *.      |
        |   ..  O      O     .. +                    *.                     |
   3.05 ++ .                   *                                            |
      3 ++.                                                         O       |
        O.                                                                  |
   2.95 *+                     O              O      O                      O
    2.9 ++                            O                                     |
        |                                                    O              |
   2.85 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                    0.49      +170.8%         1.33  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                    0.53      +155.6%         1.35  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                    4.34       +10.2%         4.78  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                    4.18        +7.7%         4.51  lkp-a04/micro/netperf/120s-200%-TCP_RR
                    2.90       +14.3%         3.32  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                    4.25       +13.2%         4.81  lkp-a04/micro/netperf/120s-200%-UDP_RR
                    3.12        -4.6%         2.97  nhm-white/micro/aim7/misc_rtns_1
                   19.81       +16.5%        23.07  TOTAL vmstat.cpu.us
\f
                          time.involuntary_context_switches

   2800 ++------------------------------------------------------------------+
        |                                                                   O
   2600 ++                                                                  |
        |                                     O                             |
        |                                            O              O       |
   2400 ++      O                                                           |
        |                      O                             O              |
   2200 ++                            O                                     |
        |              O                                                    |
   2000 O+                                                         .*...    |
        *.....        .*.....        .*..                      ....     ... |
        |     ..  ....       ..  ....    ..                ..*.            .|
   1800 ++      *.             *.          ..          ....                 *
        |                                    .    ...*.                     |
   1600 ++------------------------------------*-----------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 1818.70       +29.6%      2356.40  nhm-white/micro/aim7/misc_rtns_1
                 1818.70       +29.6%      2356.40  TOTAL time.involuntary_context_switches
\f
                               perf-stat.cpu-migrations

   140000 ++--------------------*-------------------------------------------+
          |                   ..      .                                     |
   130000 ++                 .         *.......                             |
          |                 .                  *......              .*..... |
          *......*....... ..                          *...        ..       .|
   120000 ++             *                                ..    ..          *
          |      O                                          . ..            |
   110000 ++                                                 *              |
          O                                           O                     O
   100000 ++                                   O                            |
          |                            O                     O       O      |
          |                                                                 |
    90000 ++                    O                                           |
          |                                                                 |
    80000 ++-------------O--------------------------------------------------+

\f
                               perf-stat.cpu-migrations

   220000 ++----------------------------------------------------------------+
          |                                   .*                            |
   210000 ++                                ..  +                           |
   200000 ++                              ..     +                  .*..... |
          |                             ..        +               ..       .|
   190000 ++                 ...*......*           +            ..          *
          *.....       ..*...                       +         ..            |
   180000 ++    .  ....                              +    ...*              |
          |      *.                                   *...                  |
   170000 ++                                                                |
   160000 ++                                                 O              |
          O                     O                                           |
   150000 ++     O                                    O              O      |
          |              O             O                                    O
   140000 ++-----------------------------------O----------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               124548.40       -20.5%     99076.50  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
               188828.50       -20.2%    150766.20  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
               313376.90       -20.3%    249842.70  TOTAL perf-stat.cpu-migrations
\f
                                  vmstat.system.in

   1055 ++------------------------------------------------------------------+
        |                    ..*..                                          |
   1050 ++               ....     ..         .*......*.......               |
        |           ...*.           ..    ...                *..            |
        *.......*...                    ..                      ..          |
   1045 ++                            *.                          ..      ..*
        |                                                             ....  |
   1040 ++                                    O                     *.      |
        |                                                    O              |
   1035 O+                                                                  O
        |       O      O                                                    |
        |                      O                     O                      |
   1030 ++                                                          O       |
        |                             O                                     |
   1025 ++------------------------------------------------------------------+

\f
                                  vmstat.system.in

   1450 ++------------------------------------------------------------------+
        |                                         ...*.......               |
   1400 *+......*......*.......*......*.......*...           *......*.......*
   1350 ++                                                                  |
        |                                                                   |
   1300 ++                                                                  |
   1250 ++                                                                  |
        |                                                                   |
   1200 ++                                                                  |
   1150 ++                                                                  |
        |                                                                   |
   1100 ++                                                                  |
   1050 ++                                                                  |
        O       O      O       O      O       O      O       O      O       O
   1000 ++------------------------------------------------------------------+

\f
                                  vmstat.system.in

   1080 ++------------------------------------------------------------------+
        |   ....*..            *...                                         |
   1075 *+..       .         ..    ..                       .*......*..     |
        |           .      ..        .                   ...           ..   |
   1070 ++           ..   .           *.......         ..                .. |
        |               ..                    *......*.                    .|
   1065 ++             *                                                    *
        |                                                                   |
   1060 ++                                                                  |
        |                                     O                             |
   1055 ++      O                                                           |
        |                             O                      O              O
   1050 ++                                                                  |
        O              O       O                     O              O       |
   1045 ++------------------------------------------------------------------+

\f
                                  vmstat.system.in

   1550 ++------------------------------------------------------------------+
   1500 *+......*......*.......*......*.......*......*.......*......*.......*
        |                                                                   |
   1450 ++                                                                  |
   1400 ++                                                                  |
   1350 ++                                                                  |
   1300 ++                                                                  |
        |                                                                   |
   1250 ++                                                                  |
   1200 ++                                                                  |
   1150 ++                                                                  |
   1100 ++                                                                  |
        |                                                                   |
   1050 O+      O      O       O      O       O      O       O      O       O
   1000 ++------------------------------------------------------------------+

\f
                                  vmstat.system.in

   1450 ++------------------------------------------------------------------+
        *.......*......    ....*......*.......*......    ....*......*.......|
   1400 ++             *...                          *...                   *
   1350 ++                                                                  |
        |                                                                   |
   1300 ++                                                                  |
   1250 ++                                                                  |
        |                                                                   |
   1200 ++                                                                  |
   1150 ++                                                                  |
        |                                                                   |
   1100 ++                                                                  |
   1050 ++                                                                  |
        O       O      O       O      O       O      O       O      O       O
   1000 ++------------------------------------------------------------------+

\f
                                  vmstat.system.in

   1090 ++-------------------------------------------*----------------------+
        |             .*.......*......*..          ..   ..                  |
   1080 ++          ..                   ..       .       ..                |
        |         ..                       ..    .          .    ...*.......*
        |   ....*.                           . ..            *...           |
   1070 *+..                                  *                             |
        |                                                                   |
   1060 O+      O                                                   O       |
        |              O              O       O                             O
   1050 ++                                           O                      |
        |                                                    O              |
        |                                                                   |
   1040 ++                                                                  |
        |                      O                                            |
   1030 ++------------------------------------------------------------------+

\f
                                  vmstat.system.in

   1500 ++------------------------------------*------*----------------------*
   1450 *+......*......*.......*......*...                   *......*...    |
        |                                                                   |
   1400 ++                                                                  |
   1350 ++                                                                  |
        |                                                                   |
   1300 ++                                                                  |
   1250 ++                                                                  |
   1200 ++                                                                  |
        |                                                                   |
   1150 ++                                                                  |
   1100 ++                                                                  |
        |                                                                   |
   1050 O+      O      O       O      O       O      O       O      O       O
   1000 ++------------------------------------------------------------------+

\f
                                  vmstat.system.in

   8800 ++------------------------------------------------------------------+
   8600 *+......*......*.......*......*.......*......*.......*......    ....*
        |                                                           *...    |
   8400 ++                                                                  |
   8200 ++                                                                  |
        |                                                                   |
   8000 ++                                                                  |
   7800 ++                                                                  |
   7600 ++                                                                  |
        |                                                                   |
   7400 O+      O      O                             O       O      O       |
   7200 ++                     O                                            |
        |                             O       O                             |
   7000 ++                                                                  O
   6800 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 1047.17        -1.3%      1033.30  lkp-a04/micro/netperf/120s-200%-TCP_CRR
                 1389.51       -25.2%      1038.67  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                 1071.90        -2.0%      1050.66  lkp-a04/micro/netperf/120s-200%-TCP_RR
                 1514.78       -31.4%      1038.68  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                 1408.60       -26.3%      1037.84  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                 1078.65        -2.3%      1053.82  lkp-a04/micro/netperf/120s-200%-UDP_RR
                 1473.83       -29.1%      1044.79  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                 8622.23       -16.0%      7242.96  nhm-white/micro/aim7/misc_rtns_1
                17606.67       -17.4%     14540.72  TOTAL vmstat.system.in
\f
                            time.voluntary_context_switches

   700000 ++----------------------------------------------------------------+
          |            ..*...                                               |
   680000 *+.....  ....      ..                                             |
          |      *.            .    ...*.......    ...*......*.......    ...*
   660000 ++                    *...           *...                  *...   |
          |                                                                 |
   640000 ++                                                                |
          |                                                                 |
   620000 ++                                                                |
          |                                                                 |
   600000 O+     O                                                          |
          |              O      O                            O       O      |
   580000 ++                                   O      O                     |
          |                            O                                    |
   560000 ++----------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
               669432.20       -12.7%    584367.90  nhm-white/micro/aim7/misc_rtns_1
               669432.20       -12.7%    584367.90  TOTAL time.voluntary_context_switches
\f
                                 iostat.cpu.system

     9 ++-------------------------------------------------------------------+
       |       O                             O               O              |
   8.5 O+                                                           O       O
     8 ++                                                                   |
       |                                             O                      |
   7.5 ++             O                                                     |
     7 ++                             O                                     |
       |                                                                    |
   6.5 ++                     O           ...*.......                       |
     6 ++    ..*......              ..*...           *..            *.......*
       | ....         *.......  ....                    ..        ..        |
   5.5 *+                     *.                          ..    ..          |
     5 ++                                                   . ..            |
       |                                                     *              |
   4.5 ++-------------------------------------------------------------------+

\f
                                  iostat.cpu.system

   94.9 ++------------------------------------------------------------------+
        |      .*.....                                                      |
   94.8 ++  ...       .    ....*.....             ...*.....        .*.......|
   94.7 ++..           *...          .    ....*...         ..  ....         *
        *.                            *...                   *.             |
   94.6 ++                                                                  |
   94.5 ++                                                                  |
        |                      O                                    O       |
   94.4 ++                                                                  |
   94.3 ++      O                                                           |
        |              O                                     O              O
   94.2 ++                            O              O                      |
   94.1 ++                                                                  |
        O                                     O                             |
     94 ++------------------------------------------------------------------+

\f
                                  iostat.cpu.system

   96.2 ++----------------------------*-------------------------------------+
        |                     .*...         ..                              |
   96.1 *+......           ...                *......*.......*......*.......|
        |       *......  ..    O                                            *
     96 ++             *.                                                   |
        |                                                                   |
   95.9 ++                                                                  O
        |                                                    O              |
   95.8 ++                                                                  |
        |                                                                   |
   95.7 ++                                                                  |
        |                                            O              O       |
   95.6 ++      O                     O       O                             |
        O              O                                                    |
   95.5 ++------------------------------------------------------------------+

\f
                                  iostat.cpu.system

   95.1 ++------------------------------------------------------------------+
     95 ++                                                         .*.....  |
        |                    ..*.                              ....       ..|
   94.9 ++     .*......  ....    .                           *.             *
   94.8 ++  ...        *.         ..         .*..          ..               |
   94.7 ++..                        .      ..    ..      ..                 |
   94.6 *+                           .   ..        ..   .                   |
        |                             O..             ..                    |
   94.5 ++                            *              *                      |
   94.4 ++                                                                  |
   94.3 ++      O                                    O                      |
   94.2 ++                                    O                     O       |
        O              O                                                    O
   94.1 ++                                                   O              |
     94 ++---------------------O--------------------------------------------+

\f
                                  iostat.cpu.system

   14.9 ++------------------------------------------------------------------+
        |                                    .*..                           |
   14.8 *+.                ....*.....     ...    ..                         |
        |  .           *...          .  ..         ..                       |
   14.7 ++  ..       ..               *.             O   ....*......*...    |
   14.6 ++    ..   ..                                *...               ... |
        |        ..                                          O             .|
   14.5 O+      *                                                   O       *
        |       O                                                           |
   14.4 ++                                                                  |
        |              O              O       O                             |
   14.3 ++                     O                                            |
   14.2 ++                                                                  |
        |                                                                   O
   14.1 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                    5.87       +35.3%         7.94  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                   94.71        -0.5%        94.26  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                   96.10        -0.4%        95.70  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                   94.77        -0.6%        94.22  lkp-a04/micro/netperf/120s-200%-UDP_RR
                   14.67        -1.8%        14.40  nhm-white/micro/aim7/misc_rtns_1
                  306.12        +0.1%       306.52  TOTAL iostat.cpu.system
\f
                                 time.elapsed_time

   80 ++--------------------------------------------------------------------+
      |                              O       O                              |
   79 ++                     O                                              O
   78 ++      O                                                             |
      O                                                                     |
   77 ++              O                              O      O       O       |
   76 ++                                                                    |
      |                                                                     |
   75 ++             .*..                                                   |
   74 ++           ..    .                                                  |
      |          ..       .                                                 |
   73 *+...... ..          ..                                   ....*.......|
   72 ++      *                  ....*.......    ....*......*...            *
      |                      *...            *...                           |
   71 ++--------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                   72.62        +7.1%        77.79  nhm-white/micro/aim7/misc_rtns_1
                   72.62        +7.1%        77.79  TOTAL time.elapsed_time
\f
                                   vmstat.cpu.sy

     9 ++-------------------------------------------------------------------+
       |       O                             O               O              |
   8.5 O+                                                           O       |
     8 ++                                                                   O
       |                                                                    |
   7.5 ++             O                              O                      |
     7 ++                             O                                     |
       |                                                                    |
   6.5 ++                     O           ...*.......                       |
     6 ++    ..*......              ..*...           *..            *.......*
       | ....         *.......  ....                    ..        ..        |
   5.5 *+                     *.                          ..    ..          |
     5 ++                                                   . ..            |
       |                                                     *              |
   4.5 ++-------------------------------------------------------------------+

\f
                                    vmstat.cpu.sy

   94.6 ++------*--------------*--------------------------------------------+
        | ....         *...        ..             ...*.......    ...*.......*
   94.5 *+                           .    ....*...           *...           |
        |                             *...                                  |
   94.4 ++                                                                  |
   94.3 ++                                                          O       |
        |                      O                                            |
   94.2 ++                                                                  |
        |       O      O                                                    |
   94.1 ++                                                   O              |
        |                             O                                     O
     94 ++                                                                  |
   93.9 ++                                    O      O                      |
        O                                                                   |
   93.8 ++------------------------------------------------------------------+

\f
                                    vmstat.cpu.sy

   95.2 ++------------------------------------------------------------------+
        |                                                                   |
     95 ++                                    *                             |
        |                                   .. :                            |
        |                                 ..    :                ...*.......*
   94.8 ++             *.                .       :           *...           |
        |            ..O .             ..         :        ..               |
   94.6 *+....     ..     .           *           :      ..                 |
        |     .. ..        ..       ..             :    .           O       |
   94.4 ++      *            .     .  O             : ..                    |
        O                     .   .           O      *       O              |
        |       O              O..                                          |
   94.2 ++                     *                     O                      |
        |                                                                   |
     94 ++------------------------------------------------------------------O

\f
                                    vmstat.cpu.sy

     96 ++---------------------*------*----------------------*--------------+
        *.......           ...              ..         ....         *.......|
   95.9 ++      *......  ..                   *......*.                     *
        |              *.                                                   |
   95.8 ++                     O                                            |
        |                                                                   O
   95.7 ++                                                                  |
        |                                                    O              |
   95.6 ++                                                                  |
        |                                                                   |
   95.5 ++                            O              O              O       |
        |       O                                                           |
   95.4 ++                                    O                             |
        O              O                                                    |
   95.3 ++------------------------------------------------------------------+

\f
                                    vmstat.cpu.sy

     95 ++------------------------------------------------------------------+
        |                                                          .*.....  |
   94.8 ++                   ..*                                 ..       ..|
        |                ....   +                              ..           *
        |     ..*......*.        +                          .*.             |
   94.6 ++....                    +          .*...        ..                |
        *.                         +       ..     ..    ..                  |
   94.4 ++                          +    ..         . ..                    |
        |                            +O..            *                      |
   94.2 ++                            *                                     |
        |       O                                    O                      |
        |                                     O                     O       O
     94 O+             O                                                    |
        |                      O                             O              |
   93.8 ++------------------------------------------------------------------+

\f
                                    vmstat.cpu.sy

   14.9 ++------------------------------------------------------------------+
        |                                    .*..                           |
   14.8 *+             *.......           ...    ..                         |
        | ..          .        *......  ..         ..O     ..*......*...    |
   14.7 ++  .        .                *.               ....             ... |
   14.6 ++   .     ..                                *.                    .|
        |     ..  .                                          O      O       *
   14.5 O+       .                                                          |
        |       *                                                           |
   14.4 ++      O                             O                             |
        |              O              O                                     |
   14.3 ++                                                                  |
   14.2 ++                     O                                            |
        |                                                                   O
   14.1 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                    5.87       +35.0%         7.93  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                   94.52        -0.5%        94.07  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                   94.66        -0.3%        94.34  lkp-a04/micro/netperf/120s-200%-TCP_RR
                   95.93        -0.4%        95.52  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                   94.61        -0.6%        94.04  lkp-a04/micro/netperf/120s-200%-UDP_RR
                   14.69        -1.9%        14.41  nhm-white/micro/aim7/misc_rtns_1
                  400.28        +0.0%       400.32  TOTAL vmstat.cpu.sy
\f
                         time.percent_of_cpu_this_job_got

   190 ++-------------------------------------------------------------------+
   188 *+......*......    ....*.......*......*.......*.......*.....       ..*
       |              *...                                         .  ....  |
   186 ++                                                           *.      |
   184 ++                                                                   |
       |                                                                    |
   182 ++                                                                   |
   180 ++                                                                   |
   178 O+                                                           O       |
       |       O      O                              O       O              |
   176 ++                                                                   |
   174 ++                                    O                              |
       |                      O                                             |
   172 ++                             O                                     |
   170 ++-------------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  188.10        -6.8%       175.40  nhm-white/micro/aim7/misc_rtns_1
                  188.10        -6.8%       175.40  TOTAL time.percent_of_cpu_this_job_got
\f
                                  iostat.sda.rkB/s

    1.4 ++------------------------------------------------------------------+
        |   ....*......    ....*......    ....*......*.......               |
   1.35 *+..           *...           *...                   *......*.......*
    1.3 O+      O                                            O              |
        |              O                                            O       O
   1.25 ++                     O      O       O                             |
    1.2 ++                                                                  |
        |                                                                   |
   1.15 ++                                                                  |
    1.1 ++                                                                  |
        |                                                                   |
   1.05 ++                                                                  |
      1 ++                                                                  |
        |                                                                   |
   0.95 ++-------------------------------------------O----------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                    1.35        -7.9%         1.24  nhm-white/micro/aim7/misc_rtns_1
                    1.35        -7.9%         1.24  TOTAL iostat.sda.rkB/s
\f
                                    vmstat.io.bi

   0.36 ++------------------------------------------------------------------+
        |                    ..*......    ....*......                       |
   0.34 *+......*......  ....         *...           *.......*......*.......*
        O       O      *.                                                   |
        |              O                                            O       |
   0.32 ++                     O      O       O                             O
        |                                                                   |
    0.3 ++                                                                  |
        |                                                                   |
   0.28 ++                                                                  |
        |                                                                   |
        |                                                                   |
   0.26 ++                                                                  |
        |                                                                   |
   0.24 ++-------------------------------------------O----------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                    0.35        -7.9%         0.32  nhm-white/micro/aim7/misc_rtns_1
                    0.35        -7.9%         0.32  TOTAL vmstat.io.bi
\f
                                   vmstat.cpu.wa

   94 ++--------------------------------------------------------------------+
      |                                                                     |
   93 ++                                                   .*...            |
      |                                                ....     ...         |
   92 *+......    ....*......*.....              ....*.            .    ....*
      |       *...                 ..    ....*...                   *...    |
   91 ++                     O       *...                                   |
      |                                                                     |
   90 ++              O              O                                      |
      |                                                                     |
   89 ++                                                                    O
      |                                              O      O       O       |
   88 ++                                     O                              |
      O       O                                                             |
   87 ++--------------------------------------------------------------------+

\f
                                   vmstat.cpu.wa

   90 ++--------------------------------------------------------------------+
   88 ++  ....*.......                             ..*.....                 |
      *...            *......*.......          ....        .            ....*
   86 ++                             *.......*.      O      *.......*...    |
   84 ++      O       O      O                                      O       |
   82 O+                                                    O               |
   80 ++                                                                    |
      |                                                                     |
   78 ++                                                                    |
   76 ++                                                                    |
   74 ++                                                                    |
   72 ++                                                                    |
      |                                      O                              |
   70 ++                                                                    |
   68 ++--------------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                   91.82        -3.3%        88.82  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                   86.38        -6.1%        81.08  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                  178.19        -4.7%       169.90  TOTAL vmstat.cpu.wa
\f
                                 iostat.cpu.iowait

   93 ++----------------------------------------------------*---------------+
      |                                                ....    ..           |
   92 *+....              ...*.....                ..*.          ..         |
      |     ..    ....*...         ..          ....                .    ....*
      |       *...                   *.......*.                     *...    |
   91 ++                     O                                              |
      |                                                                     |
   90 ++              O              O                                      |
      |                                                                     |
   89 ++                                                                    |
      |                                              O                      O
      |                                      O              O       O       |
   88 ++                                                                    |
      O       O                                                             |
   87 ++--------------------------------------------------------------------+

\f
                                 iostat.cpu.iowait

   90 ++--------------------------------------------------------------------+
   88 ++  ....*.......                                                      |
      *...            *......*.......            ....*......            ....*
   86 ++                             *.......*...    O      *.......*...    |
   84 ++      O       O      O                                      O       |
   82 O+                                                    O               |
   80 ++                                                                    |
      |                                                                     |
   78 ++                                                                    |
   76 ++                                                                    |
   74 ++                                                                    |
   72 ++                                                                    |
      |                                      O                              |
   70 ++                                                                    |
   68 ++--------------------------------------------------------------------O

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                   91.79        -3.2%        88.82  kbuildx/micro/dd-write/4HDD-RAID0-cfq-xfs-100dd
                   86.29        -6.0%        81.09  kbuildx/micro/dd-write/4HDD-RAID5-deadline-ext4-100dd
                  178.08        -4.6%       169.91  TOTAL iostat.cpu.iowait
\f
                                    boottime.dhcp

     48 ++------------------------------------------------------------------+
        |              O                                                    |
   47.5 ++                                                                  |
     47 ++                                                                  |
        |                                                                   |
   46.5 ++                                                                  |
        |                                                                   |
     46 ++                                                                  |
        |                             O                             O       *
   45.5 O+      O              O                     O       O            ..O
     45 ++                                    O                         ..  |
        |                  ....*......                                 .    |
   44.5 *+......*......*...           *.......*......                ..     |
        |                                            *.......*......*       |
     44 ++------------------------------------------------------------------+

\f
                                    boottime.dhcp

     49 ++------------------------------------------------------------------+
   48.5 ++                                    O                             |
        |                                                                   |
     48 ++                                                                  |
   47.5 ++                                                                  |
        |                                                                   |
     47 ++                                                                  |
   46.5 ++                                                                  |
     46 ++                                                                  |
        |                             O              O       O      O       |
   45.5 O+                                                                  O
     45 ++      O          ....*.....             ...*.......               |
        *.......*......*...          .    ....*...           *......*.......*
   44.5 ++                            *...                                  |
     44 ++------------------------------------------------------------------+

\f
                                    boottime.dhcp

     49 ++-------------------------------------------O----------------------+
   48.5 ++                                                                  |
        |                                                                   |
     48 ++                                                                  |
   47.5 ++                                                                  |
        |                                                                   |
     47 ++                                                                  |
   46.5 ++                                                                  |
     46 ++                                                          O       |
        |                      O                             O              |
   45.5 ++      O      O              O       O                             O
     45 O+                                                                  |
        |           ...*.......*......*.......                          ....*
   44.5 *+......*...                          *......    ....*......*...    |
     44 ++-------------------------------------------*----------------------+

\f
                                    boottime.dhcp

   46.5 ++------------------------------------------------------------------+
        |                                                           *       |
     46 ++      O                                                  : :      |
        |                      O                                   : :      |
        |              O              O                           : O :     |
   45.5 O+                                    O      O       O   :     :    O
        |                                                        :      :   |
     45 ++                                                      :       :   |
        |      .*......                           ...*...      :         :  |
   44.5 ++  ...        *.......*......*.......*...       ...  :           : |
        | ..                                                . :            :|
        *.                                                   *             :|
     44 ++                                                                  *
        |                                                                   |
   43.5 ++------------------------------------------------------------------+

\f
                                    boottime.dhcp

     46 ++------------------------------------------------------------------+
   45.8 O+                                                          O       |
        |                                                                   |
   45.6 ++             O                      O                             |
   45.4 ++                                                   O              O
        |       O              O                                            |
   45.2 ++                                                                  |
     45 ++                            O                                     |
   44.8 ++                                                                  |
        |                                                                  .*
   44.6 ++            .*..                           O                  ... |
   44.4 ++          ..    ..         .*.......*......*.......         ..    |
        *.......  ..        ..     ..                        *......*.      |
   44.2 ++      *.            .  ..                                         |
     44 ++---------------------*--------------------------------------------+

\f
                                    boottime.dhcp

   46.5 ++-------------------------------------------O----------------------+
        |                                                                   |
     46 O+                            O       O                     O       |
        |                                                                   O
        |       O                                                           |
   45.5 ++                                                   O              |
        |              O       O                                            |
     45 ++                                                                  |
        |                    ..*......*.....             ....*..           .*
   44.5 *+......         ....               ..    ...*...       ..       .. |
        |       *......*.                     *...                ..   ..   |
        |                                                            ..     |
     44 ++                                                          *       |
        |                                                                   |
   43.5 ++------------------------------------------------------------------+

\f
                                    boottime.dhcp

     48 ++---------------------O--------------------------------------------+
        |                                                                   |
   47.5 ++                                                                  |
     47 ++                                                                  |
        |                                                                   |
   46.5 ++                                                                  |
     46 ++                                                                  |
        |                             O                             O       O
   45.5 ++                                    O              O              |
     45 O+      O     .*...                                                 |
        |           ..     ...                    ...*.......               |
   44.5 *+......  ..          .    ...*.......*...           *......        |
     44 ++      *.             *...                                 *.......|
        |                                                                   *
   43.5 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                   44.62        +2.4%        45.71  lkp-a04/micro/netperf/120s-200%-TCP_CRR
                   44.68        +2.5%        45.80  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                   44.47        +3.4%        45.96  lkp-a04/micro/netperf/120s-200%-TCP_RR
                   44.66        +2.1%        45.61  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                   44.38        +2.3%        45.39  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                   44.57        +2.8%        45.83  lkp-a04/micro/netperf/120s-200%-UDP_RR
                   44.37        +2.8%        45.62  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                  311.74        +2.6%       319.92  TOTAL boottime.dhcp
\f
                                    boottime.boot

     71 ++-------------O----------------------------------------------------+
        |                                                                   |
   70.5 ++                                                                  |
     70 ++                                                                  |
        |                                                                   |
   69.5 ++                                                                 .*
        |       O                                                        .. |
     69 O+                     O                     O       O         ..   |
        |                  ....*...   O       O                     O..     O
   68.5 ++            .*...        ..                              .*       |
     68 *+......  ....               .                         ....         |
        |       *.                    *.......*...          .*.             |
   67.5 ++                                        ..     ...                |
        |                                           .  ..                   |
     67 ++-------------------------------------------*----------------------+

\f
                                   boottime.boot

   73 ++--------------------------------------------------------------------+
      |                                      O                              |
   72 ++                                                                    |
   71 ++                                                                    |
      |                                                                     |
   70 ++                                             O              O       |
   69 O+      O                      O                      O               O
      *..             O      O                                              |
   68 ++ .           .*...                          .*......*.......*..     |
   67 ++  ..       ..     ..                      ..                   .    |
      |     ..   ..         .                   ..                      ..  |
   66 ++       ..            *.....           ..                          ..|
   65 ++      *                    ..    ....*                              |
      |                              *...                                   *
   64 ++--------------------------------------------------------------------+

\f
                                   boottime.boot

   70 ++-------------------------------------O------------------------------+
      O                                                                     O
   69 ++              O                                             O       |
      |       O      .*..                        ....*                      |
      |           ...    ..  O       O   ....*...    O:                     |
   68 ++        ..         ..      ..*...              :                    |
      *.......*.               ....                    :                    |
   67 ++                     *.                         :                   |
      |                                                  :                  |
   66 ++                                                  : O               |
      |                                                   :                 |
      |                                                    :            ....*
   65 ++                                                    *.......*...    |
      |                                                                     |
   64 ++--------------------------------------------------------------------+

\f
                                   boottime.boot

   72 ++--------------------------------------------------------------------+
      |                      O                                              |
   71 ++                                                                    |
   70 ++                                                                    |
      |                                                                     O
   69 ++      O                      O       O       O                      |
      O             ..*.....                       ..*.     O       O       |
   68 *+......  ....        .    ....*.......  ....    .                    |
      |       *.             *...            *.         ..         .*.......|
   67 ++                                                  .      ..         *
   66 ++                                                   .   ..           |
      |                                                      ..             |
   65 ++                                                    *               |
      |                                                                     |
   64 ++--------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                   68.13        +1.5%        69.14  lkp-a04/micro/netperf/120s-200%-TCP_CRR
                   66.58        +4.2%        69.36  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                   67.07        +2.4%        68.66  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                   67.44        +2.6%        69.20  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                  269.22        +2.7%       276.37  TOTAL boottime.boot
\f
                                   boottime.idle

   204 ++-------------------------------------------------------------------+
       |                                                                    |
   202 ++             O                                                     |
   200 ++                                                                   |
       |                                                                    |
   198 ++                                                                   |
       |                                                                   .*
   196 ++                                            O                   .. |
       |       O                             O                         ..   |
   194 O+                     O       O                      O       ..     |
   192 *+....        .*.......*.......*......                    ...*       O
       |     ..  ....                        *.....        ..*...           |
   190 ++      *.                                  ..  ....                 |
       |                                             *.                     |
   188 ++-------------------------------------------------------------------+

\f
                                   boottime.idle

   210 ++-------------------------------------------------------------------+
       |                                     O                              |
   205 ++                                                                   |
       |                                                                    |
       |                                                                    |
   200 ++                                                                   |
       O                                                            O       |
   195 ++      O                      O              O       O              O
       *..            O       O                      *.......    ...*.      |
   190 ++ .           *..                          ..        *...     ..    |
       |   ..       ..   ..                       .                     .   |
       |     ..   ..       ..                    .                       .  |
   185 ++       ..           .                 ..                         ..|
       |       *              *.......        .                             |
   180 ++-----------------------------*------*------------------------------*

\f
                                   boottime.idle

   205 ++-------------------------------------------------------------------+
       |                      O                                             |
   200 ++                                                                   |
       |                                                                    |
       |       O                             O                              O
   195 ++                             O            ..*.      O      O       |
       *.....        .*.......*.......         ....    .                    |
   190 ++    ..  ....                 *......*.         .           *.......|
       |       *.                                        ..       ..        *
   185 ++                                                  .     .          |
       |                                                    .   .           |
       |                                                      ..            |
   180 ++                                                    *              |
       |                                                                    |
   175 ++-------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                  192.32        +1.3%       194.91  lkp-a04/micro/netperf/120s-200%-TCP_CRR
                  186.43        +5.2%       196.06  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                  189.71        +2.9%       195.22  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                  568.45        +3.1%       586.18  TOTAL boottime.idle
\f
                               time.minor_page_faults

   2725 ++------------------------------------------------------------------+
        |                                                                   |
   2720 ++                                    O                             |
        |              O                                     O              |
   2715 O+                     O                                    O       |
        |                                                                   |
   2710 ++      O                                    O                      O
        |      .*......               O                                     |
   2705 ++   ..        *.......*..                                          |
        |  ..                     .                 .*...                   |
   2700 ++.                        .            ....     ...                |
        *                           ..      ..*.            .    ...*.......|
   2695 ++                              ....                 *...           *
        |                             *.                                    |
   2690 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                 2699.20        +0.6%      2714.10  lkp-a04/micro/netperf/120s-200%-UDP_RR
                 2699.20        +0.6%      2714.10  TOTAL time.minor_page_faults
\f
                                   iostat.cpu.idle

     83 ++------------------------------------------------------------------+
   82.8 ++                                                                  O
        |                      O      O       O                             |
   82.6 ++      O      O                                     O      O       |
   82.4 ++                                           O                    ..*
   82.2 *+......*......                             .*.......         ....  |
     82 ++             *.......*......*.......  ....         *......*.      |
        |                                     *.                            |
   81.8 ++                                                                  |
   81.6 ++                                                                  |
   81.4 ++                                                                  |
   81.2 ++                                                                  |
        |                                                                   |
     81 ++                                                                  |
   80.8 O+------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                   82.14        +0.3%        82.40  nhm-white/micro/aim7/misc_rtns_1
                   82.14        +0.3%        82.40  TOTAL iostat.cpu.idle
\f
                                    vmstat.cpu.id

     83 ++------------------------------------------------------------------+
        |                      O                                            O
        |                             O       O                             |
   82.5 ++      O      O                                     O      O       |
        |   ....*......                             .*.....             ....*
        *...           *.......*......*.......  ....       ..    ...*...    |
     82 ++                                    *.             *...           |
        |                                                                   |
   81.5 ++                                                                  |
        |                                                                   |
        |                                                                   |
     81 ++                                                                  |
        O                                                                   |
        |                                                                   |
   80.5 ++------------------------------------------------------------------+

    44c1ff31853685ef76ca      126d9c20bc99ccf1d625  
------------------------  ------------------------  
                   82.12        +0.3%        82.39  nhm-white/micro/aim7/misc_rtns_1
                   82.12        +0.3%        82.39  TOTAL vmstat.cpu.id

^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2013-08-30 10:07 UTC | newest]

Thread overview: (only message) (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <20130828140240.GA28272@localhost>
2013-08-30 10:07 ` [another NUMA code dump] -3.9% netperf throughput drop Fengguang Wu

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.