From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: lianglihao@huawei.com
Cc: guohanjun@huawei.com, heng.z@huawei.com, hb.chen@huawei.com,
lihao.liang@gmail.com, linux-kernel@vger.kernel.org
Subject: Re: [PATCH RFC 00/16] A new RCU implementation based on a fast consensus protocol
Date: Wed, 24 Jan 2018 21:53:56 -0800 [thread overview]
Message-ID: <20180125055356.GT3741@linux.vnet.ibm.com> (raw)
In-Reply-To: <1516694381-20333-1-git-send-email-lianglihao@huawei.com>
On Tue, Jan 23, 2018 at 03:59:25PM +0800, lianglihao@huawei.com wrote:
> From: Lihao Liang <lianglihao@huawei.com>
>
> Dear Paul,
>
> This patch set implements a preemptive version of RCU (PRCU) based on the following paper:
>
> Fast Consensus Using Bounded Staleness for Scalable Read-mostly Synchronization.
> Haibo Chen, Heng Zhang, Ran Liu, Binyu Zang, and Haibing Guan.
> IEEE Transactions on Parallel and Distributed Systems (TPDS), 2016.
> https://dl.acm.org/citation.cfm?id=3024114.3024143
>
> We have also added preliminary callback-handling support. Thus, the current version
> provides APIs prcu_read_lock(), prcu_read_unlock(), synchronize_prcu(), call_prcu(),
> and prcu_barrier().
>
> This is an experimental patch, so it would be good to have some feedback.
>
> Known shortcoming is that the grace-period version is incremented in synchronize_prcu().
> If call_prcu() or prcu_barrier() is called but there is no synchronized_prcu() invoked,
> callbacks cannot be invoked. Later version should address this issue, e.g. adding a
> grace-period expedition mechanism. Others include to use a a hierarchical structure,
> taking into account the NUMA topology, to send IPI in synchronize_prcu().
>
> We have tested the implementation using rcutorture on both an x86 and ARM64 machine.
> PRCU passed 1h and 3h tests on all the newly added config files except PRCU07 reported BUG
> in a 1h run.
>
> [ 1593.604201] ---[ end trace b3bae911bec86152 ]---
> [ 1594.629450] prcu-torture:torture_onoff task: offlining 14
> [ 1594.755553] smpboot: CPU 14 is now offline
> [ 1594.757732] prcu-torture:torture_onoff task: offlined 14
> [ 1597.765149] prcu-torture:torture_onoff task: onlining 11
> [ 1597.766795] smpboot: Booting Node 0 Processor 11 APIC 0xb
> [ 1597.804102] prcu-torture:torture_onoff task: onlined 11
> [ 1599.365098] prcu-torture: rtc: ffffffffb0277b90 ver: 66358 tfle: 0 rta: 66358 rtaf: 0
> rtf: 66349 rtmbe: 0 rtbe: 1 rtbke: 0 rtbre: 0 rtbf: 0 rtb: 0 nt: 2233418
> onoff: 191/191:199/199 34,199:59,5102 10403:0 (HZ=1000) barrier: 188/189:1 cbflood: 225
> [ 1599.367946] prcu-torture: !!!
> [ 1599.367966] ------------[ cut here ]------------
The "rtbe: 1" indicates that your implementation of prcu_barrier()
failed to wait for all preceding call_prcu() callbacks to be invoked.
Does the immediately following "Reader Pipe:" list have any but the
first two numbers non-zero?
> We have also compared PRCU with TREE RCU using rcuperf with gp_exp set to true, that is
> synchronize_rcu_expedited was tested.
>
> The rcuperf results are as follows (average grace-period duration in ms of ten 10min runs):
>
> 16*Intel Xeon CPU@2.4GHz, 16GB memory, Ubuntu Linux 3.13.0-47-generic
>
> CPUs 2 4 8 12 15 16
> PRCU 0.14 1.07 4.15 8.02 10.79 15.16
> TREE 49.30 104.75 277.55 390.82 620.82 1381.54
>
> 64*Cortex-A72 CPU@2.4GHz, 130GB memory, Ubuntu Linux 4.10.0-21.23-generic
>
> CPUs 2 4 8 16 32 48 63 64
> PRCU 0.23 19.69 38.28 63.21 95.41 167.18 252.01 1841.44
> TREE 416.73 901.89 1060.86 743.00 920.66 1325.21 1646.20 23806.27
Well, at the very least, this is a bug report on either expedited RCU
grace-period latency or on rcuperf's measurements, and thank you for that.
I will look into this. In the meantime, could you please let me know
exactly how you invoked rcuperf?
I have a few comments on some of your patches based on a quick scan
through them.
Thanx, Paul
> Best wishes,
> Lihao.
>
>
> Lihao Liang (15):
> rcutorture: Add PRCU rcu_torture_ops
> rcutorture: Add PRCU test config files
> rcuperf: Add PRCU rcu_perf_ops
> rcuperf: Add PRCU test config files
> rcuperf: Set gp_exp to true for tests to run
> prcu: Implement call_prcu() API
> prcu: Implement PRCU callback processing
> prcu: Implement prcu_barrier() API
> rcutorture: Test call_prcu() and prcu_barrier()
> rcutorture: Add basic ARM64 support to run scripts
> prcu: Add PRCU Kconfig parameter
> prcu: Comment source code
> rcuperf: Add config files with various CONFIG_NR_CPUS
> rcutorture: Add scripts to run experiments
> Add GPLv2 license
>
> Heng Zhang (1):
> prcu: Add PRCU implementation
>
> include/linux/interrupt.h | 3 +
> include/linux/prcu.h | 122 +++++
> include/linux/rcupdate.h | 1 +
> init/Kconfig | 7 +
> init/main.c | 2 +
> kernel/rcu/Makefile | 1 +
> kernel/rcu/prcu.c | 497 +++++++++++++++++++++
> kernel/rcu/rcuperf.c | 33 +-
> kernel/rcu/rcutorture.c | 40 +-
> kernel/rcu/tree.c | 1 +
> kernel/sched/core.c | 2 +
> kernel/time/timer.c | 2 +
> kvm.sh | 452 +++++++++++++++++++
> run-rcuperf.sh | 26 ++
> .../testing/selftests/rcutorture/bin/functions.sh | 17 +-
> .../selftests/rcutorture/configs/rcu/CFLIST | 5 +
> .../selftests/rcutorture/configs/rcu/PRCU02 | 27 ++
> .../selftests/rcutorture/configs/rcu/PRCU02.boot | 1 +
> .../selftests/rcutorture/configs/rcu/PRCU03 | 23 +
> .../selftests/rcutorture/configs/rcu/PRCU03.boot | 2 +
> .../selftests/rcutorture/configs/rcu/PRCU06 | 26 ++
> .../selftests/rcutorture/configs/rcu/PRCU06.boot | 5 +
> .../selftests/rcutorture/configs/rcu/PRCU07 | 25 ++
> .../selftests/rcutorture/configs/rcu/PRCU07.boot | 2 +
> .../selftests/rcutorture/configs/rcu/PRCU09 | 19 +
> .../selftests/rcutorture/configs/rcu/PRCU09.boot | 1 +
> .../selftests/rcutorture/configs/rcuperf/CFLIST | 1 +
> .../selftests/rcutorture/configs/rcuperf/PRCU | 20 +
> .../selftests/rcutorture/configs/rcuperf/PRCU-12 | 21 +
> .../rcutorture/configs/rcuperf/PRCU-12.boot | 1 +
> .../selftests/rcutorture/configs/rcuperf/PRCU-14 | 21 +
> .../rcutorture/configs/rcuperf/PRCU-14.boot | 1 +
> .../selftests/rcutorture/configs/rcuperf/PRCU-15 | 21 +
> .../rcutorture/configs/rcuperf/PRCU-15.boot | 1 +
> .../selftests/rcutorture/configs/rcuperf/PRCU-16 | 21 +
> .../rcutorture/configs/rcuperf/PRCU-16.boot | 1 +
> .../selftests/rcutorture/configs/rcuperf/PRCU-2 | 21 +
> .../rcutorture/configs/rcuperf/PRCU-2.boot | 1 +
> .../selftests/rcutorture/configs/rcuperf/PRCU-32 | 21 +
> .../rcutorture/configs/rcuperf/PRCU-32.boot | 1 +
> .../selftests/rcutorture/configs/rcuperf/PRCU-4 | 21 +
> .../rcutorture/configs/rcuperf/PRCU-4.boot | 1 +
> .../selftests/rcutorture/configs/rcuperf/PRCU-48 | 21 +
> .../rcutorture/configs/rcuperf/PRCU-48.boot | 1 +
> .../selftests/rcutorture/configs/rcuperf/PRCU-56 | 21 +
> .../rcutorture/configs/rcuperf/PRCU-56.boot | 1 +
> .../selftests/rcutorture/configs/rcuperf/PRCU-60 | 21 +
> .../rcutorture/configs/rcuperf/PRCU-60.boot | 1 +
> .../selftests/rcutorture/configs/rcuperf/PRCU-62 | 21 +
> .../rcutorture/configs/rcuperf/PRCU-62.boot | 1 +
> .../selftests/rcutorture/configs/rcuperf/PRCU-64 | 21 +
> .../rcutorture/configs/rcuperf/PRCU-64.boot | 1 +
> .../selftests/rcutorture/configs/rcuperf/PRCU-8 | 21 +
> .../rcutorture/configs/rcuperf/PRCU-8.boot | 1 +
> .../selftests/rcutorture/configs/rcuperf/PRCU.boot | 1 +
> .../selftests/rcutorture/configs/rcuperf/TREE-12 | 21 +
> .../selftests/rcutorture/configs/rcuperf/TREE-14 | 21 +
> .../selftests/rcutorture/configs/rcuperf/TREE-15 | 21 +
> .../selftests/rcutorture/configs/rcuperf/TREE-16 | 21 +
> .../selftests/rcutorture/configs/rcuperf/TREE-2 | 21 +
> .../selftests/rcutorture/configs/rcuperf/TREE-32 | 21 +
> .../selftests/rcutorture/configs/rcuperf/TREE-4 | 21 +
> .../selftests/rcutorture/configs/rcuperf/TREE-48 | 21 +
> .../selftests/rcutorture/configs/rcuperf/TREE-56 | 21 +
> .../selftests/rcutorture/configs/rcuperf/TREE-60 | 21 +
> .../selftests/rcutorture/configs/rcuperf/TREE-62 | 21 +
> .../selftests/rcutorture/configs/rcuperf/TREE-64 | 21 +
> .../selftests/rcutorture/configs/rcuperf/TREE-8 | 21 +
> 68 files changed, 1918 insertions(+), 5 deletions(-)
> create mode 100644 include/linux/prcu.h
> create mode 100644 kernel/rcu/prcu.c
> create mode 100755 kvm.sh
> create mode 100755 run-rcuperf.sh
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU02
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU02.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU03
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU03.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU06
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU06.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU07
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU07.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU09
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU09.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-12
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-12.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-14
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-14.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-15
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-15.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-16
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-16.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-2
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-2.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-32
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-32.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-4
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-4.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-48
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-48.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-56
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-56.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-60
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-60.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-62
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-62.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-64
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-64.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-8
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-8.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU.boot
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-12
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-14
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-15
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-16
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-2
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-32
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-4
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-48
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-56
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-60
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-62
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-64
> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-8
>
> --
> 2.14.1.729.g59c0ea183
>
next prev parent reply other threads:[~2018-01-25 6:30 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-01-23 7:59 [PATCH RFC 00/16] A new RCU implementation based on a fast consensus protocol lianglihao
2018-01-23 7:59 ` [PATCH RFC 01/16] prcu: Add PRCU implementation lianglihao
2018-01-24 11:26 ` Peter Zijlstra
2018-01-24 17:15 ` Lihao Liang
2018-01-24 20:19 ` Peter Zijlstra
2018-01-25 6:16 ` Paul E. McKenney
2018-01-25 7:30 ` Boqun Feng
2018-01-30 5:34 ` zhangheng (AC)
2018-01-30 6:40 ` Boqun Feng
2018-01-30 10:42 ` zhangheng (AC)
2018-01-27 7:35 ` Lihao Liang
2018-01-30 3:58 ` zhangheng (AC)
2018-01-29 9:10 ` Lai Jiangshan
2018-01-30 6:21 ` zhangheng (AC)
2018-01-23 7:59 ` [PATCH RFC 02/16] rcutorture: Add PRCU rcu_torture_ops lianglihao
2018-01-23 7:59 ` [PATCH RFC 03/16] rcutorture: Add PRCU test config files lianglihao
2018-01-25 6:27 ` Paul E. McKenney
2018-01-23 7:59 ` [PATCH RFC 04/16] rcuperf: Add PRCU rcu_perf_ops lianglihao
2018-01-23 7:59 ` [PATCH RFC 05/16] rcuperf: Add PRCU test config files lianglihao
2018-01-23 7:59 ` [PATCH RFC 06/16] rcuperf: Set gp_exp to true for tests to run lianglihao
2018-01-25 6:18 ` Paul E. McKenney
2018-01-26 8:33 ` Lihao Liang
2018-01-23 7:59 ` [PATCH RFC 07/16] prcu: Implement call_prcu() API lianglihao
2018-01-25 6:20 ` Paul E. McKenney
2018-01-26 8:44 ` Lihao Liang
2018-01-26 22:22 ` Paul E. McKenney
2018-01-23 7:59 ` [PATCH RFC 08/16] prcu: Implement PRCU callback processing lianglihao
2018-01-23 7:59 ` [PATCH RFC 09/16] prcu: Implement prcu_barrier() API lianglihao
2018-01-25 6:24 ` Paul E. McKenney
2018-01-23 7:59 ` [PATCH RFC 10/16] rcutorture: Test call_prcu() and prcu_barrier() lianglihao
2018-01-23 7:59 ` [PATCH RFC 11/16] rcutorture: Add basic ARM64 support to run scripts lianglihao
2018-01-23 7:59 ` [PATCH RFC 12/16] prcu: Add PRCU Kconfig parameter lianglihao
2018-01-23 7:59 ` [PATCH RFC 13/16] prcu: Comment source code lianglihao
2018-01-23 7:59 ` [PATCH RFC 14/16] rcuperf: Add config files with various CONFIG_NR_CPUS lianglihao
2018-01-23 7:59 ` [PATCH RFC 15/16] rcutorture: Add scripts to run experiments lianglihao
2018-01-25 6:28 ` Paul E. McKenney
2018-01-23 7:59 ` [PATCH RFC 16/16] Add GPLv2 license lianglihao
2018-01-25 5:53 ` Paul E. McKenney [this message]
2018-01-27 7:22 ` [PATCH RFC 00/16] A new RCU implementation based on a fast consensus protocol Lihao Liang
2018-01-27 7:57 ` Paul E. McKenney
2018-01-27 9:57 ` Lihao Liang
2018-01-27 23:46 ` Paul E. McKenney
2018-01-27 23:41 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180125055356.GT3741@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=guohanjun@huawei.com \
--cc=hb.chen@huawei.com \
--cc=heng.z@huawei.com \
--cc=lianglihao@huawei.com \
--cc=lihao.liang@gmail.com \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox