* [PATCH 00/10] Intel(R) Speed Select Technology
From: Srinivas Pandruvada @ 2019-06-26 22:38 UTC (permalink / raw)
To: dvhart, andy, andriy.shevchenko, corbet
Cc: rjw, alan, lenb, prarit, darcari, linux-doc, linux-kernel,
platform-driver-x86, Srinivas Pandruvada
Intel® Speed Select Technology (Intel® SST) — A powerful new collection of
features giving more granular control over CPU performance for optimized total
cost of ownership and performance. With Intel® SST, one server can be configured
for power and performance for variety of diverse workload requirements. In the
Linux submission code. we are using ISST to specify Intel® SST to avoid confusion
with existing use of SST for "Smart Sound Technology".
Refer to these links for overview of the technology released with some Intel® Xeon®
Scalable processor (5218N, 6230N, and 6252N):
https://www.intel.com/content/www/us/en/architecture-and-technology/speed-select-technology-article.html
https://builders.intel.com/docs/networkbuilders/intel-speed-select-technology-base-frequency-enhancing-performance.pdf
The next generation of Intel® Xeon® processors are adding more features to the
Intel® Speed Select Technology and allow dynamic configuration of these features
from OS-software level instead from BIOS. This submission is adding new features
and dynamic configuration capabilities .
Intel SST Features:
Intel® SST—Performance Profile (PP or perf-profile):
This feature allows one server to be configured for different workload requirements
instead of deploying different servers based on the workload requirement reducing total
cost of ownership. With a single server deployed, the same server can be reconfigured
dynamically to one of the supported profiles to suit the specific workload requirements.
This feature introduces a mechanism that allows multiple optimized performance profiles
per system via static and/or dynamic adjustment of TDP level and other performance
parameters.
Intel® SST—Core power (CP or core-power):
An Interface that allows user to define per core priority. This defines a mechanism
to distribute power among cores when there is a power constrained scenario. This defines
a class of service configuration. Each CPU core can be tied to a class of service and hence
an associated priority.
Intel® SST—Base Frequency (BF or base-freq):
The Intel® SST-BF feature lets user control and direct base frequency. If some critical
workload threads demand constant high guaranteed performance, then this feature can be
used to execute the thread at higher base frequency on specific set of CPUs.
Intel® SST—Turbo frequency (TF or turbo-freq):
Enables the ability to set different all core turbo ratio limits to cores based on the priority.
Using this features some cores can be configured to get higher turbo frequency by designating
them as high priority at the cost of lower or no turbo frequency on the low priority cores.
Implementation
The Intel® SST features are implemented in the firmware executing in the the power
management unit (we are calling PUNIT here for short). The mechanism to control these
features are specific to firmware implementation for Intel® Xeon® CPUs and are not architectural
features. The interface mechanism and semantics of the messages can change in future Xeon
CPUs. Hence there is minimal kernel implementation by offering direct communication
to PUNIT via set of IOCTLs. The actual messages which can be sent to PUNIT are specified
in the following document link:
https://github.com/intel/CommsPowerManagement/blob/master/intel_sst_os_interface/mailbox.md
The idea here is that user space software like cloud orchestration software based on their workload
requirement configure the system. There is a full featured "Intel Speed Select" utility
submitted to kernel power tools, which can be used to validate and exercise the features.
Types of PUNIT interfaces
There are two types of interfaces. One using Mail box communications, which is facilitated
by a PCI device or in some Xeon® CPUs using MSRs; and other using an MMIO interface, which is
used primarily for core prioritization. For hiding details a single character device is created
to handle IOCTLs. The following block diagram shows the implementation overview.
User User Space tool(intel-speed-select)/Cloud Orchestration software
IOCTLs
---------------------------------------character device------------------------------
kernel Common driver handling IOCTLs
Mail Box drivers(PCI & MSR) PCI MMIO driver
--------------------------------------------------------------------------
Hardware PUNIT
Srinivas Pandruvada (10):
platform/x86: ISST: Update ioctl-number.txt for Intel Speed Select
interface
platform/x86: ISST: Add common API to register and handle ioctls
platform/x86: ISST: Store per CPU information
platform/x86: ISST: Add IOCTL to Translate Linux logical CPU to PUNIT
CPU number
platform/x86: ISST: Add Intel Speed Select mmio interface
platform/x86: ISST: Add Intel Speed Select mailbox interface via PCI
platform/x86: ISST: Add Intel Speed Select mailbox interface via MSRs
platform/x86: ISST: Add Intel Speed Select PUNIT MSR interface
platform/x86: ISST: Restore state on resume
tools/power/x86: A tool to validate Intel Speed Select commands
Documentation/ioctl/ioctl-number.txt | 1 +
drivers/platform/x86/Kconfig | 2 +
drivers/platform/x86/Makefile | 1 +
.../x86/intel_speed_select_if/Kconfig | 17 +
.../x86/intel_speed_select_if/Makefile | 10 +
.../intel_speed_select_if/isst_if_common.c | 672 +++++++
.../intel_speed_select_if/isst_if_common.h | 69 +
.../intel_speed_select_if/isst_if_mbox_msr.c | 216 +++
.../intel_speed_select_if/isst_if_mbox_pci.c | 214 +++
.../x86/intel_speed_select_if/isst_if_mmio.c | 180 ++
include/uapi/linux/isst_if.h | 172 ++
tools/power/x86/intel_speed_select/Makefile | 31 +
tools/power/x86/intel_speed_select/isst.h | 231 +++
.../x86/intel_speed_select/isst_config.c | 1607 +++++++++++++++++
.../power/x86/intel_speed_select/isst_core.c | 721 ++++++++
.../x86/intel_speed_select/isst_display.c | 479 +++++
16 files changed, 4623 insertions(+)
create mode 100644 drivers/platform/x86/intel_speed_select_if/Kconfig
create mode 100644 drivers/platform/x86/intel_speed_select_if/Makefile
create mode 100644 drivers/platform/x86/intel_speed_select_if/isst_if_common.c
create mode 100644 drivers/platform/x86/intel_speed_select_if/isst_if_common.h
create mode 100644 drivers/platform/x86/intel_speed_select_if/isst_if_mbox_msr.c
create mode 100644 drivers/platform/x86/intel_speed_select_if/isst_if_mbox_pci.c
create mode 100644 drivers/platform/x86/intel_speed_select_if/isst_if_mmio.c
create mode 100644 include/uapi/linux/isst_if.h
create mode 100644 tools/power/x86/intel_speed_select/Makefile
create mode 100644 tools/power/x86/intel_speed_select/isst.h
create mode 100644 tools/power/x86/intel_speed_select/isst_config.c
create mode 100644 tools/power/x86/intel_speed_select/isst_core.c
create mode 100644 tools/power/x86/intel_speed_select/isst_display.c
--
2.17.2
^ permalink raw reply
* Re: [PATCH v5 01/18] kunit: test: add KUnit test runner core
From: Brendan Higgins @ 2019-06-26 22:16 UTC (permalink / raw)
To: Luis Chamberlain
Cc: Frank Rowand, Greg KH, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Peter Zijlstra, Rob Herring, Stephen Boyd, shuah,
Theodore Ts'o, Masahiro Yamada, devicetree, dri-devel,
kunit-dev, open list:DOCUMENTATION, linux-fsdevel, linux-kbuild,
Linux Kernel Mailing List, open list:KERNEL SELFTEST FRAMEWORK,
linux-nvdimm, linux-um, Sasha Levin, Bird, Timothy,
Amir Goldstein, Dan Carpenter, Daniel Vetter, Jeff Dike,
Joel Stanley, Julia Lawall, Kevin Hilman, Knut Omang,
Logan Gunthorpe, Michael Ellerman, Petr Mladek, Randy Dunlap,
Richard Weinberger, David Rientjes, Steven Rostedt, wfg
In-Reply-To: <20190626033643.GX19023@42.do-not-panic.com>
On Tue, Jun 25, 2019 at 8:36 PM Luis Chamberlain <mcgrof@kernel.org> wrote:
>
> On Tue, Jun 25, 2019 at 05:07:32PM -0700, Brendan Higgins wrote:
> > On Tue, Jun 25, 2019 at 3:33 PM Luis Chamberlain <mcgrof@kernel.org> wrote:
> > >
> > > On Mon, Jun 17, 2019 at 01:25:56AM -0700, Brendan Higgins wrote:
> > > > +/**
> > > > + * module_test() - used to register a &struct kunit_module with KUnit.
> > > > + * @module: a statically allocated &struct kunit_module.
> > > > + *
> > > > + * Registers @module with the test framework. See &struct kunit_module for more
> > > > + * information.
> > > > + */
> > > > +#define module_test(module) \
> > > > + static int module_kunit_init##module(void) \
> > > > + { \
> > > > + return kunit_run_tests(&module); \
> > > > + } \
> > > > + late_initcall(module_kunit_init##module)
> > >
> > > Becuase late_initcall() is used, if these modules are built-in, this
> > > would preclude the ability to test things prior to this part of the
> > > kernel under UML or whatever architecture runs the tests. So, this
> > > limits the scope of testing. Small detail but the scope whould be
> > > documented.
> >
> > You aren't the first person to complain about this (and I am not sure
> > it is the first time you have complained about it). Anyway, I have
> > some follow on patches that will improve the late_initcall thing, and
> > people seemed okay with discussing the follow on patches as part of a
> > subsequent patchset after this gets merged.
> >
> > I will nevertheless document the restriction until then.
>
> To be clear, I am not complaining about it. I just find it simply
> critical to document its limitations, so folks don't try to invest
> time and energy on kunit right away for an early init test, if it
> cannot support it.
>
> If support for that requires some work, it may be worth mentioning
> as well.
Makes sense. And in anycase, it is something I do want to do, just not
right now. I will put a TODO here in the next revision.
> > > > +static void kunit_print_tap_version(void)
> > > > +{
> > > > + if (!kunit_has_printed_tap_version) {
> > > > + kunit_printk_emit(LOGLEVEL_INFO, "TAP version 14\n");
> > >
> > > What is this TAP thing? Why should we care what version it is on?
> > > Why are we printing this?
> >
> > It's part of the TAP specification[1]. Greg and Frank asked me to make
> > the intermediate format conform to TAP. Seems like something else I
> > should probable document...
>
> Yes thanks!
>
> > > > + kunit_has_printed_tap_version = true;
> > > > + }
> > > > +}
> > > > +
> > > > +static size_t kunit_test_cases_len(struct kunit_case *test_cases)
> > > > +{
> > > > + struct kunit_case *test_case;
> > > > + size_t len = 0;
> > > > +
> > > > + for (test_case = test_cases; test_case->run_case; test_case++)
> > >
> > > If we make the last test case NULL, we'd just check for test_case here,
> > > and save ourselves an extra few bytes per test module. Any reason why
> > > the last test case cannot be NULL?
> >
> > Is there anyway to make that work with a statically defined array?
>
> No you're right.
>
> > Basically, I want to be able to do something like:
> >
> > static struct kunit_case example_test_cases[] = {
> > KUNIT_CASE(example_simple_test),
> > KUNIT_CASE(example_mock_test),
> > {}
> > };
> >
> > FYI,
> > #define KUNIT_CASE(test_name) { .run_case = test_name, .name = #test_name }
>
> >
> > In order to do what you are proposing, I think I need an array of
> > pointers to test cases, which is not ideal.
>
> Yeah, you're right. The only other alternative is to have a:
>
> struct kunit_module {
> const char name[256];
> int (*init)(struct kunit *test);
> void (*exit)(struct kunit *test);
> struct kunit_case *test_cases;
> + unsigned int num_cases;
> };
>
> And then something like:
>
> #define KUNIT_MODULE(name, init, exit, cases) { \
> .name = name, \
> .init = init, \
> .exit = exit, \
> .test_cases = cases,
> num_cases = ARRAY_SIZE(cases), \
> }
>
> Let's evaluate which is better: one extra test case per all test cases, or
> an extra unsigned int for each kunit module.
I am in favor of the current method since init and exit are optional
arguments. I could see myself (actually I am planning on) adding more
optional things to the kunit_module, so having optional arguments will
make my life a lot easier since I won't have to go through big
refactorings around the kernel to support new features that tie in
here.
^ permalink raw reply
* Re: [PATCH v5 13/18] kunit: tool: add Python wrappers for running KUnit tests
From: Luis Chamberlain @ 2019-06-26 22:03 UTC (permalink / raw)
To: Brendan Higgins
Cc: Frank Rowand, Greg KH, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Peter Zijlstra, Rob Herring, Stephen Boyd, shuah,
Theodore Ts'o, Masahiro Yamada, devicetree, dri-devel,
kunit-dev, open list:DOCUMENTATION, linux-fsdevel, linux-kbuild,
Linux Kernel Mailing List, open list:KERNEL SELFTEST FRAMEWORK,
linux-nvdimm, linux-um, Sasha Levin, Bird, Timothy,
Amir Goldstein, Dan Carpenter, Daniel Vetter, Jeff Dike,
Joel Stanley, Julia Lawall, Kevin Hilman, Knut Omang,
Logan Gunthorpe, Michael Ellerman, Petr Mladek, Randy Dunlap,
Richard Weinberger, David Rientjes, Steven Rostedt, wfg,
Felix Guo
In-Reply-To: <CAFd5g44kkepB2hZcpYL-NB5ZHYE5tP7W-0yducGCX7Khd9gd9w@mail.gmail.com>
On Wed, Jun 26, 2019 at 01:02:55AM -0700, Brendan Higgins wrote:
> On Tue, Jun 25, 2019 at 5:01 PM Luis Chamberlain <mcgrof@kernel.org> wrote:
> >
> > On Mon, Jun 17, 2019 at 01:26:08AM -0700, Brendan Higgins wrote:
> > > create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-all_passed.log
> > > create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-crash.log
> > > create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-failure.log
> > > create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-no_tests_run.log
> > > create mode 100644 tools/testing/kunit/test_data/test_output_isolated_correctly.log
> > > create mode 100644 tools/testing/kunit/test_data/test_read_from_file.kconfig
> >
> > Why are these being added upstream? The commit log does not explain
> > this.
>
> Oh sorry, those are for testing purposes. I thought that was clear
> from being in the test_data directory. I will reference it in the
> commit log in the next revision.
Still, I don't get it. They seem to be results from a prior run. Why do
we need them for testing purposes?
Luis
^ permalink raw reply
* Re: [PATCH v5 01/18] kunit: test: add KUnit test runner core
From: Luis Chamberlain @ 2019-06-26 22:02 UTC (permalink / raw)
To: Brendan Higgins
Cc: Stephen Boyd, Frank Rowand, Greg KH, Josh Poimboeuf, Kees Cook,
Kieran Bingham, Peter Zijlstra, Rob Herring, shuah,
Theodore Ts'o, Masahiro Yamada, devicetree, dri-devel,
kunit-dev, open list:DOCUMENTATION, linux-fsdevel, linux-kbuild,
Linux Kernel Mailing List, open list:KERNEL SELFTEST FRAMEWORK,
linux-nvdimm, linux-um, Sasha Levin, Bird, Timothy,
Amir Goldstein, Dan Carpenter, Daniel Vetter, Jeff Dike,
Joel Stanley, Julia Lawall, Kevin Hilman, Knut Omang,
Logan Gunthorpe, Michael Ellerman, Petr Mladek, Randy Dunlap,
Richard Weinberger, David Rientjes, Steven Rostedt, wfg
In-Reply-To: <CAFd5g45fSdpytudDyD3Yo1ti=kU_JJ6S9yz53_L=pnZTjQFU9A@mail.gmail.com>
On Tue, Jun 25, 2019 at 11:41:47PM -0700, Brendan Higgins wrote:
> On Tue, Jun 25, 2019 at 4:02 PM Luis Chamberlain <mcgrof@kernel.org> wrote:
> >
> > On Tue, Jun 25, 2019 at 03:14:45PM -0700, Brendan Higgins wrote:
> > > On Tue, Jun 25, 2019 at 2:44 PM Luis Chamberlain <mcgrof@kernel.org> wrote:
> > > > Since its a new architecture and since you seem to imply most tests
> > > > don't require locking or even IRQs disabled, I think its worth to
> > > > consider the impact of adding such extreme locking requirements for
> > > > an initial ramp up.
> > >
> > > Fair enough, I can see the point of not wanting to use irq disabled
> > > until we get someone complaining about it, but I think making it
> > > thread safe is reasonable. It means there is one less thing to confuse
> > > a KUnit user and the only penalty paid is some very minor performance.
> >
> > One reason I'm really excited about kunit is speed... so by all means I
> > think we're at a good point to analyze performance optimizationsm if
> > they do make sense.
>
> Yeah, but I think there are much lower hanging fruit than this (as you
> point out below). I am all for making/keeping KUnit super fast, but I
> also don't want to waste time with premature optimizations and I think
> having thread safe expectations and non-thread safe expectations hurts
> usability.
>
> Still, I am on board with making this a mutex instead of a spinlock for now.
>
> > While on the topic of parallization, what about support for running
> > different test cases in parallel? Or at the very least different kunit
> > modules in parallel. Few questions come up based on this prospect:
> >
> > * Why not support parallelism from the start?
>
> Just because it is more work and there isn't much to gain from it right now.
>
> Some numbers:
> I currently have a collection of 86 test cases in the branch that this
> patchset is from.
Impressive, imagine 86 tests and kunit is not even *merged yet*.
> I turned on PRINTK_TIME and looked at the first
> KUnit output and the last. On UML, start time was 0.090000, and end
> time was 0.090000. Looks like sched_clock is not very good on UML.
Since you have a python thing that kicks tests, what if you just run
time on it?
> Still it seems quite likely that all of these tests run around 0.01
> seconds or less on UML: I ran KUnit with only 2 test cases enabled
> three times and got an average runtime of 1.55867 seconds with a
> standard deviation of 0.0346747. I then ran it another three times
> with all test cases enabled and got an average runtime of 1.535
> seconds with a standard deviation of 0.0150997. The second average is
> less, but that doesn't really mean anything because it is well within
> one standard deviation with a very small sample size. Nevertheless, we
> can conclude that the actual runtime of those 84 test cases is most
> likely within one standard deviation, so on the order of 0.01 seconds.
>
> On x86 running on QEMU, first message from KUnit was printed at
> 0.194251 and the last KUnit message was printed at 0.340915, meaning
> that all 86 test cases ran in about 0.146664 seconds.
Pretty impressive numbers. But can you blame me for expressing the
desire to possibly being able to do better? I am not saying -- let's
definitely have parallelism in place *now*. Just wanted to hear out
tangibles of why we *don't* want it now.
And.. also, since we are reviewing now, try to target so that the code
can later likely get a face lift to support parallelism without
requiring much changes.
> In any case, running KUnit tests in parallel is definitely something I
> plan on adding it eventually, but it just doesn't really seem worth it
> right now.
Makes sense!
> I find the incremental build time of the kernel to
> typically be between 3 and 30 seconds, and a clean build to be between
> 30 seconds to several minutes, depending on the number of available
> cores, so I don't think most users would even notice the amount of
> runtime contributed by the actual unit tests until we start getting
> into the 1000s of test cases. I don't suspect it will become an issue
> until we get into the 10,000s of test cases. I think we are a pretty
> long way off from that.
All makes sense, and agreed based on the numbers you are providing.
Thanks for the details!
> > * Are you opposed to eventually having this added? For instance, there is
> > enough code on lib/test_kmod.c for batching tons of kthreads each
> > one running its own thing for testing purposes which could be used
> > as template.
>
> I am not opposed to adding it eventually at all. I actually plan on
> doing so, just not in this patchset. There are a lot of additional
> features, improvements, and sugar that I really want to add, so much
> so that most of it doesn't belong in this patchset; I just think this
> is one of those things that belongs in a follow up. I tried to boil
> down this patchset to as small as I could while still being useful;
> this is basically an MVP. Maybe after this patchset gets merged I
> should post a list of things I have ready for review, or would like to
> work on, and people can comment on what things they want to see next.
Groovy.
> > * If we eventually *did* support it:
> > - Would logs be skewed?
>
> Probably, before I went with the TAP approach, I was tagging each
> message with the test case it came from and I could have parsed it and
> assembled a coherent view of the logs using that; now that I am using
> TAP conforming output, that won't work. I haven't really thought too
> hard about how to address it, but there are ways. For the UML users, I
> am planning on adding a feature to guarantee hermeticity between tests
> running in different modules by adding a feature that allows a single
> kernel to be built with all tests included, and then determine which
> tests get run by passing in command line arguments or something. This
> way you can get the isolation from running tests in separate
> environments without increasing the build cost. We could also use this
> method to achieve parallelism by dispatching multiple kernels at once.
Indeed. Or... wait for it... containers... There tools for these sorts
of things already. And I'm evaluating such prospects now for some other
tests I care for.
> That only works for UML, but I imagine you could do something similar
> for users running tests under qemu.
Or containers.
> > - Could we have a way to query: give me log for only kunit module
> > named "foo"?
>
> Yeah, I think that would make sense as part of the hermeticity thing I
> mentioned above.
>
> Hope that seems reasonable!
All groovy. Thanks for the details!
Luis
^ permalink raw reply
* Re: [PATCH] drm: fix a reference for a renamed file: fb/modedb.rst
From: Daniel Vetter @ 2019-06-26 21:27 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Maarten Lankhorst, Maxime Ripard, Sean Paul,
David Airlie, Daniel Vetter, dri-devel
In-Reply-To: <699d7618720e2808f9aa094a13ab2f3545c3c25c.1561565652.git.mchehab+samsung@kernel.org>
On Wed, Jun 26, 2019 at 01:14:13PM -0300, Mauro Carvalho Chehab wrote:
> Due to two patches being applied about the same time, the
> reference for modedb.rst file got wrong:
>
> Documentation/fb/modedb.txt is now Documentation/fb/modedb.rst.
>
> Fixes: 1bf4e09227c3 ("drm/modes: Allow to specify rotation and reflection on the commandline")
> Fixes: ab42b818954c ("docs: fb: convert docs to ReST and rename to *.rst")
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
What's the merge plan here? doc-next? If so:
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> ---
> drivers/gpu/drm/drm_modes.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/drm_modes.c b/drivers/gpu/drm/drm_modes.c
> index 57e6408288c8..4645af681ef8 100644
> --- a/drivers/gpu/drm/drm_modes.c
> +++ b/drivers/gpu/drm/drm_modes.c
> @@ -1680,7 +1680,7 @@ static int drm_mode_parse_cmdline_options(char *str, size_t len,
> *
> * Additionals options can be provided following the mode, using a comma to
> * separate each option. Valid options can be found in
> - * Documentation/fb/modedb.txt.
> + * Documentation/fb/modedb.rst.
> *
> * The intermediate drm_cmdline_mode structure is required to store additional
> * options from the command line modline like the force-enable/disable flag.
> --
> 2.21.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply
* Re: [PATCH 2/2] mm, slab: Extend vm/drop_caches to shrink kmem slabs
From: Roman Gushchin @ 2019-06-26 20:19 UTC (permalink / raw)
To: Waiman Long
Cc: Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim,
Andrew Morton, Alexander Viro, Jonathan Corbet, Luis Chamberlain,
Kees Cook, Johannes Weiner, Michal Hocko, Vladimir Davydov,
linux-mm@kvack.org, linux-doc@vger.kernel.org,
linux-fsdevel@vger.kernel.org, cgroups@vger.kernel.org,
linux-kernel@vger.kernel.org, Shakeel Butt, Andrea Arcangeli
In-Reply-To: <20190624174219.25513-3-longman@redhat.com>
On Mon, Jun 24, 2019 at 01:42:19PM -0400, Waiman Long wrote:
> With the slub memory allocator, the numbers of active slab objects
> reported in /proc/slabinfo are not real because they include objects
> that are held by the per-cpu slab structures whether they are actually
> used or not. The problem gets worse the more CPUs a system have. For
> instance, looking at the reported number of active task_struct objects,
> one will wonder where all the missing tasks gone.
>
> I know it is hard and costly to get a real count of active objects. So
> I am not advocating for that. Instead, this patch extends the
> /proc/sys/vm/drop_caches sysctl parameter by using a new bit (bit 3)
> to shrink all the kmem slabs which will flush out all the slabs in the
> per-cpu structures and give a more accurate view of how much memory are
> really used up by the active slab objects. This is a costly operation,
> of course, but it gives a way to have a clearer picture of the actual
> number of slab objects used, if the need arises.
>
> The upper range of the drop_caches sysctl parameter is increased to 15
> to allow all possible combinations of the lowest 4 bits.
>
> On a 2-socket 64-core 256-thread ARM64 system with 64k page size after
> a parallel kernel build, the amount of memory occupied by slabs before
> and after echoing to drop_caches were:
>
> # grep task_struct /proc/slabinfo
> task_struct 48376 48434 4288 61 4 : tunables 0 0
> 0 : slabdata 794 794 0
> # grep "^S[lRU]" /proc/meminfo
> Slab: 3419072 kB
> SReclaimable: 354688 kB
> SUnreclaim: 3064384 kB
> # echo 3 > /proc/sys/vm/drop_caches
> # grep "^S[lRU]" /proc/meminfo
> Slab: 3351680 kB
> SReclaimable: 316096 kB
> SUnreclaim: 3035584 kB
> # echo 8 > /proc/sys/vm/drop_caches
> # grep "^S[lRU]" /proc/meminfo
> Slab: 1008192 kB
> SReclaimable: 126912 kB
> SUnreclaim: 881280 kB
> # grep task_struct /proc/slabinfo
> task_struct 2601 6588 4288 61 4 : tunables 0 0
> 0 : slabdata 108 108 0
>
> Shrinking the slabs saves more than 2GB of memory in this case. This
> new feature certainly fulfills the promise of dropping caches.
>
> Unlike counting objects in the per-node caches done by /proc/slabinfo
> which is rather light weight, iterating all the per-cpu caches and
> shrinking them is much more heavy weight.
>
> For this particular instance, the time taken to shrinks all the root
> caches was about 30.2ms. There were 73 memory cgroup and the longest
> time taken for shrinking the largest one was about 16.4ms. The total
> shrinking time was about 101ms.
>
> Because of the potential long time to shrinks all the caches, the
> slab_mutex was taken multiple times - once for all the root caches
> and once for each memory cgroup. This is to reduce the slab_mutex hold
> time to minimize impact to other running applications that may need to
> acquire the mutex.
>
> The slab shrinking feature is only available when CONFIG_MEMCG_KMEM is
> defined as the code need to access slab_root_caches to iterate all the
> root caches.
>
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
> Documentation/sysctl/vm.txt | 11 ++++++++--
> fs/drop_caches.c | 4 ++++
> include/linux/slab.h | 1 +
> kernel/sysctl.c | 4 ++--
> mm/slab_common.c | 44 +++++++++++++++++++++++++++++++++++++
> 5 files changed, 60 insertions(+), 4 deletions(-)
>
> diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt
> index 749322060f10..b643ac8968d2 100644
> --- a/Documentation/sysctl/vm.txt
> +++ b/Documentation/sysctl/vm.txt
> @@ -207,8 +207,8 @@ Setting this to zero disables periodic writeback altogether.
> drop_caches
>
> Writing to this will cause the kernel to drop clean caches, as well as
> -reclaimable slab objects like dentries and inodes. Once dropped, their
> -memory becomes free.
> +reclaimable slab objects like dentries and inodes. It can also be used
> +to shrink the slabs. Once dropped, their memory becomes free.
>
> To free pagecache:
> echo 1 > /proc/sys/vm/drop_caches
> @@ -216,6 +216,8 @@ To free reclaimable slab objects (includes dentries and inodes):
> echo 2 > /proc/sys/vm/drop_caches
> To free slab objects and pagecache:
> echo 3 > /proc/sys/vm/drop_caches
> +To shrink the slabs:
> + echo 8 > /proc/sys/vm/drop_caches
>
> This is a non-destructive operation and will not free any dirty objects.
> To increase the number of objects freed by this operation, the user may run
> @@ -223,6 +225,11 @@ To increase the number of objects freed by this operation, the user may run
> number of dirty objects on the system and create more candidates to be
> dropped.
>
> +Shrinking the slabs can reduce the memory footprint used by the slabs.
> +It also makes the number of active objects reported in /proc/slabinfo
> +more representative of the actual number of objects used for the slub
> +memory allocator.
> +
> This file is not a means to control the growth of the various kernel caches
> (inodes, dentries, pagecache, etc...) These objects are automatically
> reclaimed by the kernel when memory is needed elsewhere on the system.
> diff --git a/fs/drop_caches.c b/fs/drop_caches.c
> index d31b6c72b476..633b99e25dab 100644
> --- a/fs/drop_caches.c
> +++ b/fs/drop_caches.c
> @@ -9,6 +9,7 @@
> #include <linux/writeback.h>
> #include <linux/sysctl.h>
> #include <linux/gfp.h>
> +#include <linux/slab.h>
> #include "internal.h"
>
> /* A global variable is a bit ugly, but it keeps the code simple */
> @@ -65,6 +66,9 @@ int drop_caches_sysctl_handler(struct ctl_table *table, int write,
> drop_slab();
> count_vm_event(DROP_SLAB);
> }
> + if (sysctl_drop_caches & 8) {
> + kmem_cache_shrink_all();
> + }
> if (!stfu) {
> pr_info("%s (%d): drop_caches: %d\n",
> current->comm, task_pid_nr(current),
> diff --git a/include/linux/slab.h b/include/linux/slab.h
> index 9449b19c5f10..f7c1626b2aa6 100644
> --- a/include/linux/slab.h
> +++ b/include/linux/slab.h
> @@ -149,6 +149,7 @@ struct kmem_cache *kmem_cache_create_usercopy(const char *name,
> void (*ctor)(void *));
> void kmem_cache_destroy(struct kmem_cache *);
> int kmem_cache_shrink(struct kmem_cache *);
> +void kmem_cache_shrink_all(void);
>
> void memcg_create_kmem_cache(struct mem_cgroup *, struct kmem_cache *);
> void memcg_deactivate_kmem_caches(struct mem_cgroup *);
> diff --git a/kernel/sysctl.c b/kernel/sysctl.c
> index 1beca96fb625..feeb867dabd7 100644
> --- a/kernel/sysctl.c
> +++ b/kernel/sysctl.c
> @@ -129,7 +129,7 @@ static int __maybe_unused neg_one = -1;
> static int zero;
> static int __maybe_unused one = 1;
> static int __maybe_unused two = 2;
> -static int __maybe_unused four = 4;
> +static int __maybe_unused fifteen = 15;
> static unsigned long zero_ul;
> static unsigned long one_ul = 1;
> static unsigned long long_max = LONG_MAX;
> @@ -1455,7 +1455,7 @@ static struct ctl_table vm_table[] = {
> .mode = 0644,
> .proc_handler = drop_caches_sysctl_handler,
> .extra1 = &one,
> - .extra2 = &four,
> + .extra2 = &fifteen,
> },
> #ifdef CONFIG_COMPACTION
> {
> diff --git a/mm/slab_common.c b/mm/slab_common.c
> index 58251ba63e4a..b3c5b64f9bfb 100644
> --- a/mm/slab_common.c
> +++ b/mm/slab_common.c
> @@ -956,6 +956,50 @@ int kmem_cache_shrink(struct kmem_cache *cachep)
> }
> EXPORT_SYMBOL(kmem_cache_shrink);
Hi Waiman!
>
> +#ifdef CONFIG_MEMCG_KMEM
> +static void kmem_cache_shrink_memcg(struct mem_cgroup *memcg,
> + void __maybe_unused *arg)
> +{
> + struct kmem_cache *s;
> +
> + if (memcg == root_mem_cgroup)
> + return;
> + mutex_lock(&slab_mutex);
> + list_for_each_entry(s, &memcg->kmem_caches,
> + memcg_params.kmem_caches_node) {
> + kmem_cache_shrink(s);
> + }
> + mutex_unlock(&slab_mutex);
> + cond_resched();
> +}
A couple of questions:
1) how about skipping already offlined kmem_caches? They are already shrunk,
so you probably won't get much out of them. Or isn't it true?
2) what's your long-term vision here? do you think that we need to shrink
kmem_caches periodically, depending on memory pressure? how a user
will use this new sysctl?
What's the problem you're trying to solve in general?
Thanks!
Roman
^ permalink raw reply
* [Linux-kernel-mentees][PATCH v5 5/5] Documentation: RCU: Add TOC tree hooks
From: Jiunn Chang @ 2019-06-26 20:07 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190626191249.21135-1-c0d1n61at3@gmail.com>
Add TOC tree hooks for:
- rcu
- listRCU
- UP
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/index.rst | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
create mode 100644 Documentation/RCU/index.rst
diff --git a/Documentation/RCU/index.rst b/Documentation/RCU/index.rst
new file mode 100644
index 000000000000..340a9725676c
--- /dev/null
+++ b/Documentation/RCU/index.rst
@@ -0,0 +1,19 @@
+.. _rcu_concepts:
+
+============
+RCU concepts
+============
+
+.. toctree::
+ :maxdepth: 1
+
+ rcu
+ listRCU
+ UP
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
--
2.22.0
^ permalink raw reply related
* [Linux-kernel-mentees][PATCH v5 4/5] Documentation: RCU: Rename txt files to rst
From: Jiunn Chang @ 2019-06-26 20:07 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190626191249.21135-1-c0d1n61at3@gmail.com>
Rename the following files to reST:
- rcu.txt
- listRCU.txt
- UP.txt
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/{UP.txt => UP.rst} | 0
Documentation/RCU/{listRCU.txt => listRCU.rst} | 0
Documentation/RCU/{rcu.txt => rcu.rst} | 0
3 files changed, 0 insertions(+), 0 deletions(-)
rename Documentation/RCU/{UP.txt => UP.rst} (100%)
rename Documentation/RCU/{listRCU.txt => listRCU.rst} (100%)
rename Documentation/RCU/{rcu.txt => rcu.rst} (100%)
diff --git a/Documentation/RCU/UP.txt b/Documentation/RCU/UP.rst
similarity index 100%
rename from Documentation/RCU/UP.txt
rename to Documentation/RCU/UP.rst
diff --git a/Documentation/RCU/listRCU.txt b/Documentation/RCU/listRCU.rst
similarity index 100%
rename from Documentation/RCU/listRCU.txt
rename to Documentation/RCU/listRCU.rst
diff --git a/Documentation/RCU/rcu.txt b/Documentation/RCU/rcu.rst
similarity index 100%
rename from Documentation/RCU/rcu.txt
rename to Documentation/RCU/rcu.rst
--
2.22.0
^ permalink raw reply
* [Linux-kernel-mentees][PATCH v5 3/5] Documentation: RCU: Convert RCU UP systems to reST
From: Jiunn Chang @ 2019-06-26 20:07 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190626191249.21135-1-c0d1n61at3@gmail.com>
RCU UP systems reST markup.
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
Documentation/RCU/UP.txt | 37 +++++++++++++++++++++++--------------
1 file changed, 23 insertions(+), 14 deletions(-)
diff --git a/Documentation/RCU/UP.txt b/Documentation/RCU/UP.txt
index 53bde717017b..67715a47ae89 100644
--- a/Documentation/RCU/UP.txt
+++ b/Documentation/RCU/UP.txt
@@ -1,17 +1,19 @@
-RCU on Uniprocessor Systems
+.. _up_doc:
+RCU on Uniprocessor Systems
+===========================
A common misconception is that, on UP systems, the call_rcu() primitive
may immediately invoke its function. The basis of this misconception
is that since there is only one CPU, it should not be necessary to
wait for anything else to get done, since there are no other CPUs for
-anything else to be happening on. Although this approach will -sort- -of-
+anything else to be happening on. Although this approach will *sort of*
work a surprising amount of the time, it is a very bad idea in general.
This document presents three examples that demonstrate exactly how bad
an idea this is.
-
Example 1: softirq Suicide
+--------------------------
Suppose that an RCU-based algorithm scans a linked list containing
elements A, B, and C in process context, and can delete elements from
@@ -28,8 +30,8 @@ your kernel.
This same problem can occur if call_rcu() is invoked from a hardware
interrupt handler.
-
Example 2: Function-Call Fatality
+---------------------------------
Of course, one could avert the suicide described in the preceding example
by having call_rcu() directly invoke its arguments only if it was called
@@ -46,11 +48,13 @@ its arguments would cause it to fail to make the fundamental guarantee
underlying RCU, namely that call_rcu() defers invoking its arguments until
all RCU read-side critical sections currently executing have completed.
-Quick Quiz #1: why is it -not- legal to invoke synchronize_rcu() in
- this case?
+Quick Quiz #1:
+ Why is it *not* legal to invoke synchronize_rcu() in this case?
+:ref:`Answers to Quick Quiz <answer_quick_quiz_up>`
Example 3: Death by Deadlock
+----------------------------
Suppose that call_rcu() is invoked while holding a lock, and that the
callback function must acquire this same lock. In this case, if
@@ -76,25 +80,30 @@ there are cases where this can be quite ugly:
If call_rcu() directly invokes the callback, painful locking restrictions
or API changes would be required.
-Quick Quiz #2: What locking restriction must RCU callbacks respect?
+Quick Quiz #2:
+ What locking restriction must RCU callbacks respect?
+:ref:`Answers to Quick Quiz <answer_quick_quiz_up>`
Summary
+-------
Permitting call_rcu() to immediately invoke its arguments breaks RCU,
even on a UP system. So do not do it! Even on a UP system, the RCU
-infrastructure -must- respect grace periods, and -must- invoke callbacks
+infrastructure *must* respect grace periods, and *must* invoke callbacks
from a known environment in which no locks are held.
-Note that it -is- safe for synchronize_rcu() to return immediately on
-UP systems, including !PREEMPT SMP builds running on UP systems.
+Note that it *is* safe for synchronize_rcu() to return immediately on
+UP systems, including PREEMPT SMP builds running on UP systems.
-Quick Quiz #3: Why can't synchronize_rcu() return immediately on
- UP systems running preemptable RCU?
+Quick Quiz #3:
+ Why can't synchronize_rcu() return immediately on UP systems running
+ preemptable RCU?
+.. _answer_quick_quiz_up:
Answer to Quick Quiz #1:
- Why is it -not- legal to invoke synchronize_rcu() in this case?
+ Why is it *not* legal to invoke synchronize_rcu() in this case?
Because the calling function is scanning an RCU-protected linked
list, and is therefore within an RCU read-side critical section.
@@ -119,7 +128,7 @@ Answer to Quick Quiz #2:
This restriction might seem gratuitous, since very few RCU
callbacks acquire locks directly. However, a great many RCU
- callbacks do acquire locks -indirectly-, for example, via
+ callbacks do acquire locks *indirectly*, for example, via
the kfree() primitive.
Answer to Quick Quiz #3:
--
2.22.0
^ permalink raw reply related
* [Linux-kernel-mentees][PATCH v5 2/5] Documentation: RCU: Convert RCU linked list to reST
From: Jiunn Chang @ 2019-06-26 20:07 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190626191249.21135-1-c0d1n61at3@gmail.com>
RCU linked list reST markup.
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/listRCU.txt | 38 ++++++++++++++++++++---------------
1 file changed, 22 insertions(+), 16 deletions(-)
diff --git a/Documentation/RCU/listRCU.txt b/Documentation/RCU/listRCU.txt
index adb5a3782846..7956ff33042b 100644
--- a/Documentation/RCU/listRCU.txt
+++ b/Documentation/RCU/listRCU.txt
@@ -1,5 +1,7 @@
-Using RCU to Protect Read-Mostly Linked Lists
+.. _list_rcu_doc:
+Using RCU to Protect Read-Mostly Linked Lists
+=============================================
One of the best applications of RCU is to protect read-mostly linked lists
("struct list_head" in list.h). One big advantage of this approach
@@ -7,8 +9,8 @@ is that all of the required memory barriers are included for you in
the list macros. This document describes several applications of RCU,
with the best fits first.
-
Example 1: Read-Side Action Taken Outside of Lock, No In-Place Updates
+----------------------------------------------------------------------
The best applications are cases where, if reader-writer locking were
used, the read-side lock would be dropped before taking any action
@@ -24,7 +26,7 @@ added or deleted, rather than being modified in place.
A straightforward example of this use of RCU may be found in the
system-call auditing support. For example, a reader-writer locked
-implementation of audit_filter_task() might be as follows:
+implementation of audit_filter_task() might be as follows::
static enum audit_state audit_filter_task(struct task_struct *tsk)
{
@@ -48,7 +50,7 @@ the corresponding value is returned. By the time that this value is acted
on, the list may well have been modified. This makes sense, since if
you are turning auditing off, it is OK to audit a few extra system calls.
-This means that RCU can be easily applied to the read side, as follows:
+This means that RCU can be easily applied to the read side, as follows::
static enum audit_state audit_filter_task(struct task_struct *tsk)
{
@@ -73,7 +75,7 @@ become list_for_each_entry_rcu(). The _rcu() list-traversal primitives
insert the read-side memory barriers that are required on DEC Alpha CPUs.
The changes to the update side are also straightforward. A reader-writer
-lock might be used as follows for deletion and insertion:
+lock might be used as follows for deletion and insertion::
static inline int audit_del_rule(struct audit_rule *rule,
struct list_head *list)
@@ -106,7 +108,7 @@ lock might be used as follows for deletion and insertion:
return 0;
}
-Following are the RCU equivalents for these two functions:
+Following are the RCU equivalents for these two functions::
static inline int audit_del_rule(struct audit_rule *rule,
struct list_head *list)
@@ -154,13 +156,13 @@ otherwise cause concurrent readers to fail spectacularly.
So, when readers can tolerate stale data and when entries are either added
or deleted, without in-place modification, it is very easy to use RCU!
-
Example 2: Handling In-Place Updates
+------------------------------------
The system-call auditing code does not update auditing rules in place.
However, if it did, reader-writer-locked code to do so might look as
follows (presumably, the field_count is only permitted to decrease,
-otherwise, the added fields would need to be filled in):
+otherwise, the added fields would need to be filled in)::
static inline int audit_upd_rule(struct audit_rule *rule,
struct list_head *list,
@@ -187,7 +189,7 @@ otherwise, the added fields would need to be filled in):
The RCU version creates a copy, updates the copy, then replaces the old
entry with the newly updated entry. This sequence of actions, allowing
concurrent reads while doing a copy to perform an update, is what gives
-RCU ("read-copy update") its name. The RCU code is as follows:
+RCU ("read-copy update") its name. The RCU code is as follows::
static inline int audit_upd_rule(struct audit_rule *rule,
struct list_head *list,
@@ -216,8 +218,8 @@ RCU ("read-copy update") its name. The RCU code is as follows:
Again, this assumes that the caller holds audit_netlink_sem. Normally,
the reader-writer lock would become a spinlock in this sort of code.
-
Example 3: Eliminating Stale Data
+---------------------------------
The auditing examples above tolerate stale data, as do most algorithms
that are tracking external state. Because there is a delay from the
@@ -231,13 +233,16 @@ per-entry spinlock, and, if the "deleted" flag is set, pretends that the
entry does not exist. For this to be helpful, the search function must
return holding the per-entry spinlock, as ipc_lock() does in fact do.
-Quick Quiz: Why does the search function need to return holding the
- per-entry lock for this deleted-flag technique to be helpful?
+Quick Quiz:
+ Why does the search function need to return holding the per-entry lock for
+ this deleted-flag technique to be helpful?
+
+:ref:`Answer to Quick Quiz <answer_quick_quiz_list>`
If the system-call audit module were to ever need to reject stale data,
one way to accomplish this would be to add a "deleted" flag and a "lock"
spinlock to the audit_entry structure, and modify audit_filter_task()
-as follows:
+as follows::
static enum audit_state audit_filter_task(struct task_struct *tsk)
{
@@ -268,7 +273,7 @@ audit_upd_rule() would need additional memory barriers to ensure
that the list_add_rcu() was really executed before the list_del_rcu().
The audit_del_rule() function would need to set the "deleted"
-flag under the spinlock as follows:
+flag under the spinlock as follows::
static inline int audit_del_rule(struct audit_rule *rule,
struct list_head *list)
@@ -290,8 +295,8 @@ flag under the spinlock as follows:
return -EFAULT; /* No matching rule */
}
-
Summary
+-------
Read-mostly list-based data structures that can tolerate stale data are
the most amenable to use of RCU. The simplest case is where entries are
@@ -302,8 +307,9 @@ If stale data cannot be tolerated, then a "deleted" flag may be used
in conjunction with a per-entry spinlock in order to allow the search
function to reject newly deleted data.
+.. _answer_quick_quiz_list:
-Answer to Quick Quiz
+Answer to Quick Quiz:
Why does the search function need to return holding the per-entry
lock for this deleted-flag technique to be helpful?
--
2.22.0
^ permalink raw reply related
* [Linux-kernel-mentees][PATCH v5 1/5] Documentation: RCU: Convert RCU basic concepts to reST
From: Jiunn Chang @ 2019-06-26 20:07 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190626191249.21135-1-c0d1n61at3@gmail.com>
RCU basic concepts reST markup.
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
Documentation/RCU/rcu.txt | 119 +++++++++++++++++++-------------------
1 file changed, 61 insertions(+), 58 deletions(-)
diff --git a/Documentation/RCU/rcu.txt b/Documentation/RCU/rcu.txt
index c818cf65c5a9..8dfb437dacc3 100644
--- a/Documentation/RCU/rcu.txt
+++ b/Documentation/RCU/rcu.txt
@@ -1,5 +1,7 @@
-RCU Concepts
+.. _rcu_doc:
+RCU Concepts
+============
The basic idea behind RCU (read-copy update) is to split destructive
operations into two parts, one that prevents anyone from seeing the data
@@ -8,82 +10,83 @@ A "grace period" must elapse between the two parts, and this grace period
must be long enough that any readers accessing the item being deleted have
since dropped their references. For example, an RCU-protected deletion
from a linked list would first remove the item from the list, wait for
-a grace period to elapse, then free the element. See the listRCU.txt
-file for more information on using RCU with linked lists.
-
+a grace period to elapse, then free the element. See the
+Documentation/RCU/listRCU.rst file for more information on using RCU with
+linked lists.
Frequently Asked Questions
+--------------------------
-o Why would anyone want to use RCU?
+- Why would anyone want to use RCU?
- The advantage of RCU's two-part approach is that RCU readers need
- not acquire any locks, perform any atomic instructions, write to
- shared memory, or (on CPUs other than Alpha) execute any memory
- barriers. The fact that these operations are quite expensive
- on modern CPUs is what gives RCU its performance advantages
- in read-mostly situations. The fact that RCU readers need not
- acquire locks can also greatly simplify deadlock-avoidance code.
+ The advantage of RCU's two-part approach is that RCU readers need
+ not acquire any locks, perform any atomic instructions, write to
+ shared memory, or (on CPUs other than Alpha) execute any memory
+ barriers. The fact that these operations are quite expensive
+ on modern CPUs is what gives RCU its performance advantages
+ in read-mostly situations. The fact that RCU readers need not
+ acquire locks can also greatly simplify deadlock-avoidance code.
-o How can the updater tell when a grace period has completed
- if the RCU readers give no indication when they are done?
+- How can the updater tell when a grace period has completed
+ if the RCU readers give no indication when they are done?
- Just as with spinlocks, RCU readers are not permitted to
- block, switch to user-mode execution, or enter the idle loop.
- Therefore, as soon as a CPU is seen passing through any of these
- three states, we know that that CPU has exited any previous RCU
- read-side critical sections. So, if we remove an item from a
- linked list, and then wait until all CPUs have switched context,
- executed in user mode, or executed in the idle loop, we can
- safely free up that item.
+ Just as with spinlocks, RCU readers are not permitted to
+ block, switch to user-mode execution, or enter the idle loop.
+ Therefore, as soon as a CPU is seen passing through any of these
+ three states, we know that that CPU has exited any previous RCU
+ read-side critical sections. So, if we remove an item from a
+ linked list, and then wait until all CPUs have switched context,
+ executed in user mode, or executed in the idle loop, we can
+ safely free up that item.
- Preemptible variants of RCU (CONFIG_PREEMPT_RCU) get the
- same effect, but require that the readers manipulate CPU-local
- counters. These counters allow limited types of blocking within
- RCU read-side critical sections. SRCU also uses CPU-local
- counters, and permits general blocking within RCU read-side
- critical sections. These variants of RCU detect grace periods
- by sampling these counters.
+ Preemptible variants of RCU (CONFIG_PREEMPT_RCU) get the
+ same effect, but require that the readers manipulate CPU-local
+ counters. These counters allow limited types of blocking within
+ RCU read-side critical sections. SRCU also uses CPU-local
+ counters, and permits general blocking within RCU read-side
+ critical sections. These variants of RCU detect grace periods
+ by sampling these counters.
-o If I am running on a uniprocessor kernel, which can only do one
- thing at a time, why should I wait for a grace period?
+- If I am running on a uniprocessor kernel, which can only do one
+ thing at a time, why should I wait for a grace period?
- See the UP.txt file in this directory.
+ See the Documentation/RCU/UP.rst file for more information.
-o How can I see where RCU is currently used in the Linux kernel?
+- How can I see where RCU is currently used in the Linux kernel?
- Search for "rcu_read_lock", "rcu_read_unlock", "call_rcu",
- "rcu_read_lock_bh", "rcu_read_unlock_bh", "srcu_read_lock",
- "srcu_read_unlock", "synchronize_rcu", "synchronize_net",
- "synchronize_srcu", and the other RCU primitives. Or grab one
- of the cscope databases from:
+ Search for "rcu_read_lock", "rcu_read_unlock", "call_rcu",
+ "rcu_read_lock_bh", "rcu_read_unlock_bh", "srcu_read_lock",
+ "srcu_read_unlock", "synchronize_rcu", "synchronize_net",
+ "synchronize_srcu", and the other RCU primitives. Or grab one
+ of the cscope databases from:
- http://www.rdrop.com/users/paulmck/RCU/linuxusage/rculocktab.html
+ (http://www.rdrop.com/users/paulmck/RCU/linuxusage/rculocktab.html).
-o What guidelines should I follow when writing code that uses RCU?
+- What guidelines should I follow when writing code that uses RCU?
- See the checklist.txt file in this directory.
+ See the checklist.txt file in this directory.
-o Why the name "RCU"?
+- Why the name "RCU"?
- "RCU" stands for "read-copy update". The file listRCU.txt has
- more information on where this name came from, search for
- "read-copy update" to find it.
+ "RCU" stands for "read-copy update". The file Documentation/RCU/listRCU.rst
+ has more information on where this name came from, search for
+ "read-copy update" to find it.
-o I hear that RCU is patented? What is with that?
+- I hear that RCU is patented? What is with that?
- Yes, it is. There are several known patents related to RCU,
- search for the string "Patent" in RTFP.txt to find them.
- Of these, one was allowed to lapse by the assignee, and the
- others have been contributed to the Linux kernel under GPL.
- There are now also LGPL implementations of user-level RCU
- available (http://liburcu.org/).
+ Yes, it is. There are several known patents related to RCU,
+ search for the string "Patent" in RTFP.txt to find them.
+ Of these, one was allowed to lapse by the assignee, and the
+ others have been contributed to the Linux kernel under GPL.
+ There are now also LGPL implementations of user-level RCU
+ available (http://liburcu.org/).
-o I hear that RCU needs work in order to support realtime kernels?
+- I hear that RCU needs work in order to support realtime kernels?
- Realtime-friendly RCU can be enabled via the CONFIG_PREEMPT_RCU
- kernel configuration parameter.
+ Realtime-friendly RCU can be enabled via the CONFIG_PREEMPT_RCU
+ kernel configuration parameter.
-o Where can I find more information on RCU?
+- Where can I find more information on RCU?
- See the RTFP.txt file in this directory.
- Or point your browser at http://www.rdrop.com/users/paulmck/RCU/.
+ See the RTFP.txt file in this directory.
+ Or point your browser at (http://www.rdrop.com/users/paulmck/RCU/).
--
2.22.0
^ permalink raw reply related
* [Linux-kernel-mentees][PATCH v5 0/5] Documentation: RCU: Convert to reST
From: Jiunn Chang @ 2019-06-26 20:07 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190626191249.21135-1-c0d1n61at3@gmail.com>
This patch series is the initial conversion of the RCU documentation
section. This includes reST markup and renaming txt files to rst. For files
converted, internal links have been created. Checkpatch was used to leverage codespell
for any spelling errors. Each patch in the series has been compiled and reviewed
for warnings and errors. Patches can be bisected.
The changes make in v5 include:
- add reviewed-by tags
The changes made in v4 include:
- Change links in rcu.rst to the path in documentation section
- Maintain the original name of the txt files
The changes made in v3 include:
- correcting markup to maintain even more of the original text
- correcting markup for line breaks
- combining all file renaming into one patch
- add reviewed-by tags
- add required public list to CC
The changes made in v2 include:
- correcting markup to maintain as much of the original text as possible
- correcting markup to reduce reader context switching
- breakout file renaming into individual patches in the series
>8---------------------------------------------------------------------------8<
Jiunn Chang (5):
Documentation: RCU: Convert RCU basic concepts to reST
Documentation: RCU: Convert RCU linked list to reST
Documentation: RCU: Convert RCU UP systems to reST
Documentation: RCU: Rename txt files to rst
Documentation: RCU: Add TOC tree hooks
Documentation/RCU/{UP.txt => UP.rst} | 37 +++++---
Documentation/RCU/index.rst | 19 ++++
.../RCU/{listRCU.txt => listRCU.rst} | 38 ++++----
Documentation/RCU/rcu.rst | 92 +++++++++++++++++++
Documentation/RCU/rcu.txt | 89 ------------------
5 files changed, 156 insertions(+), 119 deletions(-)
rename Documentation/RCU/{UP.txt => UP.rst} (84%)
create mode 100644 Documentation/RCU/index.rst
rename Documentation/RCU/{listRCU.txt => listRCU.rst} (92%)
create mode 100644 Documentation/RCU/rcu.rst
delete mode 100644 Documentation/RCU/rcu.txt
--
2.22.0
^ permalink raw reply
* Re: [PATCH] docs: move gcc_plugins.txt to core-api and rename to .rst
From: Kees Cook @ 2019-06-26 19:59 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Emese Revfy
In-Reply-To: <4937ff4f93282ed57c9859de4300b4d835880ebb.1561556794.git.mchehab+samsung@kernel.org>
On Wed, Jun 26, 2019 at 10:47:46AM -0300, Mauro Carvalho Chehab wrote:
>
>
> The gcc_plugins.txt file is already a ReST file. Move it
> to the core-api book while renaming it.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
-Kees
> ---
> Documentation/{gcc-plugins.txt => core-api/gcc-plugins.rst} | 0
> Documentation/core-api/index.rst | 2 +-
> 2 files changed, 1 insertion(+), 1 deletion(-)
> rename Documentation/{gcc-plugins.txt => core-api/gcc-plugins.rst} (100%)
>
> diff --git a/Documentation/gcc-plugins.txt b/Documentation/core-api/gcc-plugins.rst
> similarity index 100%
> rename from Documentation/gcc-plugins.txt
> rename to Documentation/core-api/gcc-plugins.rst
> diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst
> index 2466a4c51031..d1e5b95bf86d 100644
> --- a/Documentation/core-api/index.rst
> +++ b/Documentation/core-api/index.rst
> @@ -35,7 +35,7 @@ Core utilities
> boot-time-mm
> memory-hotplug
> protection-keys
> -
> + gcc-plugins
>
> Interfaces for kernel debugging
> ===============================
> --
> 2.21.0
>
>
--
Kees Cook
^ permalink raw reply
* Re: On Nitrokey Pro's ECC support
From: Jarkko Sakkinen @ 2019-06-26 19:48 UTC (permalink / raw)
To: Jonathan Corbet; +Cc: Konstantin Ryabitsev, linux-doc, linux-kernel
In-Reply-To: <20190626092817.3e5343e6@lwn.net>
On 2019-06-26 18:28, Jonathan Corbet wrote:
> On Wed, 26 Jun 2019 11:21:38 -0400
> Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
>
>> >Maybe Konstantin (copied) might be willing to supply an update to the
>> >document to reflect this?
>>
>> Hello:
>>
>> I just sent a patch with updates that reflect ECC capabilities in
>> newer
>> devices.
>
> Hey, man, that took you just under an hour to get done. We can't all
> just
> wait around while you twiddle your thumbs... :)
>
> Seriously, though, thanks for doing this,
>
> jon
+1 :-)
/Jarkko
^ permalink raw reply
* [Linux-kernel-mentees][PATCH v4 5/5] Documentation: RCU: Add TOC tree hooks
From: Jiunn Chang @ 2019-06-26 19:12 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190625062627.26378-1-c0d1n61at3@gmail.com>
Add TOC tree hooks for:
- rcu
- listRCU
- UP
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/index.rst | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
create mode 100644 Documentation/RCU/index.rst
diff --git a/Documentation/RCU/index.rst b/Documentation/RCU/index.rst
new file mode 100644
index 000000000000..340a9725676c
--- /dev/null
+++ b/Documentation/RCU/index.rst
@@ -0,0 +1,19 @@
+.. _rcu_concepts:
+
+============
+RCU concepts
+============
+
+.. toctree::
+ :maxdepth: 1
+
+ rcu
+ listRCU
+ UP
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
--
2.22.0
^ permalink raw reply related
* [Linux-kernel-mentees][PATCH v4 4/5] Documentation: RCU: Rename txt files to rst
From: Jiunn Chang @ 2019-06-26 19:12 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190625062627.26378-1-c0d1n61at3@gmail.com>
Rename the following files to reST:
- rcu.txt
- listRCU.txt
- UP.txt
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/{UP.txt => UP.rst} | 0
Documentation/RCU/{listRCU.txt => listRCU.rst} | 0
Documentation/RCU/{rcu.txt => rcu.rst} | 0
3 files changed, 0 insertions(+), 0 deletions(-)
rename Documentation/RCU/{UP.txt => UP.rst} (100%)
rename Documentation/RCU/{listRCU.txt => listRCU.rst} (100%)
rename Documentation/RCU/{rcu.txt => rcu.rst} (100%)
diff --git a/Documentation/RCU/UP.txt b/Documentation/RCU/UP.rst
similarity index 100%
rename from Documentation/RCU/UP.txt
rename to Documentation/RCU/UP.rst
diff --git a/Documentation/RCU/listRCU.txt b/Documentation/RCU/listRCU.rst
similarity index 100%
rename from Documentation/RCU/listRCU.txt
rename to Documentation/RCU/listRCU.rst
diff --git a/Documentation/RCU/rcu.txt b/Documentation/RCU/rcu.rst
similarity index 100%
rename from Documentation/RCU/rcu.txt
rename to Documentation/RCU/rcu.rst
--
2.22.0
^ permalink raw reply
* [Linux-kernel-mentees][PATCH v4 3/5] Documentation: RCU: Convert RCU UP systems to reST
From: Jiunn Chang @ 2019-06-26 19:12 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190625062627.26378-1-c0d1n61at3@gmail.com>
RCU UP systems reST markup.
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/UP.txt | 37 +++++++++++++++++++++++--------------
1 file changed, 23 insertions(+), 14 deletions(-)
diff --git a/Documentation/RCU/UP.txt b/Documentation/RCU/UP.txt
index 53bde717017b..67715a47ae89 100644
--- a/Documentation/RCU/UP.txt
+++ b/Documentation/RCU/UP.txt
@@ -1,17 +1,19 @@
-RCU on Uniprocessor Systems
+.. _up_doc:
+RCU on Uniprocessor Systems
+===========================
A common misconception is that, on UP systems, the call_rcu() primitive
may immediately invoke its function. The basis of this misconception
is that since there is only one CPU, it should not be necessary to
wait for anything else to get done, since there are no other CPUs for
-anything else to be happening on. Although this approach will -sort- -of-
+anything else to be happening on. Although this approach will *sort of*
work a surprising amount of the time, it is a very bad idea in general.
This document presents three examples that demonstrate exactly how bad
an idea this is.
-
Example 1: softirq Suicide
+--------------------------
Suppose that an RCU-based algorithm scans a linked list containing
elements A, B, and C in process context, and can delete elements from
@@ -28,8 +30,8 @@ your kernel.
This same problem can occur if call_rcu() is invoked from a hardware
interrupt handler.
-
Example 2: Function-Call Fatality
+---------------------------------
Of course, one could avert the suicide described in the preceding example
by having call_rcu() directly invoke its arguments only if it was called
@@ -46,11 +48,13 @@ its arguments would cause it to fail to make the fundamental guarantee
underlying RCU, namely that call_rcu() defers invoking its arguments until
all RCU read-side critical sections currently executing have completed.
-Quick Quiz #1: why is it -not- legal to invoke synchronize_rcu() in
- this case?
+Quick Quiz #1:
+ Why is it *not* legal to invoke synchronize_rcu() in this case?
+:ref:`Answers to Quick Quiz <answer_quick_quiz_up>`
Example 3: Death by Deadlock
+----------------------------
Suppose that call_rcu() is invoked while holding a lock, and that the
callback function must acquire this same lock. In this case, if
@@ -76,25 +80,30 @@ there are cases where this can be quite ugly:
If call_rcu() directly invokes the callback, painful locking restrictions
or API changes would be required.
-Quick Quiz #2: What locking restriction must RCU callbacks respect?
+Quick Quiz #2:
+ What locking restriction must RCU callbacks respect?
+:ref:`Answers to Quick Quiz <answer_quick_quiz_up>`
Summary
+-------
Permitting call_rcu() to immediately invoke its arguments breaks RCU,
even on a UP system. So do not do it! Even on a UP system, the RCU
-infrastructure -must- respect grace periods, and -must- invoke callbacks
+infrastructure *must* respect grace periods, and *must* invoke callbacks
from a known environment in which no locks are held.
-Note that it -is- safe for synchronize_rcu() to return immediately on
-UP systems, including !PREEMPT SMP builds running on UP systems.
+Note that it *is* safe for synchronize_rcu() to return immediately on
+UP systems, including PREEMPT SMP builds running on UP systems.
-Quick Quiz #3: Why can't synchronize_rcu() return immediately on
- UP systems running preemptable RCU?
+Quick Quiz #3:
+ Why can't synchronize_rcu() return immediately on UP systems running
+ preemptable RCU?
+.. _answer_quick_quiz_up:
Answer to Quick Quiz #1:
- Why is it -not- legal to invoke synchronize_rcu() in this case?
+ Why is it *not* legal to invoke synchronize_rcu() in this case?
Because the calling function is scanning an RCU-protected linked
list, and is therefore within an RCU read-side critical section.
@@ -119,7 +128,7 @@ Answer to Quick Quiz #2:
This restriction might seem gratuitous, since very few RCU
callbacks acquire locks directly. However, a great many RCU
- callbacks do acquire locks -indirectly-, for example, via
+ callbacks do acquire locks *indirectly*, for example, via
the kfree() primitive.
Answer to Quick Quiz #3:
--
2.22.0
^ permalink raw reply related
* [Linux-kernel-mentees][PATCH v4 2/5] Documentation: RCU: Convert RCU linked list to reST
From: Jiunn Chang @ 2019-06-26 19:12 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190625062627.26378-1-c0d1n61at3@gmail.com>
RCU linked list reST markup.
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/listRCU.txt | 38 ++++++++++++++++++++---------------
1 file changed, 22 insertions(+), 16 deletions(-)
diff --git a/Documentation/RCU/listRCU.txt b/Documentation/RCU/listRCU.txt
index adb5a3782846..7956ff33042b 100644
--- a/Documentation/RCU/listRCU.txt
+++ b/Documentation/RCU/listRCU.txt
@@ -1,5 +1,7 @@
-Using RCU to Protect Read-Mostly Linked Lists
+.. _list_rcu_doc:
+Using RCU to Protect Read-Mostly Linked Lists
+=============================================
One of the best applications of RCU is to protect read-mostly linked lists
("struct list_head" in list.h). One big advantage of this approach
@@ -7,8 +9,8 @@ is that all of the required memory barriers are included for you in
the list macros. This document describes several applications of RCU,
with the best fits first.
-
Example 1: Read-Side Action Taken Outside of Lock, No In-Place Updates
+----------------------------------------------------------------------
The best applications are cases where, if reader-writer locking were
used, the read-side lock would be dropped before taking any action
@@ -24,7 +26,7 @@ added or deleted, rather than being modified in place.
A straightforward example of this use of RCU may be found in the
system-call auditing support. For example, a reader-writer locked
-implementation of audit_filter_task() might be as follows:
+implementation of audit_filter_task() might be as follows::
static enum audit_state audit_filter_task(struct task_struct *tsk)
{
@@ -48,7 +50,7 @@ the corresponding value is returned. By the time that this value is acted
on, the list may well have been modified. This makes sense, since if
you are turning auditing off, it is OK to audit a few extra system calls.
-This means that RCU can be easily applied to the read side, as follows:
+This means that RCU can be easily applied to the read side, as follows::
static enum audit_state audit_filter_task(struct task_struct *tsk)
{
@@ -73,7 +75,7 @@ become list_for_each_entry_rcu(). The _rcu() list-traversal primitives
insert the read-side memory barriers that are required on DEC Alpha CPUs.
The changes to the update side are also straightforward. A reader-writer
-lock might be used as follows for deletion and insertion:
+lock might be used as follows for deletion and insertion::
static inline int audit_del_rule(struct audit_rule *rule,
struct list_head *list)
@@ -106,7 +108,7 @@ lock might be used as follows for deletion and insertion:
return 0;
}
-Following are the RCU equivalents for these two functions:
+Following are the RCU equivalents for these two functions::
static inline int audit_del_rule(struct audit_rule *rule,
struct list_head *list)
@@ -154,13 +156,13 @@ otherwise cause concurrent readers to fail spectacularly.
So, when readers can tolerate stale data and when entries are either added
or deleted, without in-place modification, it is very easy to use RCU!
-
Example 2: Handling In-Place Updates
+------------------------------------
The system-call auditing code does not update auditing rules in place.
However, if it did, reader-writer-locked code to do so might look as
follows (presumably, the field_count is only permitted to decrease,
-otherwise, the added fields would need to be filled in):
+otherwise, the added fields would need to be filled in)::
static inline int audit_upd_rule(struct audit_rule *rule,
struct list_head *list,
@@ -187,7 +189,7 @@ otherwise, the added fields would need to be filled in):
The RCU version creates a copy, updates the copy, then replaces the old
entry with the newly updated entry. This sequence of actions, allowing
concurrent reads while doing a copy to perform an update, is what gives
-RCU ("read-copy update") its name. The RCU code is as follows:
+RCU ("read-copy update") its name. The RCU code is as follows::
static inline int audit_upd_rule(struct audit_rule *rule,
struct list_head *list,
@@ -216,8 +218,8 @@ RCU ("read-copy update") its name. The RCU code is as follows:
Again, this assumes that the caller holds audit_netlink_sem. Normally,
the reader-writer lock would become a spinlock in this sort of code.
-
Example 3: Eliminating Stale Data
+---------------------------------
The auditing examples above tolerate stale data, as do most algorithms
that are tracking external state. Because there is a delay from the
@@ -231,13 +233,16 @@ per-entry spinlock, and, if the "deleted" flag is set, pretends that the
entry does not exist. For this to be helpful, the search function must
return holding the per-entry spinlock, as ipc_lock() does in fact do.
-Quick Quiz: Why does the search function need to return holding the
- per-entry lock for this deleted-flag technique to be helpful?
+Quick Quiz:
+ Why does the search function need to return holding the per-entry lock for
+ this deleted-flag technique to be helpful?
+
+:ref:`Answer to Quick Quiz <answer_quick_quiz_list>`
If the system-call audit module were to ever need to reject stale data,
one way to accomplish this would be to add a "deleted" flag and a "lock"
spinlock to the audit_entry structure, and modify audit_filter_task()
-as follows:
+as follows::
static enum audit_state audit_filter_task(struct task_struct *tsk)
{
@@ -268,7 +273,7 @@ audit_upd_rule() would need additional memory barriers to ensure
that the list_add_rcu() was really executed before the list_del_rcu().
The audit_del_rule() function would need to set the "deleted"
-flag under the spinlock as follows:
+flag under the spinlock as follows::
static inline int audit_del_rule(struct audit_rule *rule,
struct list_head *list)
@@ -290,8 +295,8 @@ flag under the spinlock as follows:
return -EFAULT; /* No matching rule */
}
-
Summary
+-------
Read-mostly list-based data structures that can tolerate stale data are
the most amenable to use of RCU. The simplest case is where entries are
@@ -302,8 +307,9 @@ If stale data cannot be tolerated, then a "deleted" flag may be used
in conjunction with a per-entry spinlock in order to allow the search
function to reject newly deleted data.
+.. _answer_quick_quiz_list:
-Answer to Quick Quiz
+Answer to Quick Quiz:
Why does the search function need to return holding the per-entry
lock for this deleted-flag technique to be helpful?
--
2.22.0
^ permalink raw reply related
* [Linux-kernel-mentees][PATCH v4 1/5] Documentation: RCU: Convert RCU basic concepts to reST
From: Jiunn Chang @ 2019-06-26 19:12 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190625062627.26378-1-c0d1n61at3@gmail.com>
RCU basic concepts reST markup.
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/rcu.txt | 119 +++++++++++++++++++-------------------
1 file changed, 61 insertions(+), 58 deletions(-)
diff --git a/Documentation/RCU/rcu.txt b/Documentation/RCU/rcu.txt
index c818cf65c5a9..8dfb437dacc3 100644
--- a/Documentation/RCU/rcu.txt
+++ b/Documentation/RCU/rcu.txt
@@ -1,5 +1,7 @@
-RCU Concepts
+.. _rcu_doc:
+RCU Concepts
+============
The basic idea behind RCU (read-copy update) is to split destructive
operations into two parts, one that prevents anyone from seeing the data
@@ -8,82 +10,83 @@ A "grace period" must elapse between the two parts, and this grace period
must be long enough that any readers accessing the item being deleted have
since dropped their references. For example, an RCU-protected deletion
from a linked list would first remove the item from the list, wait for
-a grace period to elapse, then free the element. See the listRCU.txt
-file for more information on using RCU with linked lists.
-
+a grace period to elapse, then free the element. See the
+Documentation/RCU/listRCU.rst file for more information on using RCU with
+linked lists.
Frequently Asked Questions
+--------------------------
-o Why would anyone want to use RCU?
+- Why would anyone want to use RCU?
- The advantage of RCU's two-part approach is that RCU readers need
- not acquire any locks, perform any atomic instructions, write to
- shared memory, or (on CPUs other than Alpha) execute any memory
- barriers. The fact that these operations are quite expensive
- on modern CPUs is what gives RCU its performance advantages
- in read-mostly situations. The fact that RCU readers need not
- acquire locks can also greatly simplify deadlock-avoidance code.
+ The advantage of RCU's two-part approach is that RCU readers need
+ not acquire any locks, perform any atomic instructions, write to
+ shared memory, or (on CPUs other than Alpha) execute any memory
+ barriers. The fact that these operations are quite expensive
+ on modern CPUs is what gives RCU its performance advantages
+ in read-mostly situations. The fact that RCU readers need not
+ acquire locks can also greatly simplify deadlock-avoidance code.
-o How can the updater tell when a grace period has completed
- if the RCU readers give no indication when they are done?
+- How can the updater tell when a grace period has completed
+ if the RCU readers give no indication when they are done?
- Just as with spinlocks, RCU readers are not permitted to
- block, switch to user-mode execution, or enter the idle loop.
- Therefore, as soon as a CPU is seen passing through any of these
- three states, we know that that CPU has exited any previous RCU
- read-side critical sections. So, if we remove an item from a
- linked list, and then wait until all CPUs have switched context,
- executed in user mode, or executed in the idle loop, we can
- safely free up that item.
+ Just as with spinlocks, RCU readers are not permitted to
+ block, switch to user-mode execution, or enter the idle loop.
+ Therefore, as soon as a CPU is seen passing through any of these
+ three states, we know that that CPU has exited any previous RCU
+ read-side critical sections. So, if we remove an item from a
+ linked list, and then wait until all CPUs have switched context,
+ executed in user mode, or executed in the idle loop, we can
+ safely free up that item.
- Preemptible variants of RCU (CONFIG_PREEMPT_RCU) get the
- same effect, but require that the readers manipulate CPU-local
- counters. These counters allow limited types of blocking within
- RCU read-side critical sections. SRCU also uses CPU-local
- counters, and permits general blocking within RCU read-side
- critical sections. These variants of RCU detect grace periods
- by sampling these counters.
+ Preemptible variants of RCU (CONFIG_PREEMPT_RCU) get the
+ same effect, but require that the readers manipulate CPU-local
+ counters. These counters allow limited types of blocking within
+ RCU read-side critical sections. SRCU also uses CPU-local
+ counters, and permits general blocking within RCU read-side
+ critical sections. These variants of RCU detect grace periods
+ by sampling these counters.
-o If I am running on a uniprocessor kernel, which can only do one
- thing at a time, why should I wait for a grace period?
+- If I am running on a uniprocessor kernel, which can only do one
+ thing at a time, why should I wait for a grace period?
- See the UP.txt file in this directory.
+ See the Documentation/RCU/UP.rst file for more information.
-o How can I see where RCU is currently used in the Linux kernel?
+- How can I see where RCU is currently used in the Linux kernel?
- Search for "rcu_read_lock", "rcu_read_unlock", "call_rcu",
- "rcu_read_lock_bh", "rcu_read_unlock_bh", "srcu_read_lock",
- "srcu_read_unlock", "synchronize_rcu", "synchronize_net",
- "synchronize_srcu", and the other RCU primitives. Or grab one
- of the cscope databases from:
+ Search for "rcu_read_lock", "rcu_read_unlock", "call_rcu",
+ "rcu_read_lock_bh", "rcu_read_unlock_bh", "srcu_read_lock",
+ "srcu_read_unlock", "synchronize_rcu", "synchronize_net",
+ "synchronize_srcu", and the other RCU primitives. Or grab one
+ of the cscope databases from:
- http://www.rdrop.com/users/paulmck/RCU/linuxusage/rculocktab.html
+ (http://www.rdrop.com/users/paulmck/RCU/linuxusage/rculocktab.html).
-o What guidelines should I follow when writing code that uses RCU?
+- What guidelines should I follow when writing code that uses RCU?
- See the checklist.txt file in this directory.
+ See the checklist.txt file in this directory.
-o Why the name "RCU"?
+- Why the name "RCU"?
- "RCU" stands for "read-copy update". The file listRCU.txt has
- more information on where this name came from, search for
- "read-copy update" to find it.
+ "RCU" stands for "read-copy update". The file Documentation/RCU/listRCU.rst
+ has more information on where this name came from, search for
+ "read-copy update" to find it.
-o I hear that RCU is patented? What is with that?
+- I hear that RCU is patented? What is with that?
- Yes, it is. There are several known patents related to RCU,
- search for the string "Patent" in RTFP.txt to find them.
- Of these, one was allowed to lapse by the assignee, and the
- others have been contributed to the Linux kernel under GPL.
- There are now also LGPL implementations of user-level RCU
- available (http://liburcu.org/).
+ Yes, it is. There are several known patents related to RCU,
+ search for the string "Patent" in RTFP.txt to find them.
+ Of these, one was allowed to lapse by the assignee, and the
+ others have been contributed to the Linux kernel under GPL.
+ There are now also LGPL implementations of user-level RCU
+ available (http://liburcu.org/).
-o I hear that RCU needs work in order to support realtime kernels?
+- I hear that RCU needs work in order to support realtime kernels?
- Realtime-friendly RCU can be enabled via the CONFIG_PREEMPT_RCU
- kernel configuration parameter.
+ Realtime-friendly RCU can be enabled via the CONFIG_PREEMPT_RCU
+ kernel configuration parameter.
-o Where can I find more information on RCU?
+- Where can I find more information on RCU?
- See the RTFP.txt file in this directory.
- Or point your browser at http://www.rdrop.com/users/paulmck/RCU/.
+ See the RTFP.txt file in this directory.
+ Or point your browser at (http://www.rdrop.com/users/paulmck/RCU/).
--
2.22.0
^ permalink raw reply related
* [Linux-kernel-mentees][PATCH v4 0/5] Documentation: RCU: Convert to reST
From: Jiunn Chang @ 2019-06-26 19:12 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190625062627.26378-1-c0d1n61at3@gmail.com>
This patch series is the initial conversion of the RCU documentation
section. This includes reST markup and renaming txt files to rst. For files
converted, internal links have been created. Checkpatch was used to leverage codespell
for any spelling errors. Each patch in the series has been compiled and reviewed
for warnings and errors. Patches can be bisected.
The changes made in v4 include:
- Change links in rcu.rst to the path in documentation section
- Maintain the original name of the txt files
The changes made in v3 include:
- correcting markup to maintain even more of the original text
- correcting markup for line breaks
- combining all file renaming into one patch
- add reviewed-by tags
- add required public list to CC
The changes made in v2 include:
- correcting markup to maintain as much of the original text as possible
- correcting markup to reduce reader context switching
- breakout file renaming into individual patches in the series
>8---------------------------------------------------------------------------8<
Jiunn Chang (5):
Documentation: RCU: Convert RCU basic concepts to reST
Documentation: RCU: Convert RCU linked list to reST
Documentation: RCU: Convert RCU UP systems to reST
Documentation: RCU: Rename txt files to rst
Documentation: RCU: Add TOC tree hooks
Documentation/RCU/{UP.txt => UP.rst} | 37 +++++---
Documentation/RCU/index.rst | 19 ++++
.../RCU/{listRCU.txt => listRCU.rst} | 38 ++++----
Documentation/RCU/rcu.rst | 92 +++++++++++++++++++
Documentation/RCU/rcu.txt | 89 ------------------
5 files changed, 156 insertions(+), 119 deletions(-)
rename Documentation/RCU/{UP.txt => UP.rst} (84%)
create mode 100644 Documentation/RCU/index.rst
rename Documentation/RCU/{listRCU.txt => listRCU.rst} (92%)
create mode 100644 Documentation/RCU/rcu.rst
delete mode 100644 Documentation/RCU/rcu.txt
--
2.22.0
^ permalink raw reply
* Re: [PATCH] docs: filesystems: Remove uneeded .rst extension on toctables
From: Jonathan Corbet @ 2019-06-26 17:54 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Theodore Ts'o, Andreas Dilger, NeilBrown, Darrick J. Wong,
Matthew Wilcox, Christian Brauner, linux-ext4
In-Reply-To: <d2e4dfee7708a3ef6130d3ffcc579429de6a05c9.1561556105.git.mchehab+samsung@kernel.org>
On Wed, 26 Jun 2019 10:35:11 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> There's no need to use a .rst on Sphinx toc tables. As most of
> the Documentation don't use, remove the remaing occurrences.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Applied, thanks.
jon
^ permalink raw reply
* Re: [PATCH v2] scripts/sphinx-pre-install: fix out-of-tree build
From: Jonathan Corbet @ 2019-06-26 17:54 UTC (permalink / raw)
To: Mike Rapoport; +Cc: Mauro Carvalho Chehab, linux-doc
In-Reply-To: <1561353907-19911-1-git-send-email-rppt@linux.ibm.com>
On Mon, 24 Jun 2019 08:25:07 +0300
Mike Rapoport <rppt@linux.ibm.com> wrote:
> Build of htmldocs fails for out-of-tree builds:
>
> $ make V=1 O=~/build/kernel/ htmldocs
> make -C /home/rppt/build/kernel -f /home/rppt/git/linux-docs/Makefile htmldocs
> make[1]: Entering directory '/home/rppt/build/kernel'
> make -f /home/rppt/git/linux-docs/scripts/Makefile.build obj=scripts/basic
> rm -f .tmp_quiet_recordmcount
> make -f /home/rppt/git/linux-docs/scripts/Makefile.build obj=Documentation htmldocs
> Can't open Documentation/conf.py at /home/rppt/git/linux-docs/scripts/sphinx-pre-install line 230.
> /home/rppt/git/linux-docs/Documentation/Makefile:80: recipe for target 'htmldocs' failed
> make[2]: *** [htmldocs] Error 2
>
> The scripts/sphinx-pre-install is trying to open files in the current
> directory which is $KBUILD_OUTPUT rather than in $srctree.
>
> Fix it.
>
> Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
Applied, thanks.
jon
^ permalink raw reply
* Re: [PATCH] docs: zh_CN: submitting-drivers.rst: Remove a duplicated Documentation/
From: Jonathan Corbet @ 2019-06-26 17:53 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Harry Wei, Alex Shi
In-Reply-To: <47f81418930438d1deab8da1307bcd89ba9afd91.1561225663.git.mchehab+samsung@kernel.org>
On Sat, 22 Jun 2019 14:47:46 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> Somehow, this file ended with Documentation/ twice.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> ---
> Documentation/translations/zh_CN/process/submitting-drivers.rst | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/Documentation/translations/zh_CN/process/submitting-drivers.rst b/Documentation/translations/zh_CN/process/submitting-drivers.rst
> index 72c6cd935821..72f4f45c98de 100644
> --- a/Documentation/translations/zh_CN/process/submitting-drivers.rst
> +++ b/Documentation/translations/zh_CN/process/submitting-drivers.rst
> @@ -22,7 +22,7 @@
> 兴趣的是显卡驱动程序，你也许应该访问 XFree86 项目(http://www.xfree86.org/)
> 和／或 X.org 项目 (http://x.org)。
>
> -另请参阅 Documentation/Documentation/translations/zh_CN/process/submitting-patches.rst 文档。
> +另请参阅 Documentation/translations/zh_CN/process/submitting-patches.rst 文档。
There is such a thing as too much Documentation! :)
Applied, thanks.
jon
^ permalink raw reply
* Re: [PATCH v2] Documentation: platform: Delete x86-laptop-drivers.txt
From: Jonathan Corbet @ 2019-06-26 17:41 UTC (permalink / raw)
To: Andy Shevchenko
Cc: Puranjay Mohan, Shuah Khan, Greg KH, Cezary Jackiewicz,
Darren Hart, Andy Shevchenko, Linux Documentation List,
linux-kernel-mentees, Linux Kernel Mailing List, Platform Driver
In-Reply-To: <CAHp75Ve+v7o=Ar=5Vc7yZndCxUNf3sn8YwpCHXMwdeJxuLKMoA@mail.gmail.com>
On Thu, 20 Jun 2019 22:50:17 +0300
Andy Shevchenko <andy.shevchenko@gmail.com> wrote:
> On Thu, Jun 20, 2019 at 9:38 PM Puranjay Mohan <puranjay12@gmail.com> wrote:
> >
> > The list of laptops supported by drivers in PDx86 subsystem is quite
> > big and growing. x86-laptop-drivers.txt contains details of very few
> > laptop models. Remove it because it does not serve any purpose.
> >
>
> Acked-by: Andy Shevchenko <andy.shevchenko@gmail.com>
>
> > Signed-off-by: Puranjay Mohan <puranjay12@gmail.com>
I have applied this, thanks.
jon
^ permalink raw reply
* Re: [PATCH] binfmt_elf: Extract .note.gnu.property from an ELF file
From: Yu-cheng Yu @ 2019-06-26 17:30 UTC (permalink / raw)
To: Andy Lutomirski, Dave Martin
Cc: X86 ML, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, LKML,
open list:DOCUMENTATION, Linux-MM, linux-arch, Linux API,
Arnd Bergmann, Balbir Singh, Cyrill Gorcunov, Dave Hansen,
Eugene Syromiatnikov, Florian Weimer, H.J. Lu, Jann Horn,
Jonathan Corbet, Kees Cook, Mike Kravetz, Nadav Amit,
Oleg Nesterov, Pavel Machek, Peter Zijlstra, Randy Dunlap,
Ravi V. Shankar, Vedvyas Shanbhogue, Szabolcs Nagy, libc-alpha
In-Reply-To: <CALCETrVZCzh+KFCF6ijuf4QEPn=R2gJ8FHLpyFd=n+pNOMMMjA@mail.gmail.com>
On Wed, 2019-06-26 at 10:14 -0700, Andy Lutomirski wrote:
> On Thu, May 2, 2019 at 4:10 AM Dave Martin <Dave.Martin@arm.com> wrote:
> >
> > On Wed, May 01, 2019 at 02:12:17PM -0700, Yu-cheng Yu wrote:
> > > An ELF file's .note.gnu.property indicates features the executable file
> > > can support. For example, the property GNU_PROPERTY_X86_FEATURE_1_AND
> > > indicates the file supports GNU_PROPERTY_X86_FEATURE_1_IBT and/or
> > > GNU_PROPERTY_X86_FEATURE_1_SHSTK.
> > >
[...]
>
> Where did PT_GNU_PROPERTY come from? Are there actual docs for it?
> Can someone here tell us what the actual semantics of this new ELF
> thingy are? From some searching, it seems like it's kind of an ELF
> note but kind of not. An actual description would be fantastic.
>
> Also, I don't think there's any actual requirement that the upstream
> kernel recognize existing CET-enabled RHEL 8 binaries as being
> CET-enabled. I tend to think that RHEL 8 jumped the gun here. While
> the upstream kernel should make some reasonble effort to make sure
> that RHEL 8 binaries will continue to run, I don't see why we need to
> go out of our way to keep the full set of mitigations available for
> binaries that were developed against a non-upstream kernel.
>
> In fact, if we handle the legacy bitmap differently from RHEL 8, we
> may *have* to make sure that we don't recognize existing RHEL 8
> binaries as CET-enabled.
We have worked out the issue. Linux will look at only PT_GNU_PROPERTY, which is
a shortcut pointing directly to .note.gnu.property. I have an updated patch,
and will send it out (although it is not yet perfect).
The Linux gABI extension draft is here: https://github.com/hjl-tools/linux-abi/w
iki/linux-abi-draft.pdf.
Yu-cheng
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox