* RE: [PATCH 2/6] hwspinlock: allow sharing of hwspinlocks
From: Loic PALLARDY @ 2019-07-31 9:22 UTC (permalink / raw)
To: Fabien DESSENNE, Ohad Ben-Cohen, Bjorn Andersson, Rob Herring,
Mark Rutland, Maxime Coquelin, Alexandre TORGUE, Jonathan Corbet,
linux-remoteproc@vger.kernel.org, devicetree@vger.kernel.org,
linux-kernel@vger.kernel.org,
linux-stm32@st-md-mailman.stormreply.com,
linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org
Cc: Fabien DESSENNE, Benjamin GAIGNARD
In-Reply-To: <1552492237-28810-3-git-send-email-fabien.dessenne@st.com>
> -----Original Message-----
> From: linux-remoteproc-owner@vger.kernel.org <linux-remoteproc-
> owner@vger.kernel.org> On Behalf Of Fabien Dessenne
> Sent: mercredi 13 mars 2019 16:51
> To: Ohad Ben-Cohen <ohad@wizery.com>; Bjorn Andersson
> <bjorn.andersson@linaro.org>; Rob Herring <robh+dt@kernel.org>; Mark
> Rutland <mark.rutland@arm.com>; Maxime Coquelin
> <mcoquelin.stm32@gmail.com>; Alexandre TORGUE
> <alexandre.torgue@st.com>; Jonathan Corbet <corbet@lwn.net>; linux-
> remoteproc@vger.kernel.org; devicetree@vger.kernel.org; linux-
> kernel@vger.kernel.org; linux-stm32@st-md-mailman.stormreply.com;
> linux-arm-kernel@lists.infradead.org; linux-doc@vger.kernel.org
> Cc: Fabien DESSENNE <fabien.dessenne@st.com>; Benjamin GAIGNARD
> <benjamin.gaignard@st.com>
> Subject: [PATCH 2/6] hwspinlock: allow sharing of hwspinlocks
>
> The current implementation does not allow different devices to use a
> common hwspinlock. Offer the possibility to use the same hwspinlock by
> several users.
> If a device registers to the framework with #hwlock-cells = 2, then
> the second parameter of the 'hwlocks' DeviceTree property defines
> whether an hwlock is requested for an exclusive or a shared usage.
> If a device registers with #hwlock-cells = 1, then all the hwlocks are
> for an exclusive usage.
>
> Signed-off-by: Fabien Dessenne <fabien.dessenne@st.com>
Looks good for me.
Acked-by: Loic Pallardy <loic.pallardy@st.com>
Regards,
Loic
> ---
> Documentation/hwspinlock.txt | 10 ++--
> drivers/hwspinlock/hwspinlock_core.c | 82
> +++++++++++++++++++++++++-------
> drivers/hwspinlock/hwspinlock_internal.h | 2 +
> 3 files changed, 73 insertions(+), 21 deletions(-)
>
> diff --git a/Documentation/hwspinlock.txt b/Documentation/hwspinlock.txt
> index ed640a2..e6ce2dd 100644
> --- a/Documentation/hwspinlock.txt
> +++ b/Documentation/hwspinlock.txt
> @@ -54,9 +54,11 @@ Should be called from a process context (might sleep).
> struct hwspinlock *hwspin_lock_request_specific(unsigned int id);
>
> Assign a specific hwspinlock id and return its address, or NULL
> -if that hwspinlock is already in use. Usually board code will
> -be calling this function in order to reserve specific hwspinlock
> -ids for predefined purposes.
> +if that hwspinlock is already in use and not shared. If that specific
> +hwspinlock is declared as shared, it can be requested and used by
> +several users.
> +Usually board code will be calling this function in order to reserve
> +specific hwspinlock ids for predefined purposes.
>
> Should be called from a process context (might sleep).
>
> @@ -368,11 +370,13 @@ of which represents a single hardware lock::
> * struct hwspinlock - this struct represents a single hwspinlock
> instance
> * @bank: the hwspinlock_device structure which owns this lock
> * @lock: initialized and used by hwspinlock core
> + * @refcount: number of users (when shared)
> * @priv: private data, owned by the underlying platform-specific
> hwspinlock drv
> */
> struct hwspinlock {
> struct hwspinlock_device *bank;
> spinlock_t lock;
> + unsigned int refcount;
> void *priv;
> };
>
> diff --git a/drivers/hwspinlock/hwspinlock_core.c
> b/drivers/hwspinlock/hwspinlock_core.c
> index 2bad40d..53afdeb 100644
> --- a/drivers/hwspinlock/hwspinlock_core.c
> +++ b/drivers/hwspinlock/hwspinlock_core.c
> @@ -25,6 +25,8 @@
>
> /* radix tree tags */
> #define HWSPINLOCK_UNUSED (0) /* tags an hwspinlock as unused
> */
> +#define HWSPINLOCK_EXCLUSIVE (1) /* tags an hwspinlock as exclusive
> */
> +#define HWSPINLOCK_SHARED (2) /* tags an hwspinlock as shared */
>
> /*
> * A radix tree is used to maintain the available hwspinlock instances.
> @@ -291,7 +293,7 @@ EXPORT_SYMBOL_GPL(__hwspin_unlock);
> * @hwlock_spec: hwlock specifier as found in the device tree
> *
> * This is a simple translation function, suitable for hwspinlock platform
> - * drivers that only has a lock specifier length of 1.
> + * drivers that only has a lock specifier length of 1 or 2.
> *
> * Returns a relative index of the lock within a specified bank on success,
> * or -EINVAL on invalid specifier cell count.
> @@ -299,7 +301,8 @@ EXPORT_SYMBOL_GPL(__hwspin_unlock);
> static inline int
> of_hwspin_lock_simple_xlate(const struct of_phandle_args *hwlock_spec)
> {
> - if (WARN_ON(hwlock_spec->args_count != 1))
> + if (WARN_ON(hwlock_spec->args_count != 1 &&
> + hwlock_spec->args_count != 2))
> return -EINVAL;
>
> return hwlock_spec->args[0];
> @@ -322,11 +325,12 @@ of_hwspin_lock_simple_xlate(const struct
> of_phandle_args *hwlock_spec)
> int of_hwspin_lock_get_id(struct device_node *np, int index)
> {
> struct of_phandle_args args;
> - struct hwspinlock *hwlock;
> + struct hwspinlock *hwlock, *tmp;
> struct radix_tree_iter iter;
> void **slot;
> int id;
> int ret;
> + unsigned int tag;
>
> ret = of_parse_phandle_with_args(np, "hwlocks", "#hwlock-cells",
> index,
> &args);
> @@ -361,6 +365,37 @@ int of_hwspin_lock_get_id(struct device_node *np,
> int index)
> }
> id += hwlock->bank->base_id;
>
> + /* Set the EXCLUSIVE / SHARED tag */
> + if (args.args_count == 2 && args.args[1]) {
> + /* Tag SHARED unless already tagged EXCLUSIVE */
> + if (radix_tree_tag_get(&hwspinlock_tree, id,
> + HWSPINLOCK_EXCLUSIVE)) {
> + ret = -EINVAL;
> + goto out;
> + }
> + tag = HWSPINLOCK_SHARED;
> + } else {
> + /* Tag EXCLUSIVE unless already tagged SHARED */
> + if (radix_tree_tag_get(&hwspinlock_tree, id,
> + HWSPINLOCK_SHARED)) {
> + ret = -EINVAL;
> + goto out;
> + }
> + tag = HWSPINLOCK_EXCLUSIVE;
> + }
> +
> + /* mark this hwspinlock */
> + hwlock = radix_tree_lookup(&hwspinlock_tree, id);
> + if (!hwlock) {
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + tmp = radix_tree_tag_set(&hwspinlock_tree, id, tag);
> +
> + /* self-sanity check which should never fail */
> + WARN_ON(tmp != hwlock);
> +
> out:
> of_node_put(args.np);
> return ret ? ret : id;
> @@ -483,6 +518,7 @@ int hwspin_lock_register(struct hwspinlock_device
> *bank, struct device *dev,
>
> spin_lock_init(&hwlock->lock);
> hwlock->bank = bank;
> + hwlock->refcount = 0;
>
> ret = hwspin_lock_register_single(hwlock, base_id + i);
> if (ret)
> @@ -625,7 +661,7 @@ static int __hwspin_lock_request(struct hwspinlock
> *hwlock)
> {
> struct device *dev = hwlock->bank->dev;
> struct hwspinlock *tmp;
> - int ret;
> + int ret, id;
>
> /* prevent underlying implementation from being removed */
> if (!try_module_get(dev->driver->owner)) {
> @@ -642,13 +678,18 @@ static int __hwspin_lock_request(struct hwspinlock
> *hwlock)
> return ret;
> }
>
> + /* update shareable refcount */
> + id = hwlock_to_id(hwlock);
> + if (radix_tree_tag_get(&hwspinlock_tree, id, HWSPINLOCK_SHARED)
> &&
> + hwlock->refcount++)
> + goto out;
> +
> /* mark hwspinlock as used, should not fail */
> - tmp = radix_tree_tag_clear(&hwspinlock_tree,
> hwlock_to_id(hwlock),
> -
> HWSPINLOCK_UNUSED);
> + tmp = radix_tree_tag_clear(&hwspinlock_tree, id,
> HWSPINLOCK_UNUSED);
>
> /* self-sanity check that should never fail */
> WARN_ON(tmp != hwlock);
> -
> +out:
> return ret;
> }
>
> @@ -742,9 +783,9 @@ struct hwspinlock
> *hwspin_lock_request_specific(unsigned int id)
> /* sanity check (this shouldn't happen) */
> WARN_ON(hwlock_to_id(hwlock) != id);
>
> - /* make sure this hwspinlock is unused */
> - ret = radix_tree_tag_get(&hwspinlock_tree, id,
> HWSPINLOCK_UNUSED);
> - if (ret == 0) {
> + /* make sure this hwspinlock is unused or shareable */
> + if (!radix_tree_tag_get(&hwspinlock_tree, id,
> HWSPINLOCK_SHARED) &&
> + !radix_tree_tag_get(&hwspinlock_tree, id,
> HWSPINLOCK_UNUSED)) {
> pr_warn("hwspinlock %u is already in use\n", id);
> hwlock = NULL;
> goto out;
> @@ -777,7 +818,7 @@ int hwspin_lock_free(struct hwspinlock *hwlock)
> {
> struct device *dev;
> struct hwspinlock *tmp;
> - int ret;
> + int ret, id;
>
> if (!hwlock) {
> pr_err("invalid hwlock\n");
> @@ -788,30 +829,35 @@ int hwspin_lock_free(struct hwspinlock *hwlock)
> mutex_lock(&hwspinlock_tree_lock);
>
> /* make sure the hwspinlock is used */
> - ret = radix_tree_tag_get(&hwspinlock_tree, hwlock_to_id(hwlock),
> -
> HWSPINLOCK_UNUSED);
> + id = hwlock_to_id(hwlock);
> + ret = radix_tree_tag_get(&hwspinlock_tree, id,
> HWSPINLOCK_UNUSED);
> if (ret == 1) {
> dev_err(dev, "%s: hwlock is already free\n", __func__);
> dump_stack();
> ret = -EINVAL;
> - goto out;
> + goto unlock;
> }
>
> /* notify the underlying device that power is not needed */
> ret = pm_runtime_put(dev);
> if (ret < 0)
> - goto out;
> + goto unlock;
> +
> + /* update shareable refcount */
> + if (radix_tree_tag_get(&hwspinlock_tree, id, HWSPINLOCK_SHARED)
> &&
> + --hwlock->refcount)
> + goto put;
>
> /* mark this hwspinlock as available */
> - tmp = radix_tree_tag_set(&hwspinlock_tree, hwlock_to_id(hwlock),
> -
> HWSPINLOCK_UNUSED);
> + tmp = radix_tree_tag_set(&hwspinlock_tree, id,
> HWSPINLOCK_UNUSED);
>
> /* sanity check (this shouldn't happen) */
> WARN_ON(tmp != hwlock);
>
> +put:
> module_put(dev->driver->owner);
>
> -out:
> +unlock:
> mutex_unlock(&hwspinlock_tree_lock);
> return ret;
> }
> diff --git a/drivers/hwspinlock/hwspinlock_internal.h
> b/drivers/hwspinlock/hwspinlock_internal.h
> index 9eb6bd0..c808e11 100644
> --- a/drivers/hwspinlock/hwspinlock_internal.h
> +++ b/drivers/hwspinlock/hwspinlock_internal.h
> @@ -35,11 +35,13 @@ struct hwspinlock_ops {
> * struct hwspinlock - this struct represents a single hwspinlock instance
> * @bank: the hwspinlock_device structure which owns this lock
> * @lock: initialized and used by hwspinlock core
> + * @refcount: number of users (when shared)
> * @priv: private data, owned by the underlying platform-specific hwspinlock
> drv
> */
> struct hwspinlock {
> struct hwspinlock_device *bank;
> spinlock_t lock;
> + unsigned int refcount;
> void *priv;
> };
>
> --
> 2.7.4
^ permalink raw reply
* [PATCH] docs: arm: Remove orphan sh-mobile directory
From: Geert Uytterhoeven @ 2019-07-31 9:02 UTC (permalink / raw)
To: Jonathan Corbet, Mauro Carvalho Chehab, Simon Horman, Magnus Damm
Cc: linux-doc, linux-renesas-soc, linux-kernel, Geert Uytterhoeven
This directory is empty, except for a .gitignore file, listing an
executable file that can no longer be built since commit
c6535e1e0361157e ("Documentation: Remove ZBOOT MMC/SDHI utility and
docs").
Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
---
Documentation/arm/sh-mobile/.gitignore | 1 -
1 file changed, 1 deletion(-)
delete mode 100644 Documentation/arm/sh-mobile/.gitignore
diff --git a/Documentation/arm/sh-mobile/.gitignore b/Documentation/arm/sh-mobile/.gitignore
deleted file mode 100644
index c928dbf3cc8806e2..0000000000000000
--- a/Documentation/arm/sh-mobile/.gitignore
+++ /dev/null
@@ -1 +0,0 @@
-vrl4
--
2.17.1
^ permalink raw reply related
* Re: [PATCH v3 1/2] mm/page_idle: Add per-pid idle page tracking using virtual indexing
From: Minchan Kim @ 2019-07-31 8:53 UTC (permalink / raw)
To: Joel Fernandes (Google)
Cc: linux-kernel, Alexey Dobriyan, Andrew Morton, Brendan Gregg,
Christian Hansen, dancol, fmayer, joaodias, joelaf,
Jonathan Corbet, Kees Cook, kernel-team, linux-api, linux-doc,
linux-fsdevel, linux-mm, Michal Hocko, Mike Rapoport, namhyung,
Roman Gushchin, Stephen Rothwell, surenb, tkjos, Vladimir Davydov,
Vlastimil Babka, wvw
In-Reply-To: <20190726152319.134152-1-joel@joelfernandes.org>
Hi Joel,
On Fri, Jul 26, 2019 at 11:23:18AM -0400, Joel Fernandes (Google) wrote:
> The page_idle tracking feature currently requires looking up the pagemap
> for a process followed by interacting with /sys/kernel/mm/page_idle.
> Looking up PFN from pagemap in Android devices is not supported by
> unprivileged process and requires SYS_ADMIN and gives 0 for the PFN.
>
> This patch adds support to directly interact with page_idle tracking at
> the PID level by introducing a /proc/<pid>/page_idle file. It follows
> the exact same semantics as the global /sys/kernel/mm/page_idle, but now
> looking up PFN through pagemap is not needed since the interface uses
> virtual frame numbers, and at the same time also does not require
> SYS_ADMIN.
>
> In Android, we are using this for the heap profiler (heapprofd) which
> profiles and pin points code paths which allocates and leaves memory
> idle for long periods of time. This method solves the security issue
> with userspace learning the PFN, and while at it is also shown to yield
> better results than the pagemap lookup, the theory being that the window
> where the address space can change is reduced by eliminating the
> intermediate pagemap look up stage. In virtual address indexing, the
> process's mmap_sem is held for the duration of the access.
>
> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
>
> ---
> v2->v3:
> Fixed a bug where I was doing a kfree that is not needed due to not
> needing to do GFP_ATOMIC allocations.
>
> v1->v2:
> Mark swap ptes as idle (Minchan)
> Avoid need for GFP_ATOMIC (Andrew)
> Get rid of idle_page_list lock by moving list to stack
>
> Internal review -> v1:
> Fixes from Suren.
> Corrections to change log, docs (Florian, Sandeep)
>
> fs/proc/base.c | 3 +
> fs/proc/internal.h | 1 +
> fs/proc/task_mmu.c | 57 +++++++
> include/linux/page_idle.h | 4 +
> mm/page_idle.c | 340 +++++++++++++++++++++++++++++++++-----
> 5 files changed, 360 insertions(+), 45 deletions(-)
>
> diff --git a/fs/proc/base.c b/fs/proc/base.c
> index 77eb628ecc7f..a58dd74606e9 100644
> --- a/fs/proc/base.c
> +++ b/fs/proc/base.c
> @@ -3021,6 +3021,9 @@ static const struct pid_entry tgid_base_stuff[] = {
> REG("smaps", S_IRUGO, proc_pid_smaps_operations),
> REG("smaps_rollup", S_IRUGO, proc_pid_smaps_rollup_operations),
> REG("pagemap", S_IRUSR, proc_pagemap_operations),
> +#ifdef CONFIG_IDLE_PAGE_TRACKING
> + REG("page_idle", S_IRUSR|S_IWUSR, proc_page_idle_operations),
> +#endif
> #endif
> #ifdef CONFIG_SECURITY
> DIR("attr", S_IRUGO|S_IXUGO, proc_attr_dir_inode_operations, proc_attr_dir_operations),
> diff --git a/fs/proc/internal.h b/fs/proc/internal.h
> index cd0c8d5ce9a1..bc9371880c63 100644
> --- a/fs/proc/internal.h
> +++ b/fs/proc/internal.h
> @@ -293,6 +293,7 @@ extern const struct file_operations proc_pid_smaps_operations;
> extern const struct file_operations proc_pid_smaps_rollup_operations;
> extern const struct file_operations proc_clear_refs_operations;
> extern const struct file_operations proc_pagemap_operations;
> +extern const struct file_operations proc_page_idle_operations;
>
> extern unsigned long task_vsize(struct mm_struct *);
> extern unsigned long task_statm(struct mm_struct *,
> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> index 4d2b860dbc3f..11ccc53da38e 100644
> --- a/fs/proc/task_mmu.c
> +++ b/fs/proc/task_mmu.c
> @@ -1642,6 +1642,63 @@ const struct file_operations proc_pagemap_operations = {
> .open = pagemap_open,
> .release = pagemap_release,
> };
> +
> +#ifdef CONFIG_IDLE_PAGE_TRACKING
> +static ssize_t proc_page_idle_read(struct file *file, char __user *buf,
> + size_t count, loff_t *ppos)
> +{
> + int ret;
> + struct task_struct *tsk = get_proc_task(file_inode(file));
> +
> + if (!tsk)
> + return -EINVAL;
> + ret = page_idle_proc_read(file, buf, count, ppos, tsk);
> + put_task_struct(tsk);
Why do you need task_struct here? You already got the task in open
and got mm there so you could pass the MM here instead of task.
> + return ret;
> +}
> +
> +static ssize_t proc_page_idle_write(struct file *file, const char __user *buf,
> + size_t count, loff_t *ppos)
> +{
> + int ret;
> + struct task_struct *tsk = get_proc_task(file_inode(file));
> +
> + if (!tsk)
> + return -EINVAL;
> + ret = page_idle_proc_write(file, (char __user *)buf, count, ppos, tsk);
> + put_task_struct(tsk);
> + return ret;
> +}
> +
> +static int proc_page_idle_open(struct inode *inode, struct file *file)
> +{
> + struct mm_struct *mm;
> +
> + mm = proc_mem_open(inode, PTRACE_MODE_READ);
> + if (IS_ERR(mm))
> + return PTR_ERR(mm);
> + file->private_data = mm;
> + return 0;
> +}
> +
> +static int proc_page_idle_release(struct inode *inode, struct file *file)
> +{
> + struct mm_struct *mm = file->private_data;
> +
> + if (mm)
> + mmdrop(mm);
> + return 0;
> +}
> +
> +const struct file_operations proc_page_idle_operations = {
> + .llseek = mem_lseek, /* borrow this */
> + .read = proc_page_idle_read,
> + .write = proc_page_idle_write,
> + .open = proc_page_idle_open,
> + .release = proc_page_idle_release,
> +};
> +#endif /* CONFIG_IDLE_PAGE_TRACKING */
> +
> #endif /* CONFIG_PROC_PAGE_MONITOR */
>
> #ifdef CONFIG_NUMA
> diff --git a/include/linux/page_idle.h b/include/linux/page_idle.h
> index 1e894d34bdce..f1bc2640d85e 100644
> --- a/include/linux/page_idle.h
> +++ b/include/linux/page_idle.h
> @@ -106,6 +106,10 @@ static inline void clear_page_idle(struct page *page)
> }
> #endif /* CONFIG_64BIT */
>
> +ssize_t page_idle_proc_write(struct file *file,
> + char __user *buf, size_t count, loff_t *ppos, struct task_struct *tsk);
> +ssize_t page_idle_proc_read(struct file *file,
> + char __user *buf, size_t count, loff_t *ppos, struct task_struct *tsk);
> #else /* !CONFIG_IDLE_PAGE_TRACKING */
>
> static inline bool page_is_young(struct page *page)
> diff --git a/mm/page_idle.c b/mm/page_idle.c
> index 295512465065..86244f7f1faa 100644
> --- a/mm/page_idle.c
> +++ b/mm/page_idle.c
> @@ -5,12 +5,15 @@
> #include <linux/sysfs.h>
> #include <linux/kobject.h>
> #include <linux/mm.h>
> -#include <linux/mmzone.h>
> -#include <linux/pagemap.h>
> -#include <linux/rmap.h>
> #include <linux/mmu_notifier.h>
> +#include <linux/mmzone.h>
> #include <linux/page_ext.h>
> #include <linux/page_idle.h>
> +#include <linux/pagemap.h>
> +#include <linux/rmap.h>
> +#include <linux/sched/mm.h>
> +#include <linux/swap.h>
> +#include <linux/swapops.h>
>
> #define BITMAP_CHUNK_SIZE sizeof(u64)
> #define BITMAP_CHUNK_BITS (BITMAP_CHUNK_SIZE * BITS_PER_BYTE)
> @@ -25,18 +28,13 @@
> * page tracking. With such an indicator of user pages we can skip isolated
> * pages, but since there are not usually many of them, it will hardly affect
> * the overall result.
> - *
> - * This function tries to get a user memory page by pfn as described above.
> */
> -static struct page *page_idle_get_page(unsigned long pfn)
> +static struct page *page_idle_get_page(struct page *page_in)
Looks weird function name after you changed the argument.
Maybe "bool check_valid_page(struct page *page)"?
> {
> struct page *page;
> pg_data_t *pgdat;
>
> - if (!pfn_valid(pfn))
> - return NULL;
> -
> - page = pfn_to_page(pfn);
> + page = page_in;
> if (!page || !PageLRU(page) ||
> !get_page_unless_zero(page))
> return NULL;
> @@ -51,6 +49,18 @@ static struct page *page_idle_get_page(unsigned long pfn)
> return page;
> }
>
> +/*
> + * This function tries to get a user memory page by pfn as described above.
> + */
> +static struct page *page_idle_get_page_pfn(unsigned long pfn)
So we could use page_idle_get_page name here.
> +{
> +
> + if (!pfn_valid(pfn))
> + return NULL;
page = pfn_to_page(pfn);
return check_valid_page(page) ? page : NULL;
> +
> + return page_idle_get_page(pfn_to_page(pfn));
> +}
> +
> static bool page_idle_clear_pte_refs_one(struct page *page,
> struct vm_area_struct *vma,
> unsigned long addr, void *arg)
> @@ -118,6 +128,47 @@ static void page_idle_clear_pte_refs(struct page *page)
> unlock_page(page);
> }
>
> +/* Helper to get the start and end frame given a pos and count */
> +static int page_idle_get_frames(loff_t pos, size_t count, struct mm_struct *mm,
> + unsigned long *start, unsigned long *end)
> +{
> + unsigned long max_frame;
> +
> + /* If an mm is not given, assume we want physical frames */
> + max_frame = mm ? (mm->task_size >> PAGE_SHIFT) : max_pfn;
> +
> + if (pos % BITMAP_CHUNK_SIZE || count % BITMAP_CHUNK_SIZE)
> + return -EINVAL;
> +
> + *start = pos * BITS_PER_BYTE;
> + if (*start >= max_frame)
> + return -ENXIO;
> +
> + *end = *start + count * BITS_PER_BYTE;
> + if (*end > max_frame)
> + *end = max_frame;
> + return 0;
> +}
> +
> +static bool page_really_idle(struct page *page)
Just minor:
Instead of creating new API, could we combine page_is_idle with
introducing furthere argument pte_check?
bool page_is_idle(struct page *page, bool pte_check);
> +{
> + if (!page)
> + return false;
> +
> + if (page_is_idle(page)) {
> + /*
> + * The page might have been referenced via a
> + * pte, in which case it is not idle. Clear
> + * refs and recheck.
> + */
> + page_idle_clear_pte_refs(page);
> + if (page_is_idle(page))
> + return true;
> + }
> +
> + return false;
> +}
> +
> static ssize_t page_idle_bitmap_read(struct file *file, struct kobject *kobj,
> struct bin_attribute *attr, char *buf,
> loff_t pos, size_t count)
> @@ -125,35 +176,21 @@ static ssize_t page_idle_bitmap_read(struct file *file, struct kobject *kobj,
> u64 *out = (u64 *)buf;
> struct page *page;
> unsigned long pfn, end_pfn;
> - int bit;
> + int bit, ret;
>
> - if (pos % BITMAP_CHUNK_SIZE || count % BITMAP_CHUNK_SIZE)
> - return -EINVAL;
> -
> - pfn = pos * BITS_PER_BYTE;
> - if (pfn >= max_pfn)
> - return 0;
> -
> - end_pfn = pfn + count * BITS_PER_BYTE;
> - if (end_pfn > max_pfn)
> - end_pfn = max_pfn;
> + ret = page_idle_get_frames(pos, count, NULL, &pfn, &end_pfn);
> + if (ret == -ENXIO)
> + return 0; /* Reads beyond max_pfn do nothing */
> + else if (ret)
> + return ret;
>
> for (; pfn < end_pfn; pfn++) {
> bit = pfn % BITMAP_CHUNK_BITS;
> if (!bit)
> *out = 0ULL;
> - page = page_idle_get_page(pfn);
> - if (page) {
> - if (page_is_idle(page)) {
> - /*
> - * The page might have been referenced via a
> - * pte, in which case it is not idle. Clear
> - * refs and recheck.
> - */
> - page_idle_clear_pte_refs(page);
> - if (page_is_idle(page))
> - *out |= 1ULL << bit;
> - }
> + page = page_idle_get_page_pfn(pfn);
> + if (page && page_really_idle(page)) {
> + *out |= 1ULL << bit;
> put_page(page);
> }
> if (bit == BITMAP_CHUNK_BITS - 1)
> @@ -170,23 +207,16 @@ static ssize_t page_idle_bitmap_write(struct file *file, struct kobject *kobj,
> const u64 *in = (u64 *)buf;
> struct page *page;
> unsigned long pfn, end_pfn;
> - int bit;
> + int bit, ret;
>
> - if (pos % BITMAP_CHUNK_SIZE || count % BITMAP_CHUNK_SIZE)
> - return -EINVAL;
> -
> - pfn = pos * BITS_PER_BYTE;
> - if (pfn >= max_pfn)
> - return -ENXIO;
> -
> - end_pfn = pfn + count * BITS_PER_BYTE;
> - if (end_pfn > max_pfn)
> - end_pfn = max_pfn;
> + ret = page_idle_get_frames(pos, count, NULL, &pfn, &end_pfn);
> + if (ret)
> + return ret;
>
> for (; pfn < end_pfn; pfn++) {
> bit = pfn % BITMAP_CHUNK_BITS;
> if ((*in >> bit) & 1) {
> - page = page_idle_get_page(pfn);
> + page = page_idle_get_page_pfn(pfn);
> if (page) {
> page_idle_clear_pte_refs(page);
> set_page_idle(page);
> @@ -224,6 +254,226 @@ struct page_ext_operations page_idle_ops = {
> };
> #endif
>
> +/* page_idle tracking for /proc/<pid>/page_idle */
> +
> +struct page_node {
> + struct page *page;
> + unsigned long addr;
> + struct list_head list;
> +};
> +
> +struct page_idle_proc_priv {
> + unsigned long start_addr;
> + char *buffer;
> + int write;
> +
> + /* Pre-allocate and provide nodes to add_page_idle_list() */
> + struct page_node *page_nodes;
> + int cur_page_node;
> + struct list_head *idle_page_list;
> +};
> +
> +/*
> + * Add a page to the idle page list. page can be NULL if pte is
> + * from a swapped page.
> + */
> +static void add_page_idle_list(struct page *page,
> + unsigned long addr, struct mm_walk *walk)
> +{
> + struct page *page_get = NULL;
> + struct page_node *pn;
> + int bit;
> + unsigned long frames;
> + struct page_idle_proc_priv *priv = walk->private;
> + u64 *chunk = (u64 *)priv->buffer;
> +
> + if (priv->write) {
> + /* Find whether this page was asked to be marked */
> + frames = (addr - priv->start_addr) >> PAGE_SHIFT;
> + bit = frames % BITMAP_CHUNK_BITS;
> + chunk = &chunk[frames / BITMAP_CHUNK_BITS];
> + if (((*chunk >> bit) & 1) == 0)
> + return;
> + }
> +
> + if (page) {
> + page_get = page_idle_get_page(page);
> + if (!page_get)
> + return;
> + }
> +
> + pn = &(priv->page_nodes[priv->cur_page_node++]);
> + pn->page = page_get;
> + pn->addr = addr;
> + list_add(&pn->list, priv->idle_page_list);
> +}
> +
> +static int pte_page_idle_proc_range(pmd_t *pmd, unsigned long addr,
> + unsigned long end,
> + struct mm_walk *walk)
> +{
> + struct vm_area_struct *vma = walk->vma;
> + pte_t *pte;
> + spinlock_t *ptl;
> + struct page *page;
> +
> + ptl = pmd_trans_huge_lock(pmd, vma);
> + if (ptl) {
> + if (pmd_present(*pmd)) {
> + page = follow_trans_huge_pmd(vma, addr, pmd,
> + FOLL_DUMP|FOLL_WRITE);
> + if (!IS_ERR_OR_NULL(page))
> + add_page_idle_list(page, addr, walk);
> + }
> + spin_unlock(ptl);
> + return 0;
> + }
> +
> + if (pmd_trans_unstable(pmd))
> + return 0;
> +
> + pte = pte_offset_map_lock(vma->vm_mm, pmd, addr, &ptl);
> + for (; addr != end; pte++, addr += PAGE_SIZE) {
> + /*
> + * We add swapped pages to the idle_page_list so that we can
> + * reported to userspace that they are idle.
> + */
> + if (is_swap_pte(*pte)) {
I suggested "let's consider every swapped out pages as IDLE" but
let's think about this case:
1. mark heap of the process as IDLE
2. process touch working set
3. process's heap pages are swap out by meory spike or madvise
4. heap profiler investigates the process's IDLE page and surprised all of
heap are idle.
It's the good scenario for other purpose because non-idle pages(IOW,
workingset) could be readahead when the app will restart.
Maybe, squeeze the idle bit in the swap pte to check it.
> + add_page_idle_list(NULL, addr, walk);
> + continue;
> + }
> +
> + if (!pte_present(*pte))
> + continue;
> +
> + page = vm_normal_page(vma, addr, *pte);
> + if (page)
> + add_page_idle_list(page, addr, walk);
> + }
> +
> + pte_unmap_unlock(pte - 1, ptl);
> + return 0;
> +}
> +
> +ssize_t page_idle_proc_generic(struct file *file, char __user *ubuff,
> + size_t count, loff_t *pos,
> + struct task_struct *tsk, int write)
> +{
> + int ret;
> + char *buffer;
> + u64 *out;
> + unsigned long start_addr, end_addr, start_frame, end_frame;
> + struct mm_struct *mm = file->private_data;
> + struct mm_walk walk = { .pmd_entry = pte_page_idle_proc_range, };
> + struct page_node *cur;
> + struct page_idle_proc_priv priv;
> + bool walk_error = false;
> + LIST_HEAD(idle_page_list);
> +
> + if (!mm || !mmget_not_zero(mm))
> + return -EINVAL;
> +
> + if (count > PAGE_SIZE)
> + count = PAGE_SIZE;
> +
> + buffer = kzalloc(PAGE_SIZE, GFP_KERNEL);
> + if (!buffer) {
> + ret = -ENOMEM;
> + goto out_mmput;
> + }
> + out = (u64 *)buffer;
> +
> + if (write && copy_from_user(buffer, ubuff, count)) {
> + ret = -EFAULT;
> + goto out;
> + }
> +
> + ret = page_idle_get_frames(*pos, count, mm, &start_frame, &end_frame);
> + if (ret)
> + goto out;
> +
> + start_addr = (start_frame << PAGE_SHIFT);
> + end_addr = (end_frame << PAGE_SHIFT);
> + priv.buffer = buffer;
> + priv.start_addr = start_addr;
> + priv.write = write;
> +
> + priv.idle_page_list = &idle_page_list;
> + priv.cur_page_node = 0;
> + priv.page_nodes = kzalloc(sizeof(struct page_node) *
> + (end_frame - start_frame), GFP_KERNEL);
> + if (!priv.page_nodes) {
> + ret = -ENOMEM;
> + goto out;
> + }
> +
> + walk.private = &priv;
> + walk.mm = mm;
> +
> + down_read(&mm->mmap_sem);
> +
> + /*
> + * idle_page_list is needed because walk_page_vma() holds ptlock which
> + * deadlocks with page_idle_clear_pte_refs(). So we have to collect all
> + * pages first, and then call page_idle_clear_pte_refs().
> + */
Thanks for the comment, I was curious why you want to have
idle_page_list and the reason is here.
How about making this /proc/<pid>/page_idle per-process granuariy,
unlike system level /sys/xxx/page_idle? What I meant is not to check
rmap to see any reference from random process but just check only
access from the target process. It would be more proper as /proc/
<pid>/ interface and good for per-process tracking as well as
fast.
> + ret = walk_page_range(start_addr, end_addr, &walk);
> + if (ret)
> + walk_error = true;
> +
> + list_for_each_entry(cur, &idle_page_list, list) {
> + int bit, index;
> + unsigned long off;
> + struct page *page = cur->page;
> +
> + if (unlikely(walk_error))
> + goto remove_page;
> +
> + if (write) {
> + if (page) {
> + page_idle_clear_pte_refs(page);
> + set_page_idle(page);
> + }
> + } else {
> + if (!page || page_really_idle(page)) {
> + off = ((cur->addr) >> PAGE_SHIFT) - start_frame;
> + bit = off % BITMAP_CHUNK_BITS;
> + index = off / BITMAP_CHUNK_BITS;
> + out[index] |= 1ULL << bit;
> + }
> + }
> +remove_page:
> + if (page)
> + put_page(page);
> + }
> +
> + if (!write && !walk_error)
> + ret = copy_to_user(ubuff, buffer, count);
> +
> + up_read(&mm->mmap_sem);
> + kfree(priv.page_nodes);
> +out:
> + kfree(buffer);
> +out_mmput:
> + mmput(mm);
> + if (!ret)
> + ret = count;
> + return ret;
> +
> +}
> +
> +ssize_t page_idle_proc_read(struct file *file, char __user *ubuff,
> + size_t count, loff_t *pos, struct task_struct *tsk)
> +{
> + return page_idle_proc_generic(file, ubuff, count, pos, tsk, 0);
> +}
> +
> +ssize_t page_idle_proc_write(struct file *file, char __user *ubuff,
> + size_t count, loff_t *pos, struct task_struct *tsk)
> +{
> + return page_idle_proc_generic(file, ubuff, count, pos, tsk, 1);
> +}
> +
> static int __init page_idle_init(void)
> {
> int err;
> --
> 2.22.0.709.g102302147b-goog
^ permalink raw reply
* Re: [RFC v2 0/6] Introduce TEE based Trusted Keys support
From: Janne Karhunen @ 2019-07-31 7:11 UTC (permalink / raw)
To: Sumit Garg
Cc: keyrings, linux-integrity, linux-security-module, jens.wiklander,
corbet, dhowells, jejb, jarkko.sakkinen, Mimi Zohar, James Morris,
Serge E. Hallyn, Casey Schaufler, ard.biesheuvel, daniel.thompson,
linux-doc, Linux Kernel Mailing List, linux-arm-kernel, tee-dev
In-Reply-To: <1564489420-677-1-git-send-email-sumit.garg@linaro.org>
Hi,
Interesting, I wrote something similar and posted it to the lists a while back:
https://github.com/jkrh/linux/commit/d77ea03afedcb5fd42234cd834da8f8a0809f6a6
Since there are no generic 'TEEs' available, I implemented the same
thing as a generic protocol translator. The shared memory binding for
instance already assumes fair amount about the TEE and how that is
physically present in the system. Besides, the help from usage of shm
is pretty limited due to the size of the keydata.
--
Janne
On Tue, Jul 30, 2019 at 3:26 PM Sumit Garg <sumit.garg@linaro.org> wrote:
>
> Add support for TEE based trusted keys where TEE provides the functionality
> to seal and unseal trusted keys using hardware unique key. Also, this is
> an alternative in case platform doesn't possess a TPM device.
>
> This series also adds some TEE features like:
>
> Patch #1, #2 enables support for registered kernel shared memory with TEE.
>
> Patch #3 enables support for private kernel login method required for
> cases like trusted keys where we don't wan't user-space to directly access
> TEE service to retrieve trusted key contents.
>
> Rest of the patches from #4 to #6 adds support for TEE based trusted keys.
>
> This patch-set has been tested with OP-TEE based pseudo TA which can be
> found here [1].
>
> Also, this patch-set is dependent on generic Trusted Keys framework
> patch-set [2].
>
> [1] https://github.com/OP-TEE/optee_os/pull/3082
> [2] https://lkml.org/lkml/2019/7/18/284
>
> Changes in v2:
> 1. Add reviewed-by tags for patch #1 and #2.
> 2. Incorporate comments from Jens for patch #3.
> 3. Switch to use generic trusted keys framework.
>
> Sumit Garg (6):
> tee: optee: allow kernel pages to register as shm
> tee: enable support to register kernel memory
> tee: add private login method for kernel clients
> KEYS: trusted: Introduce TEE based Trusted Keys
> doc: keys: Document usage of TEE based Trusted Keys
> MAINTAINERS: Add entry for TEE based Trusted Keys
>
> Documentation/security/keys/index.rst | 1 +
> Documentation/security/keys/tee-trusted.rst | 93 +++++++++
> MAINTAINERS | 9 +
> drivers/tee/optee/call.c | 7 +
> drivers/tee/tee_core.c | 6 +
> drivers/tee/tee_shm.c | 16 +-
> include/keys/trusted-type.h | 3 +
> include/keys/trusted_tee.h | 66 +++++++
> include/linux/tee_drv.h | 1 +
> include/uapi/linux/tee.h | 8 +
> security/keys/Kconfig | 3 +
> security/keys/trusted-keys/Makefile | 3 +-
> security/keys/trusted-keys/trusted-tee.c | 282 ++++++++++++++++++++++++++++
> security/keys/trusted-keys/trusted.c | 3 +
> 14 files changed, 498 insertions(+), 3 deletions(-)
> create mode 100644 Documentation/security/keys/tee-trusted.rst
> create mode 100644 include/keys/trusted_tee.h
> create mode 100644 security/keys/trusted-keys/trusted-tee.c
>
> --
> 2.7.4
>
^ permalink raw reply
* Re: [PATCH v3 2/2] doc: Update documentation for page_idle virtual address indexing
From: Mike Rapoport @ 2019-07-31 6:44 UTC (permalink / raw)
To: Joel Fernandes (Google)
Cc: linux-kernel, Alexey Dobriyan, Andrew Morton, Brendan Gregg,
Christian Hansen, dancol, fmayer, joaodias, joelaf,
Jonathan Corbet, Kees Cook, kernel-team, linux-api, linux-doc,
linux-fsdevel, linux-mm, Michal Hocko, minchan, namhyung,
Roman Gushchin, Stephen Rothwell, surenb, tkjos, Vladimir Davydov,
Vlastimil Babka, wvw
In-Reply-To: <20190726152319.134152-2-joel@joelfernandes.org>
On Fri, Jul 26, 2019 at 11:23:19AM -0400, Joel Fernandes (Google) wrote:
> This patch updates the documentation with the new page_idle tracking
> feature which uses virtual address indexing.
>
> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
One nit below, otherwise
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
> ---
> .../admin-guide/mm/idle_page_tracking.rst | 43 ++++++++++++++++---
> 1 file changed, 36 insertions(+), 7 deletions(-)
>
> diff --git a/Documentation/admin-guide/mm/idle_page_tracking.rst b/Documentation/admin-guide/mm/idle_page_tracking.rst
> index df9394fb39c2..1eeac78c94a7 100644
> --- a/Documentation/admin-guide/mm/idle_page_tracking.rst
> +++ b/Documentation/admin-guide/mm/idle_page_tracking.rst
> @@ -19,10 +19,14 @@ It is enabled by CONFIG_IDLE_PAGE_TRACKING=y.
>
> User API
> ========
> +There are 2 ways to access the idle page tracking API. One uses physical
> +address indexing, another uses a simpler virtual address indexing scheme.
>
> -The idle page tracking API is located at ``/sys/kernel/mm/page_idle``.
> -Currently, it consists of the only read-write file,
> -``/sys/kernel/mm/page_idle/bitmap``.
> +Physical address indexing
> +-------------------------
> +The idle page tracking API for physical address indexing using page frame
> +numbers (PFN) is located at ``/sys/kernel/mm/page_idle``. Currently, it
> +consists of the only read-write file, ``/sys/kernel/mm/page_idle/bitmap``.
>
> The file implements a bitmap where each bit corresponds to a memory page. The
> bitmap is represented by an array of 8-byte integers, and the page at PFN #i is
> @@ -74,6 +78,31 @@ See :ref:`Documentation/admin-guide/mm/pagemap.rst <pagemap>` for more
> information about ``/proc/pid/pagemap``, ``/proc/kpageflags``, and
> ``/proc/kpagecgroup``.
>
> +Virtual address indexing
> +------------------------
> +The idle page tracking API for virtual address indexing using virtual page
> +frame numbers (VFN) is located at ``/proc/<pid>/page_idle``. It is a bitmap
> +that follows the same semantics as ``/sys/kernel/mm/page_idle/bitmap``
> +except that it uses virtual instead of physical frame numbers.
Can you please make it more explicit that VFNs are in the <pid>'s address
space?
> +
> +This idle page tracking API does not need deal with PFN so it does not require
> +prior lookups of ``pagemap`` in order to find if page is idle or not. This is
> +an advantage on some systems where looking up PFN is considered a security
> +issue. Also in some cases, this interface could be slightly more reliable to
> +use than physical address indexing, since in physical address indexing, address
> +space changes can occur between reading the ``pagemap`` and reading the
> +``bitmap``, while in virtual address indexing, the process's ``mmap_sem`` is
> +held for the duration of the access.
> +
> +To estimate the amount of pages that are not used by a workload one should:
> +
> + 1. Mark all the workload's pages as idle by setting corresponding bits in
> + ``/proc/<pid>/page_idle``.
> +
> + 2. Wait until the workload accesses its working set.
> +
> + 3. Read ``/proc/<pid>/page_idle`` and count the number of bits set.
> +
> .. _impl_details:
>
> Implementation Details
> @@ -99,10 +128,10 @@ When a dirty page is written to swap or disk as a result of memory reclaim or
> exceeding the dirty memory limit, it is not marked referenced.
>
> The idle memory tracking feature adds a new page flag, the Idle flag. This flag
> -is set manually, by writing to ``/sys/kernel/mm/page_idle/bitmap`` (see the
> -:ref:`User API <user_api>`
> -section), and cleared automatically whenever a page is referenced as defined
> -above.
> +is set manually, by writing to ``/sys/kernel/mm/page_idle/bitmap`` for physical
> +addressing or by writing to ``/proc/<pid>/page_idle`` for virtual
> +addressing (see the :ref:`User API <user_api>` section), and cleared
> +automatically whenever a page is referenced as defined above.
>
> When a page is marked idle, the Accessed bit must be cleared in all PTEs it is
> mapped to, otherwise we will not be able to detect accesses to the page coming
> --
> 2.22.0.709.g102302147b-goog
>
--
Sincerely yours,
Mike.
^ permalink raw reply
* Re: [PATCH v2 25/26] docs: rcu: convert some articles from html to ReST
From: Mauro Carvalho Chehab @ 2019-07-31 1:33 UTC (permalink / raw)
To: Paul E. McKenney
Cc: Josh Triplett, Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan,
Joel Fernandes, Jonathan Corbet, rcu, linux-doc
In-Reply-To: <20190731010623.GN14271@linux.ibm.com>
Em Tue, 30 Jul 2019 18:06:24 -0700
"Paul E. McKenney" <paulmck@linux.ibm.com> escreveu:
> On Tue, Jul 30, 2019 at 09:47:22PM -0300, Mauro Carvalho Chehab wrote:
> > Em Tue, 30 Jul 2019 17:04:55 -0700
> > "Paul E. McKenney" <paulmck@linux.ibm.com> escreveu:
> > > This appears to come from Documentation/output/latex/RCU.tex.
> > > There is nevertheless an RCU.pdf in this directory. It is not
> > > bad, but has a figure full of XML on PDF page 21. And a few later
> > > on as well.
> >
> > PDF output is indeed an issue. The way it works is that it first
> > generates a LaTeX and then it uses texlive to produce the PDF.
>
> Would it be fair to say that html output is what is currently supported,
> and that PDF output is a future thing?
Sure.
Anyway, if you want to fix PDF later, I suspect that simply adding:
.. cssclass:: longtable
Before each quiz table should be enough to fix, as the tables there seem
to be simple enough.
After fixed, the PDF and LaTeX output are usually decent.
> > > On the HTML side, the quick quizzes have immediately visible answers,
> > > which defeats the purpose. The original HTML used a white font,
> > > so that you selected the answer with your mouse to make it visible.
> > >
> > > Can something similar be done with Sphinx? Another approach is to
> > > gather the answers into a separate file and link to them.
> >
> > Yeah, I guess you used a css style that would make the answer visible
> > when the mouse is inside it on your original lwn.net set of articles.
> >
> > Sphinx has a directive to use css, so, the short answer is: yes, you
> > can.
> >
> > For html, you would need to add a css specific for the RCU quiz,
> > placing it under Documentation/sphinx directory. Then, use the
> > ".. css" directive to handle that.
> >
> > You should notice, however, that this will be ignored for
> > LaTeX/pdf output.
> >
> > I guess you can place this on another file, or perhaps place at the
> > end of the document, having a link for the quiz answers.
> >
> > Another alternative would be to make the answer as a footnote.
>
> Making it CSS for HTML and a footnote for PDF seems eminently
> reasonable to me!
You should either do CSS or PDF, as otherwise you will end with dirty
hacks like:
.. only:: html
<some quiz table with answers using css>
.. only: latex
<some quiz table with answers using footnotes>
E. g. you'll need to place the quiz twice, making it harder to maintain
and messier.
Btw, the LaTeX may also parse a css tag, processing it via some custom
macro (with should be added at Documentation/conf.py.
> > > I believe that Joel already noted that internal links are not working.
> > > The external links that I tried work just fine, though. As do the
> > > links from the table of contents.
> >
> > Yeah. Funny enough, when I tested here, they worked fine. Maybe
> > this is due to the Sphinx version I used here at the time I wrote
> > it.
> >
> > Anyway, Joel already submitted a patch addressing this one.
>
> And it works for me, anyway! ;-)
Great!
Thanks,
Mauro
^ permalink raw reply
* Re: [PATCH v2 25/26] docs: rcu: convert some articles from html to ReST
From: Paul E. McKenney @ 2019-07-31 1:06 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Josh Triplett, Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan,
Joel Fernandes, Jonathan Corbet, rcu, linux-doc
In-Reply-To: <20190730214722.53432fe3@coco.lan>
On Tue, Jul 30, 2019 at 09:47:22PM -0300, Mauro Carvalho Chehab wrote:
> Em Tue, 30 Jul 2019 17:04:55 -0700
> "Paul E. McKenney" <paulmck@linux.ibm.com> escreveu:
>
> > On Tue, Jul 30, 2019 at 04:37:20PM -0700, Paul E. McKenney wrote:
> > > On Tue, Jul 30, 2019 at 06:50:51PM -0300, Mauro Carvalho Chehab wrote:
> > > > Em Tue, 30 Jul 2019 14:22:50 -0700
> > > > "Paul E. McKenney" <paulmck@linux.ibm.com> escreveu:
> > > >
> > > > > On Fri, Jul 26, 2019 at 09:51:35AM -0300, Mauro Carvalho Chehab wrote:
> > > > > > There are 4 RCU articles that are written on html format.
> > > > > >
> > > > > > The way they are, they can't be part of the Linux Kernel
> > > > > > documentation body nor share the styles and pdf output.
> > > > > >
> > > > > > So, convert them to ReST format.
> > > > > >
> > > > > > This way, make htmldocs and make pdfdocs will produce a
> > > > > > documentation output that will be like the original ones, but
> > > > > > will be part of the Linux Kernel documentation body.
> > > > > >
> > > > > > Part of the conversion was done with the help of pandoc, but
> > > > > > the result had some broken things that had to be manually
> > > > > > fixed.
> > > > > >
> > > > > > Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> > > > >
> > > > > I am having some trouble applying these, at least in part due to UTF-8
> > > > > sequences, for example double left quotation mark. These end up being
> > > > > "=E2=80=9C", with a few space characters turned into "=20".
> > > > >
> > > > > Any advice on how to apply these?
> > > >
> > > > Didn't notice it ended with UTF-8 chars. It is probably because it came
> > > > from the html conversion.
> > >
> > > Or maybe there are some email issues somewhere along the way.
> > >
> > > > I guess it shouldn't hurt keeping those, but if you prefer I can find
> > > > some time later to replace them.
> > > >
> > > > > Should I just pull commits from somewhere?
> > > >
> > > > Yeah, if you prefer, you can pull from this branch:
> > > >
> > > > https://git.linuxtv.org/mchehab/experimental.git/log/?h=rcu-v1
> > > >
> > > > It has just two patches: the RCU and tools/memory-model ones.
> > > >
> > > > It is based on v5.3-rc2.
> > >
> > > And that does apply, thank you!
> >
> > And "make htmldocs" does produce real HTML! At first glance anyway,
> > quite impressive.
>
> Yeah, its output is pretty decent.
>
> > However, "make pdfdocs" gives me this complaint:
> >
> > ! Dimension too large.
> > \color@b@x ... #3}\kern \fboxsep }\dimen@ \ht \z@
> > \advance \dimen@ \fboxsep ...
> > l.5092 \end{sphinxVerbatim}
> >
> > This appears to come from Documentation/output/latex/RCU.tex.
> > There is nevertheless an RCU.pdf in this directory. It is not
> > bad, but has a figure full of XML on PDF page 21. And a few later
> > on as well.
>
> PDF output is indeed an issue. The way it works is that it first
> generates a LaTeX and then it uses texlive to produce the PDF.
>
> There is a rst2pdf tool with handles it directly, but it is a way more
> problematic. I have an experimental patch with enables it. Maybe
> some day it could be applied, but upstream for the tool needs a lot
> more work.
>
> With regards to the LaTeX/PDF output, on media, we had to tweak the
> documents in order for them to produce a good LaTeX/PDF output on tables.
>
> Usually, before some tables, we add something like this:
>
> .. tabularcolumns:: |p{1.2cm}|p{2.9cm}|p{13.4cm}|
>
> in order to teach LaTeX the size of each column.
>
> If the table is really big, the only way for it to fit is to reduce
> the font size, using a raw LaTeX syntax. See, for example:
>
> Documentation/media/uapi/v4l/dev-subdev.rst
>
> There, we use things like:
>
> .. raw:: latex
>
> \scriptsize
>
> .. tabularcolumns:: |p{2.0cm}|p{2.3cm}|p{2.3cm}|p{2.3cm}|p{2.3cm}|p{2.3cm}|p{2.3cm}|
>
>
> +-----------------+ ...
> | Some long table | ...
> +-----------------+ ...
> ...
> +-----------------+ ...
>
>
> .. raw:: latex
>
> \normalsize
>
> In order to use a small font for the table.
>
> Neither "raw:: latex" nor "tabularcolumns" tag affect html.
>
> There is another trick too. By default, Sphinx LaTeX output uses a
> type of table that should fit into a single page ("tabular").
>
> If the table has more than 30 columns, it switches to another type
> ("longtable"), with can be split into multiple pages.
>
> As the quiz tables usually have only 4 columns, it will always try
> the unbreakable tabular table. If it doesn't fit, PDF will break.
>
> In order to avoid that, just add:
>
> .. cssclass:: longtable
>
> Before the offended table. This will make Sphinx to use LaTeX
> longtable instead of tabular ones.
>
> This won't affect html output.
Would it be fair to say that html output is what is currently supported,
and that PDF output is a future thing?
> > On the HTML side, the quick quizzes have immediately visible answers,
> > which defeats the purpose. The original HTML used a white font,
> > so that you selected the answer with your mouse to make it visible.
> >
> > Can something similar be done with Sphinx? Another approach is to
> > gather the answers into a separate file and link to them.
>
> Yeah, I guess you used a css style that would make the answer visible
> when the mouse is inside it on your original lwn.net set of articles.
>
> Sphinx has a directive to use css, so, the short answer is: yes, you
> can.
>
> For html, you would need to add a css specific for the RCU quiz,
> placing it under Documentation/sphinx directory. Then, use the
> ".. css" directive to handle that.
>
> You should notice, however, that this will be ignored for
> LaTeX/pdf output.
>
> I guess you can place this on another file, or perhaps place at the
> end of the document, having a link for the quiz answers.
>
> Another alternative would be to make the answer as a footnote.
Making it CSS for HTML and a footnote for PDF seems eminently
reasonable to me!
> > I believe that Joel already noted that internal links are not working.
> > The external links that I tried work just fine, though. As do the
> > links from the table of contents.
>
> Yeah. Funny enough, when I tested here, they worked fine. Maybe
> this is due to the Sphinx version I used here at the time I wrote
> it.
>
> Anyway, Joel already submitted a patch addressing this one.
And it works for me, anyway! ;-)
Thanx, Paul
^ permalink raw reply
* Re: [PATCH v2 25/26] docs: rcu: convert some articles from html to ReST
From: Mauro Carvalho Chehab @ 2019-07-31 0:47 UTC (permalink / raw)
To: Paul E. McKenney
Cc: Josh Triplett, Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan,
Joel Fernandes, Jonathan Corbet, rcu, linux-doc
In-Reply-To: <20190731000455.GA11465@linux.ibm.com>
Em Tue, 30 Jul 2019 17:04:55 -0700
"Paul E. McKenney" <paulmck@linux.ibm.com> escreveu:
> On Tue, Jul 30, 2019 at 04:37:20PM -0700, Paul E. McKenney wrote:
> > On Tue, Jul 30, 2019 at 06:50:51PM -0300, Mauro Carvalho Chehab wrote:
> > > Em Tue, 30 Jul 2019 14:22:50 -0700
> > > "Paul E. McKenney" <paulmck@linux.ibm.com> escreveu:
> > >
> > > > On Fri, Jul 26, 2019 at 09:51:35AM -0300, Mauro Carvalho Chehab wrote:
> > > > > There are 4 RCU articles that are written on html format.
> > > > >
> > > > > The way they are, they can't be part of the Linux Kernel
> > > > > documentation body nor share the styles and pdf output.
> > > > >
> > > > > So, convert them to ReST format.
> > > > >
> > > > > This way, make htmldocs and make pdfdocs will produce a
> > > > > documentation output that will be like the original ones, but
> > > > > will be part of the Linux Kernel documentation body.
> > > > >
> > > > > Part of the conversion was done with the help of pandoc, but
> > > > > the result had some broken things that had to be manually
> > > > > fixed.
> > > > >
> > > > > Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> > > >
> > > > I am having some trouble applying these, at least in part due to UTF-8
> > > > sequences, for example double left quotation mark. These end up being
> > > > "=E2=80=9C", with a few space characters turned into "=20".
> > > >
> > > > Any advice on how to apply these?
> > >
> > > Didn't notice it ended with UTF-8 chars. It is probably because it came
> > > from the html conversion.
> >
> > Or maybe there are some email issues somewhere along the way.
> >
> > > I guess it shouldn't hurt keeping those, but if you prefer I can find
> > > some time later to replace them.
> > >
> > > > Should I just pull commits from somewhere?
> > >
> > > Yeah, if you prefer, you can pull from this branch:
> > >
> > > https://git.linuxtv.org/mchehab/experimental.git/log/?h=rcu-v1
> > >
> > > It has just two patches: the RCU and tools/memory-model ones.
> > >
> > > It is based on v5.3-rc2.
> >
> > And that does apply, thank you!
>
> And "make htmldocs" does produce real HTML! At first glance anyway,
> quite impressive.
Yeah, its output is pretty decent.
> However, "make pdfdocs" gives me this complaint:
>
> ! Dimension too large.
> \color@b@x ... #3}\kern \fboxsep }\dimen@ \ht \z@
> \advance \dimen@ \fboxsep ...
> l.5092 \end{sphinxVerbatim}
>
> This appears to come from Documentation/output/latex/RCU.tex.
> There is nevertheless an RCU.pdf in this directory. It is not
> bad, but has a figure full of XML on PDF page 21. And a few later
> on as well.
PDF output is indeed an issue. The way it works is that it first
generates a LaTeX and then it uses texlive to produce the PDF.
There is a rst2pdf tool with handles it directly, but it is a way more
problematic. I have an experimental patch with enables it. Maybe
some day it could be applied, but upstream for the tool needs a lot
more work.
With regards to the LaTeX/PDF output, on media, we had to tweak the
documents in order for them to produce a good LaTeX/PDF output on tables.
Usually, before some tables, we add something like this:
.. tabularcolumns:: |p{1.2cm}|p{2.9cm}|p{13.4cm}|
in order to teach LaTeX the size of each column.
If the table is really big, the only way for it to fit is to reduce
the font size, using a raw LaTeX syntax. See, for example:
Documentation/media/uapi/v4l/dev-subdev.rst
There, we use things like:
.. raw:: latex
\scriptsize
.. tabularcolumns:: |p{2.0cm}|p{2.3cm}|p{2.3cm}|p{2.3cm}|p{2.3cm}|p{2.3cm}|p{2.3cm}|
+-----------------+ ...
| Some long table | ...
+-----------------+ ...
...
+-----------------+ ...
.. raw:: latex
\normalsize
In order to use a small font for the table.
Neither "raw:: latex" nor "tabularcolumns" tag affect html.
There is another trick too. By default, Sphinx LaTeX output uses a
type of table that should fit into a single page ("tabular").
If the table has more than 30 columns, it switches to another type
("longtable"), with can be split into multiple pages.
As the quiz tables usually have only 4 columns, it will always try
the unbreakable tabular table. If it doesn't fit, PDF will break.
In order to avoid that, just add:
.. cssclass:: longtable
Before the offended table. This will make Sphinx to use LaTeX
longtable instead of tabular ones.
This won't affect html output.
> On the HTML side, the quick quizzes have immediately visible answers,
> which defeats the purpose. The original HTML used a white font,
> so that you selected the answer with your mouse to make it visible.
>
> Can something similar be done with Sphinx? Another approach is to
> gather the answers into a separate file and link to them.
Yeah, I guess you used a css style that would make the answer visible
when the mouse is inside it on your original lwn.net set of articles.
Sphinx has a directive to use css, so, the short answer is: yes, you
can.
For html, you would need to add a css specific for the RCU quiz,
placing it under Documentation/sphinx directory. Then, use the
".. css" directive to handle that.
You should notice, however, that this will be ignored for
LaTeX/pdf output.
I guess you can place this on another file, or perhaps place at the
end of the document, having a link for the quiz answers.
Another alternative would be to make the answer as a footnote.
> I believe that Joel already noted that internal links are not working.
> The external links that I tried work just fine, though. As do the
> links from the table of contents.
Yeah. Funny enough, when I tested here, they worked fine. Maybe
this is due to the Sphinx version I used here at the time I wrote
it.
Anyway, Joel already submitted a patch addressing this one.
Thanks,
Mauro
^ permalink raw reply
* Re: [PATCH v2 25/26] docs: rcu: convert some articles from html to ReST
From: Paul E. McKenney @ 2019-07-31 0:04 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Josh Triplett, Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan,
Joel Fernandes, Jonathan Corbet, rcu, linux-doc
In-Reply-To: <20190730233720.GL14271@linux.ibm.com>
On Tue, Jul 30, 2019 at 04:37:20PM -0700, Paul E. McKenney wrote:
> On Tue, Jul 30, 2019 at 06:50:51PM -0300, Mauro Carvalho Chehab wrote:
> > Em Tue, 30 Jul 2019 14:22:50 -0700
> > "Paul E. McKenney" <paulmck@linux.ibm.com> escreveu:
> >
> > > On Fri, Jul 26, 2019 at 09:51:35AM -0300, Mauro Carvalho Chehab wrote:
> > > > There are 4 RCU articles that are written on html format.
> > > >
> > > > The way they are, they can't be part of the Linux Kernel
> > > > documentation body nor share the styles and pdf output.
> > > >
> > > > So, convert them to ReST format.
> > > >
> > > > This way, make htmldocs and make pdfdocs will produce a
> > > > documentation output that will be like the original ones, but
> > > > will be part of the Linux Kernel documentation body.
> > > >
> > > > Part of the conversion was done with the help of pandoc, but
> > > > the result had some broken things that had to be manually
> > > > fixed.
> > > >
> > > > Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> > >
> > > I am having some trouble applying these, at least in part due to UTF-8
> > > sequences, for example double left quotation mark. These end up being
> > > "=E2=80=9C", with a few space characters turned into "=20".
> > >
> > > Any advice on how to apply these?
> >
> > Didn't notice it ended with UTF-8 chars. It is probably because it came
> > from the html conversion.
>
> Or maybe there are some email issues somewhere along the way.
>
> > I guess it shouldn't hurt keeping those, but if you prefer I can find
> > some time later to replace them.
> >
> > > Should I just pull commits from somewhere?
> >
> > Yeah, if you prefer, you can pull from this branch:
> >
> > https://git.linuxtv.org/mchehab/experimental.git/log/?h=rcu-v1
> >
> > It has just two patches: the RCU and tools/memory-model ones.
> >
> > It is based on v5.3-rc2.
>
> And that does apply, thank you!
And "make htmldocs" does produce real HTML! At first glance anyway,
quite impressive.
However, "make pdfdocs" gives me this complaint:
! Dimension too large.
\color@b@x ... #3}\kern \fboxsep }\dimen@ \ht \z@
\advance \dimen@ \fboxsep ...
l.5092 \end{sphinxVerbatim}
This appears to come from Documentation/output/latex/RCU.tex.
There is nevertheless an RCU.pdf in this directory. It is not
bad, but has a figure full of XML on PDF page 21. And a few later
on as well.
On the HTML side, the quick quizzes have immediately visible answers,
which defeats the purpose. The original HTML used a white font,
so that you selected the answer with your mouse to make it visible.
Can something similar be done with Sphinx? Another approach is to
gather the answers into a separate file and link to them.
I believe that Joel already noted that internal links are not working.
The external links that I tried work just fine, though. As do the
links from the table of contents.
Thanx, Paul
^ permalink raw reply
* Re: [PATCH v3 3/3] docs: rcu: Increase toctree to 3
From: Mauro Carvalho Chehab @ 2019-07-30 23:57 UTC (permalink / raw)
To: Joel Fernandes (Google)
Cc: linux-kernel, Jonathan Corbet, Josh Triplett, Lai Jiangshan,
linux-doc, Mathieu Desnoyers, Paul E. McKenney, rcu,
Steven Rostedt
In-Reply-To: <20190730231030.27510-4-joel@joelfernandes.org>
Em Tue, 30 Jul 2019 19:10:30 -0400
"Joel Fernandes (Google)" <joel@joelfernandes.org> escreveu:
> These documents are long and have various sections. Provide a good
> toc nesting level.
>
> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> ---
> Documentation/RCU/index.rst | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/Documentation/RCU/index.rst b/Documentation/RCU/index.rst
> index 94427dc1f23d..5c99185710fa 100644
> --- a/Documentation/RCU/index.rst
> +++ b/Documentation/RCU/index.rst
> @@ -5,7 +5,7 @@ RCU concepts
> ============
>
> .. toctree::
> - :maxdepth: 1
> + :maxdepth: 3
Makes sense to me.
Reviewed-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Thanks,
Mauro
^ permalink raw reply
* Re: [PATCH v3 2/3] docs: rcu: Correct links referring to titles
From: Mauro Carvalho Chehab @ 2019-07-30 23:56 UTC (permalink / raw)
To: Joel Fernandes (Google)
Cc: linux-kernel, Jonathan Corbet, Josh Triplett, Lai Jiangshan,
linux-doc, Mathieu Desnoyers, Paul E. McKenney, rcu,
Steven Rostedt
In-Reply-To: <20190730231030.27510-3-joel@joelfernandes.org>
Em Tue, 30 Jul 2019 19:10:29 -0400
"Joel Fernandes (Google)" <joel@joelfernandes.org> escreveu:
> Mauro's auto conversion broken these links, fix them.
They actually worked here with the Sphinx I used, but yeah, the way it
is is a way cleaner.
>
> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> ---
> .../Tree-RCU-Memory-Ordering.rst | 17 ++--
> .../RCU/Design/Requirements/Requirements.rst | 90 ++++++++-----------
> 2 files changed, 47 insertions(+), 60 deletions(-)
>
> diff --git a/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst b/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
> index 1011b5db1b3d..248b1222f918 100644
> --- a/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
> +++ b/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
> @@ -230,15 +230,14 @@ Tree RCU Grace Period Memory Ordering Components
> Tree RCU's grace-period memory-ordering guarantee is provided by a
> number of RCU components:
>
> -#. `Callback Registry <#Callback%20Registry>`__
> -#. `Grace-Period Initialization <#Grace-Period%20Initialization>`__
> -#. `Self-Reported Quiescent
> - States <#Self-Reported%20Quiescent%20States>`__
> -#. `Dynamic Tick Interface <#Dynamic%20Tick%20Interface>`__
> -#. `CPU-Hotplug Interface <#CPU-Hotplug%20Interface>`__
> -#. `Forcing Quiescent States <Forcing%20Quiescent%20States>`__
> -#. `Grace-Period Cleanup <Grace-Period%20Cleanup>`__
> -#. `Callback Invocation <Callback%20Invocation>`__
> +#. `Callback Registry`_
> +#. `Grace-Period Initialization`_
> +#. `Self-Reported Quiescent States`_
> +#. `Dynamic Tick Interface`_
> +#. `CPU-Hotplug Interface`_
> +#. `Forcing Quiescent States`_
> +#. `Grace-Period Cleanup`_
> +#. `Callback Invocation`_
>
> Each of the following section looks at the corresponding component in
> detail.
> diff --git a/Documentation/RCU/Design/Requirements/Requirements.rst b/Documentation/RCU/Design/Requirements/Requirements.rst
> index 876e0038bb58..a33b5fb331b4 100644
> --- a/Documentation/RCU/Design/Requirements/Requirements.rst
> +++ b/Documentation/RCU/Design/Requirements/Requirements.rst
> @@ -36,16 +36,14 @@ technologies in interesting new ways.
> All that aside, here are the categories of currently known RCU
> requirements:
>
> -#. `Fundamental Requirements <#Fundamental%20Requirements>`__
> -#. `Fundamental Non-Requirements <#Fundamental%20Non-Requirements>`__
> -#. `Parallelism Facts of Life <#Parallelism%20Facts%20of%20Life>`__
> -#. `Quality-of-Implementation
> - Requirements <#Quality-of-Implementation%20Requirements>`__
> -#. `Linux Kernel Complications <#Linux%20Kernel%20Complications>`__
> -#. `Software-Engineering
> - Requirements <#Software-Engineering%20Requirements>`__
> -#. `Other RCU Flavors <#Other%20RCU%20Flavors>`__
> -#. `Possible Future Changes <#Possible%20Future%20Changes>`__
> +#. `Fundamental Requirements`_
> +#. `Fundamental Non-Requirements`_
> +#. `Parallelism Facts of Life`_
> +#. `Quality-of-Implementation Requirements`_
> +#. `Linux Kernel Complications`_
> +#. `Software-Engineering Requirements`_
> +#. `Other RCU Flavors`_
> +#. `Possible Future Changes`_
>
> This is followed by a `summary <#Summary>`__, however, the answers to
> each quick quiz immediately follows the quiz. Select the big white space
> @@ -57,13 +55,11 @@ Fundamental Requirements
> RCU's fundamental requirements are the closest thing RCU has to hard
> mathematical requirements. These are:
>
> -#. `Grace-Period Guarantee <#Grace-Period%20Guarantee>`__
> -#. `Publish-Subscribe Guarantee <#Publish-Subscribe%20Guarantee>`__
> -#. `Memory-Barrier Guarantees <#Memory-Barrier%20Guarantees>`__
> -#. `RCU Primitives Guaranteed to Execute
> - Unconditionally <#RCU%20Primitives%20Guaranteed%20to%20Execute%20Unconditionally>`__
> -#. `Guaranteed Read-to-Write
> - Upgrade <#Guaranteed%20Read-to-Write%20Upgrade>`__
> +#. `Grace-Period Guarantee`_
> +#. `Publish/Subscribe Guarantee`_
> +#. `Memory-Barrier Guarantees`_
> +#. `RCU Primitives Guaranteed to Execute Unconditionally`_
> +#. `Guaranteed Read-to-Write Upgrade`_
>
> Grace-Period Guarantee
> ~~~~~~~~~~~~~~~~~~~~~~
> @@ -689,16 +685,11 @@ infinitely long, however, the following sections list a few
> non-guarantees that have caused confusion. Except where otherwise noted,
> these non-guarantees were premeditated.
>
> -#. `Readers Impose Minimal
> - Ordering <#Readers%20Impose%20Minimal%20Ordering>`__
> -#. `Readers Do Not Exclude
> - Updaters <#Readers%20Do%20Not%20Exclude%20Updaters>`__
> -#. `Updaters Only Wait For Old
> - Readers <#Updaters%20Only%20Wait%20For%20Old%20Readers>`__
> -#. `Grace Periods Don't Partition Read-Side Critical
> - Sections <#Grace%20Periods%20Don't%20Partition%20Read-Side%20Critical%20Sections>`__
> -#. `Read-Side Critical Sections Don't Partition Grace
> - Periods <#Read-Side%20Critical%20Sections%20Don't%20Partition%20Grace%20Periods>`__
> +#. `Readers Impose Minimal Ordering`_
> +#. `Readers Do Not Exclude Updaters`_
> +#. `Updaters Only Wait For Old Readers`_
> +#. `Grace Periods Don't Partition Read-Side Critical Sections`_
> +#. `Read-Side Critical Sections Don't Partition Grace Periods`_
>
> Readers Impose Minimal Ordering
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> @@ -1056,11 +1047,11 @@ it would likely be subject to limitations that would make it
> inappropriate for industrial-strength production use. Classes of
> quality-of-implementation requirements are as follows:
>
> -#. `Specialization <#Specialization>`__
> -#. `Performance and Scalability <#Performance%20and%20Scalability>`__
> -#. `Forward Progress <#Forward%20Progress>`__
> -#. `Composability <#Composability>`__
> -#. `Corner Cases <#Corner%20Cases>`__
> +#. `Specialization`_
> +#. `Performance and Scalability`_
> +#. `Forward Progress`_
> +#. `Composability`_
> +#. `Corner Cases`_
>
> These classes is covered in the following sections.
>
> @@ -1692,21 +1683,18 @@ The Linux kernel provides an interesting environment for all kinds of
> software, including RCU. Some of the relevant points of interest are as
> follows:
>
> -#. `Configuration <#Configuration>`__.
> -#. `Firmware Interface <#Firmware%20Interface>`__.
> -#. `Early Boot <#Early%20Boot>`__.
> -#. `Interrupts and non-maskable interrupts
> - (NMIs) <#Interrupts%20and%20NMIs>`__.
> -#. `Loadable Modules <#Loadable%20Modules>`__.
> -#. `Hotplug CPU <#Hotplug%20CPU>`__.
> -#. `Scheduler and RCU <#Scheduler%20and%20RCU>`__.
> -#. `Tracing and RCU <#Tracing%20and%20RCU>`__.
> -#. `Energy Efficiency <#Energy%20Efficiency>`__.
> -#. `Scheduling-Clock Interrupts and
> - RCU <#Scheduling-Clock%20Interrupts%20and%20RCU>`__.
> -#. `Memory Efficiency <#Memory%20Efficiency>`__.
> -#. `Performance, Scalability, Response Time, and
> - Reliability <#Performance,%20Scalability,%20Response%20Time,%20and%20Reliability>`__.
> +#. `Configuration`_
> +#. `Firmware Interface`_
> +#. `Early Boot`_
> +#. `Interrupts and NMIs`_
> +#. `Loadable Modules`_
> +#. `Hotplug CPU`_
> +#. `Scheduler and RCU`_
> +#. `Tracing and RCU`_
> +#. `Energy Efficiency`_
> +#. `Scheduling-Clock Interrupts and RCU`_
> +#. `Memory Efficiency`_
> +#. `Performance, Scalability, Response Time, and Reliability`_
>
> This list is probably incomplete, but it does give a feel for the most
> notable Linux-kernel complications. Each of the following sections
> @@ -2344,10 +2332,10 @@ implementations, non-preemptible and preemptible. The other four flavors
> are listed below, with requirements for each described in a separate
> section.
>
> -#. `Bottom-Half Flavor (Historical) <#Bottom-Half%20Flavor>`__
> -#. `Sched Flavor (Historical) <#Sched%20Flavor>`__
> -#. `Sleepable RCU <#Sleepable%20RCU>`__
> -#. `Tasks RCU <#Tasks%20RCU>`__
> +#. `Bottom-Half Flavor (Historical)`_
> +#. `Sched Flavor (Historical)`_
> +#. `Sleepable RCU`_
> +#. `Tasks RCU`_
>
> Bottom-Half Flavor (Historical)
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Sounds a lot better to me.
Reviewed-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Thanks,
Mauro
^ permalink raw reply
* Re: [PATCH v2 25/26] docs: rcu: convert some articles from html to ReST
From: Paul E. McKenney @ 2019-07-30 23:37 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Josh Triplett, Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan,
Joel Fernandes, Jonathan Corbet, rcu, linux-doc
In-Reply-To: <20190730185040.3fbc44ca@coco.lan>
On Tue, Jul 30, 2019 at 06:50:51PM -0300, Mauro Carvalho Chehab wrote:
> Em Tue, 30 Jul 2019 14:22:50 -0700
> "Paul E. McKenney" <paulmck@linux.ibm.com> escreveu:
>
> > On Fri, Jul 26, 2019 at 09:51:35AM -0300, Mauro Carvalho Chehab wrote:
> > > There are 4 RCU articles that are written on html format.
> > >
> > > The way they are, they can't be part of the Linux Kernel
> > > documentation body nor share the styles and pdf output.
> > >
> > > So, convert them to ReST format.
> > >
> > > This way, make htmldocs and make pdfdocs will produce a
> > > documentation output that will be like the original ones, but
> > > will be part of the Linux Kernel documentation body.
> > >
> > > Part of the conversion was done with the help of pandoc, but
> > > the result had some broken things that had to be manually
> > > fixed.
> > >
> > > Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> >
> > I am having some trouble applying these, at least in part due to UTF-8
> > sequences, for example double left quotation mark. These end up being
> > "=E2=80=9C", with a few space characters turned into "=20".
> >
> > Any advice on how to apply these?
>
> Didn't notice it ended with UTF-8 chars. It is probably because it came
> from the html conversion.
Or maybe there are some email issues somewhere along the way.
> I guess it shouldn't hurt keeping those, but if you prefer I can find
> some time later to replace them.
>
> > Should I just pull commits from somewhere?
>
> Yeah, if you prefer, you can pull from this branch:
>
> https://git.linuxtv.org/mchehab/experimental.git/log/?h=rcu-v1
>
> It has just two patches: the RCU and tools/memory-model ones.
>
> It is based on v5.3-rc2.
And that does apply, thank you!
Thanx, Paul
^ permalink raw reply
* Re: [PATCH v2 25/26] docs: rcu: convert some articles from html to ReST
From: Joel Fernandes @ 2019-07-30 23:15 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Paul E. McKenney, Josh Triplett, Steven Rostedt,
Mathieu Desnoyers, Lai Jiangshan, Jonathan Corbet, rcu, linux-doc
In-Reply-To: <20190730200057.6ddcc2ce@coco.lan>
On Tue, Jul 30, 2019 at 08:00:57PM -0300, Mauro Carvalho Chehab wrote:
> Em Tue, 30 Jul 2019 18:21:30 -0400
> Joel Fernandes <joel@joelfernandes.org> escreveu:
>
> > On Tue, Jul 30, 2019 at 07:00:28PM -0300, Mauro Carvalho Chehab wrote:
> > > Em Tue, 30 Jul 2019 17:50:07 -0400
> > > Joel Fernandes <joel@joelfernandes.org> escreveu:
> > >
> > > > On Tue, Jul 30, 2019 at 02:22:50PM -0700, Paul E. McKenney wrote:
> > > > > On Fri, Jul 26, 2019 at 09:51:35AM -0300, Mauro Carvalho Chehab wrote:
> > > > > > There are 4 RCU articles that are written on html format.
> > > > > >
> > > > > > The way they are, they can't be part of the Linux Kernel
> > > > > > documentation body nor share the styles and pdf output.
> > > > > >
> > > > > > So, convert them to ReST format.
> > > > > >
> > > > > > This way, make htmldocs and make pdfdocs will produce a
> > > > > > documentation output that will be like the original ones, but
> > > > > > will be part of the Linux Kernel documentation body.
> > > > > >
> > > > > > Part of the conversion was done with the help of pandoc, but
> > > > > > the result had some broken things that had to be manually
> > > > > > fixed.
> > > > > >
> > > > > > Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> > > > >
> > > > > I am having some trouble applying these, at least in part due to UTF-8
> > > > > sequences, for example double left quotation mark. These end up being
> > > > > "=E2=80=9C", with a few space characters turned into "=20".
> > > > >
> > > > > Any advice on how to apply these? Should I just pull commits from
> > > > > somewhere?
> > > >
> > > > I was able to successfully apply and build this particular patch. I think
> > > > this is the only one in the series that applies to RCU.
> > > >
> > > > Sadly, I can't find the patch in any of the public archives, but I could
> > > > perhaps email it to you as an .mbox attach which 'git am' should be able to
> > > > apply.
> > > >
> > > > Mauro did say he was going to add some more details to changelog, or it could
> > > > be added when it is applied:
> > > > https://lore.kernel.org/rcu/20190726154550.5eeae294@coco.lan/
> > >
> > > Yeah, I'm also planning to address at least some of the issues you
> > > pointed, in order to improve the html output, but got sidetracked by something
> > > else and didn't find any time yet to finish. I'm adding some CI automation for
> > > the media subsystem in order to help us dealing with the huge amount of patches
> > > we receive there.
> > >
> > > Feel free to add those details to the changelog. I may find some spare time
> > > this week or the next one for the improvements you suggested, but those
> > > could be sent on followup patches, once done.
> >
> > Ok, I will re-send this RCU patch with the changes, leave this one to me.
> >
> > The other memory model one, needs a lot more work so we can keep that aside
> > for now till someone has the time.
>
> Yeah, feel free to do what fits best with regards to this one.
>
> If you prefer to merge it instead, I'm enclosing the last version of it,
> with that quick hack I just wrote, plus the change of the toctree for
> it to produce the 2-level index.
Ok thanks a lot Mauro! It looks much better to me now. I can probably send
this one out too to Paul after the RCU ones are in, after some more careful
review..
However, I am curious what Paul and Alan think about it since some folks such
as PeterZ actively deal with memory model stuff and probably refer a lot to
this document, but don't like ReST (unlike us) and prefer plain text editors
without any markup. Hopefully this is not the case any more..
thanks,
- Joel
> Thanks,
> Mauro
>
> [PATCH] tools: memory-model: add it to the Documentation body
>
> The books at tools/memory-model/Documentation are very well
> formatted. Congrats to the ones that wrote them!
>
> The manual conversion to ReST is really trivial:
>
> - Add document titles;
> - change the bullets on some lists;
> - mark code blocks.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
>
> diff --git a/Documentation/core-api/refcount-vs-atomic.rst b/Documentation/core-api/refcount-vs-atomic.rst
> index 976e85adffe8..7e6500a6fa76 100644
> --- a/Documentation/core-api/refcount-vs-atomic.rst
> +++ b/Documentation/core-api/refcount-vs-atomic.rst
> @@ -17,7 +17,7 @@ in order to help maintainers validate their code against the change in
> these memory ordering guarantees.
>
> The terms used through this document try to follow the formal LKMM defined in
> -tools/memory-model/Documentation/explanation.txt.
> +tools/memory-model/Documentation/explanation.rst.
>
> memory-barriers.txt and atomic_t.txt provide more background to the
> memory ordering in general and for atomic operations specifically.
> diff --git a/Documentation/index.rst b/Documentation/index.rst
> index 2df5a3da563c..5123770ba77e 100644
> --- a/Documentation/index.rst
> +++ b/Documentation/index.rst
> @@ -36,6 +36,7 @@ trying to get it to work optimally on a given system.
>
> admin-guide/index
> kbuild/index
> + tools/index
>
> Firmware-related documentation
> ------------------------------
> diff --git a/Documentation/tools/index.rst b/Documentation/tools/index.rst
> new file mode 100644
> index 000000000000..f540d9cc75a1
> --- /dev/null
> +++ b/Documentation/tools/index.rst
> @@ -0,0 +1,17 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +===========
> +Linux Tools
> +===========
> +
> +.. toctree::
> + :maxdepth: 2
> +
> + memory-model/index
> +
> +.. only:: subproject and html
> +
> + Indices
> + =======
> +
> + * :ref:`genindex`
> diff --git a/Documentation/tools/memory-model b/Documentation/tools/memory-model
> new file mode 120000
> index 000000000000..020ed38ce302
> --- /dev/null
> +++ b/Documentation/tools/memory-model
> @@ -0,0 +1 @@
> +../../tools/memory-model/Documentation/
> \ No newline at end of file
> diff --git a/tools/memory-model/Documentation/cheatsheet.rst b/tools/memory-model/Documentation/cheatsheet.rst
> new file mode 100644
> index 000000000000..35f8749b8a53
> --- /dev/null
> +++ b/tools/memory-model/Documentation/cheatsheet.rst
> @@ -0,0 +1,36 @@
> +===========
> +Cheat Sheet
> +===========
> +
> +::
> +
> + Prior Operation Subsequent Operation
> + --------------- ---------------------------
> + C Self R W RMW Self R W DR DW RMW SV
> + -- ---- - - --- ---- - - -- -- --- --
> +
> + Store, e.g., WRITE_ONCE() Y Y
> + Load, e.g., READ_ONCE() Y Y Y Y
> + Unsuccessful RMW operation Y Y Y Y
> + rcu_dereference() Y Y Y Y
> + Successful *_acquire() R Y Y Y Y Y Y
> + Successful *_release() C Y Y Y W Y
> + smp_rmb() Y R Y Y R
> + smp_wmb() Y W Y Y W
> + smp_mb() & synchronize_rcu() CP Y Y Y Y Y Y Y Y
> + Successful full non-void RMW CP Y Y Y Y Y Y Y Y Y Y Y
> + smp_mb__before_atomic() CP Y Y Y a a a a Y
> + smp_mb__after_atomic() CP a a Y Y Y Y Y Y
> +
> +
> + Key: C: Ordering is cumulative
> + P: Ordering propagates
> + R: Read, for example, READ_ONCE(), or read portion of RMW
> + W: Write, for example, WRITE_ONCE(), or write portion of RMW
> + Y: Provides ordering
> + a: Provides ordering given intervening RMW atomic operation
> + DR: Dependent read (address dependency)
> + DW: Dependent write (address, data, or control dependency)
> + RMW: Atomic read-modify-write operation
> + SELF: Orders self, as opposed to accesses before and/or after
> + SV: Orders later accesses to the same variable
> diff --git a/tools/memory-model/Documentation/cheatsheet.txt b/tools/memory-model/Documentation/cheatsheet.txt
> deleted file mode 100644
> index 33ba98d72b16..000000000000
> --- a/tools/memory-model/Documentation/cheatsheet.txt
> +++ /dev/null
> @@ -1,30 +0,0 @@
> - Prior Operation Subsequent Operation
> - --------------- ---------------------------
> - C Self R W RMW Self R W DR DW RMW SV
> - -- ---- - - --- ---- - - -- -- --- --
> -
> -Store, e.g., WRITE_ONCE() Y Y
> -Load, e.g., READ_ONCE() Y Y Y Y
> -Unsuccessful RMW operation Y Y Y Y
> -rcu_dereference() Y Y Y Y
> -Successful *_acquire() R Y Y Y Y Y Y
> -Successful *_release() C Y Y Y W Y
> -smp_rmb() Y R Y Y R
> -smp_wmb() Y W Y Y W
> -smp_mb() & synchronize_rcu() CP Y Y Y Y Y Y Y Y
> -Successful full non-void RMW CP Y Y Y Y Y Y Y Y Y Y Y
> -smp_mb__before_atomic() CP Y Y Y a a a a Y
> -smp_mb__after_atomic() CP a a Y Y Y Y Y Y
> -
> -
> -Key: C: Ordering is cumulative
> - P: Ordering propagates
> - R: Read, for example, READ_ONCE(), or read portion of RMW
> - W: Write, for example, WRITE_ONCE(), or write portion of RMW
> - Y: Provides ordering
> - a: Provides ordering given intervening RMW atomic operation
> - DR: Dependent read (address dependency)
> - DW: Dependent write (address, data, or control dependency)
> - RMW: Atomic read-modify-write operation
> - SELF: Orders self, as opposed to accesses before and/or after
> - SV: Orders later accesses to the same variable
> diff --git a/tools/memory-model/Documentation/explanation.txt b/tools/memory-model/Documentation/explanation.rst
> similarity index 92%
> rename from tools/memory-model/Documentation/explanation.txt
> rename to tools/memory-model/Documentation/explanation.rst
> index 68caa9a976d0..9b5d10cef0c2 100644
> --- a/tools/memory-model/Documentation/explanation.txt
> +++ b/tools/memory-model/Documentation/explanation.rst
> @@ -1,5 +1,6 @@
> +========================================================
> Explanation of the Linux-Kernel Memory Consistency Model
> -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +========================================================
>
> :Author: Alan Stern <stern@rowland.harvard.edu>
> :Created: October 2017
> @@ -107,7 +108,7 @@ and the flag are memory locations shared between the two CPUs.
>
> We can abstract out the important pieces of the driver code as follows
> (the reason for using WRITE_ONCE() and READ_ONCE() instead of simple
> -assignment statements is discussed later):
> +assignment statements is discussed later)::
>
> int buf = 0, flag = 0;
>
> @@ -219,7 +220,7 @@ Ordering). This model predicts that the undesired outcome for the MP
> pattern cannot occur, but in other respects it differs from Sequential
> Consistency. One example is the Store Buffer (SB) pattern, in which
> each CPU stores to its own shared location and then loads from the
> -other CPU's location:
> +other CPU's location::
>
> int x = 0, y = 0;
>
> @@ -264,7 +265,7 @@ The counterpart to ordering is a cycle. Ordering rules out cycles:
> It's not possible to have X ordered before Y, Y ordered before Z, and
> Z ordered before X, because this would mean that X is ordered before
> itself. The analysis of the MP example under Sequential Consistency
> -involved just such an impossible cycle:
> +involved just such an impossible cycle::
>
> W: P0 stores 1 to flag executes before
> X: P1 loads 1 from flag executes before
> @@ -331,7 +332,7 @@ can think of it as the order in which statements occur in the source
> code after branches are taken into account and loops have been
> unrolled. A better description might be the order in which
> instructions are presented to a CPU's execution unit. Thus, we say
> -that X is po-before Y (written as "X ->po Y" in formulas) if X occurs
> +that X is po-before Y (written as ``X ->po Y`` in formulas) if X occurs
> before Y in the instruction stream.
>
> This is inherently a single-CPU relation; two instructions executing
> @@ -388,7 +389,7 @@ The protections provided by READ_ONCE(), WRITE_ONCE(), and others are
> not perfect; and under some circumstances it is possible for the
> compiler to undermine the memory model. Here is an example. Suppose
> both branches of an "if" statement store the same value to the same
> -location:
> +location::
>
> r1 = READ_ONCE(x);
> if (r1) {
> @@ -402,7 +403,7 @@ location:
> For this code, the LKMM predicts that the load from x will always be
> executed before either of the stores to y. However, a compiler could
> lift the stores out of the conditional, transforming the code into
> -something resembling:
> +something resembling::
>
> r1 = READ_ONCE(x);
> WRITE_ONCE(y, 2);
> @@ -418,7 +419,7 @@ model's original prediction could be invalidated by the compiler.
>
> Another issue arises from the fact that in C, arguments to many
> operators and function calls can be evaluated in any order. For
> -example:
> +example::
>
> r1 = f(5) + g(6);
>
> @@ -440,7 +441,7 @@ control (ctrl).
>
> A read and a write event are linked by a data dependency if the value
> obtained by the read affects the value stored by the write. As a very
> -simple example:
> +simple example::
>
> int x, y;
>
> @@ -455,7 +456,7 @@ values of multiple reads.
> A read event and another memory access event are linked by an address
> dependency if the value obtained by the read affects the location
> accessed by the other event. The second event can be either a read or
> -a write. Here's another simple example:
> +a write. Here's another simple example::
>
> int a[20];
> int i;
> @@ -471,7 +472,7 @@ pointer.
>
> Finally, a read event and another memory access event are linked by a
> control dependency if the value obtained by the read affects whether
> -the second event is executed at all. Simple example:
> +the second event is executed at all. Simple example::
>
> int x, y;
>
> @@ -484,9 +485,9 @@ which depends on the value obtained by the READ_ONCE(); hence there is
> a control dependency from the load to the store.
>
> It should be pretty obvious that events can only depend on reads that
> -come earlier in program order. Symbolically, if we have R ->data X,
> -R ->addr X, or R ->ctrl X (where R is a read event), then we must also
> -have R ->po X. It wouldn't make sense for a computation to depend
> +come earlier in program order. Symbolically, if we have ``R ->data X``,
> +``R ->addr X``, or ``R ->ctrl X`` (where R is a read event), then we must also
> +have ``R ->po X``. It wouldn't make sense for a computation to depend
> somehow on a value that doesn't get loaded from shared memory until
> later in the code!
>
> @@ -497,7 +498,7 @@ THE READS-FROM RELATION: rf, rfi, and rfe
> The reads-from relation (rf) links a write event to a read event when
> the value loaded by the read is the value that was stored by the
> write. In colloquial terms, the load "reads from" the store. We
> -write W ->rf R to indicate that the load R reads from the store W. We
> +write ``W ->rf R`` to indicate that the load R reads from the store W. We
> further distinguish the cases where the load and the store occur on
> the same CPU (internal reads-from, or rfi) and where they occur on
> different CPUs (external reads-from, or rfe).
> @@ -510,7 +511,7 @@ Usage of the rf relation implicitly assumes that loads will always
> read from a single store. It doesn't apply properly in the presence
> of load-tearing, where a load obtains some of its bits from one store
> and some of them from another store. Fortunately, use of READ_ONCE()
> -and WRITE_ONCE() will prevent load-tearing; it's not possible to have:
> +and WRITE_ONCE() will prevent load-tearing; it's not possible to have::
>
> int x = 0;
>
> @@ -530,7 +531,7 @@ and end up with r1 = 0x1200 (partly from x's initial value and partly
> from the value stored by P0).
>
> On the other hand, load-tearing is unavoidable when mixed-size
> -accesses are used. Consider this example:
> +accesses are used. Consider this example::
>
> union {
> u32 w;
> @@ -578,26 +579,26 @@ that value comes third, and so on.
> You can think of the coherence order as being the order in which the
> stores reach x's location in memory (or if you prefer a more
> hardware-centric view, the order in which the stores get written to
> -x's cache line). We write W ->co W' if W comes before W' in the
> +x's cache line). We write ``W ->co W'`` if W comes before W' in the
> coherence order, that is, if the value stored by W gets overwritten,
> directly or indirectly, by the value stored by W'.
>
> Coherence order is required to be consistent with program order. This
> requirement takes the form of four coherency rules:
>
> - Write-write coherence: If W ->po-loc W' (i.e., W comes before
> + Write-write coherence: If ``W ->po-loc W'`` (i.e., W comes before
> W' in program order and they access the same location), where W
> - and W' are two stores, then W ->co W'.
> + and W' are two stores, then ``W ->co W'``.
>
> - Write-read coherence: If W ->po-loc R, where W is a store and R
> + Write-read coherence: If ``W ->po-loc R``, where W is a store and R
> is a load, then R must read from W or from some other store
> which comes after W in the coherence order.
>
> - Read-write coherence: If R ->po-loc W, where R is a load and W
> + Read-write coherence: If ``R ->po-loc W``, where R is a load and W
> is a store, then the store which R reads from must come before
> W in the coherence order.
>
> - Read-read coherence: If R ->po-loc R', where R and R' are two
> + Read-read coherence: If ``R ->po-loc R'``, where R and R' are two
> loads, then either they read from the same store or else the
> store read by R comes before the store read by R' in the
> coherence order.
> @@ -612,7 +613,7 @@ requirement that every store eventually becomes visible to every CPU.)
> Any reasonable memory model will include cache coherence. Indeed, our
> expectation of cache coherence is so deeply ingrained that violations
> of its requirements look more like hardware bugs than programming
> -errors:
> +errors::
>
> int x;
>
> @@ -628,6 +629,8 @@ write-write coherence rule: Since the store of 23 comes later in
> program order, it must also come later in x's coherence order and
> thus must overwrite the store of 17.
>
> +::
> +
> int x = 0;
>
> P0()
> @@ -644,6 +647,8 @@ program order, so it must not read from that store but rather from one
> coming earlier in the coherence order (in this case, x's initial
> value).
>
> +::
> +
> int x = 0;
>
> P0()
> @@ -689,12 +694,12 @@ THE FROM-READS RELATION: fr, fri, and fre
>
> The from-reads relation (fr) can be a little difficult for people to
> grok. It describes the situation where a load reads a value that gets
> -overwritten by a store. In other words, we have R ->fr W when the
> +overwritten by a store. In other words, we have ``R ->fr W`` when the
> value that R reads is overwritten (directly or indirectly) by W, or
> equivalently, when R reads from a store which comes earlier than W in
> the coherence order.
>
> -For example:
> +For example::
>
> int x = 0;
>
> @@ -718,9 +723,11 @@ different CPUs).
>
> Note that the fr relation is determined entirely by the rf and co
> relations; it is not independent. Given a read event R and a write
> -event W for the same location, we will have R ->fr W if and only if
> +event W for the same location, we will have ``R ->fr W`` if and only if
> the write which R reads from is co-before W. In symbols,
>
> +::
> +
> (R ->fr W) := (there exists W' with W' ->rf R and W' ->co W).
>
>
> @@ -843,13 +850,13 @@ defined to link memory access events E and F whenever:
> event occurs between them in program order; or
>
> F is a release fence and some X comes before F in program order,
> - where either X = E or else E ->rf X; or
> + where either ``X = E`` or else ``E ->rf X``; or
>
> A strong fence event occurs between some X and F in program
> - order, where either X = E or else E ->rf X.
> + order, where either ``X = E`` or else ``E ->rf X``.
>
> The operational model requires that whenever W and W' are both stores
> -and W ->cumul-fence W', then W must propagate to any given CPU
> +and ``W ->cumul-fence W'``, then W must propagate to any given CPU
> before W' does. However, for different CPUs C and C', it does not
> require W to propagate to C before W' propagates to C'.
>
> @@ -903,11 +910,11 @@ first for CPU 0, then CPU 1, etc.
>
> You can check that the four coherency rules imply that the rf, co, fr,
> and po-loc relations agree with this global ordering; in other words,
> -whenever we have X ->rf Y or X ->co Y or X ->fr Y or X ->po-loc Y, the
> +whenever we have ``X ->rf Y`` or ``X ->co Y`` or ``X ->fr Y`` or ``X ->po-loc Y``, the
> X event comes before the Y event in the global ordering. The LKMM's
> "coherence" axiom expresses this by requiring the union of these
> relations not to have any cycles. This means it must not be possible
> -to find events
> +to find events::
>
> X0 -> X1 -> X2 -> ... -> Xn -> X0,
>
> @@ -929,7 +936,7 @@ this case) does not get altered between the read and the write events
> making up the atomic operation. In particular, if two CPUs perform
> atomic_inc(&x) concurrently, it must be guaranteed that the final
> value of x will be the initial value plus two. We should never have
> -the following sequence of events:
> +the following sequence of events::
>
> CPU 0 loads x obtaining 13;
> CPU 1 loads x obtaining 13;
> @@ -951,6 +958,8 @@ atomic read-modify-write and W' is the write event which R reads from,
> there must not be any stores coming between W' and W in the coherence
> order. Equivalently,
>
> +::
> +
> (R ->rmw W) implies (there is no X with R ->fr X and X ->co W),
>
> where the rmw relation links the read and write events making up each
> @@ -968,7 +977,7 @@ po.
>
> The operational model already includes a description of one such
> situation: Fences are a source of ppo links. Suppose X and Y are
> -memory accesses with X ->po Y; then the CPU must execute X before Y if
> +memory accesses with ``X ->po Y``; then the CPU must execute X before Y if
> any of the following hold:
>
> A strong (smp_mb() or synchronize_rcu()) fence occurs between
> @@ -987,7 +996,7 @@ any of the following hold:
> Another possibility, not mentioned earlier but discussed in the next
> section, is:
>
> - X and Y are both loads, X ->addr Y (i.e., there is an address
> + X and Y are both loads, ``X ->addr Y`` (i.e., there is an address
> dependency from X to Y), and X is a READ_ONCE() or an atomic
> access.
>
> @@ -1021,7 +1030,7 @@ includes address dependencies to loads in the ppo relation.
>
> On the other hand, dependencies can indirectly affect the ordering of
> two loads. This happens when there is a dependency from a load to a
> -store and a second, po-later load reads from that store:
> +store and a second, po-later load reads from that store::
>
> R ->dep W ->rfi R',
>
> @@ -1036,7 +1045,7 @@ R; if the speculation turns out to be wrong then the CPU merely has to
> restart or abandon R'.
>
> (In theory, a CPU might forward a store to a load when it runs across
> -an address dependency like this:
> +an address dependency like this::
>
> r1 = READ_ONCE(ptr);
> WRITE_ONCE(*r1, 17);
> @@ -1048,7 +1057,7 @@ However, none of the architectures supported by the Linux kernel do
> this.)
>
> Two memory accesses of the same location must always be executed in
> -program order if the second access is a store. Thus, if we have
> +program order if the second access is a store. Thus, if we have::
>
> R ->po-loc W
>
> @@ -1056,7 +1065,7 @@ program order if the second access is a store. Thus, if we have
> access the same location), the CPU is obliged to execute W after R.
> If it executed W first then the memory subsystem would respond to R's
> read request with the value stored by W (or an even later store), in
> -violation of the read-write coherence rule. Similarly, if we had
> +violation of the read-write coherence rule. Similarly, if we had::
>
> W ->po-loc W'
>
> @@ -1074,7 +1083,7 @@ AND THEN THERE WAS ALPHA
>
> As mentioned above, the Alpha architecture is unique in that it does
> not appear to respect address dependencies to loads. This means that
> -code such as the following:
> +code such as the following::
>
> int x = 0;
> int y = -1;
> @@ -1128,7 +1137,7 @@ nothing at all on non-Alpha builds) after every READ_ONCE() and atomic
> load. The effect of the fence is to cause the CPU not to execute any
> po-later instructions until after the local cache has finished
> processing all the stores it has already received. Thus, if the code
> -was changed to:
> +was changed to::
>
> P1()
> {
> @@ -1167,25 +1176,25 @@ The happens-before relation (hb) links memory accesses that have to
> execute in a certain order. hb includes the ppo relation and two
> others, one of which is rfe.
>
> -W ->rfe R implies that W and R are on different CPUs. It also means
> +``W ->rfe R`` implies that W and R are on different CPUs. It also means
> that W's store must have propagated to R's CPU before R executed;
> otherwise R could not have read the value stored by W. Therefore W
> -must have executed before R, and so we have W ->hb R.
> +must have executed before R, and so we have ``W ->hb R``.
>
> -The equivalent fact need not hold if W ->rfi R (i.e., W and R are on
> +The equivalent fact need not hold if ``W ->rfi R`` (i.e., W and R are on
> the same CPU). As we have already seen, the operational model allows
> W's value to be forwarded to R in such cases, meaning that R may well
> execute before W does.
>
> It's important to understand that neither coe nor fre is included in
> hb, despite their similarities to rfe. For example, suppose we have
> -W ->coe W'. This means that W and W' are stores to the same location,
> +``W ->coe W'``. This means that W and W' are stores to the same location,
> they execute on different CPUs, and W comes before W' in the coherence
> order (i.e., W' overwrites W). Nevertheless, it is possible for W' to
> execute before W, because the decision as to which store overwrites
> the other is made later by the memory subsystem. When the stores are
> nearly simultaneous, either one can come out on top. Similarly,
> -R ->fre W means that W overwrites the value which R reads, but it
> +``R ->fre W`` means that W overwrites the value which R reads, but it
> doesn't mean that W has to execute after R. All that's necessary is
> for the memory subsystem not to propagate W to R's CPU until after R
> has executed, which is possible if W executes shortly before R.
> @@ -1199,7 +1208,7 @@ the first event in the coherence order and propagates to C before the
> second event executes.
>
> This is best explained with some examples. The simplest case looks
> -like this:
> +like this::
>
> int x;
>
> @@ -1225,7 +1234,7 @@ event, because P1's store came after P0's store in x's coherence
> order, and P1's store propagated to P0 before P0's load executed.
>
> An equally simple case involves two loads of the same location that
> -read from different stores:
> +read from different stores::
>
> int x = 0;
>
> @@ -1252,7 +1261,7 @@ P1's store propagated to P0 before P0's second load executed.
>
> Less trivial examples of prop all involve fences. Unlike the simple
> examples above, they can require that some instructions are executed
> -out of program order. This next one should look familiar:
> +out of program order. This next one should look familiar::
>
> int buf = 0, flag = 0;
>
> @@ -1303,7 +1312,7 @@ rfe link. You can concoct more exotic examples, containing more than
> one fence, although this quickly leads to diminishing returns in terms
> of complexity. For instance, here's an example containing a coe link
> followed by two fences and an rfe link, utilizing the fact that
> -release fences are A-cumulative:
> +release fences are A-cumulative::
>
> int x, y, z;
>
> @@ -1363,7 +1372,7 @@ links. Let's see how this definition works out.
>
> Consider first the case where E is a store (implying that the sequence
> of links begins with coe). Then there are events W, X, Y, and Z such
> -that:
> +that::
>
> E ->coe W ->cumul-fence* X ->rfe? Y ->strong-fence Z ->hb* F,
>
> @@ -1384,13 +1393,13 @@ The existence of a pb link from E to F implies that E must execute
> before F. To see why, suppose that F executed first. Then W would
> have propagated to E's CPU before E executed. If E was a store, the
> memory subsystem would then be forced to make E come after W in the
> -coherence order, contradicting the fact that E ->coe W. If E was a
> +coherence order, contradicting the fact that ``E ->coe W``. If E was a
> load, the memory subsystem would then be forced to satisfy E's read
> request with the value stored by W or an even later store,
> -contradicting the fact that E ->fre W.
> +contradicting the fact that ``E ->fre W``.
>
> A good example illustrating how pb works is the SB pattern with strong
> -fences:
> +fences::
>
> int x = 0, y = 0;
>
> @@ -1449,18 +1458,18 @@ span a full grace period. In more detail, the Guarantee says:
> For any critical section C and any grace period G, at least
> one of the following statements must hold:
>
> -(1) C ends before G does, and in addition, every store that
> - propagates to C's CPU before the end of C must propagate to
> - every CPU before G ends.
> + (1) C ends before G does, and in addition, every store that
> + propagates to C's CPU before the end of C must propagate to
> + every CPU before G ends.
>
> -(2) G starts before C does, and in addition, every store that
> - propagates to G's CPU before the start of G must propagate
> - to every CPU before C starts.
> + (2) G starts before C does, and in addition, every store that
> + propagates to G's CPU before the start of G must propagate
> + to every CPU before C starts.
>
> In particular, it is not possible for a critical section to both start
> before and end after a grace period.
>
> -Here is a simple example of RCU in action:
> +Here is a simple example of RCU in action::
>
> int x, y;
>
> @@ -1509,9 +1518,9 @@ entirely clear. The LKMM formalizes this notion by means of the
> rcu-link relation. rcu-link encompasses a very general notion of
> "before": If E and F are RCU fence events (i.e., rcu_read_lock(),
> rcu_read_unlock(), or synchronize_rcu()) then among other things,
> -E ->rcu-link F includes cases where E is po-before some memory-access
> +``E ->rcu-link F`` includes cases where E is po-before some memory-access
> event X, F is po-after some memory-access event Y, and we have any of
> -X ->rfe Y, X ->co Y, or X ->fr Y.
> +``X ->rfe Y``, ``X ->co Y``, or ``X ->fr Y``.
>
> The formal definition of the rcu-link relation is more than a little
> obscure, and we won't give it here. It is closely related to the pb
> @@ -1523,50 +1532,50 @@ The LKMM also defines the rcu-gp and rcu-rscsi relations. They bring
> grace periods and read-side critical sections into the picture, in the
> following way:
>
> - E ->rcu-gp F means that E and F are in fact the same event,
> + ``E ->rcu-gp F`` means that E and F are in fact the same event,
> and that event is a synchronize_rcu() fence (i.e., a grace
> period).
>
> - E ->rcu-rscsi F means that E and F are the rcu_read_unlock()
> + ``E ->rcu-rscsi F`` means that E and F are the rcu_read_unlock()
> and rcu_read_lock() fence events delimiting some read-side
> critical section. (The 'i' at the end of the name emphasizes
> that this relation is "inverted": It links the end of the
> critical section to the start.)
>
> If we think of the rcu-link relation as standing for an extended
> -"before", then X ->rcu-gp Y ->rcu-link Z roughly says that X is a
> +"before", then ``X ->rcu-gp Y ->rcu-link Z`` roughly says that X is a
> grace period which ends before Z begins. (In fact it covers more than
> this, because it also includes cases where some store propagates to
> Z's CPU before Z begins but doesn't propagate to some other CPU until
> -after X ends.) Similarly, X ->rcu-rscsi Y ->rcu-link Z says that X is
> +after X ends.) Similarly, ``X ->rcu-rscsi Y ->rcu-link Z`` says that X is
> the end of a critical section which starts before Z begins.
>
> The LKMM goes on to define the rcu-fence relation as a sequence of
> rcu-gp and rcu-rscsi links separated by rcu-link links, in which the
> number of rcu-gp links is >= the number of rcu-rscsi links. For
> -example:
> +example::
>
> X ->rcu-gp Y ->rcu-link Z ->rcu-rscsi T ->rcu-link U ->rcu-gp V
>
> -would imply that X ->rcu-fence V, because this sequence contains two
> +would imply that ``X ->rcu-fence V``, because this sequence contains two
> rcu-gp links and one rcu-rscsi link. (It also implies that
> -X ->rcu-fence T and Z ->rcu-fence V.) On the other hand:
> +``X ->rcu-fence T`` and ``Z ->rcu-fence V``.) On the other hand::
>
> X ->rcu-rscsi Y ->rcu-link Z ->rcu-rscsi T ->rcu-link U ->rcu-gp V
>
> -does not imply X ->rcu-fence V, because the sequence contains only
> +does not imply ``X ->rcu-fence V``, because the sequence contains only
> one rcu-gp link but two rcu-rscsi links.
>
> The rcu-fence relation is important because the Grace Period Guarantee
> means that rcu-fence acts kind of like a strong fence. In particular,
> -E ->rcu-fence F implies not only that E begins before F ends, but also
> +``E ->rcu-fence F`` implies not only that E begins before F ends, but also
> that any write po-before E will propagate to every CPU before any
> instruction po-after F can execute. (However, it does not imply that
> E must execute before F; in fact, each synchronize_rcu() fence event
> is linked to itself by rcu-fence as a degenerate case.)
>
> To prove this in full generality requires some intellectual effort.
> -We'll consider just a very simple case:
> +We'll consider just a very simple case::
>
> G ->rcu-gp W ->rcu-link Z ->rcu-rscsi F.
>
> @@ -1595,13 +1604,13 @@ covered by rcu-fence.
> Finally, the LKMM defines the RCU-before (rb) relation in terms of
> rcu-fence. This is done in essentially the same way as the pb
> relation was defined in terms of strong-fence. We will omit the
> -details; the end result is that E ->rb F implies E must execute
> -before F, just as E ->pb F does (and for much the same reasons).
> +details; the end result is that ``E ->rb F`` implies E must execute
> +before F, just as ``E ->pb F`` does (and for much the same reasons).
>
> Putting this all together, the LKMM expresses the Grace Period
> Guarantee by requiring that the rb relation does not contain a cycle.
> Equivalently, this "rcu" axiom requires that there are no events E
> -and F with E ->rcu-link F ->rcu-fence E. Or to put it a third way,
> +and F with ``E ->rcu-link F ->rcu-fence E``. Or to put it a third way,
> the axiom requires that there are no cycles consisting of rcu-gp and
> rcu-rscsi alternating with rcu-link, where the number of rcu-gp links
> is >= the number of rcu-rscsi links.
> @@ -1621,7 +1630,7 @@ question, and let S be the synchronize_rcu() fence event for the grace
> period. Saying that the critical section starts before S means there
> are events Q and R where Q is po-after L (which marks the start of the
> critical section), Q is "before" R in the sense used by the rcu-link
> -relation, and R is po-before the grace period S. Thus we have:
> +relation, and R is po-before the grace period S. Thus we have::
>
> L ->rcu-link S.
>
> @@ -1629,20 +1638,20 @@ Let W be the store mentioned above, let Y come before the end of the
> critical section and witness that W propagates to the critical
> section's CPU by reading from W, and let Z on some arbitrary CPU be a
> witness that W has not propagated to that CPU, where Z happens after
> -some event X which is po-after S. Symbolically, this amounts to:
> +some event X which is po-after S. Symbolically, this amounts to::
>
> S ->po X ->hb* Z ->fr W ->rf Y ->po U.
>
> The fr link from Z to W indicates that W has not propagated to Z's CPU
> at the time that Z executes. From this, it can be shown (see the
> discussion of the rcu-link relation earlier) that S and U are related
> -by rcu-link:
> +by rcu-link::
>
> S ->rcu-link U.
>
> -Since S is a grace period we have S ->rcu-gp S, and since L and U are
> -the start and end of the critical section C we have U ->rcu-rscsi L.
> -From this we obtain:
> +Since S is a grace period we have ``S ->rcu-gp S``, and since L and U are
> +the start and end of the critical section C we have ``U ->rcu-rscsi L``.
> +From this we obtain::
>
> S ->rcu-gp S ->rcu-link U ->rcu-rscsi L ->rcu-link S,
>
> @@ -1651,7 +1660,7 @@ the Grace Period Guarantee.
>
> For something a little more down-to-earth, let's see how the axiom
> works out in practice. Consider the RCU code example from above, this
> -time with statement labels added:
> +time with statement labels added::
>
> int x, y;
>
> @@ -1674,20 +1683,20 @@ time with statement labels added:
>
>
> If r2 = 0 at the end then P0's store at Y overwrites the value that
> -P1's load at W reads from, so we have W ->fre Y. Since S ->po W and
> -also Y ->po U, we get S ->rcu-link U. In addition, S ->rcu-gp S
> +P1's load at W reads from, so we have ``W ->fre Y``. Since ``S ->po W`` and
> +also ``Y ->po U``, we get ``S ->rcu-link U``. In addition, ``S ->rcu-gp S``
> because S is a grace period.
>
> If r1 = 1 at the end then P1's load at Z reads from P0's store at X,
> -so we have X ->rfe Z. Together with L ->po X and Z ->po S, this
> -yields L ->rcu-link S. And since L and U are the start and end of a
> -critical section, we have U ->rcu-rscsi L.
> +so we have ``X ->rfe Z``. Together with ``L ->po X`` and ``Z ->po S``, this
> +yields ``L ->rcu-link S``. And since L and U are the start and end of a
> +critical section, we have ``U ->rcu-rscsi L``.
>
> -Then U ->rcu-rscsi L ->rcu-link S ->rcu-gp S ->rcu-link U is a
> +Then ``U ->rcu-rscsi L ->rcu-link S ->rcu-gp S ->rcu-link U`` is a
> forbidden cycle, violating the "rcu" axiom. Hence the outcome is not
> allowed by the LKMM, as we would expect.
>
> -For contrast, let's see what can happen in a more complicated example:
> +For contrast, let's see what can happen in a more complicated example::
>
> int x, y, z;
>
> @@ -1720,28 +1729,28 @@ For contrast, let's see what can happen in a more complicated example:
> U2: rcu_read_unlock();
> }
>
> -If r0 = r1 = r2 = 1 at the end, then similar reasoning to before shows
> -that U0 ->rcu-rscsi L0 ->rcu-link S1 ->rcu-gp S1 ->rcu-link U2 ->rcu-rscsi
> -L2 ->rcu-link U0. However this cycle is not forbidden, because the
> +If ``r0 = r1 = r2 = 1`` at the end, then similar reasoning to before shows
> +that ``U0 ->rcu-rscsi L0 ->rcu-link S1 ->rcu-gp S1 ->rcu-link U2 ->rcu-rscsi
> +L2 ->rcu-link U0``. However this cycle is not forbidden, because the
> sequence of relations contains fewer instances of rcu-gp (one) than of
> rcu-rscsi (two). Consequently the outcome is allowed by the LKMM.
> The following instruction timing diagram shows how it might actually
> -occur:
> -
> -P0 P1 P2
> --------------------- -------------------- --------------------
> -rcu_read_lock()
> -WRITE_ONCE(y, 1)
> - r1 = READ_ONCE(y)
> - synchronize_rcu() starts
> - . rcu_read_lock()
> - . WRITE_ONCE(x, 1)
> -r0 = READ_ONCE(x) .
> -rcu_read_unlock() .
> - synchronize_rcu() ends
> - WRITE_ONCE(z, 1)
> - r2 = READ_ONCE(z)
> - rcu_read_unlock()
> +occur::
> +
> + P0 P1 P2
> + -------------------- -------------------- --------------------
> + rcu_read_lock()
> + WRITE_ONCE(y, 1)
> + r1 = READ_ONCE(y)
> + synchronize_rcu() starts
> + . rcu_read_lock()
> + . WRITE_ONCE(x, 1)
> + r0 = READ_ONCE(x) .
> + rcu_read_unlock() .
> + synchronize_rcu() ends
> + WRITE_ONCE(z, 1)
> + r2 = READ_ONCE(z)
> + rcu_read_unlock()
>
> This requires P0 and P2 to execute their loads and stores out of
> program order, but of course they are allowed to do so. And as you
> @@ -1767,26 +1776,26 @@ The LKMM includes locking. In fact, there is special code for locking
> in the formal model, added in order to make tools run faster.
> However, this special code is intended to be more or less equivalent
> to concepts we have already covered. A spinlock_t variable is treated
> -the same as an int, and spin_lock(&s) is treated almost the same as:
> +the same as an int, and spin_lock(&s) is treated almost the same as::
>
> while (cmpxchg_acquire(&s, 0, 1) != 0)
> cpu_relax();
>
> This waits until s is equal to 0 and then atomically sets it to 1,
> and the read part of the cmpxchg operation acts as an acquire fence.
> -An alternate way to express the same thing would be:
> +An alternate way to express the same thing would be::
>
> r = xchg_acquire(&s, 1);
>
> along with a requirement that at the end, r = 0. Similarly,
> -spin_trylock(&s) is treated almost the same as:
> +spin_trylock(&s) is treated almost the same as::
>
> return !cmpxchg_acquire(&s, 0, 1);
>
> which atomically sets s to 1 if it is currently equal to 0 and returns
> true if it succeeds (the read part of the cmpxchg operation acts as an
> acquire fence only if the operation is successful). spin_unlock(&s)
> -is treated almost the same as:
> +is treated almost the same as::
>
> smp_store_release(&s, 0);
>
> @@ -1802,7 +1811,7 @@ requires that every instruction po-before the lock-release must
> execute before any instruction po-after the lock-acquire. This would
> naturally hold if the release and acquire operations were on different
> CPUs, but the LKMM says it holds even when they are on the same CPU.
> -For example:
> +For example::
>
> int x, y;
> spinlock_t s;
> @@ -1833,7 +1842,7 @@ MP pattern).
>
> This requirement does not apply to ordinary release and acquire
> fences, only to lock-related operations. For instance, suppose P0()
> -in the example had been written as:
> +in the example had been written as::
>
> P0()
> {
> @@ -1847,7 +1856,7 @@ in the example had been written as:
>
> Then the CPU would be allowed to forward the s = 1 value from the
> smp_store_release() to the smp_load_acquire(), executing the
> -instructions in the following order:
> +instructions in the following order::
>
> r3 = smp_load_acquire(&s); // Obtains r3 = 1
> r2 = READ_ONCE(y);
> @@ -1859,7 +1868,7 @@ and thus it could load y before x, obtaining r2 = 0 and r1 = 1.
> Second, when a lock-acquire reads from a lock-release, and some other
> stores W and W' occur po-before the lock-release and po-after the
> lock-acquire respectively, the LKMM requires that W must propagate to
> -each CPU before W' does. For example, consider:
> +each CPU before W' does. For example, consider::
>
> int x, y;
> spinlock_t x;
> @@ -1928,7 +1937,7 @@ smp_store_release().) The final effect is the same.
> Although we didn't mention it above, the instruction execution
> ordering provided by the smp_rmb() fence doesn't apply to read events
> that are part of a non-value-returning atomic update. For instance,
> -given:
> +given::
>
> atomic_inc(&x);
> smp_rmb();
> @@ -1967,14 +1976,14 @@ they behave as follows:
> events.
>
> Interestingly, RCU and locking each introduce the possibility of
> -deadlock. When faced with code sequences such as:
> +deadlock. When faced with code sequences such as::
>
> spin_lock(&s);
> spin_lock(&s);
> spin_unlock(&s);
> spin_unlock(&s);
>
> -or:
> +or::
>
> rcu_read_lock();
> synchronize_rcu();
> @@ -1984,7 +1993,7 @@ what does the LKMM have to say? Answer: It says there are no allowed
> executions at all, which makes sense. But this can also lead to
> misleading results, because if a piece of code has multiple possible
> executions, some of which deadlock, the model will report only on the
> -non-deadlocking executions. For example:
> +non-deadlocking executions. For example::
>
> int x, y;
>
> diff --git a/tools/memory-model/Documentation/index.rst b/tools/memory-model/Documentation/index.rst
> new file mode 100644
> index 000000000000..0e53d83a5a48
> --- /dev/null
> +++ b/tools/memory-model/Documentation/index.rst
> @@ -0,0 +1,20 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +========================
> +Linux Memory Model Tools
> +========================
> +
> +.. toctree::
> + :maxdepth: 1
> +
> + explanation
> + recipes
> + references
> + cheatsheet
> +
> +.. only:: subproject and html
> +
> + Indices
> + =======
> +
> + * :ref:`genindex`
> diff --git a/tools/memory-model/Documentation/recipes.txt b/tools/memory-model/Documentation/recipes.rst
> similarity index 96%
> rename from tools/memory-model/Documentation/recipes.txt
> rename to tools/memory-model/Documentation/recipes.rst
> index 7fe8d7aa3029..0229a431b1a2 100644
> --- a/tools/memory-model/Documentation/recipes.txt
> +++ b/tools/memory-model/Documentation/recipes.rst
> @@ -1,3 +1,8 @@
> +=======
> +Recipes
> +=======
> +
> +
> This document provides "recipes", that is, litmus tests for commonly
> occurring situations, as well as a few that illustrate subtly broken but
> attractive nuisances. Many of these recipes include example code from
> @@ -67,7 +72,7 @@ has acquired a given lock sees any changes previously seen or made by any
> CPU before it released that same lock. Note that this statement is a bit
> stronger than "Any CPU holding a given lock sees all changes made by any
> CPU during the time that CPU was holding this same lock". For example,
> -consider the following pair of code fragments:
> +consider the following pair of code fragments::
>
> /* See MP+polocks.litmus. */
> void CPU0(void)
> @@ -93,7 +98,7 @@ value of r1 must also be equal to 1. In contrast, the weaker rule would
> say nothing about the final value of r1.
>
> The converse to the basic rule also holds, as illustrated by the
> -following litmus test:
> +following litmus test::
>
> /* See MP+porevlocks.litmus. */
> void CPU0(void)
> @@ -124,7 +129,7 @@ across multiple CPUs.
>
> However, it is not necessarily the case that accesses ordered by
> locking will be seen as ordered by CPUs not holding that lock.
> -Consider this example:
> +Consider this example::
>
> /* See Z6.0+pooncerelease+poacquirerelease+fencembonceonce.litmus. */
> void CPU0(void)
> @@ -157,7 +162,7 @@ CPU2() never acquired the lock, and thus did not benefit from the
> lock's ordering properties.
>
> Ordering can be extended to CPUs not holding the lock by careful use
> -of smp_mb__after_spinlock():
> +of smp_mb__after_spinlock()::
>
> /* See Z6.0+pooncelock+poonceLock+pombonce.litmus. */
> void CPU0(void)
> @@ -214,7 +219,7 @@ Release and acquire
> ~~~~~~~~~~~~~~~~~~~
>
> Use of smp_store_release() and smp_load_acquire() is one way to force
> -the desired MP ordering. The general approach is shown below:
> +the desired MP ordering. The general approach is shown below::
>
> /* See MP+pooncerelease+poacquireonce.litmus. */
> void CPU0(void)
> @@ -245,7 +250,7 @@ Assign and dereference
> Use of rcu_assign_pointer() and rcu_dereference() is quite similar to the
> use of smp_store_release() and smp_load_acquire(), except that both
> rcu_assign_pointer() and rcu_dereference() operate on RCU-protected
> -pointers. The general approach is shown below:
> +pointers. The general approach is shown below::
>
> /* See MP+onceassign+derefonce.litmus. */
> int z;
> @@ -290,7 +295,7 @@ Write and read memory barriers
> It is usually better to use smp_store_release() instead of smp_wmb()
> and to use smp_load_acquire() instead of smp_rmb(). However, the older
> smp_wmb() and smp_rmb() APIs are still heavily used, so it is important
> -to understand their use cases. The general approach is shown below:
> +to understand their use cases. The general approach is shown below::
>
> /* See MP+fencewmbonceonce+fencermbonceonce.litmus. */
> void CPU0(void)
> @@ -312,7 +317,7 @@ smp_rmb() macro orders prior loads against later loads. Therefore, if
> the final value of r0 is 1, the final value of r1 must also be 1.
>
> The xlog_state_switch_iclogs() function in fs/xfs/xfs_log.c contains
> -the following write-side code fragment:
> +the following write-side code fragment::
>
> log->l_curr_block -= log->l_logBBsize;
> ASSERT(log->l_curr_block >= 0);
> @@ -327,7 +332,7 @@ the corresponding read-side code fragment:
> cur_block = READ_ONCE(log->l_curr_block);
>
> Alternatively, consider the following comment in function
> -perf_output_put_handle() in kernel/events/ring_buffer.c:
> +perf_output_put_handle() in kernel/events/ring_buffer.c::
>
> * kernel user
> *
> @@ -358,7 +363,7 @@ absence of any ordering it is quite possible that this may happen, as
> can be seen in the LB+poonceonces.litmus litmus test.
>
> One way of avoiding the counter-intuitive outcome is through the use of a
> -control dependency paired with a full memory barrier:
> +control dependency paired with a full memory barrier::
>
> /* See LB+fencembonceonce+ctrlonceonce.litmus. */
> void CPU0(void)
> @@ -382,7 +387,7 @@ The A/D pairing from the ring-buffer use case shown earlier also
> illustrates LB. Here is a repeat of the comment in
> perf_output_put_handle() in kernel/events/ring_buffer.c, showing a
> control dependency on the kernel side and a full memory barrier on
> -the user side:
> +the user side::
>
> * kernel user
> *
> @@ -407,7 +412,7 @@ Release-acquire chains
>
> Release-acquire chains are a low-overhead, flexible, and easy-to-use
> method of maintaining order. However, they do have some limitations that
> -need to be fully understood. Here is an example that maintains order:
> +need to be fully understood. Here is an example that maintains order::
>
> /* See ISA2+pooncerelease+poacquirerelease+poacquireonce.litmus. */
> void CPU0(void)
> @@ -436,7 +441,7 @@ example, ordering would still be preserved if CPU1()'s smp_load_acquire()
> invocation was replaced with READ_ONCE().
>
> It is tempting to assume that CPU0()'s store to x is globally ordered
> -before CPU1()'s store to z, but this is not the case:
> +before CPU1()'s store to z, but this is not the case::
>
> /* See Z6.0+pooncerelease+poacquirerelease+mbonceonce.litmus. */
> void CPU0(void)
> @@ -474,7 +479,7 @@ Store buffering
> Store buffering can be thought of as upside-down load buffering, so
> that one CPU first stores to one variable and then loads from a second,
> while another CPU stores to the second variable and then loads from the
> -first. Preserving order requires nothing less than full barriers:
> +first. Preserving order requires nothing less than full barriers::
>
> /* See SB+fencembonceonces.litmus. */
> void CPU0(void)
> @@ -498,7 +503,7 @@ this counter-intuitive outcome.
> This pattern most famously appears as part of Dekker's locking
> algorithm, but it has a much more practical use within the Linux kernel
> of ordering wakeups. The following comment taken from waitqueue_active()
> -in include/linux/wait.h shows the canonical pattern:
> +in include/linux/wait.h shows the canonical pattern::
>
> * CPU0 - waker CPU1 - waiter
> *
> @@ -550,16 +555,16 @@ The strength of memory ordering required for a given litmus test to
> avoid a counter-intuitive outcome depends on the types of relations
> linking the memory accesses for the outcome in question:
>
> -o If all links are write-to-read links, then the weakest
> +- If all links are write-to-read links, then the weakest
> possible ordering within each CPU suffices. For example, in
> the LB litmus test, a control dependency was enough to do the
> job.
>
> -o If all but one of the links are write-to-read links, then a
> +- If all but one of the links are write-to-read links, then a
> release-acquire chain suffices. Both the MP and the ISA2
> litmus tests illustrate this case.
>
> -o If more than one of the links are something other than
> +- If more than one of the links are something other than
> write-to-read links, then a full memory barrier is required
> between each successive pair of non-write-to-read links. This
> case is illustrated by the Z6.0 litmus tests, both in the
> diff --git a/tools/memory-model/Documentation/references.txt b/tools/memory-model/Documentation/references.rst
> similarity index 71%
> rename from tools/memory-model/Documentation/references.txt
> rename to tools/memory-model/Documentation/references.rst
> index b177f3e4a614..275876cd10b8 100644
> --- a/tools/memory-model/Documentation/references.txt
> +++ b/tools/memory-model/Documentation/references.rst
> @@ -1,3 +1,7 @@
> +==========
> +References
> +==========
> +
> This document provides background reading for memory models and related
> tools. These documents are aimed at kernel hackers who are interested
> in memory models.
> @@ -6,64 +10,64 @@ in memory models.
> Hardware manuals and models
> ===========================
>
> -o SPARC International Inc. (Ed.). 1994. "The SPARC Architecture
> +- SPARC International Inc. (Ed.). 1994. "The SPARC Architecture
> Reference Manual Version 9". SPARC International Inc.
>
> -o Compaq Computer Corporation (Ed.). 2002. "Alpha Architecture
> +- Compaq Computer Corporation (Ed.). 2002. "Alpha Architecture
> Reference Manual". Compaq Computer Corporation.
>
> -o Intel Corporation (Ed.). 2002. "A Formal Specification of Intel
> +- Intel Corporation (Ed.). 2002. "A Formal Specification of Intel
> Itanium Processor Family Memory Ordering". Intel Corporation.
>
> -o Intel Corporation (Ed.). 2002. "Intel 64 and IA-32 Architectures
> +- Intel Corporation (Ed.). 2002. "Intel 64 and IA-32 Architectures
> Software Developer’s Manual". Intel Corporation.
>
> -o Peter Sewell, Susmit Sarkar, Scott Owens, Francesco Zappa Nardelli,
> +- Peter Sewell, Susmit Sarkar, Scott Owens, Francesco Zappa Nardelli,
> and Magnus O. Myreen. 2010. "x86-TSO: A Rigorous and Usable
> Programmer's Model for x86 Multiprocessors". Commun. ACM 53, 7
> (July, 2010), 89-97. http://doi.acm.org/10.1145/1785414.1785443
>
> -o IBM Corporation (Ed.). 2009. "Power ISA Version 2.06". IBM
> +- IBM Corporation (Ed.). 2009. "Power ISA Version 2.06". IBM
> Corporation.
>
> -o ARM Ltd. (Ed.). 2009. "ARM Barrier Litmus Tests and Cookbook".
> +- ARM Ltd. (Ed.). 2009. "ARM Barrier Litmus Tests and Cookbook".
> ARM Ltd.
>
> -o Susmit Sarkar, Peter Sewell, Jade Alglave, Luc Maranget, and
> +- Susmit Sarkar, Peter Sewell, Jade Alglave, Luc Maranget, and
> Derek Williams. 2011. "Understanding POWER Multiprocessors". In
> Proceedings of the 32Nd ACM SIGPLAN Conference on Programming
> Language Design and Implementation (PLDI ’11). ACM, New York,
> NY, USA, 175–186.
>
> -o Susmit Sarkar, Kayvan Memarian, Scott Owens, Mark Batty,
> +- Susmit Sarkar, Kayvan Memarian, Scott Owens, Mark Batty,
> Peter Sewell, Luc Maranget, Jade Alglave, and Derek Williams.
> 2012. "Synchronising C/C++ and POWER". In Proceedings of the 33rd
> ACM SIGPLAN Conference on Programming Language Design and
> Implementation (PLDI '12). ACM, New York, NY, USA, 311-322.
>
> -o ARM Ltd. (Ed.). 2014. "ARM Architecture Reference Manual (ARMv8,
> +- ARM Ltd. (Ed.). 2014. "ARM Architecture Reference Manual (ARMv8,
> for ARMv8-A architecture profile)". ARM Ltd.
>
> -o Imagination Technologies, LTD. 2015. "MIPS(R) Architecture
> +- Imagination Technologies, LTD. 2015. "MIPS(R) Architecture
> For Programmers, Volume II-A: The MIPS64(R) Instruction,
> Set Reference Manual". Imagination Technologies,
> LTD. https://imgtec.com/?do-download=4302.
>
> -o Shaked Flur, Kathryn E. Gray, Christopher Pulte, Susmit
> +- Shaked Flur, Kathryn E. Gray, Christopher Pulte, Susmit
> Sarkar, Ali Sezgin, Luc Maranget, Will Deacon, and Peter
> Sewell. 2016. "Modelling the ARMv8 Architecture, Operationally:
> Concurrency and ISA". In Proceedings of the 43rd Annual ACM
> SIGPLAN-SIGACT Symposium on Principles of Programming Languages
> (POPL ’16). ACM, New York, NY, USA, 608–621.
>
> -o Shaked Flur, Susmit Sarkar, Christopher Pulte, Kyndylan Nienhuis,
> +- Shaked Flur, Susmit Sarkar, Christopher Pulte, Kyndylan Nienhuis,
> Luc Maranget, Kathryn E. Gray, Ali Sezgin, Mark Batty, and Peter
> Sewell. 2017. "Mixed-size Concurrency: ARM, POWER, C/C++11,
> and SC". In Proceedings of the 44th ACM SIGPLAN Symposium on
> Principles of Programming Languages (POPL 2017). ACM, New York,
> NY, USA, 429–442.
>
> -o Christopher Pulte, Shaked Flur, Will Deacon, Jon French,
> +- Christopher Pulte, Shaked Flur, Will Deacon, Jon French,
> Susmit Sarkar, and Peter Sewell. 2018. "Simplifying ARM concurrency:
> multicopy-atomic axiomatic and operational models for ARMv8". In
> Proceedings of the ACM on Programming Languages, Volume 2, Issue
> @@ -73,18 +77,18 @@ o Christopher Pulte, Shaked Flur, Will Deacon, Jon French,
> Linux-kernel memory model
> =========================
>
> -o Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and
> +- Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and
> Alan Stern. 2018. "Frightening small children and disconcerting
> grown-ups: Concurrency in the Linux kernel". In Proceedings of
> the 23rd International Conference on Architectural Support for
> Programming Languages and Operating Systems (ASPLOS 2018). ACM,
> New York, NY, USA, 405-418. Webpage: http://diy.inria.fr/linux/.
>
> -o Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and
> +- Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and
> Alan Stern. 2017. "A formal kernel memory-ordering model (part 1)"
> Linux Weekly News. https://lwn.net/Articles/718628/
>
> -o Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and
> +- Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and
> Alan Stern. 2017. "A formal kernel memory-ordering model (part 2)"
> Linux Weekly News. https://lwn.net/Articles/720550/
>
> @@ -92,16 +96,16 @@ o Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and
> Memory-model tooling
> ====================
>
> -o Daniel Jackson. 2002. "Alloy: A Lightweight Object Modelling
> +- Daniel Jackson. 2002. "Alloy: A Lightweight Object Modelling
> Notation". ACM Trans. Softw. Eng. Methodol. 11, 2 (April 2002),
> 256–290. http://doi.acm.org/10.1145/505145.505149
>
> -o Jade Alglave, Luc Maranget, and Michael Tautschnig. 2014. "Herding
> +- Jade Alglave, Luc Maranget, and Michael Tautschnig. 2014. "Herding
> Cats: Modelling, Simulation, Testing, and Data Mining for Weak
> Memory". ACM Trans. Program. Lang. Syst. 36, 2, Article 7 (July
> 2014), 7:1–7:74 pages.
>
> -o Jade Alglave, Patrick Cousot, and Luc Maranget. 2016. "Syntax and
> +- Jade Alglave, Patrick Cousot, and Luc Maranget. 2016. "Syntax and
> semantics of the weak consistency model specification language
> cat". CoRR abs/1608.07531 (2016). http://arxiv.org/abs/1608.07531
>
> @@ -109,6 +113,6 @@ o Jade Alglave, Patrick Cousot, and Luc Maranget. 2016. "Syntax and
> Memory-model comparisons
> ========================
>
> -o Paul E. McKenney, Ulrich Weigand, Andrea Parri, and Boqun
> +- Paul E. McKenney, Ulrich Weigand, Andrea Parri, and Boqun
> Feng. 2016. "Linux-Kernel Memory Model". (6 June 2016).
> http://open-std.org/JTC1/SC22/WG21/docs/papers/2016/p0124r2.html.
> diff --git a/tools/memory-model/README b/tools/memory-model/README
> index 2b87f3971548..04bb1fa9ed76 100644
> --- a/tools/memory-model/README
> +++ b/tools/memory-model/README
> @@ -105,16 +105,16 @@ for more information.
> DESCRIPTION OF FILES
> ====================
>
> -Documentation/cheatsheet.txt
> +Documentation/cheatsheet.rst
> Quick-reference guide to the Linux-kernel memory model.
>
> -Documentation/explanation.txt
> +Documentation/explanation.rst
> Describes the memory model in detail.
>
> -Documentation/recipes.txt
> +Documentation/recipes.rst
> Lists common memory-ordering patterns.
>
> -Documentation/references.txt
> +Documentation/references.rst
> Provides background reading.
>
> linux-kernel.bell
> @@ -173,7 +173,7 @@ The Linux-kernel memory model has the following limitations:
> of READ_ONCE() and WRITE_ONCE() limits the compiler's ability
> to optimize, but there is Linux-kernel code that uses bare C
> memory accesses. Handling this code is on the to-do list.
> - For more information, see Documentation/explanation.txt (in
> + For more information, see Documentation/explanation.rst (in
> particular, the "THE PROGRAM ORDER RELATION: po AND po-loc"
> and "A WARNING" sections).
>
>
>
^ permalink raw reply
* [PATCH v3 3/3] docs: rcu: Increase toctree to 3
From: Joel Fernandes (Google) @ 2019-07-30 23:10 UTC (permalink / raw)
To: linux-kernel
Cc: Joel Fernandes (Google), Jonathan Corbet, Josh Triplett,
Lai Jiangshan, linux-doc, Mathieu Desnoyers,
Mauro Carvalho Chehab, Paul E. McKenney, rcu, Steven Rostedt
In-Reply-To: <20190730231030.27510-1-joel@joelfernandes.org>
These documents are long and have various sections. Provide a good
toc nesting level.
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
Documentation/RCU/index.rst | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Documentation/RCU/index.rst b/Documentation/RCU/index.rst
index 94427dc1f23d..5c99185710fa 100644
--- a/Documentation/RCU/index.rst
+++ b/Documentation/RCU/index.rst
@@ -5,7 +5,7 @@ RCU concepts
============
.. toctree::
- :maxdepth: 1
+ :maxdepth: 3
rcu
listRCU
--
2.22.0.709.g102302147b-goog
^ permalink raw reply related
* [PATCH v3 2/3] docs: rcu: Correct links referring to titles
From: Joel Fernandes (Google) @ 2019-07-30 23:10 UTC (permalink / raw)
To: linux-kernel
Cc: Joel Fernandes (Google), Jonathan Corbet, Josh Triplett,
Lai Jiangshan, linux-doc, Mathieu Desnoyers,
Mauro Carvalho Chehab, Paul E. McKenney, rcu, Steven Rostedt
In-Reply-To: <20190730231030.27510-1-joel@joelfernandes.org>
Mauro's auto conversion broken these links, fix them.
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
.../Tree-RCU-Memory-Ordering.rst | 17 ++--
.../RCU/Design/Requirements/Requirements.rst | 90 ++++++++-----------
2 files changed, 47 insertions(+), 60 deletions(-)
diff --git a/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst b/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
index 1011b5db1b3d..248b1222f918 100644
--- a/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
+++ b/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
@@ -230,15 +230,14 @@ Tree RCU Grace Period Memory Ordering Components
Tree RCU's grace-period memory-ordering guarantee is provided by a
number of RCU components:
-#. `Callback Registry <#Callback%20Registry>`__
-#. `Grace-Period Initialization <#Grace-Period%20Initialization>`__
-#. `Self-Reported Quiescent
- States <#Self-Reported%20Quiescent%20States>`__
-#. `Dynamic Tick Interface <#Dynamic%20Tick%20Interface>`__
-#. `CPU-Hotplug Interface <#CPU-Hotplug%20Interface>`__
-#. `Forcing Quiescent States <Forcing%20Quiescent%20States>`__
-#. `Grace-Period Cleanup <Grace-Period%20Cleanup>`__
-#. `Callback Invocation <Callback%20Invocation>`__
+#. `Callback Registry`_
+#. `Grace-Period Initialization`_
+#. `Self-Reported Quiescent States`_
+#. `Dynamic Tick Interface`_
+#. `CPU-Hotplug Interface`_
+#. `Forcing Quiescent States`_
+#. `Grace-Period Cleanup`_
+#. `Callback Invocation`_
Each of the following section looks at the corresponding component in
detail.
diff --git a/Documentation/RCU/Design/Requirements/Requirements.rst b/Documentation/RCU/Design/Requirements/Requirements.rst
index 876e0038bb58..a33b5fb331b4 100644
--- a/Documentation/RCU/Design/Requirements/Requirements.rst
+++ b/Documentation/RCU/Design/Requirements/Requirements.rst
@@ -36,16 +36,14 @@ technologies in interesting new ways.
All that aside, here are the categories of currently known RCU
requirements:
-#. `Fundamental Requirements <#Fundamental%20Requirements>`__
-#. `Fundamental Non-Requirements <#Fundamental%20Non-Requirements>`__
-#. `Parallelism Facts of Life <#Parallelism%20Facts%20of%20Life>`__
-#. `Quality-of-Implementation
- Requirements <#Quality-of-Implementation%20Requirements>`__
-#. `Linux Kernel Complications <#Linux%20Kernel%20Complications>`__
-#. `Software-Engineering
- Requirements <#Software-Engineering%20Requirements>`__
-#. `Other RCU Flavors <#Other%20RCU%20Flavors>`__
-#. `Possible Future Changes <#Possible%20Future%20Changes>`__
+#. `Fundamental Requirements`_
+#. `Fundamental Non-Requirements`_
+#. `Parallelism Facts of Life`_
+#. `Quality-of-Implementation Requirements`_
+#. `Linux Kernel Complications`_
+#. `Software-Engineering Requirements`_
+#. `Other RCU Flavors`_
+#. `Possible Future Changes`_
This is followed by a `summary <#Summary>`__, however, the answers to
each quick quiz immediately follows the quiz. Select the big white space
@@ -57,13 +55,11 @@ Fundamental Requirements
RCU's fundamental requirements are the closest thing RCU has to hard
mathematical requirements. These are:
-#. `Grace-Period Guarantee <#Grace-Period%20Guarantee>`__
-#. `Publish-Subscribe Guarantee <#Publish-Subscribe%20Guarantee>`__
-#. `Memory-Barrier Guarantees <#Memory-Barrier%20Guarantees>`__
-#. `RCU Primitives Guaranteed to Execute
- Unconditionally <#RCU%20Primitives%20Guaranteed%20to%20Execute%20Unconditionally>`__
-#. `Guaranteed Read-to-Write
- Upgrade <#Guaranteed%20Read-to-Write%20Upgrade>`__
+#. `Grace-Period Guarantee`_
+#. `Publish/Subscribe Guarantee`_
+#. `Memory-Barrier Guarantees`_
+#. `RCU Primitives Guaranteed to Execute Unconditionally`_
+#. `Guaranteed Read-to-Write Upgrade`_
Grace-Period Guarantee
~~~~~~~~~~~~~~~~~~~~~~
@@ -689,16 +685,11 @@ infinitely long, however, the following sections list a few
non-guarantees that have caused confusion. Except where otherwise noted,
these non-guarantees were premeditated.
-#. `Readers Impose Minimal
- Ordering <#Readers%20Impose%20Minimal%20Ordering>`__
-#. `Readers Do Not Exclude
- Updaters <#Readers%20Do%20Not%20Exclude%20Updaters>`__
-#. `Updaters Only Wait For Old
- Readers <#Updaters%20Only%20Wait%20For%20Old%20Readers>`__
-#. `Grace Periods Don't Partition Read-Side Critical
- Sections <#Grace%20Periods%20Don't%20Partition%20Read-Side%20Critical%20Sections>`__
-#. `Read-Side Critical Sections Don't Partition Grace
- Periods <#Read-Side%20Critical%20Sections%20Don't%20Partition%20Grace%20Periods>`__
+#. `Readers Impose Minimal Ordering`_
+#. `Readers Do Not Exclude Updaters`_
+#. `Updaters Only Wait For Old Readers`_
+#. `Grace Periods Don't Partition Read-Side Critical Sections`_
+#. `Read-Side Critical Sections Don't Partition Grace Periods`_
Readers Impose Minimal Ordering
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -1056,11 +1047,11 @@ it would likely be subject to limitations that would make it
inappropriate for industrial-strength production use. Classes of
quality-of-implementation requirements are as follows:
-#. `Specialization <#Specialization>`__
-#. `Performance and Scalability <#Performance%20and%20Scalability>`__
-#. `Forward Progress <#Forward%20Progress>`__
-#. `Composability <#Composability>`__
-#. `Corner Cases <#Corner%20Cases>`__
+#. `Specialization`_
+#. `Performance and Scalability`_
+#. `Forward Progress`_
+#. `Composability`_
+#. `Corner Cases`_
These classes is covered in the following sections.
@@ -1692,21 +1683,18 @@ The Linux kernel provides an interesting environment for all kinds of
software, including RCU. Some of the relevant points of interest are as
follows:
-#. `Configuration <#Configuration>`__.
-#. `Firmware Interface <#Firmware%20Interface>`__.
-#. `Early Boot <#Early%20Boot>`__.
-#. `Interrupts and non-maskable interrupts
- (NMIs) <#Interrupts%20and%20NMIs>`__.
-#. `Loadable Modules <#Loadable%20Modules>`__.
-#. `Hotplug CPU <#Hotplug%20CPU>`__.
-#. `Scheduler and RCU <#Scheduler%20and%20RCU>`__.
-#. `Tracing and RCU <#Tracing%20and%20RCU>`__.
-#. `Energy Efficiency <#Energy%20Efficiency>`__.
-#. `Scheduling-Clock Interrupts and
- RCU <#Scheduling-Clock%20Interrupts%20and%20RCU>`__.
-#. `Memory Efficiency <#Memory%20Efficiency>`__.
-#. `Performance, Scalability, Response Time, and
- Reliability <#Performance,%20Scalability,%20Response%20Time,%20and%20Reliability>`__.
+#. `Configuration`_
+#. `Firmware Interface`_
+#. `Early Boot`_
+#. `Interrupts and NMIs`_
+#. `Loadable Modules`_
+#. `Hotplug CPU`_
+#. `Scheduler and RCU`_
+#. `Tracing and RCU`_
+#. `Energy Efficiency`_
+#. `Scheduling-Clock Interrupts and RCU`_
+#. `Memory Efficiency`_
+#. `Performance, Scalability, Response Time, and Reliability`_
This list is probably incomplete, but it does give a feel for the most
notable Linux-kernel complications. Each of the following sections
@@ -2344,10 +2332,10 @@ implementations, non-preemptible and preemptible. The other four flavors
are listed below, with requirements for each described in a separate
section.
-#. `Bottom-Half Flavor (Historical) <#Bottom-Half%20Flavor>`__
-#. `Sched Flavor (Historical) <#Sched%20Flavor>`__
-#. `Sleepable RCU <#Sleepable%20RCU>`__
-#. `Tasks RCU <#Tasks%20RCU>`__
+#. `Bottom-Half Flavor (Historical)`_
+#. `Sched Flavor (Historical)`_
+#. `Sleepable RCU`_
+#. `Tasks RCU`_
Bottom-Half Flavor (Historical)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--
2.22.0.709.g102302147b-goog
^ permalink raw reply related
* [PATCH v3 0/3] Convert some RCU articles to ReST
From: Joel Fernandes (Google) @ 2019-07-30 23:10 UTC (permalink / raw)
To: linux-kernel
Cc: Joel Fernandes (Google), Jonathan Corbet, Josh Triplett,
Lai Jiangshan, linux-doc, Mathieu Desnoyers,
Mauro Carvalho Chehab, Paul E. McKenney, rcu, Steven Rostedt
This patch is a respin of the RCU ReST patch from Mauro [1].
I updated his changelog, and made some fixes.
[1] https://www.spinics.net/lists/rcu/msg00750.html
Joel Fernandes (Google) (2):
docs: rcu: Correct links referring to titles
docs: rcu: Increase toctree to 3
Mauro Carvalho Chehab (1):
docs: rcu: convert some articles from html to ReST
.../Data-Structures/Data-Structures.html | 1391 -------
.../Data-Structures/Data-Structures.rst | 1163 ++++++
.../Expedited-Grace-Periods.html | 668 ----
.../Expedited-Grace-Periods.rst | 521 +++
.../Memory-Ordering/Tree-RCU-Diagram.html | 9 -
.../Tree-RCU-Memory-Ordering.html | 704 ----
.../Tree-RCU-Memory-Ordering.rst | 624 +++
.../RCU/Design/Requirements/Requirements.html | 3330 -----------------
.../RCU/Design/Requirements/Requirements.rst | 2650 +++++++++++++
Documentation/RCU/index.rst | 7 +-
Documentation/RCU/whatisRCU.txt | 4 +-
11 files changed, 4966 insertions(+), 6105 deletions(-)
delete mode 100644 Documentation/RCU/Design/Data-Structures/Data-Structures.html
create mode 100644 Documentation/RCU/Design/Data-Structures/Data-Structures.rst
delete mode 100644 Documentation/RCU/Design/Expedited-Grace-Periods/Expedited-Grace-Periods.html
create mode 100644 Documentation/RCU/Design/Expedited-Grace-Periods/Expedited-Grace-Periods.rst
delete mode 100644 Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Diagram.html
delete mode 100644 Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.html
create mode 100644 Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
delete mode 100644 Documentation/RCU/Design/Requirements/Requirements.html
create mode 100644 Documentation/RCU/Design/Requirements/Requirements.rst
--
2.22.0.709.g102302147b-goog
^ permalink raw reply
* Re: [PATCH v2 25/26] docs: rcu: convert some articles from html to ReST
From: Mauro Carvalho Chehab @ 2019-07-30 23:00 UTC (permalink / raw)
To: Joel Fernandes
Cc: Paul E. McKenney, Josh Triplett, Steven Rostedt,
Mathieu Desnoyers, Lai Jiangshan, Jonathan Corbet, rcu, linux-doc
In-Reply-To: <20190730222130.GD254050@google.com>
Em Tue, 30 Jul 2019 18:21:30 -0400
Joel Fernandes <joel@joelfernandes.org> escreveu:
> On Tue, Jul 30, 2019 at 07:00:28PM -0300, Mauro Carvalho Chehab wrote:
> > Em Tue, 30 Jul 2019 17:50:07 -0400
> > Joel Fernandes <joel@joelfernandes.org> escreveu:
> >
> > > On Tue, Jul 30, 2019 at 02:22:50PM -0700, Paul E. McKenney wrote:
> > > > On Fri, Jul 26, 2019 at 09:51:35AM -0300, Mauro Carvalho Chehab wrote:
> > > > > There are 4 RCU articles that are written on html format.
> > > > >
> > > > > The way they are, they can't be part of the Linux Kernel
> > > > > documentation body nor share the styles and pdf output.
> > > > >
> > > > > So, convert them to ReST format.
> > > > >
> > > > > This way, make htmldocs and make pdfdocs will produce a
> > > > > documentation output that will be like the original ones, but
> > > > > will be part of the Linux Kernel documentation body.
> > > > >
> > > > > Part of the conversion was done with the help of pandoc, but
> > > > > the result had some broken things that had to be manually
> > > > > fixed.
> > > > >
> > > > > Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> > > >
> > > > I am having some trouble applying these, at least in part due to UTF-8
> > > > sequences, for example double left quotation mark. These end up being
> > > > "=E2=80=9C", with a few space characters turned into "=20".
> > > >
> > > > Any advice on how to apply these? Should I just pull commits from
> > > > somewhere?
> > >
> > > I was able to successfully apply and build this particular patch. I think
> > > this is the only one in the series that applies to RCU.
> > >
> > > Sadly, I can't find the patch in any of the public archives, but I could
> > > perhaps email it to you as an .mbox attach which 'git am' should be able to
> > > apply.
> > >
> > > Mauro did say he was going to add some more details to changelog, or it could
> > > be added when it is applied:
> > > https://lore.kernel.org/rcu/20190726154550.5eeae294@coco.lan/
> >
> > Yeah, I'm also planning to address at least some of the issues you
> > pointed, in order to improve the html output, but got sidetracked by something
> > else and didn't find any time yet to finish. I'm adding some CI automation for
> > the media subsystem in order to help us dealing with the huge amount of patches
> > we receive there.
> >
> > Feel free to add those details to the changelog. I may find some spare time
> > this week or the next one for the improvements you suggested, but those
> > could be sent on followup patches, once done.
>
> Ok, I will re-send this RCU patch with the changes, leave this one to me.
>
> The other memory model one, needs a lot more work so we can keep that aside
> for now till someone has the time.
Yeah, feel free to do what fits best with regards to this one.
If you prefer to merge it instead, I'm enclosing the last version of it,
with that quick hack I just wrote, plus the change of the toctree for
it to produce the 2-level index.
Thanks,
Mauro
[PATCH] tools: memory-model: add it to the Documentation body
The books at tools/memory-model/Documentation are very well
formatted. Congrats to the ones that wrote them!
The manual conversion to ReST is really trivial:
- Add document titles;
- change the bullets on some lists;
- mark code blocks.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
diff --git a/Documentation/core-api/refcount-vs-atomic.rst b/Documentation/core-api/refcount-vs-atomic.rst
index 976e85adffe8..7e6500a6fa76 100644
--- a/Documentation/core-api/refcount-vs-atomic.rst
+++ b/Documentation/core-api/refcount-vs-atomic.rst
@@ -17,7 +17,7 @@ in order to help maintainers validate their code against the change in
these memory ordering guarantees.
The terms used through this document try to follow the formal LKMM defined in
-tools/memory-model/Documentation/explanation.txt.
+tools/memory-model/Documentation/explanation.rst.
memory-barriers.txt and atomic_t.txt provide more background to the
memory ordering in general and for atomic operations specifically.
diff --git a/Documentation/index.rst b/Documentation/index.rst
index 2df5a3da563c..5123770ba77e 100644
--- a/Documentation/index.rst
+++ b/Documentation/index.rst
@@ -36,6 +36,7 @@ trying to get it to work optimally on a given system.
admin-guide/index
kbuild/index
+ tools/index
Firmware-related documentation
------------------------------
diff --git a/Documentation/tools/index.rst b/Documentation/tools/index.rst
new file mode 100644
index 000000000000..f540d9cc75a1
--- /dev/null
+++ b/Documentation/tools/index.rst
@@ -0,0 +1,17 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+===========
+Linux Tools
+===========
+
+.. toctree::
+ :maxdepth: 2
+
+ memory-model/index
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
diff --git a/Documentation/tools/memory-model b/Documentation/tools/memory-model
new file mode 120000
index 000000000000..020ed38ce302
--- /dev/null
+++ b/Documentation/tools/memory-model
@@ -0,0 +1 @@
+../../tools/memory-model/Documentation/
\ No newline at end of file
diff --git a/tools/memory-model/Documentation/cheatsheet.rst b/tools/memory-model/Documentation/cheatsheet.rst
new file mode 100644
index 000000000000..35f8749b8a53
--- /dev/null
+++ b/tools/memory-model/Documentation/cheatsheet.rst
@@ -0,0 +1,36 @@
+===========
+Cheat Sheet
+===========
+
+::
+
+ Prior Operation Subsequent Operation
+ --------------- ---------------------------
+ C Self R W RMW Self R W DR DW RMW SV
+ -- ---- - - --- ---- - - -- -- --- --
+
+ Store, e.g., WRITE_ONCE() Y Y
+ Load, e.g., READ_ONCE() Y Y Y Y
+ Unsuccessful RMW operation Y Y Y Y
+ rcu_dereference() Y Y Y Y
+ Successful *_acquire() R Y Y Y Y Y Y
+ Successful *_release() C Y Y Y W Y
+ smp_rmb() Y R Y Y R
+ smp_wmb() Y W Y Y W
+ smp_mb() & synchronize_rcu() CP Y Y Y Y Y Y Y Y
+ Successful full non-void RMW CP Y Y Y Y Y Y Y Y Y Y Y
+ smp_mb__before_atomic() CP Y Y Y a a a a Y
+ smp_mb__after_atomic() CP a a Y Y Y Y Y Y
+
+
+ Key: C: Ordering is cumulative
+ P: Ordering propagates
+ R: Read, for example, READ_ONCE(), or read portion of RMW
+ W: Write, for example, WRITE_ONCE(), or write portion of RMW
+ Y: Provides ordering
+ a: Provides ordering given intervening RMW atomic operation
+ DR: Dependent read (address dependency)
+ DW: Dependent write (address, data, or control dependency)
+ RMW: Atomic read-modify-write operation
+ SELF: Orders self, as opposed to accesses before and/or after
+ SV: Orders later accesses to the same variable
diff --git a/tools/memory-model/Documentation/cheatsheet.txt b/tools/memory-model/Documentation/cheatsheet.txt
deleted file mode 100644
index 33ba98d72b16..000000000000
--- a/tools/memory-model/Documentation/cheatsheet.txt
+++ /dev/null
@@ -1,30 +0,0 @@
- Prior Operation Subsequent Operation
- --------------- ---------------------------
- C Self R W RMW Self R W DR DW RMW SV
- -- ---- - - --- ---- - - -- -- --- --
-
-Store, e.g., WRITE_ONCE() Y Y
-Load, e.g., READ_ONCE() Y Y Y Y
-Unsuccessful RMW operation Y Y Y Y
-rcu_dereference() Y Y Y Y
-Successful *_acquire() R Y Y Y Y Y Y
-Successful *_release() C Y Y Y W Y
-smp_rmb() Y R Y Y R
-smp_wmb() Y W Y Y W
-smp_mb() & synchronize_rcu() CP Y Y Y Y Y Y Y Y
-Successful full non-void RMW CP Y Y Y Y Y Y Y Y Y Y Y
-smp_mb__before_atomic() CP Y Y Y a a a a Y
-smp_mb__after_atomic() CP a a Y Y Y Y Y Y
-
-
-Key: C: Ordering is cumulative
- P: Ordering propagates
- R: Read, for example, READ_ONCE(), or read portion of RMW
- W: Write, for example, WRITE_ONCE(), or write portion of RMW
- Y: Provides ordering
- a: Provides ordering given intervening RMW atomic operation
- DR: Dependent read (address dependency)
- DW: Dependent write (address, data, or control dependency)
- RMW: Atomic read-modify-write operation
- SELF: Orders self, as opposed to accesses before and/or after
- SV: Orders later accesses to the same variable
diff --git a/tools/memory-model/Documentation/explanation.txt b/tools/memory-model/Documentation/explanation.rst
similarity index 92%
rename from tools/memory-model/Documentation/explanation.txt
rename to tools/memory-model/Documentation/explanation.rst
index 68caa9a976d0..9b5d10cef0c2 100644
--- a/tools/memory-model/Documentation/explanation.txt
+++ b/tools/memory-model/Documentation/explanation.rst
@@ -1,5 +1,6 @@
+========================================================
Explanation of the Linux-Kernel Memory Consistency Model
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+========================================================
:Author: Alan Stern <stern@rowland.harvard.edu>
:Created: October 2017
@@ -107,7 +108,7 @@ and the flag are memory locations shared between the two CPUs.
We can abstract out the important pieces of the driver code as follows
(the reason for using WRITE_ONCE() and READ_ONCE() instead of simple
-assignment statements is discussed later):
+assignment statements is discussed later)::
int buf = 0, flag = 0;
@@ -219,7 +220,7 @@ Ordering). This model predicts that the undesired outcome for the MP
pattern cannot occur, but in other respects it differs from Sequential
Consistency. One example is the Store Buffer (SB) pattern, in which
each CPU stores to its own shared location and then loads from the
-other CPU's location:
+other CPU's location::
int x = 0, y = 0;
@@ -264,7 +265,7 @@ The counterpart to ordering is a cycle. Ordering rules out cycles:
It's not possible to have X ordered before Y, Y ordered before Z, and
Z ordered before X, because this would mean that X is ordered before
itself. The analysis of the MP example under Sequential Consistency
-involved just such an impossible cycle:
+involved just such an impossible cycle::
W: P0 stores 1 to flag executes before
X: P1 loads 1 from flag executes before
@@ -331,7 +332,7 @@ can think of it as the order in which statements occur in the source
code after branches are taken into account and loops have been
unrolled. A better description might be the order in which
instructions are presented to a CPU's execution unit. Thus, we say
-that X is po-before Y (written as "X ->po Y" in formulas) if X occurs
+that X is po-before Y (written as ``X ->po Y`` in formulas) if X occurs
before Y in the instruction stream.
This is inherently a single-CPU relation; two instructions executing
@@ -388,7 +389,7 @@ The protections provided by READ_ONCE(), WRITE_ONCE(), and others are
not perfect; and under some circumstances it is possible for the
compiler to undermine the memory model. Here is an example. Suppose
both branches of an "if" statement store the same value to the same
-location:
+location::
r1 = READ_ONCE(x);
if (r1) {
@@ -402,7 +403,7 @@ location:
For this code, the LKMM predicts that the load from x will always be
executed before either of the stores to y. However, a compiler could
lift the stores out of the conditional, transforming the code into
-something resembling:
+something resembling::
r1 = READ_ONCE(x);
WRITE_ONCE(y, 2);
@@ -418,7 +419,7 @@ model's original prediction could be invalidated by the compiler.
Another issue arises from the fact that in C, arguments to many
operators and function calls can be evaluated in any order. For
-example:
+example::
r1 = f(5) + g(6);
@@ -440,7 +441,7 @@ control (ctrl).
A read and a write event are linked by a data dependency if the value
obtained by the read affects the value stored by the write. As a very
-simple example:
+simple example::
int x, y;
@@ -455,7 +456,7 @@ values of multiple reads.
A read event and another memory access event are linked by an address
dependency if the value obtained by the read affects the location
accessed by the other event. The second event can be either a read or
-a write. Here's another simple example:
+a write. Here's another simple example::
int a[20];
int i;
@@ -471,7 +472,7 @@ pointer.
Finally, a read event and another memory access event are linked by a
control dependency if the value obtained by the read affects whether
-the second event is executed at all. Simple example:
+the second event is executed at all. Simple example::
int x, y;
@@ -484,9 +485,9 @@ which depends on the value obtained by the READ_ONCE(); hence there is
a control dependency from the load to the store.
It should be pretty obvious that events can only depend on reads that
-come earlier in program order. Symbolically, if we have R ->data X,
-R ->addr X, or R ->ctrl X (where R is a read event), then we must also
-have R ->po X. It wouldn't make sense for a computation to depend
+come earlier in program order. Symbolically, if we have ``R ->data X``,
+``R ->addr X``, or ``R ->ctrl X`` (where R is a read event), then we must also
+have ``R ->po X``. It wouldn't make sense for a computation to depend
somehow on a value that doesn't get loaded from shared memory until
later in the code!
@@ -497,7 +498,7 @@ THE READS-FROM RELATION: rf, rfi, and rfe
The reads-from relation (rf) links a write event to a read event when
the value loaded by the read is the value that was stored by the
write. In colloquial terms, the load "reads from" the store. We
-write W ->rf R to indicate that the load R reads from the store W. We
+write ``W ->rf R`` to indicate that the load R reads from the store W. We
further distinguish the cases where the load and the store occur on
the same CPU (internal reads-from, or rfi) and where they occur on
different CPUs (external reads-from, or rfe).
@@ -510,7 +511,7 @@ Usage of the rf relation implicitly assumes that loads will always
read from a single store. It doesn't apply properly in the presence
of load-tearing, where a load obtains some of its bits from one store
and some of them from another store. Fortunately, use of READ_ONCE()
-and WRITE_ONCE() will prevent load-tearing; it's not possible to have:
+and WRITE_ONCE() will prevent load-tearing; it's not possible to have::
int x = 0;
@@ -530,7 +531,7 @@ and end up with r1 = 0x1200 (partly from x's initial value and partly
from the value stored by P0).
On the other hand, load-tearing is unavoidable when mixed-size
-accesses are used. Consider this example:
+accesses are used. Consider this example::
union {
u32 w;
@@ -578,26 +579,26 @@ that value comes third, and so on.
You can think of the coherence order as being the order in which the
stores reach x's location in memory (or if you prefer a more
hardware-centric view, the order in which the stores get written to
-x's cache line). We write W ->co W' if W comes before W' in the
+x's cache line). We write ``W ->co W'`` if W comes before W' in the
coherence order, that is, if the value stored by W gets overwritten,
directly or indirectly, by the value stored by W'.
Coherence order is required to be consistent with program order. This
requirement takes the form of four coherency rules:
- Write-write coherence: If W ->po-loc W' (i.e., W comes before
+ Write-write coherence: If ``W ->po-loc W'`` (i.e., W comes before
W' in program order and they access the same location), where W
- and W' are two stores, then W ->co W'.
+ and W' are two stores, then ``W ->co W'``.
- Write-read coherence: If W ->po-loc R, where W is a store and R
+ Write-read coherence: If ``W ->po-loc R``, where W is a store and R
is a load, then R must read from W or from some other store
which comes after W in the coherence order.
- Read-write coherence: If R ->po-loc W, where R is a load and W
+ Read-write coherence: If ``R ->po-loc W``, where R is a load and W
is a store, then the store which R reads from must come before
W in the coherence order.
- Read-read coherence: If R ->po-loc R', where R and R' are two
+ Read-read coherence: If ``R ->po-loc R'``, where R and R' are two
loads, then either they read from the same store or else the
store read by R comes before the store read by R' in the
coherence order.
@@ -612,7 +613,7 @@ requirement that every store eventually becomes visible to every CPU.)
Any reasonable memory model will include cache coherence. Indeed, our
expectation of cache coherence is so deeply ingrained that violations
of its requirements look more like hardware bugs than programming
-errors:
+errors::
int x;
@@ -628,6 +629,8 @@ write-write coherence rule: Since the store of 23 comes later in
program order, it must also come later in x's coherence order and
thus must overwrite the store of 17.
+::
+
int x = 0;
P0()
@@ -644,6 +647,8 @@ program order, so it must not read from that store but rather from one
coming earlier in the coherence order (in this case, x's initial
value).
+::
+
int x = 0;
P0()
@@ -689,12 +694,12 @@ THE FROM-READS RELATION: fr, fri, and fre
The from-reads relation (fr) can be a little difficult for people to
grok. It describes the situation where a load reads a value that gets
-overwritten by a store. In other words, we have R ->fr W when the
+overwritten by a store. In other words, we have ``R ->fr W`` when the
value that R reads is overwritten (directly or indirectly) by W, or
equivalently, when R reads from a store which comes earlier than W in
the coherence order.
-For example:
+For example::
int x = 0;
@@ -718,9 +723,11 @@ different CPUs).
Note that the fr relation is determined entirely by the rf and co
relations; it is not independent. Given a read event R and a write
-event W for the same location, we will have R ->fr W if and only if
+event W for the same location, we will have ``R ->fr W`` if and only if
the write which R reads from is co-before W. In symbols,
+::
+
(R ->fr W) := (there exists W' with W' ->rf R and W' ->co W).
@@ -843,13 +850,13 @@ defined to link memory access events E and F whenever:
event occurs between them in program order; or
F is a release fence and some X comes before F in program order,
- where either X = E or else E ->rf X; or
+ where either ``X = E`` or else ``E ->rf X``; or
A strong fence event occurs between some X and F in program
- order, where either X = E or else E ->rf X.
+ order, where either ``X = E`` or else ``E ->rf X``.
The operational model requires that whenever W and W' are both stores
-and W ->cumul-fence W', then W must propagate to any given CPU
+and ``W ->cumul-fence W'``, then W must propagate to any given CPU
before W' does. However, for different CPUs C and C', it does not
require W to propagate to C before W' propagates to C'.
@@ -903,11 +910,11 @@ first for CPU 0, then CPU 1, etc.
You can check that the four coherency rules imply that the rf, co, fr,
and po-loc relations agree with this global ordering; in other words,
-whenever we have X ->rf Y or X ->co Y or X ->fr Y or X ->po-loc Y, the
+whenever we have ``X ->rf Y`` or ``X ->co Y`` or ``X ->fr Y`` or ``X ->po-loc Y``, the
X event comes before the Y event in the global ordering. The LKMM's
"coherence" axiom expresses this by requiring the union of these
relations not to have any cycles. This means it must not be possible
-to find events
+to find events::
X0 -> X1 -> X2 -> ... -> Xn -> X0,
@@ -929,7 +936,7 @@ this case) does not get altered between the read and the write events
making up the atomic operation. In particular, if two CPUs perform
atomic_inc(&x) concurrently, it must be guaranteed that the final
value of x will be the initial value plus two. We should never have
-the following sequence of events:
+the following sequence of events::
CPU 0 loads x obtaining 13;
CPU 1 loads x obtaining 13;
@@ -951,6 +958,8 @@ atomic read-modify-write and W' is the write event which R reads from,
there must not be any stores coming between W' and W in the coherence
order. Equivalently,
+::
+
(R ->rmw W) implies (there is no X with R ->fr X and X ->co W),
where the rmw relation links the read and write events making up each
@@ -968,7 +977,7 @@ po.
The operational model already includes a description of one such
situation: Fences are a source of ppo links. Suppose X and Y are
-memory accesses with X ->po Y; then the CPU must execute X before Y if
+memory accesses with ``X ->po Y``; then the CPU must execute X before Y if
any of the following hold:
A strong (smp_mb() or synchronize_rcu()) fence occurs between
@@ -987,7 +996,7 @@ any of the following hold:
Another possibility, not mentioned earlier but discussed in the next
section, is:
- X and Y are both loads, X ->addr Y (i.e., there is an address
+ X and Y are both loads, ``X ->addr Y`` (i.e., there is an address
dependency from X to Y), and X is a READ_ONCE() or an atomic
access.
@@ -1021,7 +1030,7 @@ includes address dependencies to loads in the ppo relation.
On the other hand, dependencies can indirectly affect the ordering of
two loads. This happens when there is a dependency from a load to a
-store and a second, po-later load reads from that store:
+store and a second, po-later load reads from that store::
R ->dep W ->rfi R',
@@ -1036,7 +1045,7 @@ R; if the speculation turns out to be wrong then the CPU merely has to
restart or abandon R'.
(In theory, a CPU might forward a store to a load when it runs across
-an address dependency like this:
+an address dependency like this::
r1 = READ_ONCE(ptr);
WRITE_ONCE(*r1, 17);
@@ -1048,7 +1057,7 @@ However, none of the architectures supported by the Linux kernel do
this.)
Two memory accesses of the same location must always be executed in
-program order if the second access is a store. Thus, if we have
+program order if the second access is a store. Thus, if we have::
R ->po-loc W
@@ -1056,7 +1065,7 @@ program order if the second access is a store. Thus, if we have
access the same location), the CPU is obliged to execute W after R.
If it executed W first then the memory subsystem would respond to R's
read request with the value stored by W (or an even later store), in
-violation of the read-write coherence rule. Similarly, if we had
+violation of the read-write coherence rule. Similarly, if we had::
W ->po-loc W'
@@ -1074,7 +1083,7 @@ AND THEN THERE WAS ALPHA
As mentioned above, the Alpha architecture is unique in that it does
not appear to respect address dependencies to loads. This means that
-code such as the following:
+code such as the following::
int x = 0;
int y = -1;
@@ -1128,7 +1137,7 @@ nothing at all on non-Alpha builds) after every READ_ONCE() and atomic
load. The effect of the fence is to cause the CPU not to execute any
po-later instructions until after the local cache has finished
processing all the stores it has already received. Thus, if the code
-was changed to:
+was changed to::
P1()
{
@@ -1167,25 +1176,25 @@ The happens-before relation (hb) links memory accesses that have to
execute in a certain order. hb includes the ppo relation and two
others, one of which is rfe.
-W ->rfe R implies that W and R are on different CPUs. It also means
+``W ->rfe R`` implies that W and R are on different CPUs. It also means
that W's store must have propagated to R's CPU before R executed;
otherwise R could not have read the value stored by W. Therefore W
-must have executed before R, and so we have W ->hb R.
+must have executed before R, and so we have ``W ->hb R``.
-The equivalent fact need not hold if W ->rfi R (i.e., W and R are on
+The equivalent fact need not hold if ``W ->rfi R`` (i.e., W and R are on
the same CPU). As we have already seen, the operational model allows
W's value to be forwarded to R in such cases, meaning that R may well
execute before W does.
It's important to understand that neither coe nor fre is included in
hb, despite their similarities to rfe. For example, suppose we have
-W ->coe W'. This means that W and W' are stores to the same location,
+``W ->coe W'``. This means that W and W' are stores to the same location,
they execute on different CPUs, and W comes before W' in the coherence
order (i.e., W' overwrites W). Nevertheless, it is possible for W' to
execute before W, because the decision as to which store overwrites
the other is made later by the memory subsystem. When the stores are
nearly simultaneous, either one can come out on top. Similarly,
-R ->fre W means that W overwrites the value which R reads, but it
+``R ->fre W`` means that W overwrites the value which R reads, but it
doesn't mean that W has to execute after R. All that's necessary is
for the memory subsystem not to propagate W to R's CPU until after R
has executed, which is possible if W executes shortly before R.
@@ -1199,7 +1208,7 @@ the first event in the coherence order and propagates to C before the
second event executes.
This is best explained with some examples. The simplest case looks
-like this:
+like this::
int x;
@@ -1225,7 +1234,7 @@ event, because P1's store came after P0's store in x's coherence
order, and P1's store propagated to P0 before P0's load executed.
An equally simple case involves two loads of the same location that
-read from different stores:
+read from different stores::
int x = 0;
@@ -1252,7 +1261,7 @@ P1's store propagated to P0 before P0's second load executed.
Less trivial examples of prop all involve fences. Unlike the simple
examples above, they can require that some instructions are executed
-out of program order. This next one should look familiar:
+out of program order. This next one should look familiar::
int buf = 0, flag = 0;
@@ -1303,7 +1312,7 @@ rfe link. You can concoct more exotic examples, containing more than
one fence, although this quickly leads to diminishing returns in terms
of complexity. For instance, here's an example containing a coe link
followed by two fences and an rfe link, utilizing the fact that
-release fences are A-cumulative:
+release fences are A-cumulative::
int x, y, z;
@@ -1363,7 +1372,7 @@ links. Let's see how this definition works out.
Consider first the case where E is a store (implying that the sequence
of links begins with coe). Then there are events W, X, Y, and Z such
-that:
+that::
E ->coe W ->cumul-fence* X ->rfe? Y ->strong-fence Z ->hb* F,
@@ -1384,13 +1393,13 @@ The existence of a pb link from E to F implies that E must execute
before F. To see why, suppose that F executed first. Then W would
have propagated to E's CPU before E executed. If E was a store, the
memory subsystem would then be forced to make E come after W in the
-coherence order, contradicting the fact that E ->coe W. If E was a
+coherence order, contradicting the fact that ``E ->coe W``. If E was a
load, the memory subsystem would then be forced to satisfy E's read
request with the value stored by W or an even later store,
-contradicting the fact that E ->fre W.
+contradicting the fact that ``E ->fre W``.
A good example illustrating how pb works is the SB pattern with strong
-fences:
+fences::
int x = 0, y = 0;
@@ -1449,18 +1458,18 @@ span a full grace period. In more detail, the Guarantee says:
For any critical section C and any grace period G, at least
one of the following statements must hold:
-(1) C ends before G does, and in addition, every store that
- propagates to C's CPU before the end of C must propagate to
- every CPU before G ends.
+ (1) C ends before G does, and in addition, every store that
+ propagates to C's CPU before the end of C must propagate to
+ every CPU before G ends.
-(2) G starts before C does, and in addition, every store that
- propagates to G's CPU before the start of G must propagate
- to every CPU before C starts.
+ (2) G starts before C does, and in addition, every store that
+ propagates to G's CPU before the start of G must propagate
+ to every CPU before C starts.
In particular, it is not possible for a critical section to both start
before and end after a grace period.
-Here is a simple example of RCU in action:
+Here is a simple example of RCU in action::
int x, y;
@@ -1509,9 +1518,9 @@ entirely clear. The LKMM formalizes this notion by means of the
rcu-link relation. rcu-link encompasses a very general notion of
"before": If E and F are RCU fence events (i.e., rcu_read_lock(),
rcu_read_unlock(), or synchronize_rcu()) then among other things,
-E ->rcu-link F includes cases where E is po-before some memory-access
+``E ->rcu-link F`` includes cases where E is po-before some memory-access
event X, F is po-after some memory-access event Y, and we have any of
-X ->rfe Y, X ->co Y, or X ->fr Y.
+``X ->rfe Y``, ``X ->co Y``, or ``X ->fr Y``.
The formal definition of the rcu-link relation is more than a little
obscure, and we won't give it here. It is closely related to the pb
@@ -1523,50 +1532,50 @@ The LKMM also defines the rcu-gp and rcu-rscsi relations. They bring
grace periods and read-side critical sections into the picture, in the
following way:
- E ->rcu-gp F means that E and F are in fact the same event,
+ ``E ->rcu-gp F`` means that E and F are in fact the same event,
and that event is a synchronize_rcu() fence (i.e., a grace
period).
- E ->rcu-rscsi F means that E and F are the rcu_read_unlock()
+ ``E ->rcu-rscsi F`` means that E and F are the rcu_read_unlock()
and rcu_read_lock() fence events delimiting some read-side
critical section. (The 'i' at the end of the name emphasizes
that this relation is "inverted": It links the end of the
critical section to the start.)
If we think of the rcu-link relation as standing for an extended
-"before", then X ->rcu-gp Y ->rcu-link Z roughly says that X is a
+"before", then ``X ->rcu-gp Y ->rcu-link Z`` roughly says that X is a
grace period which ends before Z begins. (In fact it covers more than
this, because it also includes cases where some store propagates to
Z's CPU before Z begins but doesn't propagate to some other CPU until
-after X ends.) Similarly, X ->rcu-rscsi Y ->rcu-link Z says that X is
+after X ends.) Similarly, ``X ->rcu-rscsi Y ->rcu-link Z`` says that X is
the end of a critical section which starts before Z begins.
The LKMM goes on to define the rcu-fence relation as a sequence of
rcu-gp and rcu-rscsi links separated by rcu-link links, in which the
number of rcu-gp links is >= the number of rcu-rscsi links. For
-example:
+example::
X ->rcu-gp Y ->rcu-link Z ->rcu-rscsi T ->rcu-link U ->rcu-gp V
-would imply that X ->rcu-fence V, because this sequence contains two
+would imply that ``X ->rcu-fence V``, because this sequence contains two
rcu-gp links and one rcu-rscsi link. (It also implies that
-X ->rcu-fence T and Z ->rcu-fence V.) On the other hand:
+``X ->rcu-fence T`` and ``Z ->rcu-fence V``.) On the other hand::
X ->rcu-rscsi Y ->rcu-link Z ->rcu-rscsi T ->rcu-link U ->rcu-gp V
-does not imply X ->rcu-fence V, because the sequence contains only
+does not imply ``X ->rcu-fence V``, because the sequence contains only
one rcu-gp link but two rcu-rscsi links.
The rcu-fence relation is important because the Grace Period Guarantee
means that rcu-fence acts kind of like a strong fence. In particular,
-E ->rcu-fence F implies not only that E begins before F ends, but also
+``E ->rcu-fence F`` implies not only that E begins before F ends, but also
that any write po-before E will propagate to every CPU before any
instruction po-after F can execute. (However, it does not imply that
E must execute before F; in fact, each synchronize_rcu() fence event
is linked to itself by rcu-fence as a degenerate case.)
To prove this in full generality requires some intellectual effort.
-We'll consider just a very simple case:
+We'll consider just a very simple case::
G ->rcu-gp W ->rcu-link Z ->rcu-rscsi F.
@@ -1595,13 +1604,13 @@ covered by rcu-fence.
Finally, the LKMM defines the RCU-before (rb) relation in terms of
rcu-fence. This is done in essentially the same way as the pb
relation was defined in terms of strong-fence. We will omit the
-details; the end result is that E ->rb F implies E must execute
-before F, just as E ->pb F does (and for much the same reasons).
+details; the end result is that ``E ->rb F`` implies E must execute
+before F, just as ``E ->pb F`` does (and for much the same reasons).
Putting this all together, the LKMM expresses the Grace Period
Guarantee by requiring that the rb relation does not contain a cycle.
Equivalently, this "rcu" axiom requires that there are no events E
-and F with E ->rcu-link F ->rcu-fence E. Or to put it a third way,
+and F with ``E ->rcu-link F ->rcu-fence E``. Or to put it a third way,
the axiom requires that there are no cycles consisting of rcu-gp and
rcu-rscsi alternating with rcu-link, where the number of rcu-gp links
is >= the number of rcu-rscsi links.
@@ -1621,7 +1630,7 @@ question, and let S be the synchronize_rcu() fence event for the grace
period. Saying that the critical section starts before S means there
are events Q and R where Q is po-after L (which marks the start of the
critical section), Q is "before" R in the sense used by the rcu-link
-relation, and R is po-before the grace period S. Thus we have:
+relation, and R is po-before the grace period S. Thus we have::
L ->rcu-link S.
@@ -1629,20 +1638,20 @@ Let W be the store mentioned above, let Y come before the end of the
critical section and witness that W propagates to the critical
section's CPU by reading from W, and let Z on some arbitrary CPU be a
witness that W has not propagated to that CPU, where Z happens after
-some event X which is po-after S. Symbolically, this amounts to:
+some event X which is po-after S. Symbolically, this amounts to::
S ->po X ->hb* Z ->fr W ->rf Y ->po U.
The fr link from Z to W indicates that W has not propagated to Z's CPU
at the time that Z executes. From this, it can be shown (see the
discussion of the rcu-link relation earlier) that S and U are related
-by rcu-link:
+by rcu-link::
S ->rcu-link U.
-Since S is a grace period we have S ->rcu-gp S, and since L and U are
-the start and end of the critical section C we have U ->rcu-rscsi L.
-From this we obtain:
+Since S is a grace period we have ``S ->rcu-gp S``, and since L and U are
+the start and end of the critical section C we have ``U ->rcu-rscsi L``.
+From this we obtain::
S ->rcu-gp S ->rcu-link U ->rcu-rscsi L ->rcu-link S,
@@ -1651,7 +1660,7 @@ the Grace Period Guarantee.
For something a little more down-to-earth, let's see how the axiom
works out in practice. Consider the RCU code example from above, this
-time with statement labels added:
+time with statement labels added::
int x, y;
@@ -1674,20 +1683,20 @@ time with statement labels added:
If r2 = 0 at the end then P0's store at Y overwrites the value that
-P1's load at W reads from, so we have W ->fre Y. Since S ->po W and
-also Y ->po U, we get S ->rcu-link U. In addition, S ->rcu-gp S
+P1's load at W reads from, so we have ``W ->fre Y``. Since ``S ->po W`` and
+also ``Y ->po U``, we get ``S ->rcu-link U``. In addition, ``S ->rcu-gp S``
because S is a grace period.
If r1 = 1 at the end then P1's load at Z reads from P0's store at X,
-so we have X ->rfe Z. Together with L ->po X and Z ->po S, this
-yields L ->rcu-link S. And since L and U are the start and end of a
-critical section, we have U ->rcu-rscsi L.
+so we have ``X ->rfe Z``. Together with ``L ->po X`` and ``Z ->po S``, this
+yields ``L ->rcu-link S``. And since L and U are the start and end of a
+critical section, we have ``U ->rcu-rscsi L``.
-Then U ->rcu-rscsi L ->rcu-link S ->rcu-gp S ->rcu-link U is a
+Then ``U ->rcu-rscsi L ->rcu-link S ->rcu-gp S ->rcu-link U`` is a
forbidden cycle, violating the "rcu" axiom. Hence the outcome is not
allowed by the LKMM, as we would expect.
-For contrast, let's see what can happen in a more complicated example:
+For contrast, let's see what can happen in a more complicated example::
int x, y, z;
@@ -1720,28 +1729,28 @@ For contrast, let's see what can happen in a more complicated example:
U2: rcu_read_unlock();
}
-If r0 = r1 = r2 = 1 at the end, then similar reasoning to before shows
-that U0 ->rcu-rscsi L0 ->rcu-link S1 ->rcu-gp S1 ->rcu-link U2 ->rcu-rscsi
-L2 ->rcu-link U0. However this cycle is not forbidden, because the
+If ``r0 = r1 = r2 = 1`` at the end, then similar reasoning to before shows
+that ``U0 ->rcu-rscsi L0 ->rcu-link S1 ->rcu-gp S1 ->rcu-link U2 ->rcu-rscsi
+L2 ->rcu-link U0``. However this cycle is not forbidden, because the
sequence of relations contains fewer instances of rcu-gp (one) than of
rcu-rscsi (two). Consequently the outcome is allowed by the LKMM.
The following instruction timing diagram shows how it might actually
-occur:
-
-P0 P1 P2
--------------------- -------------------- --------------------
-rcu_read_lock()
-WRITE_ONCE(y, 1)
- r1 = READ_ONCE(y)
- synchronize_rcu() starts
- . rcu_read_lock()
- . WRITE_ONCE(x, 1)
-r0 = READ_ONCE(x) .
-rcu_read_unlock() .
- synchronize_rcu() ends
- WRITE_ONCE(z, 1)
- r2 = READ_ONCE(z)
- rcu_read_unlock()
+occur::
+
+ P0 P1 P2
+ -------------------- -------------------- --------------------
+ rcu_read_lock()
+ WRITE_ONCE(y, 1)
+ r1 = READ_ONCE(y)
+ synchronize_rcu() starts
+ . rcu_read_lock()
+ . WRITE_ONCE(x, 1)
+ r0 = READ_ONCE(x) .
+ rcu_read_unlock() .
+ synchronize_rcu() ends
+ WRITE_ONCE(z, 1)
+ r2 = READ_ONCE(z)
+ rcu_read_unlock()
This requires P0 and P2 to execute their loads and stores out of
program order, but of course they are allowed to do so. And as you
@@ -1767,26 +1776,26 @@ The LKMM includes locking. In fact, there is special code for locking
in the formal model, added in order to make tools run faster.
However, this special code is intended to be more or less equivalent
to concepts we have already covered. A spinlock_t variable is treated
-the same as an int, and spin_lock(&s) is treated almost the same as:
+the same as an int, and spin_lock(&s) is treated almost the same as::
while (cmpxchg_acquire(&s, 0, 1) != 0)
cpu_relax();
This waits until s is equal to 0 and then atomically sets it to 1,
and the read part of the cmpxchg operation acts as an acquire fence.
-An alternate way to express the same thing would be:
+An alternate way to express the same thing would be::
r = xchg_acquire(&s, 1);
along with a requirement that at the end, r = 0. Similarly,
-spin_trylock(&s) is treated almost the same as:
+spin_trylock(&s) is treated almost the same as::
return !cmpxchg_acquire(&s, 0, 1);
which atomically sets s to 1 if it is currently equal to 0 and returns
true if it succeeds (the read part of the cmpxchg operation acts as an
acquire fence only if the operation is successful). spin_unlock(&s)
-is treated almost the same as:
+is treated almost the same as::
smp_store_release(&s, 0);
@@ -1802,7 +1811,7 @@ requires that every instruction po-before the lock-release must
execute before any instruction po-after the lock-acquire. This would
naturally hold if the release and acquire operations were on different
CPUs, but the LKMM says it holds even when they are on the same CPU.
-For example:
+For example::
int x, y;
spinlock_t s;
@@ -1833,7 +1842,7 @@ MP pattern).
This requirement does not apply to ordinary release and acquire
fences, only to lock-related operations. For instance, suppose P0()
-in the example had been written as:
+in the example had been written as::
P0()
{
@@ -1847,7 +1856,7 @@ in the example had been written as:
Then the CPU would be allowed to forward the s = 1 value from the
smp_store_release() to the smp_load_acquire(), executing the
-instructions in the following order:
+instructions in the following order::
r3 = smp_load_acquire(&s); // Obtains r3 = 1
r2 = READ_ONCE(y);
@@ -1859,7 +1868,7 @@ and thus it could load y before x, obtaining r2 = 0 and r1 = 1.
Second, when a lock-acquire reads from a lock-release, and some other
stores W and W' occur po-before the lock-release and po-after the
lock-acquire respectively, the LKMM requires that W must propagate to
-each CPU before W' does. For example, consider:
+each CPU before W' does. For example, consider::
int x, y;
spinlock_t x;
@@ -1928,7 +1937,7 @@ smp_store_release().) The final effect is the same.
Although we didn't mention it above, the instruction execution
ordering provided by the smp_rmb() fence doesn't apply to read events
that are part of a non-value-returning atomic update. For instance,
-given:
+given::
atomic_inc(&x);
smp_rmb();
@@ -1967,14 +1976,14 @@ they behave as follows:
events.
Interestingly, RCU and locking each introduce the possibility of
-deadlock. When faced with code sequences such as:
+deadlock. When faced with code sequences such as::
spin_lock(&s);
spin_lock(&s);
spin_unlock(&s);
spin_unlock(&s);
-or:
+or::
rcu_read_lock();
synchronize_rcu();
@@ -1984,7 +1993,7 @@ what does the LKMM have to say? Answer: It says there are no allowed
executions at all, which makes sense. But this can also lead to
misleading results, because if a piece of code has multiple possible
executions, some of which deadlock, the model will report only on the
-non-deadlocking executions. For example:
+non-deadlocking executions. For example::
int x, y;
diff --git a/tools/memory-model/Documentation/index.rst b/tools/memory-model/Documentation/index.rst
new file mode 100644
index 000000000000..0e53d83a5a48
--- /dev/null
+++ b/tools/memory-model/Documentation/index.rst
@@ -0,0 +1,20 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+========================
+Linux Memory Model Tools
+========================
+
+.. toctree::
+ :maxdepth: 1
+
+ explanation
+ recipes
+ references
+ cheatsheet
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
diff --git a/tools/memory-model/Documentation/recipes.txt b/tools/memory-model/Documentation/recipes.rst
similarity index 96%
rename from tools/memory-model/Documentation/recipes.txt
rename to tools/memory-model/Documentation/recipes.rst
index 7fe8d7aa3029..0229a431b1a2 100644
--- a/tools/memory-model/Documentation/recipes.txt
+++ b/tools/memory-model/Documentation/recipes.rst
@@ -1,3 +1,8 @@
+=======
+Recipes
+=======
+
+
This document provides "recipes", that is, litmus tests for commonly
occurring situations, as well as a few that illustrate subtly broken but
attractive nuisances. Many of these recipes include example code from
@@ -67,7 +72,7 @@ has acquired a given lock sees any changes previously seen or made by any
CPU before it released that same lock. Note that this statement is a bit
stronger than "Any CPU holding a given lock sees all changes made by any
CPU during the time that CPU was holding this same lock". For example,
-consider the following pair of code fragments:
+consider the following pair of code fragments::
/* See MP+polocks.litmus. */
void CPU0(void)
@@ -93,7 +98,7 @@ value of r1 must also be equal to 1. In contrast, the weaker rule would
say nothing about the final value of r1.
The converse to the basic rule also holds, as illustrated by the
-following litmus test:
+following litmus test::
/* See MP+porevlocks.litmus. */
void CPU0(void)
@@ -124,7 +129,7 @@ across multiple CPUs.
However, it is not necessarily the case that accesses ordered by
locking will be seen as ordered by CPUs not holding that lock.
-Consider this example:
+Consider this example::
/* See Z6.0+pooncerelease+poacquirerelease+fencembonceonce.litmus. */
void CPU0(void)
@@ -157,7 +162,7 @@ CPU2() never acquired the lock, and thus did not benefit from the
lock's ordering properties.
Ordering can be extended to CPUs not holding the lock by careful use
-of smp_mb__after_spinlock():
+of smp_mb__after_spinlock()::
/* See Z6.0+pooncelock+poonceLock+pombonce.litmus. */
void CPU0(void)
@@ -214,7 +219,7 @@ Release and acquire
~~~~~~~~~~~~~~~~~~~
Use of smp_store_release() and smp_load_acquire() is one way to force
-the desired MP ordering. The general approach is shown below:
+the desired MP ordering. The general approach is shown below::
/* See MP+pooncerelease+poacquireonce.litmus. */
void CPU0(void)
@@ -245,7 +250,7 @@ Assign and dereference
Use of rcu_assign_pointer() and rcu_dereference() is quite similar to the
use of smp_store_release() and smp_load_acquire(), except that both
rcu_assign_pointer() and rcu_dereference() operate on RCU-protected
-pointers. The general approach is shown below:
+pointers. The general approach is shown below::
/* See MP+onceassign+derefonce.litmus. */
int z;
@@ -290,7 +295,7 @@ Write and read memory barriers
It is usually better to use smp_store_release() instead of smp_wmb()
and to use smp_load_acquire() instead of smp_rmb(). However, the older
smp_wmb() and smp_rmb() APIs are still heavily used, so it is important
-to understand their use cases. The general approach is shown below:
+to understand their use cases. The general approach is shown below::
/* See MP+fencewmbonceonce+fencermbonceonce.litmus. */
void CPU0(void)
@@ -312,7 +317,7 @@ smp_rmb() macro orders prior loads against later loads. Therefore, if
the final value of r0 is 1, the final value of r1 must also be 1.
The xlog_state_switch_iclogs() function in fs/xfs/xfs_log.c contains
-the following write-side code fragment:
+the following write-side code fragment::
log->l_curr_block -= log->l_logBBsize;
ASSERT(log->l_curr_block >= 0);
@@ -327,7 +332,7 @@ the corresponding read-side code fragment:
cur_block = READ_ONCE(log->l_curr_block);
Alternatively, consider the following comment in function
-perf_output_put_handle() in kernel/events/ring_buffer.c:
+perf_output_put_handle() in kernel/events/ring_buffer.c::
* kernel user
*
@@ -358,7 +363,7 @@ absence of any ordering it is quite possible that this may happen, as
can be seen in the LB+poonceonces.litmus litmus test.
One way of avoiding the counter-intuitive outcome is through the use of a
-control dependency paired with a full memory barrier:
+control dependency paired with a full memory barrier::
/* See LB+fencembonceonce+ctrlonceonce.litmus. */
void CPU0(void)
@@ -382,7 +387,7 @@ The A/D pairing from the ring-buffer use case shown earlier also
illustrates LB. Here is a repeat of the comment in
perf_output_put_handle() in kernel/events/ring_buffer.c, showing a
control dependency on the kernel side and a full memory barrier on
-the user side:
+the user side::
* kernel user
*
@@ -407,7 +412,7 @@ Release-acquire chains
Release-acquire chains are a low-overhead, flexible, and easy-to-use
method of maintaining order. However, they do have some limitations that
-need to be fully understood. Here is an example that maintains order:
+need to be fully understood. Here is an example that maintains order::
/* See ISA2+pooncerelease+poacquirerelease+poacquireonce.litmus. */
void CPU0(void)
@@ -436,7 +441,7 @@ example, ordering would still be preserved if CPU1()'s smp_load_acquire()
invocation was replaced with READ_ONCE().
It is tempting to assume that CPU0()'s store to x is globally ordered
-before CPU1()'s store to z, but this is not the case:
+before CPU1()'s store to z, but this is not the case::
/* See Z6.0+pooncerelease+poacquirerelease+mbonceonce.litmus. */
void CPU0(void)
@@ -474,7 +479,7 @@ Store buffering
Store buffering can be thought of as upside-down load buffering, so
that one CPU first stores to one variable and then loads from a second,
while another CPU stores to the second variable and then loads from the
-first. Preserving order requires nothing less than full barriers:
+first. Preserving order requires nothing less than full barriers::
/* See SB+fencembonceonces.litmus. */
void CPU0(void)
@@ -498,7 +503,7 @@ this counter-intuitive outcome.
This pattern most famously appears as part of Dekker's locking
algorithm, but it has a much more practical use within the Linux kernel
of ordering wakeups. The following comment taken from waitqueue_active()
-in include/linux/wait.h shows the canonical pattern:
+in include/linux/wait.h shows the canonical pattern::
* CPU0 - waker CPU1 - waiter
*
@@ -550,16 +555,16 @@ The strength of memory ordering required for a given litmus test to
avoid a counter-intuitive outcome depends on the types of relations
linking the memory accesses for the outcome in question:
-o If all links are write-to-read links, then the weakest
+- If all links are write-to-read links, then the weakest
possible ordering within each CPU suffices. For example, in
the LB litmus test, a control dependency was enough to do the
job.
-o If all but one of the links are write-to-read links, then a
+- If all but one of the links are write-to-read links, then a
release-acquire chain suffices. Both the MP and the ISA2
litmus tests illustrate this case.
-o If more than one of the links are something other than
+- If more than one of the links are something other than
write-to-read links, then a full memory barrier is required
between each successive pair of non-write-to-read links. This
case is illustrated by the Z6.0 litmus tests, both in the
diff --git a/tools/memory-model/Documentation/references.txt b/tools/memory-model/Documentation/references.rst
similarity index 71%
rename from tools/memory-model/Documentation/references.txt
rename to tools/memory-model/Documentation/references.rst
index b177f3e4a614..275876cd10b8 100644
--- a/tools/memory-model/Documentation/references.txt
+++ b/tools/memory-model/Documentation/references.rst
@@ -1,3 +1,7 @@
+==========
+References
+==========
+
This document provides background reading for memory models and related
tools. These documents are aimed at kernel hackers who are interested
in memory models.
@@ -6,64 +10,64 @@ in memory models.
Hardware manuals and models
===========================
-o SPARC International Inc. (Ed.). 1994. "The SPARC Architecture
+- SPARC International Inc. (Ed.). 1994. "The SPARC Architecture
Reference Manual Version 9". SPARC International Inc.
-o Compaq Computer Corporation (Ed.). 2002. "Alpha Architecture
+- Compaq Computer Corporation (Ed.). 2002. "Alpha Architecture
Reference Manual". Compaq Computer Corporation.
-o Intel Corporation (Ed.). 2002. "A Formal Specification of Intel
+- Intel Corporation (Ed.). 2002. "A Formal Specification of Intel
Itanium Processor Family Memory Ordering". Intel Corporation.
-o Intel Corporation (Ed.). 2002. "Intel 64 and IA-32 Architectures
+- Intel Corporation (Ed.). 2002. "Intel 64 and IA-32 Architectures
Software Developer’s Manual". Intel Corporation.
-o Peter Sewell, Susmit Sarkar, Scott Owens, Francesco Zappa Nardelli,
+- Peter Sewell, Susmit Sarkar, Scott Owens, Francesco Zappa Nardelli,
and Magnus O. Myreen. 2010. "x86-TSO: A Rigorous and Usable
Programmer's Model for x86 Multiprocessors". Commun. ACM 53, 7
(July, 2010), 89-97. http://doi.acm.org/10.1145/1785414.1785443
-o IBM Corporation (Ed.). 2009. "Power ISA Version 2.06". IBM
+- IBM Corporation (Ed.). 2009. "Power ISA Version 2.06". IBM
Corporation.
-o ARM Ltd. (Ed.). 2009. "ARM Barrier Litmus Tests and Cookbook".
+- ARM Ltd. (Ed.). 2009. "ARM Barrier Litmus Tests and Cookbook".
ARM Ltd.
-o Susmit Sarkar, Peter Sewell, Jade Alglave, Luc Maranget, and
+- Susmit Sarkar, Peter Sewell, Jade Alglave, Luc Maranget, and
Derek Williams. 2011. "Understanding POWER Multiprocessors". In
Proceedings of the 32Nd ACM SIGPLAN Conference on Programming
Language Design and Implementation (PLDI ’11). ACM, New York,
NY, USA, 175–186.
-o Susmit Sarkar, Kayvan Memarian, Scott Owens, Mark Batty,
+- Susmit Sarkar, Kayvan Memarian, Scott Owens, Mark Batty,
Peter Sewell, Luc Maranget, Jade Alglave, and Derek Williams.
2012. "Synchronising C/C++ and POWER". In Proceedings of the 33rd
ACM SIGPLAN Conference on Programming Language Design and
Implementation (PLDI '12). ACM, New York, NY, USA, 311-322.
-o ARM Ltd. (Ed.). 2014. "ARM Architecture Reference Manual (ARMv8,
+- ARM Ltd. (Ed.). 2014. "ARM Architecture Reference Manual (ARMv8,
for ARMv8-A architecture profile)". ARM Ltd.
-o Imagination Technologies, LTD. 2015. "MIPS(R) Architecture
+- Imagination Technologies, LTD. 2015. "MIPS(R) Architecture
For Programmers, Volume II-A: The MIPS64(R) Instruction,
Set Reference Manual". Imagination Technologies,
LTD. https://imgtec.com/?do-download=4302.
-o Shaked Flur, Kathryn E. Gray, Christopher Pulte, Susmit
+- Shaked Flur, Kathryn E. Gray, Christopher Pulte, Susmit
Sarkar, Ali Sezgin, Luc Maranget, Will Deacon, and Peter
Sewell. 2016. "Modelling the ARMv8 Architecture, Operationally:
Concurrency and ISA". In Proceedings of the 43rd Annual ACM
SIGPLAN-SIGACT Symposium on Principles of Programming Languages
(POPL ’16). ACM, New York, NY, USA, 608–621.
-o Shaked Flur, Susmit Sarkar, Christopher Pulte, Kyndylan Nienhuis,
+- Shaked Flur, Susmit Sarkar, Christopher Pulte, Kyndylan Nienhuis,
Luc Maranget, Kathryn E. Gray, Ali Sezgin, Mark Batty, and Peter
Sewell. 2017. "Mixed-size Concurrency: ARM, POWER, C/C++11,
and SC". In Proceedings of the 44th ACM SIGPLAN Symposium on
Principles of Programming Languages (POPL 2017). ACM, New York,
NY, USA, 429–442.
-o Christopher Pulte, Shaked Flur, Will Deacon, Jon French,
+- Christopher Pulte, Shaked Flur, Will Deacon, Jon French,
Susmit Sarkar, and Peter Sewell. 2018. "Simplifying ARM concurrency:
multicopy-atomic axiomatic and operational models for ARMv8". In
Proceedings of the ACM on Programming Languages, Volume 2, Issue
@@ -73,18 +77,18 @@ o Christopher Pulte, Shaked Flur, Will Deacon, Jon French,
Linux-kernel memory model
=========================
-o Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and
+- Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and
Alan Stern. 2018. "Frightening small children and disconcerting
grown-ups: Concurrency in the Linux kernel". In Proceedings of
the 23rd International Conference on Architectural Support for
Programming Languages and Operating Systems (ASPLOS 2018). ACM,
New York, NY, USA, 405-418. Webpage: http://diy.inria.fr/linux/.
-o Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and
+- Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and
Alan Stern. 2017. "A formal kernel memory-ordering model (part 1)"
Linux Weekly News. https://lwn.net/Articles/718628/
-o Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and
+- Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and
Alan Stern. 2017. "A formal kernel memory-ordering model (part 2)"
Linux Weekly News. https://lwn.net/Articles/720550/
@@ -92,16 +96,16 @@ o Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and
Memory-model tooling
====================
-o Daniel Jackson. 2002. "Alloy: A Lightweight Object Modelling
+- Daniel Jackson. 2002. "Alloy: A Lightweight Object Modelling
Notation". ACM Trans. Softw. Eng. Methodol. 11, 2 (April 2002),
256–290. http://doi.acm.org/10.1145/505145.505149
-o Jade Alglave, Luc Maranget, and Michael Tautschnig. 2014. "Herding
+- Jade Alglave, Luc Maranget, and Michael Tautschnig. 2014. "Herding
Cats: Modelling, Simulation, Testing, and Data Mining for Weak
Memory". ACM Trans. Program. Lang. Syst. 36, 2, Article 7 (July
2014), 7:1–7:74 pages.
-o Jade Alglave, Patrick Cousot, and Luc Maranget. 2016. "Syntax and
+- Jade Alglave, Patrick Cousot, and Luc Maranget. 2016. "Syntax and
semantics of the weak consistency model specification language
cat". CoRR abs/1608.07531 (2016). http://arxiv.org/abs/1608.07531
@@ -109,6 +113,6 @@ o Jade Alglave, Patrick Cousot, and Luc Maranget. 2016. "Syntax and
Memory-model comparisons
========================
-o Paul E. McKenney, Ulrich Weigand, Andrea Parri, and Boqun
+- Paul E. McKenney, Ulrich Weigand, Andrea Parri, and Boqun
Feng. 2016. "Linux-Kernel Memory Model". (6 June 2016).
http://open-std.org/JTC1/SC22/WG21/docs/papers/2016/p0124r2.html.
diff --git a/tools/memory-model/README b/tools/memory-model/README
index 2b87f3971548..04bb1fa9ed76 100644
--- a/tools/memory-model/README
+++ b/tools/memory-model/README
@@ -105,16 +105,16 @@ for more information.
DESCRIPTION OF FILES
====================
-Documentation/cheatsheet.txt
+Documentation/cheatsheet.rst
Quick-reference guide to the Linux-kernel memory model.
-Documentation/explanation.txt
+Documentation/explanation.rst
Describes the memory model in detail.
-Documentation/recipes.txt
+Documentation/recipes.rst
Lists common memory-ordering patterns.
-Documentation/references.txt
+Documentation/references.rst
Provides background reading.
linux-kernel.bell
@@ -173,7 +173,7 @@ The Linux-kernel memory model has the following limitations:
of READ_ONCE() and WRITE_ONCE() limits the compiler's ability
to optimize, but there is Linux-kernel code that uses bare C
memory accesses. Handling this code is on the to-do list.
- For more information, see Documentation/explanation.txt (in
+ For more information, see Documentation/explanation.rst (in
particular, the "THE PROGRAM ORDER RELATION: po AND po-loc"
and "A WARNING" sections).
^ permalink raw reply related
* Re: [PATCH] tools: memory-model: add it to the Documentation body
From: Mauro Carvalho Chehab @ 2019-07-30 22:57 UTC (permalink / raw)
To: Joel Fernandes
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Alan Stern, Andrea Parri, Will Deacon,
Peter Zijlstra, Boqun Feng, Nicholas Piggin, David Howells,
Jade Alglave, Luc Maranget, Paul E. McKenney, Akira Yokosawa,
Daniel Lustig, Ingo Molnar, Jason Gunthorpe, SeongJae Park,
linux-arch
In-Reply-To: <20190730221701.GC254050@google.com>
Em Tue, 30 Jul 2019 18:17:01 -0400
Joel Fernandes <joel@joelfernandes.org> escreveu:
> On Sat, Jul 27, 2019 at 12:37:54PM -0300, Mauro Carvalho Chehab wrote:
> > Em Sat, 27 Jul 2019 14:14:53 +0000
> > Joel Fernandes <joel@joelfernandes.org> escreveu:
> >
> > > On Fri, Jul 26, 2019 at 04:01:37PM -0300, Mauro Carvalho Chehab wrote:
> > > > The books at tools/memory-model/Documentation are very well
> > > > formatted. Congrats to the ones that wrote them!
> > > >
> > > > The manual conversion to ReST is really trivial:
> > > >
> > > > - Add document titles;
> > > > - change the bullets on some lists;
> > > > - mark code blocks.
> > >
> > > Thanks so much, some feedback:
> > > >
> > > > Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> > >
> > > (1)
> > > I could not find the table of contents appear in the HTML output for this.
> > > Basically this list in the beginning doesn't render:
> > > 1. INTRODUCTION
> > > 2. BACKGROUND
> > > 3. A SIMPLE EXAMPLE
> > > 4. A SELECTION OF MEMORY MODELS
> > > 5. ORDERING AND CYCLES
> >
> > Yes. It is written as a comment, like:
> >
> > .. foo This is a comment block
> >
> > Everything on this block
> >
> > won't be parsed.
> >
> > So it won't be parsed, but having a TOC like this isn't need, as
> > Sphinx generates it automatically via "toctree" markup.
>
> Ok.
>
> > > Could we add a proper TOC with sections? My motivation for ReST here would be
> > > to make the sections jumpable since it is a large document.
> >
> > Just change the toctree depth at index.rst to 2 and you'll see an index
> > produced by Sphinx with both levels 1 (doc name) and level 2 (chapters):
> >
> > .. toctree::
> > :maxdepth: 2
>
> Admittedly, I don't have much time at the moment to do these experiments :(
>
> > > Also could we make the different sections appear as a tree in the left
> > > sidebar?
> >
> > The sidebar follows the maxdepth too.
> >
> > >
> > > (2) Arguably several function names in the document HTML output should appear
> > > in monospace fonting and/or referring to the documentation for real function
> > > names, but these can be fixed as we go, I guess.
> >
> > If you want monospaced fonts, just use: ``monospaced_symbol_foo`` within
> > any paragraph, or place the monospaced data inside a code-block:
> >
> > ::
> >
> > This will be monospaced.
> >
> > >
> > > (3) Things like smp_load_acquire() and spin_lock() should probably refer to
> > > the documentation for those elsewhere..
> >
> > Jon added an automarkup extension on Kernel 5.2. So, all functions that
> > are defined elsewhere will automatically generate an hyperlink. For that to
> > happen, you need to add the kernel-doc markup at the *.h or *.c file where
> > the function is declared and use the kernel-doc markup somewhere within the
> > Kernel Documentation/.
> >
> > >
> > > (4) I would argue that every occurence of
> > > A ->(some dependency) B should be replaced with fixed size font in the HTML
> > > results.
> >
> > Just place those with ``A -> (some dependency)``. This will make them use
> > a fixed size font.
>
> Ok, understood all these. I guess my point was all of these will need to be
> done to make this document useful from a ReST conversion standpoint. Until
> then it is probably just better off being plain text - since there are so
> many of those ``A -> (dep) B`` things.
>
> > > Arguably it is better IMO if the whole document is fixed size font in the
> > > HTML output because so many things need to be fixed size, but that my just be
> > > my opinion.
> >
> > Just my 2 cents here, but having the entire document using a fixed size
> > font makes it more boring to read. Having just the symbols with a fixed size
> > is a common convention used on technical books, and helps to make easier
> > to identify the symbols while reading the docs.
> >
> > That's said, Sphinx doesn't have any tag to switch the font for the entire
> > document. All it can be done is to define a CSS and apply it for the
> > doc - or to place everything within a code-block, with will suppress all
> > markup tags, including cross-references for functions.
>
> Ok, got it.
>
> > The problem with CSS is that you need to write both an html CSS file
> > and add LaTeX macros associated to this "CSS style" (technically, LaTeX
> > doesn't have a CSS concept, but Sphinx emulates it).
>
> Yeah I don't think we want to do CSS here. So the correct thing to do would
> be to place all fixed-width things within double backticks, if someone had
> the time to do it. I am currently spending time understanding the document's
> content itself..
>
> thanks for the effort, it could probably serve as a good future reference,
On a very quick look, it seems that, if we replace:
(\S+\s->\S*\s\w+)
by:
``\1``
On an editor that would allow to manually replace the regex (like kate),
most of those can be get.
See patch enclosed.
Thanks,
Mauro
[PATCH] Use monotonic fonts for ``A -> (dep) B``
Manually replace:
(\S+\s->\S*\s\w+)
by:
``\1``
On their occurrences and fix a couple of places where it doesn't
hit well.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
diff --git a/tools/memory-model/Documentation/explanation.rst b/tools/memory-model/Documentation/explanation.rst
index 227ec75f8dc4..9b5d10cef0c2 100644
--- a/tools/memory-model/Documentation/explanation.rst
+++ b/tools/memory-model/Documentation/explanation.rst
@@ -332,7 +332,7 @@ can think of it as the order in which statements occur in the source
code after branches are taken into account and loops have been
unrolled. A better description might be the order in which
instructions are presented to a CPU's execution unit. Thus, we say
-that X is po-before Y (written as "X ->po Y" in formulas) if X occurs
+that X is po-before Y (written as ``X ->po Y`` in formulas) if X occurs
before Y in the instruction stream.
This is inherently a single-CPU relation; two instructions executing
@@ -485,9 +485,9 @@ which depends on the value obtained by the READ_ONCE(); hence there is
a control dependency from the load to the store.
It should be pretty obvious that events can only depend on reads that
-come earlier in program order. Symbolically, if we have R ->data X,
-R ->addr X, or R ->ctrl X (where R is a read event), then we must also
-have R ->po X. It wouldn't make sense for a computation to depend
+come earlier in program order. Symbolically, if we have ``R ->data X``,
+``R ->addr X``, or ``R ->ctrl X`` (where R is a read event), then we must also
+have ``R ->po X``. It wouldn't make sense for a computation to depend
somehow on a value that doesn't get loaded from shared memory until
later in the code!
@@ -498,7 +498,7 @@ THE READS-FROM RELATION: rf, rfi, and rfe
The reads-from relation (rf) links a write event to a read event when
the value loaded by the read is the value that was stored by the
write. In colloquial terms, the load "reads from" the store. We
-write W ->rf R to indicate that the load R reads from the store W. We
+write ``W ->rf R`` to indicate that the load R reads from the store W. We
further distinguish the cases where the load and the store occur on
the same CPU (internal reads-from, or rfi) and where they occur on
different CPUs (external reads-from, or rfe).
@@ -579,26 +579,26 @@ that value comes third, and so on.
You can think of the coherence order as being the order in which the
stores reach x's location in memory (or if you prefer a more
hardware-centric view, the order in which the stores get written to
-x's cache line). We write W ->co W' if W comes before W' in the
+x's cache line). We write ``W ->co W'`` if W comes before W' in the
coherence order, that is, if the value stored by W gets overwritten,
directly or indirectly, by the value stored by W'.
Coherence order is required to be consistent with program order. This
requirement takes the form of four coherency rules:
- Write-write coherence: If W ->po-loc W' (i.e., W comes before
+ Write-write coherence: If ``W ->po-loc W'`` (i.e., W comes before
W' in program order and they access the same location), where W
- and W' are two stores, then W ->co W'.
+ and W' are two stores, then ``W ->co W'``.
- Write-read coherence: If W ->po-loc R, where W is a store and R
+ Write-read coherence: If ``W ->po-loc R``, where W is a store and R
is a load, then R must read from W or from some other store
which comes after W in the coherence order.
- Read-write coherence: If R ->po-loc W, where R is a load and W
+ Read-write coherence: If ``R ->po-loc W``, where R is a load and W
is a store, then the store which R reads from must come before
W in the coherence order.
- Read-read coherence: If R ->po-loc R', where R and R' are two
+ Read-read coherence: If ``R ->po-loc R'``, where R and R' are two
loads, then either they read from the same store or else the
store read by R comes before the store read by R' in the
coherence order.
@@ -694,7 +694,7 @@ THE FROM-READS RELATION: fr, fri, and fre
The from-reads relation (fr) can be a little difficult for people to
grok. It describes the situation where a load reads a value that gets
-overwritten by a store. In other words, we have R ->fr W when the
+overwritten by a store. In other words, we have ``R ->fr W`` when the
value that R reads is overwritten (directly or indirectly) by W, or
equivalently, when R reads from a store which comes earlier than W in
the coherence order.
@@ -723,7 +723,7 @@ different CPUs).
Note that the fr relation is determined entirely by the rf and co
relations; it is not independent. Given a read event R and a write
-event W for the same location, we will have R ->fr W if and only if
+event W for the same location, we will have ``R ->fr W`` if and only if
the write which R reads from is co-before W. In symbols,
::
@@ -850,13 +850,13 @@ defined to link memory access events E and F whenever:
event occurs between them in program order; or
F is a release fence and some X comes before F in program order,
- where either X = E or else E ->rf X; or
+ where either ``X = E`` or else ``E ->rf X``; or
A strong fence event occurs between some X and F in program
- order, where either X = E or else E ->rf X.
+ order, where either ``X = E`` or else ``E ->rf X``.
The operational model requires that whenever W and W' are both stores
-and W ->cumul-fence W', then W must propagate to any given CPU
+and ``W ->cumul-fence W'``, then W must propagate to any given CPU
before W' does. However, for different CPUs C and C', it does not
require W to propagate to C before W' propagates to C'.
@@ -910,7 +910,7 @@ first for CPU 0, then CPU 1, etc.
You can check that the four coherency rules imply that the rf, co, fr,
and po-loc relations agree with this global ordering; in other words,
-whenever we have X ->rf Y or X ->co Y or X ->fr Y or X ->po-loc Y, the
+whenever we have ``X ->rf Y`` or ``X ->co Y`` or ``X ->fr Y`` or ``X ->po-loc Y``, the
X event comes before the Y event in the global ordering. The LKMM's
"coherence" axiom expresses this by requiring the union of these
relations not to have any cycles. This means it must not be possible
@@ -977,7 +977,7 @@ po.
The operational model already includes a description of one such
situation: Fences are a source of ppo links. Suppose X and Y are
-memory accesses with X ->po Y; then the CPU must execute X before Y if
+memory accesses with ``X ->po Y``; then the CPU must execute X before Y if
any of the following hold:
A strong (smp_mb() or synchronize_rcu()) fence occurs between
@@ -996,7 +996,7 @@ any of the following hold:
Another possibility, not mentioned earlier but discussed in the next
section, is:
- X and Y are both loads, X ->addr Y (i.e., there is an address
+ X and Y are both loads, ``X ->addr Y`` (i.e., there is an address
dependency from X to Y), and X is a READ_ONCE() or an atomic
access.
@@ -1176,25 +1176,25 @@ The happens-before relation (hb) links memory accesses that have to
execute in a certain order. hb includes the ppo relation and two
others, one of which is rfe.
-W ->rfe R implies that W and R are on different CPUs. It also means
+``W ->rfe R`` implies that W and R are on different CPUs. It also means
that W's store must have propagated to R's CPU before R executed;
otherwise R could not have read the value stored by W. Therefore W
-must have executed before R, and so we have W ->hb R.
+must have executed before R, and so we have ``W ->hb R``.
-The equivalent fact need not hold if W ->rfi R (i.e., W and R are on
+The equivalent fact need not hold if ``W ->rfi R`` (i.e., W and R are on
the same CPU). As we have already seen, the operational model allows
W's value to be forwarded to R in such cases, meaning that R may well
execute before W does.
It's important to understand that neither coe nor fre is included in
hb, despite their similarities to rfe. For example, suppose we have
-W ->coe W'. This means that W and W' are stores to the same location,
+``W ->coe W'``. This means that W and W' are stores to the same location,
they execute on different CPUs, and W comes before W' in the coherence
order (i.e., W' overwrites W). Nevertheless, it is possible for W' to
execute before W, because the decision as to which store overwrites
the other is made later by the memory subsystem. When the stores are
nearly simultaneous, either one can come out on top. Similarly,
-R ->fre W means that W overwrites the value which R reads, but it
+``R ->fre W`` means that W overwrites the value which R reads, but it
doesn't mean that W has to execute after R. All that's necessary is
for the memory subsystem not to propagate W to R's CPU until after R
has executed, which is possible if W executes shortly before R.
@@ -1393,10 +1393,10 @@ The existence of a pb link from E to F implies that E must execute
before F. To see why, suppose that F executed first. Then W would
have propagated to E's CPU before E executed. If E was a store, the
memory subsystem would then be forced to make E come after W in the
-coherence order, contradicting the fact that E ->coe W. If E was a
+coherence order, contradicting the fact that ``E ->coe W``. If E was a
load, the memory subsystem would then be forced to satisfy E's read
request with the value stored by W or an even later store,
-contradicting the fact that E ->fre W.
+contradicting the fact that ``E ->fre W``.
A good example illustrating how pb works is the SB pattern with strong
fences::
@@ -1518,9 +1518,9 @@ entirely clear. The LKMM formalizes this notion by means of the
rcu-link relation. rcu-link encompasses a very general notion of
"before": If E and F are RCU fence events (i.e., rcu_read_lock(),
rcu_read_unlock(), or synchronize_rcu()) then among other things,
-E ->rcu-link F includes cases where E is po-before some memory-access
+``E ->rcu-link F`` includes cases where E is po-before some memory-access
event X, F is po-after some memory-access event Y, and we have any of
-X ->rfe Y, X ->co Y, or X ->fr Y.
+``X ->rfe Y``, ``X ->co Y``, or ``X ->fr Y``.
The formal definition of the rcu-link relation is more than a little
obscure, and we won't give it here. It is closely related to the pb
@@ -1532,22 +1532,22 @@ The LKMM also defines the rcu-gp and rcu-rscsi relations. They bring
grace periods and read-side critical sections into the picture, in the
following way:
- E ->rcu-gp F means that E and F are in fact the same event,
+ ``E ->rcu-gp F`` means that E and F are in fact the same event,
and that event is a synchronize_rcu() fence (i.e., a grace
period).
- E ->rcu-rscsi F means that E and F are the rcu_read_unlock()
+ ``E ->rcu-rscsi F`` means that E and F are the rcu_read_unlock()
and rcu_read_lock() fence events delimiting some read-side
critical section. (The 'i' at the end of the name emphasizes
that this relation is "inverted": It links the end of the
critical section to the start.)
If we think of the rcu-link relation as standing for an extended
-"before", then X ->rcu-gp Y ->rcu-link Z roughly says that X is a
+"before", then ``X ->rcu-gp Y ->rcu-link Z`` roughly says that X is a
grace period which ends before Z begins. (In fact it covers more than
this, because it also includes cases where some store propagates to
Z's CPU before Z begins but doesn't propagate to some other CPU until
-after X ends.) Similarly, X ->rcu-rscsi Y ->rcu-link Z says that X is
+after X ends.) Similarly, ``X ->rcu-rscsi Y ->rcu-link Z`` says that X is
the end of a critical section which starts before Z begins.
The LKMM goes on to define the rcu-fence relation as a sequence of
@@ -1557,18 +1557,18 @@ example::
X ->rcu-gp Y ->rcu-link Z ->rcu-rscsi T ->rcu-link U ->rcu-gp V
-would imply that X ->rcu-fence V, because this sequence contains two
+would imply that ``X ->rcu-fence V``, because this sequence contains two
rcu-gp links and one rcu-rscsi link. (It also implies that
-X ->rcu-fence T and Z ->rcu-fence V.) On the other hand::
+``X ->rcu-fence T`` and ``Z ->rcu-fence V``.) On the other hand::
X ->rcu-rscsi Y ->rcu-link Z ->rcu-rscsi T ->rcu-link U ->rcu-gp V
-does not imply X ->rcu-fence V, because the sequence contains only
+does not imply ``X ->rcu-fence V``, because the sequence contains only
one rcu-gp link but two rcu-rscsi links.
The rcu-fence relation is important because the Grace Period Guarantee
means that rcu-fence acts kind of like a strong fence. In particular,
-E ->rcu-fence F implies not only that E begins before F ends, but also
+``E ->rcu-fence F`` implies not only that E begins before F ends, but also
that any write po-before E will propagate to every CPU before any
instruction po-after F can execute. (However, it does not imply that
E must execute before F; in fact, each synchronize_rcu() fence event
@@ -1604,13 +1604,13 @@ covered by rcu-fence.
Finally, the LKMM defines the RCU-before (rb) relation in terms of
rcu-fence. This is done in essentially the same way as the pb
relation was defined in terms of strong-fence. We will omit the
-details; the end result is that E ->rb F implies E must execute
-before F, just as E ->pb F does (and for much the same reasons).
+details; the end result is that ``E ->rb F`` implies E must execute
+before F, just as ``E ->pb F`` does (and for much the same reasons).
Putting this all together, the LKMM expresses the Grace Period
Guarantee by requiring that the rb relation does not contain a cycle.
Equivalently, this "rcu" axiom requires that there are no events E
-and F with E ->rcu-link F ->rcu-fence E. Or to put it a third way,
+and F with ``E ->rcu-link F ->rcu-fence E``. Or to put it a third way,
the axiom requires that there are no cycles consisting of rcu-gp and
rcu-rscsi alternating with rcu-link, where the number of rcu-gp links
is >= the number of rcu-rscsi links.
@@ -1649,8 +1649,8 @@ by rcu-link::
S ->rcu-link U.
-Since S is a grace period we have S ->rcu-gp S, and since L and U are
-the start and end of the critical section C we have U ->rcu-rscsi L.
+Since S is a grace period we have ``S ->rcu-gp S``, and since L and U are
+the start and end of the critical section C we have ``U ->rcu-rscsi L``.
From this we obtain::
S ->rcu-gp S ->rcu-link U ->rcu-rscsi L ->rcu-link S,
@@ -1683,16 +1683,16 @@ time with statement labels added::
If r2 = 0 at the end then P0's store at Y overwrites the value that
-P1's load at W reads from, so we have W ->fre Y. Since S ->po W and
-also Y ->po U, we get S ->rcu-link U. In addition, S ->rcu-gp S
+P1's load at W reads from, so we have ``W ->fre Y``. Since ``S ->po W`` and
+also ``Y ->po U``, we get ``S ->rcu-link U``. In addition, ``S ->rcu-gp S``
because S is a grace period.
If r1 = 1 at the end then P1's load at Z reads from P0's store at X,
-so we have X ->rfe Z. Together with L ->po X and Z ->po S, this
-yields L ->rcu-link S. And since L and U are the start and end of a
-critical section, we have U ->rcu-rscsi L.
+so we have ``X ->rfe Z``. Together with ``L ->po X`` and ``Z ->po S``, this
+yields ``L ->rcu-link S``. And since L and U are the start and end of a
+critical section, we have ``U ->rcu-rscsi L``.
-Then U ->rcu-rscsi L ->rcu-link S ->rcu-gp S ->rcu-link U is a
+Then ``U ->rcu-rscsi L ->rcu-link S ->rcu-gp S ->rcu-link U`` is a
forbidden cycle, violating the "rcu" axiom. Hence the outcome is not
allowed by the LKMM, as we would expect.
@@ -1729,9 +1729,9 @@ For contrast, let's see what can happen in a more complicated example::
U2: rcu_read_unlock();
}
-If r0 = r1 = r2 = 1 at the end, then similar reasoning to before shows
-that U0 ->rcu-rscsi L0 ->rcu-link S1 ->rcu-gp S1 ->rcu-link U2 ->rcu-rscsi
-L2 ->rcu-link U0. However this cycle is not forbidden, because the
+If ``r0 = r1 = r2 = 1`` at the end, then similar reasoning to before shows
+that ``U0 ->rcu-rscsi L0 ->rcu-link S1 ->rcu-gp S1 ->rcu-link U2 ->rcu-rscsi
+L2 ->rcu-link U0``. However this cycle is not forbidden, because the
sequence of relations contains fewer instances of rcu-gp (one) than of
rcu-rscsi (two). Consequently the outcome is allowed by the LKMM.
The following instruction timing diagram shows how it might actually
^ permalink raw reply related
* Re: [PATCH v2 25/26] docs: rcu: convert some articles from html to ReST
From: Joel Fernandes @ 2019-07-30 22:21 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Paul E. McKenney, Josh Triplett, Steven Rostedt,
Mathieu Desnoyers, Lai Jiangshan, Jonathan Corbet, rcu, linux-doc
In-Reply-To: <20190730190028.740c131e@coco.lan>
On Tue, Jul 30, 2019 at 07:00:28PM -0300, Mauro Carvalho Chehab wrote:
> Em Tue, 30 Jul 2019 17:50:07 -0400
> Joel Fernandes <joel@joelfernandes.org> escreveu:
>
> > On Tue, Jul 30, 2019 at 02:22:50PM -0700, Paul E. McKenney wrote:
> > > On Fri, Jul 26, 2019 at 09:51:35AM -0300, Mauro Carvalho Chehab wrote:
> > > > There are 4 RCU articles that are written on html format.
> > > >
> > > > The way they are, they can't be part of the Linux Kernel
> > > > documentation body nor share the styles and pdf output.
> > > >
> > > > So, convert them to ReST format.
> > > >
> > > > This way, make htmldocs and make pdfdocs will produce a
> > > > documentation output that will be like the original ones, but
> > > > will be part of the Linux Kernel documentation body.
> > > >
> > > > Part of the conversion was done with the help of pandoc, but
> > > > the result had some broken things that had to be manually
> > > > fixed.
> > > >
> > > > Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> > >
> > > I am having some trouble applying these, at least in part due to UTF-8
> > > sequences, for example double left quotation mark. These end up being
> > > "=E2=80=9C", with a few space characters turned into "=20".
> > >
> > > Any advice on how to apply these? Should I just pull commits from
> > > somewhere?
> >
> > I was able to successfully apply and build this particular patch. I think
> > this is the only one in the series that applies to RCU.
> >
> > Sadly, I can't find the patch in any of the public archives, but I could
> > perhaps email it to you as an .mbox attach which 'git am' should be able to
> > apply.
> >
> > Mauro did say he was going to add some more details to changelog, or it could
> > be added when it is applied:
> > https://lore.kernel.org/rcu/20190726154550.5eeae294@coco.lan/
>
> Yeah, I'm also planning to address at least some of the issues you
> pointed, in order to improve the html output, but got sidetracked by something
> else and didn't find any time yet to finish. I'm adding some CI automation for
> the media subsystem in order to help us dealing with the huge amount of patches
> we receive there.
>
> Feel free to add those details to the changelog. I may find some spare time
> this week or the next one for the improvements you suggested, but those
> could be sent on followup patches, once done.
Ok, I will re-send this RCU patch with the changes, leave this one to me.
The other memory model one, needs a lot more work so we can keep that aside
for now till someone has the time.
thanks,
- Joel
^ permalink raw reply
* Re: [PATCH] tools: memory-model: add it to the Documentation body
From: Joel Fernandes @ 2019-07-30 22:17 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Alan Stern, Andrea Parri, Will Deacon,
Peter Zijlstra, Boqun Feng, Nicholas Piggin, David Howells,
Jade Alglave, Luc Maranget, Paul E. McKenney, Akira Yokosawa,
Daniel Lustig, Ingo Molnar, Jason Gunthorpe, SeongJae Park,
linux-arch
In-Reply-To: <20190727123754.5d91d4a4@coco.lan>
On Sat, Jul 27, 2019 at 12:37:54PM -0300, Mauro Carvalho Chehab wrote:
> Em Sat, 27 Jul 2019 14:14:53 +0000
> Joel Fernandes <joel@joelfernandes.org> escreveu:
>
> > On Fri, Jul 26, 2019 at 04:01:37PM -0300, Mauro Carvalho Chehab wrote:
> > > The books at tools/memory-model/Documentation are very well
> > > formatted. Congrats to the ones that wrote them!
> > >
> > > The manual conversion to ReST is really trivial:
> > >
> > > - Add document titles;
> > > - change the bullets on some lists;
> > > - mark code blocks.
> >
> > Thanks so much, some feedback:
> > >
> > > Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> >
> > (1)
> > I could not find the table of contents appear in the HTML output for this.
> > Basically this list in the beginning doesn't render:
> > 1. INTRODUCTION
> > 2. BACKGROUND
> > 3. A SIMPLE EXAMPLE
> > 4. A SELECTION OF MEMORY MODELS
> > 5. ORDERING AND CYCLES
>
> Yes. It is written as a comment, like:
>
> .. foo This is a comment block
>
> Everything on this block
>
> won't be parsed.
>
> So it won't be parsed, but having a TOC like this isn't need, as
> Sphinx generates it automatically via "toctree" markup.
Ok.
> > Could we add a proper TOC with sections? My motivation for ReST here would be
> > to make the sections jumpable since it is a large document.
>
> Just change the toctree depth at index.rst to 2 and you'll see an index
> produced by Sphinx with both levels 1 (doc name) and level 2 (chapters):
>
> .. toctree::
> :maxdepth: 2
Admittedly, I don't have much time at the moment to do these experiments :(
> > Also could we make the different sections appear as a tree in the left
> > sidebar?
>
> The sidebar follows the maxdepth too.
>
> >
> > (2) Arguably several function names in the document HTML output should appear
> > in monospace fonting and/or referring to the documentation for real function
> > names, but these can be fixed as we go, I guess.
>
> If you want monospaced fonts, just use: ``monospaced_symbol_foo`` within
> any paragraph, or place the monospaced data inside a code-block:
>
> ::
>
> This will be monospaced.
>
> >
> > (3) Things like smp_load_acquire() and spin_lock() should probably refer to
> > the documentation for those elsewhere..
>
> Jon added an automarkup extension on Kernel 5.2. So, all functions that
> are defined elsewhere will automatically generate an hyperlink. For that to
> happen, you need to add the kernel-doc markup at the *.h or *.c file where
> the function is declared and use the kernel-doc markup somewhere within the
> Kernel Documentation/.
>
> >
> > (4) I would argue that every occurence of
> > A ->(some dependency) B should be replaced with fixed size font in the HTML
> > results.
>
> Just place those with ``A -> (some dependency)``. This will make them use
> a fixed size font.
Ok, understood all these. I guess my point was all of these will need to be
done to make this document useful from a ReST conversion standpoint. Until
then it is probably just better off being plain text - since there are so
many of those ``A -> (dep) B`` things.
> > Arguably it is better IMO if the whole document is fixed size font in the
> > HTML output because so many things need to be fixed size, but that my just be
> > my opinion.
>
> Just my 2 cents here, but having the entire document using a fixed size
> font makes it more boring to read. Having just the symbols with a fixed size
> is a common convention used on technical books, and helps to make easier
> to identify the symbols while reading the docs.
>
> That's said, Sphinx doesn't have any tag to switch the font for the entire
> document. All it can be done is to define a CSS and apply it for the
> doc - or to place everything within a code-block, with will suppress all
> markup tags, including cross-references for functions.
Ok, got it.
> The problem with CSS is that you need to write both an html CSS file
> and add LaTeX macros associated to this "CSS style" (technically, LaTeX
> doesn't have a CSS concept, but Sphinx emulates it).
Yeah I don't think we want to do CSS here. So the correct thing to do would
be to place all fixed-width things within double backticks, if someone had
the time to do it. I am currently spending time understanding the document's
content itself..
thanks for the effort, it could probably serve as a good future reference,
- Joel
^ permalink raw reply
* Re: [PATCH v2 25/26] docs: rcu: convert some articles from html to ReST
From: Joel Fernandes @ 2019-07-30 22:06 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Paul E. McKenney, Josh Triplett, Steven Rostedt,
Mathieu Desnoyers, Lai Jiangshan, Jonathan Corbet, rcu, linux-doc,
Markus Heiser
In-Reply-To: <20190726161405.278e7cfc@coco.lan>
On Fri, Jul 26, 2019 at 04:14:05PM -0300, Mauro Carvalho Chehab wrote:
> Em Fri, 26 Jul 2019 14:02:01 -0400
> Joel Fernandes <joel@joelfernandes.org> escreveu:
>
> > On Fri, Jul 26, 2019 at 09:51:35AM -0300, Mauro Carvalho Chehab wrote:
> > [snip]
> > > +| until the assignment to ``gp``, by which time both fields are fully |
> > > +| initialized. So reordering the assignments to ``p->a`` and ``p->b`` |
> > > +| cannot possibly cause any problems. |
> > > ++-----------------------------------------------------------------------+
> > > +
> > > +It is tempting to assume that the reader need not do anything special to
> > > +control its accesses to the RCU-protected data, as shown in
> > > +``do_something_gp_buggy()`` below:
> > > +
> > > + ::
> > > +
> > > + 1 bool do_something_gp_buggy(void)
> > > + 2 {
> > > + 3 rcu_read_lock();
> > > + 4 p = gp; /* OPTIMIZATIONS GALORE!!! */
> > > + 5 if (p) {
> > > + 6 do_something(p->a, p->b);
> > > + 7 rcu_read_unlock();
> > > + 8 return true;
> > > + 9 }
> > > + 10 rcu_read_unlock();
> > > + 11 return false;
> > > + 12 }
> > > +
> > > +However, this temptation must be resisted because there are a
> > > +surprisingly large number of ways that the compiler (to say nothing of
> > > +`DEC Alpha CPUs <https://h71000.www7.hp.com/wizard/wiz_2637.html>`__)
> > > +can trip this code up. For but one example, if the compiler were short
> > > +of registers, it might choose to refetch from ``gp`` rather than keeping
> > > +a separate copy in ``p`` as follows:
> > > +
> > > + ::
> > > +
> > > + 1 bool do_something_gp_buggy_optimized(void)
> > > + 2 {
> > > + 3 rcu_read_lock();
> > > + 4 if (gp) { /* OPTIMIZATIONS GALORE!!! */
> > > + 5 do_something(gp->a, gp->b);
> > > + 6 rcu_read_unlock();
> > > + 7 return true;
> > > + 8 }
> > > + 9 rcu_read_unlock();
> > > + 10 return false;
> > > + 11 }
> > > +
> > > +If this function ran concurrently with a series of updates that replaced
> > > +the current structure with a new one, the fetches of ``gp->a`` and
> > > +``gp->b`` might well come from two different structures, which could
> > > +cause serious confusion. To prevent this (and much else besides),
> > > +``do_something_gp()`` uses ``rcu_dereference()`` to fetch from ``gp``:
> > > +
> > > + ::
> > > +
> > > + 1 bool do_something_gp(void)
> > > + 2 {
> > > + 3 rcu_read_lock();
> > > + 4 p = rcu_dereference(gp);
> > > + 5 if (p) {
> > > + 6 do_something(p->a, p->b);
> > > + 7 rcu_read_unlock();
> > > + 8 return true;
> > > + 9 }
> > > + 10 rcu_read_unlock();
> > > + 11 return false;
> > > + 12 }
> > > +
> > > +The ``rcu_dereference()`` uses volatile casts and (for DEC Alpha) memory
> > > +barriers in the Linux kernel. Should a `high-quality implementation of
> > > +C11 ``memory_order_consume``
> > > +[PDF] <http://www.rdrop.com/users/paulmck/RCU/consume.2015.07.13a.pdf>`__
> > > +ever appear, then ``rcu_dereference()`` could be implemented as a
> > > +``memory_order_consume`` load. Regardless of the exact implementation, a
> > > +pointer fetched by ``rcu_dereference()`` may not be used outside of the
> > > +outermost RCU read-side critical section containing that
> > > +``rcu_dereference()``, unless protection of the corresponding data
> > > +element has been passed from RCU to some other synchronization
> > > +mechanism, most commonly locking or `reference
> > > +counting <https://www.kernel.org/doc/Documentation/RCU/rcuref.txt>`__.
> >
> > From the make htmldocs output, this appears very poorly for me, I get
> > something like this in the browser:
> >
> > The rcu_dereference() uses volatile casts and (for DEC Alpha) memory barriers
> > in the Linux kernel. Should a high-quality implementation of C11
> > ``memory_order_consume` [PDF]
> > <http://www.rdrop.com/users/paulmck/RCU/consume.2015.07.13a.pdf>`__ ever
> > appear, then rcu_dereference() could be implemented as a memory_order_consume
> > load.
> >
> > Is there a syntax issue here?
>
> Maybe. I tested those with Sphinx 2.0.1. Didn't test with older versions.
>
> I'll do some tests with Sphinx 1.7.9 (with is the current minimal
> recommended version) and do some cleanup on those references.
Ok, one more thing is broken, clicking links such as "Parallelism Facts of
Life" does not jump to the corresponding section.
Would you mind fixing this, add the description of changes you made (which
you shared in an earlier reply), fixing the above Sphinx issue, and then
resend it?
Otherwise, I believe it looks sane.
> > One more feedback,
> > the image under "RCU read-side critical section that started before the current
> > grace period:" should probably be blown up a bit.
>
> Unfortunately, the Kernel Sphinx image extension doesn't allow image scaling.
The current scale appears fine to me, it is not a big deal since it is clear.
> We had to add our own image extension at the Kernel, as otherwise,
> for every image, we would need to add one parser for PDF and another
> one for SVG.
>
> We would also need an extra parser for DOT.
>
> Markus solved all the 3 image formats with a single extension, but
> it currently doesn't allow passing the image size.
Cool.
^ permalink raw reply
* Re: [PATCH v2 25/26] docs: rcu: convert some articles from html to ReST
From: Mauro Carvalho Chehab @ 2019-07-30 22:00 UTC (permalink / raw)
To: Joel Fernandes
Cc: Paul E. McKenney, Josh Triplett, Steven Rostedt,
Mathieu Desnoyers, Lai Jiangshan, Jonathan Corbet, rcu, linux-doc
In-Reply-To: <20190730215007.GA254050@google.com>
Em Tue, 30 Jul 2019 17:50:07 -0400
Joel Fernandes <joel@joelfernandes.org> escreveu:
> On Tue, Jul 30, 2019 at 02:22:50PM -0700, Paul E. McKenney wrote:
> > On Fri, Jul 26, 2019 at 09:51:35AM -0300, Mauro Carvalho Chehab wrote:
> > > There are 4 RCU articles that are written on html format.
> > >
> > > The way they are, they can't be part of the Linux Kernel
> > > documentation body nor share the styles and pdf output.
> > >
> > > So, convert them to ReST format.
> > >
> > > This way, make htmldocs and make pdfdocs will produce a
> > > documentation output that will be like the original ones, but
> > > will be part of the Linux Kernel documentation body.
> > >
> > > Part of the conversion was done with the help of pandoc, but
> > > the result had some broken things that had to be manually
> > > fixed.
> > >
> > > Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> >
> > I am having some trouble applying these, at least in part due to UTF-8
> > sequences, for example double left quotation mark. These end up being
> > "=E2=80=9C", with a few space characters turned into "=20".
> >
> > Any advice on how to apply these? Should I just pull commits from
> > somewhere?
>
> I was able to successfully apply and build this particular patch. I think
> this is the only one in the series that applies to RCU.
>
> Sadly, I can't find the patch in any of the public archives, but I could
> perhaps email it to you as an .mbox attach which 'git am' should be able to
> apply.
>
> Mauro did say he was going to add some more details to changelog, or it could
> be added when it is applied:
> https://lore.kernel.org/rcu/20190726154550.5eeae294@coco.lan/
Yeah, I'm also planning to address at least some of the issues you
pointed, in order to improve the html output, but got sidetracked by something
else and didn't find any time yet to finish. I'm adding some CI automation for
the media subsystem in order to help us dealing with the huge amount of patches
we receive there.
Feel free to add those details to the changelog. I may find some spare time
this week or the next one for the improvements you suggested, but those
could be sent on followup patches, once done.
Thanks,
Mauro
^ permalink raw reply
* Re: [PATCH v2 25/26] docs: rcu: convert some articles from html to ReST
From: Mauro Carvalho Chehab @ 2019-07-30 21:50 UTC (permalink / raw)
To: Paul E. McKenney
Cc: Josh Triplett, Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan,
Joel Fernandes, Jonathan Corbet, rcu, linux-doc
In-Reply-To: <20190730212250.GJ14271@linux.ibm.com>
Em Tue, 30 Jul 2019 14:22:50 -0700
"Paul E. McKenney" <paulmck@linux.ibm.com> escreveu:
> On Fri, Jul 26, 2019 at 09:51:35AM -0300, Mauro Carvalho Chehab wrote:
> > There are 4 RCU articles that are written on html format.
> >
> > The way they are, they can't be part of the Linux Kernel
> > documentation body nor share the styles and pdf output.
> >
> > So, convert them to ReST format.
> >
> > This way, make htmldocs and make pdfdocs will produce a
> > documentation output that will be like the original ones, but
> > will be part of the Linux Kernel documentation body.
> >
> > Part of the conversion was done with the help of pandoc, but
> > the result had some broken things that had to be manually
> > fixed.
> >
> > Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
>
> I am having some trouble applying these, at least in part due to UTF-8
> sequences, for example double left quotation mark. These end up being
> "=E2=80=9C", with a few space characters turned into "=20".
>
> Any advice on how to apply these?
Didn't notice it ended with UTF-8 chars. It is probably because it came
from the html conversion.
I guess it shouldn't hurt keeping those, but if you prefer I can find
some time later to replace them.
> Should I just pull commits from somewhere?
Yeah, if you prefer, you can pull from this branch:
https://git.linuxtv.org/mchehab/experimental.git/log/?h=rcu-v1
It has just two patches: the RCU and tools/memory-model ones.
It is based on v5.3-rc2.
Thanks,
Mauro
^ permalink raw reply
* Re: [PATCH v2 25/26] docs: rcu: convert some articles from html to ReST
From: Joel Fernandes @ 2019-07-30 21:50 UTC (permalink / raw)
To: Paul E. McKenney
Cc: Mauro Carvalho Chehab, Josh Triplett, Steven Rostedt,
Mathieu Desnoyers, Lai Jiangshan, Jonathan Corbet, rcu, linux-doc
In-Reply-To: <20190730212250.GJ14271@linux.ibm.com>
On Tue, Jul 30, 2019 at 02:22:50PM -0700, Paul E. McKenney wrote:
> On Fri, Jul 26, 2019 at 09:51:35AM -0300, Mauro Carvalho Chehab wrote:
> > There are 4 RCU articles that are written on html format.
> >
> > The way they are, they can't be part of the Linux Kernel
> > documentation body nor share the styles and pdf output.
> >
> > So, convert them to ReST format.
> >
> > This way, make htmldocs and make pdfdocs will produce a
> > documentation output that will be like the original ones, but
> > will be part of the Linux Kernel documentation body.
> >
> > Part of the conversion was done with the help of pandoc, but
> > the result had some broken things that had to be manually
> > fixed.
> >
> > Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
>
> I am having some trouble applying these, at least in part due to UTF-8
> sequences, for example double left quotation mark. These end up being
> "=E2=80=9C", with a few space characters turned into "=20".
>
> Any advice on how to apply these? Should I just pull commits from
> somewhere?
I was able to successfully apply and build this particular patch. I think
this is the only one in the series that applies to RCU.
Sadly, I can't find the patch in any of the public archives, but I could
perhaps email it to you as an .mbox attach which 'git am' should be able to
apply.
Mauro did say he was going to add some more details to changelog, or it could
be added when it is applied:
https://lore.kernel.org/rcu/20190726154550.5eeae294@coco.lan/
Let me know how else I can help! I am reviewing this patch further today.
thanks,
- Joel
^ permalink raw reply
* Re: [PATCH v12 0/5] overlayfs override_creds=off
From: Casey Schaufler @ 2019-07-30 21:37 UTC (permalink / raw)
To: Mark Salyzyn, linux-kernel
Cc: kernel-team, Miklos Szeredi, Jonathan Corbet, Vivek Goyal,
Eric W . Biederman, Amir Goldstein, Randy Dunlap, Stephen Smalley,
linux-unionfs, linux-doc, Linux Security Module list
In-Reply-To: <20190730172904.79146-1-salyzyn@android.com>
On 7/30/2019 10:28 AM, Mark Salyzyn wrote:
> Patch series:
Please add linux-security-module@vger.kernel.org to the CC
for all changes affecting handling of security xattrs.
>
> overlayfs: check CAP_DAC_READ_SEARCH before issuing exportfs_decode_fh
> Add flags option to get xattr method paired to __vfs_getxattr
> overlayfs: handle XATTR_NOSECURITY flag for get xattr method
> overlayfs: internal getxattr operations without sepolicy checking
> overlayfs: override_creds=off option bypass creator_cred
>
> The first four patches address fundamental security issues that should
> be solved regardless of the override_creds=off feature.
> on them).
>
> The fifth adds the feature depends on these other fixes.
>
> By default, all access to the upper, lower and work directories is the
> recorded mounter's MAC and DAC credentials. The incoming accesses are
> checked against the caller's credentials.
>
> If the principles of least privilege are applied for sepolicy, the
> mounter's credentials might not overlap the credentials of the caller's
> when accessing the overlayfs filesystem. For example, a file that a
> lower DAC privileged caller can execute, is MAC denied to the
> generally higher DAC privileged mounter, to prevent an attack vector.
>
> We add the option to turn off override_creds in the mount options; all
> subsequent operations after mount on the filesystem will be only the
> caller's credentials. The module boolean parameter and mount option
> override_creds is also added as a presence check for this "feature",
> existence of /sys/module/overlay/parameters/overlay_creds
>
> Signed-off-by: Mark Salyzyn <salyzyn@android.com>
> Cc: Miklos Szeredi <miklos@szeredi.hu>
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: Vivek Goyal <vgoyal@redhat.com>
> Cc: Eric W. Biederman <ebiederm@xmission.com>
> Cc: Amir Goldstein <amir73il@gmail.com>
> Cc: Randy Dunlap <rdunlap@infradead.org>
> Cc: Stephen Smalley <sds@tycho.nsa.gov>
> Cc: linux-unionfs@vger.kernel.org
> Cc: linux-doc@vger.kernel.org
> Cc: linux-kernel@vger.kernel.org
>
> ---
> v12:
> - Restore squished out patch 2 and 3 in the series,
> then change algorithm to add flags argument.
> Per-thread flag is a large security surface.
>
> v11:
> - Squish out v10 introduced patch 2 and 3 in the series,
> then and use per-thread flag instead for nesting.
> - Switch name to ovl_do_vds_getxattr for __vds_getxattr wrapper.
> - Add sb argument to ovl_revert_creds to match future work.
>
> v10:
> - Return NULL on CAP_DAC_READ_SEARCH
> - Add __get xattr method to solve sepolicy logging issue
> - Drop unnecessary sys_admin sepolicy checking for administrative
> driver internal xattr functions.
>
> v6:
> - Drop CONFIG_OVERLAY_FS_OVERRIDE_CREDS.
> - Do better with the documentation, drop rationalizations.
> - pr_warn message adjusted to report consequences.
>
> v5:
> - beefed up the caveats in the Documentation
> - Is dependent on
> "overlayfs: check CAP_DAC_READ_SEARCH before issuing exportfs_decode_fh"
> "overlayfs: check CAP_MKNOD before issuing vfs_whiteout"
> - Added prwarn when override_creds=off
>
> v4:
> - spelling and grammar errors in text
>
> v3:
> - Change name from caller_credentials / creator_credentials to the
> boolean override_creds.
> - Changed from creator to mounter credentials.
> - Updated and fortified the documentation.
> - Added CONFIG_OVERLAY_FS_OVERRIDE_CREDS
>
> v2:
> - Forward port changed attr to stat, resulting in a build error.
> - altered commit message.
>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox