* [PATCH v3 2/4] docs: remove :c:func: annotations from xarray.rst
From: Jonathan Corbet @ 2019-06-26 17:28 UTC (permalink / raw)
To: linux-doc
Cc: Mauro Carvalho Chehab, Jani Nikula, linux-kernel, Jonathan Corbet,
Matthew Wilcox
In-Reply-To: <20190626172859.16113-1-corbet@lwn.net>
Now that the build system automatically marks up function references, we
don't have to clutter the source files, so take it out.
[Some paragraphs could now benefit from refilling, but that was left out to
avoid obscuring the real changes.]
Acked-by: Matthew Wilcox <willy@infradead.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
Documentation/core-api/xarray.rst | 270 +++++++++++++++---------------
1 file changed, 135 insertions(+), 135 deletions(-)
diff --git a/Documentation/core-api/xarray.rst b/Documentation/core-api/xarray.rst
index ef6f9f98f595..fcedc5349ace 100644
--- a/Documentation/core-api/xarray.rst
+++ b/Documentation/core-api/xarray.rst
@@ -30,27 +30,27 @@ it called marks. Each mark may be set or cleared independently of
the others. You can iterate over entries which are marked.
Normal pointers may be stored in the XArray directly. They must be 4-byte
-aligned, which is true for any pointer returned from :c:func:`kmalloc` and
-:c:func:`alloc_page`. It isn't true for arbitrary user-space pointers,
+aligned, which is true for any pointer returned from kmalloc() and
+alloc_page(). It isn't true for arbitrary user-space pointers,
nor for function pointers. You can store pointers to statically allocated
objects, as long as those objects have an alignment of at least 4.
You can also store integers between 0 and ``LONG_MAX`` in the XArray.
-You must first convert it into an entry using :c:func:`xa_mk_value`.
+You must first convert it into an entry using xa_mk_value().
When you retrieve an entry from the XArray, you can check whether it is
-a value entry by calling :c:func:`xa_is_value`, and convert it back to
-an integer by calling :c:func:`xa_to_value`.
+a value entry by calling xa_is_value(), and convert it back to
+an integer by calling xa_to_value().
Some users want to store tagged pointers instead of using the marks
-described above. They can call :c:func:`xa_tag_pointer` to create an
-entry with a tag, :c:func:`xa_untag_pointer` to turn a tagged entry
-back into an untagged pointer and :c:func:`xa_pointer_tag` to retrieve
+described above. They can call xa_tag_pointer() to create an
+entry with a tag, xa_untag_pointer() to turn a tagged entry
+back into an untagged pointer and xa_pointer_tag() to retrieve
the tag of an entry. Tagged pointers use the same bits that are used
to distinguish value entries from normal pointers, so each user must
decide whether they want to store value entries or tagged pointers in
any particular XArray.
-The XArray does not support storing :c:func:`IS_ERR` pointers as some
+The XArray does not support storing IS_ERR() pointers as some
conflict with value entries or internal entries.
An unusual feature of the XArray is the ability to create entries which
@@ -64,89 +64,89 @@ entry will cause the XArray to forget about the range.
Normal API
==========
-Start by initialising an XArray, either with :c:func:`DEFINE_XARRAY`
-for statically allocated XArrays or :c:func:`xa_init` for dynamically
+Start by initialising an XArray, either with DEFINE_XARRAY()
+for statically allocated XArrays or xa_init() for dynamically
allocated ones. A freshly-initialised XArray contains a ``NULL``
pointer at every index.
-You can then set entries using :c:func:`xa_store` and get entries
-using :c:func:`xa_load`. xa_store will overwrite any entry with the
+You can then set entries using xa_store() and get entries
+using xa_load(). xa_store will overwrite any entry with the
new entry and return the previous entry stored at that index. You can
-use :c:func:`xa_erase` instead of calling :c:func:`xa_store` with a
+use xa_erase() instead of calling xa_store() with a
``NULL`` entry. There is no difference between an entry that has never
been stored to, one that has been erased and one that has most recently
had ``NULL`` stored to it.
You can conditionally replace an entry at an index by using
-:c:func:`xa_cmpxchg`. Like :c:func:`cmpxchg`, it will only succeed if
+xa_cmpxchg(). Like cmpxchg(), it will only succeed if
the entry at that index has the 'old' value. It also returns the entry
which was at that index; if it returns the same entry which was passed as
-'old', then :c:func:`xa_cmpxchg` succeeded.
+'old', then xa_cmpxchg() succeeded.
If you want to only store a new entry to an index if the current entry
-at that index is ``NULL``, you can use :c:func:`xa_insert` which
+at that index is ``NULL``, you can use xa_insert() which
returns ``-EBUSY`` if the entry is not empty.
You can enquire whether a mark is set on an entry by using
-:c:func:`xa_get_mark`. If the entry is not ``NULL``, you can set a mark
-on it by using :c:func:`xa_set_mark` and remove the mark from an entry by
-calling :c:func:`xa_clear_mark`. You can ask whether any entry in the
-XArray has a particular mark set by calling :c:func:`xa_marked`.
+xa_get_mark(). If the entry is not ``NULL``, you can set a mark
+on it by using xa_set_mark() and remove the mark from an entry by
+calling xa_clear_mark(). You can ask whether any entry in the
+XArray has a particular mark set by calling xa_marked().
You can copy entries out of the XArray into a plain array by calling
-:c:func:`xa_extract`. Or you can iterate over the present entries in
-the XArray by calling :c:func:`xa_for_each`. You may prefer to use
-:c:func:`xa_find` or :c:func:`xa_find_after` to move to the next present
+xa_extract(). Or you can iterate over the present entries in
+the XArray by calling xa_for_each(). You may prefer to use
+xa_find() or xa_find_after() to move to the next present
entry in the XArray.
-Calling :c:func:`xa_store_range` stores the same entry in a range
+Calling xa_store_range() stores the same entry in a range
of indices. If you do this, some of the other operations will behave
in a slightly odd way. For example, marking the entry at one index
may result in the entry being marked at some, but not all of the other
indices. Storing into one index may result in the entry retrieved by
some, but not all of the other indices changing.
-Sometimes you need to ensure that a subsequent call to :c:func:`xa_store`
-will not need to allocate memory. The :c:func:`xa_reserve` function
+Sometimes you need to ensure that a subsequent call to xa_store()
+will not need to allocate memory. The xa_reserve() function
will store a reserved entry at the indicated index. Users of the
normal API will see this entry as containing ``NULL``. If you do
-not need to use the reserved entry, you can call :c:func:`xa_release`
+not need to use the reserved entry, you can call xa_release()
to remove the unused entry. If another user has stored to the entry
-in the meantime, :c:func:`xa_release` will do nothing; if instead you
-want the entry to become ``NULL``, you should use :c:func:`xa_erase`.
-Using :c:func:`xa_insert` on a reserved entry will fail.
+in the meantime, xa_release() will do nothing; if instead you
+want the entry to become ``NULL``, you should use xa_erase().
+Using xa_insert() on a reserved entry will fail.
-If all entries in the array are ``NULL``, the :c:func:`xa_empty` function
+If all entries in the array are ``NULL``, the xa_empty() function
will return ``true``.
Finally, you can remove all entries from an XArray by calling
-:c:func:`xa_destroy`. If the XArray entries are pointers, you may wish
+xa_destroy(). If the XArray entries are pointers, you may wish
to free the entries first. You can do this by iterating over all present
-entries in the XArray using the :c:func:`xa_for_each` iterator.
+entries in the XArray using the xa_for_each() iterator.
Allocating XArrays
------------------
-If you use :c:func:`DEFINE_XARRAY_ALLOC` to define the XArray, or
-initialise it by passing ``XA_FLAGS_ALLOC`` to :c:func:`xa_init_flags`,
+If you use DEFINE_XARRAY_ALLOC() to define the XArray, or
+initialise it by passing ``XA_FLAGS_ALLOC`` to xa_init_flags(),
the XArray changes to track whether entries are in use or not.
-You can call :c:func:`xa_alloc` to store the entry at an unused index
+You can call xa_alloc() to store the entry at an unused index
in the XArray. If you need to modify the array from interrupt context,
-you can use :c:func:`xa_alloc_bh` or :c:func:`xa_alloc_irq` to disable
+you can use xa_alloc_bh() or xa_alloc_irq() to disable
interrupts while allocating the ID.
-Using :c:func:`xa_store`, :c:func:`xa_cmpxchg` or :c:func:`xa_insert` will
+Using xa_store(), xa_cmpxchg() or xa_insert() will
also mark the entry as being allocated. Unlike a normal XArray, storing
-``NULL`` will mark the entry as being in use, like :c:func:`xa_reserve`.
-To free an entry, use :c:func:`xa_erase` (or :c:func:`xa_release` if
+``NULL`` will mark the entry as being in use, like xa_reserve().
+To free an entry, use xa_erase() (or xa_release() if
you only want to free the entry if it's ``NULL``).
By default, the lowest free entry is allocated starting from 0. If you
want to allocate entries starting at 1, it is more efficient to use
-:c:func:`DEFINE_XARRAY_ALLOC1` or ``XA_FLAGS_ALLOC1``. If you want to
+DEFINE_XARRAY_ALLOC1() or ``XA_FLAGS_ALLOC1``. If you want to
allocate IDs up to a maximum, then wrap back around to the lowest free
-ID, you can use :c:func:`xa_alloc_cyclic`.
+ID, you can use xa_alloc_cyclic().
You cannot use ``XA_MARK_0`` with an allocating XArray as this mark
is used to track whether an entry is free or not. The other marks are
@@ -155,17 +155,17 @@ available for your use.
Memory allocation
-----------------
-The :c:func:`xa_store`, :c:func:`xa_cmpxchg`, :c:func:`xa_alloc`,
-:c:func:`xa_reserve` and :c:func:`xa_insert` functions take a gfp_t
+The xa_store(), xa_cmpxchg(), xa_alloc(),
+xa_reserve() and xa_insert() functions take a gfp_t
parameter in case the XArray needs to allocate memory to store this entry.
If the entry is being deleted, no memory allocation needs to be performed,
and the GFP flags specified will be ignored.
It is possible for no memory to be allocatable, particularly if you pass
a restrictive set of GFP flags. In that case, the functions return a
-special value which can be turned into an errno using :c:func:`xa_err`.
+special value which can be turned into an errno using xa_err().
If you don't need to know exactly which error occurred, using
-:c:func:`xa_is_err` is slightly more efficient.
+xa_is_err() is slightly more efficient.
Locking
-------
@@ -174,54 +174,54 @@ When using the Normal API, you do not have to worry about locking.
The XArray uses RCU and an internal spinlock to synchronise access:
No lock needed:
- * :c:func:`xa_empty`
- * :c:func:`xa_marked`
+ * xa_empty()
+ * xa_marked()
Takes RCU read lock:
- * :c:func:`xa_load`
- * :c:func:`xa_for_each`
- * :c:func:`xa_find`
- * :c:func:`xa_find_after`
- * :c:func:`xa_extract`
- * :c:func:`xa_get_mark`
+ * xa_load()
+ * xa_for_each()
+ * xa_find()
+ * xa_find_after()
+ * xa_extract()
+ * xa_get_mark()
Takes xa_lock internally:
- * :c:func:`xa_store`
- * :c:func:`xa_store_bh`
- * :c:func:`xa_store_irq`
- * :c:func:`xa_insert`
- * :c:func:`xa_insert_bh`
- * :c:func:`xa_insert_irq`
- * :c:func:`xa_erase`
- * :c:func:`xa_erase_bh`
- * :c:func:`xa_erase_irq`
- * :c:func:`xa_cmpxchg`
- * :c:func:`xa_cmpxchg_bh`
- * :c:func:`xa_cmpxchg_irq`
- * :c:func:`xa_store_range`
- * :c:func:`xa_alloc`
- * :c:func:`xa_alloc_bh`
- * :c:func:`xa_alloc_irq`
- * :c:func:`xa_reserve`
- * :c:func:`xa_reserve_bh`
- * :c:func:`xa_reserve_irq`
- * :c:func:`xa_destroy`
- * :c:func:`xa_set_mark`
- * :c:func:`xa_clear_mark`
+ * xa_store()
+ * xa_store_bh()
+ * xa_store_irq()
+ * xa_insert()
+ * xa_insert_bh()
+ * xa_insert_irq()
+ * xa_erase()
+ * xa_erase_bh()
+ * xa_erase_irq()
+ * xa_cmpxchg()
+ * xa_cmpxchg_bh()
+ * xa_cmpxchg_irq()
+ * xa_store_range()
+ * xa_alloc()
+ * xa_alloc_bh()
+ * xa_alloc_irq()
+ * xa_reserve()
+ * xa_reserve_bh()
+ * xa_reserve_irq()
+ * xa_destroy()
+ * xa_set_mark()
+ * xa_clear_mark()
Assumes xa_lock held on entry:
- * :c:func:`__xa_store`
- * :c:func:`__xa_insert`
- * :c:func:`__xa_erase`
- * :c:func:`__xa_cmpxchg`
- * :c:func:`__xa_alloc`
- * :c:func:`__xa_set_mark`
- * :c:func:`__xa_clear_mark`
+ * __xa_store()
+ * __xa_insert()
+ * __xa_erase()
+ * __xa_cmpxchg()
+ * __xa_alloc()
+ * __xa_set_mark()
+ * __xa_clear_mark()
If you want to take advantage of the lock to protect the data structures
-that you are storing in the XArray, you can call :c:func:`xa_lock`
-before calling :c:func:`xa_load`, then take a reference count on the
-object you have found before calling :c:func:`xa_unlock`. This will
+that you are storing in the XArray, you can call xa_lock()
+before calling xa_load(), then take a reference count on the
+object you have found before calling xa_unlock(). This will
prevent stores from removing the object from the array between looking
up the object and incrementing the refcount. You can also use RCU to
avoid dereferencing freed memory, but an explanation of that is beyond
@@ -261,7 +261,7 @@ context and then erase them in softirq context, you can do that this way::
}
If you are going to modify the XArray from interrupt or softirq context,
-you need to initialise the array using :c:func:`xa_init_flags`, passing
+you need to initialise the array using xa_init_flags(), passing
``XA_FLAGS_LOCK_IRQ`` or ``XA_FLAGS_LOCK_BH``.
The above example also shows a common pattern of wanting to extend the
@@ -269,20 +269,20 @@ coverage of the xa_lock on the store side to protect some statistics
associated with the array.
Sharing the XArray with interrupt context is also possible, either
-using :c:func:`xa_lock_irqsave` in both the interrupt handler and process
-context, or :c:func:`xa_lock_irq` in process context and :c:func:`xa_lock`
+using xa_lock_irqsave() in both the interrupt handler and process
+context, or xa_lock_irq() in process context and xa_lock()
in the interrupt handler. Some of the more common patterns have helper
-functions such as :c:func:`xa_store_bh`, :c:func:`xa_store_irq`,
-:c:func:`xa_erase_bh`, :c:func:`xa_erase_irq`, :c:func:`xa_cmpxchg_bh`
-and :c:func:`xa_cmpxchg_irq`.
+functions such as xa_store_bh(), xa_store_irq(),
+xa_erase_bh(), xa_erase_irq(), xa_cmpxchg_bh()
+and xa_cmpxchg_irq().
Sometimes you need to protect access to the XArray with a mutex because
that lock sits above another mutex in the locking hierarchy. That does
-not entitle you to use functions like :c:func:`__xa_erase` without taking
+not entitle you to use functions like __xa_erase() without taking
the xa_lock; the xa_lock is used for lockdep validation and will be used
for other purposes in the future.
-The :c:func:`__xa_set_mark` and :c:func:`__xa_clear_mark` functions are also
+The __xa_set_mark() and __xa_clear_mark() functions are also
available for situations where you look up an entry and want to atomically
set or clear a mark. It may be more efficient to use the advanced API
in this case, as it will save you from walking the tree twice.
@@ -300,27 +300,27 @@ indeed the normal API is implemented in terms of the advanced API. The
advanced API is only available to modules with a GPL-compatible license.
The advanced API is based around the xa_state. This is an opaque data
-structure which you declare on the stack using the :c:func:`XA_STATE`
+structure which you declare on the stack using the XA_STATE()
macro. This macro initialises the xa_state ready to start walking
around the XArray. It is used as a cursor to maintain the position
in the XArray and let you compose various operations together without
having to restart from the top every time.
The xa_state is also used to store errors. You can call
-:c:func:`xas_error` to retrieve the error. All operations check whether
+xas_error() to retrieve the error. All operations check whether
the xa_state is in an error state before proceeding, so there's no need
for you to check for an error after each call; you can make multiple
calls in succession and only check at a convenient point. The only
errors currently generated by the XArray code itself are ``ENOMEM`` and
``EINVAL``, but it supports arbitrary errors in case you want to call
-:c:func:`xas_set_err` yourself.
+xas_set_err() yourself.
-If the xa_state is holding an ``ENOMEM`` error, calling :c:func:`xas_nomem`
+If the xa_state is holding an ``ENOMEM`` error, calling xas_nomem()
will attempt to allocate more memory using the specified gfp flags and
cache it in the xa_state for the next attempt. The idea is that you take
the xa_lock, attempt the operation and drop the lock. The operation
attempts to allocate memory while holding the lock, but it is more
-likely to fail. Once you have dropped the lock, :c:func:`xas_nomem`
+likely to fail. Once you have dropped the lock, xas_nomem()
can try harder to allocate more memory. It will return ``true`` if it
is worth retrying the operation (i.e. that there was a memory error *and*
more memory was allocated). If it has previously allocated memory, and
@@ -333,7 +333,7 @@ Internal Entries
The XArray reserves some entries for its own purposes. These are never
exposed through the normal API, but when using the advanced API, it's
possible to see them. Usually the best way to handle them is to pass them
-to :c:func:`xas_retry`, and retry the operation if it returns ``true``.
+to xas_retry(), and retry the operation if it returns ``true``.
.. flat-table::
:widths: 1 1 6
@@ -343,89 +343,89 @@ to :c:func:`xas_retry`, and retry the operation if it returns ``true``.
- Usage
* - Node
- - :c:func:`xa_is_node`
+ - xa_is_node()
- An XArray node. May be visible when using a multi-index xa_state.
* - Sibling
- - :c:func:`xa_is_sibling`
+ - xa_is_sibling()
- A non-canonical entry for a multi-index entry. The value indicates
which slot in this node has the canonical entry.
* - Retry
- - :c:func:`xa_is_retry`
+ - xa_is_retry()
- This entry is currently being modified by a thread which has the
xa_lock. The node containing this entry may be freed at the end
of this RCU period. You should restart the lookup from the head
of the array.
* - Zero
- - :c:func:`xa_is_zero`
+ - xa_is_zero()
- Zero entries appear as ``NULL`` through the Normal API, but occupy
an entry in the XArray which can be used to reserve the index for
future use. This is used by allocating XArrays for allocated entries
which are ``NULL``.
Other internal entries may be added in the future. As far as possible, they
-will be handled by :c:func:`xas_retry`.
+will be handled by xas_retry().
Additional functionality
------------------------
-The :c:func:`xas_create_range` function allocates all the necessary memory
+The xas_create_range() function allocates all the necessary memory
to store every entry in a range. It will set ENOMEM in the xa_state if
it cannot allocate memory.
-You can use :c:func:`xas_init_marks` to reset the marks on an entry
+You can use xas_init_marks() to reset the marks on an entry
to their default state. This is usually all marks clear, unless the
XArray is marked with ``XA_FLAGS_TRACK_FREE``, in which case mark 0 is set
and all other marks are clear. Replacing one entry with another using
-:c:func:`xas_store` will not reset the marks on that entry; if you want
+xas_store() will not reset the marks on that entry; if you want
the marks reset, you should do that explicitly.
-The :c:func:`xas_load` will walk the xa_state as close to the entry
+The xas_load() will walk the xa_state as close to the entry
as it can. If you know the xa_state has already been walked to the
entry and need to check that the entry hasn't changed, you can use
-:c:func:`xas_reload` to save a function call.
+xas_reload() to save a function call.
If you need to move to a different index in the XArray, call
-:c:func:`xas_set`. This resets the cursor to the top of the tree, which
+xas_set(). This resets the cursor to the top of the tree, which
will generally make the next operation walk the cursor to the desired
spot in the tree. If you want to move to the next or previous index,
-call :c:func:`xas_next` or :c:func:`xas_prev`. Setting the index does
+call xas_next() or xas_prev(). Setting the index does
not walk the cursor around the array so does not require a lock to be
held, while moving to the next or previous index does.
-You can search for the next present entry using :c:func:`xas_find`. This
-is the equivalent of both :c:func:`xa_find` and :c:func:`xa_find_after`;
+You can search for the next present entry using xas_find(). This
+is the equivalent of both xa_find() and xa_find_after();
if the cursor has been walked to an entry, then it will find the next
entry after the one currently referenced. If not, it will return the
-entry at the index of the xa_state. Using :c:func:`xas_next_entry` to
-move to the next present entry instead of :c:func:`xas_find` will save
+entry at the index of the xa_state. Using xas_next_entry() to
+move to the next present entry instead of xas_find() will save
a function call in the majority of cases at the expense of emitting more
inline code.
-The :c:func:`xas_find_marked` function is similar. If the xa_state has
+The xas_find_marked() function is similar. If the xa_state has
not been walked, it will return the entry at the index of the xa_state,
if it is marked. Otherwise, it will return the first marked entry after
-the entry referenced by the xa_state. The :c:func:`xas_next_marked`
-function is the equivalent of :c:func:`xas_next_entry`.
+the entry referenced by the xa_state. The xas_next_marked()
+function is the equivalent of xas_next_entry().
-When iterating over a range of the XArray using :c:func:`xas_for_each`
-or :c:func:`xas_for_each_marked`, it may be necessary to temporarily stop
-the iteration. The :c:func:`xas_pause` function exists for this purpose.
+When iterating over a range of the XArray using xas_for_each()
+or xas_for_each_marked(), it may be necessary to temporarily stop
+the iteration. The xas_pause() function exists for this purpose.
After you have done the necessary work and wish to resume, the xa_state
is in an appropriate state to continue the iteration after the entry
you last processed. If you have interrupts disabled while iterating,
then it is good manners to pause the iteration and reenable interrupts
every ``XA_CHECK_SCHED`` entries.
-The :c:func:`xas_get_mark`, :c:func:`xas_set_mark` and
-:c:func:`xas_clear_mark` functions require the xa_state cursor to have
+The xas_get_mark(), xas_set_mark() and
+xas_clear_mark() functions require the xa_state cursor to have
been moved to the appropriate location in the xarray; they will do
-nothing if you have called :c:func:`xas_pause` or :c:func:`xas_set`
+nothing if you have called xas_pause() or xas_set()
immediately before.
-You can call :c:func:`xas_set_update` to have a callback function
+You can call xas_set_update() to have a callback function
called each time the XArray updates a node. This is used by the page
cache workingset code to maintain its list of nodes which contain only
shadow entries.
@@ -443,25 +443,25 @@ eg indices 64-127 may be tied together, but 2-6 may not be. This may
save substantial quantities of memory; for example tying 512 entries
together will save over 4kB.
-You can create a multi-index entry by using :c:func:`XA_STATE_ORDER`
-or :c:func:`xas_set_order` followed by a call to :c:func:`xas_store`.
-Calling :c:func:`xas_load` with a multi-index xa_state will walk the
+You can create a multi-index entry by using XA_STATE_ORDER()
+or xas_set_order() followed by a call to xas_store().
+Calling xas_load() with a multi-index xa_state will walk the
xa_state to the right location in the tree, but the return value is not
meaningful, potentially being an internal entry or ``NULL`` even when there
-is an entry stored within the range. Calling :c:func:`xas_find_conflict`
+is an entry stored within the range. Calling xas_find_conflict()
will return the first entry within the range or ``NULL`` if there are no
-entries in the range. The :c:func:`xas_for_each_conflict` iterator will
+entries in the range. The xas_for_each_conflict() iterator will
iterate over every entry which overlaps the specified range.
-If :c:func:`xas_load` encounters a multi-index entry, the xa_index
+If xas_load() encounters a multi-index entry, the xa_index
in the xa_state will not be changed. When iterating over an XArray
-or calling :c:func:`xas_find`, if the initial index is in the middle
+or calling xas_find(), if the initial index is in the middle
of a multi-index entry, it will not be altered. Subsequent calls
or iterations will move the index to the first index in the range.
Each entry will only be returned once, no matter how many indices it
occupies.
-Using :c:func:`xas_next` or :c:func:`xas_prev` with a multi-index xa_state
+Using xas_next() or xas_prev() with a multi-index xa_state
is not supported. Using either of these functions on a multi-index entry
will reveal sibling entries; these should be skipped over by the caller.
--
2.21.0
^ permalink raw reply related
* [PATCH v3 4/4] docs: Note that :c:func: should no longer be used
From: Jonathan Corbet @ 2019-06-26 17:28 UTC (permalink / raw)
To: linux-doc
Cc: Mauro Carvalho Chehab, Jani Nikula, linux-kernel, Jonathan Corbet
In-Reply-To: <20190626172859.16113-1-corbet@lwn.net>
Now that we can mark up function() automatically, there is no reason to use
:c:func: and every reason to avoid it. Adjust the documentation to reflect
that fact.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
Documentation/doc-guide/sphinx.rst | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index e60a56640c63..f71ddd592aaa 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -241,11 +241,14 @@ The C domain of the kernel-doc has some additional features. E.g. you can
The func-name (e.g. ioctl) remains in the output but the ref-name changed from
``ioctl`` to ``VIDIOC_LOG_STATUS``. The index entry for this function is also
-changed to ``VIDIOC_LOG_STATUS`` and the function can now referenced by:
-
-.. code-block:: rst
-
- :c:func:`VIDIOC_LOG_STATUS`
+changed to ``VIDIOC_LOG_STATUS``.
+
+Please note that there is no need to use ``c:func:`` to generate cross
+references to function documentation. Due to some Sphinx extension magic,
+the documentation build system will automatically turn a reference to
+``function()`` into a cross reference if an index entry for the given
+function name exists. If you see ``c:func:`` use in a kernel document,
+please feel free to remove it.
list tables
--
2.21.0
^ permalink raw reply related
* [PATCH v3 3/4] kernel-doc: Don't try to mark up function names
From: Jonathan Corbet @ 2019-06-26 17:28 UTC (permalink / raw)
To: linux-doc
Cc: Mauro Carvalho Chehab, Jani Nikula, linux-kernel, Jonathan Corbet
In-Reply-To: <20190626172859.16113-1-corbet@lwn.net>
We now have better automarkup in sphinx itself and, besides, this markup
was incorrect and left :c:func: gunk in the processed docs. Sort of
discouraging that nobody ever noticed...:)
As a first step toward the removal of impenetrable regex magic from
kernel-doc it's a tiny one, but you have to start somewhere.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
scripts/kernel-doc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index c0cb41e65b9b..6b03012750da 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -249,7 +249,7 @@ my @highlights_rst = (
[$type_member_func, "\\:c\\:type\\:`\$1\$2\$3\\\\(\\\\) <\$1>`"],
[$type_member, "\\:c\\:type\\:`\$1\$2\$3 <\$1>`"],
[$type_fp_param, "**\$1\\\\(\\\\)**"],
- [$type_func, "\\:c\\:func\\:`\$1()`"],
+ [$type_func, "\$1()"],
[$type_enum, "\\:c\\:type\\:`\$1 <\$2>`"],
[$type_struct, "\\:c\\:type\\:`\$1 <\$2>`"],
[$type_typedef, "\\:c\\:type\\:`\$1 <\$2>`"],
--
2.21.0
^ permalink raw reply related
* Re: [PATCH] binfmt_elf: Extract .note.gnu.property from an ELF file
From: Yu-cheng Yu @ 2019-06-26 17:30 UTC (permalink / raw)
To: Andy Lutomirski, Dave Martin
Cc: X86 ML, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, LKML,
open list:DOCUMENTATION, Linux-MM, linux-arch, Linux API,
Arnd Bergmann, Balbir Singh, Cyrill Gorcunov, Dave Hansen,
Eugene Syromiatnikov, Florian Weimer, H.J. Lu, Jann Horn,
Jonathan Corbet, Kees Cook, Mike Kravetz, Nadav Amit,
Oleg Nesterov, Pavel Machek, Peter Zijlstra, Randy Dunlap,
Ravi V. Shankar, Vedvyas Shanbhogue, Szabolcs Nagy, libc-alpha
In-Reply-To: <CALCETrVZCzh+KFCF6ijuf4QEPn=R2gJ8FHLpyFd=n+pNOMMMjA@mail.gmail.com>
On Wed, 2019-06-26 at 10:14 -0700, Andy Lutomirski wrote:
> On Thu, May 2, 2019 at 4:10 AM Dave Martin <Dave.Martin@arm.com> wrote:
> >
> > On Wed, May 01, 2019 at 02:12:17PM -0700, Yu-cheng Yu wrote:
> > > An ELF file's .note.gnu.property indicates features the executable file
> > > can support. For example, the property GNU_PROPERTY_X86_FEATURE_1_AND
> > > indicates the file supports GNU_PROPERTY_X86_FEATURE_1_IBT and/or
> > > GNU_PROPERTY_X86_FEATURE_1_SHSTK.
> > >
[...]
>
> Where did PT_GNU_PROPERTY come from? Are there actual docs for it?
> Can someone here tell us what the actual semantics of this new ELF
> thingy are? From some searching, it seems like it's kind of an ELF
> note but kind of not. An actual description would be fantastic.
>
> Also, I don't think there's any actual requirement that the upstream
> kernel recognize existing CET-enabled RHEL 8 binaries as being
> CET-enabled. I tend to think that RHEL 8 jumped the gun here. While
> the upstream kernel should make some reasonble effort to make sure
> that RHEL 8 binaries will continue to run, I don't see why we need to
> go out of our way to keep the full set of mitigations available for
> binaries that were developed against a non-upstream kernel.
>
> In fact, if we handle the legacy bitmap differently from RHEL 8, we
> may *have* to make sure that we don't recognize existing RHEL 8
> binaries as CET-enabled.
We have worked out the issue. Linux will look at only PT_GNU_PROPERTY, which is
a shortcut pointing directly to .note.gnu.property. I have an updated patch,
and will send it out (although it is not yet perfect).
The Linux gABI extension draft is here: https://github.com/hjl-tools/linux-abi/w
iki/linux-abi-draft.pdf.
Yu-cheng
^ permalink raw reply
* Re: [PATCH v2] Documentation: platform: Delete x86-laptop-drivers.txt
From: Jonathan Corbet @ 2019-06-26 17:41 UTC (permalink / raw)
To: Andy Shevchenko
Cc: Puranjay Mohan, Shuah Khan, Greg KH, Cezary Jackiewicz,
Darren Hart, Andy Shevchenko, Linux Documentation List,
linux-kernel-mentees, Linux Kernel Mailing List, Platform Driver
In-Reply-To: <CAHp75Ve+v7o=Ar=5Vc7yZndCxUNf3sn8YwpCHXMwdeJxuLKMoA@mail.gmail.com>
On Thu, 20 Jun 2019 22:50:17 +0300
Andy Shevchenko <andy.shevchenko@gmail.com> wrote:
> On Thu, Jun 20, 2019 at 9:38 PM Puranjay Mohan <puranjay12@gmail.com> wrote:
> >
> > The list of laptops supported by drivers in PDx86 subsystem is quite
> > big and growing. x86-laptop-drivers.txt contains details of very few
> > laptop models. Remove it because it does not serve any purpose.
> >
>
> Acked-by: Andy Shevchenko <andy.shevchenko@gmail.com>
>
> > Signed-off-by: Puranjay Mohan <puranjay12@gmail.com>
I have applied this, thanks.
jon
^ permalink raw reply
* Re: [PATCH] docs: zh_CN: submitting-drivers.rst: Remove a duplicated Documentation/
From: Jonathan Corbet @ 2019-06-26 17:53 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Harry Wei, Alex Shi
In-Reply-To: <47f81418930438d1deab8da1307bcd89ba9afd91.1561225663.git.mchehab+samsung@kernel.org>
On Sat, 22 Jun 2019 14:47:46 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> Somehow, this file ended with Documentation/ twice.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> ---
> Documentation/translations/zh_CN/process/submitting-drivers.rst | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/Documentation/translations/zh_CN/process/submitting-drivers.rst b/Documentation/translations/zh_CN/process/submitting-drivers.rst
> index 72c6cd935821..72f4f45c98de 100644
> --- a/Documentation/translations/zh_CN/process/submitting-drivers.rst
> +++ b/Documentation/translations/zh_CN/process/submitting-drivers.rst
> @@ -22,7 +22,7 @@
> 兴趣的是显卡驱动程序，你也许应该访问 XFree86 项目(http://www.xfree86.org/)
> 和／或 X.org 项目 (http://x.org)。
>
> -另请参阅 Documentation/Documentation/translations/zh_CN/process/submitting-patches.rst 文档。
> +另请参阅 Documentation/translations/zh_CN/process/submitting-patches.rst 文档。
There is such a thing as too much Documentation! :)
Applied, thanks.
jon
^ permalink raw reply
* Re: [PATCH v2] scripts/sphinx-pre-install: fix out-of-tree build
From: Jonathan Corbet @ 2019-06-26 17:54 UTC (permalink / raw)
To: Mike Rapoport; +Cc: Mauro Carvalho Chehab, linux-doc
In-Reply-To: <1561353907-19911-1-git-send-email-rppt@linux.ibm.com>
On Mon, 24 Jun 2019 08:25:07 +0300
Mike Rapoport <rppt@linux.ibm.com> wrote:
> Build of htmldocs fails for out-of-tree builds:
>
> $ make V=1 O=~/build/kernel/ htmldocs
> make -C /home/rppt/build/kernel -f /home/rppt/git/linux-docs/Makefile htmldocs
> make[1]: Entering directory '/home/rppt/build/kernel'
> make -f /home/rppt/git/linux-docs/scripts/Makefile.build obj=scripts/basic
> rm -f .tmp_quiet_recordmcount
> make -f /home/rppt/git/linux-docs/scripts/Makefile.build obj=Documentation htmldocs
> Can't open Documentation/conf.py at /home/rppt/git/linux-docs/scripts/sphinx-pre-install line 230.
> /home/rppt/git/linux-docs/Documentation/Makefile:80: recipe for target 'htmldocs' failed
> make[2]: *** [htmldocs] Error 2
>
> The scripts/sphinx-pre-install is trying to open files in the current
> directory which is $KBUILD_OUTPUT rather than in $srctree.
>
> Fix it.
>
> Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
Applied, thanks.
jon
^ permalink raw reply
* Re: [PATCH] docs: filesystems: Remove uneeded .rst extension on toctables
From: Jonathan Corbet @ 2019-06-26 17:54 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Theodore Ts'o, Andreas Dilger, NeilBrown, Darrick J. Wong,
Matthew Wilcox, Christian Brauner, linux-ext4
In-Reply-To: <d2e4dfee7708a3ef6130d3ffcc579429de6a05c9.1561556105.git.mchehab+samsung@kernel.org>
On Wed, 26 Jun 2019 10:35:11 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> There's no need to use a .rst on Sphinx toc tables. As most of
> the Documentation don't use, remove the remaing occurrences.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Applied, thanks.
jon
^ permalink raw reply
* [Linux-kernel-mentees][PATCH v4 0/5] Documentation: RCU: Convert to reST
From: Jiunn Chang @ 2019-06-26 19:12 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190625062627.26378-1-c0d1n61at3@gmail.com>
This patch series is the initial conversion of the RCU documentation
section. This includes reST markup and renaming txt files to rst. For files
converted, internal links have been created. Checkpatch was used to leverage codespell
for any spelling errors. Each patch in the series has been compiled and reviewed
for warnings and errors. Patches can be bisected.
The changes made in v4 include:
- Change links in rcu.rst to the path in documentation section
- Maintain the original name of the txt files
The changes made in v3 include:
- correcting markup to maintain even more of the original text
- correcting markup for line breaks
- combining all file renaming into one patch
- add reviewed-by tags
- add required public list to CC
The changes made in v2 include:
- correcting markup to maintain as much of the original text as possible
- correcting markup to reduce reader context switching
- breakout file renaming into individual patches in the series
>8---------------------------------------------------------------------------8<
Jiunn Chang (5):
Documentation: RCU: Convert RCU basic concepts to reST
Documentation: RCU: Convert RCU linked list to reST
Documentation: RCU: Convert RCU UP systems to reST
Documentation: RCU: Rename txt files to rst
Documentation: RCU: Add TOC tree hooks
Documentation/RCU/{UP.txt => UP.rst} | 37 +++++---
Documentation/RCU/index.rst | 19 ++++
.../RCU/{listRCU.txt => listRCU.rst} | 38 ++++----
Documentation/RCU/rcu.rst | 92 +++++++++++++++++++
Documentation/RCU/rcu.txt | 89 ------------------
5 files changed, 156 insertions(+), 119 deletions(-)
rename Documentation/RCU/{UP.txt => UP.rst} (84%)
create mode 100644 Documentation/RCU/index.rst
rename Documentation/RCU/{listRCU.txt => listRCU.rst} (92%)
create mode 100644 Documentation/RCU/rcu.rst
delete mode 100644 Documentation/RCU/rcu.txt
--
2.22.0
^ permalink raw reply
* [Linux-kernel-mentees][PATCH v4 1/5] Documentation: RCU: Convert RCU basic concepts to reST
From: Jiunn Chang @ 2019-06-26 19:12 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190625062627.26378-1-c0d1n61at3@gmail.com>
RCU basic concepts reST markup.
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/rcu.txt | 119 +++++++++++++++++++-------------------
1 file changed, 61 insertions(+), 58 deletions(-)
diff --git a/Documentation/RCU/rcu.txt b/Documentation/RCU/rcu.txt
index c818cf65c5a9..8dfb437dacc3 100644
--- a/Documentation/RCU/rcu.txt
+++ b/Documentation/RCU/rcu.txt
@@ -1,5 +1,7 @@
-RCU Concepts
+.. _rcu_doc:
+RCU Concepts
+============
The basic idea behind RCU (read-copy update) is to split destructive
operations into two parts, one that prevents anyone from seeing the data
@@ -8,82 +10,83 @@ A "grace period" must elapse between the two parts, and this grace period
must be long enough that any readers accessing the item being deleted have
since dropped their references. For example, an RCU-protected deletion
from a linked list would first remove the item from the list, wait for
-a grace period to elapse, then free the element. See the listRCU.txt
-file for more information on using RCU with linked lists.
-
+a grace period to elapse, then free the element. See the
+Documentation/RCU/listRCU.rst file for more information on using RCU with
+linked lists.
Frequently Asked Questions
+--------------------------
-o Why would anyone want to use RCU?
+- Why would anyone want to use RCU?
- The advantage of RCU's two-part approach is that RCU readers need
- not acquire any locks, perform any atomic instructions, write to
- shared memory, or (on CPUs other than Alpha) execute any memory
- barriers. The fact that these operations are quite expensive
- on modern CPUs is what gives RCU its performance advantages
- in read-mostly situations. The fact that RCU readers need not
- acquire locks can also greatly simplify deadlock-avoidance code.
+ The advantage of RCU's two-part approach is that RCU readers need
+ not acquire any locks, perform any atomic instructions, write to
+ shared memory, or (on CPUs other than Alpha) execute any memory
+ barriers. The fact that these operations are quite expensive
+ on modern CPUs is what gives RCU its performance advantages
+ in read-mostly situations. The fact that RCU readers need not
+ acquire locks can also greatly simplify deadlock-avoidance code.
-o How can the updater tell when a grace period has completed
- if the RCU readers give no indication when they are done?
+- How can the updater tell when a grace period has completed
+ if the RCU readers give no indication when they are done?
- Just as with spinlocks, RCU readers are not permitted to
- block, switch to user-mode execution, or enter the idle loop.
- Therefore, as soon as a CPU is seen passing through any of these
- three states, we know that that CPU has exited any previous RCU
- read-side critical sections. So, if we remove an item from a
- linked list, and then wait until all CPUs have switched context,
- executed in user mode, or executed in the idle loop, we can
- safely free up that item.
+ Just as with spinlocks, RCU readers are not permitted to
+ block, switch to user-mode execution, or enter the idle loop.
+ Therefore, as soon as a CPU is seen passing through any of these
+ three states, we know that that CPU has exited any previous RCU
+ read-side critical sections. So, if we remove an item from a
+ linked list, and then wait until all CPUs have switched context,
+ executed in user mode, or executed in the idle loop, we can
+ safely free up that item.
- Preemptible variants of RCU (CONFIG_PREEMPT_RCU) get the
- same effect, but require that the readers manipulate CPU-local
- counters. These counters allow limited types of blocking within
- RCU read-side critical sections. SRCU also uses CPU-local
- counters, and permits general blocking within RCU read-side
- critical sections. These variants of RCU detect grace periods
- by sampling these counters.
+ Preemptible variants of RCU (CONFIG_PREEMPT_RCU) get the
+ same effect, but require that the readers manipulate CPU-local
+ counters. These counters allow limited types of blocking within
+ RCU read-side critical sections. SRCU also uses CPU-local
+ counters, and permits general blocking within RCU read-side
+ critical sections. These variants of RCU detect grace periods
+ by sampling these counters.
-o If I am running on a uniprocessor kernel, which can only do one
- thing at a time, why should I wait for a grace period?
+- If I am running on a uniprocessor kernel, which can only do one
+ thing at a time, why should I wait for a grace period?
- See the UP.txt file in this directory.
+ See the Documentation/RCU/UP.rst file for more information.
-o How can I see where RCU is currently used in the Linux kernel?
+- How can I see where RCU is currently used in the Linux kernel?
- Search for "rcu_read_lock", "rcu_read_unlock", "call_rcu",
- "rcu_read_lock_bh", "rcu_read_unlock_bh", "srcu_read_lock",
- "srcu_read_unlock", "synchronize_rcu", "synchronize_net",
- "synchronize_srcu", and the other RCU primitives. Or grab one
- of the cscope databases from:
+ Search for "rcu_read_lock", "rcu_read_unlock", "call_rcu",
+ "rcu_read_lock_bh", "rcu_read_unlock_bh", "srcu_read_lock",
+ "srcu_read_unlock", "synchronize_rcu", "synchronize_net",
+ "synchronize_srcu", and the other RCU primitives. Or grab one
+ of the cscope databases from:
- http://www.rdrop.com/users/paulmck/RCU/linuxusage/rculocktab.html
+ (http://www.rdrop.com/users/paulmck/RCU/linuxusage/rculocktab.html).
-o What guidelines should I follow when writing code that uses RCU?
+- What guidelines should I follow when writing code that uses RCU?
- See the checklist.txt file in this directory.
+ See the checklist.txt file in this directory.
-o Why the name "RCU"?
+- Why the name "RCU"?
- "RCU" stands for "read-copy update". The file listRCU.txt has
- more information on where this name came from, search for
- "read-copy update" to find it.
+ "RCU" stands for "read-copy update". The file Documentation/RCU/listRCU.rst
+ has more information on where this name came from, search for
+ "read-copy update" to find it.
-o I hear that RCU is patented? What is with that?
+- I hear that RCU is patented? What is with that?
- Yes, it is. There are several known patents related to RCU,
- search for the string "Patent" in RTFP.txt to find them.
- Of these, one was allowed to lapse by the assignee, and the
- others have been contributed to the Linux kernel under GPL.
- There are now also LGPL implementations of user-level RCU
- available (http://liburcu.org/).
+ Yes, it is. There are several known patents related to RCU,
+ search for the string "Patent" in RTFP.txt to find them.
+ Of these, one was allowed to lapse by the assignee, and the
+ others have been contributed to the Linux kernel under GPL.
+ There are now also LGPL implementations of user-level RCU
+ available (http://liburcu.org/).
-o I hear that RCU needs work in order to support realtime kernels?
+- I hear that RCU needs work in order to support realtime kernels?
- Realtime-friendly RCU can be enabled via the CONFIG_PREEMPT_RCU
- kernel configuration parameter.
+ Realtime-friendly RCU can be enabled via the CONFIG_PREEMPT_RCU
+ kernel configuration parameter.
-o Where can I find more information on RCU?
+- Where can I find more information on RCU?
- See the RTFP.txt file in this directory.
- Or point your browser at http://www.rdrop.com/users/paulmck/RCU/.
+ See the RTFP.txt file in this directory.
+ Or point your browser at (http://www.rdrop.com/users/paulmck/RCU/).
--
2.22.0
^ permalink raw reply related
* [Linux-kernel-mentees][PATCH v4 2/5] Documentation: RCU: Convert RCU linked list to reST
From: Jiunn Chang @ 2019-06-26 19:12 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190625062627.26378-1-c0d1n61at3@gmail.com>
RCU linked list reST markup.
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/listRCU.txt | 38 ++++++++++++++++++++---------------
1 file changed, 22 insertions(+), 16 deletions(-)
diff --git a/Documentation/RCU/listRCU.txt b/Documentation/RCU/listRCU.txt
index adb5a3782846..7956ff33042b 100644
--- a/Documentation/RCU/listRCU.txt
+++ b/Documentation/RCU/listRCU.txt
@@ -1,5 +1,7 @@
-Using RCU to Protect Read-Mostly Linked Lists
+.. _list_rcu_doc:
+Using RCU to Protect Read-Mostly Linked Lists
+=============================================
One of the best applications of RCU is to protect read-mostly linked lists
("struct list_head" in list.h). One big advantage of this approach
@@ -7,8 +9,8 @@ is that all of the required memory barriers are included for you in
the list macros. This document describes several applications of RCU,
with the best fits first.
-
Example 1: Read-Side Action Taken Outside of Lock, No In-Place Updates
+----------------------------------------------------------------------
The best applications are cases where, if reader-writer locking were
used, the read-side lock would be dropped before taking any action
@@ -24,7 +26,7 @@ added or deleted, rather than being modified in place.
A straightforward example of this use of RCU may be found in the
system-call auditing support. For example, a reader-writer locked
-implementation of audit_filter_task() might be as follows:
+implementation of audit_filter_task() might be as follows::
static enum audit_state audit_filter_task(struct task_struct *tsk)
{
@@ -48,7 +50,7 @@ the corresponding value is returned. By the time that this value is acted
on, the list may well have been modified. This makes sense, since if
you are turning auditing off, it is OK to audit a few extra system calls.
-This means that RCU can be easily applied to the read side, as follows:
+This means that RCU can be easily applied to the read side, as follows::
static enum audit_state audit_filter_task(struct task_struct *tsk)
{
@@ -73,7 +75,7 @@ become list_for_each_entry_rcu(). The _rcu() list-traversal primitives
insert the read-side memory barriers that are required on DEC Alpha CPUs.
The changes to the update side are also straightforward. A reader-writer
-lock might be used as follows for deletion and insertion:
+lock might be used as follows for deletion and insertion::
static inline int audit_del_rule(struct audit_rule *rule,
struct list_head *list)
@@ -106,7 +108,7 @@ lock might be used as follows for deletion and insertion:
return 0;
}
-Following are the RCU equivalents for these two functions:
+Following are the RCU equivalents for these two functions::
static inline int audit_del_rule(struct audit_rule *rule,
struct list_head *list)
@@ -154,13 +156,13 @@ otherwise cause concurrent readers to fail spectacularly.
So, when readers can tolerate stale data and when entries are either added
or deleted, without in-place modification, it is very easy to use RCU!
-
Example 2: Handling In-Place Updates
+------------------------------------
The system-call auditing code does not update auditing rules in place.
However, if it did, reader-writer-locked code to do so might look as
follows (presumably, the field_count is only permitted to decrease,
-otherwise, the added fields would need to be filled in):
+otherwise, the added fields would need to be filled in)::
static inline int audit_upd_rule(struct audit_rule *rule,
struct list_head *list,
@@ -187,7 +189,7 @@ otherwise, the added fields would need to be filled in):
The RCU version creates a copy, updates the copy, then replaces the old
entry with the newly updated entry. This sequence of actions, allowing
concurrent reads while doing a copy to perform an update, is what gives
-RCU ("read-copy update") its name. The RCU code is as follows:
+RCU ("read-copy update") its name. The RCU code is as follows::
static inline int audit_upd_rule(struct audit_rule *rule,
struct list_head *list,
@@ -216,8 +218,8 @@ RCU ("read-copy update") its name. The RCU code is as follows:
Again, this assumes that the caller holds audit_netlink_sem. Normally,
the reader-writer lock would become a spinlock in this sort of code.
-
Example 3: Eliminating Stale Data
+---------------------------------
The auditing examples above tolerate stale data, as do most algorithms
that are tracking external state. Because there is a delay from the
@@ -231,13 +233,16 @@ per-entry spinlock, and, if the "deleted" flag is set, pretends that the
entry does not exist. For this to be helpful, the search function must
return holding the per-entry spinlock, as ipc_lock() does in fact do.
-Quick Quiz: Why does the search function need to return holding the
- per-entry lock for this deleted-flag technique to be helpful?
+Quick Quiz:
+ Why does the search function need to return holding the per-entry lock for
+ this deleted-flag technique to be helpful?
+
+:ref:`Answer to Quick Quiz <answer_quick_quiz_list>`
If the system-call audit module were to ever need to reject stale data,
one way to accomplish this would be to add a "deleted" flag and a "lock"
spinlock to the audit_entry structure, and modify audit_filter_task()
-as follows:
+as follows::
static enum audit_state audit_filter_task(struct task_struct *tsk)
{
@@ -268,7 +273,7 @@ audit_upd_rule() would need additional memory barriers to ensure
that the list_add_rcu() was really executed before the list_del_rcu().
The audit_del_rule() function would need to set the "deleted"
-flag under the spinlock as follows:
+flag under the spinlock as follows::
static inline int audit_del_rule(struct audit_rule *rule,
struct list_head *list)
@@ -290,8 +295,8 @@ flag under the spinlock as follows:
return -EFAULT; /* No matching rule */
}
-
Summary
+-------
Read-mostly list-based data structures that can tolerate stale data are
the most amenable to use of RCU. The simplest case is where entries are
@@ -302,8 +307,9 @@ If stale data cannot be tolerated, then a "deleted" flag may be used
in conjunction with a per-entry spinlock in order to allow the search
function to reject newly deleted data.
+.. _answer_quick_quiz_list:
-Answer to Quick Quiz
+Answer to Quick Quiz:
Why does the search function need to return holding the per-entry
lock for this deleted-flag technique to be helpful?
--
2.22.0
^ permalink raw reply related
* [Linux-kernel-mentees][PATCH v4 3/5] Documentation: RCU: Convert RCU UP systems to reST
From: Jiunn Chang @ 2019-06-26 19:12 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190625062627.26378-1-c0d1n61at3@gmail.com>
RCU UP systems reST markup.
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/UP.txt | 37 +++++++++++++++++++++++--------------
1 file changed, 23 insertions(+), 14 deletions(-)
diff --git a/Documentation/RCU/UP.txt b/Documentation/RCU/UP.txt
index 53bde717017b..67715a47ae89 100644
--- a/Documentation/RCU/UP.txt
+++ b/Documentation/RCU/UP.txt
@@ -1,17 +1,19 @@
-RCU on Uniprocessor Systems
+.. _up_doc:
+RCU on Uniprocessor Systems
+===========================
A common misconception is that, on UP systems, the call_rcu() primitive
may immediately invoke its function. The basis of this misconception
is that since there is only one CPU, it should not be necessary to
wait for anything else to get done, since there are no other CPUs for
-anything else to be happening on. Although this approach will -sort- -of-
+anything else to be happening on. Although this approach will *sort of*
work a surprising amount of the time, it is a very bad idea in general.
This document presents three examples that demonstrate exactly how bad
an idea this is.
-
Example 1: softirq Suicide
+--------------------------
Suppose that an RCU-based algorithm scans a linked list containing
elements A, B, and C in process context, and can delete elements from
@@ -28,8 +30,8 @@ your kernel.
This same problem can occur if call_rcu() is invoked from a hardware
interrupt handler.
-
Example 2: Function-Call Fatality
+---------------------------------
Of course, one could avert the suicide described in the preceding example
by having call_rcu() directly invoke its arguments only if it was called
@@ -46,11 +48,13 @@ its arguments would cause it to fail to make the fundamental guarantee
underlying RCU, namely that call_rcu() defers invoking its arguments until
all RCU read-side critical sections currently executing have completed.
-Quick Quiz #1: why is it -not- legal to invoke synchronize_rcu() in
- this case?
+Quick Quiz #1:
+ Why is it *not* legal to invoke synchronize_rcu() in this case?
+:ref:`Answers to Quick Quiz <answer_quick_quiz_up>`
Example 3: Death by Deadlock
+----------------------------
Suppose that call_rcu() is invoked while holding a lock, and that the
callback function must acquire this same lock. In this case, if
@@ -76,25 +80,30 @@ there are cases where this can be quite ugly:
If call_rcu() directly invokes the callback, painful locking restrictions
or API changes would be required.
-Quick Quiz #2: What locking restriction must RCU callbacks respect?
+Quick Quiz #2:
+ What locking restriction must RCU callbacks respect?
+:ref:`Answers to Quick Quiz <answer_quick_quiz_up>`
Summary
+-------
Permitting call_rcu() to immediately invoke its arguments breaks RCU,
even on a UP system. So do not do it! Even on a UP system, the RCU
-infrastructure -must- respect grace periods, and -must- invoke callbacks
+infrastructure *must* respect grace periods, and *must* invoke callbacks
from a known environment in which no locks are held.
-Note that it -is- safe for synchronize_rcu() to return immediately on
-UP systems, including !PREEMPT SMP builds running on UP systems.
+Note that it *is* safe for synchronize_rcu() to return immediately on
+UP systems, including PREEMPT SMP builds running on UP systems.
-Quick Quiz #3: Why can't synchronize_rcu() return immediately on
- UP systems running preemptable RCU?
+Quick Quiz #3:
+ Why can't synchronize_rcu() return immediately on UP systems running
+ preemptable RCU?
+.. _answer_quick_quiz_up:
Answer to Quick Quiz #1:
- Why is it -not- legal to invoke synchronize_rcu() in this case?
+ Why is it *not* legal to invoke synchronize_rcu() in this case?
Because the calling function is scanning an RCU-protected linked
list, and is therefore within an RCU read-side critical section.
@@ -119,7 +128,7 @@ Answer to Quick Quiz #2:
This restriction might seem gratuitous, since very few RCU
callbacks acquire locks directly. However, a great many RCU
- callbacks do acquire locks -indirectly-, for example, via
+ callbacks do acquire locks *indirectly*, for example, via
the kfree() primitive.
Answer to Quick Quiz #3:
--
2.22.0
^ permalink raw reply related
* [Linux-kernel-mentees][PATCH v4 4/5] Documentation: RCU: Rename txt files to rst
From: Jiunn Chang @ 2019-06-26 19:12 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190625062627.26378-1-c0d1n61at3@gmail.com>
Rename the following files to reST:
- rcu.txt
- listRCU.txt
- UP.txt
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/{UP.txt => UP.rst} | 0
Documentation/RCU/{listRCU.txt => listRCU.rst} | 0
Documentation/RCU/{rcu.txt => rcu.rst} | 0
3 files changed, 0 insertions(+), 0 deletions(-)
rename Documentation/RCU/{UP.txt => UP.rst} (100%)
rename Documentation/RCU/{listRCU.txt => listRCU.rst} (100%)
rename Documentation/RCU/{rcu.txt => rcu.rst} (100%)
diff --git a/Documentation/RCU/UP.txt b/Documentation/RCU/UP.rst
similarity index 100%
rename from Documentation/RCU/UP.txt
rename to Documentation/RCU/UP.rst
diff --git a/Documentation/RCU/listRCU.txt b/Documentation/RCU/listRCU.rst
similarity index 100%
rename from Documentation/RCU/listRCU.txt
rename to Documentation/RCU/listRCU.rst
diff --git a/Documentation/RCU/rcu.txt b/Documentation/RCU/rcu.rst
similarity index 100%
rename from Documentation/RCU/rcu.txt
rename to Documentation/RCU/rcu.rst
--
2.22.0
^ permalink raw reply
* [Linux-kernel-mentees][PATCH v4 5/5] Documentation: RCU: Add TOC tree hooks
From: Jiunn Chang @ 2019-06-26 19:12 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190625062627.26378-1-c0d1n61at3@gmail.com>
Add TOC tree hooks for:
- rcu
- listRCU
- UP
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/index.rst | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
create mode 100644 Documentation/RCU/index.rst
diff --git a/Documentation/RCU/index.rst b/Documentation/RCU/index.rst
new file mode 100644
index 000000000000..340a9725676c
--- /dev/null
+++ b/Documentation/RCU/index.rst
@@ -0,0 +1,19 @@
+.. _rcu_concepts:
+
+============
+RCU concepts
+============
+
+.. toctree::
+ :maxdepth: 1
+
+ rcu
+ listRCU
+ UP
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
--
2.22.0
^ permalink raw reply related
* Re: On Nitrokey Pro's ECC support
From: Jarkko Sakkinen @ 2019-06-26 19:48 UTC (permalink / raw)
To: Jonathan Corbet; +Cc: Konstantin Ryabitsev, linux-doc, linux-kernel
In-Reply-To: <20190626092817.3e5343e6@lwn.net>
On 2019-06-26 18:28, Jonathan Corbet wrote:
> On Wed, 26 Jun 2019 11:21:38 -0400
> Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
>
>> >Maybe Konstantin (copied) might be willing to supply an update to the
>> >document to reflect this?
>>
>> Hello:
>>
>> I just sent a patch with updates that reflect ECC capabilities in
>> newer
>> devices.
>
> Hey, man, that took you just under an hour to get done. We can't all
> just
> wait around while you twiddle your thumbs... :)
>
> Seriously, though, thanks for doing this,
>
> jon
+1 :-)
/Jarkko
^ permalink raw reply
* Re: [PATCH] docs: move gcc_plugins.txt to core-api and rename to .rst
From: Kees Cook @ 2019-06-26 19:59 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Emese Revfy
In-Reply-To: <4937ff4f93282ed57c9859de4300b4d835880ebb.1561556794.git.mchehab+samsung@kernel.org>
On Wed, Jun 26, 2019 at 10:47:46AM -0300, Mauro Carvalho Chehab wrote:
>
>
> The gcc_plugins.txt file is already a ReST file. Move it
> to the core-api book while renaming it.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
-Kees
> ---
> Documentation/{gcc-plugins.txt => core-api/gcc-plugins.rst} | 0
> Documentation/core-api/index.rst | 2 +-
> 2 files changed, 1 insertion(+), 1 deletion(-)
> rename Documentation/{gcc-plugins.txt => core-api/gcc-plugins.rst} (100%)
>
> diff --git a/Documentation/gcc-plugins.txt b/Documentation/core-api/gcc-plugins.rst
> similarity index 100%
> rename from Documentation/gcc-plugins.txt
> rename to Documentation/core-api/gcc-plugins.rst
> diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst
> index 2466a4c51031..d1e5b95bf86d 100644
> --- a/Documentation/core-api/index.rst
> +++ b/Documentation/core-api/index.rst
> @@ -35,7 +35,7 @@ Core utilities
> boot-time-mm
> memory-hotplug
> protection-keys
> -
> + gcc-plugins
>
> Interfaces for kernel debugging
> ===============================
> --
> 2.21.0
>
>
--
Kees Cook
^ permalink raw reply
* [Linux-kernel-mentees][PATCH v5 0/5] Documentation: RCU: Convert to reST
From: Jiunn Chang @ 2019-06-26 20:07 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190626191249.21135-1-c0d1n61at3@gmail.com>
This patch series is the initial conversion of the RCU documentation
section. This includes reST markup and renaming txt files to rst. For files
converted, internal links have been created. Checkpatch was used to leverage codespell
for any spelling errors. Each patch in the series has been compiled and reviewed
for warnings and errors. Patches can be bisected.
The changes make in v5 include:
- add reviewed-by tags
The changes made in v4 include:
- Change links in rcu.rst to the path in documentation section
- Maintain the original name of the txt files
The changes made in v3 include:
- correcting markup to maintain even more of the original text
- correcting markup for line breaks
- combining all file renaming into one patch
- add reviewed-by tags
- add required public list to CC
The changes made in v2 include:
- correcting markup to maintain as much of the original text as possible
- correcting markup to reduce reader context switching
- breakout file renaming into individual patches in the series
>8---------------------------------------------------------------------------8<
Jiunn Chang (5):
Documentation: RCU: Convert RCU basic concepts to reST
Documentation: RCU: Convert RCU linked list to reST
Documentation: RCU: Convert RCU UP systems to reST
Documentation: RCU: Rename txt files to rst
Documentation: RCU: Add TOC tree hooks
Documentation/RCU/{UP.txt => UP.rst} | 37 +++++---
Documentation/RCU/index.rst | 19 ++++
.../RCU/{listRCU.txt => listRCU.rst} | 38 ++++----
Documentation/RCU/rcu.rst | 92 +++++++++++++++++++
Documentation/RCU/rcu.txt | 89 ------------------
5 files changed, 156 insertions(+), 119 deletions(-)
rename Documentation/RCU/{UP.txt => UP.rst} (84%)
create mode 100644 Documentation/RCU/index.rst
rename Documentation/RCU/{listRCU.txt => listRCU.rst} (92%)
create mode 100644 Documentation/RCU/rcu.rst
delete mode 100644 Documentation/RCU/rcu.txt
--
2.22.0
^ permalink raw reply
* [Linux-kernel-mentees][PATCH v5 1/5] Documentation: RCU: Convert RCU basic concepts to reST
From: Jiunn Chang @ 2019-06-26 20:07 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190626191249.21135-1-c0d1n61at3@gmail.com>
RCU basic concepts reST markup.
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
Documentation/RCU/rcu.txt | 119 +++++++++++++++++++-------------------
1 file changed, 61 insertions(+), 58 deletions(-)
diff --git a/Documentation/RCU/rcu.txt b/Documentation/RCU/rcu.txt
index c818cf65c5a9..8dfb437dacc3 100644
--- a/Documentation/RCU/rcu.txt
+++ b/Documentation/RCU/rcu.txt
@@ -1,5 +1,7 @@
-RCU Concepts
+.. _rcu_doc:
+RCU Concepts
+============
The basic idea behind RCU (read-copy update) is to split destructive
operations into two parts, one that prevents anyone from seeing the data
@@ -8,82 +10,83 @@ A "grace period" must elapse between the two parts, and this grace period
must be long enough that any readers accessing the item being deleted have
since dropped their references. For example, an RCU-protected deletion
from a linked list would first remove the item from the list, wait for
-a grace period to elapse, then free the element. See the listRCU.txt
-file for more information on using RCU with linked lists.
-
+a grace period to elapse, then free the element. See the
+Documentation/RCU/listRCU.rst file for more information on using RCU with
+linked lists.
Frequently Asked Questions
+--------------------------
-o Why would anyone want to use RCU?
+- Why would anyone want to use RCU?
- The advantage of RCU's two-part approach is that RCU readers need
- not acquire any locks, perform any atomic instructions, write to
- shared memory, or (on CPUs other than Alpha) execute any memory
- barriers. The fact that these operations are quite expensive
- on modern CPUs is what gives RCU its performance advantages
- in read-mostly situations. The fact that RCU readers need not
- acquire locks can also greatly simplify deadlock-avoidance code.
+ The advantage of RCU's two-part approach is that RCU readers need
+ not acquire any locks, perform any atomic instructions, write to
+ shared memory, or (on CPUs other than Alpha) execute any memory
+ barriers. The fact that these operations are quite expensive
+ on modern CPUs is what gives RCU its performance advantages
+ in read-mostly situations. The fact that RCU readers need not
+ acquire locks can also greatly simplify deadlock-avoidance code.
-o How can the updater tell when a grace period has completed
- if the RCU readers give no indication when they are done?
+- How can the updater tell when a grace period has completed
+ if the RCU readers give no indication when they are done?
- Just as with spinlocks, RCU readers are not permitted to
- block, switch to user-mode execution, or enter the idle loop.
- Therefore, as soon as a CPU is seen passing through any of these
- three states, we know that that CPU has exited any previous RCU
- read-side critical sections. So, if we remove an item from a
- linked list, and then wait until all CPUs have switched context,
- executed in user mode, or executed in the idle loop, we can
- safely free up that item.
+ Just as with spinlocks, RCU readers are not permitted to
+ block, switch to user-mode execution, or enter the idle loop.
+ Therefore, as soon as a CPU is seen passing through any of these
+ three states, we know that that CPU has exited any previous RCU
+ read-side critical sections. So, if we remove an item from a
+ linked list, and then wait until all CPUs have switched context,
+ executed in user mode, or executed in the idle loop, we can
+ safely free up that item.
- Preemptible variants of RCU (CONFIG_PREEMPT_RCU) get the
- same effect, but require that the readers manipulate CPU-local
- counters. These counters allow limited types of blocking within
- RCU read-side critical sections. SRCU also uses CPU-local
- counters, and permits general blocking within RCU read-side
- critical sections. These variants of RCU detect grace periods
- by sampling these counters.
+ Preemptible variants of RCU (CONFIG_PREEMPT_RCU) get the
+ same effect, but require that the readers manipulate CPU-local
+ counters. These counters allow limited types of blocking within
+ RCU read-side critical sections. SRCU also uses CPU-local
+ counters, and permits general blocking within RCU read-side
+ critical sections. These variants of RCU detect grace periods
+ by sampling these counters.
-o If I am running on a uniprocessor kernel, which can only do one
- thing at a time, why should I wait for a grace period?
+- If I am running on a uniprocessor kernel, which can only do one
+ thing at a time, why should I wait for a grace period?
- See the UP.txt file in this directory.
+ See the Documentation/RCU/UP.rst file for more information.
-o How can I see where RCU is currently used in the Linux kernel?
+- How can I see where RCU is currently used in the Linux kernel?
- Search for "rcu_read_lock", "rcu_read_unlock", "call_rcu",
- "rcu_read_lock_bh", "rcu_read_unlock_bh", "srcu_read_lock",
- "srcu_read_unlock", "synchronize_rcu", "synchronize_net",
- "synchronize_srcu", and the other RCU primitives. Or grab one
- of the cscope databases from:
+ Search for "rcu_read_lock", "rcu_read_unlock", "call_rcu",
+ "rcu_read_lock_bh", "rcu_read_unlock_bh", "srcu_read_lock",
+ "srcu_read_unlock", "synchronize_rcu", "synchronize_net",
+ "synchronize_srcu", and the other RCU primitives. Or grab one
+ of the cscope databases from:
- http://www.rdrop.com/users/paulmck/RCU/linuxusage/rculocktab.html
+ (http://www.rdrop.com/users/paulmck/RCU/linuxusage/rculocktab.html).
-o What guidelines should I follow when writing code that uses RCU?
+- What guidelines should I follow when writing code that uses RCU?
- See the checklist.txt file in this directory.
+ See the checklist.txt file in this directory.
-o Why the name "RCU"?
+- Why the name "RCU"?
- "RCU" stands for "read-copy update". The file listRCU.txt has
- more information on where this name came from, search for
- "read-copy update" to find it.
+ "RCU" stands for "read-copy update". The file Documentation/RCU/listRCU.rst
+ has more information on where this name came from, search for
+ "read-copy update" to find it.
-o I hear that RCU is patented? What is with that?
+- I hear that RCU is patented? What is with that?
- Yes, it is. There are several known patents related to RCU,
- search for the string "Patent" in RTFP.txt to find them.
- Of these, one was allowed to lapse by the assignee, and the
- others have been contributed to the Linux kernel under GPL.
- There are now also LGPL implementations of user-level RCU
- available (http://liburcu.org/).
+ Yes, it is. There are several known patents related to RCU,
+ search for the string "Patent" in RTFP.txt to find them.
+ Of these, one was allowed to lapse by the assignee, and the
+ others have been contributed to the Linux kernel under GPL.
+ There are now also LGPL implementations of user-level RCU
+ available (http://liburcu.org/).
-o I hear that RCU needs work in order to support realtime kernels?
+- I hear that RCU needs work in order to support realtime kernels?
- Realtime-friendly RCU can be enabled via the CONFIG_PREEMPT_RCU
- kernel configuration parameter.
+ Realtime-friendly RCU can be enabled via the CONFIG_PREEMPT_RCU
+ kernel configuration parameter.
-o Where can I find more information on RCU?
+- Where can I find more information on RCU?
- See the RTFP.txt file in this directory.
- Or point your browser at http://www.rdrop.com/users/paulmck/RCU/.
+ See the RTFP.txt file in this directory.
+ Or point your browser at (http://www.rdrop.com/users/paulmck/RCU/).
--
2.22.0
^ permalink raw reply related
* [Linux-kernel-mentees][PATCH v5 2/5] Documentation: RCU: Convert RCU linked list to reST
From: Jiunn Chang @ 2019-06-26 20:07 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190626191249.21135-1-c0d1n61at3@gmail.com>
RCU linked list reST markup.
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/listRCU.txt | 38 ++++++++++++++++++++---------------
1 file changed, 22 insertions(+), 16 deletions(-)
diff --git a/Documentation/RCU/listRCU.txt b/Documentation/RCU/listRCU.txt
index adb5a3782846..7956ff33042b 100644
--- a/Documentation/RCU/listRCU.txt
+++ b/Documentation/RCU/listRCU.txt
@@ -1,5 +1,7 @@
-Using RCU to Protect Read-Mostly Linked Lists
+.. _list_rcu_doc:
+Using RCU to Protect Read-Mostly Linked Lists
+=============================================
One of the best applications of RCU is to protect read-mostly linked lists
("struct list_head" in list.h). One big advantage of this approach
@@ -7,8 +9,8 @@ is that all of the required memory barriers are included for you in
the list macros. This document describes several applications of RCU,
with the best fits first.
-
Example 1: Read-Side Action Taken Outside of Lock, No In-Place Updates
+----------------------------------------------------------------------
The best applications are cases where, if reader-writer locking were
used, the read-side lock would be dropped before taking any action
@@ -24,7 +26,7 @@ added or deleted, rather than being modified in place.
A straightforward example of this use of RCU may be found in the
system-call auditing support. For example, a reader-writer locked
-implementation of audit_filter_task() might be as follows:
+implementation of audit_filter_task() might be as follows::
static enum audit_state audit_filter_task(struct task_struct *tsk)
{
@@ -48,7 +50,7 @@ the corresponding value is returned. By the time that this value is acted
on, the list may well have been modified. This makes sense, since if
you are turning auditing off, it is OK to audit a few extra system calls.
-This means that RCU can be easily applied to the read side, as follows:
+This means that RCU can be easily applied to the read side, as follows::
static enum audit_state audit_filter_task(struct task_struct *tsk)
{
@@ -73,7 +75,7 @@ become list_for_each_entry_rcu(). The _rcu() list-traversal primitives
insert the read-side memory barriers that are required on DEC Alpha CPUs.
The changes to the update side are also straightforward. A reader-writer
-lock might be used as follows for deletion and insertion:
+lock might be used as follows for deletion and insertion::
static inline int audit_del_rule(struct audit_rule *rule,
struct list_head *list)
@@ -106,7 +108,7 @@ lock might be used as follows for deletion and insertion:
return 0;
}
-Following are the RCU equivalents for these two functions:
+Following are the RCU equivalents for these two functions::
static inline int audit_del_rule(struct audit_rule *rule,
struct list_head *list)
@@ -154,13 +156,13 @@ otherwise cause concurrent readers to fail spectacularly.
So, when readers can tolerate stale data and when entries are either added
or deleted, without in-place modification, it is very easy to use RCU!
-
Example 2: Handling In-Place Updates
+------------------------------------
The system-call auditing code does not update auditing rules in place.
However, if it did, reader-writer-locked code to do so might look as
follows (presumably, the field_count is only permitted to decrease,
-otherwise, the added fields would need to be filled in):
+otherwise, the added fields would need to be filled in)::
static inline int audit_upd_rule(struct audit_rule *rule,
struct list_head *list,
@@ -187,7 +189,7 @@ otherwise, the added fields would need to be filled in):
The RCU version creates a copy, updates the copy, then replaces the old
entry with the newly updated entry. This sequence of actions, allowing
concurrent reads while doing a copy to perform an update, is what gives
-RCU ("read-copy update") its name. The RCU code is as follows:
+RCU ("read-copy update") its name. The RCU code is as follows::
static inline int audit_upd_rule(struct audit_rule *rule,
struct list_head *list,
@@ -216,8 +218,8 @@ RCU ("read-copy update") its name. The RCU code is as follows:
Again, this assumes that the caller holds audit_netlink_sem. Normally,
the reader-writer lock would become a spinlock in this sort of code.
-
Example 3: Eliminating Stale Data
+---------------------------------
The auditing examples above tolerate stale data, as do most algorithms
that are tracking external state. Because there is a delay from the
@@ -231,13 +233,16 @@ per-entry spinlock, and, if the "deleted" flag is set, pretends that the
entry does not exist. For this to be helpful, the search function must
return holding the per-entry spinlock, as ipc_lock() does in fact do.
-Quick Quiz: Why does the search function need to return holding the
- per-entry lock for this deleted-flag technique to be helpful?
+Quick Quiz:
+ Why does the search function need to return holding the per-entry lock for
+ this deleted-flag technique to be helpful?
+
+:ref:`Answer to Quick Quiz <answer_quick_quiz_list>`
If the system-call audit module were to ever need to reject stale data,
one way to accomplish this would be to add a "deleted" flag and a "lock"
spinlock to the audit_entry structure, and modify audit_filter_task()
-as follows:
+as follows::
static enum audit_state audit_filter_task(struct task_struct *tsk)
{
@@ -268,7 +273,7 @@ audit_upd_rule() would need additional memory barriers to ensure
that the list_add_rcu() was really executed before the list_del_rcu().
The audit_del_rule() function would need to set the "deleted"
-flag under the spinlock as follows:
+flag under the spinlock as follows::
static inline int audit_del_rule(struct audit_rule *rule,
struct list_head *list)
@@ -290,8 +295,8 @@ flag under the spinlock as follows:
return -EFAULT; /* No matching rule */
}
-
Summary
+-------
Read-mostly list-based data structures that can tolerate stale data are
the most amenable to use of RCU. The simplest case is where entries are
@@ -302,8 +307,9 @@ If stale data cannot be tolerated, then a "deleted" flag may be used
in conjunction with a per-entry spinlock in order to allow the search
function to reject newly deleted data.
+.. _answer_quick_quiz_list:
-Answer to Quick Quiz
+Answer to Quick Quiz:
Why does the search function need to return holding the per-entry
lock for this deleted-flag technique to be helpful?
--
2.22.0
^ permalink raw reply related
* [Linux-kernel-mentees][PATCH v5 3/5] Documentation: RCU: Convert RCU UP systems to reST
From: Jiunn Chang @ 2019-06-26 20:07 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190626191249.21135-1-c0d1n61at3@gmail.com>
RCU UP systems reST markup.
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
Documentation/RCU/UP.txt | 37 +++++++++++++++++++++++--------------
1 file changed, 23 insertions(+), 14 deletions(-)
diff --git a/Documentation/RCU/UP.txt b/Documentation/RCU/UP.txt
index 53bde717017b..67715a47ae89 100644
--- a/Documentation/RCU/UP.txt
+++ b/Documentation/RCU/UP.txt
@@ -1,17 +1,19 @@
-RCU on Uniprocessor Systems
+.. _up_doc:
+RCU on Uniprocessor Systems
+===========================
A common misconception is that, on UP systems, the call_rcu() primitive
may immediately invoke its function. The basis of this misconception
is that since there is only one CPU, it should not be necessary to
wait for anything else to get done, since there are no other CPUs for
-anything else to be happening on. Although this approach will -sort- -of-
+anything else to be happening on. Although this approach will *sort of*
work a surprising amount of the time, it is a very bad idea in general.
This document presents three examples that demonstrate exactly how bad
an idea this is.
-
Example 1: softirq Suicide
+--------------------------
Suppose that an RCU-based algorithm scans a linked list containing
elements A, B, and C in process context, and can delete elements from
@@ -28,8 +30,8 @@ your kernel.
This same problem can occur if call_rcu() is invoked from a hardware
interrupt handler.
-
Example 2: Function-Call Fatality
+---------------------------------
Of course, one could avert the suicide described in the preceding example
by having call_rcu() directly invoke its arguments only if it was called
@@ -46,11 +48,13 @@ its arguments would cause it to fail to make the fundamental guarantee
underlying RCU, namely that call_rcu() defers invoking its arguments until
all RCU read-side critical sections currently executing have completed.
-Quick Quiz #1: why is it -not- legal to invoke synchronize_rcu() in
- this case?
+Quick Quiz #1:
+ Why is it *not* legal to invoke synchronize_rcu() in this case?
+:ref:`Answers to Quick Quiz <answer_quick_quiz_up>`
Example 3: Death by Deadlock
+----------------------------
Suppose that call_rcu() is invoked while holding a lock, and that the
callback function must acquire this same lock. In this case, if
@@ -76,25 +80,30 @@ there are cases where this can be quite ugly:
If call_rcu() directly invokes the callback, painful locking restrictions
or API changes would be required.
-Quick Quiz #2: What locking restriction must RCU callbacks respect?
+Quick Quiz #2:
+ What locking restriction must RCU callbacks respect?
+:ref:`Answers to Quick Quiz <answer_quick_quiz_up>`
Summary
+-------
Permitting call_rcu() to immediately invoke its arguments breaks RCU,
even on a UP system. So do not do it! Even on a UP system, the RCU
-infrastructure -must- respect grace periods, and -must- invoke callbacks
+infrastructure *must* respect grace periods, and *must* invoke callbacks
from a known environment in which no locks are held.
-Note that it -is- safe for synchronize_rcu() to return immediately on
-UP systems, including !PREEMPT SMP builds running on UP systems.
+Note that it *is* safe for synchronize_rcu() to return immediately on
+UP systems, including PREEMPT SMP builds running on UP systems.
-Quick Quiz #3: Why can't synchronize_rcu() return immediately on
- UP systems running preemptable RCU?
+Quick Quiz #3:
+ Why can't synchronize_rcu() return immediately on UP systems running
+ preemptable RCU?
+.. _answer_quick_quiz_up:
Answer to Quick Quiz #1:
- Why is it -not- legal to invoke synchronize_rcu() in this case?
+ Why is it *not* legal to invoke synchronize_rcu() in this case?
Because the calling function is scanning an RCU-protected linked
list, and is therefore within an RCU read-side critical section.
@@ -119,7 +128,7 @@ Answer to Quick Quiz #2:
This restriction might seem gratuitous, since very few RCU
callbacks acquire locks directly. However, a great many RCU
- callbacks do acquire locks -indirectly-, for example, via
+ callbacks do acquire locks *indirectly*, for example, via
the kfree() primitive.
Answer to Quick Quiz #3:
--
2.22.0
^ permalink raw reply related
* [Linux-kernel-mentees][PATCH v5 4/5] Documentation: RCU: Rename txt files to rst
From: Jiunn Chang @ 2019-06-26 20:07 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190626191249.21135-1-c0d1n61at3@gmail.com>
Rename the following files to reST:
- rcu.txt
- listRCU.txt
- UP.txt
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/{UP.txt => UP.rst} | 0
Documentation/RCU/{listRCU.txt => listRCU.rst} | 0
Documentation/RCU/{rcu.txt => rcu.rst} | 0
3 files changed, 0 insertions(+), 0 deletions(-)
rename Documentation/RCU/{UP.txt => UP.rst} (100%)
rename Documentation/RCU/{listRCU.txt => listRCU.rst} (100%)
rename Documentation/RCU/{rcu.txt => rcu.rst} (100%)
diff --git a/Documentation/RCU/UP.txt b/Documentation/RCU/UP.rst
similarity index 100%
rename from Documentation/RCU/UP.txt
rename to Documentation/RCU/UP.rst
diff --git a/Documentation/RCU/listRCU.txt b/Documentation/RCU/listRCU.rst
similarity index 100%
rename from Documentation/RCU/listRCU.txt
rename to Documentation/RCU/listRCU.rst
diff --git a/Documentation/RCU/rcu.txt b/Documentation/RCU/rcu.rst
similarity index 100%
rename from Documentation/RCU/rcu.txt
rename to Documentation/RCU/rcu.rst
--
2.22.0
^ permalink raw reply
* [Linux-kernel-mentees][PATCH v5 5/5] Documentation: RCU: Add TOC tree hooks
From: Jiunn Chang @ 2019-06-26 20:07 UTC (permalink / raw)
To: skhan
Cc: linux-kernel-mentees, rcu, linux-doc, paulmck, josh, rostedt,
mathieu.desnoyers, jiangshanlai, joel, corbet
In-Reply-To: <20190626191249.21135-1-c0d1n61at3@gmail.com>
Add TOC tree hooks for:
- rcu
- listRCU
- UP
Signed-off-by: Jiunn Chang <c0d1n61at3@gmail.com>
---
Documentation/RCU/index.rst | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
create mode 100644 Documentation/RCU/index.rst
diff --git a/Documentation/RCU/index.rst b/Documentation/RCU/index.rst
new file mode 100644
index 000000000000..340a9725676c
--- /dev/null
+++ b/Documentation/RCU/index.rst
@@ -0,0 +1,19 @@
+.. _rcu_concepts:
+
+============
+RCU concepts
+============
+
+.. toctree::
+ :maxdepth: 1
+
+ rcu
+ listRCU
+ UP
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
--
2.22.0
^ permalink raw reply related
* Re: [PATCH 2/2] mm, slab: Extend vm/drop_caches to shrink kmem slabs
From: Roman Gushchin @ 2019-06-26 20:19 UTC (permalink / raw)
To: Waiman Long
Cc: Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim,
Andrew Morton, Alexander Viro, Jonathan Corbet, Luis Chamberlain,
Kees Cook, Johannes Weiner, Michal Hocko, Vladimir Davydov,
linux-mm@kvack.org, linux-doc@vger.kernel.org,
linux-fsdevel@vger.kernel.org, cgroups@vger.kernel.org,
linux-kernel@vger.kernel.org, Shakeel Butt, Andrea Arcangeli
In-Reply-To: <20190624174219.25513-3-longman@redhat.com>
On Mon, Jun 24, 2019 at 01:42:19PM -0400, Waiman Long wrote:
> With the slub memory allocator, the numbers of active slab objects
> reported in /proc/slabinfo are not real because they include objects
> that are held by the per-cpu slab structures whether they are actually
> used or not. The problem gets worse the more CPUs a system have. For
> instance, looking at the reported number of active task_struct objects,
> one will wonder where all the missing tasks gone.
>
> I know it is hard and costly to get a real count of active objects. So
> I am not advocating for that. Instead, this patch extends the
> /proc/sys/vm/drop_caches sysctl parameter by using a new bit (bit 3)
> to shrink all the kmem slabs which will flush out all the slabs in the
> per-cpu structures and give a more accurate view of how much memory are
> really used up by the active slab objects. This is a costly operation,
> of course, but it gives a way to have a clearer picture of the actual
> number of slab objects used, if the need arises.
>
> The upper range of the drop_caches sysctl parameter is increased to 15
> to allow all possible combinations of the lowest 4 bits.
>
> On a 2-socket 64-core 256-thread ARM64 system with 64k page size after
> a parallel kernel build, the amount of memory occupied by slabs before
> and after echoing to drop_caches were:
>
> # grep task_struct /proc/slabinfo
> task_struct 48376 48434 4288 61 4 : tunables 0 0
> 0 : slabdata 794 794 0
> # grep "^S[lRU]" /proc/meminfo
> Slab: 3419072 kB
> SReclaimable: 354688 kB
> SUnreclaim: 3064384 kB
> # echo 3 > /proc/sys/vm/drop_caches
> # grep "^S[lRU]" /proc/meminfo
> Slab: 3351680 kB
> SReclaimable: 316096 kB
> SUnreclaim: 3035584 kB
> # echo 8 > /proc/sys/vm/drop_caches
> # grep "^S[lRU]" /proc/meminfo
> Slab: 1008192 kB
> SReclaimable: 126912 kB
> SUnreclaim: 881280 kB
> # grep task_struct /proc/slabinfo
> task_struct 2601 6588 4288 61 4 : tunables 0 0
> 0 : slabdata 108 108 0
>
> Shrinking the slabs saves more than 2GB of memory in this case. This
> new feature certainly fulfills the promise of dropping caches.
>
> Unlike counting objects in the per-node caches done by /proc/slabinfo
> which is rather light weight, iterating all the per-cpu caches and
> shrinking them is much more heavy weight.
>
> For this particular instance, the time taken to shrinks all the root
> caches was about 30.2ms. There were 73 memory cgroup and the longest
> time taken for shrinking the largest one was about 16.4ms. The total
> shrinking time was about 101ms.
>
> Because of the potential long time to shrinks all the caches, the
> slab_mutex was taken multiple times - once for all the root caches
> and once for each memory cgroup. This is to reduce the slab_mutex hold
> time to minimize impact to other running applications that may need to
> acquire the mutex.
>
> The slab shrinking feature is only available when CONFIG_MEMCG_KMEM is
> defined as the code need to access slab_root_caches to iterate all the
> root caches.
>
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
> Documentation/sysctl/vm.txt | 11 ++++++++--
> fs/drop_caches.c | 4 ++++
> include/linux/slab.h | 1 +
> kernel/sysctl.c | 4 ++--
> mm/slab_common.c | 44 +++++++++++++++++++++++++++++++++++++
> 5 files changed, 60 insertions(+), 4 deletions(-)
>
> diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt
> index 749322060f10..b643ac8968d2 100644
> --- a/Documentation/sysctl/vm.txt
> +++ b/Documentation/sysctl/vm.txt
> @@ -207,8 +207,8 @@ Setting this to zero disables periodic writeback altogether.
> drop_caches
>
> Writing to this will cause the kernel to drop clean caches, as well as
> -reclaimable slab objects like dentries and inodes. Once dropped, their
> -memory becomes free.
> +reclaimable slab objects like dentries and inodes. It can also be used
> +to shrink the slabs. Once dropped, their memory becomes free.
>
> To free pagecache:
> echo 1 > /proc/sys/vm/drop_caches
> @@ -216,6 +216,8 @@ To free reclaimable slab objects (includes dentries and inodes):
> echo 2 > /proc/sys/vm/drop_caches
> To free slab objects and pagecache:
> echo 3 > /proc/sys/vm/drop_caches
> +To shrink the slabs:
> + echo 8 > /proc/sys/vm/drop_caches
>
> This is a non-destructive operation and will not free any dirty objects.
> To increase the number of objects freed by this operation, the user may run
> @@ -223,6 +225,11 @@ To increase the number of objects freed by this operation, the user may run
> number of dirty objects on the system and create more candidates to be
> dropped.
>
> +Shrinking the slabs can reduce the memory footprint used by the slabs.
> +It also makes the number of active objects reported in /proc/slabinfo
> +more representative of the actual number of objects used for the slub
> +memory allocator.
> +
> This file is not a means to control the growth of the various kernel caches
> (inodes, dentries, pagecache, etc...) These objects are automatically
> reclaimed by the kernel when memory is needed elsewhere on the system.
> diff --git a/fs/drop_caches.c b/fs/drop_caches.c
> index d31b6c72b476..633b99e25dab 100644
> --- a/fs/drop_caches.c
> +++ b/fs/drop_caches.c
> @@ -9,6 +9,7 @@
> #include <linux/writeback.h>
> #include <linux/sysctl.h>
> #include <linux/gfp.h>
> +#include <linux/slab.h>
> #include "internal.h"
>
> /* A global variable is a bit ugly, but it keeps the code simple */
> @@ -65,6 +66,9 @@ int drop_caches_sysctl_handler(struct ctl_table *table, int write,
> drop_slab();
> count_vm_event(DROP_SLAB);
> }
> + if (sysctl_drop_caches & 8) {
> + kmem_cache_shrink_all();
> + }
> if (!stfu) {
> pr_info("%s (%d): drop_caches: %d\n",
> current->comm, task_pid_nr(current),
> diff --git a/include/linux/slab.h b/include/linux/slab.h
> index 9449b19c5f10..f7c1626b2aa6 100644
> --- a/include/linux/slab.h
> +++ b/include/linux/slab.h
> @@ -149,6 +149,7 @@ struct kmem_cache *kmem_cache_create_usercopy(const char *name,
> void (*ctor)(void *));
> void kmem_cache_destroy(struct kmem_cache *);
> int kmem_cache_shrink(struct kmem_cache *);
> +void kmem_cache_shrink_all(void);
>
> void memcg_create_kmem_cache(struct mem_cgroup *, struct kmem_cache *);
> void memcg_deactivate_kmem_caches(struct mem_cgroup *);
> diff --git a/kernel/sysctl.c b/kernel/sysctl.c
> index 1beca96fb625..feeb867dabd7 100644
> --- a/kernel/sysctl.c
> +++ b/kernel/sysctl.c
> @@ -129,7 +129,7 @@ static int __maybe_unused neg_one = -1;
> static int zero;
> static int __maybe_unused one = 1;
> static int __maybe_unused two = 2;
> -static int __maybe_unused four = 4;
> +static int __maybe_unused fifteen = 15;
> static unsigned long zero_ul;
> static unsigned long one_ul = 1;
> static unsigned long long_max = LONG_MAX;
> @@ -1455,7 +1455,7 @@ static struct ctl_table vm_table[] = {
> .mode = 0644,
> .proc_handler = drop_caches_sysctl_handler,
> .extra1 = &one,
> - .extra2 = &four,
> + .extra2 = &fifteen,
> },
> #ifdef CONFIG_COMPACTION
> {
> diff --git a/mm/slab_common.c b/mm/slab_common.c
> index 58251ba63e4a..b3c5b64f9bfb 100644
> --- a/mm/slab_common.c
> +++ b/mm/slab_common.c
> @@ -956,6 +956,50 @@ int kmem_cache_shrink(struct kmem_cache *cachep)
> }
> EXPORT_SYMBOL(kmem_cache_shrink);
Hi Waiman!
>
> +#ifdef CONFIG_MEMCG_KMEM
> +static void kmem_cache_shrink_memcg(struct mem_cgroup *memcg,
> + void __maybe_unused *arg)
> +{
> + struct kmem_cache *s;
> +
> + if (memcg == root_mem_cgroup)
> + return;
> + mutex_lock(&slab_mutex);
> + list_for_each_entry(s, &memcg->kmem_caches,
> + memcg_params.kmem_caches_node) {
> + kmem_cache_shrink(s);
> + }
> + mutex_unlock(&slab_mutex);
> + cond_resched();
> +}
A couple of questions:
1) how about skipping already offlined kmem_caches? They are already shrunk,
so you probably won't get much out of them. Or isn't it true?
2) what's your long-term vision here? do you think that we need to shrink
kmem_caches periodically, depending on memory pressure? how a user
will use this new sysctl?
What's the problem you're trying to solve in general?
Thanks!
Roman
^ permalink raw reply
* Re: [PATCH] drm: fix a reference for a renamed file: fb/modedb.rst
From: Daniel Vetter @ 2019-06-26 21:27 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Maarten Lankhorst, Maxime Ripard, Sean Paul,
David Airlie, Daniel Vetter, dri-devel
In-Reply-To: <699d7618720e2808f9aa094a13ab2f3545c3c25c.1561565652.git.mchehab+samsung@kernel.org>
On Wed, Jun 26, 2019 at 01:14:13PM -0300, Mauro Carvalho Chehab wrote:
> Due to two patches being applied about the same time, the
> reference for modedb.rst file got wrong:
>
> Documentation/fb/modedb.txt is now Documentation/fb/modedb.rst.
>
> Fixes: 1bf4e09227c3 ("drm/modes: Allow to specify rotation and reflection on the commandline")
> Fixes: ab42b818954c ("docs: fb: convert docs to ReST and rename to *.rst")
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
What's the merge plan here? doc-next? If so:
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> ---
> drivers/gpu/drm/drm_modes.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/drm_modes.c b/drivers/gpu/drm/drm_modes.c
> index 57e6408288c8..4645af681ef8 100644
> --- a/drivers/gpu/drm/drm_modes.c
> +++ b/drivers/gpu/drm/drm_modes.c
> @@ -1680,7 +1680,7 @@ static int drm_mode_parse_cmdline_options(char *str, size_t len,
> *
> * Additionals options can be provided following the mode, using a comma to
> * separate each option. Valid options can be found in
> - * Documentation/fb/modedb.txt.
> + * Documentation/fb/modedb.rst.
> *
> * The intermediate drm_cmdline_mode structure is required to store additional
> * options from the command line modline like the force-enable/disable flag.
> --
> 2.21.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply
* Re: [PATCH v5 01/18] kunit: test: add KUnit test runner core
From: Luis Chamberlain @ 2019-06-26 22:02 UTC (permalink / raw)
To: Brendan Higgins
Cc: Stephen Boyd, Frank Rowand, Greg KH, Josh Poimboeuf, Kees Cook,
Kieran Bingham, Peter Zijlstra, Rob Herring, shuah,
Theodore Ts'o, Masahiro Yamada, devicetree, dri-devel,
kunit-dev, open list:DOCUMENTATION, linux-fsdevel, linux-kbuild,
Linux Kernel Mailing List, open list:KERNEL SELFTEST FRAMEWORK,
linux-nvdimm, linux-um, Sasha Levin, Bird, Timothy,
Amir Goldstein, Dan Carpenter, Daniel Vetter, Jeff Dike,
Joel Stanley, Julia Lawall, Kevin Hilman, Knut Omang,
Logan Gunthorpe, Michael Ellerman, Petr Mladek, Randy Dunlap,
Richard Weinberger, David Rientjes, Steven Rostedt, wfg
In-Reply-To: <CAFd5g45fSdpytudDyD3Yo1ti=kU_JJ6S9yz53_L=pnZTjQFU9A@mail.gmail.com>
On Tue, Jun 25, 2019 at 11:41:47PM -0700, Brendan Higgins wrote:
> On Tue, Jun 25, 2019 at 4:02 PM Luis Chamberlain <mcgrof@kernel.org> wrote:
> >
> > On Tue, Jun 25, 2019 at 03:14:45PM -0700, Brendan Higgins wrote:
> > > On Tue, Jun 25, 2019 at 2:44 PM Luis Chamberlain <mcgrof@kernel.org> wrote:
> > > > Since its a new architecture and since you seem to imply most tests
> > > > don't require locking or even IRQs disabled, I think its worth to
> > > > consider the impact of adding such extreme locking requirements for
> > > > an initial ramp up.
> > >
> > > Fair enough, I can see the point of not wanting to use irq disabled
> > > until we get someone complaining about it, but I think making it
> > > thread safe is reasonable. It means there is one less thing to confuse
> > > a KUnit user and the only penalty paid is some very minor performance.
> >
> > One reason I'm really excited about kunit is speed... so by all means I
> > think we're at a good point to analyze performance optimizationsm if
> > they do make sense.
>
> Yeah, but I think there are much lower hanging fruit than this (as you
> point out below). I am all for making/keeping KUnit super fast, but I
> also don't want to waste time with premature optimizations and I think
> having thread safe expectations and non-thread safe expectations hurts
> usability.
>
> Still, I am on board with making this a mutex instead of a spinlock for now.
>
> > While on the topic of parallization, what about support for running
> > different test cases in parallel? Or at the very least different kunit
> > modules in parallel. Few questions come up based on this prospect:
> >
> > * Why not support parallelism from the start?
>
> Just because it is more work and there isn't much to gain from it right now.
>
> Some numbers:
> I currently have a collection of 86 test cases in the branch that this
> patchset is from.
Impressive, imagine 86 tests and kunit is not even *merged yet*.
> I turned on PRINTK_TIME and looked at the first
> KUnit output and the last. On UML, start time was 0.090000, and end
> time was 0.090000. Looks like sched_clock is not very good on UML.
Since you have a python thing that kicks tests, what if you just run
time on it?
> Still it seems quite likely that all of these tests run around 0.01
> seconds or less on UML: I ran KUnit with only 2 test cases enabled
> three times and got an average runtime of 1.55867 seconds with a
> standard deviation of 0.0346747. I then ran it another three times
> with all test cases enabled and got an average runtime of 1.535
> seconds with a standard deviation of 0.0150997. The second average is
> less, but that doesn't really mean anything because it is well within
> one standard deviation with a very small sample size. Nevertheless, we
> can conclude that the actual runtime of those 84 test cases is most
> likely within one standard deviation, so on the order of 0.01 seconds.
>
> On x86 running on QEMU, first message from KUnit was printed at
> 0.194251 and the last KUnit message was printed at 0.340915, meaning
> that all 86 test cases ran in about 0.146664 seconds.
Pretty impressive numbers. But can you blame me for expressing the
desire to possibly being able to do better? I am not saying -- let's
definitely have parallelism in place *now*. Just wanted to hear out
tangibles of why we *don't* want it now.
And.. also, since we are reviewing now, try to target so that the code
can later likely get a face lift to support parallelism without
requiring much changes.
> In any case, running KUnit tests in parallel is definitely something I
> plan on adding it eventually, but it just doesn't really seem worth it
> right now.
Makes sense!
> I find the incremental build time of the kernel to
> typically be between 3 and 30 seconds, and a clean build to be between
> 30 seconds to several minutes, depending on the number of available
> cores, so I don't think most users would even notice the amount of
> runtime contributed by the actual unit tests until we start getting
> into the 1000s of test cases. I don't suspect it will become an issue
> until we get into the 10,000s of test cases. I think we are a pretty
> long way off from that.
All makes sense, and agreed based on the numbers you are providing.
Thanks for the details!
> > * Are you opposed to eventually having this added? For instance, there is
> > enough code on lib/test_kmod.c for batching tons of kthreads each
> > one running its own thing for testing purposes which could be used
> > as template.
>
> I am not opposed to adding it eventually at all. I actually plan on
> doing so, just not in this patchset. There are a lot of additional
> features, improvements, and sugar that I really want to add, so much
> so that most of it doesn't belong in this patchset; I just think this
> is one of those things that belongs in a follow up. I tried to boil
> down this patchset to as small as I could while still being useful;
> this is basically an MVP. Maybe after this patchset gets merged I
> should post a list of things I have ready for review, or would like to
> work on, and people can comment on what things they want to see next.
Groovy.
> > * If we eventually *did* support it:
> > - Would logs be skewed?
>
> Probably, before I went with the TAP approach, I was tagging each
> message with the test case it came from and I could have parsed it and
> assembled a coherent view of the logs using that; now that I am using
> TAP conforming output, that won't work. I haven't really thought too
> hard about how to address it, but there are ways. For the UML users, I
> am planning on adding a feature to guarantee hermeticity between tests
> running in different modules by adding a feature that allows a single
> kernel to be built with all tests included, and then determine which
> tests get run by passing in command line arguments or something. This
> way you can get the isolation from running tests in separate
> environments without increasing the build cost. We could also use this
> method to achieve parallelism by dispatching multiple kernels at once.
Indeed. Or... wait for it... containers... There tools for these sorts
of things already. And I'm evaluating such prospects now for some other
tests I care for.
> That only works for UML, but I imagine you could do something similar
> for users running tests under qemu.
Or containers.
> > - Could we have a way to query: give me log for only kunit module
> > named "foo"?
>
> Yeah, I think that would make sense as part of the hermeticity thing I
> mentioned above.
>
> Hope that seems reasonable!
All groovy. Thanks for the details!
Luis
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox