* [PATCH 05/10] scripts/documentation-file-ref-check: better handle translations
From: Mauro Carvalho Chehab @ 2019-05-29 23:09 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
In-Reply-To: <cover.1559170790.git.mchehab+samsung@kernel.org>
Only seek for translation renames inside the translation
directory.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/documentation-file-ref-check | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/scripts/documentation-file-ref-check b/scripts/documentation-file-ref-check
index 63e9542656f1..6b622b88f4cf 100755
--- a/scripts/documentation-file-ref-check
+++ b/scripts/documentation-file-ref-check
@@ -141,6 +141,10 @@ print "Auto-fixing broken references. Please double-check the results\n";
foreach my $ref (keys %broken_ref) {
my $new =$ref;
+ my $basedir = ".";
+ # On translations, only seek inside the translations directory
+ $basedir = $1 if ($ref =~ m,(Documentation/translations/[^/]+),);
+
# get just the basename
$new =~ s,.*/,,;
@@ -161,18 +165,18 @@ foreach my $ref (keys %broken_ref) {
# usual reason for breakage: file renamed to .rst
if (!$f) {
$new =~ s/\.txt$/.rst/;
- $f=qx(find . -iname $new) if ($new);
+ $f=qx(find $basedir -iname $new) if ($new);
}
# usual reason for breakage: use dash or underline
if (!$f) {
$new =~ s/[-_]/[-_]/g;
- $f=qx(find . -iname $new) if ($new);
+ $f=qx(find $basedir -iname $new) if ($new);
}
# Wild guess: seek for the same name on another place
if (!$f) {
- $f = qx(find . -iname $new) if ($new);
+ $f = qx(find $basedir -iname $new) if ($new);
}
my @find = split /\s+/, $f;
--
2.21.0
^ permalink raw reply related
* [PATCH 09/10] docs: by default, build docs a lot faster with Sphinx >= 1.7
From: Mauro Carvalho Chehab @ 2019-05-29 23:09 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
In-Reply-To: <cover.1559170790.git.mchehab+samsung@kernel.org>
Since Sphinx version 1.7, it is possible to use "-jauto" in
order to speedup documentation builds. On older versions,
while -j was already supported, one would need to set the
number of threads manually.
So, if SPHINXOPTS is not provided, add -jauto, in order to
speed up the build. That makes it *a lot* times faster than
without -j.
If one really wants to slow things down, it can just use:
make SPHINXOPTS=-j1 htmldocs
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
Documentation/Makefile | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Documentation/Makefile b/Documentation/Makefile
index 380e24053d6f..794233d05789 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -28,6 +28,8 @@ ifeq ($(HAVE_SPHINX),0)
else # HAVE_SPHINX
+SPHINXOPTS = $(shell perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto" if ($$1 >= "1.7") } ;} close IN')
+
# User-friendly check for pdflatex and latexmk
HAVE_PDFLATEX := $(shell if which $(PDFLATEX) >/dev/null 2>&1; then echo 1; else echo 0; fi)
HAVE_LATEXMK := $(shell if which latexmk >/dev/null 2>&1; then echo 1; else echo 0; fi)
--
2.21.0
^ permalink raw reply related
* [PATCH 01/10] docs: cdomain.py: get rid of a warning since version 1.8
From: Mauro Carvalho Chehab @ 2019-05-29 23:09 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
In-Reply-To: <cover.1559170790.git.mchehab+samsung@kernel.org>
There's a new warning about a deprecation function. Add a
logic at cdomain.py to avoid that.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
Documentation/sphinx/cdomain.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/Documentation/sphinx/cdomain.py b/Documentation/sphinx/cdomain.py
index cf13ff3a656c..cbac8e608dc4 100644
--- a/Documentation/sphinx/cdomain.py
+++ b/Documentation/sphinx/cdomain.py
@@ -48,7 +48,10 @@ major, minor, patch = sphinx.version_info[:3]
def setup(app):
- app.override_domain(CDomain)
+ if (major == 1 and minor < 8):
+ app.override_domain(CDomain)
+ else:
+ app.add_domain(CDomain, override=True)
return dict(
version = __version__,
--
2.21.0
^ permalink raw reply related
* [PATCH 00/10] Improvements to the documentation build system
From: Mauro Carvalho Chehab @ 2019-05-29 23:09 UTC (permalink / raw)
To: Linux Doc Mailing List, Jonathan Corbet
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Matthew Wilcox, Federico Vaga, Joel Nider, Mike Rapoport
Hi Jon,
This series contain some improvements for the building system.
I sent already several of the patches here. They're rebased on the
top of your docs-next tree:
patch 1: gets rid of a warning since version 1.8 (I guess it starts
appearing with 1.8.6);
patches 2 to 4: improve the pre-install script;
patches 5 to 8: improve the script with checks broken doc references;
patch 9: by default, use "-jauto" with Sphinx 1.7 or upper, in order
to speed up the build.
patch 10 changes the recommended Sphinx version to 1.7.9. It keeps
the minimal supported version to 1.3.
Patch 4 contains a good description of the improvements made at
the build system.
If you prefer, you can pull those patches (and the next series I'm
submitting you) from my development git tree:
https://git.linuxtv.org/mchehab/experimental.git/log/?h=fix_doc_links_v2
Regards,
Mauro
-
Mauro Carvalho Chehab (10):
docs: cdomain.py: get rid of a warning since version 1.8
scripts/sphinx-pre-install: make activate hint smarter
scripts/sphinx-pre-install: get rid of RHEL7 explicity check
scripts/sphinx-pre-install: always check if version is compatible with
build
scripts/documentation-file-ref-check: better handle translations
scripts/documentation-file-ref-check: exclude false-positives
scripts/documentation-file-ref-check: improve tools ref handling
scripts/documentation-file-ref-check: teach about .txt -> .yaml
renames
docs: by default, build docs a lot faster with Sphinx >= 1.7
docs: requirements.txt: recommend Sphinx 1.7.9
Documentation/Makefile | 7 +++
Documentation/doc-guide/sphinx.rst | 17 +++---
Documentation/sphinx/cdomain.py | 5 +-
Documentation/sphinx/requirements.txt | 4 +-
scripts/documentation-file-ref-check | 44 ++++++++++++----
scripts/sphinx-pre-install | 75 +++++++++++++++------------
6 files changed, 97 insertions(+), 55 deletions(-)
--
2.21.0
^ permalink raw reply
* [PATCH 03/10] scripts/sphinx-pre-install: get rid of RHEL7 explicity check
From: Mauro Carvalho Chehab @ 2019-05-29 23:09 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
In-Reply-To: <cover.1559170790.git.mchehab+samsung@kernel.org>
RHEL8 was already launched. This test won't get it, and will
do the wrong thing. Ok, we could fix it, but now we check
Sphinx version to ensure that it matches the minimal (1.3),
so there's no need for an explicit check there.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/sphinx-pre-install | 13 -------------
1 file changed, 13 deletions(-)
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 11239eb29695..ded3e2ef3f8d 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -581,19 +581,6 @@ sub check_needs()
print "Unknown OS\n";
}
- # RHEL 7.x and clones have Sphinx version 1.1.x and incomplete texlive
- if (($system_release =~ /Red Hat Enterprise Linux/) ||
- ($system_release =~ /CentOS/) ||
- ($system_release =~ /Scientific Linux/) ||
- ($system_release =~ /Oracle Linux Server/)) {
- $virtualenv = 1;
- $pdf = 0;
-
- printf("NOTE: On this distro, Sphinx and TexLive shipped versions are incompatible\n");
- printf("with doc build. So, use Sphinx via a Python virtual environment.\n\n");
- printf("This script can't install a TexLive version that would provide PDF.\n");
- }
-
# Check for needed programs/tools
check_sphinx();
check_perl_module("Pod::Usage", 0);
--
2.21.0
^ permalink raw reply related
* [PATCH 06/10] scripts/documentation-file-ref-check: exclude false-positives
From: Mauro Carvalho Chehab @ 2019-05-29 23:09 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
In-Reply-To: <cover.1559170790.git.mchehab+samsung@kernel.org>
There are at least two cases where a documentation file was gone
for good, but the text still mentions it:
1) drivers/vhost/vhost.c:
the reference for Documentation/virtual/lguest/lguest.c is just
to give credits to the original work that vhost replaced;
2) Documentation/scsi/scsi_mid_low_api.txt:
It gives credit and mentions the old Documentation/Configure.help
file that used to be part of Kernel 2.4.x
As we don't want to keep the script to keep pinpoint to those
every time, let's add a logic at the script to allow it to ignore
valid false-positives like the above.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/documentation-file-ref-check | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/scripts/documentation-file-ref-check b/scripts/documentation-file-ref-check
index 6b622b88f4cf..05235775cc71 100755
--- a/scripts/documentation-file-ref-check
+++ b/scripts/documentation-file-ref-check
@@ -8,6 +8,14 @@ use warnings;
use strict;
use Getopt::Long qw(:config no_auto_abbrev);
+# NOTE: only add things here when the file was gone, but the text wants
+# to mention a past documentation file, for example, to give credits for
+# the original work.
+my %false_positives = (
+ "Documentation/scsi/scsi_mid_low_api.txt" => "Documentation/Configure.help",
+ "drivers/vhost/vhost.c" => "Documentation/virtual/lguest/lguest.c",
+);
+
my $scriptname = $0;
$scriptname =~ s,.*/([^/]+/),$1,;
@@ -122,6 +130,11 @@ while (<IN>) {
next if (grep -e, glob("$path/$ref $path/$fulref"));
}
+ # Discard known false-positives
+ if (defined($false_positives{$f})) {
+ next if ($false_positives{$f} eq $fulref);
+ }
+
if ($fix) {
if (!($ref =~ m/(scripts|Kconfig|Kbuild)/)) {
$broken_ref{$ref}++;
--
2.21.0
^ permalink raw reply related
* [PATCH 02/10] scripts/sphinx-pre-install: make activate hint smarter
From: Mauro Carvalho Chehab @ 2019-05-29 23:09 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
In-Reply-To: <cover.1559170790.git.mchehab+samsung@kernel.org>
It is possible that multiple Sphinx virtualenvs are installed
on a given kernel tree. Change the logic to get the latest
version of those, as this is probably what the user wants.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/sphinx-pre-install | 22 ++++++++++++++--------
1 file changed, 14 insertions(+), 8 deletions(-)
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 8c2d1bcf2e02..11239eb29695 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -1,7 +1,7 @@
#!/usr/bin/perl
use strict;
-# Copyright (c) 2017 Mauro Carvalho Chehab <mchehab@kernel.org>
+# Copyright (c) 2017-2019 Mauro Carvalho Chehab <mchehab@kernel.org>
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
@@ -15,6 +15,7 @@ use strict;
my $conf = "Documentation/conf.py";
my $requirement_file = "Documentation/sphinx/requirements.txt";
+my $virtenv_prefix = "sphinx_";
#
# Static vars
@@ -28,7 +29,8 @@ my $need_symlink = 0;
my $need_sphinx = 0;
my $rec_sphinx_upgrade = 0;
my $install = "";
-my $virtenv_dir = "sphinx_";
+my $virtenv_dir = "";
+my $min_version;
#
# Command line arguments
@@ -229,7 +231,6 @@ sub get_sphinx_fname()
sub check_sphinx()
{
- my $min_version;
my $rec_version;
my $cur_version;
@@ -255,7 +256,7 @@ sub check_sphinx()
die "Can't get recommended sphinx version from $requirement_file" if (!$min_version);
- $virtenv_dir .= $rec_version;
+ $virtenv_dir = $virtenv_prefix . $rec_version;
my $sphinx = get_sphinx_fname();
return if ($sphinx eq "");
@@ -612,18 +613,23 @@ sub check_needs()
which("sphinx-build-3");
}
if ($need_sphinx || $rec_sphinx_upgrade) {
- my $activate = "$virtenv_dir/bin/activate";
- if (-e "$ENV{'PWD'}/$activate") {
+ my $min_activate = "$ENV{'PWD'}/${virtenv_prefix}${min_version}/bin/activate";
+ my @activates = glob "$ENV{'PWD'}/${virtenv_prefix}*/bin/activate";
+
+ @activates = sort {$b cmp $a} @activates;
+
+ if (scalar @activates > 0 && $activates[0] ge $min_activate) {
printf "\nNeed to activate virtualenv with:\n";
- printf "\t. $activate\n";
+ printf "\t. $activates[0]\n";
} else {
+ my $rec_activate = "$virtenv_dir/bin/activate";
my $virtualenv = findprog("virtualenv-3");
$virtualenv = findprog("virtualenv-3.5") if (!$virtualenv);
$virtualenv = findprog("virtualenv") if (!$virtualenv);
$virtualenv = "virtualenv" if (!$virtualenv);
printf "\t$virtualenv $virtenv_dir\n";
- printf "\t. $activate\n";
+ printf "\t. $rec_activate\n";
printf "\tpip install -r $requirement_file\n";
$need++ if (!$rec_sphinx_upgrade);
--
2.21.0
^ permalink raw reply related
* [PATCH 04/10] scripts/sphinx-pre-install: always check if version is compatible with build
From: Mauro Carvalho Chehab @ 2019-05-29 23:09 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
In-Reply-To: <cover.1559170790.git.mchehab+samsung@kernel.org>
Call the script every time a make docs target is selected, on
a simplified check mode.
With this change, the script will set two vars:
$min_version - obtained from `needs_sphinx` var inside
conf.py (currently, '1.3')
$rec_version - obtained from sphinx/requirements.txt.
With those changes, a target like "make htmldocs" will do:
1) If no sphinx-build/sphinx-build3 is found, it will run
the script on normal mode as before, checking for all
system dependencies and providing install hints for the
needed programs and will abort the build;
2) If no sphinx-build/sphinx-build3 is found, but there is
a sphinx_${VER}/bin/activate file, and if
${VER} >= $min_version (string comparation), it will
run in full mode, and will recommend to activate the
virtualenv. If there are multiple virtualenvs, it
will string sort the versions, recommending the
highest version and will abort the build;
3) If Sphinx is detected but has a version lower than
$min_version, it will run in full mode - with will
recommend creating a virtual env using sphinx/requirements.txt,
and will abort the build.
4) If Sphinx is detected and version is lower than
$rec_version, it will run in full mode and will
recommend creating a virtual env using sphinx/requirements.txt.
In this case, it **won't** abort the build.
5) If Sphinx is detected and version is equal or righer than
$rec_version it will return just after detecting the
version ("quick mode"), not checking if are there any
missing dependencies.
Just like before, if one wants to install Sphinx from the
distro, it has to call the script manually and use `--no-virtualenv`
argument to get the hints for his OS:
You should run:
sudo dnf install -y python3-sphinx python3-sphinx_rtd_theme
While here, add a small help for the three optional arguments
for the script.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
Documentation/Makefile | 5 +++++
scripts/sphinx-pre-install | 46 +++++++++++++++++++++++++-------------
2 files changed, 35 insertions(+), 16 deletions(-)
diff --git a/Documentation/Makefile b/Documentation/Makefile
index e889e7cb8511..380e24053d6f 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -70,12 +70,14 @@ quiet_cmd_sphinx = SPHINX $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
$(abspath $(BUILDDIR)/$3/$4)
htmldocs:
+ @./scripts/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))
linkcheckdocs:
@$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,linkcheck,$(var),,$(var)))
latexdocs:
+ @./scripts/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,latex,$(var),latex,$(var)))
ifeq ($(HAVE_PDFLATEX),0)
@@ -87,14 +89,17 @@ pdfdocs:
else # HAVE_PDFLATEX
pdfdocs: latexdocs
+ @./scripts/sphinx-pre-install --version-check
$(foreach var,$(SPHINXDIRS), $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" -C $(BUILDDIR)/$(var)/latex || exit;)
endif # HAVE_PDFLATEX
epubdocs:
+ @./scripts/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var)))
xmldocs:
+ @./scripts/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,xml,$(var),xml,$(var)))
endif # HAVE_SPHINX
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index ded3e2ef3f8d..f001fc2fcf12 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -38,6 +38,7 @@ my $min_version;
my $pdf = 1;
my $virtualenv = 1;
+my $version_check = 0;
#
# List of required texlive packages on Fedora and OpenSuse
@@ -277,20 +278,22 @@ sub check_sphinx()
die "$sphinx didn't return its version" if (!$cur_version);
- printf "Sphinx version %s (minimal: %s, recommended >= %s)\n",
- $cur_version, $min_version, $rec_version;
-
if ($cur_version lt $min_version) {
- print "Warning: Sphinx version should be >= $min_version\n\n";
+ printf "ERROR: Sphinx version is %s. It should be >= %s (recommended >= %s)\n",
+ $cur_version, $min_version, $rec_version;;
$need_sphinx = 1;
return;
}
if ($cur_version lt $rec_version) {
+ printf "Sphinx version %s\n", $cur_version;
print "Warning: It is recommended at least Sphinx version $rec_version.\n";
- print " To upgrade, use:\n\n";
$rec_sphinx_upgrade = 1;
+ return;
}
+
+ # On version check mode, just assume Sphinx has all mandatory deps
+ exit (0) if ($version_check);
}
#
@@ -575,14 +578,18 @@ sub check_distros()
sub check_needs()
{
- if ($system_release) {
- print "Detected OS: $system_release.\n";
- } else {
- print "Unknown OS\n";
- }
-
# Check for needed programs/tools
check_sphinx();
+
+ if ($system_release) {
+ print "Detected OS: $system_release.\n\n";
+ } else {
+ print "Unknown OS\n\n";
+ }
+
+ print "To upgrade Sphinx, use:\n\n" if ($rec_sphinx_upgrade);
+
+ # Check for needed programs/tools
check_perl_module("Pod::Usage", 0);
check_program("make", 0);
check_program("gcc", 0);
@@ -601,13 +608,14 @@ sub check_needs()
}
if ($need_sphinx || $rec_sphinx_upgrade) {
my $min_activate = "$ENV{'PWD'}/${virtenv_prefix}${min_version}/bin/activate";
- my @activates = glob "$ENV{'PWD'}/${virtenv_prefix}*/bin/activate";
+ my @activates = glob "$ENV{'PWD'}/${virtenv_prefix}*/bin/activate";
- @activates = sort {$b cmp $a} @activates;
+ @activates = sort {$b cmp $a} @activates;
- if (scalar @activates > 0 && $activates[0] ge $min_activate) {
- printf "\nNeed to activate virtualenv with:\n";
+ if ($need_sphinx && scalar @activates > 0 && $activates[0] ge $min_activate) {
+ printf "\nNeed to activate a compatible Sphinx version on virtualenv with:\n";
printf "\t. $activates[0]\n";
+ exit (1);
} else {
my $rec_activate = "$virtenv_dir/bin/activate";
my $virtualenv = findprog("virtualenv-3");
@@ -646,8 +654,14 @@ while (@ARGV) {
$virtualenv = 0;
} elsif ($arg eq "--no-pdf"){
$pdf = 0;
+ } elsif ($arg eq "--version-check"){
+ $version_check = 1;
} else {
- print "Usage:\n\t$0 <--no-virtualenv> <--no-pdf>\n\n";
+ print "Usage:\n\t$0 <--no-virtualenv> <--no-pdf> <--version-check>\n\n";
+ print "Where:\n";
+ print "\t--no-virtualenv\t- Recommend installing Sphinx instead of using a virtualenv\n";
+ print "\t--version-check\t- if version is compatible, don't check for missing dependencies\n";
+ print "\t--no-pdf\t- don't check for dependencies required to build PDF docs\n\n";
exit -1;
}
}
--
2.21.0
^ permalink raw reply related
* [PATCH 07/10] scripts/documentation-file-ref-check: improve tools ref handling
From: Mauro Carvalho Chehab @ 2019-05-29 23:09 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
In-Reply-To: <cover.1559170790.git.mchehab+samsung@kernel.org>
There's a false positive on perf/util:
tools/perf/util/s390-cpumsf.c: Documentation/perf.data-file-format.txt
The file is there at tools/perf/Documentation/, but the logic
with detects relative documentation references inside tools is
not capable of detecting it.
So, improve it.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/documentation-file-ref-check | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/documentation-file-ref-check b/scripts/documentation-file-ref-check
index 05235775cc71..5d775ca7469b 100755
--- a/scripts/documentation-file-ref-check
+++ b/scripts/documentation-file-ref-check
@@ -127,7 +127,7 @@ while (<IN>) {
if ($f =~ m/tools/) {
my $path = $f;
$path =~ s,(.*)/.*,$1,;
- next if (grep -e, glob("$path/$ref $path/$fulref"));
+ next if (grep -e, glob("$path/$ref $path/../$ref $path/$fulref"));
}
# Discard known false-positives
--
2.21.0
^ permalink raw reply related
* [PATCH 08/10] scripts/documentation-file-ref-check: teach about .txt -> .yaml renames
From: Mauro Carvalho Chehab @ 2019-05-29 23:09 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
In-Reply-To: <cover.1559170790.git.mchehab+samsung@kernel.org>
At DT, files are being renamed to jason. Teach the script how to
handle such renames when used in fix mode.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/documentation-file-ref-check | 19 ++++++++++++++-----
1 file changed, 14 insertions(+), 5 deletions(-)
diff --git a/scripts/documentation-file-ref-check b/scripts/documentation-file-ref-check
index 5d775ca7469b..ff16db269079 100755
--- a/scripts/documentation-file-ref-check
+++ b/scripts/documentation-file-ref-check
@@ -165,13 +165,22 @@ foreach my $ref (keys %broken_ref) {
# usual reason for breakage: DT file moved around
if ($ref =~ /devicetree/) {
- my $search = $new;
- $search =~ s,^.*/,,;
- $f = qx(find Documentation/devicetree/ -iname "*$search*") if ($search);
+ # usual reason for breakage: DT file renamed to .yaml
if (!$f) {
- # Manufacturer name may have changed
- $search =~ s/^.*,//;
+ my $new_ref = $ref;
+ $new_ref =~ s/\.txt$/.yaml/;
+ $f=$new_ref if (-f $new_ref);
+ }
+
+ if (!$f) {
+ my $search = $new;
+ $search =~ s,^.*/,,;
$f = qx(find Documentation/devicetree/ -iname "*$search*") if ($search);
+ if (!$f) {
+ # Manufacturer name may have changed
+ $search =~ s/^.*,//;
+ $f = qx(find Documentation/devicetree/ -iname "*$search*") if ($search);
+ }
}
}
--
2.21.0
^ permalink raw reply related
* Re: [PATCH] Allow to exclude specific file types in LoadPin
From: Kees Cook @ 2019-05-29 23:07 UTC (permalink / raw)
To: Ke Wu
Cc: Jonathan Corbet, James Morris, Serge E. Hallyn, linux-doc,
linux-kernel, linux-security-module
In-Reply-To: <20190529224350.6460-1-mikewu@google.com>
On Wed, May 29, 2019 at 03:43:50PM -0700, Ke Wu wrote:
> Linux kernel already provide MODULE_SIG and KEXEC_VERIFY_SIG to
> make sure loaded kernel module and kernel image are trusted. This
> patch adds a kernel command line option "loadpin.exclude" which
> allows to exclude specific file types from LoadPin. This is useful
> when people want to use different mechanisms to verify module and
> kernel image while still use LoadPin to protect the integrity of
> other files kernel loads.
Cool; I like this. A few thoughts below...
>
> Signed-off-by: Ke Wu <mikewu@google.com>
> ---
> Documentation/admin-guide/LSM/LoadPin.rst | 10 ++++++
> security/loadpin/loadpin.c | 37 +++++++++++++++++++++++
> 2 files changed, 47 insertions(+)
>
> diff --git a/Documentation/admin-guide/LSM/LoadPin.rst b/Documentation/admin-guide/LSM/LoadPin.rst
> index 32070762d24c..716ad9b23c9a 100644
> --- a/Documentation/admin-guide/LSM/LoadPin.rst
> +++ b/Documentation/admin-guide/LSM/LoadPin.rst
> @@ -19,3 +19,13 @@ block device backing the filesystem is not read-only, a sysctl is
> created to toggle pinning: ``/proc/sys/kernel/loadpin/enabled``. (Having
> a mutable filesystem means pinning is mutable too, but having the
> sysctl allows for easy testing on systems with a mutable filesystem.)
> +
> +It's also possible to exclude specific file types from LoadPin using kernel
> +command line option "``loadpin.exclude``". By default, all files are
> +included, but they can be excluded using kernel command line option such
> +as "``loadpin.exclude=kernel-module,kexec-image``". This allows to use
> +different mechanisms such as ``CONFIG_MODULE_SIG`` and
> +``CONFIG_KEXEC_VERIFY_SIG`` to verify kernel module and kernel image while
> +still use LoadPin to protect the integrity of other files kernel loads. The
> +full list of valid file types can be found in ``kernel_read_file_str``
> +defined in ``include/linux/fs.h``.
> diff --git a/security/loadpin/loadpin.c b/security/loadpin/loadpin.c
> index 055fb0a64169..8ee0c58fea40 100644
> --- a/security/loadpin/loadpin.c
> +++ b/security/loadpin/loadpin.c
> @@ -45,6 +45,8 @@ static void report_load(const char *origin, struct file *file, char *operation)
> }
>
> static int enforce = IS_ENABLED(CONFIG_SECURITY_LOADPIN_ENFORCE);
> +static char *exclude_read_files[READING_MAX_ID];
> +static int ignore_read_file_id[READING_MAX_ID];
Since this is set up at init, let's mark ignore_read_file_id with
__ro_after_init.
> static struct super_block *pinned_root;
> static DEFINE_SPINLOCK(pinned_root_spinlock);
>
> @@ -129,6 +131,12 @@ static int loadpin_read_file(struct file *file, enum kernel_read_file_id id)
> struct super_block *load_root;
> const char *origin = kernel_read_file_id_str(id);
>
> + /* If the file id is excluded, ignore the pinning. */
> + if ((unsigned int)id < READING_MAX_ID && ignore_read_file_id[id]) {
Can you use ARRAY_SIZE(ignore_read_file_id) here instead of
READING_MAX_ID?
> + report_load(origin, file, "pinning-excluded");
> + return 0;
> + }
> +
> /* This handles the older init_module API that has a NULL file. */
> if (!file) {
> if (!enforce) {
> @@ -187,10 +195,37 @@ static struct security_hook_list loadpin_hooks[] __lsm_ro_after_init = {
> LSM_HOOK_INIT(kernel_load_data, loadpin_load_data),
> };
>
> +static void parse_exclude(void)
Please mark this __init (since it's called from another __init
function).
> +{
> + int i, j;
> + char *cur;
> +
> + for (i = 0; i < ARRAY_SIZE(exclude_read_files); i++) {
> + cur = exclude_read_files[i];
> + if (!cur)
> + break;
> + if (*cur == '\0')
> + continue;
> +
> + for (j = 0; j < ARRAY_SIZE(kernel_read_file_str); j++) {
> + if (strcmp(cur, kernel_read_file_str[j]) == 0) {
> + pr_info("excluding: %s\n",
> + kernel_read_file_str[j]);
> + ignore_read_file_id[j] = 1;
> + /*
> + * Can not break, because one read_file_str
> + * may map to more than on read_file_id.
> + */
> + }
> + }
> + }
> +}
> +
> static int __init loadpin_init(void)
> {
> pr_info("ready to pin (currently %senforcing)\n",
> enforce ? "" : "not ");
> + parse_exclude();
> security_add_hooks(loadpin_hooks, ARRAY_SIZE(loadpin_hooks), "loadpin");
> return 0;
> }
> @@ -203,3 +238,5 @@ DEFINE_LSM(loadpin) = {
> /* Should not be mutable after boot, so not listed in sysfs (perm == 0). */
> module_param(enforce, int, 0);
> MODULE_PARM_DESC(enforce, "Enforce module/firmware pinning");
> +module_param_array_named(exclude, exclude_read_files, charp, NULL, 0);
> +MODULE_PARM_DESC(exclude, "Exclude pinning specific read file types");
> --
> 2.22.0.rc1.257.g3120a18244-goog
>
Everything else looks good; thanks!
--
Kees Cook
^ permalink raw reply
* Re: [PATCH 4/5] docs: by default, build docs a lot faster with Sphinx >= 1.7
From: Jonathan Corbet @ 2019-05-29 23:02 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel
In-Reply-To: <baf19095789f2b2ed0c7a940703037a00cd77850.1558955082.git.mchehab+samsung@kernel.org>
On Mon, 27 May 2019 08:07:40 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> Since Sphinx version 1.7, it is possible to use "-jauto" in
> order to speedup documentation builds. On older versions,
> while -j was already supported, one would need to set the
> number of threads manually.
>
> So, if SPHINXOPTS is not provided, add -jauto, in order to
> speed up the build. That makes it *a lot* times faster than
> without -j.
>
> If one really wants to slow things down, it can just use:
>
> make SPHINXOPTS=-j1 htmldocs
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> ---
> Documentation/Makefile | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/Documentation/Makefile b/Documentation/Makefile
> index 380e24053d6f..794233d05789 100644
> --- a/Documentation/Makefile
> +++ b/Documentation/Makefile
> @@ -28,6 +28,8 @@ ifeq ($(HAVE_SPHINX),0)
>
> else # HAVE_SPHINX
>
> +SPHINXOPTS = $(shell perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto" if ($$1 >= "1.7") } ;} close IN')
> +
So this totally fails to work for me with any version of sphinx, and I'm
not enough of a Perl person to figure it out. Sometimes I'll see the
sphinx-build output, i.e.:
sphinx-build 1.8.4
and sometimes (like with 2.0) I don't, but I never get -jauto regardless.
Not sure what's going on here?
Thanks,
jon
^ permalink raw reply
* [PATCH] Allow to exclude specific file types in LoadPin
From: Ke Wu @ 2019-05-29 22:43 UTC (permalink / raw)
To: Kees Cook, Jonathan Corbet, James Morris, Serge E. Hallyn
Cc: linux-doc, linux-kernel, linux-security-module, Ke Wu
Linux kernel already provide MODULE_SIG and KEXEC_VERIFY_SIG to
make sure loaded kernel module and kernel image are trusted. This
patch adds a kernel command line option "loadpin.exclude" which
allows to exclude specific file types from LoadPin. This is useful
when people want to use different mechanisms to verify module and
kernel image while still use LoadPin to protect the integrity of
other files kernel loads.
Signed-off-by: Ke Wu <mikewu@google.com>
---
Documentation/admin-guide/LSM/LoadPin.rst | 10 ++++++
security/loadpin/loadpin.c | 37 +++++++++++++++++++++++
2 files changed, 47 insertions(+)
diff --git a/Documentation/admin-guide/LSM/LoadPin.rst b/Documentation/admin-guide/LSM/LoadPin.rst
index 32070762d24c..716ad9b23c9a 100644
--- a/Documentation/admin-guide/LSM/LoadPin.rst
+++ b/Documentation/admin-guide/LSM/LoadPin.rst
@@ -19,3 +19,13 @@ block device backing the filesystem is not read-only, a sysctl is
created to toggle pinning: ``/proc/sys/kernel/loadpin/enabled``. (Having
a mutable filesystem means pinning is mutable too, but having the
sysctl allows for easy testing on systems with a mutable filesystem.)
+
+It's also possible to exclude specific file types from LoadPin using kernel
+command line option "``loadpin.exclude``". By default, all files are
+included, but they can be excluded using kernel command line option such
+as "``loadpin.exclude=kernel-module,kexec-image``". This allows to use
+different mechanisms such as ``CONFIG_MODULE_SIG`` and
+``CONFIG_KEXEC_VERIFY_SIG`` to verify kernel module and kernel image while
+still use LoadPin to protect the integrity of other files kernel loads. The
+full list of valid file types can be found in ``kernel_read_file_str``
+defined in ``include/linux/fs.h``.
diff --git a/security/loadpin/loadpin.c b/security/loadpin/loadpin.c
index 055fb0a64169..8ee0c58fea40 100644
--- a/security/loadpin/loadpin.c
+++ b/security/loadpin/loadpin.c
@@ -45,6 +45,8 @@ static void report_load(const char *origin, struct file *file, char *operation)
}
static int enforce = IS_ENABLED(CONFIG_SECURITY_LOADPIN_ENFORCE);
+static char *exclude_read_files[READING_MAX_ID];
+static int ignore_read_file_id[READING_MAX_ID];
static struct super_block *pinned_root;
static DEFINE_SPINLOCK(pinned_root_spinlock);
@@ -129,6 +131,12 @@ static int loadpin_read_file(struct file *file, enum kernel_read_file_id id)
struct super_block *load_root;
const char *origin = kernel_read_file_id_str(id);
+ /* If the file id is excluded, ignore the pinning. */
+ if ((unsigned int)id < READING_MAX_ID && ignore_read_file_id[id]) {
+ report_load(origin, file, "pinning-excluded");
+ return 0;
+ }
+
/* This handles the older init_module API that has a NULL file. */
if (!file) {
if (!enforce) {
@@ -187,10 +195,37 @@ static struct security_hook_list loadpin_hooks[] __lsm_ro_after_init = {
LSM_HOOK_INIT(kernel_load_data, loadpin_load_data),
};
+static void parse_exclude(void)
+{
+ int i, j;
+ char *cur;
+
+ for (i = 0; i < ARRAY_SIZE(exclude_read_files); i++) {
+ cur = exclude_read_files[i];
+ if (!cur)
+ break;
+ if (*cur == '\0')
+ continue;
+
+ for (j = 0; j < ARRAY_SIZE(kernel_read_file_str); j++) {
+ if (strcmp(cur, kernel_read_file_str[j]) == 0) {
+ pr_info("excluding: %s\n",
+ kernel_read_file_str[j]);
+ ignore_read_file_id[j] = 1;
+ /*
+ * Can not break, because one read_file_str
+ * may map to more than on read_file_id.
+ */
+ }
+ }
+ }
+}
+
static int __init loadpin_init(void)
{
pr_info("ready to pin (currently %senforcing)\n",
enforce ? "" : "not ");
+ parse_exclude();
security_add_hooks(loadpin_hooks, ARRAY_SIZE(loadpin_hooks), "loadpin");
return 0;
}
@@ -203,3 +238,5 @@ DEFINE_LSM(loadpin) = {
/* Should not be mutable after boot, so not listed in sysfs (perm == 0). */
module_param(enforce, int, 0);
MODULE_PARM_DESC(enforce, "Enforce module/firmware pinning");
+module_param_array_named(exclude, exclude_read_files, charp, NULL, 0);
+MODULE_PARM_DESC(exclude, "Exclude pinning specific read file types");
--
2.22.0.rc1.257.g3120a18244-goog
^ permalink raw reply related
* Re: [PATCH] scripts/sphinx-pre-install: make activate hint smarter
From: Jonathan Corbet @ 2019-05-29 22:35 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel
In-Reply-To: <582e6b7351f619d92035db8cfe24ecec5d95a489.1558610037.git.mchehab+samsung@kernel.org>
On Thu, 23 May 2019 08:14:02 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> It is possible that multiple Sphinx virtualenvs are installed
> on a given kernel tree. Change the logic to get the latest
> version of those, as this is probably what the user wants.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Applied, thanks.
jon
^ permalink raw reply
* [PATCH 1/2] hwmon: pmbus: Add Infineon PXE1610 VR driver
From: Vijay Khemka @ 2019-05-29 22:35 UTC (permalink / raw)
To: Jean Delvare, Guenter Roeck, Jonathan Corbet, linux-hwmon,
linux-doc, linux-kernel
Cc: vijaykhemka, joel, linux-aspeed, sdasari, Greg Kroah-Hartman
Added pmbus driver for the new device Infineon pxe1610
voltage regulator. It also supports similar family device
PXE1110 and PXM1310.
Signed-off-by: Vijay Khemka <vijaykhemka@fb.com>
---
drivers/hwmon/pmbus/Kconfig | 9 +++
drivers/hwmon/pmbus/Makefile | 1 +
drivers/hwmon/pmbus/pxe1610.c | 119 ++++++++++++++++++++++++++++++++++
3 files changed, 129 insertions(+)
create mode 100644 drivers/hwmon/pmbus/pxe1610.c
diff --git a/drivers/hwmon/pmbus/Kconfig b/drivers/hwmon/pmbus/Kconfig
index 30751eb9550a..338ef9b5a395 100644
--- a/drivers/hwmon/pmbus/Kconfig
+++ b/drivers/hwmon/pmbus/Kconfig
@@ -154,6 +154,15 @@ config SENSORS_MAX8688
This driver can also be built as a module. If so, the module will
be called max8688.
+config SENSORS_PXE1610
+ tristate "Infineon PXE1610"
+ help
+ If you say yes here you get hardware monitoring support for Infineon
+ PXE1610.
+
+ This driver can also be built as a module. If so, the module will
+ be called pxe1610.
+
config SENSORS_TPS40422
tristate "TI TPS40422"
help
diff --git a/drivers/hwmon/pmbus/Makefile b/drivers/hwmon/pmbus/Makefile
index 2219b9300316..b0fbd017a91a 100644
--- a/drivers/hwmon/pmbus/Makefile
+++ b/drivers/hwmon/pmbus/Makefile
@@ -18,6 +18,7 @@ obj-$(CONFIG_SENSORS_MAX20751) += max20751.o
obj-$(CONFIG_SENSORS_MAX31785) += max31785.o
obj-$(CONFIG_SENSORS_MAX34440) += max34440.o
obj-$(CONFIG_SENSORS_MAX8688) += max8688.o
+obj-$(CONFIG_SENSORS_PXE1610) += pxe1610.o
obj-$(CONFIG_SENSORS_TPS40422) += tps40422.o
obj-$(CONFIG_SENSORS_TPS53679) += tps53679.o
obj-$(CONFIG_SENSORS_UCD9000) += ucd9000.o
diff --git a/drivers/hwmon/pmbus/pxe1610.c b/drivers/hwmon/pmbus/pxe1610.c
new file mode 100644
index 000000000000..01e267944df5
--- /dev/null
+++ b/drivers/hwmon/pmbus/pxe1610.c
@@ -0,0 +1,119 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Hardware monitoring driver for Infineon PXE1610
+ *
+ * Copyright (c) 2019 Facebook Inc
+ *
+ */
+
+#include <linux/err.h>
+#include <linux/i2c.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include "pmbus.h"
+
+/*
+ * Identify chip parameters.
+ */
+static int pxe1610_identify(struct i2c_client *client,
+ struct pmbus_driver_info *info)
+{
+ if (pmbus_check_byte_register(client, 0, PMBUS_VOUT_MODE)) {
+ int vout_mode;
+
+ vout_mode = pmbus_read_byte_data(client, 0, PMBUS_VOUT_MODE);
+ switch (vout_mode & 0x1f) {
+ case 1:
+ info->vrm_version = vr12;
+ break;
+ case 2:
+ info->vrm_version = vr13;
+ break;
+ default:
+ return -ENODEV;
+ }
+ }
+ return 0;
+}
+
+static int pxe1610_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct pmbus_driver_info *info;
+ u8 buf[I2C_SMBUS_BLOCK_MAX];
+ int ret;
+
+ if (!i2c_check_functionality(client->adapter,
+ I2C_FUNC_SMBUS_READ_BYTE_DATA
+ | I2C_FUNC_SMBUS_READ_WORD_DATA
+ | I2C_FUNC_SMBUS_READ_BLOCK_DATA))
+ return -ENODEV;
+
+ /* By default this device doesn't boot to page 0, so set page 0
+ * to access all pmbus registers.
+ */
+ i2c_smbus_write_byte_data(client, 0, 0);
+
+ /* Read Manufacturer id */
+ ret = i2c_smbus_read_block_data(client, PMBUS_MFR_ID, buf);
+ if (ret < 0) {
+ dev_err(&client->dev, "Failed to read PMBUS_MFR_ID\n");
+ return ret;
+ }
+ if (ret != 2 || strncmp(buf, "XP", strlen("XP"))) {
+ dev_err(&client->dev, "MFR_ID unrecognised\n");
+ return -ENODEV;
+ }
+
+ info = devm_kzalloc(&client->dev, sizeof(struct pmbus_driver_info),
+ GFP_KERNEL);
+ if (!info)
+ return -ENOMEM;
+
+ info->format[PSC_VOLTAGE_IN] = linear;
+ info->format[PSC_VOLTAGE_OUT] = vid;
+ info->format[PSC_CURRENT_IN] = linear;
+ info->format[PSC_CURRENT_OUT] = linear;
+ info->format[PSC_POWER] = linear;
+ info->format[PSC_TEMPERATURE] = linear;
+
+ info->func[0] = PMBUS_HAVE_VIN
+ | PMBUS_HAVE_VOUT | PMBUS_HAVE_IIN
+ | PMBUS_HAVE_IOUT | PMBUS_HAVE_PIN
+ | PMBUS_HAVE_POUT | PMBUS_HAVE_TEMP
+ | PMBUS_HAVE_STATUS_VOUT | PMBUS_HAVE_STATUS_IOUT
+ | PMBUS_HAVE_STATUS_INPUT | PMBUS_HAVE_STATUS_TEMP;
+ info->func[1] = info->func[0];
+ info->func[2] = info->func[0];
+
+ info->pages = id->driver_data;
+ info->identify = pxe1610_identify;
+
+ return pmbus_do_probe(client, id, info);
+}
+
+static const struct i2c_device_id pxe1610_id[] = {
+ {"pxe1610", 3},
+ {"pxe1110", 3},
+ {"pxm1310", 3},
+ {}
+};
+
+MODULE_DEVICE_TABLE(i2c, pxe1610_id);
+
+/* This is the driver that will be inserted */
+static struct i2c_driver pxe1610_driver = {
+ .driver = {
+ .name = "pxe1610",
+ },
+ .probe = pxe1610_probe,
+ .remove = pmbus_do_remove,
+ .id_table = pxe1610_id,
+};
+
+module_i2c_driver(pxe1610_driver);
+
+MODULE_AUTHOR("Vijay Khemka <vijaykhemka@fb.com>");
+MODULE_DESCRIPTION("PMBus driver for Infineon PXE1610, PXE1110 and PXM1310");
+MODULE_LICENSE("GPL");
--
2.17.1
^ permalink raw reply related
* [PATCH 2/2] Docs: hwmon: pmbus: Add PXE1610 driver
From: Vijay Khemka @ 2019-05-29 22:35 UTC (permalink / raw)
To: Jean Delvare, Guenter Roeck, Jonathan Corbet, linux-hwmon,
linux-doc, linux-kernel
Cc: vijaykhemka, joel, linux-aspeed, sdasari, Greg Kroah-Hartman
In-Reply-To: <20190529223511.4059120-1-vijaykhemka@fb.com>
Added support for Infenion PXE1610 driver
Signed-off-by: Vijay Khemka <vijaykhemka@fb.com>
---
Documentation/hwmon/pxe1610 | 84 +++++++++++++++++++++++++++++++++++++
1 file changed, 84 insertions(+)
create mode 100644 Documentation/hwmon/pxe1610
diff --git a/Documentation/hwmon/pxe1610 b/Documentation/hwmon/pxe1610
new file mode 100644
index 000000000000..b5c83edf027a
--- /dev/null
+++ b/Documentation/hwmon/pxe1610
@@ -0,0 +1,84 @@
+Kernel driver pxe1610
+=====================
+
+Supported chips:
+ * Infinion PXE1610
+ Prefix: 'pxe1610'
+ Addresses scanned: -
+ Datasheet: Datasheet is not publicly available.
+
+ * Infinion PXE1110
+ Prefix: 'pxe1110'
+ Addresses scanned: -
+ Datasheet: Datasheet is not publicly available.
+
+ * Infinion PXM1310
+ Prefix: 'pxm1310'
+ Addresses scanned: -
+ Datasheet: Datasheet is not publicly available.
+
+Author: Vijay Khemka <vijaykhemka@fb.com>
+
+
+Description
+-----------
+
+PXE1610 is a Multi-rail/Multiphase Digital Controllers and
+it is compliant to Intel VR13 DC-DC converter specifications.
+
+
+Usage Notes
+-----------
+
+This driver can be enabled with kernel config CONFIG_SENSORS_PXE1610
+set to 'y' or 'm'(for module).
+
+This driver does not probe for PMBus devices. You will have
+to instantiate devices explicitly.
+
+Example: the following commands will load the driver for an PXE1610
+at address 0x70 on I2C bus #4:
+
+# modprobe pxe1610
+# echo pxe1610 0x70 > /sys/bus/i2c/devices/i2c-4/new_device
+
+It can also be instantiated by declaring in device tree if it is
+built as a kernel not as a module.
+
+
+Sysfs attributes
+----------------
+
+curr1_label "iin"
+curr1_input Measured input current
+curr1_alarm Current high alarm
+
+curr[2-4]_label "iout[1-3]"
+curr[2-4]_input Measured output current
+curr[2-4]_crit Critical maximum current
+curr[2-4]_crit_alarm Current critical high alarm
+
+in1_label "vin"
+in1_input Measured input voltage
+in1_crit Critical maximum input voltage
+in1_crit_alarm Input voltage critical high alarm
+
+in[2-4]_label "vout[1-3]"
+in[2-4]_input Measured output voltage
+in[2-4]_lcrit Critical minimum output voltage
+in[2-4]_lcrit_alarm Output voltage critical low alarm
+in[2-4]_crit Critical maximum output voltage
+in[2-4]_crit_alarm Output voltage critical high alarm
+
+power1_label "pin"
+power1_input Measured input power
+power1_alarm Input power high alarm
+
+power[2-4]_label "pout[1-3]"
+power[2-4]_input Measured output power
+
+temp[1-3]_input Measured temperature
+temp[1-3]_crit Critical high temperature
+temp[1-3]_crit_alarm Chip temperature critical high alarm
+temp[1-3]_max Maximum temperature
+temp[1-3]_max_alarm Chip temperature high alarm
--
2.17.1
^ permalink raw reply related
* Re: [PATCH v6 00/12] Include linux PCI docs into Sphinx TOC tree
From: Jonathan Corbet @ 2019-05-29 22:35 UTC (permalink / raw)
To: Changbin Du; +Cc: bhelgaas, linux-pci, linux-doc, linux-kernel, mchehab+samsung
In-Reply-To: <20190520061014.qtq6tc366pnnqcio@mail.google.com>
On Mon, 20 May 2019 06:10:15 +0000
Changbin Du <changbin.du@gmail.com> wrote:
> Bjorn and Jonathan,
> Could we consider to merge this serias now? Thanks.
Somewhat belatedly, but I think we could. Bjorn, do you have a preference
for which tree this goes through? I don't remember if we'd come to an
agreement on that or not, sorry...
Thanks,
jon
^ permalink raw reply
* Re: [PATCH v4 0/9] docs: Convert VFS doc to RST
From: Jonathan Corbet @ 2019-05-29 22:30 UTC (permalink / raw)
To: Tobin C. Harding
Cc: Al Viro, Mauro Carvalho Chehab, Neil Brown, Randy Dunlap,
linux-doc, linux-fsdevel, linux-kernel
In-Reply-To: <20190515002913.12586-1-tobin@kernel.org>
On Wed, 15 May 2019 10:29:04 +1000
"Tobin C. Harding" <tobin@kernel.org> wrote:
> Here is an updated version of the VFS doc conversion. This series in no
> way represents a final point for the VFS documentation rather it is a
> small step towards getting VFS docs updated. This series does not
> update the content of vfs.txt, only does formatting.
I've finally gotten to this, sorry for taking so long. Applying it to
docs-next turned out to be a bit of a chore; there have been intervening
changes to vfs.txt that we didn't want to lose. But I did it.
Unfortunately, there's still a remaining issue. You did a lot of list
conversions like this:
> - struct file_system_type *fs_type: describes the filesystem, partly initialized
> +``struct file_system_type *fs_type``: describes the filesystem, partly initialized
> by the specific filesystem code
but that does not render the way you would like, trust me. You really
want to use the list format, something like:
``struct file_system_type *fs_type``
describes the filesystem, partly initialized by the specific
filesystem code
There are, unfortunately, a lot of these to fix... I bet it could be done
with an elisp function, but I don't have time to beat my head against that
wall right now.
Any chance you would have time to send me a followup patch fixing these
up? I'll keep my branch with this set for now so there's no need to
rebase those.
Thanks,
jon
^ permalink raw reply
* [PATCH] docs cgroups: add another example size for hugetlb
From: Odin Ugedal @ 2019-05-29 22:24 UTC (permalink / raw)
To: odin
Cc: Tejun Heo, Li Zefan, Johannes Weiner, Jonathan Corbet,
open list:CONTROL GROUP (CGROUP), open list:DOCUMENTATION,
open list
Add another example to clarify that HugePages smaller than 1MB will
be displayed using "KB", with an uppercased K (eg. 20KB), and not the
normal SI prefix kilo (small k).
Because of a misunderstanding/copy-paste error inside runc
(see https://github.com/opencontainers/runc/pull/2065), it tried
accessing the cgroup control file of a 64kB HugePage using
"hugetlb.64kB._____" instead of the correct "hugetlb.64KB._____".
Adding a new example will make it clear how sizes smaller than 1MB are
handled.
Signed-off-by: Odin Ugedal <odin@ugedal.com>
---
Documentation/cgroup-v1/hugetlb.txt | 22 +++++++++++++---------
1 file changed, 13 insertions(+), 9 deletions(-)
diff --git a/Documentation/cgroup-v1/hugetlb.txt b/Documentation/cgroup-v1/hugetlb.txt
index 106245c3aecc..1260e5369b9b 100644
--- a/Documentation/cgroup-v1/hugetlb.txt
+++ b/Documentation/cgroup-v1/hugetlb.txt
@@ -32,14 +32,18 @@ Brief summary of control files
hugetlb.<hugepagesize>.usage_in_bytes # show current usage for "hugepagesize" hugetlb
hugetlb.<hugepagesize>.failcnt # show the number of allocation failure due to HugeTLB limit
-For a system supporting two hugepage size (16M and 16G) the control
+For a system supporting three hugepage sizes (64k, 32M and 1G), the control
files include:
-hugetlb.16GB.limit_in_bytes
-hugetlb.16GB.max_usage_in_bytes
-hugetlb.16GB.usage_in_bytes
-hugetlb.16GB.failcnt
-hugetlb.16MB.limit_in_bytes
-hugetlb.16MB.max_usage_in_bytes
-hugetlb.16MB.usage_in_bytes
-hugetlb.16MB.failcnt
+hugetlb.1GB.limit_in_bytes
+hugetlb.1GB.max_usage_in_bytes
+hugetlb.1GB.usage_in_bytes
+hugetlb.1GB.failcnt
+hugetlb.64KB.limit_in_bytes
+hugetlb.64KB.max_usage_in_bytes
+hugetlb.64KB.usage_in_bytes
+hugetlb.64KB.failcnt
+hugetlb.32MB.limit_in_bytes
+hugetlb.32MB.max_usage_in_bytes
+hugetlb.32MB.usage_in_bytes
+hugetlb.32MB.failcnt
--
2.21.0
^ permalink raw reply related
* Re: [PATCH] [linux-next] docs: tracing: Fix typos in histogram.rst
From: Jonathan Corbet @ 2019-05-29 21:49 UTC (permalink / raw)
To: Steven Rostedt; +Cc: Masanari Iida, linux-doc, linux-kernel, mingo, rdunlap
In-Reply-To: <20190521085525.2963c10f@gandalf.local.home>
On Tue, 21 May 2019 08:55:25 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:
> On Tue, 21 May 2019 21:30:00 +0900
> Masanari Iida <standby24x7@gmail.com> wrote:
>
> > This patch fixes some spelling typos in histogram.rst
>
> Acked-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
>
> Jon,
>
> Care to take this in your tree?
Done, thanks.
jon
^ permalink raw reply
* Re: [PATCH] docs: cdomain.py: get rid of a warning since version 1.8
From: Jonathan Corbet @ 2019-05-29 21:48 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel
In-Reply-To: <b38a9fdfdcda49b2c6118072afac564e96406800.1558608217.git.mchehab+samsung@kernel.org>
On Thu, 23 May 2019 07:43:43 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> There's a new warning about a deprecation function. Add a
> logic at cdomain.py to avoid that.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Applied, thanks.
jon
^ permalink raw reply
* Re: [GIT PULL] Fixes for sphinx 2.0 docs build failures
From: pr-tracker-bot @ 2019-05-29 21:45 UTC (permalink / raw)
To: Jonathan Corbet; +Cc: Linus Torvalds, LKML, linux-doc
In-Reply-To: <20190529095408.1429a5ab@lwn.net>
The pull request you sent on Wed, 29 May 2019 09:54:08 -0600:
> git://git.lwn.net/linux.git tags/docs-5.2-fixes2
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/bec7550cca106c3ccc061e3e625516af63054fe4
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker
^ permalink raw reply
* Re: [PATCH v3 1/1] sched/fair: Fix low cpu usage with high throttling by removing expiration of cpu-local slices
From: bsegall @ 2019-05-29 21:05 UTC (permalink / raw)
To: Dave Chiluk
Cc: Phil Auld, Peter Oskolkov, Peter Zijlstra, Ingo Molnar, cgroups,
linux-kernel, Brendan Gregg, Kyle Anderson, Gabriel Munos,
John Hammond, Cong Wang, Jonathan Corbet, linux-doc, pjt
In-Reply-To: <1559156926-31336-2-git-send-email-chiluk+linux@indeed.com>
Dave Chiluk <chiluk+linux@indeed.com> writes:
> It has been observed, that highly-threaded, non-cpu-bound applications
> running under cpu.cfs_quota_us constraints can hit a high percentage of
> periods throttled while simultaneously not consuming the allocated
> amount of quota. This use case is typical of user-interactive non-cpu
> bound applications, such as those running in kubernetes or mesos when
> run on multiple cpu cores.
>
> This has been root caused to threads being allocated per cpu bandwidth
> slices, and then not fully using that slice within the period. At which
> point the slice and quota expires. This expiration of unused slice
> results in applications not being able to utilize the quota for which
> they are allocated.
>
> The expiration of per-cpu slices was recently fixed by
> 'commit 512ac999d275 ("sched/fair: Fix bandwidth timer clock drift
> condition")'. Prior to that it appears that this has been broken since
> at least 'commit 51f2176d74ac ("sched/fair: Fix unlocked reads of some
> cfs_b->quota/period")' which was introduced in v3.16-rc1 in 2014. That
> added the following conditional which resulted in slices never being
> expired.
Yeah, having run the test, stranding only 1 ms per cpu rather than 5
doesn't help if you only have 10 ms of quota and even 10 threads/cpus.
The slack timer isn't important in this test, though I think it probably
should be changed.
Decreasing min_cfs_rq_runtime helps, but would mean that we have to pull
quota more often / always. The worst case here I think is where you
run/sleep for ~1ns, so you wind up taking the lock twice every
min_cfs_rq_runtime: once for assign and once to return all but min,
which you then use up doing short run/sleep. I suppose that determines
how much we care about this overhead at all.
Removing expiration means that in the worst case period and quota can be
effectively twice what the user specified, but only on very particular
workloads.
I think we should at least think about instead lowering
min_cfs_rq_runtime to some smaller value.
^ permalink raw reply
* Re: [PATCH v3 1/1] sched/fair: Fix low cpu usage with high throttling by removing expiration of cpu-local slices
From: bsegall @ 2019-05-29 19:50 UTC (permalink / raw)
To: Dave Chiluk
Cc: Phil Auld, Peter Oskolkov, Peter Zijlstra, Ingo Molnar, cgroups,
linux-kernel, Brendan Gregg, Kyle Anderson, Gabriel Munos,
John Hammond, Cong Wang, Jonathan Corbet, linux-doc
In-Reply-To: <1559156926-31336-2-git-send-email-chiluk+linux@indeed.com>
Dave Chiluk <chiluk+linux@indeed.com> writes:
> It has been observed, that highly-threaded, non-cpu-bound applications
> running under cpu.cfs_quota_us constraints can hit a high percentage of
> periods throttled while simultaneously not consuming the allocated
> amount of quota. This use case is typical of user-interactive non-cpu
> bound applications, such as those running in kubernetes or mesos when
> run on multiple cpu cores.
>
> This has been root caused to threads being allocated per cpu bandwidth
> slices, and then not fully using that slice within the period. At which
> point the slice and quota expires. This expiration of unused slice
> results in applications not being able to utilize the quota for which
> they are allocated.
>
So most of the bandwidth is supposed to be returned, leaving only 1ms.
I'm setting up to try your test now, but there was a bug with the
unthrottle part of this where it could be continuously pushed into the
future. You might try this patch instead, possibly along with lowering
min_cfs_rq_runtime (currently not runtime tunable) if you see that cost
more than the cost of extra locking to acquire runtime.
From: Ben Segall <bsegall@google.com>
Date: Tue, 28 May 2019 12:30:25 -0700
Subject: [PATCH] sched/fair: don't push cfs_bandwith slack timers forward
When a cfs_rq sleeps and returns its quota, we delay for 5ms before waking any
throttled cfs_rqs to coalesce with other cfs_rqs going to sleep, as this has has
to be done outside of the rq lock we hold. The current code waits for 5ms
without any sleeps, instead of waiting for 5ms from the first sleep, which can
delay the unthrottle more than we want. Switch this around.
Change-Id: Ife536bb54af633863de486ba6ec0d20e55ada8c9
Signed-off-by: Ben Segall <bsegall@google.com>
---
kernel/sched/fair.c | 7 +++++++
kernel/sched/sched.h | 1 +
2 files changed, 8 insertions(+)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 8213ff6e365d..2ead252cfa32 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -4729,6 +4729,11 @@ static void start_cfs_slack_bandwidth(struct cfs_bandwidth *cfs_b)
if (runtime_refresh_within(cfs_b, min_left))
return;
+ /* don't push forwards an existing deferred unthrottle */
+ if (cfs_b->slack_started)
+ return;
+ cfs_b->slack_started = true;
+
hrtimer_start(&cfs_b->slack_timer,
ns_to_ktime(cfs_bandwidth_slack_period),
HRTIMER_MODE_REL);
@@ -4782,6 +4787,7 @@ static void do_sched_cfs_slack_timer(struct cfs_bandwidth *cfs_b)
/* confirm we're still not at a refresh boundary */
raw_spin_lock_irqsave(&cfs_b->lock, flags);
+ cfs_b->slack_started = false;
if (cfs_b->distribute_running) {
raw_spin_unlock_irqrestore(&cfs_b->lock, flags);
return;
@@ -4920,6 +4926,7 @@ void init_cfs_bandwidth(struct cfs_bandwidth *cfs_b)
hrtimer_init(&cfs_b->slack_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
cfs_b->slack_timer.function = sched_cfs_slack_timer;
cfs_b->distribute_running = 0;
+ cfs_b->slack_started = false;
}
static void init_cfs_rq_runtime(struct cfs_rq *cfs_rq)
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index efa686eeff26..60219acda94b 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -356,6 +356,7 @@ struct cfs_bandwidth {
u64 throttled_time;
bool distribute_running;
+ bool slack_started;
#endif
};
--
2.22.0.rc1.257.g3120a18244-goog
^ permalink raw reply related
* Re: [PATCH v3 1/1] sched/fair: Fix low cpu usage with high throttling by removing expiration of cpu-local slices
From: Phil Auld @ 2019-05-29 19:28 UTC (permalink / raw)
To: Dave Chiluk
Cc: Ben Segall, Peter Oskolkov, Peter Zijlstra, Ingo Molnar, cgroups,
linux-kernel, Brendan Gregg, Kyle Anderson, Gabriel Munos,
John Hammond, Cong Wang, Jonathan Corbet, linux-doc
In-Reply-To: <1559156926-31336-2-git-send-email-chiluk+linux@indeed.com>
On Wed, May 29, 2019 at 02:08:46PM -0500 Dave Chiluk wrote:
> It has been observed, that highly-threaded, non-cpu-bound applications
> running under cpu.cfs_quota_us constraints can hit a high percentage of
> periods throttled while simultaneously not consuming the allocated
> amount of quota. This use case is typical of user-interactive non-cpu
> bound applications, such as those running in kubernetes or mesos when
> run on multiple cpu cores.
>
> This has been root caused to threads being allocated per cpu bandwidth
> slices, and then not fully using that slice within the period. At which
> point the slice and quota expires. This expiration of unused slice
> results in applications not being able to utilize the quota for which
> they are allocated.
>
> The expiration of per-cpu slices was recently fixed by
> 'commit 512ac999d275 ("sched/fair: Fix bandwidth timer clock drift
> condition")'. Prior to that it appears that this has been broken since
> at least 'commit 51f2176d74ac ("sched/fair: Fix unlocked reads of some
> cfs_b->quota/period")' which was introduced in v3.16-rc1 in 2014. That
> added the following conditional which resulted in slices never being
> expired.
>
> if (cfs_rq->runtime_expires != cfs_b->runtime_expires) {
> /* extend local deadline, drift is bounded above by 2 ticks */
> cfs_rq->runtime_expires += TICK_NSEC;
>
> Because this was broken for nearly 5 years, and has recently been fixed
> and is now being noticed by many users running kubernetes
> (https://github.com/kubernetes/kubernetes/issues/67577) it is my opinion
> that the mechanisms around expiring runtime should be removed
> altogether.
>
> This allows only per-cpu slices to live longer than the period boundary.
> This allows threads on runqueues that do not use much CPU to continue to
> use their remaining slice over a longer period of time than
> cpu.cfs_period_us. However, this helps prevents the above condition of
> hitting throttling while also not fully utilizing your cpu quota.
>
> This theoretically allows a machine to use slightly more than it's
> allotted quota in some periods. This overflow would be bounded by the
> remaining per-cpu slice that was left un-used in the previous period.
> For CPU bound tasks this will change nothing, as they should
> theoretically fully utilize all of their quota and slices in each
> period. For user-interactive tasks as described above this provides a
> much better user/application experience as their cpu utilization will
> more closely match the amount they requested when they hit throttling.
>
> This greatly improves performance of high-thread-count, non-cpu bound
> applications with low cfs_quota_us allocation on high-core-count
> machines. In the case of an artificial testcase, this performance
> discrepancy has been observed to be almost 30x performance improvement,
> while still maintaining correct cpu quota restrictions albeit over
> longer time intervals than cpu.cfs_period_us. That testcase is
> available at https://github.com/indeedeng/fibtest.
>
> Fixes: 512ac999d275 ("sched/fair: Fix bandwidth timer clock drift condition")
> Signed-off-by: Dave Chiluk <chiluk+linux@indeed.com>
> ---
> Documentation/scheduler/sched-bwc.txt | 56 ++++++++++++++++++++++-----
> kernel/sched/fair.c | 71 +++--------------------------------
> kernel/sched/sched.h | 4 --
> 3 files changed, 53 insertions(+), 78 deletions(-)
>
> diff --git a/Documentation/scheduler/sched-bwc.txt b/Documentation/scheduler/sched-bwc.txt
> index f6b1873..260fd65 100644
> --- a/Documentation/scheduler/sched-bwc.txt
> +++ b/Documentation/scheduler/sched-bwc.txt
> @@ -8,15 +8,16 @@ CFS bandwidth control is a CONFIG_FAIR_GROUP_SCHED extension which allows the
> specification of the maximum CPU bandwidth available to a group or hierarchy.
>
> The bandwidth allowed for a group is specified using a quota and period. Within
> -each given "period" (microseconds), a group is allowed to consume only up to
> -"quota" microseconds of CPU time. When the CPU bandwidth consumption of a
> -group exceeds this limit (for that period), the tasks belonging to its
> -hierarchy will be throttled and are not allowed to run again until the next
> -period.
> -
> -A group's unused runtime is globally tracked, being refreshed with quota units
> -above at each period boundary. As threads consume this bandwidth it is
> -transferred to cpu-local "silos" on a demand basis. The amount transferred
> +each given "period" (microseconds), a task group is allocated up to "quota"
> +microseconds of CPU time. That quota is assigned to per cpu run queues in
> +slices as threads in the cgroup become runnable. Once all quota has been
> +assigned any additional requests for quota will result in those threads being
> +throttled. Throttled threads will not be able to run again until the next
> +period when the quota is replenished.
> +
> +A group's unassigned quota is globally tracked, being refreshed back to
> +cfs_quota units at each period boundary. As threads consume this bandwidth it
> +is transferred to cpu-local "silos" on a demand basis. The amount transferred
> within each of these updates is tunable and described as the "slice".
>
> Management
> @@ -90,6 +91,43 @@ There are two ways in which a group may become throttled:
> In case b) above, even though the child may have runtime remaining it will not
> be allowed to until the parent's runtime is refreshed.
>
> +Real-world behavior of slice non-expiration
> +-------------------------------------------
> +The fact that cpu-local slices do not expire results in some interesting corner
> +cases that should be understood.
> +
> +For cgroup cpu constrained applications that are cpu limited this is a
> +relatively moot point because they will naturally consume the entirety of their
> +quota as well as the entirety of each cpu-local slice in each period. As a
> +result it is expected that nr_periods roughly equal nr_throttled, and that
> +cpuacct.usage will increase roughly equal to cfs_quota_us in each period.
> +
> +However in a worst-case scenario, highly-threaded, interactive/non-cpu bound
> +applications this non-expiration nuance allows applications to briefly burst
> +past their quota limits by the amount of unused slice on each cpu that the task
> +group is running on. This slight burst requires that quota had been assigned
> +and then not fully used in previous periods. This burst amount will not be
> +transferred between cores. As a result, this mechanism still strictly limits
> +the task group to quota average usage, albeit over a longer time window than
> +period. This provides better more predictable user experience for highly
> +threaded applications with small quota limits on high core count machines. It
> +also eliminates the propensity to throttle these applications while
> +simultanously using less than quota amounts of cpu. Another way to say this,
> +is that by allowing the unused portion of a slice to remain valid across
> +periods we have decreased the possibility of wasting quota on cpu-local silos
> +that don't need a full slice's amount of cpu time.
> +
> +The interaction between cpu-bound and non-cpu-bound-interactive applications
> +should also be considered, especially when single core usage hits 100%. If you
> +gave each of these applications half of a cpu-core and they both got scheduled
> +on the same CPU it is theoretically possible that the non-cpu bound application
> +will use up to sched_cfs_bandwidth_slice_us additional quota in some periods,
> +thereby preventing the cpu-bound application from fully using it's quota by
"its quota"
> +that same amount. In these instances it will be up to the CFS algorithm (see
> +sched-design-CFS.txt) to decide which application is chosen to run, as they
> +will both be runnable and have remaining quota. This runtime discrepancy will
> +should made up in the following periods when the interactive application idles.
> +
"discrepancy will be made" or "descrepancy should be made" but not both :)
Otherwise, fwiw,
Acked-by: Phil Auld <pauld@redhat.com>
Cheers,
Phil
--
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox