[PATCH v2 0/8] Add a script to check for Sphinx install requirements

[PATCH v2 0/8] Add a script to check for Sphinx install requirements

[Mauro Carvalho Chehab]

Sphinx installation is not trivial, as not all versions are supported,
and it requires a lot of stuff for math, images and PDF/LaTeX output
to work.

So, add a script that checks if everything is fine, providing
distro-specific hints about what's needed for it to work.

I posted already RFC patches for the first patch in this series along
this weekend at linux-doc ML. The current version of the script
will provide hints for some popular distributions. I tested myself
on a couple of release-based and rolling popular distributions:

            - Fedora 25 and 26;
            - Ubuntu 17.04;
            - OpenSuse Tumbleweed;
            - Arch Linux;
            - Gentoo.

I tested compilation with both htmldocs and pdfdocs targets[1].
Yet, tests (and fix patches) are welcomed :-)

Those patches are at:

   https://git.linuxtv.org/mchehab/experimental.git/log/?h=sphinx_install_v2



PS.:

1) The first patch in this series is actually independent: it
   is just a cleanup from DocBook removal patches.

2) Debian 17.04 version of ImageMagick seems capped:
   it can't read SVG texts with things like:
	font-family="'Liberation Sans', Sans"
   It also doesn't handle other random SVG tags.
   So, I had to rewrite several media SVG files to make
   them more compliant with ImageMagick.

   So far, I can't submit those patches via e-mail, as some
   lines there are bigger than 998 characters, with violates
   VGER rules for maximum number of columns. I'll try
   to break the big lines there via some scripting.

-

v2: be sure that GraphViz and ImageMagick will generate PDF
    outputs on all platforms but OpenSuse (where GraphViz
    is built without PDf support).


Mauro Carvalho Chehab (8):
  docs: Makefile: remove no-ops targets
  scripts/sphinx-pre-install: add a script to check Sphinx install
  sphinx-pre-install: detect an existing virtualenv
  sphinx-pre-install: use a requirements file
  sphinx-pre-install: check for the need of graphviz-gd
  sphinx-pre-install: add dependencies for ImageMagick to work with svg
  sphinx-pre-install: fix USE needs for GraphViz and ImageMagick
  sphinx.rst: document scripts/sphinx-pre-install script

 Documentation/Makefile                 |  10 -
 Documentation/doc-guide/sphinx.rst     |  48 ++-
 Documentation/sphinx/requirements.txt  |   3 +
 Documentation/translations/zh_CN/HOWTO |   2 -
 Makefile                               |   2 +-
 scripts/sphinx-pre-install             | 544 +++++++++++++++++++++++++++++++++
 6 files changed, 592 insertions(+), 17 deletions(-)
 create mode 100644 Documentation/sphinx/requirements.txt
 create mode 100755 scripts/sphinx-pre-install

-- 
2.13.3

[PATCH v2 1/8] docs: Makefile: remove no-ops targets

[Mauro Carvalho Chehab]

After removal of DocBook, those targets are bogus.

Reported-by: Jim Davis <jim.epost@gmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 Documentation/Makefile                 | 10 ----------
 Documentation/translations/zh_CN/HOWTO |  2 --
 Makefile                               |  2 +-
 3 files changed, 1 insertion(+), 13 deletions(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index a42320385df3..d75c00e3aadb 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -95,16 +95,6 @@ endif # HAVE_SPHINX
 # The following targets are independent of HAVE_SPHINX, and the rules should
 # work or silently pass without Sphinx.
 
-# no-ops for the Sphinx toolchain
-sgmldocs:
-	@:
-psdocs:
-	@:
-mandocs:
-	@:
-installmandocs:
-	@:
-
 cleandocs:
 	$(Q)rm -rf $(BUILDDIR)
 	$(Q)$(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=Documentation/media clean
diff --git a/Documentation/translations/zh_CN/HOWTO b/Documentation/translations/zh_CN/HOWTO
index 11be075ba5fa..5f6d09edc9ac 100644
--- a/Documentation/translations/zh_CN/HOWTO
+++ b/Documentation/translations/zh_CN/HOWTO
@@ -149,9 +149,7 @@ Linux内核代码中包含有大量的文档。这些文档对于学习如何与
 核源码的主目录中使用以下不同命令将会分别生成PDF、Postscript、HTML和手册
 页等不同格式的文档：
     make pdfdocs
-    make psdocs
     make htmldocs
-    make mandocs
 
 
 如何成为内核开发者
diff --git a/Makefile b/Makefile
index b4fb9a1d1594..8b7237a0d66a 100644
--- a/Makefile
+++ b/Makefile
@@ -1467,7 +1467,7 @@ $(help-board-dirs): help-%:
 
 # Documentation targets
 # ---------------------------------------------------------------------------
-DOC_TARGETS := xmldocs sgmldocs psdocs latexdocs pdfdocs htmldocs mandocs installmandocs epubdocs cleandocs linkcheckdocs
+DOC_TARGETS := xmldocs latexdocs pdfdocs htmldocs epubdocs cleandocs linkcheckdocs
 PHONY += $(DOC_TARGETS)
 $(DOC_TARGETS): scripts_basic FORCE
 	$(Q)$(MAKE) $(build)=Documentation $@
-- 
2.13.3

[PATCH v2 2/8] scripts/sphinx-pre-install: add a script to check Sphinx install

[Mauro Carvalho Chehab]

Solving Sphinx dependencies can be painful. Add a script to
check if everything is ok.

Tested on:
	- Fedora 25 and 26;
	- Ubuntu 17.04;
	- OpenSuse Tumbleweed;
	- Arch Linux;
	- Gentoo.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 scripts/sphinx-pre-install | 517 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 517 insertions(+)
 create mode 100755 scripts/sphinx-pre-install

diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
new file mode 100755
index 000000000000..5fe3b4a86174
--- /dev/null
+++ b/scripts/sphinx-pre-install
@@ -0,0 +1,517 @@
+#!/usr/bin/perl
+use strict;
+
+# Copyright (c) 2017 Mauro Carvalho Chehab <mchehab@kernel.org>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+#
+# Static vars
+#
+
+my %missing;
+my $system_release;
+my $need = 0;
+my $optional = 0;
+my $need_symlink = 0;
+my $need_sphinx = 0;
+my $install = "";
+
+#
+# Command line arguments
+#
+
+my $pdf = 1;
+my $virtualenv = 1;
+
+#
+# List of required texlive packages on Fedora and OpenSuse
+#
+
+my %texlive = (
+	'adjustbox.sty'      => 'texlive-adjustbox',
+	'amsfonts.sty'       => 'texlive-amsfonts',
+	'amsmath.sty'        => 'texlive-amsmath',
+	'amssymb.sty'        => 'texlive-amsfonts',
+	'amsthm.sty'         => 'texlive-amscls',
+	'anyfontsize.sty'    => 'texlive-anyfontsize',
+	'atbegshi.sty'       => 'texlive-oberdiek',
+	'bm.sty'             => 'texlive-tools',
+	'capt-of.sty'        => 'texlive-capt-of',
+	'cmap.sty'           => 'texlive-cmap',
+	'ecrm1000.tfm'       => 'texlive-ec',
+	'eqparbox.sty'       => 'texlive-eqparbox',
+	'eu1enc.def'         => 'texlive-euenc',
+	'fancybox.sty'       => 'texlive-fancybox',
+	'fancyvrb.sty'       => 'texlive-fancyvrb',
+	'float.sty'          => 'texlive-float',
+	'fncychap.sty'       => 'texlive-fncychap',
+	'footnote.sty'       => 'texlive-mdwtools',
+	'framed.sty'         => 'texlive-framed',
+	'luatex85.sty'       => 'texlive-luatex85',
+	'multirow.sty'       => 'texlive-multirow',
+	'needspace.sty'      => 'texlive-needspace',
+	'palatino.sty'       => 'texlive-psnfss',
+	'parskip.sty'        => 'texlive-parskip',
+	'polyglossia.sty'    => 'texlive-polyglossia',
+	'tabulary.sty'       => 'texlive-tabulary',
+	'threeparttable.sty' => 'texlive-threeparttable',
+	'titlesec.sty'       => 'texlive-titlesec',
+	'ucs.sty'            => 'texlive-ucs',
+	'upquote.sty'        => 'texlive-upquote',
+	'wrapfig.sty'        => 'texlive-wrapfig',
+);
+
+#
+# Subroutines that checks if a feature exists
+#
+
+sub check_missing(%)
+{
+	my %map = %{$_[0]};
+
+	foreach my $prog (sort keys %missing) {
+		my $is_optional = $missing{$prog};
+
+		if ($is_optional) {
+			print "Warning: better to also install \"$prog\".\n";
+		} else {
+			print "ERROR: please install \"$prog\", otherwise, build won't work.\n";
+		}
+		if (defined($map{$prog})) {
+			$install .= " " . $map{$prog};
+		} else {
+			$install .= " " . $prog;
+		}
+	}
+
+	$install =~ s/^\s//;
+}
+
+sub add_package($$)
+{
+	my $package = shift;
+	my $is_optional = shift;
+
+	$missing{$package} = $is_optional;
+	if ($is_optional) {
+		$optional++;
+	} else {
+		$need++;
+	}
+}
+
+sub check_missing_file($$$)
+{
+	my $file = shift;
+	my $package = shift;
+	my $is_optional = shift;
+
+	return if(-e $file);
+
+	add_package($package, $is_optional);
+}
+
+sub findprog($)
+{
+	foreach(split(/:/, $ENV{PATH})) {
+		return "$_/$_[0]" if(-x "$_/$_[0]");
+	}
+}
+
+sub check_program($$)
+{
+	my $prog = shift;
+	my $is_optional = shift;
+
+	return if findprog($prog);
+
+	add_package($prog, $is_optional);
+}
+
+sub check_perl_module($$)
+{
+	my $prog = shift;
+	my $is_optional = shift;
+
+	my $err = system("perl -M$prog -e 1 2>/dev/null /dev/null");
+	return if ($err == 0);
+
+	add_package($prog, $is_optional);
+}
+
+sub check_python_module($$)
+{
+	my $prog = shift;
+	my $is_optional = shift;
+
+	my $err = system("python3 -c 'import $prog' 2>/dev/null /dev/null");
+	return if ($err == 0);
+	my $err = system("python -c 'import $prog' 2>/dev/null /dev/null");
+	return if ($err == 0);
+
+	add_package($prog, $is_optional);
+}
+
+sub check_rpm_missing($$)
+{
+	my @pkgs = @{$_[0]};
+	my $is_optional = $_[1];
+
+	foreach my $prog(@pkgs) {
+		my $err = system("rpm -q '$prog' 2>/dev/null >/dev/null");
+		add_package($prog, $is_optional) if ($err);
+	}
+}
+
+sub check_pacman_missing($$)
+{
+	my @pkgs = @{$_[0]};
+	my $is_optional = $_[1];
+
+	foreach my $prog(@pkgs) {
+		my $err = system("pacman -Q '$prog' 2>/dev/null >/dev/null");
+		add_package($prog, $is_optional) if ($err);
+	}
+}
+
+sub check_missing_tex($)
+{
+	my $is_optional = shift;
+	my $kpsewhich = findprog("kpsewhich");
+
+	foreach my $prog(keys %texlive) {
+		my $package = $texlive{$prog};
+		if (!$kpsewhich) {
+			add_package($package, $is_optional);
+			next;
+		}
+		my $file = qx($kpsewhich $prog);
+		add_package($package, $is_optional) if ($file =~ /^\s*$/);
+	}
+}
+
+sub check_sphinx()
+{
+	return if findprog("sphinx-build");
+
+	if (findprog("sphinx-build-3")) {
+		$need_symlink = 1;
+		return;
+	}
+
+	if ($virtualenv) {
+		check_program("virtualenv", 0) if (!findprog("virtualenv-3"));
+		check_program("pip", 0) if (!findprog("pip3"));
+		$need_sphinx = 1;
+	} else {
+		add_package("python-sphinx", 0);
+	}
+}
+
+#
+# Ancillary subroutines
+#
+
+sub catcheck($)
+{
+  my $res = "";
+  $res = qx(cat $_[0]) if (-r $_[0]);
+  return $res;
+}
+
+sub which($)
+{
+	my $file = shift;
+	my @path = split ":", $ENV{PATH};
+
+	foreach my $dir(@path) {
+		my $name = $dir.'/'.$file;
+		return $name if (-x $name );
+	}
+	return undef;
+}
+
+#
+# Subroutines that check distro-specific hints
+#
+
+sub give_debian_hints()
+{
+	my %map = (
+		"python-sphinx"		=> "python3-sphinx",
+		"sphinx_rtd_theme"	=> "python3-sphinx-rtd-theme",
+		"virtualenv"		=> "virtualenv",
+		"pip"			=> "python3-pip",
+		"dot"			=> "graphviz",
+		"convert"		=> "imagemagick",
+		"Pod::Usage"		=> "perl-modules",
+		"xelatex"		=> "texlive-xetex",
+	);
+
+	if ($pdf) {
+		check_missing_file("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
+				   "fonts-dejavu", 1);
+	}
+
+	check_program("dvipng", 1) if ($pdf);
+	check_missing(\%map);
+
+	return if (!$need && !$optional);
+	printf("You should run:\n\n\tsudo apt-get install $install\n");
+}
+
+sub give_redhat_hints()
+{
+	my %map = (
+		"python-sphinx"		=> "python3-sphinx",
+		"sphinx_rtd_theme"	=> "python3-sphinx_rtd_theme",
+		"virtualenv"		=> "python3-virtualenv",
+		"pip"			=> "python3-pip",
+		"dot"			=> "graphviz",
+		"convert"		=> "ImageMagick",
+		"Pod::Usage"		=> "perl-Pod-Usage",
+		"xelatex"		=> "texlive-xetex-bin",
+	);
+
+	my @fedora_tex_pkgs = (
+		"texlive-collection-fontsrecommended",
+		"texlive-collection-latex",
+		"dejavu-sans-fonts",
+		"dejavu-serif-fonts",
+		"dejavu-sans-mono-fonts",
+	);
+
+	check_rpm_missing(\@fedora_tex_pkgs, 1) if ($pdf);
+	check_missing_tex(1) if ($pdf);
+	check_missing(\%map);
+
+	return if (!$need && !$optional);
+	printf("You should run:\n\n\tsudo dnf install -y $install\n");
+}
+
+sub give_opensuse_hints()
+{
+	my %map = (
+		"python-sphinx"		=> "python3-sphinx",
+		"sphinx_rtd_theme"	=> "python3-sphinx_rtd_theme",
+		"virtualenv"		=> "python3-virtualenv",
+		"pip"			=> "python3-pip",
+		"dot"			=> "graphviz",
+		"convert"		=> "ImageMagick",
+		"Pod::Usage"		=> "perl-Pod-Usage",
+		"xelatex"		=> "texlive-xetex-bin",
+	);
+
+	my @suse_tex_pkgs = (
+		"texlive-babel-english",
+		"texlive-caption",
+		"texlive-colortbl",
+		"texlive-courier",
+		"texlive-dvips",
+		"texlive-helvetic",
+		"texlive-makeindex",
+		"texlive-metafont",
+		"texlive-metapost",
+		"texlive-palatino",
+		"texlive-preview",
+		"texlive-times",
+		"texlive-zapfchan",
+		"texlive-zapfding",
+	);
+
+	check_rpm_missing(\@suse_tex_pkgs, 1) if ($pdf);
+	check_missing_tex(1) if ($pdf);
+	check_missing(\%map);
+
+	return if (!$need && !$optional);
+	printf("You should run:\n\n\tsudo zypper install --no-recommends $install\n");
+}
+
+sub give_arch_linux_hints()
+{
+	my %map = (
+		"sphinx_rtd_theme"	=> "python-sphinx_rtd_theme",
+		"virtualenv"		=> "python-virtualenv",
+		"pip"			=> "python-pip",
+		"dot"			=> "graphviz",
+		"convert"		=> "imagemagick",
+		"xelatex"		=> "texlive-bin",
+	);
+
+	my @archlinux_tex_pkgs = (
+		"texlive-core",
+		"texlive-latexextra",
+		"ttf-dejavu",
+	);
+	check_pacman_missing(\@archlinux_tex_pkgs, 1) if ($pdf);
+	check_missing(\%map);
+
+	return if (!$need && !$optional);
+	printf("You should run:\n\n\tsudo pacman -S $install\n");
+}
+
+sub give_gentoo_hints()
+{
+	my %map = (
+		"sphinx_rtd_theme"	=> "dev-python/sphinx_rtd_theme",
+		"virtualenv"		=> "dev-python/virtualenv",
+		"pip"			=> "dev-python/pip",
+		"dot"			=> "media-gfx/graphviz",
+		"convert"		=> "media-gfx/imagemagick",
+		"xelatex"		=> "dev-texlive/texlive-xetex media-fonts/dejavu",
+	);
+
+	check_missing_file("/usr/share/fonts/dejavu/DejaVuSans.ttf",
+			   "media-fonts/dejavu", 1) if ($pdf);
+
+	check_missing(\%map);
+
+	return if (!$need && !$optional);
+	printf("You should run:\n\n\tsudo emerge --ask $install\n");
+}
+
+sub check_distros()
+{
+	# Distro-specific hints
+	if ($system_release =~ /Red Hat Enterprise Linux/) {
+		give_redhat_hints;
+		return;
+	}
+	if ($system_release =~ /Fedora/) {
+		give_redhat_hints;
+		return;
+	}
+	if ($system_release =~ /Ubuntu/) {
+		give_debian_hints;
+		return;
+	}
+	if ($system_release =~ /Debian/) {
+		give_debian_hints;
+		return;
+	}
+	if ($system_release =~ /openSUSE/) {
+		give_opensuse_hints;
+		return;
+	}
+	if ($system_release =~ /Arch Linux/) {
+		give_arch_linux_hints;
+		return;
+	}
+	if ($system_release =~ /Gentoo/) {
+		give_gentoo_hints;
+		return;
+	}
+
+	#
+	# Fall-back to generic hint code for other distros
+	# That's far from ideal, specially for LaTeX dependencies.
+	#
+	my %map = (
+		"sphinx-build" => "sphinx"
+	);
+	check_missing_tex(1) if ($pdf);
+	check_missing(\%map);
+	print "I don't know distro $system_release.\n";
+	print "So, I can't provide you a hint with the install procedure.\n";
+	print "There are likely missing dependencies.\n";
+}
+
+#
+# Common dependencies
+#
+
+sub check_needs()
+{
+	if ($system_release) {
+		print "Checking if the needed tools for $system_release are available\n";
+	} else {
+		print "Checking if the needed tools are present\n";
+	}
+
+	# Check for needed programs/tools
+	check_sphinx();
+	check_perl_module("Pod::Usage", 0);
+	check_program("make", 0);
+	check_program("gcc", 0);
+	check_python_module("sphinx_rtd_theme", 1) if (!$virtualenv);
+	check_program("xelatex", 1) if ($pdf);
+	check_program("dot", 1);
+	check_program("convert", 1);
+
+	check_distros();
+
+	if ($need_symlink) {
+		printf "\tsudo ln -sf %s /usr/bin/sphinx-build\n\n",
+		       which("sphinx-build-3");
+	}
+	if ($need_sphinx) {
+		my $virtualenv = findprog("virtualenv-3");
+		$virtualenv = findprog("virtualenv") if (!$virtualenv);
+		$virtualenv = "virtualenv" if (!$virtualenv);
+
+		printf "\t$virtualenv sphinx_1.4\n";
+		printf "\t. sphinx_1.4/bin/activate\n";
+		printf "\tpip install 'docutils==0.12'\n";
+		printf "\tpip install 'Sphinx==1.4.9'\n";
+		printf "\tpip install sphinx_rtd_theme\n";
+		$need++;
+	}
+	printf "\n";
+
+	print "All optional dependenties are met.\n" if (!$optional);
+
+	if ($need == 1) {
+		die "Can't build as $need mandatory dependency is missing";
+	} elsif ($need) {
+		die "Can't build as $need mandatory dependencies are missing";
+	}
+
+	print "Needed package dependencies are met.\n";
+}
+
+#
+# Main
+#
+
+while (@ARGV) {
+	my $arg = shift(@ARGV);
+
+	if ($arg eq "--no-virtualenv") {
+		$virtualenv = 0;
+	} elsif ($arg eq "--no-pdf"){
+		$pdf = 0;
+	} else {
+		print "Usage:\n\t$0 <--no-virtualenv> <--no-pdf>\n\n";
+		exit -1;
+	}
+}
+
+#
+# Determine the system type. There's no standard unique way that would
+# work with all distros with a minimal package install. So, several
+# methods are used here.
+#
+# By default, it will use lsb_release function. If not available, it will
+# fail back to reading the known different places where the distro name
+# is stored
+#
+
+$system_release = qx(lsb_release -d) if which("lsb_release");
+$system_release =~ s/Description:\s*// if ($system_release);
+$system_release = catcheck("/etc/system-release") if !$system_release;
+$system_release = catcheck("/etc/redhat-release") if !$system_release;
+$system_release = catcheck("/etc/lsb-release") if !$system_release;
+$system_release = catcheck("/etc/gentoo-release") if !$system_release;
+$system_release = catcheck("/etc/issue") if !$system_release;
+$system_release =~ s/\s+$//;
+
+check_needs;
-- 
2.13.3

[PATCH v2 3/8] sphinx-pre-install: detect an existing virtualenv

[Mauro Carvalho Chehab]

Detect if the script runs after creating the virtualenv,
printing the command line commands to enable the virtualenv.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 scripts/sphinx-pre-install | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)

diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 5fe3b4a86174..bcd6f7978f8d 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -13,6 +13,8 @@ use strict;
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 
+my $virtenv_dir = "sphinx_1.4";
+
 #
 # Static vars
 #
@@ -454,16 +456,22 @@ sub check_needs()
 		       which("sphinx-build-3");
 	}
 	if ($need_sphinx) {
-		my $virtualenv = findprog("virtualenv-3");
-		$virtualenv = findprog("virtualenv") if (!$virtualenv);
-		$virtualenv = "virtualenv" if (!$virtualenv);
+		my $activate = "$virtenv_dir/bin/activate";
+		if (-e "$ENV{'PWD'}/$activate") {
+			printf "\nNeed to activate virtualenv with:\n";
+			printf "\t. $activate\n";
+		} else {
+			my $virtualenv = findprog("virtualenv-3");
+			$virtualenv = findprog("virtualenv") if (!$virtualenv);
+			$virtualenv = "virtualenv" if (!$virtualenv);
 
-		printf "\t$virtualenv sphinx_1.4\n";
-		printf "\t. sphinx_1.4/bin/activate\n";
-		printf "\tpip install 'docutils==0.12'\n";
-		printf "\tpip install 'Sphinx==1.4.9'\n";
-		printf "\tpip install sphinx_rtd_theme\n";
-		$need++;
+			printf "\t$virtualenv $virtenv_dir\n";
+			printf "\t. $activate\n";
+			printf "\tpip install 'docutils==0.12'\n";
+			printf "\tpip install 'Sphinx==1.4.9'\n";
+			printf "\tpip install sphinx_rtd_theme\n";
+			$need++;
+		}
 	}
 	printf "\n";
 
-- 
2.13.3

[PATCH v2 4/8] sphinx-pre-install: use a requirements file

[Mauro Carvalho Chehab]

Instead of using 3 commands to install a virtualenv, use
a single one, reading the requirements from this file:

	Documentation/sphinx/requirements.txt

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 Documentation/sphinx/requirements.txt | 3 +++
 scripts/sphinx-pre-install            | 5 ++---
 2 files changed, 5 insertions(+), 3 deletions(-)
 create mode 100644 Documentation/sphinx/requirements.txt

diff --git a/Documentation/sphinx/requirements.txt b/Documentation/sphinx/requirements.txt
new file mode 100644
index 000000000000..742be3e12619
--- /dev/null
+++ b/Documentation/sphinx/requirements.txt
@@ -0,0 +1,3 @@
+docutils==0.12
+Sphinx==1.4.9
+sphinx_rtd_theme
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index bcd6f7978f8d..230d9cc7ee1d 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -14,6 +14,7 @@ use strict;
 # GNU General Public License for more details.
 
 my $virtenv_dir = "sphinx_1.4";
+my $requirement_file = "Documentation/sphinx/requirements.txt";
 
 #
 # Static vars
@@ -467,9 +468,7 @@ sub check_needs()
 
 			printf "\t$virtualenv $virtenv_dir\n";
 			printf "\t. $activate\n";
-			printf "\tpip install 'docutils==0.12'\n";
-			printf "\tpip install 'Sphinx==1.4.9'\n";
-			printf "\tpip install sphinx_rtd_theme\n";
+			printf "\tpip install -r $requirement_file\n";
 			$need++;
 		}
 	}
-- 
2.13.3

[PATCH v2 5/8] sphinx-pre-install: check for the need of graphviz-gd

[Mauro Carvalho Chehab]

On newer versions of graphviz packaging on Fedora, it is needed to
install a separate package for PDF support.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 scripts/sphinx-pre-install | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 230d9cc7ee1d..817241c674d3 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -285,6 +285,10 @@ sub give_redhat_hints()
 		"xelatex"		=> "texlive-xetex-bin",
 	);
 
+	my @fedora26_opt_pkgs = (
+		"graphviz-gd",		# Fedora 26: needed for PDF support
+	);
+
 	my @fedora_tex_pkgs = (
 		"texlive-collection-fontsrecommended",
 		"texlive-collection-latex",
@@ -293,6 +297,11 @@ sub give_redhat_hints()
 		"dejavu-sans-mono-fonts",
 	);
 
+	my $release;
+
+	$release = $1 if ($system_release =~ /Fedora\s+release\s+(\d+)/);
+
+	check_rpm_missing(\@fedora26_opt_pkgs, 1) if ($pdf && $release >= 26);
 	check_rpm_missing(\@fedora_tex_pkgs, 1) if ($pdf);
 	check_missing_tex(1) if ($pdf);
 	check_missing(\%map);
-- 
2.13.3

[PATCH v2 6/8] sphinx-pre-install: add dependencies for ImageMagick to work with svg

[Mauro Carvalho Chehab]

ImageMagick actually uses librsvg for conversions when converiting
from SVG (actually, it uses rsvg-convert). That causes the build to
fail with:

	WARNING: Error #1 when calling: /usr/bin/convert /home/mchehab/docs/Documentation/media/uapi/v4l/selection.svg /home/mchehab/docs/Documentation/output/latex/selection.pdf
	convert: delegate failed `'rsvg-convert' -o '%o' '%i'' @ error/delegate.c/InvokeDelegate/1919.
	convert: unable to open file `/tmp/magick-8883oOQfHzrA5trM': No such file or directory @ error/constitute.c/ReadImage/544.

Add the corresponding dependencies.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 scripts/sphinx-pre-install | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 817241c674d3..fc9891ce5c85 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -258,6 +258,7 @@ sub give_debian_hints()
 		"convert"		=> "imagemagick",
 		"Pod::Usage"		=> "perl-modules",
 		"xelatex"		=> "texlive-xetex",
+		"rsvg-convert"		=> "librsvg2-bin",
 	);
 
 	if ($pdf) {
@@ -283,6 +284,7 @@ sub give_redhat_hints()
 		"convert"		=> "ImageMagick",
 		"Pod::Usage"		=> "perl-Pod-Usage",
 		"xelatex"		=> "texlive-xetex-bin",
+		"rsvg-convert"		=> "librsvg2-tools",
 	);
 
 	my @fedora26_opt_pkgs = (
@@ -321,6 +323,7 @@ sub give_opensuse_hints()
 		"convert"		=> "ImageMagick",
 		"Pod::Usage"		=> "perl-Pod-Usage",
 		"xelatex"		=> "texlive-xetex-bin",
+		"rsvg-convert"		=> "rsvg-view",
 	);
 
 	my @suse_tex_pkgs = (
@@ -357,6 +360,7 @@ sub give_arch_linux_hints()
 		"dot"			=> "graphviz",
 		"convert"		=> "imagemagick",
 		"xelatex"		=> "texlive-bin",
+		"rsvg-convert"		=> "extra/librsvg",
 	);
 
 	my @archlinux_tex_pkgs = (
@@ -380,6 +384,7 @@ sub give_gentoo_hints()
 		"dot"			=> "media-gfx/graphviz",
 		"convert"		=> "media-gfx/imagemagick",
 		"xelatex"		=> "dev-texlive/texlive-xetex media-fonts/dejavu",
+		"rsvg-convert"		=> "gnome-base/librsvg",
 	);
 
 	check_missing_file("/usr/share/fonts/dejavu/DejaVuSans.ttf",
@@ -458,6 +463,7 @@ sub check_needs()
 	check_program("xelatex", 1) if ($pdf);
 	check_program("dot", 1);
 	check_program("convert", 1);
+	check_program("rsvg-convert", 1) if ($pdf);
 
 	check_distros();
 
-- 
2.13.3

[PATCH v2 7/8] sphinx-pre-install: fix USE needs for GraphViz and ImageMagick

[Mauro Carvalho Chehab]

Gentoo need some USE for GraphViz and ImageMagick to have
the features required by kfigure.py.

Output that when providing instructions for Gentoo.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 scripts/sphinx-pre-install | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index fc9891ce5c85..2d18064d57b4 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -393,7 +393,12 @@ sub give_gentoo_hints()
 	check_missing(\%map);
 
 	return if (!$need && !$optional);
-	printf("You should run:\n\n\tsudo emerge --ask $install\n");
+
+	printf("You should run:\n\n");
+	printf("\tsudo su -c 'echo \"media-gfx/imagemagick svg png\" > /etc/portage/package.use/imagemagick'\n");
+	printf("\tsudo su -c 'echo \"media-gfx/graphviz cairo pdf\" > /etc/portage/package.use/graphviz'\n");
+	printf("\tsudo emerge --ask $install\n");
+
 }
 
 sub check_distros()
-- 
2.13.3

[PATCH v2 8/8] sphinx.rst: document scripts/sphinx-pre-install script

[Mauro Carvalho Chehab]

Now that we have a script to check for Sphinx dependencies,
document it.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 Documentation/doc-guide/sphinx.rst | 48 ++++++++++++++++++++++++++++++++++----
 1 file changed, 44 insertions(+), 4 deletions(-)

diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index 5521213efa68..8faafb9b2d86 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -28,6 +28,9 @@ The ReST markups currently used by the Documentation/ files are meant to be
 built with ``Sphinx`` version 1.3 or upper. If you're desiring to build
 PDF outputs, it is recommended to use version 1.4.6 or upper.
 
+There's a script that checks for the Spinx requirements. Please see
+:ref:`sphinx-pre-install` for further details.
+
 Most distributions are shipped with Sphinx, but its toolchain is fragile,
 and it is not uncommon that upgrading it or some other Python packages
 on your machine would cause the documentation build to break.
@@ -47,13 +50,15 @@ or ``virtualenv``, depending on how your distribution packaged Python 3.
       on the Sphinx version, it should be installed  in separate,
       with ``pip install sphinx_rtd_theme``.
 
+   #) Some ReST pages contain math expressions. Due to the way Sphinx work,
+      those expressions are written using LaTeX notation. It needs texlive
+      installed with amdfonts and amsmath in order to evaluate them.
+
 In summary, if you want to install Sphinx version 1.4.9, you should do::
 
        $ virtualenv sphinx_1.4
        $ . sphinx_1.4/bin/activate
-       (sphinx_1.4) $ pip install 'docutils==0.12'
-       (sphinx_1.4) $ pip install 'Sphinx==1.4.9'
-       (sphinx_1.4) $ pip install sphinx_rtd_theme
+       (sphinx_1.4) $ pip install -r Documentation/sphinx/requirements.txt
 
 After running ``. sphinx_1.4/bin/activate``, the prompt will change,
 in order to indicate that you're using the new environment. If you
@@ -83,7 +88,42 @@ For PDF and LaTeX output, you'll also need ``XeLaTeX`` version 3.14159265.
 
 Depending on the distribution, you may also need to install a series of
 ``texlive`` packages that provide the minimal set of functionalities
-required for ``XeLaTex`` to work.
+required for ``XeLaTeX`` to work.
+
+.. _sphinx-pre-install:
+
+Checking for Sphinx dependencies
+--------------------------------
+
+There's a script that automatically check for Sphinx dependencies. If it can
+recognize your distribution, it will also give a hint about the install
+command line options for your distro::
+
+	$ ./scripts/sphinx-pre-install
+	Checking if the needed tools for Fedora release 26 (Twenty Six) are available
+	Warning: better to also install "texlive-luatex85".
+	You should run:
+
+		sudo dnf install -y texlive-luatex85
+		/usr/bin/virtualenv sphinx_1.4
+		. sphinx_1.4/bin/activate
+		pip install -r Documentation/sphinx/requirements.txt
+
+	Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468.
+
+By default, it checks all the requirements for both html and PDF, including
+the requirements for images, math expressions and LaTeX build, and assumes
+that a virtual Python environment will be used. The ones needed for html
+builds are assumed to be mandatory; the others to be optional.
+
+It supports two optional parameters:
+
+``--no-pdf``
+	Disable checks for PDF;
+
+``--no-virtualenv``
+	Use OS packaging for Sphinx instead of Python virtual environment.
+
 
 Sphinx Build
 ============
-- 
2.13.3

Re: [PATCH v2 0/8] Add a script to check for Sphinx install requirements

[Jonathan Corbet]

On Mon, 17 Jul 2017 18:46:34 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> wrote:

> Sphinx installation is not trivial, as not all versions are supported,
> and it requires a lot of stuff for math, images and PDF/LaTeX output
> to work.
> 
> So, add a script that checks if everything is fine, providing
> distro-specific hints about what's needed for it to work.

I've applied these (including the Mageia one), thanks.

It occurs to me that adding CentOS would be easy, just treat it like an
RHEL system.  Maybe I'll give that a shot in a bit.

Thanks,

jon

Re: [PATCH v2 0/8] Add a script to check for Sphinx install requirements

[Mauro Carvalho Chehab]

Em Sun, 23 Jul 2017 16:08:29 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> On Mon, 17 Jul 2017 18:46:34 -0300
> Mauro Carvalho Chehab <mchehab@s-opensource.com> wrote:
> 
> > Sphinx installation is not trivial, as not all versions are supported,
> > and it requires a lot of stuff for math, images and PDF/LaTeX output
> > to work.
> > 
> > So, add a script that checks if everything is fine, providing
> > distro-specific hints about what's needed for it to work.  
> 
> I've applied these (including the Mageia one), thanks.
> 
> It occurs to me that adding CentOS would be easy, just treat it like an
> RHEL system.  Maybe I'll give that a shot in a bit.

Heh, I sent a patch for it today :-)

There's a problem with CentOS/RHEL: it doesn't have texlive tabulary.
Without that, PDF won't work.

So, CentOS basic support is easy; PDF would require some manual
install (or some advanced way to do the right thing).

Thanks,
Mauro

Re: [PATCH v2 0/8] Add a script to check for Sphinx install requirements

[Mauro Carvalho Chehab]

Em Sun, 23 Jul 2017 20:01:44 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:

> Em Sun, 23 Jul 2017 16:08:29 -0600
> Jonathan Corbet <corbet@lwn.net> escreveu:
> 
> > On Mon, 17 Jul 2017 18:46:34 -0300
> > Mauro Carvalho Chehab <mchehab@s-opensource.com> wrote:
> >   
> > > Sphinx installation is not trivial, as not all versions are supported,
> > > and it requires a lot of stuff for math, images and PDF/LaTeX output
> > > to work.
> > > 
> > > So, add a script that checks if everything is fine, providing
> > > distro-specific hints about what's needed for it to work.    
> > 
> > I've applied these (including the Mageia one), thanks.
> > 
> > It occurs to me that adding CentOS would be easy, just treat it like an
> > RHEL system.  Maybe I'll give that a shot in a bit.  
> 
> Heh, I sent a patch for it today :-)

Sorry, the e-mail was queued, but was actually sent only today.


Thanks,
Mauro