* Re: [PATCH] docs: translations: add translations links when they exist
2023-10-28 16:29 [PATCH] docs: translations: add translations links when they exist Vegard Nossum
@ 2023-10-28 18:51 ` Vegard Nossum
2023-11-03 2:00 ` Yanteng Si
2023-10-30 11:30 ` Federico Vaga
` (3 subsequent siblings)
4 siblings, 1 reply; 9+ messages in thread
From: Vegard Nossum @ 2023-10-28 18:51 UTC (permalink / raw)
To: Jonathan Corbet
Cc: linux-doc, Federico Vaga, Akira Yokosawa, Carlos Bilbao, Alex Shi,
Yanteng Si, Hu Haowen, linux-kernel
This went a bit fast... corrections below.
On 28/10/2023 18:29, Vegard Nossum wrote:
> Add a new Sphinx extension that knows about the translations of kernel
> documentation and can insert links to the translations at the top of
> the document.
[...]
> Testing: make htmldocs with v7.3.0.
*Sphinx v7.3.0.
> +all_languages = {
> + # English is always first
> + None: 'English',
> +
> + # Keep the rest sorted alphabetically
> + 'zh_CN': 'Chinese',
> + 'it_IT': 'Italian',
> + 'ja_JP': 'Japanese',
> + 'ko_KR': 'Korean',
> + 'sp_SP': 'Spanish',
> + 'zh_TW': 'Taiwanese',
> +}
I went with my naive understanding of the language codes without double
checking but I think these might be better names:
'zh_CN': 'Chinese (simplified)'
'zh_TW': 'Chinese (traditional)',
Thoughts?
Vegard
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH] docs: translations: add translations links when they exist
2023-10-28 18:51 ` Vegard Nossum
@ 2023-11-03 2:00 ` Yanteng Si
0 siblings, 0 replies; 9+ messages in thread
From: Yanteng Si @ 2023-11-03 2:00 UTC (permalink / raw)
To: Vegard Nossum, Jonathan Corbet
Cc: linux-doc, Federico Vaga, Akira Yokosawa, Carlos Bilbao, Alex Shi,
Hu Haowen, linux-kernel
在 2023/10/29 02:51, Vegard Nossum 写道:
>
> This went a bit fast... corrections below.
>
> On 28/10/2023 18:29, Vegard Nossum wrote:
>> Add a new Sphinx extension that knows about the translations of kernel
>> documentation and can insert links to the translations at the top of
>> the document.
>
> [...]
>
>> Testing: make htmldocs with v7.3.0.
>
> *Sphinx v7.3.0.
>
>> +all_languages = {
>> + # English is always first
>> + None: 'English',
>> +
>> + # Keep the rest sorted alphabetically
>> + 'zh_CN': 'Chinese',
>> + 'it_IT': 'Italian',
>> + 'ja_JP': 'Japanese',
>> + 'ko_KR': 'Korean',
>> + 'sp_SP': 'Spanish',
>> + 'zh_TW': 'Taiwanese',
>> +}
>
> I went with my naive understanding of the language codes without double
> checking but I think these might be better names:
>
> 'zh_CN': 'Chinese (simplified)'
> 'zh_TW': 'Chinese (traditional)',
Yes, but we need to capitalize the first letter， just like:
'zh_CN': 'Chinese (Simplified)'
'zh_TW': 'Chinese (Traditional)',
see <https://translations.launchpad.net/ubuntu>
Thanks,
Yanteng
>
> Thoughts?
>
>
> Vegard
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] docs: translations: add translations links when they exist
2023-10-28 16:29 [PATCH] docs: translations: add translations links when they exist Vegard Nossum
2023-10-28 18:51 ` Vegard Nossum
@ 2023-10-30 11:30 ` Federico Vaga
2023-10-30 17:11 ` Jani Nikula
` (2 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Federico Vaga @ 2023-10-30 11:30 UTC (permalink / raw)
To: Vegard Nossum
Cc: Jonathan Corbet, linux-doc, Akira Yokosawa, Carlos Bilbao,
Alex Shi, Yanteng Si, Hu Haowen, linux-kernel
I find it a nice initiative :)
On Sat, Oct 28, 2023 at 06:29:31PM +0200, Vegard Nossum wrote:
>Add a new Sphinx extension that knows about the translations of kernel
>documentation and can insert links to the translations at the top of
>the document.
>
>It basically works like this:
>
>1. Register a new node type, LanguagesNode.
>
>2. Register a new transform, TranslationsTransform, that inserts a new
> LanguageNode at the top of every document. The LanguageNode contains
> "pending references" to translations of the document. The key here
> is that these are pending (i.e. unresolved) references that may or
> may not actually exist.
>
>3. Register a 'doctree-resolved' event that iterates over all the
> LanguageNode nodes. Any unresolved references are filtered out; the
> list of resolved references is passed to the 'translations.html'
> template and rendered as an HTML node (if HTML output is selected).
>
>Testing: make htmldocs with v7.3.0.
>
>Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
>---
> Documentation/conf.py | 2 +-
> Documentation/sphinx-static/custom.css | 8 ++
> .../sphinx/templates/translations.html | 12 +++
> Documentation/sphinx/translations.py | 96 +++++++++++++++++++
> 4 files changed, 117 insertions(+), 1 deletion(-)
> create mode 100644 Documentation/sphinx/templates/translations.html
> create mode 100644 Documentation/sphinx/translations.py
>
>diff --git a/Documentation/conf.py b/Documentation/conf.py
>index d4fdf6a3875a..64eab500b2cd 100644
>--- a/Documentation/conf.py
>+++ b/Documentation/conf.py
>@@ -55,7 +55,7 @@ needs_sphinx = '1.7'
> extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include',
> 'kfigure', 'sphinx.ext.ifconfig', 'automarkup',
> 'maintainers_include', 'sphinx.ext.autosectionlabel',
>- 'kernel_abi', 'kernel_feat']
>+ 'kernel_abi', 'kernel_feat', 'translations']
>
> if major >= 3:
> if (major > 3) or (minor > 0 or patch >= 2):
>diff --git a/Documentation/sphinx-static/custom.css b/Documentation/sphinx-static/custom.css
>index 084a884f6fb7..33adee4a35d9 100644
>--- a/Documentation/sphinx-static/custom.css
>+++ b/Documentation/sphinx-static/custom.css
>@@ -73,3 +73,11 @@ input.kernel-toc-toggle { display: none; }
> h3.kernel-toc-contents { display: inline; }
> div.kerneltoc a { color: black; }
> }
>+
>+/* Language selection bar */
>+div.language-selection {
>+ background: #eeeeee;
>+ border: 1px solid #cccccc;
>+ margin-bottom: 1em;
>+ padding: .5em;
>+}
>diff --git a/Documentation/sphinx/templates/translations.html b/Documentation/sphinx/templates/translations.html
>new file mode 100644
>index 000000000000..08afb595c203
>--- /dev/null
>+++ b/Documentation/sphinx/templates/translations.html
>@@ -0,0 +1,12 @@
>+<!-- SPDX-License-Identifier: GPL-2.0 -->
>+<!-- Copyright © 2023, Oracle and/or its affiliates. -->
>+
>+{# Create a language bar for translations #}
>+{% if languages|length > 0: %}
>+<div class="language-selection">
>+Languages:
Should also "Languages" subject to translation?
Or perhaps, nothing. Just the list of languages.
>+{% for ref in languages: %}
>+<a href="{{ ref.refuri }}">{{ ref.astext() }}</a>{% if not loop.last %}, {% endif %}
>+{% endfor %}
>+</div>
>+{% endif %}
>diff --git a/Documentation/sphinx/translations.py b/Documentation/sphinx/translations.py
>new file mode 100644
>index 000000000000..e1da811bdaf0
>--- /dev/null
>+++ b/Documentation/sphinx/translations.py
>@@ -0,0 +1,96 @@
>+# SPDX-License-Identifier: GPL-2.0
>+#
>+# Copyright © 2023, Oracle and/or its affiliates.
>+# Author: Vegard Nossum <vegard.nossum@oracle.com>
>+#
>+# Add translation links to the top of the document.
>+#
>+
>+import os
>+
>+from docutils import nodes
>+from docutils.transforms import Transform
>+
>+import sphinx
>+from sphinx import addnodes
>+from sphinx.errors import NoUri
>+
>+all_languages = {
>+ # English is always first
>+ None: 'English',
>+
>+ # Keep the rest sorted alphabetically
>+ 'zh_CN': 'Chinese',
>+ 'it_IT': 'Italian',
>+ 'ja_JP': 'Japanese',
>+ 'ko_KR': 'Korean',
>+ 'sp_SP': 'Spanish',
>+ 'zh_TW': 'Taiwanese',
Minor comment. Should we use the language name in its original language?
Example:
[... snip ...]
'it_IT': Italiano,
[... snip ...]
'sp_SP': Español,
[... snip ...]
>+}
>+
>+class LanguagesNode(nodes.Element):
>+ pass
>+
>+class TranslationsTransform(Transform):
>+ default_priority = 900
>+
>+ def apply(self):
>+ app = self.document.settings.env.app
>+ if app.builder.format not in ['html']:
>+ return
>+
>+ docname = self.document.settings.env.docname
>+
>+ this_lang_code = None
>+ components = docname.split(os.sep)
>+ if components[0] == 'translations' and len(components) > 2:
>+ this_lang_code = components[1]
>+
>+ # normalize docname to be the untranslated one
>+ docname = os.path.join(*components[2:])
>+
>+ new_nodes = LanguagesNode()
>+
>+ for lang_code, lang_name in all_languages.items():
>+ if lang_code == this_lang_code:
>+ continue
We could show all languages (remove the above two lines) to have a consistent
output among pages.
>+ if lang_code is None:
>+ target_name = docname
>+ else:
>+ target_name = os.path.join('translations', lang_code, docname)
>+
>+ pxref = addnodes.pending_xref('', refdomain='std',
>+ reftype='doc', reftarget='/' + target_name, modname=None,
>+ classname=None, refexplicit=True)
>+ pxref += nodes.Text(lang_name)
>+ new_nodes += pxref
>+
>+ self.document.insert(0, new_nodes)
>+
>+def process_languages(app, doctree, docname):
>+ for node in doctree.traverse(LanguagesNode):
>+ languages = []
>+
>+ # Iterate over the child nodes; any resolved links will have
>+ # the type 'nodes.reference', while unresolved links will be
>+ # type 'nodes.Text'.
>+ languages = list(filter(lambda xref:
>+ isinstance(xref, nodes.reference), node.children))
>+
>+ html_content = app.builder.templates.render('translations.html',
>+ context={
>+ 'languages': languages,
>+ })
>+
>+ node.replace_self(nodes.raw('', html_content, format='html'))
>+
>+def setup(app):
>+ app.add_node(LanguagesNode)
>+ app.add_transform(TranslationsTransform)
>+ app.connect('doctree-resolved', process_languages)
>+
>+ return {
>+ 'parallel_read_safe': True,
>+ 'parallel_write_safe': True,
>+ }
>--
>2.34.1
>
--
Federico Vaga
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH] docs: translations: add translations links when they exist
2023-10-28 16:29 [PATCH] docs: translations: add translations links when they exist Vegard Nossum
2023-10-28 18:51 ` Vegard Nossum
2023-10-30 11:30 ` Federico Vaga
@ 2023-10-30 17:11 ` Jani Nikula
2023-11-01 14:56 ` Akira Yokosawa
2023-11-17 20:54 ` Jonathan Corbet
4 siblings, 0 replies; 9+ messages in thread
From: Jani Nikula @ 2023-10-30 17:11 UTC (permalink / raw)
To: Vegard Nossum, Jonathan Corbet
Cc: linux-doc, Federico Vaga, Akira Yokosawa, Carlos Bilbao, Alex Shi,
Yanteng Si, Hu Haowen, linux-kernel, Vegard Nossum
On Sat, 28 Oct 2023, Vegard Nossum <vegard.nossum@oracle.com> wrote:
> Add a new Sphinx extension that knows about the translations of kernel
> documentation and can insert links to the translations at the top of
> the document.
>
> It basically works like this:
>
> 1. Register a new node type, LanguagesNode.
>
> 2. Register a new transform, TranslationsTransform, that inserts a new
> LanguageNode at the top of every document. The LanguageNode contains
> "pending references" to translations of the document. The key here
> is that these are pending (i.e. unresolved) references that may or
> may not actually exist.
>
> 3. Register a 'doctree-resolved' event that iterates over all the
> LanguageNode nodes. Any unresolved references are filtered out; the
> list of resolved references is passed to the 'translations.html'
> template and rendered as an HTML node (if HTML output is selected).
>
> Testing: make htmldocs with v7.3.0.
I'm just observing the kernel documentation has a system of its own for
translations. Maybe it's fine this way, but it's certainly different
from what the Sphinx developers had in mind. (But I'm not an expert on
this field, don't ask me how this should be done!)
Regardless, the implication is that this builds on the translation
solution chosen for kernel documentation, for better and for worse, and
raises the bar for switching later. It's something to be aware of.
>
> Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
> ---
> Documentation/conf.py | 2 +-
> Documentation/sphinx-static/custom.css | 8 ++
> .../sphinx/templates/translations.html | 12 +++
> Documentation/sphinx/translations.py | 96 +++++++++++++++++++
> 4 files changed, 117 insertions(+), 1 deletion(-)
> create mode 100644 Documentation/sphinx/templates/translations.html
> create mode 100644 Documentation/sphinx/translations.py
>
> diff --git a/Documentation/conf.py b/Documentation/conf.py
> index d4fdf6a3875a..64eab500b2cd 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -55,7 +55,7 @@ needs_sphinx = '1.7'
> extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include',
> 'kfigure', 'sphinx.ext.ifconfig', 'automarkup',
> 'maintainers_include', 'sphinx.ext.autosectionlabel',
> - 'kernel_abi', 'kernel_feat']
> + 'kernel_abi', 'kernel_feat', 'translations']
>
> if major >= 3:
> if (major > 3) or (minor > 0 or patch >= 2):
> diff --git a/Documentation/sphinx-static/custom.css b/Documentation/sphinx-static/custom.css
> index 084a884f6fb7..33adee4a35d9 100644
> --- a/Documentation/sphinx-static/custom.css
> +++ b/Documentation/sphinx-static/custom.css
> @@ -73,3 +73,11 @@ input.kernel-toc-toggle { display: none; }
> h3.kernel-toc-contents { display: inline; }
> div.kerneltoc a { color: black; }
> }
> +
> +/* Language selection bar */
> +div.language-selection {
> + background: #eeeeee;
> + border: 1px solid #cccccc;
> + margin-bottom: 1em;
> + padding: .5em;
> +}
> diff --git a/Documentation/sphinx/templates/translations.html b/Documentation/sphinx/templates/translations.html
> new file mode 100644
> index 000000000000..08afb595c203
> --- /dev/null
> +++ b/Documentation/sphinx/templates/translations.html
> @@ -0,0 +1,12 @@
> +<!-- SPDX-License-Identifier: GPL-2.0 -->
> +<!-- Copyright © 2023, Oracle and/or its affiliates. -->
> +
> +{# Create a language bar for translations #}
> +{% if languages|length > 0: %}
> +<div class="language-selection">
> +Languages:
> +{% for ref in languages: %}
> +<a href="{{ ref.refuri }}">{{ ref.astext() }}</a>{% if not loop.last %}, {% endif %}
> +{% endfor %}
> +</div>
> +{% endif %}
This could also be part of the menu on the left, I think it's fairly
easy to extend in alabaster. But then again it's already pretty crowded,
and it's all a matter of taste.
> diff --git a/Documentation/sphinx/translations.py b/Documentation/sphinx/translations.py
> new file mode 100644
> index 000000000000..e1da811bdaf0
> --- /dev/null
> +++ b/Documentation/sphinx/translations.py
> @@ -0,0 +1,96 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Copyright © 2023, Oracle and/or its affiliates.
> +# Author: Vegard Nossum <vegard.nossum@oracle.com>
> +#
> +# Add translation links to the top of the document.
> +#
> +
> +import os
> +
> +from docutils import nodes
> +from docutils.transforms import Transform
> +
> +import sphinx
> +from sphinx import addnodes
> +from sphinx.errors import NoUri
> +
> +all_languages = {
> + # English is always first
> + None: 'English',
> +
> + # Keep the rest sorted alphabetically
> + 'zh_CN': 'Chinese',
> + 'it_IT': 'Italian',
> + 'ja_JP': 'Japanese',
> + 'ko_KR': 'Korean',
> + 'sp_SP': 'Spanish',
> + 'zh_TW': 'Taiwanese',
> +}
Maybe the path to translations should be a configuration option for the
extension, and it could read the translations from the directory instead
of hard coding them here.
> +
> +class LanguagesNode(nodes.Element):
> + pass
> +
> +class TranslationsTransform(Transform):
> + default_priority = 900
> +
> + def apply(self):
> + app = self.document.settings.env.app
> + if app.builder.format not in ['html']:
> + return
> +
> + docname = self.document.settings.env.docname
> +
> + this_lang_code = None
> + components = docname.split(os.sep)
> + if components[0] == 'translations' and len(components) > 2:
> + this_lang_code = components[1]
> +
> + # normalize docname to be the untranslated one
> + docname = os.path.join(*components[2:])
Need to rename these files, and keep the translated document names
up-to-date:
for f in $(find Documentation/translations/ -name "*.rst"); do if ! test -f Documentation/${f##Documentation/translations/??_??/}; then echo $f; fi; done
Maybe the extension should warn about documents under translations that
do not have a corresponding original.
BR,
Jani.
> +
> + new_nodes = LanguagesNode()
> +
> + for lang_code, lang_name in all_languages.items():
> + if lang_code == this_lang_code:
> + continue
> +
> + if lang_code is None:
> + target_name = docname
> + else:
> + target_name = os.path.join('translations', lang_code, docname)
> +
> + pxref = addnodes.pending_xref('', refdomain='std',
> + reftype='doc', reftarget='/' + target_name, modname=None,
> + classname=None, refexplicit=True)
> + pxref += nodes.Text(lang_name)
> + new_nodes += pxref
> +
> + self.document.insert(0, new_nodes)
> +
> +def process_languages(app, doctree, docname):
> + for node in doctree.traverse(LanguagesNode):
> + languages = []
> +
> + # Iterate over the child nodes; any resolved links will have
> + # the type 'nodes.reference', while unresolved links will be
> + # type 'nodes.Text'.
> + languages = list(filter(lambda xref:
> + isinstance(xref, nodes.reference), node.children))
> +
> + html_content = app.builder.templates.render('translations.html',
> + context={
> + 'languages': languages,
> + })
> +
> + node.replace_self(nodes.raw('', html_content, format='html'))
> +
> +def setup(app):
> + app.add_node(LanguagesNode)
> + app.add_transform(TranslationsTransform)
> + app.connect('doctree-resolved', process_languages)
> +
> + return {
> + 'parallel_read_safe': True,
> + 'parallel_write_safe': True,
> + }
--
Jani Nikula, Intel
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH] docs: translations: add translations links when they exist
2023-10-28 16:29 [PATCH] docs: translations: add translations links when they exist Vegard Nossum
` (2 preceding siblings ...)
2023-10-30 17:11 ` Jani Nikula
@ 2023-11-01 14:56 ` Akira Yokosawa
2023-11-02 11:07 ` Vegard Nossum
2023-11-17 20:54 ` Jonathan Corbet
4 siblings, 1 reply; 9+ messages in thread
From: Akira Yokosawa @ 2023-11-01 14:56 UTC (permalink / raw)
To: vegard.nossum
Cc: akiyks, alexs, carlos.bilbao, corbet, federico.vaga, linux-doc,
linux-kernel, siyanteng, src.res.211
Hi,
On Sat, 28 Oct 2023 18:29:31 +0200, Vegard Nossum wrote:
> Add a new Sphinx extension that knows about the translations of kernel
> documentation and can insert links to the translations at the top of
> the document.
>
> It basically works like this:
>
> 1. Register a new node type, LanguagesNode.
>
> 2. Register a new transform, TranslationsTransform, that inserts a new
> LanguageNode at the top of every document. The LanguageNode contains
> "pending references" to translations of the document. The key here
> is that these are pending (i.e. unresolved) references that may or
> may not actually exist.
>
> 3. Register a 'doctree-resolved' event that iterates over all the
> LanguageNode nodes. Any unresolved references are filtered out; the
> list of resolved references is passed to the 'translations.html'
> template and rendered as an HTML node (if HTML output is selected).
>
> Testing: make htmldocs with v7.3.0.
So, I've started playing with this.
It looks like this introduces hysteresis in successive runs of
"make htmldocs" and "make latexdocs".
Steps to reproduce
1. Run "make cleandocs"
2. Run "make htmldocs"
3. Run "make latexdocs"
This aborts with the message (under Sphinx 7.2.6):
Extension error (translations):
Handler <function process_languages at 0x7f122f343420> for event 'doctree-resolved' threw an exception (exception: 'LaTeXBuilder' object has no attribute 'templates')
make[2]: *** [Documentation/Makefile:128: latexdocs] Error 2
make[1]: *** [/linux/Makefile:1695: latexdocs] Error 2
make: *** [Makefile:234: __sub-make] Error 2
Command exited with non-zero status 2
If I run "make latexdocs" in step 2 and "make htmldocs" in step 3,
both runs complete successfully, but html pages don't have the
expected links to other translations.
All I can do is to report the symptoms.
Vegard, can you look into them?
>
> Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
> ---
> Documentation/conf.py | 2 +-
> Documentation/sphinx-static/custom.css | 8 ++
> .../sphinx/templates/translations.html | 12 +++
> Documentation/sphinx/translations.py | 96 +++++++++++++++++++
> 4 files changed, 117 insertions(+), 1 deletion(-)
> create mode 100644 Documentation/sphinx/templates/translations.html
> create mode 100644 Documentation/sphinx/translations.py
>
> diff --git a/Documentation/conf.py b/Documentation/conf.py
> index d4fdf6a3875a..64eab500b2cd 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -55,7 +55,7 @@ needs_sphinx = '1.7'
> extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include',
> 'kfigure', 'sphinx.ext.ifconfig', 'automarkup',
> 'maintainers_include', 'sphinx.ext.autosectionlabel',
> - 'kernel_abi', 'kernel_feat']
> + 'kernel_abi', 'kernel_feat', 'translations']
>
> if major >= 3:
> if (major > 3) or (minor > 0 or patch >= 2):
> diff --git a/Documentation/sphinx-static/custom.css b/Documentation/sphinx-static/custom.css
> index 084a884f6fb7..33adee4a35d9 100644
> --- a/Documentation/sphinx-static/custom.css
> +++ b/Documentation/sphinx-static/custom.css
> @@ -73,3 +73,11 @@ input.kernel-toc-toggle { display: none; }
> h3.kernel-toc-contents { display: inline; }
> div.kerneltoc a { color: black; }
> }
> +
> +/* Language selection bar */
> +div.language-selection {
> + background: #eeeeee;
> + border: 1px solid #cccccc;
> + margin-bottom: 1em;
> + padding: .5em;
> +}
> diff --git a/Documentation/sphinx/templates/translations.html b/Documentation/sphinx/templates/translations.html
> new file mode 100644
> index 000000000000..08afb595c203
> --- /dev/null
> +++ b/Documentation/sphinx/templates/translations.html
> @@ -0,0 +1,12 @@
> +<!-- SPDX-License-Identifier: GPL-2.0 -->
> +<!-- Copyright © 2023, Oracle and/or its affiliates. -->
> +
> +{# Create a language bar for translations #}
> +{% if languages|length > 0: %}
> +<div class="language-selection">
> +Languages:
> +{% for ref in languages: %}
> +<a href="{{ ref.refuri }}">{{ ref.astext() }}</a>{% if not loop.last %}, {% endif %}
> +{% endfor %}
> +</div>
> +{% endif %}
> diff --git a/Documentation/sphinx/translations.py b/Documentation/sphinx/translations.py
> new file mode 100644
> index 000000000000..e1da811bdaf0
> --- /dev/null
> +++ b/Documentation/sphinx/translations.py
> @@ -0,0 +1,96 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Copyright © 2023, Oracle and/or its affiliates.
> +# Author: Vegard Nossum <vegard.nossum@oracle.com>
> +#
> +# Add translation links to the top of the document.
> +#
> +
> +import os
> +
> +from docutils import nodes
> +from docutils.transforms import Transform
> +
> +import sphinx
> +from sphinx import addnodes
> +from sphinx.errors import NoUri
NoUri is not in sphinx.errors prior to Sphinx 2.
I think it is a good chance to finally get rid of Sphinx 1.7.x
support.
Thanks, Akira
> +
> +all_languages = {
> + # English is always first
> + None: 'English',
> +
> + # Keep the rest sorted alphabetically
> + 'zh_CN': 'Chinese',
> + 'it_IT': 'Italian',
> + 'ja_JP': 'Japanese',
> + 'ko_KR': 'Korean',
> + 'sp_SP': 'Spanish',
> + 'zh_TW': 'Taiwanese',
> +}
> +
> +class LanguagesNode(nodes.Element):
> + pass
> +
> +class TranslationsTransform(Transform):
> + default_priority = 900
> +
> + def apply(self):
> + app = self.document.settings.env.app
> + if app.builder.format not in ['html']:
> + return
> +
> + docname = self.document.settings.env.docname
> +
> + this_lang_code = None
> + components = docname.split(os.sep)
> + if components[0] == 'translations' and len(components) > 2:
> + this_lang_code = components[1]
> +
> + # normalize docname to be the untranslated one
> + docname = os.path.join(*components[2:])
> +
> + new_nodes = LanguagesNode()
> +
> + for lang_code, lang_name in all_languages.items():
> + if lang_code == this_lang_code:
> + continue
> +
> + if lang_code is None:
> + target_name = docname
> + else:
> + target_name = os.path.join('translations', lang_code, docname)
> +
> + pxref = addnodes.pending_xref('', refdomain='std',
> + reftype='doc', reftarget='/' + target_name, modname=None,
> + classname=None, refexplicit=True)
> + pxref += nodes.Text(lang_name)
> + new_nodes += pxref
> +
> + self.document.insert(0, new_nodes)
> +
> +def process_languages(app, doctree, docname):
> + for node in doctree.traverse(LanguagesNode):
> + languages = []
> +
> + # Iterate over the child nodes; any resolved links will have
> + # the type 'nodes.reference', while unresolved links will be
> + # type 'nodes.Text'.
> + languages = list(filter(lambda xref:
> + isinstance(xref, nodes.reference), node.children))
> +
> + html_content = app.builder.templates.render('translations.html',
> + context={
> + 'languages': languages,
> + })
> +
> + node.replace_self(nodes.raw('', html_content, format='html'))
> +
> +def setup(app):
> + app.add_node(LanguagesNode)
> + app.add_transform(TranslationsTransform)
> + app.connect('doctree-resolved', process_languages)
> +
> + return {
> + 'parallel_read_safe': True,
> + 'parallel_write_safe': True,
> + }
> --
> 2.34.1
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH] docs: translations: add translations links when they exist
2023-11-01 14:56 ` Akira Yokosawa
@ 2023-11-02 11:07 ` Vegard Nossum
2023-11-03 0:09 ` Akira Yokosawa
0 siblings, 1 reply; 9+ messages in thread
From: Vegard Nossum @ 2023-11-02 11:07 UTC (permalink / raw)
To: Akira Yokosawa
Cc: alexs, carlos.bilbao, corbet, federico.vaga, linux-doc,
linux-kernel, siyanteng, src.res.211, Jani Nikula
[-- Attachment #1: Type: text/plain, Size: 1856 bytes --]
On 01/11/2023 15:56, Akira Yokosawa wrote:
> It looks like this introduces hysteresis in successive runs of
> "make htmldocs" and "make latexdocs".
>
> Steps to reproduce
>
> 1. Run "make cleandocs"
>
> 2. Run "make htmldocs"
>
> 3. Run "make latexdocs"
>
> This aborts with the message (under Sphinx 7.2.6):
>
> Extension error (translations):
> Handler <function process_languages at 0x7f122f343420> for event 'doctree-resolved' threw an exception (exception: 'LaTeXBuilder' object has no attribute 'templates')
> make[2]: *** [Documentation/Makefile:128: latexdocs] Error 2
> make[1]: *** [/linux/Makefile:1695: latexdocs] Error 2
> make: *** [Makefile:234: __sub-make] Error 2
> Command exited with non-zero status 2
>
> If I run "make latexdocs" in step 2 and "make htmldocs" in step 3,
> both runs complete successfully, but html pages don't have the
> expected links to other translations.
>
> All I can do is to report the symptoms.
> Vegard, can you look into them?
Thanks for testing this out and reporting!
I think we can fix this by moving the "is this html output?" check from
the TranslationsTransform into the 'doctree-resolved' handler (which, as
far as I can tell, runs after the doctree has been serialized to disk
but before output is generated).
I've attached an incremental patch, does that seem to work for you? I
test both (clean/html/latex + clean/latex/html) and it seemed to work here.
I had a look at using a custom "visit" callback that would just render
the HTML in place instead of manipulating the doctree, but it also
doesn't feel right as then you need to specify callbacks for every
output writer; there doesn't seem to be a way to ignore the node by
default. Maybe I should ask on the Sphinx/docutils mailing lists what
the "proper" way to do this would be.
Thanks again!
Vegard
[-- Attachment #2: latexdocs-fix.patch --]
[-- Type: text/x-patch, Size: 864 bytes --]
diff --git a/Documentation/sphinx/translations.py b/Documentation/sphinx/translations.py
index 5f2866a12533..99a73075697a 100644
--- a/Documentation/sphinx/translations.py
+++ b/Documentation/sphinx/translations.py
@@ -36,9 +36,6 @@ class TranslationsTransform(Transform):
def apply(self):
app = self.document.settings.env.app
- if app.builder.format not in ['html']:
- return
-
docname = self.document.settings.env.docname
this_lang_code = None
@@ -70,6 +67,10 @@ class TranslationsTransform(Transform):
def process_languages(app, doctree, docname):
for node in doctree.traverse(LanguagesNode):
+ if app.builder.format not in ['html']:
+ node.parent.remove(node)
+ continue
+
languages = []
# Iterate over the child nodes; any resolved links will have
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PATCH] docs: translations: add translations links when they exist
2023-11-02 11:07 ` Vegard Nossum
@ 2023-11-03 0:09 ` Akira Yokosawa
0 siblings, 0 replies; 9+ messages in thread
From: Akira Yokosawa @ 2023-11-03 0:09 UTC (permalink / raw)
To: Vegard Nossum
Cc: alexs, carlos.bilbao, corbet, federico.vaga, linux-doc,
linux-kernel, siyanteng, src.res.211, Jani Nikula, Akira Yokosawa
On 2023/11/02 20:07, Vegard Nossum wrote:
> On 01/11/2023 15:56, Akira Yokosawa wrote:
>> It looks like this introduces hysteresis in successive runs of
>> "make htmldocs" and "make latexdocs".
>>
>> Steps to reproduce
>>
>> 1. Run "make cleandocs"
>>
>> 2. Run "make htmldocs"
>>
>> 3. Run "make latexdocs"
>>
>> This aborts with the message (under Sphinx 7.2.6):
>>
>> Extension error (translations):
>> Handler <function process_languages at 0x7f122f343420> for event 'doctree-resolved' threw an exception (exception: 'LaTeXBuilder' object has no attribute 'templates')
>> make[2]: *** [Documentation/Makefile:128: latexdocs] Error 2
>> make[1]: *** [/linux/Makefile:1695: latexdocs] Error 2
>> make: *** [Makefile:234: __sub-make] Error 2
>> Command exited with non-zero status 2
>>
>> If I run "make latexdocs" in step 2 and "make htmldocs" in step 3,
>> both runs complete successfully, but html pages don't have the
>> expected links to other translations.
>>
>> All I can do is to report the symptoms.
>> Vegard, can you look into them?
>
> Thanks for testing this out and reporting!
>
> I think we can fix this by moving the "is this html output?" check from
> the TranslationsTransform into the 'doctree-resolved' handler (which, as
> far as I can tell, runs after the doctree has been serialized to disk
> but before output is generated).
>
> I've attached an incremental patch, does that seem to work for you? I> test both (clean/html/latex + clean/latex/html) and it seemed to work here.
Yes, it works here as well.
Thanks, Akira
>
> I had a look at using a custom "visit" callback that would just render
> the HTML in place instead of manipulating the doctree, but it also
> doesn't feel right as then you need to specify callbacks for every
> output writer; there doesn't seem to be a way to ignore the node by
> default. Maybe I should ask on the Sphinx/docutils mailing lists what
> the "proper" way to do this would be.
>
> Thanks again!
>
>
> Vegard
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] docs: translations: add translations links when they exist
2023-10-28 16:29 [PATCH] docs: translations: add translations links when they exist Vegard Nossum
` (3 preceding siblings ...)
2023-11-01 14:56 ` Akira Yokosawa
@ 2023-11-17 20:54 ` Jonathan Corbet
4 siblings, 0 replies; 9+ messages in thread
From: Jonathan Corbet @ 2023-11-17 20:54 UTC (permalink / raw)
To: Vegard Nossum
Cc: linux-doc, Federico Vaga, Akira Yokosawa, Carlos Bilbao, Alex Shi,
Yanteng Si, Hu Haowen, linux-kernel, Vegard Nossum
Vegard Nossum <vegard.nossum@oracle.com> writes:
> Add a new Sphinx extension that knows about the translations of kernel
> documentation and can insert links to the translations at the top of
> the document.
>
> It basically works like this:
>
> 1. Register a new node type, LanguagesNode.
>
> 2. Register a new transform, TranslationsTransform, that inserts a new
> LanguageNode at the top of every document. The LanguageNode contains
> "pending references" to translations of the document. The key here
> is that these are pending (i.e. unresolved) references that may or
> may not actually exist.
>
> 3. Register a 'doctree-resolved' event that iterates over all the
> LanguageNode nodes. Any unresolved references are filtered out; the
> list of resolved references is passed to the 'translations.html'
> template and rendered as an HTML node (if HTML output is selected).
>
> Testing: make htmldocs with v7.3.0.
>
> Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
> ---
> Documentation/conf.py | 2 +-
> Documentation/sphinx-static/custom.css | 8 ++
> .../sphinx/templates/translations.html | 12 +++
> Documentation/sphinx/translations.py | 96 +++++++++++++++++++
> 4 files changed, 117 insertions(+), 1 deletion(-)
> create mode 100644 Documentation/sphinx/templates/translations.html
> create mode 100644 Documentation/sphinx/translations.py
OK, this does seem to work.
The naming of the translations definitely needs to change; if we put out
something with "Taiwanese" in it, experience tells me, there will be
objections - and that's not what the translation was called when it was
added.
I'm unsure about putting the languages in the top bar like that; it will
already become pretty wide with the relabeled translations, and may not
look great on a small-screen device. Perhaps a pulldown would be
better?
The build problem reported by Akira definitely needs to be fixed as
well.
Thanks,
jon
^ permalink raw reply [flat|nested] 9+ messages in thread