[PATCH rdma-core] Support writing man pages in MarkDown

[PATCH rdma-core] Support writing man pages in MarkDown

[Jason Gunthorpe]

The pages are converted during the build into *roff mdoc using
the pandoc tool.

Signed-off-by: Jason Gunthorpe <jgg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 .travis.yml                   |  1 +
 CMakeLists.txt                |  4 +++
 buildlib/Findpandoc.cmake     | 21 +++++++++++++
 buildlib/cbuild               |  3 ++
 buildlib/rdma_functions.cmake | 33 --------------------
 buildlib/rdma_man.cmake       | 70 +++++++++++++++++++++++++++++++++++++++++++
 debian/control                |  1 +
 7 files changed, 100 insertions(+), 33 deletions(-)
 create mode 100644 buildlib/Findpandoc.cmake
 create mode 100644 buildlib/rdma_man.cmake

This is a very big series, I am only sending the first patch to the
mailing list. The whole series is on github here:

https://github.com/linux-rdma/rdma-core/pull/274

Overall, I'd like to see the man pages be more editable and viewable
when in patch format. Longer term I'd like to see them all rendered to
HTML and hosted on the web as libfabric and systemd are doing
successfully. While some of this can be done with the classic roff
format, it feels very painful.

Markdown was selected because we are already using it successfully for
other documentation, and libfabric has already demonstrated using it
for man pages. github will natively render the markdown documentation
internally for instance:

https://github.com/jgunthorpe/rdma-plumbing/blob/ea581c2d3d99d69083a8f4eaa9e55ae7ae233246/libibverbs/man/ibv_alloc_mw.3.md

Although some places are using features specific to the 'pandoc'
flavour of markdown, github still renders them in a readable format.

The next most popular choice is docbook XML, which I suspect would not
be well liked (and it is very baroque as well, just in a more modern
way)

The actual conversion of the man pages was done largely by script,
with some manual fixes and post-inspection. There doesn't seem to be a
useful mdoc to markdown conversion script, so I built a quick and
dirty one that works on our .3 man page style.

The converter is here:

https://github.com/jgunthorpe/rdma-plumbing/blob/migration-tooling/buildlib/man2md.py

I'm no mdoc expert, but it sure looks to me like there are many markup
problems in our man pages, even if they happen to render mostly properly.

Overall the conversion to markdown is very good in terms of resulting
'man ibv_foo' display with a few notable variances:
 - The 'synopsis' is shifted right by 8 spaces and isn't bolded in
   quite the same way
 - I ran all the C code through clang-format and asked it fit it to
   the standard 80 col terminal. Some of the results are 'interesting'
   to say the least. Others are much better than before.
 - Some of the hypenation choices are a little different
 
pandoc is fairly widely available in the distros with the notable
problem that centos 6 and 7 can only get it from EPEL.

Since this may be a problem, I am thinking about following the
libfabric approach and either including the precompiled man pages in
the source tree, or just providing a framework for downstream to
add-on precompiled files (with a cbuild command to make them)
Looking for feedback on what path is best..

diff --git a/.travis.yml b/.travis.yml
index 7af2aeb9072db5..b7723f38295bc0 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -32,6 +32,7 @@ addons:
       - libudev-dev
       - make
       - ninja-build
+      - pandoc
       - pkg-config
       - python
       - valgrind
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 6c5d6b7c539515..42fc5db0523687 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -135,6 +135,7 @@ include(RDMA_BuildType)
 include(RDMA_DoFixup)
 include(publish_headers)
 include(rdma_functions)
+include(rdma_man)
 
 if (NOT DEFINED ENABLE_STATIC)
   set(ENABLE_STATIC "OFF" CACHE BOOL "Produce static linking libraries as well as shared libraries.")
@@ -282,6 +283,9 @@ endif()
 # Look for Python
 FIND_PACKAGE (PythonInterp)
 
+# Look for pandoc
+FIND_PACKAGE(pandoc REQUIRED)
+
 #-------------------------
 # Find libraries
 # pthread
diff --git a/buildlib/Findpandoc.cmake b/buildlib/Findpandoc.cmake
new file mode 100644
index 00000000000000..f753165a7b6593
--- /dev/null
+++ b/buildlib/Findpandoc.cmake
@@ -0,0 +1,21 @@
+# COPYRIGHT (c) 2017 Mellanox Technologies Ltd
+# Licensed under BSD (MIT variant) or GPLv2. See COPYING.
+find_program(PANDOC_EXECUTABLE NAMES pandoc)
+
+if(PANDOC_EXECUTABLE)
+  execute_process(COMMAND "${PANDOC_EXECUTABLE}" -v
+    OUTPUT_VARIABLE _VERSION
+    RESULT_VARIABLE _VERSION_RESULT
+    ERROR_QUIET)
+
+  if(NOT _PYTHON_VERSION_RESULT)
+    string(REGEX REPLACE "^pandoc ([^\n]+)\n.*" "\\1" PANDOC_VERSION_STRING "${_VERSION}")
+  endif()
+  unset(_VERSION_RESULT)
+  unset(_VERSION)
+endif()
+
+include(FindPackageHandleStandardArgs)
+find_package_handle_standard_args(pandoc REQUIRED_VARS PANDOC_EXECUTABLE PANDOC_VERSION_STRING VERSION_VAR PANDOC_VERSION_STRING)
+
+mark_as_advanced(PANDOC_EXECUTABLE)
diff --git a/buildlib/cbuild b/buildlib/cbuild
index 29c5b4b87d9548..92672a20dafc54 100755
--- a/buildlib/cbuild
+++ b/buildlib/cbuild
@@ -98,6 +98,7 @@ class centos6(YumEnvironment):
         'libnl3-devel',
         'libudev-devel',
         'make',
+        'pandoc',
         'pkgconfig',
         'python',
         'rpm-build',
@@ -164,6 +165,7 @@ class trusty(APTEnvironment):
         'libudev-dev',
         'make',
         'ninja-build',
+        'pandoc',
         'pkg-config',
         'python',
         'valgrind',
@@ -309,6 +311,7 @@ class leap(ZypperEnvironment):
         'udev',
         'make',
         'ninja',
+        'pandoc',
         'pkg-config',
         'python3',
         'rpm-build',
diff --git a/buildlib/rdma_functions.cmake b/buildlib/rdma_functions.cmake
index 53a978e896acfa..0d858c56eee696 100644
--- a/buildlib/rdma_functions.cmake
+++ b/buildlib/rdma_functions.cmake
@@ -229,39 +229,6 @@ function(rdma_test_executable EXEC)
   set_target_properties(${EXEC} PROPERTIES RUNTIME_OUTPUT_DIRECTORY "${BUILD_BIN}")
 endfunction()
 
-# Install man pages. This deduces the section from the trailing integer in the
-# filename
-function(rdma_man_pages)
-  foreach(I ${ARGN})
-    if ("${I}" MATCHES "\\.in$")
-      string(REGEX REPLACE "^.+[.](.+)\\.in$" "\\1" MAN_SECT "${I}")
-      string(REGEX REPLACE "^(.+)\\.in$" "\\1" BASE_NAME "${I}")
-      get_filename_component(BASE_NAME "${BASE_NAME}" NAME)
-      rdma_subst_install(FILES "${I}"
-	DESTINATION "${CMAKE_INSTALL_MANDIR}/man${MAN_SECT}/"
-	RENAME "${BASE_NAME}")
-    else()
-      string(REGEX REPLACE "^.+[.](.+)$" "\\1" MAN_SECT "${I}")
-      install(FILES "${I}" DESTINATION "${CMAKE_INSTALL_MANDIR}/man${MAN_SECT}/")
-    endif()
-  endforeach()
-endfunction()
-
-# Create an alias for a man page, using a symlink.
-# Input is a list of pairs of names (MAN_PAGE ALIAS)
-# NOTE: The section must currently be the same for both.
-function(rdma_alias_man_pages)
-  list(LENGTH ARGN LEN)
-  math(EXPR LEN ${LEN}-1)
-  foreach(I RANGE 0 ${LEN} 2)
-    list(GET ARGN ${I} FROM)
-    math(EXPR I ${I}+1)
-    list(GET ARGN ${I} TO)
-    string(REGEX REPLACE "^.+[.](.+)$" "\\1" MAN_SECT ${FROM})
-    rdma_install_symlink("${FROM}" "${CMAKE_INSTALL_MANDIR}/man${MAN_SECT}/${TO}")
-  endforeach()
-endfunction()
-
 # Finalize the setup of the static libraries by copying the meta information
 # from the shared and setting up the libtool .la files.
 function(rdma_finalize_libs)
diff --git a/buildlib/rdma_man.cmake b/buildlib/rdma_man.cmake
new file mode 100644
index 00000000000000..83d7745ab0d608
--- /dev/null
+++ b/buildlib/rdma_man.cmake
@@ -0,0 +1,70 @@
+# COPYRIGHT (c) 2017 Mellanox Technologies Ltd
+# Licensed under BSD (MIT variant) or GPLv2. See COPYING.
+
+function(rdma_md_man_page SRC MAN_SECT MANFN)
+  set(OBJ "${CMAKE_CURRENT_BINARY_DIR}/${MANFN}")
+
+  add_custom_command(
+    OUTPUT "${OBJ}"
+    COMMAND "${PANDOC_EXECUTABLE}" -s -t man "${SRC}" -o "${OBJ}"
+    MAIN_DEPENDENCY "${SRC}"
+    WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}"
+    COMMENT "Creating man page ${MANFN}"
+    VERBATIM)
+
+  add_custom_target("man-${MANFN}.${MAN_SECT}" ALL DEPENDS "${OBJ}")
+  install(FILES "${OBJ}"
+    DESTINATION "${CMAKE_INSTALL_MANDIR}/man${MAN_SECT}/")
+endfunction()
+
+# Install man pages. This deduces the section from the trailing integer in the
+# filename
+function(rdma_man_pages)
+  foreach(I ${ARGN})
+    if ("${I}" MATCHES "\\.md.in$")
+      string(REGEX REPLACE "^.+[.](.+)\\.md.in$" "\\1" MAN_SECT "${I}")
+      string(REGEX REPLACE "^(.+)\\.md.in$" "\\1" BASE_NAME "${I}")
+      get_filename_component(BASE_NAME "${BASE_NAME}" NAME)
+
+      configure_file("${I}" "${CMAKE_CURRENT_BINARY_DIR}/sub-${BASE_NAME}" @ONLY)
+      rdma_md_man_page(
+	"${CMAKE_CURRENT_BINARY_DIR}/sub-${BASE_NAME}"
+	"${MAN_SECT}"
+	"${BASE_NAME}")
+    elseif ("${I}" MATCHES "\\.md$")
+      string(REGEX REPLACE "^.+[.](.+)\\.md$" "\\1" MAN_SECT "${I}")
+      string(REGEX REPLACE "^(.+)\\.md$" "\\1" BASE_NAME "${I}")
+      get_filename_component(BASE_NAME "${BASE_NAME}" NAME)
+
+      rdma_md_man_page(
+	"${I}"
+	"${MAN_SECT}"
+	"${BASE_NAME}")
+    elseif ("${I}" MATCHES "\\.in$")
+      string(REGEX REPLACE "^.+[.](.+)\\.in$" "\\1" MAN_SECT "${I}")
+      string(REGEX REPLACE "^(.+)\\.in$" "\\1" BASE_NAME "${I}")
+      get_filename_component(BASE_NAME "${BASE_NAME}" NAME)
+      rdma_subst_install(FILES "${I}"
+	DESTINATION "${CMAKE_INSTALL_MANDIR}/man${MAN_SECT}/"
+	RENAME "${BASE_NAME}")
+    else()
+      string(REGEX REPLACE "^.+[.](.+)$" "\\1" MAN_SECT "${I}")
+      install(FILES "${I}" DESTINATION "${CMAKE_INSTALL_MANDIR}/man${MAN_SECT}/")
+    endif()
+  endforeach()
+endfunction()
+
+# Create an alias for a man page, using a symlink.
+# Input is a list of pairs of names (MAN_PAGE ALIAS)
+# NOTE: The section must currently be the same for both.
+function(rdma_alias_man_pages)
+  list(LENGTH ARGN LEN)
+  math(EXPR LEN ${LEN}-1)
+  foreach(I RANGE 0 ${LEN} 2)
+    list(GET ARGN ${I} FROM)
+    math(EXPR I ${I}+1)
+    list(GET ARGN ${I} TO)
+    string(REGEX REPLACE "^.+[.](.+)$" "\\1" MAN_SECT ${FROM})
+    rdma_install_symlink("${FROM}" "${CMAKE_INSTALL_MANDIR}/man${MAN_SECT}/${TO}")
+  endforeach()
+endfunction()
diff --git a/debian/control b/debian/control
index 451b379f01ae92..cfcdcc4e22884d 100644
--- a/debian/control
+++ b/debian/control
@@ -13,6 +13,7 @@ Build-Depends: cmake (>= 2.8.11),
                libsystemd-dev,
                libudev-dev,
                ninja-build,
+               pandoc,
                pkg-config,
                python,
                valgrind [!alpha !armel !hppa !m68k !powerpcspe !sh4 !sparc64 !x32]
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH rdma-core] Support writing man pages in MarkDown

[Hefty, Sean]

> Since this may be a problem, I am thinking about following the
> libfabric approach and either including the precompiled man pages in
> the source tree, or just providing a framework for downstream to add-
> on precompiled files (with a cbuild command to make them) Looking for
> feedback on what path is best..

FWIW, one of the reasons libfabric went with pre-compiled man pages was to include a date for the last edit.  A service automatically updates the date when updating the page.

There's probably some way to get that in the markdown directly.

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core] Support writing man pages in MarkDown

[Jason Gunthorpe]

On Tue, Dec 12, 2017 at 11:11:15PM +0000, Hefty, Sean wrote:
> > Since this may be a problem, I am thinking about following the
> > libfabric approach and either including the precompiled man pages in
> > the source tree, or just providing a framework for downstream to add-
> > on precompiled files (with a cbuild command to make them) Looking for
> > feedback on what path is best..
> 
> FWIW, one of the reasons libfabric went with pre-compiled man pages
> was to include a date for the last edit.  A service automatically
> updates the date when updating the page.

This isn't going to be so useful here since all the markdown pages
were created today..

What I did was to copy the date from the original man page over to the
markdown header:

date: 2015-01-29

So the date visible in the man page doesn't change from what it was
before (which in turn is a kind of meaningless value these days since
we never bothered to keep it up to date in most cases)

Other projects don't bother with the date, for instance systemd just
drops in in the package version number in that space, which actually
seems more usefull to me.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core] Support writing man pages in MarkDown

[Yishai Hadas]

On 12/13/2017 12:59 AM, Jason Gunthorpe wrote:
> The pages are converted during the build into *roff mdoc using
> the pandoc tool.
> 
> Signed-off-by: Jason Gunthorpe <jgg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> ---
>   .travis.yml                   |  1 +
>   CMakeLists.txt                |  4 +++
>   buildlib/Findpandoc.cmake     | 21 +++++++++++++
>   buildlib/cbuild               |  3 ++
>   buildlib/rdma_functions.cmake | 33 --------------------
>   buildlib/rdma_man.cmake       | 70 +++++++++++++++++++++++++++++++++++++++++++
>   debian/control                |  1 +
>   7 files changed, 100 insertions(+), 33 deletions(-)
>   create mode 100644 buildlib/Findpandoc.cmake
>   create mode 100644 buildlib/rdma_man.cmake
> 
> This is a very big series, I am only sending the first patch to the
> mailing list. The whole series is on github here:
> 
> https://github.com/linux-rdma/rdma-core/pull/274
> 
> Overall, I'd like to see the man pages be more editable and viewable
> when in patch format. Longer term I'd like to see them all rendered to
> HTML and hosted on the web as libfabric and systemd are doing
> successfully. While some of this can be done with the classic roff
> format, it feels very painful.
> 

This makes sense to me.

> Markdown was selected because we are already using it successfully for
> other documentation, and libfabric has already demonstrated using it
> for man pages. github will natively render the markdown documentation
> internally for instance:
>

Makes sense as well.

> https://github.com/jgunthorpe/rdma-plumbing/blob/ea581c2d3d99d69083a8f4eaa9e55ae7ae233246/libibverbs/man/ibv_alloc_mw.3.md
> 
> Although some places are using features specific to the 'pandoc'
> flavour of markdown, github still renders them in a readable format.
> 
> The next most popular choice is docbook XML, which I suspect would not
> be well liked (and it is very baroque as well, just in a more modern
> way)
> 
> The actual conversion of the man pages was done largely by script,
> with some manual fixes and post-inspection. There doesn't seem to be a
> useful mdoc to markdown conversion script, so I built a quick and
> dirty one that works on our .3 man page style.
> 
> The converter is here:
> 
> https://github.com/jgunthorpe/rdma-plumbing/blob/migration-tooling/buildlib/man2md.py
> 
> I'm no mdoc expert, but it sure looks to me like there are many markup
> problems in our man pages, even if they happen to render mostly properly.
>  > Overall the conversion to markdown is very good in terms of resulting
> 'man ibv_foo' display with a few notable variances:
>   - The 'synopsis' is shifted right by 8 spaces and isn't bolded in
>     quite the same way
>   - I ran all the C code through clang-format and asked it fit it to
>     the standard 80 col terminal. Some of the results are 'interesting'
>     to say the least. Others are much better than before.
>   - Some of the hypenation choices are a little different
> 

Looked at few generated man pages, there are places that it looks *ugly* 
comparing to the original ones. Especially talking on the 'C' structures 
that were wrapped around the 80 column. (see ibv_query_device). There 
are cases that even in the same man page there are lines larger than 80 
while others are not. We can't just drop the 'C' structures as in many 
places few fields may include some meaningful information, need to go 
per man page and consider how we would really like to see it once moving 
to the new format.

> pandoc is fairly widely available in the distros with the notable
> problem that centos 6 and 7 can only get it from EPEL.
> 

There might be even a worse case scenario where pandoc may not exist as 
a package to be installed, for example how about SLES OS ?

> Since this may be a problem, I am thinking about following the
> libfabric approach and either including the precompiled man pages in
> the source tree, or just providing a framework for downstream to
> add-on precompiled files (with a cbuild command to make them)
> Looking for feedback on what path is best..
> 

An approach to have an option via the cbuild to generate them on demand 
makes sense to me. It should use the docker/containers mechanism to 
enable that. Upon regular compile/build the already generated man pages 
will be taken from the tree, upon a man page change the cbuild 
environment should be used and both the man and the .md files will be 
pushed into the tree.

However, need to consider below notes:
1) Need to have a clear documentation to describe the required steps to 
do that. Man pages may be changed by many people and as such, the 
process to generate them must be fully clear and easy to use.

2) As the tree will hold both the 'md' files and the man pages, someone 
might forget to use the pandoc and upload only a change in a man page, 
this should be prevented by Travis or other checker script.

3) How about cases that the a man page will be generated by pandoc but 
someone will do manual changes to have it nicer/cleaner ?
This might end-up having man page which doesn't fit to its source that 
may bring an issue on the next generation by someone else.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core] Support writing man pages in MarkDown

[Benjamin Drung]

> > Am Sonntag, den 24.12.2017, 14:03 +0200 schrieb Yishai Hadas:
> On 12/13/2017 12:59 AM, Jason Gunthorpe wrote:
> > > pandoc is fairly widely available in the distros with the notable
> > problem that centos 6 and 7 can only get it from EPEL.
> > 
> 
> There might be even a worse case scenario where pandoc may not exist
> as 
> a package to be installed, for example how about SLES OS ?

The release tarball could contain pre-built man page. So in case the OS
does provide pandoc to regenerate the man pages, the pre-built can be
used.

> > Since this may be a problem, I am thinking about following the
> > libfabric approach and either including the precompiled man pages
> > in
> > the source tree, or just providing a framework for downstream to
> > add-on precompiled files (with a cbuild command to make them)
> > Looking for feedback on what path is best..
> > 

Please do not place generated files in the git repository. 

-- 
Benjamin Drung
System Developer
Debian & Ubuntu Developer

ProfitBricks GmbH
Greifswalder Str. 207
D - 10405 Berlin

Email: benjamin.drung-EIkl63zCoXaH+58JC4qpiA@public.gmane.org
URL: https://www.profitbricks.de

Sitz der Gesellschaft: Berlin
Registergericht: Amtsgericht Charlottenburg, HRB 125506 B
Geschäftsführer: Achim Weiss, Matthias Steinberg
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core] Support writing man pages in MarkDown

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1851 bytes --]

On Wed, Dec 27, 2017 at 10:53:53AM +0100, Benjamin Drung wrote:
> > > Am Sonntag, den 24.12.2017, 14:03 +0200 schrieb Yishai Hadas:
> > On 12/13/2017 12:59 AM, Jason Gunthorpe wrote:
> > > > pandoc is fairly widely available in the distros with the notable
> > > problem that centos 6 and 7 can only get it from EPEL.
> > >
> >
> > There might be even a worse case scenario where pandoc may not exist
> > as
> > a package to be installed, for example how about SLES OS ?
>
> The release tarball could contain pre-built man page. So in case the OS
> does provide pandoc to regenerate the man pages, the pre-built can be
> used.

Why should OS be dependent on pandoc? IMHO all releases should contain
pre-built man pages and they should be used without need to regenerate
them at all.

>
> > > Since this may be a problem, I am thinking about following the
> > > libfabric approach and either including the precompiled man pages
> > > in
> > > the source tree, or just providing a framework for downstream to
> > > add-on precompiled files (with a cbuild command to make them)
> > > Looking for feedback on what path is best..
> > >
>
> Please do not place generated files in the git repository.
>
> --
> Benjamin Drung
> System Developer
> Debian & Ubuntu Developer
>
> ProfitBricks GmbH
> Greifswalder Str. 207
> D - 10405 Berlin
>
> Email: benjamin.drung-EIkl63zCoXaH+58JC4qpiA@public.gmane.org
> URL: https://www.profitbricks.de
>
> Sitz der Gesellschaft: Berlin
> Registergericht: Amtsgericht Charlottenburg, HRB 125506 B
> Geschäftsführer: Achim Weiss, Matthias Steinberg
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH rdma-core] Support writing man pages in MarkDown

[Benjamin Drung]

[-- Attachment #1: Type: text/plain, Size: 1612 bytes --]

Am Mittwoch, den 27.12.2017, 13:13 +0200 schrieb Leon Romanovsky:
> On Wed, Dec 27, 2017 at 10:53:53AM +0100, Benjamin Drung wrote:
> > > > Am Sonntag, den 24.12.2017, 14:03 +0200 schrieb Yishai Hadas:
> > > 
> > > On 12/13/2017 12:59 AM, Jason Gunthorpe wrote:
> > > > > pandoc is fairly widely available in the distros with the
> > > > > notable
> > > > 
> > > > problem that centos 6 and 7 can only get it from EPEL.
> > > > 
> > > 
> > > There might be even a worse case scenario where pandoc may not
> > > exist
> > > as
> > > a package to be installed, for example how about SLES OS ?
> > 
> > The release tarball could contain pre-built man page. So in case
> > the OS
> > does provide pandoc to regenerate the man pages, the pre-built can
> > be
> > used.
> 
> Why should OS be dependent on pandoc? IMHO all releases should
> contain
> pre-built man pages and they should be used without need to
> regenerate
> them at all.

Debian's rule is to build everything from source where the source is in
the preferred format for modification. So we will drop the pre-build
man pages and build them from source - even if that would not be
needed. This is useful in case we have to modify it (e.g. fix typos).

-- 
Benjamin Drung
System Developer
Debian & Ubuntu Developer

ProfitBricks GmbH
Greifswalder Str. 207
D - 10405 Berlin

Email: benjamin.drung-EIkl63zCoXaH+58JC4qpiA@public.gmane.org
URL: https://www.profitbricks.de

Sitz der Gesellschaft: Berlin
Registergericht: Amtsgericht Charlottenburg, HRB 125506 B
Geschäftsführer: Achim Weiss, Matthias Steinberg

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH rdma-core] Support writing man pages in MarkDown

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1798 bytes --]

On Wed, Dec 27, 2017 at 12:27:01PM +0100, Benjamin Drung wrote:
> Am Mittwoch, den 27.12.2017, 13:13 +0200 schrieb Leon Romanovsky:
> > On Wed, Dec 27, 2017 at 10:53:53AM +0100, Benjamin Drung wrote:
> > > > > Am Sonntag, den 24.12.2017, 14:03 +0200 schrieb Yishai Hadas:
> > > >
> > > > On 12/13/2017 12:59 AM, Jason Gunthorpe wrote:
> > > > > > pandoc is fairly widely available in the distros with the
> > > > > > notable
> > > > >
> > > > > problem that centos 6 and 7 can only get it from EPEL.
> > > > >
> > > >
> > > > There might be even a worse case scenario where pandoc may not
> > > > exist
> > > > as
> > > > a package to be installed, for example how about SLES OS ?
> > >
> > > The release tarball could contain pre-built man page. So in case
> > > the OS
> > > does provide pandoc to regenerate the man pages, the pre-built can
> > > be
> > > used.
> >
> > Why should OS be dependent on pandoc? IMHO all releases should
> > contain
> > pre-built man pages and they should be used without need to
> > regenerate
> > them at all.
>
> Debian's rule is to build everything from source where the source is in
> the preferred format for modification. So we will drop the pre-build
> man pages and build them from source - even if that would not be
> needed. This is useful in case we have to modify it (e.g. fix typos).

Thanks for the explanation.

>
> --
> Benjamin Drung
> System Developer
> Debian & Ubuntu Developer
>
> ProfitBricks GmbH
> Greifswalder Str. 207
> D - 10405 Berlin
>
> Email: benjamin.drung-EIkl63zCoXaH+58JC4qpiA@public.gmane.org
> URL: https://www.profitbricks.de
>
> Sitz der Gesellschaft: Berlin
> Registergericht: Amtsgericht Charlottenburg, HRB 125506 B
> Geschäftsführer: Achim Weiss, Matthias Steinberg



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH rdma-core] Support writing man pages in MarkDown

[Benjamin Drung]

[-- Attachment #1: Type: text/plain, Size: 2072 bytes --]

Am Mittwoch, den 27.12.2017, 14:20 +0200 schrieb Leon Romanovsky:
> On Wed, Dec 27, 2017 at 12:27:01PM +0100, Benjamin Drung wrote:
> > Am Mittwoch, den 27.12.2017, 13:13 +0200 schrieb Leon Romanovsky:
> > > On Wed, Dec 27, 2017 at 10:53:53AM +0100, Benjamin Drung wrote:
> > > > > > Am Sonntag, den 24.12.2017, 14:03 +0200 schrieb Yishai
> > > > > > Hadas:
> > > > > 
> > > > > On 12/13/2017 12:59 AM, Jason Gunthorpe wrote:
> > > > > > > pandoc is fairly widely available in the distros with the
> > > > > > > notable
> > > > > > 
> > > > > > problem that centos 6 and 7 can only get it from EPEL.
> > > > > > 
> > > > > 
> > > > > There might be even a worse case scenario where pandoc may
> > > > > not
> > > > > exist
> > > > > as
> > > > > a package to be installed, for example how about SLES OS ?
> > > > 
> > > > The release tarball could contain pre-built man page. So in
> > > > case
> > > > the OS
> > > > does provide pandoc to regenerate the man pages, the pre-built
> > > > can
> > > > be
> > > > used.
> > > 
> > > Why should OS be dependent on pandoc? IMHO all releases should
> > > contain
> > > pre-built man pages and they should be used without need to
> > > regenerate
> > > them at all.
> > 
> > Debian's rule is to build everything from source where the source
> > is in
> > the preferred format for modification. So we will drop the pre-
> > build
> > man pages and build them from source - even if that would not be
> > needed. This is useful in case we have to modify it (e.g. fix
> > typos).
> 
> Thanks for the explanation.

FYI: Debian has a wiki page dedicated for upstream developers:
https://wiki.debian.org/UpstreamGuide

-- 
Benjamin Drung
System Developer
Debian & Ubuntu Developer

ProfitBricks GmbH
Greifswalder Str. 207
D - 10405 Berlin

Email: benjamin.drung-EIkl63zCoXaH+58JC4qpiA@public.gmane.org
URL: https://www.profitbricks.de

Sitz der Gesellschaft: Berlin
Registergericht: Amtsgericht Charlottenburg, HRB 125506 B
Geschäftsführer: Achim Weiss, Matthias Steinberg

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH rdma-core] Support writing man pages in MarkDown

[Jason Gunthorpe]

On Wed, Dec 27, 2017 at 10:53:53AM +0100, Benjamin Drung wrote:
> > > Am Sonntag, den 24.12.2017, 14:03 +0200 schrieb Yishai Hadas:
> > On 12/13/2017 12:59 AM, Jason Gunthorpe wrote:
> > > > pandoc is fairly widely available in the distros with the notable
> > > problem that centos 6 and 7 can only get it from EPEL.
> > > 
> > 
> > There might be even a worse case scenario where pandoc may not exist
> > as 
> > a package to be installed, for example how about SLES OS ?
> 
> The release tarball could contain pre-built man page. So in case the OS
> does provide pandoc to regenerate the man pages, the pre-built can be
> used.

Yes, this is my basic thinking. If pandoc is installed then the
pre-built can be ignored, otherwise they are used so the build can
succeed.

I was thinking of a scheme like
 buildlib/man-cache/<sha1 of .md file>

So if the .md files are changed for any reason then the build will
fail.

> Please do not place generated files in the git repository. 

Realistically, I don't know if this is possible... Too many developers
are using quite old distros these days, and they still have to be
able to build..

I guess we just turn off man page generation in ./build.sh ?

But, how will 'cubild pkg centos6' work?

The release .tar.gz files are created by travis
(buildilb/github-release) so it would not be too hard to
have it also build the cache.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core] Support writing man pages in MarkDown

[Benjamin Drung]

Am Mittwoch, den 27.12.2017, 10:44 -0700 schrieb Jason Gunthorpe:
> On Wed, Dec 27, 2017 at 10:53:53AM +0100, Benjamin Drung wrote:
> > > > Am Sonntag, den 24.12.2017, 14:03 +0200 schrieb Yishai Hadas:
> > > 
> > > On 12/13/2017 12:59 AM, Jason Gunthorpe wrote:
> > > > > pandoc is fairly widely available in the distros with the
> > > > > notable
> > > > 
> > > > problem that centos 6 and 7 can only get it from EPEL.
> > > > 
> > > 
> > > There might be even a worse case scenario where pandoc may not
> > > exist
> > > as 
> > > a package to be installed, for example how about SLES OS ?
> > 
> > The release tarball could contain pre-built man page. So in case
> > the OS
> > does provide pandoc to regenerate the man pages, the pre-built can
> > be
> > used.
> 
> Yes, this is my basic thinking. If pandoc is installed then the
> pre-built can be ignored, otherwise they are used so the build can
> succeed.
> 
> I was thinking of a scheme like
>  buildlib/man-cache/<sha1 of .md file>
>
> So if the .md files are changed for any reason then the build will
> fail.

None of the projects that I know build a cache using checksums. The
projects are just fine with not failing when the source changed, but
the man pages are not updated.

> > Please do not place generated files in the git repository. 
> 
> Realistically, I don't know if this is possible... Too many
> developers
> are using quite old distros these days, and they still have to be
> able to build..
> 
> I guess we just turn off man page generation in ./build.sh ?

A configuration option would be good that allows one to either enable
or disable (re)generating the man pages. By default, cmake would check
if pandoc is available and set the value based on that.

> But, how will 'cubild pkg centos6' work?
> 
> The release .tar.gz files are created by travis
> (buildilb/github-release) so it would not be too hard to
> have it also build the cache.

Yes, let travis build the man pages for the release tarballs.

-- 
Benjamin Drung
System Developer
Debian & Ubuntu Developer

ProfitBricks GmbH
Greifswalder Str. 207
D - 10405 Berlin

Email: benjamin.drung-EIkl63zCoXaH+58JC4qpiA@public.gmane.org
URL: https://www.profitbricks.de

Sitz der Gesellschaft: Berlin
Registergericht: Amtsgericht Charlottenburg, HRB 125506 B
Geschäftsführer: Achim Weiss, Matthias Steinberg
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html