Re: [PATCH v1 1/6] doc: add hugepage mapping details

[Thomas Monjalon <thomas@monjalon.net>]

Thanks for the nice addition to the documentation, this is really needed.
Some comments below.

17/01/2022 09:07, Dmitry Kozlyuk:
> --- a/doc/guides/prog_guide/env_abstraction_layer.rst
> +++ b/doc/guides/prog_guide/env_abstraction_layer.rst
> -    Memory reservations done using the APIs provided by rte_malloc are also backed by pages from the hugetlbfs filesystem.
> +    Memory reservations done using the APIs provided by rte_malloc are also backed by hugepages.

Should we mention except if --no-huge is used?

> +Hugepage Mapping
> +^^^^^^^^^^^^^^^^
> +
> +Below is an overview of methods used for each OS to obtain hugepages,
> +explaining why certain limitations and options exist in EAL.
> +See the user guide for a specific OS for configuration details.
> +
> +FreeBSD uses ``contigmem`` kernel module
> +to reserve a fixed number of hugepages at system start,
> +which are mapped by EAL at initialization using a specific ``sysctl()``.
> +
> +Windows EAL allocates hugepages from the OS as needed using Win32 API,
> +so available amount depends on the system load.
> +It uses ``virt2phys`` kernel module to obtain physical addresses,
> +unless running in IOVA-as-VA mode (e.g. forced with ``--iova-mode=va``).
> +
> +Linux implements a variety of methods:
> +
> +* mapping each hugepage from its own file in hugetlbfs;
> +* mapping multiple hugepages from a shared file in hugetlbfs;
> +* anonymous mapping.
> +
> +Mapping hugepages from files in hugetlbfs is essential for multi-process,
> +because secondary processes need to map the same hugepages.
> +EAL creates files like ``rtemap_0``
> +in directories specified with ``--huge-dir`` option
> +(or in the mount point for a specific hugepage size).
> +The ``rtemap_`` prefix can be changed using ``--file-prefix``.
> +This may be needed for running multiple primary processes
> +that share a hugetlbfs mount point.
> +Each backing file by default corresponds to one hugepage,
> +it is opened and locked for the entire time the hugepage is used.
> +See :ref:`segment-file-descriptors` section
> +on how the number of open backing file descriptors can be reduced.
> +
> +Backing files may persist after the corresponding hugepage is freed
> +and even after the application terminates,
> +reducing the number of hugepages available to other processes.
> +EAL removes existing files at startup
> +and can remove newly created files before mapping them with ``--huge-unlink``.

This sentence require more explanations, as it is not clear when and why.

> +However, since it disables multi-process anyway,
> +using anonymous mapping (``--in-memory``) is recommended instead.
> +
> +:ref:`EAL memory allocator <malloc>` relies on hugepages being zero-filled.
> +Hugepages are cleared by the kernel when a file in hugetlbfs or its part
> +is mapped for the first time system-wide
> +to prevent data leaks from previous users of the same hugepage.
> +EAL ensures this behavior by removing existing backing files at startup
> +and by recreating them before opening for mapping (as a precaution).
> +
> +Anonymous mapping does not allow multi-process architecture,
> +but it is free of filename conflicts and leftover files on hugetlbfs.

It is also easier to run as non-root.

> +If memfd_create(2) is supported both at build and run time,
> +DPDK memory manager can provide file descriptors for memory segments,
> +which are required for VirtIO with vhost-user backend.
> +This means open file descriptor issues may also affect this mode,
> +with the same solution.

This is not clear. Which issues? Which mode? Which solution?