Re: Despite `-DCMAKE_INSTALL_LIBDIR=/usr/lib` libraries installed to `/usr/usr/lib`

[Alan Maguire <alan.maguire@oracle.com>]

On 16/04/2025 11:39, Paul Menzel wrote:
> [Cc: +Ben, +Alan]
> 
> Dear pahole folks,
> 
> 
> To build pahole 1.30, I configure it with CMake 3.25.2
> 
>     cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_INSTALL_LIBDIR=/usr/lib -
> DCMAKE_COLOR_MAKEFILE=OFF -DCMAKE_VERBOSE_MAKEFILE=ON -
> DCMAKE_BUILD_TYPE=RelWithDebInfo -DLIBBPF_EMBEDDED=OFF /dev/shm/bee-
> pmenzel/pahole/pahole-1.30-1/source
> 
> and then build it with
> 
>     make -j300
> 
> and install it with
> 
>     make -j300 install DESTDIR=/dev/shm/bee-pmenzel/pahole/
> pahole-1.30-1/image
> 
> Unfortunately, the shared libraries are installed int `/usr` as the
> prefix seems to be prepended despite LIBDIR containing the full path:
> 
>     -- Installing: /dev/shm/bee-pmenzel/pahole/pahole-1.30-1/image/usr/
> usr/lib/libdwarves.so.1.0.0
>     -- Installing: /dev/shm/bee-pmenzel/pahole/pahole-1.30-1/image/usr/
> usr/lib/libdwarves.so.1
>     -- Installing: /dev/shm/bee-pmenzel/pahole/pahole-1.30-1/image/usr/
> usr/lib/libdwarves.so
>     -- Up-to-date: /dev/shm/bee-pmenzel/pahole/pahole-1.30-1/image/usr/
> usr/lib/libdwarves.so.1.0.0
>     -- Up-to-date: /dev/shm/bee-pmenzel/pahole/pahole-1.30-1/image/usr/
> usr/lib/libdwarves.so.1
>     -- Up-to-date: /dev/shm/bee-pmenzel/pahole/pahole-1.30-1/image/usr/
> usr/lib/libdwarves.so
>     -- Installing: /dev/shm/bee-pmenzel/pahole/pahole-1.30-1/image/usr/
> usr/lib/libdwarves_emit.so.1.0.0
>     -- Installing: /dev/shm/bee-pmenzel/pahole/pahole-1.30-1/image/usr/
> usr/lib/libdwarves_emit.so.1
>     -- Set runtime path of "/dev/shm/bee-pmenzel/pahole/pahole-1.30-1/
> image/usr/usr/lib/libdwarves_emit.so.1.0.0" to ""
>     -- Installing: /dev/shm/bee-pmenzel/pahole/pahole-1.30-1/image/usr/
> usr/lib/libdwarves_emit.so
>     -- Installing: /dev/shm/bee-pmenzel/pahole/pahole-1.30-1/image/usr/
> usr/lib/libdwarves_reorganize.so.1.0.0
>     -- Installing: /dev/shm/bee-pmenzel/pahole/pahole-1.30-1/image/usr/
> usr/lib/libdwarves_reorganize.so.1
>     -- Set runtime path of "/dev/shm/bee-pmenzel/pahole/pahole-1.30-1/
> image/usr/usr/lib/libdwarves_reorganize.so.1.0.0" to ""
>     -- Installing: /dev/shm/bee-pmenzel/pahole/pahole-1.30-1/image/usr/
> usr/lib/libdwarves_reorganize.so
> 
> Looking quickly into `CMakeLists.txt` there is
> 
>     […]
>     # Use the standard library installation directory
>     include(GNUInstallDirs)
>     set(CMAKE_INSTALL_LIBDIR "lib" CACHE STRING "libdir name")
>     […]
>     if (NOT DEFINED LIB_INSTALL_DIR)
>         set(LIB_INSTALL_DIR
> "${EXEC_INSTALL_PREFIX}${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}")
>     endif()
>     […]
>     install(TARGETS codiff ctracer dtagnames pahole pdwtags
>                 pfunct pglobal prefcnt scncopy syscse RUNTIME DESTINATION
>                     ${CMAKE_INSTALL_PREFIX}/bin)
>     install(TARGETS dwarves LIBRARY DESTINATION ${LIB_INSTALL_DIR}
> ARCHIVE DESTINATION ${LIB_INSTALL_DIR})
>     install(TARGETS dwarves dwarves_emit dwarves_reorganize LIBRARY
> DESTINATION ${LIB_INSTALL_DIR} ARCHIVE DESTINATION ${LIB_INSTALL_DIR})
>     install(FILES dwarves.h dwarves_emit.h dwarves_reorganize.h
>                   dutil.h gobuffer.h list.h rbtree.h
>                   btf_encoder.h config.h ctf.h
>                   elfcreator.h elf_symtab.h hash.h libctf.h
>             DESTINATION ${CMAKE_INSTALL_PREFIX}/include/dwarves/)
>     install(FILES man-pages/pahole.1 DESTINATION
> ${CMAKE_INSTALL_PREFIX}/share/man/man1/)
> 
> Passing `-DLIB_INSTALL_DIR=/usr/lib` to `cmake` works around this. It
> looks like it was supposed to be fixed by Ben’s commit a2abd9482162
> (CMakeList.txt: Respect CMAKE_INSTALL_LIBDIR) [1], but it doesn’t work
> for me.
> 
> 

hi Paul, thanks for the report! I'm not sure that the behaviour you
describe is unexpected; [1] says:

"If make install is invoked or INSTALL is built, this directory is
prepended onto all install directories."


I couldn't find any official documentation that described the
interaction between CMAKE_INSTALL_LIBDIR and CMAKE_INSTALL_PREFIX but it
seems reasonable to me that the prefix from CMAKE_INSTALL_PREFIX applies
to CMAKE_INSTALL_LIBDIR given that the latter can specify relative
paths. Commit a2abd9482162 prepends CMAKE_INSTALL_PREFIX explicitly:

+if (NOT DEFINED LIB_INSTALL_DIR)
+    set(LIB_INSTALL_DIR
"${EXEC_INSTALL_PREFIX}${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}")
+endif()

So you could either update your cmake options to specify a path relative
to CMAKE_INSTALL_PREFIX:

     cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_INSTALL_LIBDIR=lib
-DCMAKE_COLOR_MAKEFILE=OFF -DCMAKE_VERBOSE_MAKEFILE=ON -
DCMAKE_BUILD_TYPE=RelWithDebInfo -DLIBBPF_EMBEDDED=OFF /dev/shm/bee-
pmenzel/pahole/pahole-1.30-1/source

That seems to work for me - I get the install into /usr/lib .
Alternatively you could use LIB_INSTALL_DIR, i.e.

     cmake -DCMAKE_INSTALL_PREFIX=/usr -DLIB_INSTALL_DIR=/usr/lib
-DCMAKE_COLOR_MAKEFILE=OFF -DCMAKE_VERBOSE_MAKEFILE=ON -
DCMAKE_BUILD_TYPE=RelWithDebInfo -DLIBBPF_EMBEDDED=OFF /dev/shm/bee-
pmenzel/pahole/pahole-1.30-1/source

...if you want to specify an absolute path.

Are there other projects that use different conventions around the
CMAKE_INSTALL_PREFIX/CMAKE_INSTALL_LIBDIR relationship perhaps? I'm not
sure on the conventions here myself to be honest. Thanks!


[1] https://cmake.org/cmake/help/latest/variable/CMAKE_INSTALL_PREFIX.html