* [RFC PATCH v1 0/5] Add threading support to e2fsprogs
@ 2021-01-14 0:27 Saranya Muruganandam
2021-01-14 0:27 ` [RFC PATCH v1 1/5] Add configure and build support for the pthreads library Saranya Muruganandam
` (5 more replies)
0 siblings, 6 replies; 7+ messages in thread
From: Saranya Muruganandam @ 2021-01-14 0:27 UTC (permalink / raw)
To: linux-ext4; +Cc: tytso, Saranya Muruganandam
This patch set adds the infrastructure to support threading to
libext2fs. It makes the unix_io I/O Manager thread-aware. Wang's
parallel bitmap code has been adapted to use the new threading
infrastructure.
The code has been tested with TSAN and ASAN built into gcc 10.2:
configure 'CFLAGS=-g -fsanitize=thread' 'LDFLAGS=-fsanitize=thread'
make clean ; make -j16 ; make -j16 check
configure 'CFLAGS=-g -fsanitize=address' 'LDFLAGS=-fsanitize=address'
make clean ; make -j16 ; make -j16 check
As I(tytso) needed to excerpt out some of the changes to generated patches in
"Add configure and build support for the pthreads", the full patch
series can be found in git:
git fetch https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git pthreads
Changes with V1:
Fix review remarks for "ext2fs: parallel bitmap loading".
Tested stat_mutex performance on instance with 60 CPUs and fragmented 3TB Local SSD.
No noticable contention seem for the stat_mutex.
Theodore Ts'o (4):
Add configure and build support for the pthreads library
libext2fs: add threading support to the I/O manager abstraction
libext2fs: allow the unix_io manager's cache to be disabled and
re-enabled
Enable threaded support for e2fsprogs' applications.
Wang Shilong (1):
ext2fs: parallel bitmap loading
MCONFIG.in | 12 +-
aclocal.m4 | 560 ++++++++++++++++++++++++++--------------
configure | 213 ++++++++++++---
configure.ac | 24 ++
debugfs/debugfs.c | 6 +-
e2fsck/unix.c | 2 +-
lib/config.h.in | 83 +++++-
lib/ext2fs/ext2_io.h | 3 +
lib/ext2fs/ext2fs.h | 9 +
lib/ext2fs/openfs.c | 2 +
lib/ext2fs/rw_bitmaps.c | 332 ++++++++++++++++++++----
lib/ext2fs/test_io.c | 6 +-
lib/ext2fs/undo_io.c | 2 +
lib/ext2fs/unix_io.c | 156 +++++++++--
misc/dumpe2fs.c | 2 +-
misc/e2freefrag.c | 2 +-
misc/e2fuzz.c | 4 +-
misc/e2image.c | 3 +-
misc/fuse2fs.c | 3 +-
misc/tune2fs.c | 3 +-
resize/main.c | 2 +-
21 files changed, 1093 insertions(+), 336 deletions(-)
--
2.30.0.284.gd98b1dd5eaa7-goog
^ permalink raw reply [flat|nested] 7+ messages in thread
* [RFC PATCH v1 1/5] Add configure and build support for the pthreads library
2021-01-14 0:27 [RFC PATCH v1 0/5] Add threading support to e2fsprogs Saranya Muruganandam
@ 2021-01-14 0:27 ` Saranya Muruganandam
2021-01-14 0:27 ` [RFC PATCH v1 2/5] libext2fs: add threading support to the I/O manager abstraction Saranya Muruganandam
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Saranya Muruganandam @ 2021-01-14 0:27 UTC (permalink / raw)
To: linux-ext4; +Cc: tytso
From: Theodore Ts'o <tytso@mit.edu>
Support for pthreads can be forcibly disabled by passing
"--without-pthread" to the configure script.
The actual changes in this commit are in configure.ac and MCONFIG.in;
the other files were generated as a result of running aclocal,
autoconf, and autoheader on a Debian testing system.
Note: the autoconf-archive package must now be installed before
rerunning aclocal, to supply the AX_PTHREAD macro.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
MCONFIG.in | 12 +-
aclocal.m4 | 560 +++++++++++++++++++++++++++++++-----------------
configure | 213 ++++++++++++++----
configure.ac | 24 +++
lib/config.h.in | 83 +++++--
5 files changed, 636 insertions(+), 256 deletions(-)
diff --git a/MCONFIG.in b/MCONFIG.in
index 0598f21b..69f30674 100644
--- a/MCONFIG.in
+++ b/MCONFIG.in
@@ -86,15 +86,17 @@ SYSTEMD_SYSTEM_UNIT_DIR = @systemd_system_unit_dir@
SANITIZER_CFLAGS = @lto_cflags@ @ubsan_cflags@ @addrsan_cflags@ @threadsan_cflags@
SANITIZER_LDFLAGS = @lto_ldflags@ @ubsan_ldflags@ @addrsan_ldflags@ @threadsan_ldflags@
-CC = @CC@
+CC = @PTHREAD_CC@
BUILD_CC = @BUILD_CC@
+PTHREAD_CFLAGS = @PTHREAD_CFLAGS@
+PTHREAD_LIBS = @PTHREAD_LIBS@
CFLAGS = @CFLAGS@
CFLAGS_SHLIB = @CFLAGS_SHLIB@
CFLAGS_STLIB = @CFLAGS_STLIB@
CPPFLAGS = @INCLUDES@
-ALL_CFLAGS = $(CPPFLAGS) $(SANITIZER_CFLAGS) $(CFLAGS) $(CFLAGS_WARN) @DEFS@ $(LOCAL_CFLAGS)
-ALL_CFLAGS_SHLIB = $(CPPFLAGS) $(SANITIZER_CFLAGS) $(CFLAGS_SHLIB) $(CFLAGS_WARN) @DEFS@ $(LOCAL_CFLAGS)
-ALL_CFLAGS_STLIB = $(CPPFLAGS) $(SANITIZER_CFLAGS) $(CFLAGS_STLIB) $(CFLAGS_WARN) @DEFS@ $(LOCAL_CFLAGS)
+ALL_CFLAGS = $(CPPFLAGS) $(SANITIZER_CFLAGS) $(CFLAGS) $(PTHREAD_CFLAGS) $(CFLAGS_WARN) @DEFS@ $(LOCAL_CFLAGS)
+ALL_CFLAGS_SHLIB = $(CPPFLAGS) $(SANITIZER_CFLAGS) $(CFLAGS_SHLIB) $(PTHREAD_CFLAGS) $(CFLAGS_WARN) @DEFS@ $(LOCAL_CFLAGS)
+ALL_CFLAGS_STLIB = $(CPPFLAGS) $(SANITIZER_CFLAGS) $(CFLAGS_STLIB) $(PTHREAD_CFLAGS) $(CFLAGS_WARN) @DEFS@ $(LOCAL_CFLAGS)
LDFLAGS = $(SANITIZER_LDFLAGS) @LDFLAGS@
LDFLAGS_SHLIB = $(SANITIZER_LDFLAGS) @LDFLAGS_SHLIB@
ALL_LDFLAGS = $(LDFLAGS) @LDFLAG_DYNAMIC@
@@ -139,7 +141,7 @@ LIBFUSE = @FUSE_LIB@
LIBSUPPORT = $(LIBINTL) $(LIB)/libsupport@STATIC_LIB_EXT@
LIBBLKID = @LIBBLKID@ @PRIVATE_LIBS_CMT@ $(LIBUUID)
LIBINTL = @LIBINTL@
-SYSLIBS = @LIBS@
+SYSLIBS = @LIBS@ @PTHREAD_LIBS@
DEPLIBSS = $(LIB)/libss@LIB_EXT@
DEPLIBCOM_ERR = $(LIB)/libcom_err@LIB_EXT@
DEPLIBUUID = @DEPLIBUUID@
diff --git a/aclocal.m4 b/aclocal.m4
index 394e90d2..af47cc8a 100644
--- a/aclocal.m4
+++ b/aclocal.m4
@@ -1,6 +1,6 @@
-# generated automatically by aclocal 1.14.1 -*- Autoconf -*-
+# generated automatically by aclocal 1.16.2 -*- Autoconf -*-
-# Copyright (C) 1996-2013 Free Software Foundation, Inc.
+# Copyright (C) 1996-2020 Free Software Foundation, Inc.
# This file is free software; the Free Software Foundation
# gives unlimited permission to copy and/or distribute it,
@@ -13,7 +13,8 @@
m4_ifndef([AC_CONFIG_MACRO_DIRS], [m4_defun([_AM_CONFIG_MACRO_DIRS], [])m4_defun([AC_CONFIG_MACRO_DIRS], [_AM_CONFIG_MACRO_DIRS($@)])])
# codeset.m4 serial 5 (gettext-0.18.2)
-dnl Copyright (C) 2000-2002, 2006, 2008-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 2000-2002, 2006, 2008-2014, 2016 Free Software Foundation,
+dnl Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -38,7 +39,7 @@ AC_DEFUN([AM_LANGINFO_CODESET],
dnl 'extern inline' a la ISO C99.
-dnl Copyright 2012-2013 Free Software Foundation, Inc.
+dnl Copyright 2012-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -57,43 +58,75 @@ AC_DEFUN([gl_EXTERN_INLINE],
'reference to static identifier "f" in extern inline function'.
This bug was observed with Sun C 5.12 SunOS_i386 2011/11/16.
- Suppress the use of extern inline on Apple's platforms, as Libc at least
- through Libc-825.26 (2013-04-09) is incompatible with it; see, e.g.,
- <http://lists.gnu.org/archive/html/bug-gnulib/2012-12/msg00023.html>.
- Perhaps Apple will fix this some day. */
+ Suppress extern inline (with or without __attribute__ ((__gnu_inline__)))
+ on configurations that mistakenly use 'static inline' to implement
+ functions or macros in standard C headers like <ctype.h>. For example,
+ if isdigit is mistakenly implemented via a static inline function,
+ a program containing an extern inline function that calls isdigit
+ may not work since the C standard prohibits extern inline functions
+ from calling static functions. This bug is known to occur on:
+
+ OS X 10.8 and earlier; see:
+ http://lists.gnu.org/archive/html/bug-gnulib/2012-12/msg00023.html
+
+ DragonFly; see
+ http://muscles.dragonflybsd.org/bulk/bleeding-edge-potential/latest-per-pkg/ah-tty-0.3.12.log
+
+ FreeBSD; see:
+ http://lists.gnu.org/archive/html/bug-gnulib/2014-07/msg00104.html
+
+ OS X 10.9 has a macro __header_inline indicating the bug is fixed for C and
+ for clang but remains for g++; see <http://trac.macports.org/ticket/41033>.
+ Assume DragonFly and FreeBSD will be similar. */
+#if (((defined __APPLE__ && defined __MACH__) \
+ || defined __DragonFly__ || defined __FreeBSD__) \
+ && (defined __header_inline \
+ ? (defined __cplusplus && defined __GNUC_STDC_INLINE__ \
+ && ! defined __clang__) \
+ : ((! defined _DONT_USE_CTYPE_INLINE_ \
+ && (defined __GNUC__ || defined __cplusplus)) \
+ || (defined _FORTIFY_SOURCE && 0 < _FORTIFY_SOURCE \
+ && defined __GNUC__ && ! defined __cplusplus))))
+# define _GL_EXTERN_INLINE_STDHEADER_BUG
+#endif
#if ((__GNUC__ \
? defined __GNUC_STDC_INLINE__ && __GNUC_STDC_INLINE__ \
: (199901L <= __STDC_VERSION__ \
&& !defined __HP_cc \
+ && !defined __PGI \
&& !(defined __SUNPRO_C && __STDC__))) \
- && !defined __APPLE__)
+ && !defined _GL_EXTERN_INLINE_STDHEADER_BUG)
# define _GL_INLINE inline
# define _GL_EXTERN_INLINE extern inline
+# define _GL_EXTERN_INLINE_IN_USE
#elif (2 < __GNUC__ + (7 <= __GNUC_MINOR__) && !defined __STRICT_ANSI__ \
- && !defined __APPLE__)
-# if __GNUC_GNU_INLINE__
+ && !defined _GL_EXTERN_INLINE_STDHEADER_BUG)
+# if defined __GNUC_GNU_INLINE__ && __GNUC_GNU_INLINE__
/* __gnu_inline__ suppresses a GCC 4.2 diagnostic. */
# define _GL_INLINE extern inline __attribute__ ((__gnu_inline__))
# else
# define _GL_INLINE extern inline
# endif
# define _GL_EXTERN_INLINE extern
+# define _GL_EXTERN_INLINE_IN_USE
#else
# define _GL_INLINE static _GL_UNUSED
# define _GL_EXTERN_INLINE static _GL_UNUSED
#endif
-#if 4 < __GNUC__ + (6 <= __GNUC_MINOR__)
+/* In GCC 4.6 (inclusive) to 5.1 (exclusive),
+ suppress bogus "no previous prototype for 'FOO'"
+ and "no previous declaration for 'FOO'" diagnostics,
+ when FOO is an inline function in the header; see
+ <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54113> and
+ <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63877>. */
+#if __GNUC__ == 4 && 6 <= __GNUC_MINOR__
# if defined __GNUC_STDC_INLINE__ && __GNUC_STDC_INLINE__
# define _GL_INLINE_HEADER_CONST_PRAGMA
# else
# define _GL_INLINE_HEADER_CONST_PRAGMA \
_Pragma ("GCC diagnostic ignored \"-Wsuggest-attribute=const\"")
# endif
- /* Suppress GCC's bogus "no previous prototype for 'FOO'"
- and "no previous declaration for 'FOO'" diagnostics,
- when FOO is an inline function in the header; see
- <http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54113>. */
# define _GL_INLINE_HEADER_BEGIN \
_Pragma ("GCC diagnostic push") \
_Pragma ("GCC diagnostic ignored \"-Wmissing-prototypes\"") \
@@ -108,7 +141,7 @@ AC_DEFUN([gl_EXTERN_INLINE],
])
# fcntl-o.m4 serial 4
-dnl Copyright (C) 2006, 2009-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 2006, 2009-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -242,19 +275,19 @@ AC_DEFUN([gl_FCNTL_O_FLAGS],
[Define to 1 if O_NOFOLLOW works.])
])
-# gettext.m4 serial 66 (gettext-0.18.2)
-dnl Copyright (C) 1995-2013 Free Software Foundation, Inc.
+# gettext.m4 serial 68 (gettext-0.19.8)
+dnl Copyright (C) 1995-2014, 2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
dnl
-dnl This file can can be used in projects which are not available under
+dnl This file can be used in projects which are not available under
dnl the GNU General Public License or the GNU Library General Public
dnl License but which still want to provide support for the GNU gettext
dnl functionality.
dnl Please note that the actual code of the GNU gettext library is covered
dnl by the GNU Library General Public License, and the rest of the GNU
-dnl gettext package package is covered by the GNU General Public License.
+dnl gettext package is covered by the GNU General Public License.
dnl They are *not* in the public domain.
dnl Authors:
@@ -405,13 +438,18 @@ changequote([,])dnl
[AC_LANG_PROGRAM(
[[
#include <libintl.h>
-$gt_revision_test_code
+#ifndef __GNU_GETTEXT_SUPPORTED_REVISION
extern int _nl_msg_cat_cntr;
extern int *_nl_domain_bindings;
+#define __GNU_GETTEXT_SYMBOL_EXPRESSION (_nl_msg_cat_cntr + *_nl_domain_bindings)
+#else
+#define __GNU_GETTEXT_SYMBOL_EXPRESSION 0
+#endif
+$gt_revision_test_code
]],
[[
bindtextdomain ("", "");
-return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_domain_bindings
+return * gettext ("")$gt_expression_test_code + __GNU_GETTEXT_SYMBOL_EXPRESSION
]])],
[eval "$gt_func_gnugettext_libc=yes"],
[eval "$gt_func_gnugettext_libc=no"])])
@@ -437,17 +475,22 @@ return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_domain_b
[AC_LANG_PROGRAM(
[[
#include <libintl.h>
-$gt_revision_test_code
+#ifndef __GNU_GETTEXT_SUPPORTED_REVISION
extern int _nl_msg_cat_cntr;
extern
#ifdef __cplusplus
"C"
#endif
const char *_nl_expand_alias (const char *);
+#define __GNU_GETTEXT_SYMBOL_EXPRESSION (_nl_msg_cat_cntr + *_nl_expand_alias (""))
+#else
+#define __GNU_GETTEXT_SYMBOL_EXPRESSION 0
+#endif
+$gt_revision_test_code
]],
[[
bindtextdomain ("", "");
-return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_expand_alias ("")
+return * gettext ("")$gt_expression_test_code + __GNU_GETTEXT_SYMBOL_EXPRESSION
]])],
[eval "$gt_func_gnugettext_libintl=yes"],
[eval "$gt_func_gnugettext_libintl=no"])
@@ -458,17 +501,22 @@ return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_expand_a
[AC_LANG_PROGRAM(
[[
#include <libintl.h>
-$gt_revision_test_code
+#ifndef __GNU_GETTEXT_SUPPORTED_REVISION
extern int _nl_msg_cat_cntr;
extern
#ifdef __cplusplus
"C"
#endif
const char *_nl_expand_alias (const char *);
+#define __GNU_GETTEXT_SYMBOL_EXPRESSION (_nl_msg_cat_cntr + *_nl_expand_alias (""))
+#else
+#define __GNU_GETTEXT_SYMBOL_EXPRESSION 0
+#endif
+$gt_revision_test_code
]],
[[
bindtextdomain ("", "");
-return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_expand_alias ("")
+return * gettext ("")$gt_expression_test_code + __GNU_GETTEXT_SYMBOL_EXPRESSION
]])],
[LIBINTL="$LIBINTL $LIBICONV"
LTLIBINTL="$LTLIBINTL $LTLIBICONV"
@@ -644,8 +692,12 @@ AC_DEFUN([AM_GNU_GETTEXT_NEED],
dnl Usage: AM_GNU_GETTEXT_VERSION([gettext-version])
AC_DEFUN([AM_GNU_GETTEXT_VERSION], [])
+
+dnl Usage: AM_GNU_GETTEXT_REQUIRE_VERSION([gettext-version])
+AC_DEFUN([AM_GNU_GETTEXT_REQUIRE_VERSION], [])
+
# glibc2.m4 serial 3
-dnl Copyright (C) 2000-2002, 2004, 2008, 2010-2013 Free Software Foundation,
+dnl Copyright (C) 2000-2002, 2004, 2008, 2010-2016 Free Software Foundation,
dnl Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
@@ -677,7 +729,7 @@ AC_DEFUN([gt_GLIBC2],
)
# glibc21.m4 serial 5
-dnl Copyright (C) 2000-2002, 2004, 2008, 2010-2013 Free Software Foundation,
+dnl Copyright (C) 2000-2002, 2004, 2008, 2010-2016 Free Software Foundation,
dnl Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
@@ -711,8 +763,8 @@ AC_DEFUN([gl_GLIBC21],
]
)
-# iconv.m4 serial 18 (gettext-0.18.2)
-dnl Copyright (C) 2000-2002, 2007-2013 Free Software Foundation, Inc.
+# iconv.m4 serial 19 (gettext-0.18.2)
+dnl Copyright (C) 2000-2002, 2007-2014, 2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -785,27 +837,33 @@ AC_DEFUN([AM_ICONV_LINK],
if test $am_cv_lib_iconv = yes; then
LIBS="$LIBS $LIBICONV"
fi
- AC_RUN_IFELSE(
- [AC_LANG_SOURCE([[
+ am_cv_func_iconv_works=no
+ for ac_iconv_const in '' 'const'; do
+ AC_RUN_IFELSE(
+ [AC_LANG_PROGRAM(
+ [[
#include <iconv.h>
#include <string.h>
-int main ()
-{
- int result = 0;
+
+#ifndef ICONV_CONST
+# define ICONV_CONST $ac_iconv_const
+#endif
+ ]],
+ [[int result = 0;
/* Test against AIX 5.1 bug: Failures are not distinguishable from successful
returns. */
{
iconv_t cd_utf8_to_88591 = iconv_open ("ISO8859-1", "UTF-8");
if (cd_utf8_to_88591 != (iconv_t)(-1))
{
- static const char input[] = "\342\202\254"; /* EURO SIGN */
+ static ICONV_CONST char input[] = "\342\202\254"; /* EURO SIGN */
char buf[10];
- const char *inptr = input;
+ ICONV_CONST char *inptr = input;
size_t inbytesleft = strlen (input);
char *outptr = buf;
size_t outbytesleft = sizeof (buf);
size_t res = iconv (cd_utf8_to_88591,
- (char **) &inptr, &inbytesleft,
+ &inptr, &inbytesleft,
&outptr, &outbytesleft);
if (res == 0)
result |= 1;
@@ -818,14 +876,14 @@ int main ()
iconv_t cd_ascii_to_88591 = iconv_open ("ISO8859-1", "646");
if (cd_ascii_to_88591 != (iconv_t)(-1))
{
- static const char input[] = "\263";
+ static ICONV_CONST char input[] = "\263";
char buf[10];
- const char *inptr = input;
+ ICONV_CONST char *inptr = input;
size_t inbytesleft = strlen (input);
char *outptr = buf;
size_t outbytesleft = sizeof (buf);
size_t res = iconv (cd_ascii_to_88591,
- (char **) &inptr, &inbytesleft,
+ &inptr, &inbytesleft,
&outptr, &outbytesleft);
if (res == 0)
result |= 2;
@@ -837,14 +895,14 @@ int main ()
iconv_t cd_88591_to_utf8 = iconv_open ("UTF-8", "ISO-8859-1");
if (cd_88591_to_utf8 != (iconv_t)(-1))
{
- static const char input[] = "\304";
+ static ICONV_CONST char input[] = "\304";
static char buf[2] = { (char)0xDE, (char)0xAD };
- const char *inptr = input;
+ ICONV_CONST char *inptr = input;
size_t inbytesleft = 1;
char *outptr = buf;
size_t outbytesleft = 1;
size_t res = iconv (cd_88591_to_utf8,
- (char **) &inptr, &inbytesleft,
+ &inptr, &inbytesleft,
&outptr, &outbytesleft);
if (res != (size_t)(-1) || outptr - buf > 1 || buf[1] != (char)0xAD)
result |= 4;
@@ -857,14 +915,14 @@ int main ()
iconv_t cd_88591_to_utf8 = iconv_open ("utf8", "iso88591");
if (cd_88591_to_utf8 != (iconv_t)(-1))
{
- static const char input[] = "\304rger mit b\366sen B\374bchen ohne Augenma\337";
+ static ICONV_CONST char input[] = "\304rger mit b\366sen B\374bchen ohne Augenma\337";
char buf[50];
- const char *inptr = input;
+ ICONV_CONST char *inptr = input;
size_t inbytesleft = strlen (input);
char *outptr = buf;
size_t outbytesleft = sizeof (buf);
size_t res = iconv (cd_88591_to_utf8,
- (char **) &inptr, &inbytesleft,
+ &inptr, &inbytesleft,
&outptr, &outbytesleft);
if ((int)res > 0)
result |= 8;
@@ -884,17 +942,14 @@ int main ()
&& iconv_open ("utf8", "eucJP") == (iconv_t)(-1))
result |= 16;
return result;
-}]])],
- [am_cv_func_iconv_works=yes],
- [am_cv_func_iconv_works=no],
- [
-changequote(,)dnl
- case "$host_os" in
- aix* | hpux*) am_cv_func_iconv_works="guessing no" ;;
- *) am_cv_func_iconv_works="guessing yes" ;;
- esac
-changequote([,])dnl
- ])
+]])],
+ [am_cv_func_iconv_works=yes], ,
+ [case "$host_os" in
+ aix* | hpux*) am_cv_func_iconv_works="guessing no" ;;
+ *) am_cv_func_iconv_works="guessing yes" ;;
+ esac])
+ test "$am_cv_func_iconv_works" = no || break
+ done
LIBS="$am_save_LIBS"
])
case "$am_cv_func_iconv_works" in
@@ -981,7 +1036,7 @@ size_t iconv();
])
# intdiv0.m4 serial 6 (gettext-0.18.2)
-dnl Copyright (C) 2002, 2007-2008, 2010-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 2002, 2007-2008, 2010-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -1020,7 +1075,7 @@ static void
sigfpe_handler (int sig)
{
/* Exit with code 0 if SIGFPE, with code 1 if any other signal. */
- exit (sig != SIGFPE);
+ _exit (sig != SIGFPE);
}
int x = 1;
@@ -1068,19 +1123,19 @@ changequote([,])dnl
[Define if integer division by zero raises signal SIGFPE.])
])
-# intl.m4 serial 25 (gettext-0.18.3)
-dnl Copyright (C) 1995-2013 Free Software Foundation, Inc.
+# intl.m4 serial 29 (gettext-0.19)
+dnl Copyright (C) 1995-2014, 2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
dnl
-dnl This file can can be used in projects which are not available under
+dnl This file can be used in projects which are not available under
dnl the GNU General Public License or the GNU Library General Public
dnl License but which still want to provide support for the GNU gettext
dnl functionality.
dnl Please note that the actual code of the GNU gettext library is covered
dnl by the GNU Library General Public License, and the rest of the GNU
-dnl gettext package package is covered by the GNU General Public License.
+dnl gettext package is covered by the GNU General Public License.
dnl They are *not* in the public domain.
dnl Authors:
@@ -1113,6 +1168,7 @@ AC_DEFUN([AM_INTL_SUBDIR],
AC_REQUIRE([gl_FCNTL_O_FLAGS])dnl
AC_REQUIRE([gt_INTL_MACOSX])dnl
AC_REQUIRE([gl_EXTERN_INLINE])dnl
+ AC_REQUIRE([gt_GL_ATTRIBUTE])dnl
dnl Support for automake's --enable-silent-rules.
case "$enable_silent_rules" in
@@ -1301,6 +1357,12 @@ AC_DEFUN([gt_INTL_SUBDIR_CORE],
stpcpy strcasecmp strdup strtoul tsearch uselocale argz_count \
argz_stringify argz_next __fsetlocking])
+ dnl Solaris 12 provides getlocalename_l, while Illumos doesn't have
+ dnl it nor the equivalent.
+ if test $ac_cv_func_uselocale = yes; then
+ AC_CHECK_FUNCS([getlocalename_l])
+ fi
+
dnl Use the *_unlocked functions only if they are declared.
dnl (because some of them were defined without being declared in Solaris
dnl 2.5.1 but were removed in Solaris 2.6, whereas we want binaries built
@@ -1311,8 +1373,7 @@ AC_DEFUN([gt_INTL_SUBDIR_CORE],
dnl intl/plural.c is generated from intl/plural.y. It requires bison,
dnl because plural.y uses bison specific features. It requires at least
- dnl bison-1.26 because earlier versions generate a plural.c that doesn't
- dnl compile.
+ dnl bison-2.7 for %define api.pure.
dnl bison is only needed for the maintainer (who touches plural.y). But in
dnl order to avoid separate Makefiles or --enable-maintainer-mode, we put
dnl the rule in general Makefile. Now, some people carelessly touch the
@@ -1329,7 +1390,7 @@ changequote(<<,>>)dnl
ac_prog_version=`$INTLBISON --version 2>&1 | sed -n 's/^.*GNU Bison.* \([0-9]*\.[0-9.]*\).*$/\1/p'`
case $ac_prog_version in
'') ac_prog_version="v. ?.??, bad"; ac_verc_fail=yes;;
- 1.2[6-9]* | 1.[3-9][0-9]* | [2-9].*)
+ 2.[7-9]* | [3-9].*)
changequote([,])dnl
ac_prog_version="$ac_prog_version, ok"; ac_verc_fail=no;;
*) ac_prog_version="$ac_prog_version, bad"; ac_verc_fail=yes;;
@@ -1341,19 +1402,45 @@ changequote([,])dnl
fi
])
+dnl Copies _GL_UNUSED and _GL_ATTRIBUTE_PURE definitions from
+dnl gnulib-common.m4 as a fallback, if the project isn't using Gnulib.
+AC_DEFUN([gt_GL_ATTRIBUTE], [
+ m4_ifndef([gl_[]COMMON],
+ AH_VERBATIM([gt_gl_attribute],
+[/* Define as a marker that can be attached to declarations that might not
+ be used. This helps to reduce warnings, such as from
+ GCC -Wunused-parameter. */
+#ifndef _GL_UNUSED
+# if __GNUC__ >= 3 || (__GNUC__ == 2 && __GNUC_MINOR__ >= 7)
+# define _GL_UNUSED __attribute__ ((__unused__))
+# else
+# define _GL_UNUSED
+# endif
+#endif
+
+/* The __pure__ attribute was added in gcc 2.96. */
+#ifndef _GL_ATTRIBUTE_PURE
+# if __GNUC__ > 2 || (__GNUC__ == 2 && __GNUC_MINOR__ >= 96)
+# define _GL_ATTRIBUTE_PURE __attribute__ ((__pure__))
+# else
+# define _GL_ATTRIBUTE_PURE /* empty */
+# endif
+#endif
+]))])
+
# intlmacosx.m4 serial 5 (gettext-0.18.2)
-dnl Copyright (C) 2004-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 2004-2014, 2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
dnl
-dnl This file can can be used in projects which are not available under
+dnl This file can be used in projects which are not available under
dnl the GNU General Public License or the GNU Library General Public
dnl License but which still want to provide support for the GNU gettext
dnl functionality.
dnl Please note that the actual code of the GNU gettext library is covered
dnl by the GNU Library General Public License, and the rest of the GNU
-dnl gettext package package is covered by the GNU General Public License.
+dnl gettext package is covered by the GNU General Public License.
dnl They are *not* in the public domain.
dnl Checks for special options needed on Mac OS X.
@@ -1399,7 +1486,7 @@ AC_DEFUN([gt_INTL_MACOSX],
])
# intmax.m4 serial 6 (gettext-0.18.2)
-dnl Copyright (C) 2002-2005, 2008-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 2002-2005, 2008-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -1436,7 +1523,7 @@ AC_DEFUN([gt_TYPE_INTMAX_T],
])
# inttypes-pri.m4 serial 7 (gettext-0.18.2)
-dnl Copyright (C) 1997-2002, 2006, 2008-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 1997-2002, 2006, 2008-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -1479,7 +1566,7 @@ char *p = PRId32;
])
# inttypes_h.m4 serial 10
-dnl Copyright (C) 1997-2004, 2006, 2008-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 1997-2004, 2006, 2008-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -1509,19 +1596,19 @@ AC_DEFUN([gl_AC_HEADER_INTTYPES_H],
])
# lcmessage.m4 serial 7 (gettext-0.18.2)
-dnl Copyright (C) 1995-2002, 2004-2005, 2008-2013 Free Software Foundation,
-dnl Inc.
+dnl Copyright (C) 1995-2002, 2004-2005, 2008-2014, 2016 Free Software
+dnl Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
dnl
-dnl This file can can be used in projects which are not available under
+dnl This file can be used in projects which are not available under
dnl the GNU General Public License or the GNU Library General Public
dnl License but which still want to provide support for the GNU gettext
dnl functionality.
dnl Please note that the actual code of the GNU gettext library is covered
dnl by the GNU Library General Public License, and the rest of the GNU
-dnl gettext package package is covered by the GNU General Public License.
+dnl gettext package is covered by the GNU General Public License.
dnl They are *not* in the public domain.
dnl Authors:
@@ -1545,7 +1632,7 @@ AC_DEFUN([gt_LC_MESSAGES],
])
# lib-ld.m4 serial 6
-dnl Copyright (C) 1996-2003, 2009-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 1996-2003, 2009-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -1665,7 +1752,7 @@ AC_LIB_PROG_LD_GNU
])
# lib-link.m4 serial 26 (gettext-0.18.2)
-dnl Copyright (C) 2001-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 2001-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -1884,7 +1971,7 @@ AC_DEFUN([AC_LIB_LINKFLAGS_BODY],
fi
])
dnl Search the library and its dependencies in $additional_libdir and
- dnl $LDFLAGS. Using breadth-first-search.
+ dnl $LDFLAGS. Using breadth-first-seach.
LIB[]NAME=
LTLIB[]NAME=
INC[]NAME=
@@ -2443,7 +2530,7 @@ AC_DEFUN([AC_LIB_LINKFLAGS_FROM_LIBS],
])
# lib-prefix.m4 serial 7 (gettext-0.18)
-dnl Copyright (C) 2001-2005, 2008-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 2001-2005, 2008-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -2668,7 +2755,7 @@ sixtyfour bits
])
# lock.m4 serial 13 (gettext-0.18.2)
-dnl Copyright (C) 2005-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 2005-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -2711,7 +2798,7 @@ return !x;
AC_DEFUN([gl_PREREQ_LOCK], [:])
# longlong.m4 serial 17
-dnl Copyright (C) 1999-2007, 2009-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 1999-2007, 2009-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -2825,19 +2912,19 @@ AC_DEFUN([_AC_TYPE_LONG_LONG_SNIPPET],
])
# nls.m4 serial 5 (gettext-0.18)
-dnl Copyright (C) 1995-2003, 2005-2006, 2008-2013 Free Software Foundation,
-dnl Inc.
+dnl Copyright (C) 1995-2003, 2005-2006, 2008-2014, 2016 Free Software
+dnl Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
dnl
-dnl This file can can be used in projects which are not available under
+dnl This file can be used in projects which are not available under
dnl the GNU General Public License or the GNU Library General Public
dnl License but which still want to provide support for the GNU gettext
dnl functionality.
dnl Please note that the actual code of the GNU gettext library is covered
dnl by the GNU Library General Public License, and the rest of the GNU
-dnl gettext package package is covered by the GNU General Public License.
+dnl gettext package is covered by the GNU General Public License.
dnl They are *not* in the public domain.
dnl Authors:
@@ -2857,32 +2944,63 @@ AC_DEFUN([AM_NLS],
AC_SUBST([USE_NLS])
])
-# pkg.m4 - Macros to locate and utilise pkg-config. -*- Autoconf -*-
-# serial 1 (pkg-config-0.24)
-#
-# Copyright © 2004 Scott James Remnant <scott@netsplit.com>.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful, but
-# WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program; if not, write to the Free Software
-# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
-#
-# As a special exception to the GNU General Public License, if you
-# distribute this file as part of a program that contains a
-# configuration script generated by Autoconf, you may include it under
-# the same distribution terms that you use for the rest of that program.
-
-# PKG_PROG_PKG_CONFIG([MIN-VERSION])
-# ----------------------------------
+# pkg.m4 - Macros to locate and utilise pkg-config. -*- Autoconf -*-
+# serial 12 (pkg-config-0.29.2)
+
+dnl Copyright © 2004 Scott James Remnant <scott@netsplit.com>.
+dnl Copyright © 2012-2015 Dan Nicholson <dbn.lists@gmail.com>
+dnl
+dnl This program is free software; you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation; either version 2 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful, but
+dnl WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+dnl General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program; if not, write to the Free Software
+dnl Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
+dnl 02111-1307, USA.
+dnl
+dnl As a special exception to the GNU General Public License, if you
+dnl distribute this file as part of a program that contains a
+dnl configuration script generated by Autoconf, you may include it under
+dnl the same distribution terms that you use for the rest of that
+dnl program.
+
+dnl PKG_PREREQ(MIN-VERSION)
+dnl -----------------------
+dnl Since: 0.29
+dnl
+dnl Verify that the version of the pkg-config macros are at least
+dnl MIN-VERSION. Unlike PKG_PROG_PKG_CONFIG, which checks the user's
+dnl installed version of pkg-config, this checks the developer's version
+dnl of pkg.m4 when generating configure.
+dnl
+dnl To ensure that this macro is defined, also add:
+dnl m4_ifndef([PKG_PREREQ],
+dnl [m4_fatal([must install pkg-config 0.29 or later before running autoconf/autogen])])
+dnl
+dnl See the "Since" comment for each macro you use to see what version
+dnl of the macros you require.
+m4_defun([PKG_PREREQ],
+[m4_define([PKG_MACROS_VERSION], [0.29.2])
+m4_if(m4_version_compare(PKG_MACROS_VERSION, [$1]), -1,
+ [m4_fatal([pkg.m4 version $1 or higher is required but ]PKG_MACROS_VERSION[ found])])
+])dnl PKG_PREREQ
+
+dnl PKG_PROG_PKG_CONFIG([MIN-VERSION])
+dnl ----------------------------------
+dnl Since: 0.16
+dnl
+dnl Search for the pkg-config tool and set the PKG_CONFIG variable to
+dnl first found in the path. Checks that the version of pkg-config found
+dnl is at least MIN-VERSION. If MIN-VERSION is not specified, 0.9.0 is
+dnl used since that's the first version where most current features of
+dnl pkg-config existed.
AC_DEFUN([PKG_PROG_PKG_CONFIG],
[m4_pattern_forbid([^_?PKG_[A-Z_]+$])
m4_pattern_allow([^PKG_CONFIG(_(PATH|LIBDIR|SYSROOT_DIR|ALLOW_SYSTEM_(CFLAGS|LIBS)))?$])
@@ -2904,18 +3022,19 @@ if test -n "$PKG_CONFIG"; then
PKG_CONFIG=""
fi
fi[]dnl
-])# PKG_PROG_PKG_CONFIG
-
-# PKG_CHECK_EXISTS(MODULES, [ACTION-IF-FOUND], [ACTION-IF-NOT-FOUND])
-#
-# Check to see whether a particular set of modules exists. Similar
-# to PKG_CHECK_MODULES(), but does not set variables or print errors.
-#
-# Please remember that m4 expands AC_REQUIRE([PKG_PROG_PKG_CONFIG])
-# only at the first occurrence in configure.ac, so if the first place
-# it's called might be skipped (such as if it is within an "if", you
-# have to call PKG_CHECK_EXISTS manually
-# --------------------------------------------------------------
+])dnl PKG_PROG_PKG_CONFIG
+
+dnl PKG_CHECK_EXISTS(MODULES, [ACTION-IF-FOUND], [ACTION-IF-NOT-FOUND])
+dnl -------------------------------------------------------------------
+dnl Since: 0.18
+dnl
+dnl Check to see whether a particular set of modules exists. Similar to
+dnl PKG_CHECK_MODULES(), but does not set variables or print errors.
+dnl
+dnl Please remember that m4 expands AC_REQUIRE([PKG_PROG_PKG_CONFIG])
+dnl only at the first occurence in configure.ac, so if the first place
+dnl it's called might be skipped (such as if it is within an "if", you
+dnl have to call PKG_CHECK_EXISTS manually
AC_DEFUN([PKG_CHECK_EXISTS],
[AC_REQUIRE([PKG_PROG_PKG_CONFIG])dnl
if test -n "$PKG_CONFIG" && \
@@ -2925,8 +3044,10 @@ m4_ifvaln([$3], [else
$3])dnl
fi])
-# _PKG_CONFIG([VARIABLE], [COMMAND], [MODULES])
-# ---------------------------------------------
+dnl _PKG_CONFIG([VARIABLE], [COMMAND], [MODULES])
+dnl ---------------------------------------------
+dnl Internal wrapper calling pkg-config via PKG_CONFIG and setting
+dnl pkg_failed based on the result.
m4_define([_PKG_CONFIG],
[if test -n "$$1"; then
pkg_cv_[]$1="$$1"
@@ -2938,10 +3059,11 @@ m4_define([_PKG_CONFIG],
else
pkg_failed=untried
fi[]dnl
-])# _PKG_CONFIG
+])dnl _PKG_CONFIG
-# _PKG_SHORT_ERRORS_SUPPORTED
-# -----------------------------
+dnl _PKG_SHORT_ERRORS_SUPPORTED
+dnl ---------------------------
+dnl Internal check to see if pkg-config supports short errors.
AC_DEFUN([_PKG_SHORT_ERRORS_SUPPORTED],
[AC_REQUIRE([PKG_PROG_PKG_CONFIG])
if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
@@ -2949,26 +3071,24 @@ if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
else
_pkg_short_errors_supported=no
fi[]dnl
-])# _PKG_SHORT_ERRORS_SUPPORTED
-
-
-# PKG_CHECK_MODULES(VARIABLE-PREFIX, MODULES, [ACTION-IF-FOUND],
-# [ACTION-IF-NOT-FOUND])
-#
-#
-# Note that if there is a possibility the first call to
-# PKG_CHECK_MODULES might not happen, you should be sure to include an
-# explicit call to PKG_PROG_PKG_CONFIG in your configure.ac
-#
-#
-# --------------------------------------------------------------
+])dnl _PKG_SHORT_ERRORS_SUPPORTED
+
+
+dnl PKG_CHECK_MODULES(VARIABLE-PREFIX, MODULES, [ACTION-IF-FOUND],
+dnl [ACTION-IF-NOT-FOUND])
+dnl --------------------------------------------------------------
+dnl Since: 0.4.0
+dnl
+dnl Note that if there is a possibility the first call to
+dnl PKG_CHECK_MODULES might not happen, you should be sure to include an
+dnl explicit call to PKG_PROG_PKG_CONFIG in your configure.ac
AC_DEFUN([PKG_CHECK_MODULES],
[AC_REQUIRE([PKG_PROG_PKG_CONFIG])dnl
AC_ARG_VAR([$1][_CFLAGS], [C compiler flags for $1, overriding pkg-config])dnl
AC_ARG_VAR([$1][_LIBS], [linker flags for $1, overriding pkg-config])dnl
pkg_failed=no
-AC_MSG_CHECKING([for $1])
+AC_MSG_CHECKING([for $2])
_PKG_CONFIG([$1][_CFLAGS], [cflags], [$2])
_PKG_CONFIG([$1][_LIBS], [libs], [$2])
@@ -2978,11 +3098,11 @@ and $1[]_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.])
if test $pkg_failed = yes; then
- AC_MSG_RESULT([no])
+ AC_MSG_RESULT([no])
_PKG_SHORT_ERRORS_SUPPORTED
if test $_pkg_short_errors_supported = yes; then
$1[]_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "$2" 2>&1`
- else
+ else
$1[]_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "$2" 2>&1`
fi
# Put the nasty error message in config.log where it belongs
@@ -2999,7 +3119,7 @@ installed software in a non-standard prefix.
_PKG_TEXT])[]dnl
])
elif test $pkg_failed = untried; then
- AC_MSG_RESULT([no])
+ AC_MSG_RESULT([no])
m4_default([$4], [AC_MSG_FAILURE(
[The pkg-config script could not be found or is too old. Make sure it
is in your PATH or set the PKG_CONFIG environment variable to the full
@@ -3015,16 +3135,40 @@ else
AC_MSG_RESULT([yes])
$3
fi[]dnl
-])# PKG_CHECK_MODULES
+])dnl PKG_CHECK_MODULES
+
+
+dnl PKG_CHECK_MODULES_STATIC(VARIABLE-PREFIX, MODULES, [ACTION-IF-FOUND],
+dnl [ACTION-IF-NOT-FOUND])
+dnl ---------------------------------------------------------------------
+dnl Since: 0.29
+dnl
+dnl Checks for existence of MODULES and gathers its build flags with
+dnl static libraries enabled. Sets VARIABLE-PREFIX_CFLAGS from --cflags
+dnl and VARIABLE-PREFIX_LIBS from --libs.
+dnl
+dnl Note that if there is a possibility the first call to
+dnl PKG_CHECK_MODULES_STATIC might not happen, you should be sure to
+dnl include an explicit call to PKG_PROG_PKG_CONFIG in your
+dnl configure.ac.
+AC_DEFUN([PKG_CHECK_MODULES_STATIC],
+[AC_REQUIRE([PKG_PROG_PKG_CONFIG])dnl
+_save_PKG_CONFIG=$PKG_CONFIG
+PKG_CONFIG="$PKG_CONFIG --static"
+PKG_CHECK_MODULES($@)
+PKG_CONFIG=$_save_PKG_CONFIG[]dnl
+])dnl PKG_CHECK_MODULES_STATIC
-# PKG_INSTALLDIR(DIRECTORY)
-# -------------------------
-# Substitutes the variable pkgconfigdir as the location where a module
-# should install pkg-config .pc files. By default the directory is
-# $libdir/pkgconfig, but the default can be changed by passing
-# DIRECTORY. The user can override through the --with-pkgconfigdir
-# parameter.
+dnl PKG_INSTALLDIR([DIRECTORY])
+dnl -------------------------
+dnl Since: 0.27
+dnl
+dnl Substitutes the variable pkgconfigdir as the location where a module
+dnl should install pkg-config .pc files. By default the directory is
+dnl $libdir/pkgconfig, but the default can be changed by passing
+dnl DIRECTORY. The user can override through the --with-pkgconfigdir
+dnl parameter.
AC_DEFUN([PKG_INSTALLDIR],
[m4_pushdef([pkg_default], [m4_default([$1], ['${libdir}/pkgconfig'])])
m4_pushdef([pkg_description],
@@ -3035,16 +3179,18 @@ AC_ARG_WITH([pkgconfigdir],
AC_SUBST([pkgconfigdir], [$with_pkgconfigdir])
m4_popdef([pkg_default])
m4_popdef([pkg_description])
-]) dnl PKG_INSTALLDIR
+])dnl PKG_INSTALLDIR
-# PKG_NOARCH_INSTALLDIR(DIRECTORY)
-# -------------------------
-# Substitutes the variable noarch_pkgconfigdir as the location where a
-# module should install arch-independent pkg-config .pc files. By
-# default the directory is $datadir/pkgconfig, but the default can be
-# changed by passing DIRECTORY. The user can override through the
-# --with-noarch-pkgconfigdir parameter.
+dnl PKG_NOARCH_INSTALLDIR([DIRECTORY])
+dnl --------------------------------
+dnl Since: 0.27
+dnl
+dnl Substitutes the variable noarch_pkgconfigdir as the location where a
+dnl module should install arch-independent pkg-config .pc files. By
+dnl default the directory is $datadir/pkgconfig, but the default can be
+dnl changed by passing DIRECTORY. The user can override through the
+dnl --with-noarch-pkgconfigdir parameter.
AC_DEFUN([PKG_NOARCH_INSTALLDIR],
[m4_pushdef([pkg_default], [m4_default([$1], ['${datadir}/pkgconfig'])])
m4_pushdef([pkg_description],
@@ -3055,13 +3201,15 @@ AC_ARG_WITH([noarch-pkgconfigdir],
AC_SUBST([noarch_pkgconfigdir], [$with_noarch_pkgconfigdir])
m4_popdef([pkg_default])
m4_popdef([pkg_description])
-]) dnl PKG_NOARCH_INSTALLDIR
+])dnl PKG_NOARCH_INSTALLDIR
-# PKG_CHECK_VAR(VARIABLE, MODULE, CONFIG-VARIABLE,
-# [ACTION-IF-FOUND], [ACTION-IF-NOT-FOUND])
-# -------------------------------------------
-# Retrieves the value of the pkg-config variable for the given module.
+dnl PKG_CHECK_VAR(VARIABLE, MODULE, CONFIG-VARIABLE,
+dnl [ACTION-IF-FOUND], [ACTION-IF-NOT-FOUND])
+dnl -------------------------------------------
+dnl Since: 0.28
+dnl
+dnl Retrieves the value of the pkg-config variable for the given module.
AC_DEFUN([PKG_CHECK_VAR],
[AC_REQUIRE([PKG_PROG_PKG_CONFIG])dnl
AC_ARG_VAR([$1], [value of $3 for $2, overriding pkg-config])dnl
@@ -3070,21 +3218,21 @@ _PKG_CONFIG([$1], [variable="][$3]["], [$2])
AS_VAR_COPY([$1], [pkg_cv_][$1])
AS_VAR_IF([$1], [""], [$5], [$4])dnl
-])# PKG_CHECK_VAR
+])dnl PKG_CHECK_VAR
-# po.m4 serial 21 (gettext-0.18.3)
-dnl Copyright (C) 1995-2013 Free Software Foundation, Inc.
+# po.m4 serial 24 (gettext-0.19)
+dnl Copyright (C) 1995-2014, 2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
dnl
-dnl This file can can be used in projects which are not available under
+dnl This file can be used in projects which are not available under
dnl the GNU General Public License or the GNU Library General Public
dnl License but which still want to provide support for the GNU gettext
dnl functionality.
dnl Please note that the actual code of the GNU gettext library is covered
dnl by the GNU Library General Public License, and the rest of the GNU
-dnl gettext package package is covered by the GNU General Public License.
+dnl gettext package is covered by the GNU General Public License.
dnl They are *not* in the public domain.
dnl Authors:
@@ -3104,7 +3252,7 @@ AC_DEFUN([AM_PO_SUBDIRS],
dnl Release version of the gettext macros. This is used to ensure that
dnl the gettext macros and po/Makefile.in.in are in sync.
- AC_SUBST([GETTEXT_MACRO_VERSION], [0.18])
+ AC_SUBST([GETTEXT_MACRO_VERSION], [0.19])
dnl Perform the following tests also if --disable-nls has been given,
dnl because they are needed for "make dist" to work.
@@ -3527,7 +3675,7 @@ AC_DEFUN([AM_XGETTEXT_OPTION],
])
# printf-posix.m4 serial 6 (gettext-0.18.2)
-dnl Copyright (C) 2003, 2007, 2009-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 2003, 2007, 2009-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -3576,18 +3724,18 @@ int main ()
])
# progtest.m4 serial 7 (gettext-0.18.2)
-dnl Copyright (C) 1996-2003, 2005, 2008-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 1996-2003, 2005, 2008-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
dnl
-dnl This file can can be used in projects which are not available under
+dnl This file can be used in projects which are not available under
dnl the GNU General Public License or the GNU Library General Public
dnl License but which still want to provide support for the GNU gettext
dnl functionality.
dnl Please note that the actual code of the GNU gettext library is covered
dnl by the GNU Library General Public License, and the rest of the GNU
-dnl gettext package package is covered by the GNU General Public License.
+dnl gettext package is covered by the GNU General Public License.
dnl They are *not* in the public domain.
dnl Authors:
@@ -3668,7 +3816,7 @@ AC_SUBST([$1])dnl
])
# size_max.m4 serial 10
-dnl Copyright (C) 2003, 2005-2006, 2008-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 2003, 2005-2006, 2008-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -3748,7 +3896,7 @@ m4_ifdef([AC_COMPUTE_INT], [], [
])
# stdint_h.m4 serial 9
-dnl Copyright (C) 1997-2004, 2006, 2008-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 1997-2004, 2006, 2008-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -3775,8 +3923,8 @@ AC_DEFUN([gl_AC_HEADER_STDINT_H],
fi
])
-# threadlib.m4 serial 10 (gettext-0.18.2)
-dnl Copyright (C) 2005-2013 Free Software Foundation, Inc.
+# threadlib.m4 serial 11 (gettext-0.18.2)
+dnl Copyright (C) 2005-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -3799,7 +3947,7 @@ dnl libtool).
dnl Sets the variables LIBMULTITHREAD and LTLIBMULTITHREAD similarly, for
dnl programs that really need multithread functionality. The difference
dnl between LIBTHREAD and LIBMULTITHREAD is that on platforms supporting weak
-dnl symbols, typically LIBTHREAD="" whereas LIBMULTITHREAD="-lpthread".
+dnl symbols, typically LIBTHREAD is empty whereas LIBMULTITHREAD is not.
dnl Adds to CPPFLAGS the flag -D_REENTRANT or -D_THREAD_SAFE if needed for
dnl multithread-safe programs.
@@ -3939,15 +4087,31 @@ int main ()
# Test whether both pthread_mutex_lock and pthread_mutexattr_init exist
# in libc. IRIX 6.5 has the first one in both libc and libpthread, but
# the second one only in libpthread, and lock.c needs it.
- AC_LINK_IFELSE(
- [AC_LANG_PROGRAM(
- [[#include <pthread.h>]],
- [[pthread_mutex_lock((pthread_mutex_t*)0);
- pthread_mutexattr_init((pthread_mutexattr_t*)0);]])],
- [gl_have_pthread=yes])
+ #
+ # If -pthread works, prefer it to -lpthread, since Ubuntu 14.04
+ # needs -pthread for some reason. See:
+ # http://lists.gnu.org/archive/html/bug-gnulib/2014-09/msg00023.html
+ save_LIBS=$LIBS
+ for gl_pthread in '' '-pthread'; do
+ LIBS="$LIBS $gl_pthread"
+ AC_LINK_IFELSE(
+ [AC_LANG_PROGRAM(
+ [[#include <pthread.h>
+ pthread_mutex_t m;
+ pthread_mutexattr_t ma;
+ ]],
+ [[pthread_mutex_lock (&m);
+ pthread_mutexattr_init (&ma);]])],
+ [gl_have_pthread=yes
+ LIBTHREAD=$gl_pthread LTLIBTHREAD=$gl_pthread
+ LIBMULTITHREAD=$gl_pthread LTLIBMULTITHREAD=$gl_pthread])
+ LIBS=$save_LIBS
+ test -n "$gl_have_pthread" && break
+ done
+
# Test for libpthread by looking for pthread_kill. (Not pthread_self,
# since it is defined as a macro on OSF/1.)
- if test -n "$gl_have_pthread"; then
+ if test -n "$gl_have_pthread" && test -z "$LIBTHREAD"; then
# The program links fine without libpthread. But it may actually
# need to link with libpthread in order to create multiple threads.
AC_CHECK_LIB([pthread], [pthread_kill],
@@ -3962,7 +4126,7 @@ int main ()
[Define if the pthread_in_use() detection is hard.])
esac
])
- else
+ elif test -z "$gl_have_pthread"; then
# Some library is needed. Try libpthread and libc_r.
AC_CHECK_LIB([pthread], [pthread_kill],
[gl_have_pthread=yes
@@ -4103,6 +4267,8 @@ dnl Linux 2.4/glibc posix -lpthread Y OK
dnl
dnl GNU Hurd/glibc posix
dnl
+dnl Ubuntu 14.04 posix -pthread Y OK
+dnl
dnl FreeBSD 5.3 posix -lc_r Y
dnl posix -lkse ? Y
dnl posix -lpthread ? Y
@@ -4148,7 +4314,7 @@ dnl 0.5 if the first test terminates OK but the second one loops endlessly,
dnl 0.0 if the first test already loops endlessly.
# uintmax_t.m4 serial 12
-dnl Copyright (C) 1997-2004, 2007-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 1997-2004, 2007-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -4179,7 +4345,7 @@ AC_DEFUN([gl_AC_TYPE_UINTMAX_T],
])
# visibility.m4 serial 5 (gettext-0.18.2)
-dnl Copyright (C) 2005, 2008, 2010-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 2005, 2008, 2010-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -4257,7 +4423,7 @@ AC_DEFUN([gl_VISIBILITY],
])
# wchar_t.m4 serial 4 (gettext-0.18.2)
-dnl Copyright (C) 2002-2003, 2008-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 2002-2003, 2008-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -4282,7 +4448,7 @@ AC_DEFUN([gt_TYPE_WCHAR_T],
])
# wint_t.m4 serial 5 (gettext-0.18.2)
-dnl Copyright (C) 2003, 2007-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 2003, 2007-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@@ -4315,7 +4481,7 @@ AC_DEFUN([gt_TYPE_WINT_T],
])
# xsize.m4 serial 5
-dnl Copyright (C) 2003-2004, 2008-2013 Free Software Foundation, Inc.
+dnl Copyright (C) 2003-2004, 2008-2016 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
diff --git a/configure b/configure
index d90188af..194345bc 100755
--- a/configure
+++ b/configure
@@ -758,6 +758,7 @@ GETTEXT_PACKAGE
TDB_MAN_COMMENT
TDB_CMT
UUIDD_CMT
+PTHREAD_CC
E2INITRD_MAN
E2INITRD_PROG
FSCK_MAN
@@ -907,6 +908,7 @@ enable_defrag
enable_fsck
enable_e2initrd_helper
enable_tls
+with_pthread
enable_uuidd
enable_mmp
enable_tdb
@@ -1618,6 +1620,7 @@ Optional Packages:
--with-ccopts no longer supported, use CFLAGS= instead
--with-ldopts no longer supported, use LDFLAGS= instead
--with-root-prefix=PREFIX override prefix variable for files to be placed in the root
+ --without-pthread disable use of pthread support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-libpth-prefix[=DIR] search for libpth in DIR/include and DIR/lib
--without-libpth-prefix don't search for libpth in includedir and libdir
@@ -5988,6 +5991,34 @@ $as_echo "$ac_cv_tls" >&6; }
fi
+# Check whether --with-pthread was given.
+if test "${with_pthread+set}" = set; then :
+ withval=$with_pthread; if test "$withval" = "no"
+then
+ try_pthread=""
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: Disabling pthread support" >&5
+$as_echo "Disabling pthread support" >&6; }
+else
+ try_pthread="yes"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: Testing for pthread support" >&5
+$as_echo "Testing for pthread support" >&6; }
+fi
+
+else
+ try_pthread="yes"
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: Try testing for pthread support by default" >&5
+$as_echo "Try testing for pthread support by default" >&6; }
+
+fi
+
+if test "$try_pthread" = "yes"
+then
+AX_PTHREAD
+else
+test -n "$PTHREAD_CC" || PTHREAD_CC="$CC"
+
+fi
+
# Check whether --enable-uuidd was given.
if test "${enable_uuidd+set}" = set; then :
enableval=$enable_uuidd; if test "$enableval" = "no"
@@ -6392,7 +6423,7 @@ $as_echo "$USE_NLS" >&6; }
- GETTEXT_MACRO_VERSION=0.18
+ GETTEXT_MACRO_VERSION=0.19
@@ -7404,7 +7435,7 @@ static void
sigfpe_handler (int sig)
{
/* Exit with code 0 if SIGFPE, with code 1 if any other signal. */
- exit (sig != SIGFPE);
+ _exit (sig != SIGFPE);
}
int x = 1;
@@ -8002,26 +8033,42 @@ fi
# Test whether both pthread_mutex_lock and pthread_mutexattr_init exist
# in libc. IRIX 6.5 has the first one in both libc and libpthread, but
# the second one only in libpthread, and lock.c needs it.
- cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+ #
+ # If -pthread works, prefer it to -lpthread, since Ubuntu 14.04
+ # needs -pthread for some reason. See:
+ # http://lists.gnu.org/archive/html/bug-gnulib/2014-09/msg00023.html
+ save_LIBS=$LIBS
+ for gl_pthread in '' '-pthread'; do
+ LIBS="$LIBS $gl_pthread"
+ cat confdefs.h - <<_ACEOF >conftest.$ac_ext
/* end confdefs.h. */
#include <pthread.h>
+ pthread_mutex_t m;
+ pthread_mutexattr_t ma;
+
int
main ()
{
-pthread_mutex_lock((pthread_mutex_t*)0);
- pthread_mutexattr_init((pthread_mutexattr_t*)0);
+pthread_mutex_lock (&m);
+ pthread_mutexattr_init (&ma);
;
return 0;
}
_ACEOF
if ac_fn_c_try_link "$LINENO"; then :
gl_have_pthread=yes
+ LIBTHREAD=$gl_pthread LTLIBTHREAD=$gl_pthread
+ LIBMULTITHREAD=$gl_pthread LTLIBMULTITHREAD=$gl_pthread
fi
rm -f core conftest.err conftest.$ac_objext \
conftest$ac_exeext conftest.$ac_ext
+ LIBS=$save_LIBS
+ test -n "$gl_have_pthread" && break
+ done
+
# Test for libpthread by looking for pthread_kill. (Not pthread_self,
# since it is defined as a macro on OSF/1.)
- if test -n "$gl_have_pthread"; then
+ if test -n "$gl_have_pthread" && test -z "$LIBTHREAD"; then
# The program links fine without libpthread. But it may actually
# need to link with libpthread in order to create multiple threads.
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for pthread_kill in -lpthread" >&5
@@ -8075,7 +8122,7 @@ $as_echo "#define PTHREAD_IN_USE_DETECTION_HARD 1" >>confdefs.h
fi
- else
+ elif test -z "$gl_have_pthread"; then
# Some library is needed. Try libpthread and libc_r.
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for pthread_kill in -lpthread" >&5
$as_echo_n "checking for pthread_kill in -lpthread... " >&6; }
@@ -9426,36 +9473,42 @@ else
if test $am_cv_lib_iconv = yes; then
LIBS="$LIBS $LIBICONV"
fi
- if test "$cross_compiling" = yes; then :
-
- case "$host_os" in
- aix* | hpux*) am_cv_func_iconv_works="guessing no" ;;
- *) am_cv_func_iconv_works="guessing yes" ;;
- esac
-
+ am_cv_func_iconv_works=no
+ for ac_iconv_const in '' 'const'; do
+ if test "$cross_compiling" = yes; then :
+ case "$host_os" in
+ aix* | hpux*) am_cv_func_iconv_works="guessing no" ;;
+ *) am_cv_func_iconv_works="guessing yes" ;;
+ esac
else
cat confdefs.h - <<_ACEOF >conftest.$ac_ext
/* end confdefs.h. */
#include <iconv.h>
#include <string.h>
-int main ()
+
+#ifndef ICONV_CONST
+# define ICONV_CONST $ac_iconv_const
+#endif
+
+int
+main ()
{
- int result = 0;
+int result = 0;
/* Test against AIX 5.1 bug: Failures are not distinguishable from successful
returns. */
{
iconv_t cd_utf8_to_88591 = iconv_open ("ISO8859-1", "UTF-8");
if (cd_utf8_to_88591 != (iconv_t)(-1))
{
- static const char input[] = "\342\202\254"; /* EURO SIGN */
+ static ICONV_CONST char input[] = "\342\202\254"; /* EURO SIGN */
char buf[10];
- const char *inptr = input;
+ ICONV_CONST char *inptr = input;
size_t inbytesleft = strlen (input);
char *outptr = buf;
size_t outbytesleft = sizeof (buf);
size_t res = iconv (cd_utf8_to_88591,
- (char **) &inptr, &inbytesleft,
+ &inptr, &inbytesleft,
&outptr, &outbytesleft);
if (res == 0)
result |= 1;
@@ -9468,14 +9521,14 @@ int main ()
iconv_t cd_ascii_to_88591 = iconv_open ("ISO8859-1", "646");
if (cd_ascii_to_88591 != (iconv_t)(-1))
{
- static const char input[] = "\263";
+ static ICONV_CONST char input[] = "\263";
char buf[10];
- const char *inptr = input;
+ ICONV_CONST char *inptr = input;
size_t inbytesleft = strlen (input);
char *outptr = buf;
size_t outbytesleft = sizeof (buf);
size_t res = iconv (cd_ascii_to_88591,
- (char **) &inptr, &inbytesleft,
+ &inptr, &inbytesleft,
&outptr, &outbytesleft);
if (res == 0)
result |= 2;
@@ -9487,14 +9540,14 @@ int main ()
iconv_t cd_88591_to_utf8 = iconv_open ("UTF-8", "ISO-8859-1");
if (cd_88591_to_utf8 != (iconv_t)(-1))
{
- static const char input[] = "\304";
+ static ICONV_CONST char input[] = "\304";
static char buf[2] = { (char)0xDE, (char)0xAD };
- const char *inptr = input;
+ ICONV_CONST char *inptr = input;
size_t inbytesleft = 1;
char *outptr = buf;
size_t outbytesleft = 1;
size_t res = iconv (cd_88591_to_utf8,
- (char **) &inptr, &inbytesleft,
+ &inptr, &inbytesleft,
&outptr, &outbytesleft);
if (res != (size_t)(-1) || outptr - buf > 1 || buf[1] != (char)0xAD)
result |= 4;
@@ -9507,14 +9560,14 @@ int main ()
iconv_t cd_88591_to_utf8 = iconv_open ("utf8", "iso88591");
if (cd_88591_to_utf8 != (iconv_t)(-1))
{
- static const char input[] = "\304rger mit b\366sen B\374bchen ohne Augenma\337";
+ static ICONV_CONST char input[] = "\304rger mit b\366sen B\374bchen ohne Augenma\337";
char buf[50];
- const char *inptr = input;
+ ICONV_CONST char *inptr = input;
size_t inbytesleft = strlen (input);
char *outptr = buf;
size_t outbytesleft = sizeof (buf);
size_t res = iconv (cd_88591_to_utf8,
- (char **) &inptr, &inbytesleft,
+ &inptr, &inbytesleft,
&outptr, &outbytesleft);
if ((int)res > 0)
result |= 8;
@@ -9534,17 +9587,20 @@ int main ()
&& iconv_open ("utf8", "eucJP") == (iconv_t)(-1))
result |= 16;
return result;
+
+ ;
+ return 0;
}
_ACEOF
if ac_fn_c_try_run "$LINENO"; then :
am_cv_func_iconv_works=yes
-else
- am_cv_func_iconv_works=no
fi
rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \
conftest.$ac_objext conftest.beam conftest.$ac_ext
fi
+ test "$am_cv_func_iconv_works" = no || break
+ done
LIBS="$am_save_LIBS"
fi
@@ -9677,6 +9733,20 @@ fi
done
+ if test $ac_cv_func_uselocale = yes; then
+ for ac_func in getlocalename_l
+do :
+ ac_fn_c_check_func "$LINENO" "getlocalename_l" "ac_cv_func_getlocalename_l"
+if test "x$ac_cv_func_getlocalename_l" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_GETLOCALENAME_L 1
+_ACEOF
+
+fi
+done
+
+ fi
+
ac_fn_c_check_decl "$LINENO" "feof_unlocked" "ac_cv_have_decl_feof_unlocked" "#include <stdio.h>
"
if test "x$ac_cv_have_decl_feof_unlocked" = xyes; then :
@@ -9703,7 +9773,7 @@ _ACEOF
- for ac_prog in bison
+ for ac_prog in bison
do
# Extract the first word of "$ac_prog", so it can be a program name with args.
set dummy $ac_prog; ac_word=$2
@@ -9753,7 +9823,7 @@ $as_echo_n "checking version of bison... " >&6; }
ac_prog_version=`$INTLBISON --version 2>&1 | sed -n 's/^.*GNU Bison.* \([0-9]*\.[0-9.]*\).*$/\1/p'`
case $ac_prog_version in
'') ac_prog_version="v. ?.??, bad"; ac_verc_fail=yes;;
- 1.2[6-9]* | 1.[3-9][0-9]* | [2-9].*)
+ 2.[7-9]* | [3-9].*)
ac_prog_version="$ac_prog_version, ok"; ac_verc_fail=no;;
*) ac_prog_version="$ac_prog_version, bad"; ac_verc_fail=yes;;
esac
@@ -10399,6 +10469,8 @@ $as_echo "#define HAVE_CFLOCALECOPYCURRENT 1" >>confdefs.h
+
+
case "$enable_silent_rules" in
yes) INTL_DEFAULT_VERBOSITY=0;;
no) INTL_DEFAULT_VERBOSITY=1;;
@@ -10859,16 +10931,21 @@ else
/* end confdefs.h. */
#include <libintl.h>
-$gt_revision_test_code
+#ifndef __GNU_GETTEXT_SUPPORTED_REVISION
extern int _nl_msg_cat_cntr;
extern int *_nl_domain_bindings;
+#define __GNU_GETTEXT_SYMBOL_EXPRESSION (_nl_msg_cat_cntr + *_nl_domain_bindings)
+#else
+#define __GNU_GETTEXT_SYMBOL_EXPRESSION 0
+#endif
+$gt_revision_test_code
int
main ()
{
bindtextdomain ("", "");
-return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_domain_bindings
+return * gettext ("")$gt_expression_test_code + __GNU_GETTEXT_SYMBOL_EXPRESSION
;
return 0;
@@ -11362,20 +11439,25 @@ else
/* end confdefs.h. */
#include <libintl.h>
-$gt_revision_test_code
+#ifndef __GNU_GETTEXT_SUPPORTED_REVISION
extern int _nl_msg_cat_cntr;
extern
#ifdef __cplusplus
"C"
#endif
const char *_nl_expand_alias (const char *);
+#define __GNU_GETTEXT_SYMBOL_EXPRESSION (_nl_msg_cat_cntr + *_nl_expand_alias (""))
+#else
+#define __GNU_GETTEXT_SYMBOL_EXPRESSION 0
+#endif
+$gt_revision_test_code
int
main ()
{
bindtextdomain ("", "");
-return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_expand_alias ("")
+return * gettext ("")$gt_expression_test_code + __GNU_GETTEXT_SYMBOL_EXPRESSION
;
return 0;
@@ -11394,20 +11476,25 @@ rm -f core conftest.err conftest.$ac_objext \
/* end confdefs.h. */
#include <libintl.h>
-$gt_revision_test_code
+#ifndef __GNU_GETTEXT_SUPPORTED_REVISION
extern int _nl_msg_cat_cntr;
extern
#ifdef __cplusplus
"C"
#endif
const char *_nl_expand_alias (const char *);
+#define __GNU_GETTEXT_SYMBOL_EXPRESSION (_nl_msg_cat_cntr + *_nl_expand_alias (""))
+#else
+#define __GNU_GETTEXT_SYMBOL_EXPRESSION 0
+#endif
+$gt_revision_test_code
int
main ()
{
bindtextdomain ("", "");
-return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_expand_alias ("")
+return * gettext ("")$gt_expression_test_code + __GNU_GETTEXT_SYMBOL_EXPRESSION
;
return 0;
@@ -13180,6 +13267,48 @@ if test "x$ac_cv_lib_blkid_blkid_probe_get_topology" = xyes; then :
$as_echo "#define HAVE_BLKID_PROBE_GET_TOPOLOGY 1" >>confdefs.h
+fi
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for blkid_topology_get_dax in -lblkid" >&5
+$as_echo_n "checking for blkid_topology_get_dax in -lblkid... " >&6; }
+if ${ac_cv_lib_blkid_blkid_topology_get_dax+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lblkid $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char blkid_topology_get_dax ();
+int
+main ()
+{
+return blkid_topology_get_dax ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_blkid_blkid_topology_get_dax=yes
+else
+ ac_cv_lib_blkid_blkid_topology_get_dax=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_blkid_blkid_topology_get_dax" >&5
+$as_echo "$ac_cv_lib_blkid_blkid_topology_get_dax" >&6; }
+if test "x$ac_cv_lib_blkid_blkid_topology_get_dax" = xyes; then :
+
+$as_echo "#define HAVE_BLKID_TOPOLOGY_GET_DAX 1" >>confdefs.h
+
fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for blkid_probe_enable_partitions in -lblkid" >&5
@@ -14425,7 +14554,7 @@ fi
if test $pkg_failed = yes; then
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
$as_echo "no" >&6; }
if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
@@ -14445,7 +14574,7 @@ fi
with_udev_rules_dir=""
elif test $pkg_failed = untried; then
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
$as_echo "no" >&6; }
with_udev_rules_dir=""
@@ -14585,7 +14714,7 @@ fi
if test $pkg_failed = yes; then
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
$as_echo "no" >&6; }
if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
@@ -14605,7 +14734,7 @@ fi
with_systemd_unit_dir=""
elif test $pkg_failed = untried; then
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
$as_echo "no" >&6; }
with_systemd_unit_dir=""
diff --git a/configure.ac b/configure.ac
index 7d921074..bd0e06f3 100644
--- a/configure.ac
+++ b/configure.ac
@@ -773,6 +773,30 @@ fi
dnl
dnl
dnl
+AC_ARG_WITH([pthread],
+[ --without-pthread disable use of pthread support],
+[if test "$withval" = "no"
+then
+ try_pthread=""
+ AC_MSG_RESULT([Disabling pthread support])
+else
+ try_pthread="yes"
+ AC_MSG_RESULT([Testing for pthread support])
+fi]
+,
+try_pthread="yes"
+AC_MSG_RESULT([Try testing for pthread support by default])
+)
+if test "$try_pthread" = "yes"
+then
+AX_PTHREAD
+else
+test -n "$PTHREAD_CC" || PTHREAD_CC="$CC"
+AC_SUBST([PTHREAD_CC])
+fi
+dnl
+dnl
+dnl
AH_TEMPLATE([USE_UUIDD], [Define to 1 to build uuidd])
AC_ARG_ENABLE([uuidd],
[ --disable-uuidd disable building the uuid daemon],
diff --git a/lib/config.h.in b/lib/config.h.in
index b448482c..f6e4b307 100644
--- a/lib/config.h.in
+++ b/lib/config.h.in
@@ -82,6 +82,9 @@
/* Define to 1 if blkid has blkid_probe_get_topology */
#undef HAVE_BLKID_PROBE_GET_TOPOLOGY
+/* Define to 1 if blkid has blkid_topology_get_dax */
+#undef HAVE_BLKID_TOPOLOGY_GET_DAX
+
/* Define to 1 if the compiler understands __builtin_expect. */
#undef HAVE_BUILTIN_EXPECT
@@ -195,6 +198,9 @@
/* Define to 1 if you have the `gethostname' function. */
#undef HAVE_GETHOSTNAME
+/* Define to 1 if you have the `getlocalename_l' function. */
+#undef HAVE_GETLOCALENAME_L
+
/* Define to 1 if you have the `getmntinfo' function. */
#undef HAVE_GETMNTINFO
@@ -798,43 +804,75 @@
'reference to static identifier "f" in extern inline function'.
This bug was observed with Sun C 5.12 SunOS_i386 2011/11/16.
- Suppress the use of extern inline on Apple's platforms, as Libc at least
- through Libc-825.26 (2013-04-09) is incompatible with it; see, e.g.,
- <http://lists.gnu.org/archive/html/bug-gnulib/2012-12/msg00023.html>.
- Perhaps Apple will fix this some day. */
+ Suppress extern inline (with or without __attribute__ ((__gnu_inline__)))
+ on configurations that mistakenly use 'static inline' to implement
+ functions or macros in standard C headers like <ctype.h>. For example,
+ if isdigit is mistakenly implemented via a static inline function,
+ a program containing an extern inline function that calls isdigit
+ may not work since the C standard prohibits extern inline functions
+ from calling static functions. This bug is known to occur on:
+
+ OS X 10.8 and earlier; see:
+ http://lists.gnu.org/archive/html/bug-gnulib/2012-12/msg00023.html
+
+ DragonFly; see
+ http://muscles.dragonflybsd.org/bulk/bleeding-edge-potential/latest-per-pkg/ah-tty-0.3.12.log
+
+ FreeBSD; see:
+ http://lists.gnu.org/archive/html/bug-gnulib/2014-07/msg00104.html
+
+ OS X 10.9 has a macro __header_inline indicating the bug is fixed for C and
+ for clang but remains for g++; see <http://trac.macports.org/ticket/41033>.
+ Assume DragonFly and FreeBSD will be similar. */
+#if (((defined __APPLE__ && defined __MACH__) \
+ || defined __DragonFly__ || defined __FreeBSD__) \
+ && (defined __header_inline \
+ ? (defined __cplusplus && defined __GNUC_STDC_INLINE__ \
+ && ! defined __clang__) \
+ : ((! defined _DONT_USE_CTYPE_INLINE_ \
+ && (defined __GNUC__ || defined __cplusplus)) \
+ || (defined _FORTIFY_SOURCE && 0 < _FORTIFY_SOURCE \
+ && defined __GNUC__ && ! defined __cplusplus))))
+# define _GL_EXTERN_INLINE_STDHEADER_BUG
+#endif
#if ((__GNUC__ \
? defined __GNUC_STDC_INLINE__ && __GNUC_STDC_INLINE__ \
: (199901L <= __STDC_VERSION__ \
&& !defined __HP_cc \
+ && !defined __PGI \
&& !(defined __SUNPRO_C && __STDC__))) \
- && !defined __APPLE__)
+ && !defined _GL_EXTERN_INLINE_STDHEADER_BUG)
# define _GL_INLINE inline
# define _GL_EXTERN_INLINE extern inline
+# define _GL_EXTERN_INLINE_IN_USE
#elif (2 < __GNUC__ + (7 <= __GNUC_MINOR__) && !defined __STRICT_ANSI__ \
- && !defined __APPLE__)
-# if __GNUC_GNU_INLINE__
+ && !defined _GL_EXTERN_INLINE_STDHEADER_BUG)
+# if defined __GNUC_GNU_INLINE__ && __GNUC_GNU_INLINE__
/* __gnu_inline__ suppresses a GCC 4.2 diagnostic. */
# define _GL_INLINE extern inline __attribute__ ((__gnu_inline__))
# else
# define _GL_INLINE extern inline
# endif
# define _GL_EXTERN_INLINE extern
+# define _GL_EXTERN_INLINE_IN_USE
#else
# define _GL_INLINE static _GL_UNUSED
# define _GL_EXTERN_INLINE static _GL_UNUSED
#endif
-#if 4 < __GNUC__ + (6 <= __GNUC_MINOR__)
+/* In GCC 4.6 (inclusive) to 5.1 (exclusive),
+ suppress bogus "no previous prototype for 'FOO'"
+ and "no previous declaration for 'FOO'" diagnostics,
+ when FOO is an inline function in the header; see
+ <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54113> and
+ <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63877>. */
+#if __GNUC__ == 4 && 6 <= __GNUC_MINOR__
# if defined __GNUC_STDC_INLINE__ && __GNUC_STDC_INLINE__
# define _GL_INLINE_HEADER_CONST_PRAGMA
# else
# define _GL_INLINE_HEADER_CONST_PRAGMA \
_Pragma ("GCC diagnostic ignored \"-Wsuggest-attribute=const\"")
# endif
- /* Suppress GCC's bogus "no previous prototype for 'FOO'"
- and "no previous declaration for 'FOO'" diagnostics,
- when FOO is an inline function in the header; see
- <http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54113>. */
# define _GL_INLINE_HEADER_BEGIN \
_Pragma ("GCC diagnostic push") \
_Pragma ("GCC diagnostic ignored \"-Wmissing-prototypes\"") \
@@ -847,6 +885,27 @@
# define _GL_INLINE_HEADER_END
#endif
+/* Define as a marker that can be attached to declarations that might not
+ be used. This helps to reduce warnings, such as from
+ GCC -Wunused-parameter. */
+#ifndef _GL_UNUSED
+# if __GNUC__ >= 3 || (__GNUC__ == 2 && __GNUC_MINOR__ >= 7)
+# define _GL_UNUSED __attribute__ ((__unused__))
+# else
+# define _GL_UNUSED
+# endif
+#endif
+
+/* The __pure__ attribute was added in gcc 2.96. */
+#ifndef _GL_ATTRIBUTE_PURE
+# if __GNUC__ > 2 || (__GNUC__ == 2 && __GNUC_MINOR__ >= 96)
+# define _GL_ATTRIBUTE_PURE __attribute__ ((__pure__))
+# else
+# define _GL_ATTRIBUTE_PURE /* empty */
+# endif
+#endif
+
+
/* Define to `__inline__' or `__inline' if that's what the C compiler
calls it, or to nothing if 'inline' is not supported under any name. */
#ifndef __cplusplus
--
2.30.0.284.gd98b1dd5eaa7-goog
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [RFC PATCH v1 2/5] libext2fs: add threading support to the I/O manager abstraction
2021-01-14 0:27 [RFC PATCH v1 0/5] Add threading support to e2fsprogs Saranya Muruganandam
2021-01-14 0:27 ` [RFC PATCH v1 1/5] Add configure and build support for the pthreads library Saranya Muruganandam
@ 2021-01-14 0:27 ` Saranya Muruganandam
2021-01-14 0:27 ` [RFC PATCH v1 3/5] libext2fs: allow the unix_io manager's cache to be disabled and re-enabled Saranya Muruganandam
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Saranya Muruganandam @ 2021-01-14 0:27 UTC (permalink / raw)
To: linux-ext4; +Cc: tytso
From: Theodore Ts'o <tytso@mit.edu>
Add initial implementation support for the unix_io manager.
Applications which want to use threading should pass in
IO_FLAG_THREADS when opening the channel. Channels which support
threading (which as of this commit is unix_io and test_io if the
backing io_manager supports threading) will set the
CHANNEL_FLAGS_THREADS bit in io->flags. Library code or applications
can test if threading is enabled by checking this flag.
Applications using libext2fs can pass in EXT2_FLAG_THREADS to
ext2fs_open() or ext2fs_open2() to request threading support.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
lib/ext2fs/ext2_io.h | 2 +
lib/ext2fs/ext2fs.h | 1 +
lib/ext2fs/openfs.c | 2 +
lib/ext2fs/test_io.c | 6 +-
lib/ext2fs/undo_io.c | 2 +
lib/ext2fs/unix_io.c | 137 ++++++++++++++++++++++++++++++++++++++-----
6 files changed, 133 insertions(+), 17 deletions(-)
diff --git a/lib/ext2fs/ext2_io.h b/lib/ext2fs/ext2_io.h
index 5540900a..2e0da5a5 100644
--- a/lib/ext2fs/ext2_io.h
+++ b/lib/ext2fs/ext2_io.h
@@ -33,6 +33,7 @@ typedef struct struct_io_stats *io_stats;
#define CHANNEL_FLAGS_WRITETHROUGH 0x01
#define CHANNEL_FLAGS_DISCARD_ZEROES 0x02
#define CHANNEL_FLAGS_BLOCK_DEVICE 0x04
+#define CHANNEL_FLAGS_THREADS 0x08
#define io_channel_discard_zeroes_data(i) (i->flags & CHANNEL_FLAGS_DISCARD_ZEROES)
@@ -104,6 +105,7 @@ struct struct_io_manager {
#define IO_FLAG_EXCLUSIVE 0x0002
#define IO_FLAG_DIRECT_IO 0x0004
#define IO_FLAG_FORCE_BOUNCE 0x0008
+#define IO_FLAG_THREADS 0x0010
/*
* Convenience functions....
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 69c8a3ff..5955c3ae 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -206,6 +206,7 @@ typedef struct ext2_file *ext2_file_t;
#define EXT2_FLAG_IGNORE_SB_ERRORS 0x800000
#define EXT2_FLAG_BBITMAP_TAIL_PROBLEM 0x1000000
#define EXT2_FLAG_IBITMAP_TAIL_PROBLEM 0x2000000
+#define EXT2_FLAG_THREADS 0x4000000
/*
* Special flag in the ext2 inode i_flag field that means that this is
diff --git a/lib/ext2fs/openfs.c b/lib/ext2fs/openfs.c
index 3ed1e25c..5ec8ed5c 100644
--- a/lib/ext2fs/openfs.c
+++ b/lib/ext2fs/openfs.c
@@ -170,6 +170,8 @@ errcode_t ext2fs_open2(const char *name, const char *io_options,
io_flags |= IO_FLAG_EXCLUSIVE;
if (flags & EXT2_FLAG_DIRECT_IO)
io_flags |= IO_FLAG_DIRECT_IO;
+ if (flags & EXT2_FLAG_THREADS)
+ io_flags |= IO_FLAG_THREADS;
retval = manager->open(fs->device_name, io_flags, &fs->io);
if (retval)
goto cleanup;
diff --git a/lib/ext2fs/test_io.c b/lib/ext2fs/test_io.c
index ee828be7..480e68fc 100644
--- a/lib/ext2fs/test_io.c
+++ b/lib/ext2fs/test_io.c
@@ -197,6 +197,7 @@ static errcode_t test_open(const char *name, int flags, io_channel *channel)
io->read_error = 0;
io->write_error = 0;
io->refcount = 1;
+ io->flags = 0;
memset(data, 0, sizeof(struct test_private_data));
data->magic = EXT2_ET_MAGIC_TEST_IO_CHANNEL;
@@ -237,8 +238,11 @@ static errcode_t test_open(const char *name, int flags, io_channel *channel)
if ((value = safe_getenv("TEST_IO_WRITE_ABORT")) != NULL)
data->write_abort_count = strtoul(value, NULL, 0);
- if (data->real)
+ if (data->real) {
io->align = data->real->align;
+ if (data->real->flags & CHANNEL_FLAGS_THREADS)
+ io->flags |= CHANNEL_FLAGS_THREADS;
+ }
*channel = io;
return 0;
diff --git a/lib/ext2fs/undo_io.c b/lib/ext2fs/undo_io.c
index 19862414..eb56f53d 100644
--- a/lib/ext2fs/undo_io.c
+++ b/lib/ext2fs/undo_io.c
@@ -698,6 +698,8 @@ static errcode_t undo_open(const char *name, int flags, io_channel *channel)
int undo_fd = -1;
errcode_t retval;
+ /* We don't support multi-threading, at least for now */
+ flags &= ~IO_FLAG_THREADS;
if (name == 0)
return EXT2_ET_BAD_DEVICE_NAME;
retval = ext2fs_get_mem(sizeof(struct struct_io_channel), &io);
diff --git a/lib/ext2fs/unix_io.c b/lib/ext2fs/unix_io.c
index 628e60c3..9385487d 100644
--- a/lib/ext2fs/unix_io.c
+++ b/lib/ext2fs/unix_io.c
@@ -67,6 +67,9 @@
#if HAVE_LINUX_FALLOC_H
#include <linux/falloc.h>
#endif
+#ifdef HAVE_PTHREAD
+#include <pthread.h>
+#endif
#if defined(__linux__) && defined(_IO) && !defined(BLKROGET)
#define BLKROGET _IO(0x12, 94) /* Get read-only status (0 = read_write). */
@@ -107,11 +110,58 @@ struct unix_private_data {
struct unix_cache cache[CACHE_SIZE];
void *bounce;
struct struct_io_stats io_stats;
+#ifdef HAVE_PTHREAD
+ pthread_mutex_t cache_mutex;
+ pthread_mutex_t bounce_mutex;
+ pthread_mutex_t stats_mutex;
+#endif
};
#define IS_ALIGNED(n, align) ((((uintptr_t) n) & \
((uintptr_t) ((align)-1))) == 0)
+typedef enum lock_kind {
+ CACHE_MTX, BOUNCE_MTX, STATS_MTX
+} kind_t;
+
+#ifdef HAVE_PTHREAD
+static inline pthread_mutex_t *get_mutex(struct unix_private_data *data,
+ kind_t kind)
+{
+ if (data->flags & IO_FLAG_THREADS) {
+ switch (kind) {
+ case CACHE_MTX:
+ return &data->cache_mutex;
+ case BOUNCE_MTX:
+ return &data->bounce_mutex;
+ case STATS_MTX:
+ return &data->stats_mutex;
+ }
+ }
+ return NULL;
+}
+#endif
+
+static inline void mutex_lock(struct unix_private_data *data, kind_t kind)
+{
+#ifdef HAVE_PTHREAD
+ pthread_mutex_t *mtx = get_mutex(data,kind);
+
+ if (mtx)
+ pthread_mutex_lock(mtx);
+#endif
+}
+
+static inline void mutex_unlock(struct unix_private_data *data, kind_t kind)
+{
+#ifdef HAVE_PTHREAD
+ pthread_mutex_t *mtx = get_mutex(data,kind);
+
+ if (mtx)
+ pthread_mutex_unlock(mtx);
+#endif
+}
+
static errcode_t unix_get_stats(io_channel channel, io_stats *stats)
{
errcode_t retval = 0;
@@ -122,8 +172,11 @@ static errcode_t unix_get_stats(io_channel channel, io_stats *stats)
data = (struct unix_private_data *) channel->private_data;
EXT2_CHECK_MAGIC(data, EXT2_ET_MAGIC_UNIX_IO_CHANNEL);
- if (stats)
+ if (stats) {
+ mutex_lock(data, STATS_MTX);
*stats = &data->io_stats;
+ mutex_unlock(data, STATS_MTX);
+ }
return retval;
}
@@ -167,7 +220,9 @@ static errcode_t raw_read_blk(io_channel channel,
ssize_t really_read = 0;
size = (count < 0) ? -count : (ext2_loff_t) count * channel->block_size;
+ mutex_lock(data, STATS_MTX);
data->io_stats.bytes_read += size;
+ mutex_unlock(data, STATS_MTX);
location = ((ext2_loff_t) block * channel->block_size) + data->offset;
if (data->flags & IO_FLAG_FORCE_BOUNCE) {
@@ -232,8 +287,10 @@ static errcode_t raw_read_blk(io_channel channel,
*/
bounce_read:
while (size > 0) {
+ mutex_lock(data, BOUNCE_MTX);
actual = read(data->dev, data->bounce, channel->block_size);
if (actual != channel->block_size) {
+ mutex_unlock(data, BOUNCE_MTX);
actual = really_read;
buf -= really_read;
size += really_read;
@@ -246,6 +303,7 @@ bounce_read:
really_read += actual;
size -= actual;
buf += actual;
+ mutex_unlock(data, BOUNCE_MTX);
}
return 0;
@@ -277,7 +335,9 @@ static errcode_t raw_write_blk(io_channel channel,
else
size = (ext2_loff_t) count * channel->block_size;
}
+ mutex_lock(data, STATS_MTX);
data->io_stats.bytes_written += size;
+ mutex_unlock(data, STATS_MTX);
location = ((ext2_loff_t) block * channel->block_size) + data->offset;
@@ -341,11 +401,13 @@ static errcode_t raw_write_blk(io_channel channel,
*/
bounce_write:
while (size > 0) {
+ mutex_lock(data, BOUNCE_MTX);
if (size < channel->block_size) {
actual = read(data->dev, data->bounce,
channel->block_size);
if (actual != channel->block_size) {
if (actual < 0) {
+ mutex_unlock(data, BOUNCE_MTX);
retval = errno;
goto error_out;
}
@@ -362,6 +424,7 @@ bounce_write:
goto error_out;
}
actual = write(data->dev, data->bounce, channel->block_size);
+ mutex_unlock(data, BOUNCE_MTX);
if (actual < 0) {
retval = errno;
goto error_out;
@@ -481,24 +544,28 @@ static void reuse_cache(io_channel channel, struct unix_private_data *data,
cache->access_time = ++data->access_time;
}
+#define FLUSH_INVALIDATE 0x01
+#define FLUSH_NOLOCK 0x02
+
/*
* Flush all of the blocks in the cache
*/
static errcode_t flush_cached_blocks(io_channel channel,
struct unix_private_data *data,
- int invalidate)
-
+ int flags)
{
struct unix_cache *cache;
errcode_t retval, retval2;
int i;
retval2 = 0;
+ if ((flags & FLUSH_NOLOCK) == 0)
+ mutex_lock(data, CACHE_MTX);
for (i=0, cache = data->cache; i < CACHE_SIZE; i++, cache++) {
if (!cache->in_use)
continue;
- if (invalidate)
+ if (flags & FLUSH_INVALIDATE)
cache->in_use = 0;
if (!cache->dirty)
@@ -511,6 +578,8 @@ static errcode_t flush_cached_blocks(io_channel channel,
else
cache->dirty = 0;
}
+ if ((flags & FLUSH_NOLOCK) == 0)
+ mutex_unlock(data, CACHE_MTX);
return retval2;
}
#endif /* NO_IO_CACHE */
@@ -597,6 +666,7 @@ static errcode_t unix_open_channel(const char *name, int fd,
io->read_error = 0;
io->write_error = 0;
io->refcount = 1;
+ io->flags = 0;
memset(data, 0, sizeof(struct unix_private_data));
data->magic = EXT2_ET_MAGIC_UNIX_IO_CHANNEL;
@@ -703,6 +773,25 @@ static errcode_t unix_open_channel(const char *name, int fd,
setrlimit(RLIMIT_FSIZE, &rlim);
}
}
+#endif
+#ifdef HAVE_PTHREAD
+ if (flags & IO_FLAG_THREADS) {
+ io->flags |= CHANNEL_FLAGS_THREADS;
+ retval = pthread_mutex_init(&data->cache_mutex, NULL);
+ if (retval)
+ goto cleanup;
+ retval = pthread_mutex_init(&data->bounce_mutex, NULL);
+ if (retval) {
+ pthread_mutex_destroy(&data->cache_mutex);
+ goto cleanup;
+ }
+ retval = pthread_mutex_init(&data->stats_mutex, NULL);
+ if (retval) {
+ pthread_mutex_destroy(&data->cache_mutex);
+ pthread_mutex_destroy(&data->bounce_mutex);
+ goto cleanup;
+ }
+ }
#endif
*channel = io;
return 0;
@@ -796,6 +885,13 @@ static errcode_t unix_close(io_channel channel)
if (close(data->dev) < 0)
retval = errno;
free_cache(data);
+#ifdef HAVE_PTHREAD
+ if (data->flags & IO_FLAG_THREADS) {
+ pthread_mutex_destroy(&data->cache_mutex);
+ pthread_mutex_destroy(&data->bounce_mutex);
+ pthread_mutex_destroy(&data->stats_mutex);
+ }
+#endif
ext2fs_free_mem(&channel->private_data);
if (channel->name)
@@ -807,24 +903,27 @@ static errcode_t unix_close(io_channel channel)
static errcode_t unix_set_blksize(io_channel channel, int blksize)
{
struct unix_private_data *data;
- errcode_t retval;
+ errcode_t retval = 0;
EXT2_CHECK_MAGIC(channel, EXT2_ET_MAGIC_IO_CHANNEL);
data = (struct unix_private_data *) channel->private_data;
EXT2_CHECK_MAGIC(data, EXT2_ET_MAGIC_UNIX_IO_CHANNEL);
if (channel->block_size != blksize) {
+ mutex_lock(data, CACHE_MTX);
+ mutex_lock(data, BOUNCE_MTX);
#ifndef NO_IO_CACHE
- if ((retval = flush_cached_blocks(channel, data, 0)))
+ if ((retval = flush_cached_blocks(channel, data, FLUSH_NOLOCK)))
return retval;
#endif
channel->block_size = blksize;
free_cache(data);
- if ((retval = alloc_cache(channel, data)))
- return retval;
+ retval = alloc_cache(channel, data);
+ mutex_unlock(data, BOUNCE_MTX);
+ mutex_unlock(data, CACHE_MTX);
}
- return 0;
+ return retval;
}
static errcode_t unix_read_blk64(io_channel channel, unsigned long long block,
@@ -832,7 +931,7 @@ static errcode_t unix_read_blk64(io_channel channel, unsigned long long block,
{
struct unix_private_data *data;
struct unix_cache *cache, *reuse[READ_DIRECT_SIZE];
- errcode_t retval;
+ errcode_t retval = 0;
char *cp;
int i, j;
@@ -854,6 +953,7 @@ static errcode_t unix_read_blk64(io_channel channel, unsigned long long block,
}
cp = buf;
+ mutex_lock(data, CACHE_MTX);
while (count > 0) {
/* If it's in the cache, use it! */
if ((cache = find_cached_block(data, block, &reuse[0]))) {
@@ -876,10 +976,11 @@ static errcode_t unix_read_blk64(io_channel channel, unsigned long long block,
if ((retval = raw_read_blk(channel, data, block, 1,
cache->buf))) {
cache->in_use = 0;
- return retval;
+ break;
}
memcpy(cp, cache->buf, channel->block_size);
- return 0;
+ retval = 0;
+ break;
}
/*
@@ -893,7 +994,7 @@ static errcode_t unix_read_blk64(io_channel channel, unsigned long long block,
printf("Reading %d blocks starting at %lu\n", i, block);
#endif
if ((retval = raw_read_blk(channel, data, block, i, cp)))
- return retval;
+ break;
/* Save the results in the cache */
for (j=0; j < i; j++) {
@@ -904,7 +1005,8 @@ static errcode_t unix_read_blk64(io_channel channel, unsigned long long block,
cp += channel->block_size;
}
}
- return 0;
+ mutex_unlock(data, CACHE_MTX);
+ return retval;
#endif /* NO_IO_CACHE */
}
@@ -935,7 +1037,8 @@ static errcode_t unix_write_blk64(io_channel channel, unsigned long long block,
* flush out the cache completely and then do a direct write.
*/
if (count < 0 || count > WRITE_DIRECT_SIZE) {
- if ((retval = flush_cached_blocks(channel, data, 1)))
+ if ((retval = flush_cached_blocks(channel, data,
+ FLUSH_INVALIDATE)))
return retval;
return raw_write_blk(channel, data, block, count, buf);
}
@@ -950,6 +1053,7 @@ static errcode_t unix_write_blk64(io_channel channel, unsigned long long block,
retval = raw_write_blk(channel, data, block, count, buf);
cp = buf;
+ mutex_lock(data, CACHE_MTX);
while (count > 0) {
cache = find_cached_block(data, block, &reuse);
if (!cache) {
@@ -963,6 +1067,7 @@ static errcode_t unix_write_blk64(io_channel channel, unsigned long long block,
block++;
cp += channel->block_size;
}
+ mutex_unlock(data, CACHE_MTX);
return retval;
#endif /* NO_IO_CACHE */
}
@@ -1013,7 +1118,7 @@ static errcode_t unix_write_byte(io_channel channel, unsigned long offset,
/*
* Flush out the cache completely
*/
- if ((retval = flush_cached_blocks(channel, data, 1)))
+ if ((retval = flush_cached_blocks(channel, data, FLUSH_INVALIDATE)))
return retval;
#endif
--
2.30.0.284.gd98b1dd5eaa7-goog
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [RFC PATCH v1 3/5] libext2fs: allow the unix_io manager's cache to be disabled and re-enabled
2021-01-14 0:27 [RFC PATCH v1 0/5] Add threading support to e2fsprogs Saranya Muruganandam
2021-01-14 0:27 ` [RFC PATCH v1 1/5] Add configure and build support for the pthreads library Saranya Muruganandam
2021-01-14 0:27 ` [RFC PATCH v1 2/5] libext2fs: add threading support to the I/O manager abstraction Saranya Muruganandam
@ 2021-01-14 0:27 ` Saranya Muruganandam
2021-01-14 0:27 ` [RFC PATCH v1 4/5] ext2fs: parallel bitmap loading Saranya Muruganandam
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Saranya Muruganandam @ 2021-01-14 0:27 UTC (permalink / raw)
To: linux-ext4; +Cc: tytso
From: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
lib/ext2fs/ext2_io.h | 1 +
lib/ext2fs/unix_io.c | 19 +++++++++++++++++++
2 files changed, 20 insertions(+)
diff --git a/lib/ext2fs/ext2_io.h b/lib/ext2fs/ext2_io.h
index 2e0da5a5..95c25a7b 100644
--- a/lib/ext2fs/ext2_io.h
+++ b/lib/ext2fs/ext2_io.h
@@ -106,6 +106,7 @@ struct struct_io_manager {
#define IO_FLAG_DIRECT_IO 0x0004
#define IO_FLAG_FORCE_BOUNCE 0x0008
#define IO_FLAG_THREADS 0x0010
+#define IO_FLAG_NOCACHE 0x0020
/*
* Convenience functions....
diff --git a/lib/ext2fs/unix_io.c b/lib/ext2fs/unix_io.c
index 9385487d..2df53e51 100644
--- a/lib/ext2fs/unix_io.c
+++ b/lib/ext2fs/unix_io.c
@@ -942,6 +942,8 @@ static errcode_t unix_read_blk64(io_channel channel, unsigned long long block,
#ifdef NO_IO_CACHE
return raw_read_blk(channel, data, block, count, buf);
#else
+ if (data->flags & IO_FLAG_NOCACHE)
+ return raw_read_blk(channel, data, block, count, buf);
/*
* If we're doing an odd-sized read or a very large read,
* flush out the cache and then do a direct read.
@@ -1032,6 +1034,8 @@ static errcode_t unix_write_blk64(io_channel channel, unsigned long long block,
#ifdef NO_IO_CACHE
return raw_write_blk(channel, data, block, count, buf);
#else
+ if (data->flags & IO_FLAG_NOCACHE)
+ return raw_write_blk(channel, data, block, count, buf);
/*
* If we're doing an odd-sized write or a very large write,
* flush out the cache completely and then do a direct write.
@@ -1161,6 +1165,7 @@ static errcode_t unix_set_option(io_channel channel, const char *option,
{
struct unix_private_data *data;
unsigned long long tmp;
+ errcode_t retval;
char *end;
EXT2_CHECK_MAGIC(channel, EXT2_ET_MAGIC_IO_CHANNEL);
@@ -1179,6 +1184,20 @@ static errcode_t unix_set_option(io_channel channel, const char *option,
return EXT2_ET_INVALID_ARGUMENT;
return 0;
}
+ if (!strcmp(option, "cache")) {
+ if (!arg)
+ return EXT2_ET_INVALID_ARGUMENT;
+ if (!strcmp(arg, "on")) {
+ data->flags &= ~IO_FLAG_NOCACHE;
+ return 0;
+ }
+ if (!strcmp(arg, "off")) {
+ retval = flush_cached_blocks(channel, data, 0);
+ data->flags |= IO_FLAG_NOCACHE;
+ return retval;
+ }
+ return EXT2_ET_INVALID_ARGUMENT;
+ }
return EXT2_ET_INVALID_ARGUMENT;
}
--
2.30.0.284.gd98b1dd5eaa7-goog
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [RFC PATCH v1 4/5] ext2fs: parallel bitmap loading
2021-01-14 0:27 [RFC PATCH v1 0/5] Add threading support to e2fsprogs Saranya Muruganandam
` (2 preceding siblings ...)
2021-01-14 0:27 ` [RFC PATCH v1 3/5] libext2fs: allow the unix_io manager's cache to be disabled and re-enabled Saranya Muruganandam
@ 2021-01-14 0:27 ` Saranya Muruganandam
2021-01-14 0:27 ` [RFC PATCH v1 5/5] Enable threaded support for e2fsprogs' applications Saranya Muruganandam
2021-01-21 15:51 ` [RFC PATCH v1 0/5] Add threading support to e2fsprogs Theodore Ts'o
5 siblings, 0 replies; 7+ messages in thread
From: Saranya Muruganandam @ 2021-01-14 0:27 UTC (permalink / raw)
To: linux-ext4; +Cc: tytso, Wang Shilong, Saranya Muruganandam
From: Wang Shilong <wshilong@ddn.com>
In our benchmarking for PiB size filesystem, pass5 takes
10446s to finish and 99.5% of time takes on reading bitmaps.
It makes sense to reading bitmaps using multiple threads,
a quickly benchmark show 10446s to 626s with 64 threads.
[ This has all of many bug fixes for rw_bitmaps.c from the original
luster patch set collapsed into a single commit. In addition it has
the new ext2fs_rw_bitmaps() api proposed by Ted. ]
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
lib/ext2fs/ext2fs.h | 8 +
lib/ext2fs/rw_bitmaps.c | 332 +++++++++++++++++++++++++++++++++-------
2 files changed, 288 insertions(+), 52 deletions(-)
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 5955c3ae..82ce9126 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -688,6 +688,14 @@ struct ext2_xattr_handle;
#define XATTR_ABORT 1
#define XATTR_CHANGED 2
+/*
+ * flags for ext2fs_rw_bitmaps()
+ */
+#define EXT2FS_BITMAPS_WRITE 0x0001
+#define EXT2FS_BITMAPS_BLOCK 0x0002
+#define EXT2FS_BITMAPS_INODE 0x0004
+#define EXT2FS_BITMAPS_VALID_FLAGS 0x0007
+
/*
* function prototypes
*/
diff --git a/lib/ext2fs/rw_bitmaps.c b/lib/ext2fs/rw_bitmaps.c
index d80c9eb8..7e4f7c6a 100644
--- a/lib/ext2fs/rw_bitmaps.c
+++ b/lib/ext2fs/rw_bitmaps.c
@@ -23,11 +23,33 @@
#ifdef HAVE_SYS_TYPES_H
#include <sys/types.h>
#endif
+#ifdef HAVE_PTHREAD_H
+#include <pthread.h>
+#endif
#include "ext2_fs.h"
#include "ext2fs.h"
#include "e2image.h"
+#ifdef HAVE_PTHREAD
+typedef pthread_mutex_t mutex_t;
+
+static void unix_pthread_mutex_lock(mutex_t *mutex)
+{
+ if (mutex)
+ pthread_mutex_lock(mutex);
+}
+static void unix_pthread_mutex_unlock(mutex_t *mutex)
+{
+ if (mutex)
+ pthread_mutex_unlock(mutex);
+}
+#else
+typedef int mutex_t;
+#define unix_pthread_mutex_lock(mutex_t) do {} while (0)
+#define unix_pthread_mutex_unlock(mutex_t) do {} while (0)
+#endif
+
static errcode_t write_bitmaps(ext2_filsys fs, int do_inode, int do_block)
{
dgrp_t i;
@@ -205,22 +227,12 @@ static int bitmap_tail_verify(unsigned char *bitmap, int first, int last)
return 1;
}
-static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
+static errcode_t read_bitmaps_range_prepare(ext2_filsys fs, int flags)
{
- dgrp_t i;
- char *block_bitmap = 0, *inode_bitmap = 0;
- char *buf;
errcode_t retval;
int block_nbytes = EXT2_CLUSTERS_PER_GROUP(fs->super) / 8;
int inode_nbytes = EXT2_INODES_PER_GROUP(fs->super) / 8;
- int tail_flags = 0;
- int csum_flag;
- unsigned int cnt;
- blk64_t blk;
- blk64_t blk_itr = EXT2FS_B2C(fs, fs->super->s_first_data_block);
- blk64_t blk_cnt;
- ext2_ino_t ino_itr = 1;
- ext2_ino_t ino_cnt;
+ char *buf;
EXT2_CHECK_MAGIC(fs, EXT2_ET_MAGIC_EXT2FS_FILSYS);
@@ -230,12 +242,11 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
fs->write_bitmaps = ext2fs_write_bitmaps;
- csum_flag = ext2fs_has_group_desc_csum(fs);
-
retval = ext2fs_get_mem(strlen(fs->device_name) + 80, &buf);
if (retval)
return retval;
- if (do_block) {
+
+ if (flags & EXT2FS_BITMAPS_BLOCK) {
if (fs->block_map)
ext2fs_free_block_bitmap(fs->block_map);
strcpy(buf, "block bitmap for ");
@@ -243,12 +254,9 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
retval = ext2fs_allocate_block_bitmap(fs, buf, &fs->block_map);
if (retval)
goto cleanup;
- retval = io_channel_alloc_buf(fs->io, 0, &block_bitmap);
- if (retval)
- goto cleanup;
- } else
- block_nbytes = 0;
- if (do_inode) {
+ }
+
+ if (flags & EXT2FS_BITMAPS_INODE) {
if (fs->inode_map)
ext2fs_free_inode_bitmap(fs->inode_map);
strcpy(buf, "inode bitmap for ");
@@ -256,13 +264,62 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
retval = ext2fs_allocate_inode_bitmap(fs, buf, &fs->inode_map);
if (retval)
goto cleanup;
+ }
+ ext2fs_free_mem(&buf);
+
+ return retval;
+
+cleanup:
+ if (flags & EXT2FS_BITMAPS_BLOCK) {
+ ext2fs_free_block_bitmap(fs->block_map);
+ fs->block_map = 0;
+ }
+ if (flags & EXT2FS_BITMAPS_INODE) {
+ ext2fs_free_inode_bitmap(fs->inode_map);
+ fs->inode_map = 0;
+ }
+ if (buf)
+ ext2fs_free_mem(&buf);
+ return retval;
+}
+
+static errcode_t read_bitmaps_range_start(ext2_filsys fs, int flags,
+ dgrp_t start, dgrp_t end,
+ mutex_t *mutex,
+ int *tail_flags)
+{
+ dgrp_t i;
+ char *block_bitmap = 0, *inode_bitmap = 0;
+ errcode_t retval = 0;
+ int block_nbytes = EXT2_CLUSTERS_PER_GROUP(fs->super) / 8;
+ int inode_nbytes = EXT2_INODES_PER_GROUP(fs->super) / 8;
+ int csum_flag;
+ unsigned int cnt;
+ blk64_t blk;
+ blk64_t blk_itr = EXT2FS_B2C(fs, fs->super->s_first_data_block);
+ blk64_t blk_cnt;
+ ext2_ino_t ino_itr = 1;
+ ext2_ino_t ino_cnt;
+
+ csum_flag = ext2fs_has_group_desc_csum(fs);
+
+ if (flags & EXT2FS_BITMAPS_BLOCK) {
+ retval = io_channel_alloc_buf(fs->io, 0, &block_bitmap);
+ if (retval)
+ goto cleanup;
+ } else {
+ block_nbytes = 0;
+ }
+
+ if (flags & EXT2FS_BITMAPS_INODE) {
retval = io_channel_alloc_buf(fs->io, 0, &inode_bitmap);
if (retval)
goto cleanup;
- } else
+ } else {
inode_nbytes = 0;
- ext2fs_free_mem(&buf);
+ }
+ /* io should be null */
if (fs->flags & EXT2_FLAG_IMAGE_FILE) {
blk = (ext2fs_le32_to_cpu(fs->image_header->offset_inodemap) / fs->blocksize);
ino_cnt = fs->super->s_inodes_count;
@@ -300,10 +357,12 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
blk_itr += cnt;
blk_cnt -= cnt;
}
- goto success_cleanup;
+ goto cleanup;
}
- for (i = 0; i < fs->group_desc_count; i++) {
+ blk_itr += ((blk64_t)start * (block_nbytes << 3));
+ ino_itr += ((blk64_t)start * (inode_nbytes << 3));
+ for (i = start; i <= end; i++) {
if (block_bitmap) {
blk = ext2fs_block_bitmap_loc(fs, i);
if ((csum_flag &&
@@ -329,12 +388,14 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
}
if (!bitmap_tail_verify((unsigned char *) block_bitmap,
block_nbytes, fs->blocksize - 1))
- tail_flags |= EXT2_FLAG_BBITMAP_TAIL_PROBLEM;
+ *tail_flags |= EXT2_FLAG_BBITMAP_TAIL_PROBLEM;
} else
memset(block_bitmap, 0, block_nbytes);
cnt = block_nbytes << 3;
+ unix_pthread_mutex_lock(mutex);
retval = ext2fs_set_block_bitmap_range2(fs->block_map,
blk_itr, cnt, block_bitmap);
+ unix_pthread_mutex_unlock(mutex);
if (retval)
goto cleanup;
blk_itr += block_nbytes << 3;
@@ -365,63 +426,225 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
}
if (!bitmap_tail_verify((unsigned char *) inode_bitmap,
inode_nbytes, fs->blocksize - 1))
- tail_flags |= EXT2_FLAG_IBITMAP_TAIL_PROBLEM;
+ *tail_flags |= EXT2_FLAG_IBITMAP_TAIL_PROBLEM;
} else
memset(inode_bitmap, 0, inode_nbytes);
cnt = inode_nbytes << 3;
+ unix_pthread_mutex_lock(mutex);
retval = ext2fs_set_inode_bitmap_range2(fs->inode_map,
ino_itr, cnt, inode_bitmap);
+ unix_pthread_mutex_unlock(mutex);
if (retval)
goto cleanup;
ino_itr += inode_nbytes << 3;
}
}
+cleanup:
+ if (inode_bitmap)
+ ext2fs_free_mem(&inode_bitmap);
+ if (block_bitmap)
+ ext2fs_free_mem(&block_bitmap);
+ return retval;
+}
+
+static errcode_t read_bitmaps_range_end(ext2_filsys fs, int flags,
+ int tail_flags)
+{
+ errcode_t retval;
+
/* Mark group blocks for any BLOCK_UNINIT groups */
- if (do_block) {
+ if (flags & EXT2FS_BITMAPS_BLOCK) {
retval = mark_uninit_bg_group_blocks(fs);
if (retval)
- goto cleanup;
- }
-
-success_cleanup:
- if (inode_bitmap) {
- ext2fs_free_mem(&inode_bitmap);
- fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM;
- }
- if (block_bitmap) {
- ext2fs_free_mem(&block_bitmap);
+ return retval;
fs->flags &= ~EXT2_FLAG_BBITMAP_TAIL_PROBLEM;
}
+ if (flags & EXT2FS_BITMAPS_INODE)
+ fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM;
fs->flags |= tail_flags;
+
return 0;
+}
-cleanup:
- if (do_block) {
+static void read_bitmaps_cleanup_on_error(ext2_filsys fs, int flags)
+{
+ if (flags & EXT2FS_BITMAPS_BLOCK) {
ext2fs_free_block_bitmap(fs->block_map);
fs->block_map = 0;
}
- if (do_inode) {
+ if (flags & EXT2FS_BITMAPS_INODE) {
ext2fs_free_inode_bitmap(fs->inode_map);
fs->inode_map = 0;
}
- if (inode_bitmap)
- ext2fs_free_mem(&inode_bitmap);
- if (block_bitmap)
- ext2fs_free_mem(&block_bitmap);
- if (buf)
- ext2fs_free_mem(&buf);
+}
+
+static errcode_t read_bitmaps_range(ext2_filsys fs, int flags,
+ dgrp_t start, dgrp_t end)
+{
+ errcode_t retval;
+ int tail_flags = 0;
+
+ retval = read_bitmaps_range_prepare(fs, flags);
+ if (retval)
+ return retval;
+
+ retval = read_bitmaps_range_start(fs, flags, start, end,
+ NULL, &tail_flags);
+ if (retval == 0)
+ retval = read_bitmaps_range_end(fs, flags, tail_flags);
+ if (retval)
+ read_bitmaps_cleanup_on_error(fs, flags);
+ return retval;
+}
+
+#ifdef HAVE_PTHREAD
+struct read_bitmaps_thread_info {
+ ext2_filsys rbt_fs;
+ int rbt_flags;
+ dgrp_t rbt_grp_start;
+ dgrp_t rbt_grp_end;
+ errcode_t rbt_retval;
+ pthread_mutex_t *rbt_mutex;
+ int rbt_tail_flags;
+};
+
+static void *read_bitmaps_thread(void *data)
+{
+ struct read_bitmaps_thread_info *rbt = data;
+
+ rbt->rbt_retval = read_bitmaps_range_start(rbt->rbt_fs, rbt->rbt_flags,
+ rbt->rbt_grp_start, rbt->rbt_grp_end,
+ rbt->rbt_mutex, &rbt->rbt_tail_flags);
+ return NULL;
+}
+#endif
+
+errcode_t ext2fs_rw_bitmaps(ext2_filsys fs, int flags, int num_threads)
+{
+#ifdef HAVE_PTHREAD
+ pthread_attr_t attr;
+ pthread_t *thread_ids = NULL;
+ struct read_bitmaps_thread_info *thread_infos = NULL;
+ pthread_mutex_t rbt_mutex = PTHREAD_MUTEX_INITIALIZER;
+ errcode_t retval;
+ errcode_t rc;
+ unsigned flexbg_size = 1 << fs->super->s_log_groups_per_flex;
+ dgrp_t average_group;
+ int i, tail_flags = 0;
+ io_manager manager = unix_io_manager;
+#endif
+
+ if (flags & ~EXT2FS_BITMAPS_VALID_FLAGS)
+ return EXT2_ET_INVALID_ARGUMENT;
+
+ if (flags & EXT2FS_BITMAPS_WRITE)
+ return write_bitmaps(fs, flags & EXT2FS_BITMAPS_INODE,
+ flags & EXT2FS_BITMAPS_BLOCK);
+
+#ifdef HAVE_PTHREAD
+ if (((fs->io->flags & CHANNEL_FLAGS_THREADS) == 0) ||
+ (num_threads == 1) || (fs->flags & EXT2_FLAG_IMAGE_FILE))
+ goto fallback;
+
+ if (num_threads < 0) {
+#if defined(HAVE_SYSCONF) && defined(_SC_NPROCESSORS_CONF)
+ num_threads = sysconf(_SC_NPROCESSORS_CONF);
+#else
+ /*
+ * Guess for now; eventually we should probably define
+ * ext2fs_get_num_cpus() and teach it how to get this info on
+ * MacOS, FreeBSD, etc.
+ * ref: https://stackoverflow.com/questions/150355
+ */
+ num_threads = 4;
+#endif /* HAVE_SYSCONF */
+ }
+ if (num_threads > fs->group_desc_count)
+ num_threads = fs->group_desc_count;
+ average_group = fs->group_desc_count / num_threads;
+ if (ext2fs_has_feature_flex_bg(fs->super)) {
+ average_group = (average_group / flexbg_size) * flexbg_size;
+ }
+ if (average_group == 0)
+ goto fallback;
+
+ io_channel_set_options(fs->io, "cache=off");
+ retval = pthread_attr_init(&attr);
+ if (retval)
+ return retval;
+
+ thread_ids = calloc(sizeof(pthread_t), num_threads);
+ if (!thread_ids)
+ return -ENOMEM;
+
+ thread_infos = calloc(sizeof(struct read_bitmaps_thread_info),
+ num_threads);
+ if (!thread_infos)
+ goto out;
+
+ retval = read_bitmaps_range_prepare(fs, flags);
+ if (retval)
+ goto out;
+
+// fprintf(stdout, "Multiple threads triggered to read bitmaps\n");
+ for (i = 0; i < num_threads; i++) {
+ thread_infos[i].rbt_fs = fs;
+ thread_infos[i].rbt_flags = flags;
+ thread_infos[i].rbt_mutex = &rbt_mutex;
+ thread_infos[i].rbt_tail_flags = 0;
+ if (i == 0)
+ thread_infos[i].rbt_grp_start = 0;
+ else
+ thread_infos[i].rbt_grp_start = average_group * i + 1;
+
+ if (i == num_threads - 1)
+ thread_infos[i].rbt_grp_end = fs->group_desc_count - 1;
+ else
+ thread_infos[i].rbt_grp_end = average_group * (i + 1);
+ retval = pthread_create(&thread_ids[i], &attr,
+ &read_bitmaps_thread, &thread_infos[i]);
+ if (retval)
+ break;
+ }
+ for (i = 0; i < num_threads; i++) {
+ if (!thread_ids[i])
+ break;
+ rc = pthread_join(thread_ids[i], NULL);
+ if (rc && !retval)
+ retval = rc;
+ rc = thread_infos[i].rbt_retval;
+ if (rc && !retval)
+ retval = rc;
+ tail_flags |= thread_infos[i].rbt_tail_flags;
+ }
+out:
+ rc = pthread_attr_destroy(&attr);
+ if (rc && !retval)
+ retval = rc;
+ free(thread_infos);
+ free(thread_ids);
+
+ if (retval == 0)
+ retval = read_bitmaps_range_end(fs, flags, tail_flags);
+ if (retval)
+ read_bitmaps_cleanup_on_error(fs, flags);
+ /* XXX should save and restore cache setting */
+ io_channel_set_options(fs->io, "cache=on");
return retval;
+fallback:
+#endif /* HAVE_PTHREAD */
+ return read_bitmaps_range(fs, flags, 0, fs->group_desc_count - 1);
}
errcode_t ext2fs_read_inode_bitmap(ext2_filsys fs)
{
- return read_bitmaps(fs, 1, 0);
+ return ext2fs_rw_bitmaps(fs, EXT2FS_BITMAPS_INODE, -1);
}
errcode_t ext2fs_read_block_bitmap(ext2_filsys fs)
{
- return read_bitmaps(fs, 0, 1);
+ return ext2fs_rw_bitmaps(fs, EXT2FS_BITMAPS_BLOCK, -1);
}
errcode_t ext2fs_write_inode_bitmap(ext2_filsys fs)
@@ -436,10 +659,15 @@ errcode_t ext2fs_write_block_bitmap (ext2_filsys fs)
errcode_t ext2fs_read_bitmaps(ext2_filsys fs)
{
- if (fs->inode_map && fs->block_map)
- return 0;
+ int flags = 0;
- return read_bitmaps(fs, !fs->inode_map, !fs->block_map);
+ if (!fs->inode_map)
+ flags |= EXT2FS_BITMAPS_INODE;
+ if (!fs->block_map)
+ flags |= EXT2FS_BITMAPS_BLOCK;
+ if (flags == 0)
+ return 0;
+ return ext2fs_rw_bitmaps(fs, flags, -1);
}
errcode_t ext2fs_write_bitmaps(ext2_filsys fs)
--
2.30.0.284.gd98b1dd5eaa7-goog
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [RFC PATCH v1 5/5] Enable threaded support for e2fsprogs' applications.
2021-01-14 0:27 [RFC PATCH v1 0/5] Add threading support to e2fsprogs Saranya Muruganandam
` (3 preceding siblings ...)
2021-01-14 0:27 ` [RFC PATCH v1 4/5] ext2fs: parallel bitmap loading Saranya Muruganandam
@ 2021-01-14 0:27 ` Saranya Muruganandam
2021-01-21 15:51 ` [RFC PATCH v1 0/5] Add threading support to e2fsprogs Theodore Ts'o
5 siblings, 0 replies; 7+ messages in thread
From: Saranya Muruganandam @ 2021-01-14 0:27 UTC (permalink / raw)
To: linux-ext4; +Cc: tytso
From: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
debugfs/debugfs.c | 6 ++++--
e2fsck/unix.c | 2 +-
misc/dumpe2fs.c | 2 +-
misc/e2freefrag.c | 2 +-
misc/e2fuzz.c | 4 ++--
misc/e2image.c | 3 ++-
misc/fuse2fs.c | 3 ++-
misc/tune2fs.c | 3 ++-
resize/main.c | 2 +-
9 files changed, 16 insertions(+), 11 deletions(-)
diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
index 78e55779..132c5f9d 100644
--- a/debugfs/debugfs.c
+++ b/debugfs/debugfs.c
@@ -231,7 +231,8 @@ void do_open_filesys(int argc, char **argv, int sci_idx EXT2FS_ATTR((unused)),
int catastrophic = 0;
blk64_t superblock = 0;
blk64_t blocksize = 0;
- int open_flags = EXT2_FLAG_SOFTSUPP_FEATURES | EXT2_FLAG_64BITS;
+ int open_flags = EXT2_FLAG_SOFTSUPP_FEATURES | EXT2_FLAG_64BITS |
+ EXT2_FLAG_THREADS;
char *data_filename = 0;
char *undo_file = NULL;
@@ -2532,7 +2533,8 @@ int main(int argc, char **argv)
#endif
"[-c]] [device]";
int c;
- int open_flags = EXT2_FLAG_SOFTSUPP_FEATURES | EXT2_FLAG_64BITS;
+ int open_flags = EXT2_FLAG_SOFTSUPP_FEATURES |
+ EXT2_FLAG_64BITS | EXT2_FLAG_THREADS;
char *request = 0;
int exit_status = 0;
char *cmd_file = 0;
diff --git a/e2fsck/unix.c b/e2fsck/unix.c
index 1cb51672..dbeaeef5 100644
--- a/e2fsck/unix.c
+++ b/e2fsck/unix.c
@@ -1474,7 +1474,7 @@ int main (int argc, char *argv[])
}
ctx->superblock = ctx->use_superblock;
- flags = EXT2_FLAG_SKIP_MMP;
+ flags = EXT2_FLAG_SKIP_MMP | EXT2_FLAG_THREADS;
restart:
#ifdef CONFIG_TESTIO_DEBUG
if (getenv("TEST_IO_FLAGS") || getenv("TEST_IO_BLOCK")) {
diff --git a/misc/dumpe2fs.c b/misc/dumpe2fs.c
index d295ba4d..82fb4e63 100644
--- a/misc/dumpe2fs.c
+++ b/misc/dumpe2fs.c
@@ -665,7 +665,7 @@ int main (int argc, char ** argv)
device_name = argv[optind++];
flags = EXT2_FLAG_JOURNAL_DEV_OK | EXT2_FLAG_SOFTSUPP_FEATURES |
- EXT2_FLAG_64BITS;
+ EXT2_FLAG_64BITS | EXT2_FLAG_THREADS;
if (force)
flags |= EXT2_FLAG_FORCE;
if (image_dump)
diff --git a/misc/e2freefrag.c b/misc/e2freefrag.c
index 9c23fadc..a9d16fc4 100644
--- a/misc/e2freefrag.c
+++ b/misc/e2freefrag.c
@@ -363,7 +363,7 @@ static void collect_info(ext2_filsys fs, struct chunk_info *chunk_info, FILE *f)
static void open_device(char *device_name, ext2_filsys *fs)
{
int retval;
- int flag = EXT2_FLAG_FORCE | EXT2_FLAG_64BITS;
+ int flag = EXT2_FLAG_FORCE | EXT2_FLAG_64BITS | EXT2_FLAG_THREADS;
retval = ext2fs_open(device_name, flag, 0, 0, unix_io_manager, fs);
if (retval) {
diff --git a/misc/e2fuzz.c b/misc/e2fuzz.c
index 685cdbe2..1ace1df5 100644
--- a/misc/e2fuzz.c
+++ b/misc/e2fuzz.c
@@ -201,8 +201,8 @@ static int process_fs(const char *fsname)
}
/* Ensure the fs is clean and does not have errors */
- ret = ext2fs_open(fsname, EXT2_FLAG_64BITS, 0, 0, unix_io_manager,
- &fs);
+ ret = ext2fs_open(fsname, EXT2_FLAG_64BITS | EXT2_FLAG_THREADS,
+ 0, 0, unix_io_manager, &fs);
if (ret) {
fprintf(stderr, "%s: failed to open filesystem.\n",
fsname);
diff --git a/misc/e2image.c b/misc/e2image.c
index 892c5371..e5e47565 100644
--- a/misc/e2image.c
+++ b/misc/e2image.c
@@ -1482,7 +1482,8 @@ int main (int argc, char ** argv)
ext2_filsys fs;
char *image_fn, offset_opt[64];
struct ext2_qcow2_hdr *header = NULL;
- int open_flag = EXT2_FLAG_64BITS | EXT2_FLAG_IGNORE_CSUM_ERRORS;
+ int open_flag = EXT2_FLAG_64BITS | EXT2_FLAG_THREADS |
+ EXT2_FLAG_IGNORE_CSUM_ERRORS;
int img_type = 0;
int flags = 0;
int mount_flags = 0;
diff --git a/misc/fuse2fs.c b/misc/fuse2fs.c
index 4005894d..c5957212 100644
--- a/misc/fuse2fs.c
+++ b/misc/fuse2fs.c
@@ -3727,7 +3727,8 @@ int main(int argc, char *argv[])
errcode_t err;
char *logfile;
char extra_args[BUFSIZ];
- int ret = 0, flags = EXT2_FLAG_64BITS | EXT2_FLAG_EXCLUSIVE;
+ int ret = 0;
+ int flags = EXT2_FLAG_64BITS | EXT2_FLAG_THREADS | EXT2_FLAG_EXCLUSIVE;
memset(&fctx, 0, sizeof(fctx));
fctx.magic = FUSE2FS_MAGIC;
diff --git a/misc/tune2fs.c b/misc/tune2fs.c
index f942c698..e5186fe0 100644
--- a/misc/tune2fs.c
+++ b/misc/tune2fs.c
@@ -2950,7 +2950,8 @@ retry_open:
if ((open_flag & EXT2_FLAG_RW) == 0 || f_flag)
open_flag |= EXT2_FLAG_SKIP_MMP;
- open_flag |= EXT2_FLAG_64BITS | EXT2_FLAG_JOURNAL_DEV_OK;
+ open_flag |= EXT2_FLAG_64BITS | EXT2_FLAG_THREADS |
+ EXT2_FLAG_JOURNAL_DEV_OK;
/* keep the filesystem struct around to dump MMP data */
open_flag |= EXT2_FLAG_NOFREE_ON_ERROR;
diff --git a/resize/main.c b/resize/main.c
index cb0bf6a0..72a703f6 100644
--- a/resize/main.c
+++ b/resize/main.c
@@ -402,7 +402,7 @@ int main (int argc, char ** argv)
if (!(mount_flags & EXT2_MF_MOUNTED))
io_flags = EXT2_FLAG_RW | EXT2_FLAG_EXCLUSIVE;
- io_flags |= EXT2_FLAG_64BITS;
+ io_flags |= EXT2_FLAG_64BITS | EXT2_FLAG_THREADS;
if (undo_file) {
retval = resize2fs_setup_tdb(device_name, undo_file, &io_ptr);
if (retval)
--
2.30.0.284.gd98b1dd5eaa7-goog
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [RFC PATCH v1 0/5] Add threading support to e2fsprogs
2021-01-14 0:27 [RFC PATCH v1 0/5] Add threading support to e2fsprogs Saranya Muruganandam
` (4 preceding siblings ...)
2021-01-14 0:27 ` [RFC PATCH v1 5/5] Enable threaded support for e2fsprogs' applications Saranya Muruganandam
@ 2021-01-21 15:51 ` Theodore Ts'o
5 siblings, 0 replies; 7+ messages in thread
From: Theodore Ts'o @ 2021-01-21 15:51 UTC (permalink / raw)
To: Saranya Muruganandam; +Cc: linux-ext4
On Wed, Jan 13, 2021 at 04:27:18PM -0800, Saranya Muruganandam wrote:
> This patch set adds the infrastructure to support threading to
> libext2fs. It makes the unix_io I/O Manager thread-aware. Wang's
> parallel bitmap code has been adapted to use the new threading
> infrastructure.
>
> The code has been tested with TSAN and ASAN built into gcc 10.2:
>
> configure 'CFLAGS=-g -fsanitize=thread' 'LDFLAGS=-fsanitize=thread'
> make clean ; make -j16 ; make -j16 check
> configure 'CFLAGS=-g -fsanitize=address' 'LDFLAGS=-fsanitize=address'
> make clean ; make -j16 ; make -j16 check
>
> As I(tytso) needed to excerpt out some of the changes to generated patches in
> "Add configure and build support for the pthreads", the full patch
> series can be found in git:
>
> git fetch https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git pthreads
Thanks, I've applied this patch series. (There were some slight merge
conflicts due to some other changes from the maint branch so I had to
regenerate the aclocal.m4 and configure files.)
- Ted
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-01-21 15:54 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-01-14 0:27 [RFC PATCH v1 0/5] Add threading support to e2fsprogs Saranya Muruganandam
2021-01-14 0:27 ` [RFC PATCH v1 1/5] Add configure and build support for the pthreads library Saranya Muruganandam
2021-01-14 0:27 ` [RFC PATCH v1 2/5] libext2fs: add threading support to the I/O manager abstraction Saranya Muruganandam
2021-01-14 0:27 ` [RFC PATCH v1 3/5] libext2fs: allow the unix_io manager's cache to be disabled and re-enabled Saranya Muruganandam
2021-01-14 0:27 ` [RFC PATCH v1 4/5] ext2fs: parallel bitmap loading Saranya Muruganandam
2021-01-14 0:27 ` [RFC PATCH v1 5/5] Enable threaded support for e2fsprogs' applications Saranya Muruganandam
2021-01-21 15:51 ` [RFC PATCH v1 0/5] Add threading support to e2fsprogs Theodore Ts'o
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).