* [PATCH 1/8] xfsdump: link with libpthread
2011-11-02 21:10 [PATCH 0/8] xfsdump: enable support for multiple streams Bill Kendall
@ 2011-11-02 21:10 ` Bill Kendall
2011-11-03 8:21 ` Christoph Hellwig
2011-11-02 21:10 ` [PATCH 2/8] xfsdump: remove multi-stream synchronous dir dump Bill Kendall
` (7 subsequent siblings)
8 siblings, 1 reply; 19+ messages in thread
From: Bill Kendall @ 2011-11-02 21:10 UTC (permalink / raw)
To: xfs
This patch changes xfsdump/xfsrestore to link with libpthread in
preparation for enabling multi-stream support. The libpthread checks
are the same as those used in xfsprogs.
Signed-off-by: Bill Kendall <wkendall@sgi.com>
---
configure.in | 3 +++
dump/Makefile | 2 +-
include/builddefs.in | 1 +
m4/Makefile | 1 +
m4/package_pthread.m4 | 19 +++++++++++++++++++
restore/Makefile | 2 +-
6 files changed, 26 insertions(+), 2 deletions(-)
create mode 100644 m4/package_pthread.m4
diff --git a/configure.in b/configure.in
index b7ea1eb..8d4a256 100644
--- a/configure.in
+++ b/configure.in
@@ -67,6 +67,9 @@ AC_MULTILIB($enable_lib64)
AC_PACKAGE_NEED_UUID_H
AC_PACKAGE_NEED_UUIDCOMPARE
+AC_PACKAGE_NEED_PTHREAD_H
+AC_PACKAGE_NEED_PTHREADMUTEXINIT
+
AC_PACKAGE_NEED_NCURSES_H
AC_PACKAGE_WANT_WORKING_LIBNCURSES
diff --git a/dump/Makefile b/dump/Makefile
index ce964a0..d193f68 100644
--- a/dump/Makefile
+++ b/dump/Makefile
@@ -89,7 +89,7 @@ HFILES = $(LOCALINCL)
LHFILES = $(COMMINCL) $(INVINCL)
LINKS = $(COMMINCL) $(COMMON) $(INVINCL) $(INVCOMMON)
LDIRT = $(LINKS)
-LLDLIBS = $(LIBUUID) $(LIBHANDLE) $(LIBATTR) $(LIBRMT)
+LLDLIBS = $(LIBUUID) $(LIBHANDLE) $(LIBATTR) $(LIBRMT) $(LIBPTHREAD)
LTDEPENDENCIES = $(LIBRMT)
LCFLAGS = -DDUMP
diff --git a/include/builddefs.in b/include/builddefs.in
index f69c038..d5cc8d3 100644
--- a/include/builddefs.in
+++ b/include/builddefs.in
@@ -16,6 +16,7 @@ CFLAGS = @CFLAGS@
LIBRMT = $(TOPDIR)/librmt/librmt.la
LIBXFS = @libxfs@
LIBATTR = @libattr@
+LIBPTHREAD = @libpthread@
LIBUUID = @libuuid@
LIBCURSES = @libcurses@
LIBHANDLE = @libhdl@
diff --git a/m4/Makefile b/m4/Makefile
index 8ce63f8..9a35056 100644
--- a/m4/Makefile
+++ b/m4/Makefile
@@ -17,6 +17,7 @@ LSRCFILES = \
package_attrdev.m4 \
package_globals.m4 \
package_ncurses.m4 \
+ package_pthread.m4 \
package_utilies.m4 \
package_uuiddev.m4 \
package_xfslibs.m4 \
diff --git a/m4/package_pthread.m4 b/m4/package_pthread.m4
new file mode 100644
index 0000000..be21d29
--- /dev/null
+++ b/m4/package_pthread.m4
@@ -0,0 +1,19 @@
+AC_DEFUN([AC_PACKAGE_NEED_PTHREAD_H],
+ [ AC_CHECK_HEADERS(pthread.h)
+ if test $ac_cv_header_pthread_h = no; then
+ AC_CHECK_HEADERS(pthread.h,, [
+ echo
+ echo 'FATAL ERROR: could not find a valid pthread header.'
+ exit 1])
+ fi
+ ])
+
+AC_DEFUN([AC_PACKAGE_NEED_PTHREADMUTEXINIT],
+ [ AC_CHECK_LIB(pthread, pthread_mutex_init,, [
+ echo
+ echo 'FATAL ERROR: could not find a valid pthread library.'
+ exit 1
+ ])
+ libpthread=-lpthread
+ AC_SUBST(libpthread)
+ ])
diff --git a/restore/Makefile b/restore/Makefile
index 003ea16..ce3d6b4 100644
--- a/restore/Makefile
+++ b/restore/Makefile
@@ -99,7 +99,7 @@ HFILES = $(LOCALINCL)
LHFILES = $(COMMINCL) $(INVINCL)
LINKS = $(COMMINCL) $(COMMON) $(INVINCL) $(INVCOMMON)
LDIRT = $(LINKS)
-LLDLIBS = $(LIBUUID) $(LIBHANDLE) $(LIBATTR) $(LIBRMT)
+LLDLIBS = $(LIBUUID) $(LIBHANDLE) $(LIBATTR) $(LIBRMT) $(LIBPTHREAD)
LTDEPENDENCIES = $(LIBRMT)
LCFLAGS = -DRESTORE
--
1.7.0.4
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 2/8] xfsdump: remove multi-stream synchronous dir dump
2011-11-02 21:10 [PATCH 0/8] xfsdump: enable support for multiple streams Bill Kendall
2011-11-02 21:10 ` [PATCH 1/8] xfsdump: link with libpthread Bill Kendall
@ 2011-11-02 21:10 ` Bill Kendall
2011-11-03 8:22 ` Christoph Hellwig
2011-11-02 21:10 ` [PATCH 3/8] xfsdump: implement lock abstraction with pthreads Bill Kendall
` (6 subsequent siblings)
8 siblings, 1 reply; 19+ messages in thread
From: Bill Kendall @ 2011-11-02 21:10 UTC (permalink / raw)
To: xfs
When doing multi-stream dumps, the directories are dumped by each
stream in lock-step fashion. A stream backs up a given directory,
then waits for all others to finish that directory, then they all
move on to the next one. During restore a single stream is chosen to
apply the directory dump.
This patch changes xfsdump so that only one stream dumps the
directories. The other streams begin dumping files immediately.
This results in a faster backup, which is presumably why multiple
streams are being used in the first place.
It also removes the barrier routines in qlock.c as they were only
used by the "SYNCDIR" code.
Signed-off-by: Bill Kendall <wkendall@sgi.com>
---
common/qlock.c | 37 ----------
common/qlock.h | 11 ---
dump/content.c | 193 ++++++++++++-----------------------------------------
restore/content.c | 33 +++++++++
4 files changed, 75 insertions(+), 199 deletions(-)
diff --git a/common/qlock.c b/common/qlock.c
index fd0e98d..6c9ee16 100644
--- a/common/qlock.c
+++ b/common/qlock.c
@@ -568,43 +568,6 @@ return 0;
#endif /* HIDDEN */
}
-qbarrierh_t
-qbarrier_alloc( void )
-{
-#ifdef HIDDEN
- barrier_t *barrierp;
-
- /* sanity checks
- */
- ASSERT( qlock_inited );
- ASSERT( qlock_usp );
-
- /* allocate a us barrier
- */
- barrierp = new_barrier( qlock_usp );
- ASSERT( barrierp );
-
- return ( qbarrierh_t )barrierp;
-#else
-return 0;
-#endif /* HIDDEN */
-}
-
-void
-qbarrier( qbarrierh_t qbarrierh, size_t thrdcnt )
-{
-#ifdef HIDDEN
- barrier_t *barrierp = ( barrier_t * )qbarrierh;
-
- /* sanity checks
- */
- ASSERT( qlock_inited );
- ASSERT( qlock_usp );
-
- barrier( barrierp, thrdcnt );
-#endif /* HIDDEN */
-}
-
/* internal ordinal map abstraction
*/
#ifdef HIDDEN
diff --git a/common/qlock.h b/common/qlock.h
index 54469ea..1880aff 100644
--- a/common/qlock.h
+++ b/common/qlock.h
@@ -102,15 +102,4 @@ extern size_t qsemPblocked( qsemh_t qsemh );
/* number of threads currently blocked on this semaphore
*/
-typedef void *qbarrierh_t;
-#define QBARRIERH_NULL 0
- /* opaque handle
- */
-extern qbarrierh_t qbarrier_alloc( void );
- /* allocates a rendezvous barrier
- */
-extern void qbarrier( qbarrierh_t barrierh, size_t thrdcnt );
- /* causes thrdcnt threads to rendezvous
- */
-
#endif /* QLOCK_H */
diff --git a/dump/content.c b/dump/content.c
index 54c97d1..64bfe54 100644
--- a/dump/content.c
+++ b/dump/content.c
@@ -64,9 +64,6 @@
#include "getdents.h"
#include "arch_xlate.h"
-#undef SYNCDIR
-#define SYNCDIR
-
/* max "unsigned long long int"
*/
#define ULONGLONG_MAX 18446744073709551615LLU
@@ -231,7 +228,6 @@ typedef struct extent_group_context extent_group_context_t;
struct pds {
enum { PDS_NULL, /* per-drive activity not begun */
PDS_INOMAP, /* dumping inomap */
- PDS_DIRRENDEZVOUS, /* waiting to dump dirs */
PDS_DIRDUMP, /* dumping dirs */
PDS_NONDIR, /* dumping nondirs */
PDS_INVSYNC, /* waiting for inventory */
@@ -263,9 +259,6 @@ static rv_t dump_dirs( ix_t strmix,
xfs_bstat_t *bstatbufp,
size_t bstatbuflen,
void *inomap_contextp );
-#ifdef SYNCDIR
-static rv_t dump_dirs_rendezvous( void );
-#endif /* SYNCDIR */
static rv_t dump_dir( ix_t strmix,
jdm_fshandle_t *,
intgen_t,
@@ -485,12 +478,6 @@ static bool_t sc_dumpextattrpr = BOOL_TRUE;
static bool_t sc_dumpasoffline = BOOL_FALSE;
/* dump dual-residency HSM files as offline
*/
-#ifdef SYNCDIR
-static size_t sc_thrdsdirdumpsynccnt = 0;
-static size_t sc_thrdswaitingdirdumpsync1 = 0;
-static size_t sc_thrdswaitingdirdumpsync2 = 0;
-static qbarrierh_t sc_barrierh;
-#endif /* SYNCDIR */
static bool_t sc_savequotas = BOOL_TRUE;
/* save quota information in dump
@@ -1466,14 +1453,13 @@ baseuuidbypass:
var_skip( &fsid, inomap_skip );
/* fill in write header template content info. always produce
- * an inomap and dir dump for each media file.
+ * an inomap for each media file. the dirdump flag will be set
+ * in content_stream_dump() for streams which dump the directories.
*/
ASSERT( sizeof( cwhdrtemplatep->ch_specific ) >= sizeof( *scwhdrtemplatep ));
scwhdrtemplatep->cih_mediafiletype = CIH_MEDIAFILETYPE_DATA;
scwhdrtemplatep->cih_level = ( int32_t )sc_level;
- scwhdrtemplatep->cih_dumpattr = CIH_DUMPATTR_INOMAP
- |
- CIH_DUMPATTR_DIRDUMP;
+ scwhdrtemplatep->cih_dumpattr = CIH_DUMPATTR_INOMAP;
if ( subtreecnt ) {
scwhdrtemplatep->cih_dumpattr |= CIH_DUMPATTR_SUBTREE;
}
@@ -1714,22 +1700,6 @@ baseuuidbypass:
}
}
-#ifdef SYNCDIR
- /* allocate a barrier to synchronize directory dumping
- */
- if ( drivecnt > 1 ) {
- sc_barrierh = qbarrier_alloc( );
- }
-
- /* initialize the number of players in the synchronized dir dump.
- * they drop out when last media file complete. MUST be modified
- * under lock( ).
- */
- sc_thrdsdirdumpsynccnt = drivecnt;
-
-#endif /* SYNCDIR */
-
-
return BOOL_TRUE;
}
@@ -1877,10 +1847,6 @@ content_statline( char **linespp[ ] )
strcat( statline[ statlinecnt ],
"dumping inomap" );
break;
- case PDS_DIRRENDEZVOUS:
- strcat( statline[ statlinecnt ],
- "waiting for synchronized directory dump" );
- break;
case PDS_DIRDUMP:
sprintf( &statline[ statlinecnt ]
[ strlen( statline[ statlinecnt ] ) ],
@@ -2157,6 +2123,11 @@ content_stream_dump( ix_t strmix )
scwhdrp->cih_endpt.sp_flags = STARTPT_FLAGS_END;
}
+ // the first stream dumps the directories
+ if ( strmix == 0 ) {
+ scwhdrp->cih_dumpattr |= CIH_DUMPATTR_DIRDUMP;
+ }
+
/* fill in inomap fields of write hdr
*/
inomap_writehdr( scwhdrp );
@@ -2322,39 +2293,41 @@ content_stream_dump( ix_t strmix )
return mlog_exit(EXIT_FAULT, rv);
}
- /* now dump the directories. use the bigstat iterator
- * capability to call my dump_dir function
- * for each directory in the bitmap.
+ /* now dump the directories, if this is a stream that dumps
+ * directories. use the bigstat iterator capability to call
+ * my dump_dir function for each directory in the bitmap.
*/
- sc_stat_pds[ strmix ].pds_dirdone = 0;
- rv = dump_dirs( strmix,
- bstatbufp,
- bstatbuflen,
- inomap_contextp );
- if ( rv == RV_INTR ) {
- stop_requested = BOOL_TRUE;
- goto decision_more;
- }
- if ( rv == RV_EOM ) {
- hit_eom = BOOL_TRUE;
- goto decision_more;
- }
- if ( rv == RV_DRIVE ) {
- free( ( void * )bstatbufp );
- return mlog_exit(EXIT_NORMAL, rv);
- }
- if ( rv == RV_ERROR ) {
- free( ( void * )bstatbufp );
- return mlog_exit(EXIT_ERROR, rv);
- }
- if ( rv == RV_CORE ) {
- free( ( void * )bstatbufp );
- return mlog_exit(EXIT_FAULT, rv);
- }
- ASSERT( rv == RV_OK );
- if ( rv != RV_OK ) {
- free( ( void * )bstatbufp );
- return mlog_exit(EXIT_FAULT, rv);
+ if ( scwhdrp->cih_dumpattr & CIH_DUMPATTR_DIRDUMP ) {
+ sc_stat_pds[ strmix ].pds_dirdone = 0;
+ rv = dump_dirs( strmix,
+ bstatbufp,
+ bstatbuflen,
+ inomap_contextp );
+ if ( rv == RV_INTR ) {
+ stop_requested = BOOL_TRUE;
+ goto decision_more;
+ }
+ if ( rv == RV_EOM ) {
+ hit_eom = BOOL_TRUE;
+ goto decision_more;
+ }
+ if ( rv == RV_DRIVE ) {
+ free( ( void * )bstatbufp );
+ return mlog_exit(EXIT_NORMAL, rv);
+ }
+ if ( rv == RV_ERROR ) {
+ free( ( void * )bstatbufp );
+ return mlog_exit(EXIT_ERROR, rv);
+ }
+ if ( rv == RV_CORE ) {
+ free( ( void * )bstatbufp );
+ return mlog_exit(EXIT_FAULT, rv);
+ }
+ ASSERT( rv == RV_OK );
+ if ( rv != RV_OK ) {
+ free( ( void * )bstatbufp );
+ return mlog_exit(EXIT_FAULT, rv);
+ }
}
/* finally, dump the non-directory files beginning with this
@@ -2527,20 +2500,6 @@ decision_more:
*/
done = all_nondirs_committed;
-#ifdef SYNCDIR
- /* drop out of the synchronous dump game if done
- */
- if ( done ) {
- /* REFERENCED */
- size_t tmpthrdsdirdumpsynccnt;
- lock( );
- tmpthrdsdirdumpsynccnt = sc_thrdsdirdumpsynccnt;
- sc_thrdsdirdumpsynccnt--;
- unlock( );
- ASSERT( tmpthrdsdirdumpsynccnt > 0 );
- }
-#endif /* SYNCDIR */
-
/* tell the inventory about the media file
*/
if ( inv_stmt != INV_TOKEN_NULL ) {
@@ -2833,22 +2792,6 @@ dump_dirs( ix_t strmix,
__s32 buflenout;
intgen_t rval;
-#ifdef SYNCDIR
- /* have all threads rendezvous
- */
- if ( sc_thrdsdirdumpsynccnt > 1 && stream_cnt( ) > 1 ) {
- rv_t rv;
- mlog( bulkstatcallcnt == 0 ? MLOG_VERBOSE : MLOG_NITTY,
- _("waiting for synchronized directory dump\n") );
- sc_stat_pds[ strmix ].pds_phase = PDS_DIRRENDEZVOUS;
- rv = dump_dirs_rendezvous( );
- if ( rv == RV_INTR ) {
- return RV_INTR;
- }
- ASSERT( rv == RV_OK );
- }
-#endif /* SYNCDIR */
-
if ( bulkstatcallcnt == 0 ) {
mlog( MLOG_VERBOSE, _(
"dumping directories\n") );
@@ -2948,58 +2891,6 @@ dump_dirs( ix_t strmix,
/* NOTREACHED */
}
-#ifdef SYNCDIR
-static rv_t
-dump_dirs_rendezvous( void )
-{
- static size_t localsync1;
- static size_t localsync2;
-
- sc_thrdswaitingdirdumpsync2 = 0;
- lock( );
- sc_thrdswaitingdirdumpsync1++;
- localsync1 = sc_thrdswaitingdirdumpsync1;
- localsync2 = sc_thrdswaitingdirdumpsync2;
- unlock( );
- while ( localsync2 == 0
- &&
- localsync1 < min( stream_cnt( ), sc_thrdsdirdumpsynccnt )) {
- sleep( 1 );
- if ( cldmgr_stop_requested( )) {
- lock( );
- sc_thrdswaitingdirdumpsync1--;
- unlock( );
- return RV_INTR;
- }
- lock( );
- localsync1 = sc_thrdswaitingdirdumpsync1;
- localsync2 = sc_thrdswaitingdirdumpsync2;
- unlock( );
- }
- lock( );
- sc_thrdswaitingdirdumpsync1--;
- sc_thrdswaitingdirdumpsync2++;
- localsync2 = sc_thrdswaitingdirdumpsync2;
- unlock( );
- while ( localsync2 < min( stream_cnt( ), sc_thrdsdirdumpsynccnt )) {
- sleep( 1 );
- if ( cldmgr_stop_requested( )) {
- return RV_INTR;
- }
- lock( );
- localsync2 = sc_thrdswaitingdirdumpsync2;
- unlock( );
- }
- if ( cldmgr_stop_requested( )) {
- return RV_INTR;
- }
-
- qbarrier( sc_barrierh, min( stream_cnt( ), sc_thrdsdirdumpsynccnt ));
-
- return RV_OK;
-}
-#endif /* SYNCDIR */
-
static rv_t
dump_dir( ix_t strmix,
jdm_fshandle_t *fshandlep,
diff --git a/restore/content.c b/restore/content.c
index 34fc4a0..2228a7f 100644
--- a/restore/content.c
+++ b/restore/content.c
@@ -625,6 +625,9 @@ struct tran {
intgen_t t_persfd;
/* file descriptor of the persistent state file
*/
+ size64_t t_dirdumps;
+ /* bitset of streams which contain a directory dump
+ */
sync_t t_sync1;
/* to single-thread attempt to validate command line
* selection of dump with online inventory
@@ -1184,6 +1187,12 @@ content_init( intgen_t argc, char *argv[ ], size64_t vmsz )
return BOOL_FALSE;
}
+ /* assume all streams contain a directory dump. streams will remove
+ * themselves from this bitset if they do not contain a directory dump.
+ */
+ ASSERT( drivecnt <= sizeof(tranp->t_dirdumps) * NBBY );
+ tranp->t_dirdumps = ( 1ULL << drivecnt ) - 1;
+
/* the user may specify stdin as the restore source stream,
* by a single dash ('-') with no option letter. This must
* appear between the last lettered argument and the destination
@@ -2237,6 +2246,30 @@ content_stream_restore( ix_t thrdix )
unlock( );
continue;
}
+ if ( !(scrhdrp->cih_dumpattr & CIH_DUMPATTR_DIRDUMP) ) {
+ /* if no streams have a directory dump, issue a
+ * message and exit. first set SYNC_BUSY to prevent
+ * other threads from coming through here and issuing
+ * the same message.
+ */
+ tranp->t_dirdumps &= ~(1ULL << thrdix);
+ if ( !tranp->t_dirdumps ) {
+ tranp->t_sync3 = SYNC_BUSY;
+ }
+ unlock( );
+ if ( !tranp->t_dirdumps ) {
+ mlog( MLOG_VERBOSE | MLOG_ERROR, _(
+ "no directory dump found\n") );
+ Media_end( Mediap );
+ return mlog_exit(EXIT_NORMAL, RV_ERROR);
+ }
+ sleep( 1 );
+ if ( cldmgr_stop_requested( )) {
+ Media_end( Mediap );
+ return mlog_exit(EXIT_NORMAL, RV_INTR);
+ }
+ continue;
+ }
tranp->t_sync3 = SYNC_BUSY;
unlock( );
if ( ! tranp->t_dirattrinitdonepr ) {
--
1.7.0.4
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH 2/8] xfsdump: remove multi-stream synchronous dir dump
2011-11-02 21:10 ` [PATCH 2/8] xfsdump: remove multi-stream synchronous dir dump Bill Kendall
@ 2011-11-03 8:22 ` Christoph Hellwig
0 siblings, 0 replies; 19+ messages in thread
From: Christoph Hellwig @ 2011-11-03 8:22 UTC (permalink / raw)
To: Bill Kendall; +Cc: xfs
On Wed, Nov 02, 2011 at 04:10:48PM -0500, Bill Kendall wrote:
> When doing multi-stream dumps, the directories are dumped by each
> stream in lock-step fashion. A stream backs up a given directory,
> then waits for all others to finish that directory, then they all
> move on to the next one. During restore a single stream is chosen to
> apply the directory dump.
>
> This patch changes xfsdump so that only one stream dumps the
> directories. The other streams begin dumping files immediately.
> This results in a faster backup, which is presumably why multiple
> streams are being used in the first place.
>
> It also removes the barrier routines in qlock.c as they were only
> used by the "SYNCDIR" code.
>
> Signed-off-by: Bill Kendall <wkendall@sgi.com>
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH 3/8] xfsdump: implement lock abstraction with pthreads
2011-11-02 21:10 [PATCH 0/8] xfsdump: enable support for multiple streams Bill Kendall
2011-11-02 21:10 ` [PATCH 1/8] xfsdump: link with libpthread Bill Kendall
2011-11-02 21:10 ` [PATCH 2/8] xfsdump: remove multi-stream synchronous dir dump Bill Kendall
@ 2011-11-02 21:10 ` Bill Kendall
2011-11-03 7:10 ` Christoph Hellwig
2011-11-02 21:10 ` [PATCH 4/8] xfsdump: simplify qlock ordinal bitmap Bill Kendall
` (5 subsequent siblings)
8 siblings, 1 reply; 19+ messages in thread
From: Bill Kendall @ 2011-11-02 21:10 UTC (permalink / raw)
To: xfs
This patch implements xfsdump's locking abstraction (qlock_t and qsem_t)
using pthread locking primitives. The obsolete IRIX locking code has
been removed.
A few debug-level log calls had to be changed to use the "no lock"
flag to prevent locks from being acquired out of order. This is
used in various debug messages throughout xfsdump.
Signed-off-by: Bill Kendall <wkendall@sgi.com>
---
common/main.c | 2 +-
common/qlock.c | 407 +++++++++++++++++++----------------------------------
common/qlock.h | 10 +-
restore/content.c | 17 ++-
4 files changed, 158 insertions(+), 278 deletions(-)
diff --git a/common/main.c b/common/main.c
index 08be16e..be8a921 100644
--- a/common/main.c
+++ b/common/main.c
@@ -359,7 +359,7 @@ main( int argc, char *argv[] )
/* initialize the spinlock allocator
*/
- ok = qlock_init( miniroot );
+ ok = qlock_init( );
if ( ! ok ) {
return mlog_exit(EXIT_ERROR, RV_INIT);
}
diff --git a/common/qlock.c b/common/qlock.c
index 6c9ee16..5b87b3b 100644
--- a/common/qlock.c
+++ b/common/qlock.c
@@ -19,6 +19,8 @@
#include <xfs/xfs.h>
#include <xfs/jdm.h>
+#include <pthread.h>
+
#include "types.h"
#include "qlock.h"
#include "mlog.h"
@@ -27,23 +29,24 @@ struct qlock {
ix_t ql_ord;
/* ordinal position of this lock
*/
- pid_t ql_owner;
- /* who owns this lock
- */
-#ifdef HIDDEN
- ulock_t ql_uslockh;
- /* us lock handle
- */
-#endif /* HIDDEN */
+ pthread_mutex_t ql_mutex;
};
typedef struct qlock qlock_t;
/* internal qlock
*/
-#define QLOCK_SPINS 0x1000
- /* how many times to spin on lock before sleeping for it
- */
+struct qsem {
+ ix_t qs_count;
+ /* current count of this semaphore
+ */
+ ix_t qs_waiters;
+ /* number of threads waiting on semaphore
+ */
+ pthread_mutex_t qs_mutex;
+ pthread_cond_t qs_cond;
+};
+typedef struct qsem qsem_t;
#define QLOCK_THRDCNTMAX 256
/* arbitrary limit on number of threads supported
@@ -67,15 +70,13 @@ static ordmap_t qlock_ordalloced;
*/
struct thrddesc {
- pid_t td_pid;
+ pthread_t td_tid;
ordmap_t td_ordmap;
};
typedef struct thrddesc thrddesc_t;
-#ifdef HIDDEN
static thrddesc_t qlock_thrddesc[ QLOCK_THRDCNTMAX ];
/* holds the ordmap for each thread
*/
-#endif
#define QLOCK_ORDMAP_SET( ordmap, ord ) ( ordmap |= 1U << ord )
/* sets the ordinal bit in an ordmap
@@ -93,21 +94,6 @@ static thrddesc_t qlock_thrddesc[ QLOCK_THRDCNTMAX ];
/* checks if any bits less than ord are set in the ordmap
*/
-#ifdef HIDDEN
-static usptr_t *qlock_usp;
-#else
-static void *qlock_usp;
-#endif /* HIDDEN */
-
- /* pointer to shared arena from which locks are allocated
- */
-
-#ifdef HIDDEN
-static char *qlock_arenaroot = "xfsrestoreqlockarena";
- /* shared arena file name root
- */
-#endif
-
/* REFERENCED */
static bool_t qlock_inited = BOOL_FALSE;
/* to sanity check initialization
@@ -115,22 +101,13 @@ static bool_t qlock_inited = BOOL_FALSE;
/* forward declarations
*/
-#ifdef HIDDEN
-static void qlock_ordmap_add( pid_t pid );
-static ordmap_t *qlock_ordmapp_get( pid_t pid );
-static ix_t qlock_thrdix_get( pid_t pid );
-#endif
+static void qlock_ordmap_add( pthread_t tid );
+static ordmap_t *qlock_ordmapp_get( pthread_t tid );
+static ix_t qlock_thrdix_get( pthread_t tid );
bool_t
-qlock_init( bool_t miniroot )
+qlock_init( void )
{
-#ifdef HIDDEN
- char arenaname[ 100 ];
- /* REFERENCED */
- intgen_t nwritten;
- intgen_t rval;
-#endif
-
/* sanity checks
*/
ASSERT( ! qlock_inited );
@@ -143,44 +120,6 @@ qlock_init( bool_t miniroot )
*/
qlock_ordalloced = 0;
- /* if miniroot, fake it
- */
- if ( miniroot ) {
- qlock_inited = BOOL_TRUE;
- qlock_usp = 0;
- return BOOL_TRUE;
- }
-#ifdef HIDDEN
-
- /* generate the arena name
- */
- nwritten = sprintf( arenaname,
- "/tmp/%s.%d",
- qlock_arenaroot,
- get_pid() );
- ASSERT( nwritten > 0 );
- ASSERT( ( size_t )nwritten < sizeof( arenaname ));
-
- /* configure shared arenas to automatically unlink on last close
- */
- rval = usconfig( CONF_ARENATYPE, ( u_intgen_t )US_SHAREDONLY );
- if ( rval ) {
- mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_NOLOCK,
- _("unable to configure shared arena for auto unlink: %s\n"),
- strerror( errno ));
- return BOOL_FALSE;
- }
-
- /* allocate a shared arena for the locks
- */
- qlock_usp = usinit( arenaname );
- if ( ! qlock_usp ) {
- mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_NOLOCK,
- _("unable to allocate shared arena for thread locks: %s\n"),
- strerror( errno ));
- return BOOL_FALSE;
- }
-
/* now say we are initialized
*/
qlock_inited = BOOL_TRUE;
@@ -191,7 +130,6 @@ qlock_init( bool_t miniroot )
qlock_inited = BOOL_FALSE;
return BOOL_FALSE;
}
-#endif /* HIDDEN */
return BOOL_TRUE;
}
@@ -199,28 +137,13 @@ qlock_init( bool_t miniroot )
bool_t
qlock_thrdinit( void )
{
-#ifdef HIDDEN
- intgen_t rval;
-
/* sanity checks
*/
ASSERT( qlock_inited );
- ASSERT( qlock_usp );
-
- /* add thread to shared arena
- */
- rval = usadd( qlock_usp );
- if ( rval ) {
- mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_NOLOCK,
- _("unable to add thread to shared arena: %s\n"),
- strerror( errno ));
- return BOOL_FALSE;
- }
/* add thread to ordmap list
*/
- qlock_ordmap_add( get_pid() );
-#endif /* HIDDEN */
+ qlock_ordmap_add( pthread_self() );
return BOOL_TRUE;
}
@@ -244,14 +167,9 @@ qlock_alloc( ix_t ord )
qlockp = ( qlock_t * )calloc( 1, sizeof( qlock_t ));
ASSERT( qlockp );
-#ifdef HIDDEN
- /* allocate a us lock: bypass if miniroot
+ /* initialize the mutex
*/
- if ( qlock_usp ) {
- qlockp->ql_uslockh = usnewlock( qlock_usp );
- ASSERT( qlockp->ql_uslockh );
- }
-#endif /* HIDDEN */
+ pthread_mutex_init( &qlockp->ql_mutex, NULL );
/* assign the ordinal position
*/
@@ -263,44 +181,34 @@ qlock_alloc( ix_t ord )
void
qlock_lock( qlockh_t qlockh )
{
-#ifdef HIDDEN
qlock_t *qlockp = ( qlock_t * )qlockh;
- pid_t pid;
+ pthread_t tid;
ix_t thrdix;
ordmap_t *ordmapp;
/* REFERENCED */
- bool_t lockacquired;
-#endif
+ intgen_t rval;
/* sanity checks
*/
ASSERT( qlock_inited );
- /* bypass if miniroot
- */
- if ( ! qlock_usp ) {
- return;
- }
-
-#ifdef HIDDEN
-
- /* get the caller's pid and thread index
+ /* get the caller's tid and thread index
*/
- pid = get_pid();
+ tid = pthread_self();
- thrdix = qlock_thrdix_get( pid );
+ thrdix = qlock_thrdix_get( tid );
/* get the ordmap for this thread
*/
- ordmapp = qlock_ordmapp_get( pid );
+ ordmapp = qlock_ordmapp_get( tid );
- /* assert that this lock not already held
+ /* assert that this lock not already held by this thread
*/
if ( QLOCK_ORDMAP_GET( *ordmapp, qlockp->ql_ord )) {
mlog( MLOG_NORMAL | MLOG_WARNING | MLOG_NOLOCK,
- _("lock already held: thrd %d pid %d ord %d map %x\n"),
+ _("lock already held: thrd %d tid %lu ord %d map %x\n"),
thrdix,
- pid,
+ tid,
qlockp->ql_ord,
*ordmapp );
}
@@ -310,278 +218,253 @@ qlock_lock( qlockh_t qlockh )
*/
if ( QLOCK_ORDMAP_CHK( *ordmapp, qlockp->ql_ord )) {
mlog( MLOG_NORMAL | MLOG_WARNING | MLOG_NOLOCK,
- _("lock ordinal violation: thrd %d pid %d ord %d map %x\n"),
+ _("lock ordinal violation: thrd %d tid %lu ord %d map %x\n"),
thrdix,
- pid,
+ tid,
qlockp->ql_ord,
*ordmapp );
}
ASSERT( ! QLOCK_ORDMAP_CHK( *ordmapp, qlockp->ql_ord ));
- /* acquire the us lock
+ /* acquire the lock
*/
- lockacquired = uswsetlock( qlockp->ql_uslockh, QLOCK_SPINS );
- ASSERT( lockacquired );
-
- /* verify lock is not already held
- */
- ASSERT( ! qlockp->ql_owner );
+ rval = pthread_mutex_lock( &qlockp->ql_mutex );
+ ASSERT( !rval );
/* add ordinal to this threads ordmap
*/
QLOCK_ORDMAP_SET( *ordmapp, qlockp->ql_ord );
-
- /* indicate the lock's owner
- */
- qlockp->ql_owner = pid;
-#endif /* HIDDEN */
}
void
qlock_unlock( qlockh_t qlockh )
{
-#ifdef HIDDEN
qlock_t *qlockp = ( qlock_t * )qlockh;
- pid_t pid;
ordmap_t *ordmapp;
/* REFERENCED */
intgen_t rval;
-#endif
/* sanity checks
*/
ASSERT( qlock_inited );
- /* bypass if miniroot
- */
- if ( ! qlock_usp ) {
- return;
- }
-
-#ifdef HIDDEN
- /* get the caller's pid
- */
- pid = get_pid();
-
/* get the ordmap for this thread
*/
- ordmapp = qlock_ordmapp_get( pid );
+ ordmapp = qlock_ordmapp_get( pthread_self() );
/* verify lock is held by this thread
*/
ASSERT( QLOCK_ORDMAP_GET( *ordmapp, qlockp->ql_ord ));
- ASSERT( qlockp->ql_owner == pid );
-
- /* clear lock owner
- */
- qlockp->ql_owner = 0;
/* clear lock's ord from thread's ord map
*/
QLOCK_ORDMAP_CLR( *ordmapp, qlockp->ql_ord );
- /* release the us lock
+ /* release the lock
*/
- rval = usunsetlock( qlockp->ql_uslockh );
+ rval = pthread_mutex_unlock( &qlockp->ql_mutex );
ASSERT( ! rval );
-#endif /* HIDDEN */
}
qsemh_t
qsem_alloc( ix_t cnt )
{
-#ifdef HIDDEN
- usema_t *usemap;
+ qsem_t *qsemp;
/* sanity checks
*/
ASSERT( qlock_inited );
- ASSERT( qlock_usp );
- /* allocate a us semaphore
+ /* allocate a semaphore
*/
- usemap = usnewsema( qlock_usp, ( intgen_t )cnt );
- ASSERT( usemap );
+ qsemp = ( qsem_t * )calloc( 1, sizeof( qsem_t ));
+ ASSERT( qsemp );
- return ( qsemh_t )usemap;
-#else
- return 0;
-#endif /* HIDDEN */
+ /* initialize the mutex and condition
+ */
+ pthread_mutex_init(&qsemp->qs_mutex, NULL);
+ pthread_cond_init(&qsemp->qs_cond, NULL);
+
+ /* assign the initial count
+ */
+ qsemp->qs_count = cnt;
+
+ return ( qsemh_t )qsemp;
}
void
qsem_free( qsemh_t qsemh )
{
-#ifdef HIDDEN
- usema_t *usemap = ( usema_t * )qsemh;
+ qsem_t *qsemp = ( qsem_t * )qsemh;
/* sanity checks
*/
ASSERT( qlock_inited );
- ASSERT( qlock_usp );
- /* free the us semaphore
+ /* destroy the mutex and condition
+ */
+ pthread_mutex_destroy( &qsemp->qs_mutex );
+ pthread_cond_destroy( &qsemp->qs_cond );
+
+ /* free the semaphore
*/
- usfreesema( usemap, qlock_usp );
-#endif /* HIDDEN */
+ free( qsemp );
}
void
qsemP( qsemh_t qsemh )
{
-#ifdef HIDDEN
- usema_t *usemap = ( usema_t * )qsemh;
+ qsem_t *qsemp = ( qsem_t * )qsemh;
intgen_t rval;
/* sanity checks
*/
ASSERT( qlock_inited );
- ASSERT( qlock_usp );
+
+ /* acquire the lock
+ */
+ rval = pthread_mutex_lock( &qsemp->qs_mutex );
+ ASSERT( !rval );
/* "P" the semaphore
*/
- rval = uspsema( usemap );
- if ( rval != 1 ) {
- mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_NOLOCK,
- _("unable to \"P\" semaphore: "
- "rval == %d, errno == %d (%s)\n"),
- rval,
- errno,
- strerror( errno ));
+ qsemp->qs_waiters++;
+ while ( qsemp->qs_count == 0 ) {
+ pthread_cond_wait( &qsemp->qs_cond, &qsemp->qs_mutex );
}
- ASSERT( rval == 1 );
-#endif /* HIDDEN */
+ qsemp->qs_waiters--;
+ qsemp->qs_count--;
+
+ /* release the lock
+ */
+ rval = pthread_mutex_unlock( &qsemp->qs_mutex );
+ ASSERT( !rval );
}
void
qsemV( qsemh_t qsemh )
{
-#ifdef HIDDEN
- usema_t *usemap = ( usema_t * )qsemh;
+ qsem_t *qsemp = ( qsem_t * )qsemh;
intgen_t rval;
/* sanity checks
*/
ASSERT( qlock_inited );
- ASSERT( qlock_usp );
+
+ /* acquire the lock
+ */
+ rval = pthread_mutex_lock( &qsemp->qs_mutex );
+ ASSERT( !rval );
/* "V" the semaphore
*/
- rval = usvsema( usemap );
- if ( rval != 0 ) {
- mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_NOLOCK,
- _("unable to \"V\" semaphore: "
- "rval == %d, errno == %d (%s)\n"),
- rval,
- errno,
- strerror( errno ));
- }
- ASSERT( rval == 0 );
-#endif /* HIDDEN */
+ qsemp->qs_count++;
+ if ( qsemp->qs_waiters )
+ pthread_cond_signal( &qsemp->qs_cond );
+
+ /* release the lock
+ */
+ rval = pthread_mutex_unlock( &qsemp->qs_mutex );
+ ASSERT( !rval );
}
bool_t
qsemPwouldblock( qsemh_t qsemh )
{
-#ifdef HIDDEN
- usema_t *usemap = ( usema_t * )qsemh;
+ qsem_t *qsemp = ( qsem_t * )qsemh;
+ ix_t count;
intgen_t rval;
-
+
/* sanity checks
*/
ASSERT( qlock_inited );
- ASSERT( qlock_usp );
- /* check the semaphore
+ /* acquire the lock
*/
- rval = ustestsema( usemap );
+ rval = pthread_mutex_lock( &qsemp->qs_mutex );
+ ASSERT( !rval );
- /* if equal to zero, no tokens left. if less than zero, other thread(s)
- * currently waiting.
+ /* check the count
*/
- if ( rval <= 0 ) {
- return BOOL_TRUE;
- } else {
- return BOOL_FALSE;
- }
-#else
-return BOOL_FALSE;
-#endif /* HIDDEN */
+ count = qsemp->qs_count;
+
+ /* release the lock
+ */
+ rval = pthread_mutex_unlock( &qsemp->qs_mutex );
+ ASSERT( !rval );
+
+ return count == 0 ? BOOL_TRUE : BOOL_FALSE;
}
size_t
qsemPavail( qsemh_t qsemh )
{
-#ifdef HIDDEN
- usema_t *usemap = ( usema_t * )qsemh;
+ qsem_t *qsemp = ( qsem_t * )qsemh;
+ ix_t count;
intgen_t rval;
-
+
/* sanity checks
*/
ASSERT( qlock_inited );
- ASSERT( qlock_usp );
- /* check the semaphore
+ /* acquire the lock
*/
- rval = ustestsema( usemap );
+ rval = pthread_mutex_lock( &qsemp->qs_mutex );
+ ASSERT( !rval );
- /* if greater or equal to zero, no one is blocked and that is the number
- * of resources available. if less than zero, absolute value is the
- * number of blocked threads.
+ /* check the count
*/
- if ( rval < 0 ) {
- return 0;
- } else {
- return ( size_t )rval;
- }
-#else
-return 0;
-#endif /* HIDDEN */
+ count = qsemp->qs_count;
+
+ /* release the lock
+ */
+ rval = pthread_mutex_unlock( &qsemp->qs_mutex );
+ ASSERT( !rval );
+
+ return count;
}
size_t
qsemPblocked( qsemh_t qsemh )
{
-#ifdef HIDDEN
- usema_t *usemap = ( usema_t * )qsemh;
+ qsem_t *qsemp = ( qsem_t * )qsemh;
+ ix_t waiters;
intgen_t rval;
-
+
/* sanity checks
*/
ASSERT( qlock_inited );
- ASSERT( qlock_usp );
- /* check the semaphore
+ /* acquire the lock
*/
- rval = ustestsema( usemap );
+ rval = pthread_mutex_lock( &qsemp->qs_mutex );
+ ASSERT( !rval );
- /* if greater or equal to zero, no one is blocked. if less than zero,
- * absolute value is the number of blocked threads.
+ /* check the waiters
*/
- if ( rval < 0 ) {
- return ( size_t )( 0 - rval );
- } else {
- return 0;
- }
-#else
-return 0;
-#endif /* HIDDEN */
+ waiters = qsemp->qs_waiters;
+
+ /* release the lock
+ */
+ rval = pthread_mutex_unlock( &qsemp->qs_mutex );
+ ASSERT( !rval );
+
+ return waiters;
}
/* internal ordinal map abstraction
*/
-#ifdef HIDDEN
static void
-qlock_ordmap_add( pid_t pid )
+qlock_ordmap_add( pthread_t tid )
{
ASSERT( qlock_thrdcnt < QLOCK_THRDCNTMAX );
- qlock_thrddesc[ qlock_thrdcnt ].td_pid = pid;
+ qlock_thrddesc[ qlock_thrdcnt ].td_tid = tid;
qlock_thrddesc[ qlock_thrdcnt ].td_ordmap = 0;
qlock_thrdcnt++;
}
static thrddesc_t *
-qlock_thrddesc_get( pid_t pid )
+qlock_thrddesc_get( pthread_t tid )
{
thrddesc_t *p;
thrddesc_t *endp;
@@ -592,29 +475,27 @@ qlock_thrddesc_get( pid_t pid )
p < endp
;
p++ ) {
- if ( p->td_pid == pid ) {
+ if ( pthread_equal( p->td_tid, tid ) ) {
return p;
}
}
- ASSERT( 0 );
return 0;
}
static ordmap_t *
-qlock_ordmapp_get( pid_t pid )
+qlock_ordmapp_get( pthread_t tid )
{
thrddesc_t *p;
- p = qlock_thrddesc_get( pid );
+ p = qlock_thrddesc_get( tid );
return &p->td_ordmap;
}
static ix_t
-qlock_thrdix_get( pid_t pid )
+qlock_thrdix_get( pthread_t tid )
{
thrddesc_t *p;
- p = qlock_thrddesc_get( pid );
+ p = qlock_thrddesc_get( tid );
ASSERT( p >= &qlock_thrddesc[ 0 ] );
return ( ix_t )( p - &qlock_thrddesc[ 0 ] );
}
-#endif
diff --git a/common/qlock.h b/common/qlock.h
index 1880aff..5ac51bd 100644
--- a/common/qlock.h
+++ b/common/qlock.h
@@ -21,19 +21,13 @@
/* qlock - quick locks abstraction
*
* threads may allocate quick locks using qlock_alloc, and free them with
- * qlock_free. the abstraction is initialized with qlock_init. the underlying
- * mechanism is the IRIX us lock primitive. in order to use this, a temporary
- * shared arena is created in /tmp. this will be automatically unlinked
- * when the last thread exits.
+ * qlock_free. the abstraction is initialized with qlock_init.
*
* deadlock detection is accomplished by giving an ordinal number to each
* lock allocated, and record all locks held by each thread. locks may not
* be acquired out of order. that is, subsequently acquired locks must have
* a lower ordinal than all locks currently held. for convenience, the ordinals
* of all locks to be allocated will be defined in this file.
- *
- * ADDITION: added counting semaphores. simpler to do here since same
- * shared arena can be used.
*/
#define QLOCK_ORD_CRIT 0
@@ -54,7 +48,7 @@ typedef void *qlockh_t;
/* opaque handle
*/
-extern bool_t qlock_init( bool_t miniroot );
+extern bool_t qlock_init( void );
/* called by main to initialize abstraction. returns FALSE if
* utility should abort.
*/
diff --git a/restore/content.c b/restore/content.c
index 2228a7f..8dfa456 100644
--- a/restore/content.c
+++ b/restore/content.c
@@ -8911,7 +8911,8 @@ partial_reg( ix_t d_index,
/* If not found, find a free one, fill it in and return */
if ( ! isptr ) {
- mlog(MLOG_NITTY, "partial_reg: no entry found for %llu\n", ino);
+ mlog(MLOG_NITTY | MLOG_NOLOCK,
+ "partial_reg: no entry found for %llu\n", ino);
/* find a free one */
for (i=0; i < partialmax; i++ ) {
if (persp->a.parrest[i].is_ino == 0) {
@@ -8950,8 +8951,10 @@ found:
/* no existing entry for this drive, fill in the values */
bsptr->offset = offset;
bsptr->endoffset = endoffset;
- mlog(MLOG_NITTY, "partial_reg: update entry [%d]: "
- "<off = %lld, end = %lld>\n", d_index, offset, endoffset);
+ mlog(MLOG_NITTY | MLOG_NOLOCK,
+ "partial_reg: update entry [%d]: "
+ "<off = %lld, end = %lld>\n",
+ d_index, offset, endoffset);
} else {
bool_t ret;
@@ -8960,9 +8963,11 @@ found:
*/
bsptr->endoffset = endoffset;
ret = partial_check2(isptr, fsize);
- mlog(MLOG_NITTY, "partial_reg: extend entry [%d]: "
- "<end = %lld>\n", d_index, endoffset);
- mlog(MLOG_NITTY, "partial_reg: partial_check returns: %d\n", ret);
+ mlog(MLOG_NITTY | MLOG_NOLOCK,
+ "partial_reg: extend entry [%d]: <end = %lld>\n",
+ d_index, endoffset);
+ mlog(MLOG_NITTY | MLOG_NOLOCK,
+ "partial_reg: partial_check returns: %d\n", ret);
}
pi_unlock();
--
1.7.0.4
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 4/8] xfsdump: simplify qlock ordinal bitmap
2011-11-02 21:10 [PATCH 0/8] xfsdump: enable support for multiple streams Bill Kendall
` (2 preceding siblings ...)
2011-11-02 21:10 ` [PATCH 3/8] xfsdump: implement lock abstraction with pthreads Bill Kendall
@ 2011-11-02 21:10 ` Bill Kendall
2011-11-03 8:30 ` Christoph Hellwig
2011-11-02 21:10 ` [PATCH 5/8] xfsdump: convert IRIX sproc threads to pthreads Bill Kendall
` (4 subsequent siblings)
8 siblings, 1 reply; 19+ messages in thread
From: Bill Kendall @ 2011-11-02 21:10 UTC (permalink / raw)
To: xfs
The qlock abstraction manages an array of ordinal bitmaps, one bitmap
for each thread. The bitmap indicates which locks a thread is holding
and is used to verify that the locks are obtained in the correct order.
There is no need to use an array to store the bitmaps, and in fact
adding entries to the array is broken because qlock_thrdcnt is not
protected by a lock. A simpler approach is to create a per-thread bitmap
using thread local storage.
With this change, there is no need for each new thread to register with
the qlock abstraction, so qlock_thrdinit() goes away. Remove
qlock_init() since it can be statically initialized, and that means
there's no need for a flag (qlock_inited) indicating that the module has
been initialized. Also there's no longer a need to track or limit the
number of threads that the qlock abstraction can handle.
Signed-off-by: Bill Kendall <wkendall@sgi.com>
---
common/cldmgr.c | 4 -
common/main.c | 7 --
common/qlock.c | 195 ++++---------------------------------------------------
common/qlock.h | 11 +---
4 files changed, 14 insertions(+), 203 deletions(-)
diff --git a/common/cldmgr.c b/common/cldmgr.c
index 7784a15..d327bab 100644
--- a/common/cldmgr.c
+++ b/common/cldmgr.c
@@ -213,12 +213,8 @@ cldmgr_entry( void *arg1 )
{
cld_t *cldp = ( cld_t * )arg1;
pid_t pid = getpid( );
- /* REFERENCED */
- bool_t ok;
cldp->c_pid = pid;
- ok = qlock_thrdinit( );
- ASSERT( ok );
if ( ( intgen_t )( cldp->c_streamix ) >= 0 ) {
stream_register( pid, ( intgen_t )cldp->c_streamix );
}
diff --git a/common/main.c b/common/main.c
index be8a921..25c0838 100644
--- a/common/main.c
+++ b/common/main.c
@@ -357,13 +357,6 @@ main( int argc, char *argv[] )
miniroot = BOOL_TRUE;
}
- /* initialize the spinlock allocator
- */
- ok = qlock_init( );
- if ( ! ok ) {
- return mlog_exit(EXIT_ERROR, RV_INIT);
- }
-
/* initialize message logging (stage 2) - allocate the message lock
*/
ok = mlog_init2( );
diff --git a/common/qlock.c b/common/qlock.c
index 5b87b3b..67d1291 100644
--- a/common/qlock.c
+++ b/common/qlock.c
@@ -48,14 +48,6 @@ struct qsem {
};
typedef struct qsem qsem_t;
-#define QLOCK_THRDCNTMAX 256
- /* arbitrary limit on number of threads supported
- */
-
-static size_t qlock_thrdcnt;
- /* how many threads have checked in
- */
-
typedef size_t ordmap_t;
/* bitmap of ordinals. used to track what ordinals have
* been allocated.
@@ -69,12 +61,7 @@ static ordmap_t qlock_ordalloced;
/* to enforce allocation of only one lock to each ordinal value
*/
-struct thrddesc {
- pthread_t td_tid;
- ordmap_t td_ordmap;
-};
-typedef struct thrddesc thrddesc_t;
-static thrddesc_t qlock_thrddesc[ QLOCK_THRDCNTMAX ];
+static __thread ordmap_t thread_ordmap;
/* holds the ordmap for each thread
*/
@@ -94,69 +81,12 @@ static thrddesc_t qlock_thrddesc[ QLOCK_THRDCNTMAX ];
/* checks if any bits less than ord are set in the ordmap
*/
-/* REFERENCED */
-static bool_t qlock_inited = BOOL_FALSE;
- /* to sanity check initialization
- */
-
-/* forward declarations
- */
-static void qlock_ordmap_add( pthread_t tid );
-static ordmap_t *qlock_ordmapp_get( pthread_t tid );
-static ix_t qlock_thrdix_get( pthread_t tid );
-
-bool_t
-qlock_init( void )
-{
- /* sanity checks
- */
- ASSERT( ! qlock_inited );
-
- /* initially no threads checked in
- */
- qlock_thrdcnt = 0;
-
- /* initially no ordinals allocated
- */
- qlock_ordalloced = 0;
-
- /* now say we are initialized
- */
- qlock_inited = BOOL_TRUE;
-
- /* add the parent thread to the thread list
- */
- if ( ! qlock_thrdinit( )) {
- qlock_inited = BOOL_FALSE;
- return BOOL_FALSE;
- }
-
- return BOOL_TRUE;
-}
-
-bool_t
-qlock_thrdinit( void )
-{
- /* sanity checks
- */
- ASSERT( qlock_inited );
-
- /* add thread to ordmap list
- */
- qlock_ordmap_add( pthread_self() );
-
- return BOOL_TRUE;
-}
qlockh_t
qlock_alloc( ix_t ord )
{
qlock_t *qlockp;
- /* sanity checks
- */
- ASSERT( qlock_inited );
-
/* verify the ordinal is not already taken, and mark as taken
*/
ASSERT( ! QLOCK_ORDMAP_GET( qlock_ordalloced, ord ));
@@ -183,48 +113,34 @@ qlock_lock( qlockh_t qlockh )
{
qlock_t *qlockp = ( qlock_t * )qlockh;
pthread_t tid;
- ix_t thrdix;
- ordmap_t *ordmapp;
/* REFERENCED */
intgen_t rval;
- /* sanity checks
- */
- ASSERT( qlock_inited );
-
- /* get the caller's tid and thread index
+ /* get the caller's tid
*/
tid = pthread_self();
- thrdix = qlock_thrdix_get( tid );
-
- /* get the ordmap for this thread
- */
- ordmapp = qlock_ordmapp_get( tid );
-
/* assert that this lock not already held by this thread
*/
- if ( QLOCK_ORDMAP_GET( *ordmapp, qlockp->ql_ord )) {
+ if ( QLOCK_ORDMAP_GET( thread_ordmap, qlockp->ql_ord )) {
mlog( MLOG_NORMAL | MLOG_WARNING | MLOG_NOLOCK,
- _("lock already held: thrd %d tid %lu ord %d map %x\n"),
- thrdix,
+ _("lock already held: tid %lu ord %d map %x\n"),
tid,
qlockp->ql_ord,
- *ordmapp );
+ thread_ordmap );
}
- ASSERT( ! QLOCK_ORDMAP_GET( *ordmapp, qlockp->ql_ord ));
+ ASSERT( ! QLOCK_ORDMAP_GET( thread_ordmap, qlockp->ql_ord ));
/* assert that no locks with a lesser ordinal are held by this thread
*/
- if ( QLOCK_ORDMAP_CHK( *ordmapp, qlockp->ql_ord )) {
+ if ( QLOCK_ORDMAP_CHK( thread_ordmap, qlockp->ql_ord )) {
mlog( MLOG_NORMAL | MLOG_WARNING | MLOG_NOLOCK,
- _("lock ordinal violation: thrd %d tid %lu ord %d map %x\n"),
- thrdix,
+ _("lock ordinal violation: tid %lu ord %d map %x\n"),
tid,
qlockp->ql_ord,
- *ordmapp );
+ thread_ordmap );
}
- ASSERT( ! QLOCK_ORDMAP_CHK( *ordmapp, qlockp->ql_ord ));
+ ASSERT( ! QLOCK_ORDMAP_CHK( thread_ordmap, qlockp->ql_ord ));
/* acquire the lock
*/
@@ -233,32 +149,23 @@ qlock_lock( qlockh_t qlockh )
/* add ordinal to this threads ordmap
*/
- QLOCK_ORDMAP_SET( *ordmapp, qlockp->ql_ord );
+ QLOCK_ORDMAP_SET( thread_ordmap, qlockp->ql_ord );
}
void
qlock_unlock( qlockh_t qlockh )
{
qlock_t *qlockp = ( qlock_t * )qlockh;
- ordmap_t *ordmapp;
/* REFERENCED */
intgen_t rval;
- /* sanity checks
- */
- ASSERT( qlock_inited );
-
- /* get the ordmap for this thread
- */
- ordmapp = qlock_ordmapp_get( pthread_self() );
-
/* verify lock is held by this thread
*/
- ASSERT( QLOCK_ORDMAP_GET( *ordmapp, qlockp->ql_ord ));
+ ASSERT( QLOCK_ORDMAP_GET( thread_ordmap, qlockp->ql_ord ));
/* clear lock's ord from thread's ord map
*/
- QLOCK_ORDMAP_CLR( *ordmapp, qlockp->ql_ord );
+ QLOCK_ORDMAP_CLR( thread_ordmap, qlockp->ql_ord );
/* release the lock
*/
@@ -271,10 +178,6 @@ qsem_alloc( ix_t cnt )
{
qsem_t *qsemp;
- /* sanity checks
- */
- ASSERT( qlock_inited );
-
/* allocate a semaphore
*/
qsemp = ( qsem_t * )calloc( 1, sizeof( qsem_t ));
@@ -297,10 +200,6 @@ qsem_free( qsemh_t qsemh )
{
qsem_t *qsemp = ( qsem_t * )qsemh;
- /* sanity checks
- */
- ASSERT( qlock_inited );
-
/* destroy the mutex and condition
*/
pthread_mutex_destroy( &qsemp->qs_mutex );
@@ -317,10 +216,6 @@ qsemP( qsemh_t qsemh )
qsem_t *qsemp = ( qsem_t * )qsemh;
intgen_t rval;
- /* sanity checks
- */
- ASSERT( qlock_inited );
-
/* acquire the lock
*/
rval = pthread_mutex_lock( &qsemp->qs_mutex );
@@ -347,10 +242,6 @@ qsemV( qsemh_t qsemh )
qsem_t *qsemp = ( qsem_t * )qsemh;
intgen_t rval;
- /* sanity checks
- */
- ASSERT( qlock_inited );
-
/* acquire the lock
*/
rval = pthread_mutex_lock( &qsemp->qs_mutex );
@@ -375,10 +266,6 @@ qsemPwouldblock( qsemh_t qsemh )
ix_t count;
intgen_t rval;
- /* sanity checks
- */
- ASSERT( qlock_inited );
-
/* acquire the lock
*/
rval = pthread_mutex_lock( &qsemp->qs_mutex );
@@ -403,10 +290,6 @@ qsemPavail( qsemh_t qsemh )
ix_t count;
intgen_t rval;
- /* sanity checks
- */
- ASSERT( qlock_inited );
-
/* acquire the lock
*/
rval = pthread_mutex_lock( &qsemp->qs_mutex );
@@ -431,10 +314,6 @@ qsemPblocked( qsemh_t qsemh )
ix_t waiters;
intgen_t rval;
- /* sanity checks
- */
- ASSERT( qlock_inited );
-
/* acquire the lock
*/
rval = pthread_mutex_lock( &qsemp->qs_mutex );
@@ -451,51 +330,3 @@ qsemPblocked( qsemh_t qsemh )
return waiters;
}
-
-/* internal ordinal map abstraction
- */
-static void
-qlock_ordmap_add( pthread_t tid )
-{
- ASSERT( qlock_thrdcnt < QLOCK_THRDCNTMAX );
- qlock_thrddesc[ qlock_thrdcnt ].td_tid = tid;
- qlock_thrddesc[ qlock_thrdcnt ].td_ordmap = 0;
- qlock_thrdcnt++;
-}
-
-static thrddesc_t *
-qlock_thrddesc_get( pthread_t tid )
-{
- thrddesc_t *p;
- thrddesc_t *endp;
-
- for ( p = &qlock_thrddesc[ 0 ],
- endp = &qlock_thrddesc[ qlock_thrdcnt ]
- ;
- p < endp
- ;
- p++ ) {
- if ( pthread_equal( p->td_tid, tid ) ) {
- return p;
- }
- }
-
- return 0;
-}
-
-static ordmap_t *
-qlock_ordmapp_get( pthread_t tid )
-{
- thrddesc_t *p;
- p = qlock_thrddesc_get( tid );
- return &p->td_ordmap;
-}
-
-static ix_t
-qlock_thrdix_get( pthread_t tid )
-{
- thrddesc_t *p;
- p = qlock_thrddesc_get( tid );
- ASSERT( p >= &qlock_thrddesc[ 0 ] );
- return ( ix_t )( p - &qlock_thrddesc[ 0 ] );
-}
diff --git a/common/qlock.h b/common/qlock.h
index 5ac51bd..da2031c 100644
--- a/common/qlock.h
+++ b/common/qlock.h
@@ -21,7 +21,7 @@
/* qlock - quick locks abstraction
*
* threads may allocate quick locks using qlock_alloc, and free them with
- * qlock_free. the abstraction is initialized with qlock_init.
+ * qlock_free.
*
* deadlock detection is accomplished by giving an ordinal number to each
* lock allocated, and record all locks held by each thread. locks may not
@@ -48,15 +48,6 @@ typedef void *qlockh_t;
/* opaque handle
*/
-extern bool_t qlock_init( void );
- /* called by main to initialize abstraction. returns FALSE if
- * utility should abort.
- */
-
-extern bool_t qlock_thrdinit( void );
- /* called by each thread to prepare it for participation
- */
-
extern qlockh_t qlock_alloc( ix_t ord );
/* allocates a qlock with the specified ordinal. returns
* NULL if lock can't be allocated.
--
1.7.0.4
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 5/8] xfsdump: convert IRIX sproc threads to pthreads
2011-11-02 21:10 [PATCH 0/8] xfsdump: enable support for multiple streams Bill Kendall
` (3 preceding siblings ...)
2011-11-02 21:10 ` [PATCH 4/8] xfsdump: simplify qlock ordinal bitmap Bill Kendall
@ 2011-11-02 21:10 ` Bill Kendall
2011-11-03 8:31 ` Christoph Hellwig
2011-11-02 21:10 ` [PATCH 6/8] xfsdump: process thread exit status Bill Kendall
` (3 subsequent siblings)
8 siblings, 1 reply; 19+ messages in thread
From: Bill Kendall @ 2011-11-02 21:10 UTC (permalink / raw)
To: xfs
The existing (disabled) threading code in xfsdump is based on IRIX
sprocs. This patch converts the code to use pthreads. The threading code
remains disabled at this point.
Changes:
- pid_t to pthread_t
- getpid() to pthread_self()
- "pid1 == pid2" to pthread_equal(tid1, tid2)
- sigprocmask() to pthread_sigmask()
- sproc() to pthread_create()
Also the following are not referenced and have been removed:
- cldmgr_pid2streamix()
- PROCMAX
- r_slavepid
- sproc.c and sproc.h
Signed-off-by: Bill Kendall <wkendall@sgi.com>
---
common/Makefile | 2 +-
common/cldmgr.c | 56 ++++++++++++++--------------
common/cldmgr.h | 8 +---
common/dlog.c | 4 +-
common/drive_minrmt.c | 2 +-
common/drive_scsitape.c | 2 +-
common/main.c | 20 +++++-----
common/mlog.c | 44 +++++++++++-----------
common/ring.c | 9 +---
common/ring.h | 1 -
common/sproc.c | 42 ---------------------
common/sproc.h | 23 -----------
common/stream.c | 94 +++++++++++++++++++++--------------------------
common/stream.h | 22 +++++-----
dump/Makefile | 4 +-
dump/content.c | 4 +-
restore/Makefile | 2 -
restore/content.c | 5 +-
18 files changed, 127 insertions(+), 217 deletions(-)
delete mode 100644 common/sproc.c
delete mode 100644 common/sproc.h
diff --git a/common/Makefile b/common/Makefile
index 8d9d868..ad3d61a 100644
--- a/common/Makefile
+++ b/common/Makefile
@@ -13,7 +13,7 @@ LSRCFILES = arch_xlate.c arch_xlate.h \
hsmapi.c hsmapi.h inventory.c inventory.h lock.c lock.h \
main.c media.c media.h media_rmvtape.h mlog.c mlog.h \
openutil.c openutil.h path.c path.h qlock.c qlock.h \
- rec_hdr.h ring.c ring.h sproc.c sproc.h stream.c \
+ rec_hdr.h ring.c ring.h stream.c \
stream.h timeutil.c timeutil.h ts_mtio.h types.h util.c util.h
default install install-dev :
diff --git a/common/cldmgr.c b/common/cldmgr.c
index d327bab..4574834 100644
--- a/common/cldmgr.c
+++ b/common/cldmgr.c
@@ -24,6 +24,7 @@
#include <sys/sem.h>
#include <sys/prctl.h>
#include <errno.h>
+#include <pthread.h>
#include "types.h"
#include "lock.h"
@@ -31,14 +32,13 @@
#include "stream.h"
#include "mlog.h"
#include "cldmgr.h"
-#include "sproc.h"
extern size_t pgsz;
#define CLD_MAX ( STREAM_SIMMAX * 2 )
struct cld {
bool_t c_busy;
- pid_t c_pid;
+ pthread_t c_tid;
ix_t c_streamix;
int ( * c_entry )( void *arg1 );
void * c_arg1;
@@ -50,32 +50,31 @@ static cld_t cld[ CLD_MAX ];
static bool_t cldmgr_stopflag;
static cld_t *cldmgr_getcld( void );
-static cld_t * cldmgr_findbypid( pid_t );
-static int cldmgr_entry( void * );
+static cld_t * cldmgr_findbytid( pthread_t );
+static void *cldmgr_entry( void * );
/* REFERENCED */
-static pid_t cldmgr_parentpid;
+static pthread_t cldmgr_parenttid;
bool_t
cldmgr_init( void )
{
( void )memset( ( void * )cld, 0, sizeof( cld ));
cldmgr_stopflag = BOOL_FALSE;
- cldmgr_parentpid = getpid( );
+ cldmgr_parenttid = pthread_self( );
return BOOL_TRUE;
}
bool_t
cldmgr_create( int ( * entry )( void *arg1 ),
- u_intgen_t inh,
ix_t streamix,
char *descstr,
void *arg1 )
{
cld_t *cldp;
- pid_t cldpid;
+ intgen_t rval;
- ASSERT( getpid( ) == cldmgr_parentpid );
+ ASSERT( pthread_equal( pthread_self( ), cldmgr_parenttid ) );
cldp = cldmgr_getcld( );
if ( ! cldp ) {
@@ -91,22 +90,22 @@ cldmgr_create( int ( * entry )( void *arg1 ),
cldp->c_streamix = streamix;
cldp->c_entry = entry;
cldp->c_arg1 = arg1;
- cldpid = ( pid_t )sproc( cldmgr_entry, inh, ( void * )cldp );
- if ( cldpid < 0 ) {
+ rval = pthread_create( &cldp->c_tid, NULL, cldmgr_entry, cldp );
+ if ( rval ) {
mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_PROC, _(
- "sproc failed creating %s thread for stream %u: %s\n"),
+ "failed creating %s thread for stream %u: %s\n"),
descstr,
streamix,
- strerror( errno ));
+ strerror( rval ));
} else {
mlog( MLOG_NITTY | MLOG_PROC,
- "%s thread created for stream %u: pid %d\n",
+ "%s thread created for stream %u: tid %lu\n",
descstr,
streamix,
- cldpid );
+ cldp->c_tid );
}
- return cldpid < 0 ? BOOL_FALSE : BOOL_TRUE;
+ return rval ? BOOL_FALSE : BOOL_TRUE;
}
void
@@ -119,16 +118,16 @@ cldmgr_stop( void )
}
void
-cldmgr_died( pid_t pid )
+cldmgr_died( pthread_t tid )
{
- cld_t *cldp = cldmgr_findbypid( pid );
+ cld_t *cldp = cldmgr_findbytid( tid );
if ( ! cldp ) {
return;
}
cldp->c_busy = BOOL_FALSE;
if ( ( intgen_t )( cldp->c_streamix ) >= 0 ) {
- stream_dead( pid );
+ stream_dead( tid );
}
}
@@ -194,13 +193,13 @@ cldmgr_getcld( void )
}
static cld_t *
-cldmgr_findbypid( pid_t pid )
+cldmgr_findbytid( pthread_t tid )
{
cld_t *p = cld;
cld_t *ep = cld + sizeof( cld ) / sizeof( cld[ 0 ] );
for ( ; p < ep ; p++ ) {
- if ( p->c_busy && p->c_pid == pid ) {
+ if ( p->c_busy && pthread_equal( p->c_tid, tid )) {
break;
}
}
@@ -208,19 +207,20 @@ cldmgr_findbypid( pid_t pid )
return ( p < ep ) ? p : 0;
}
-static int
+static void *
cldmgr_entry( void *arg1 )
{
cld_t *cldp = ( cld_t * )arg1;
- pid_t pid = getpid( );
+ pthread_t tid = pthread_self( );
- cldp->c_pid = pid;
if ( ( intgen_t )( cldp->c_streamix ) >= 0 ) {
- stream_register( pid, ( intgen_t )cldp->c_streamix );
+ stream_register( tid, ( intgen_t )cldp->c_streamix );
}
mlog( MLOG_DEBUG | MLOG_PROC,
- "child %d created for stream %d\n",
- pid,
+ "thread %lu created for stream %d\n",
+ tid,
cldp->c_streamix );
- return ( * cldp->c_entry )( cldp->c_arg1 );
+
+ ( * cldp->c_entry )( cldp->c_arg1 );
+ return NULL;
}
diff --git a/common/cldmgr.h b/common/cldmgr.h
index bb3f612..e393b82 100644
--- a/common/cldmgr.h
+++ b/common/cldmgr.h
@@ -30,7 +30,6 @@ extern bool_t cldmgr_init( void );
* encountered
*/
extern bool_t cldmgr_create( int ( * entry )( void *arg1 ),
- u_intgen_t inh,
ix_t streamix,
char *descstr,
void *arg1 );
@@ -42,18 +41,13 @@ extern void cldmgr_stop( void );
/* cldmgr_died - tells the child manager that the child died
*/
-extern void cldmgr_died( pid_t pid );
+extern void cldmgr_died( pthread_t tid );
/* cldmgr_stop_requested - returns TRUE if the child should gracefully
* terminate.
*/
extern bool_t cldmgr_stop_requested( void );
-/* cldmgr_pid2streamix - retrieves the stream index. returns -1 if
- * not associated with any stream.
- */
-extern intgen_t cldmgr_pid2streamix( pid_t pid );
-
/* cldmgr_remainingcnt - returns number of children remaining
*/
extern size_t cldmgr_remainingcnt( void );
diff --git a/common/dlog.c b/common/dlog.c
index 8cf9a65..51666cf 100644
--- a/common/dlog.c
+++ b/common/dlog.c
@@ -429,7 +429,7 @@ promptinput( char *buf,
sigaddset( &dlog_registered_sigs, SIGQUIT );
}
- sigprocmask( SIG_UNBLOCK, &dlog_registered_sigs, &orig_set );
+ pthread_sigmask( SIG_UNBLOCK, &dlog_registered_sigs, &orig_set );
/* wait for input, timeout, or interrupt.
* note we come out of the select() frequently in order to
@@ -455,7 +455,7 @@ promptinput( char *buf,
/* restore signal handling
*/
- sigprocmask( SIG_SETMASK, &orig_set, NULL );
+ pthread_sigmask( SIG_SETMASK, &orig_set, NULL );
sigemptyset( &dlog_registered_sigs );
/* check for timeout or interrupt
diff --git a/common/drive_minrmt.c b/common/drive_minrmt.c
index 836b663..3ff4d0f 100644
--- a/common/drive_minrmt.c
+++ b/common/drive_minrmt.c
@@ -577,7 +577,7 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded )
drivep->d_cap_est = -1;
drivep->d_rate_est = -1;
- /* if sproc not allowed, allocate a record buffer. otherwise
+ /* if threads not allowed, allocate a record buffer. otherwise
* create a ring, from which buffers will be taken.
*/
if ( singlethreaded ) {
diff --git a/common/drive_scsitape.c b/common/drive_scsitape.c
index 8c1bd49..f24d604 100644
--- a/common/drive_scsitape.c
+++ b/common/drive_scsitape.c
@@ -667,7 +667,7 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded )
drivep->d_cap_est = -1;
drivep->d_rate_est = -1;
- /* if sproc not allowed, allocate a record buffer. otherwise
+ /* if threads not allowed, allocate a record buffer. otherwise
* create a ring, from which buffers will be taken.
*/
if ( singlethreaded ) {
diff --git a/common/main.c b/common/main.c
index 25c0838..d4dbe28 100644
--- a/common/main.c
+++ b/common/main.c
@@ -32,6 +32,7 @@
#include <getopt.h>
#include <stdint.h>
#include <sched.h>
+#include <pthread.h>
#include "exit.h"
#include "types.h"
@@ -117,7 +118,7 @@ bool_t miniroot = BOOL_TRUE;
#endif /* HIDDEN */
bool_t pipeline = BOOL_FALSE;
bool_t stdoutpiped = BOOL_FALSE;
-pid_t parentpid;
+pthread_t parenttid;
char *sistr;
size_t pgsz;
size_t pgmask;
@@ -195,10 +196,10 @@ main( int argc, char *argv[] )
*/
mlog_init0();
- /* Get the parent's pid. will be used in signal handling
+ /* Get the parent's pthread id. will be used
* to differentiate parent from children.
*/
- parentpid = getpid( );
+ parenttid = pthread_self( );
rval = atexit(mlog_exit_flush);
assert(rval == 0);
@@ -395,11 +396,11 @@ main( int argc, char *argv[] )
ASSERT( ( intgen_t )pgsz > 0 );
pgmask = pgsz - 1;
- /* report parent pid
+ /* report parent tid
*/
mlog( MLOG_DEBUG | MLOG_PROC,
- "parent pid is %d\n",
- parentpid );
+ "parent tid is %lu\n",
+ parenttid );
/* get the current working directory: this is where we will dump
* core, if necessary. some tmp files may be placed here as well.
@@ -572,7 +573,7 @@ main( int argc, char *argv[] )
sigaddset( &blocked_set, SIGTERM );
sigaddset( &blocked_set, SIGQUIT );
sigaddset( &blocked_set, SIGALRM );
- sigprocmask( SIG_SETMASK, &blocked_set, NULL );
+ pthread_sigmask( SIG_SETMASK, &blocked_set, NULL );
sa.sa_handler = sighandler;
sigaction( SIGINT, &sa, NULL );
@@ -676,7 +677,6 @@ main( int argc, char *argv[] )
if ( ! init_error ) {
for ( stix = 0 ; stix < drivecnt ; stix++ ) {
ok = cldmgr_create( childmain,
- CLONE_VM,
stix,
"child",
( void * )stix );
@@ -895,7 +895,7 @@ main( int argc, char *argv[] )
if ( coredump_requested ) {
mlog( MLOG_DEBUG | MLOG_PROC,
"core dump requested, aborting (pid %d)\n",
- parentpid );
+ getpid() );
abort();
}
@@ -1560,7 +1560,7 @@ childmain( void *arg1 )
drivep = drivepp[ stix ];
( * drivep->d_opsp->do_quit )( drivep );
- exit( exitcode );
+ return exitcode;
}
diff --git a/common/mlog.c b/common/mlog.c
index 2265895..b0135b9 100644
--- a/common/mlog.c
+++ b/common/mlog.c
@@ -26,6 +26,7 @@
#include <unistd.h>
#include <time.h>
#include <getopt.h>
+#include <pthread.h>
#include "types.h"
#include "qlock.h"
@@ -40,7 +41,7 @@
extern char *progname;
extern void usage( void );
-extern pid_t parentpid;
+extern pthread_t parenttid;
#ifdef DUMP
static FILE *mlog_fp = NULL; /* stderr */;
@@ -385,7 +386,7 @@ mlog_va( intgen_t levelarg, char *fmt, va_list args )
if ( ! ( levelarg & MLOG_BARE )) {
intgen_t streamix;
- streamix = stream_getix( getpid() );
+ streamix = stream_getix( pthread_self( ) );
if ( mlog_showss ) {
sprintf( mlog_ssstr, ":%s", mlog_ss_names[ ss ] );
@@ -568,10 +569,10 @@ rv_getdesc(rv_t rv)
int
_mlog_exit( const char *file, int line, int exit_code, rv_t rv )
{
- pid_t pid;
+ pthread_t tid;
const struct rv_map *rvp;
- pid = getpid();
+ tid = pthread_self();
rvp = rv_getdesc(rv);
@@ -595,7 +596,7 @@ _mlog_exit( const char *file, int line, int exit_code, rv_t rv )
* most accurate information about the termination condition.
*/
- if (pid == parentpid) {
+ if ( pthread_equal( tid, parenttid ) ) {
if (mlog_main_exit_code == -1) {
mlog_main_exit_code = exit_code;
mlog_main_exit_return = rv;
@@ -608,7 +609,7 @@ _mlog_exit( const char *file, int line, int exit_code, rv_t rv )
int exit_code;
rv_t exit_return, exit_hint;
- if (stream_get_exit_status(pid,
+ if (stream_get_exit_status(tid,
states,
N(states),
&state,
@@ -618,8 +619,8 @@ _mlog_exit( const char *file, int line, int exit_code, rv_t rv )
&exit_hint))
{
if (exit_code == -1) {
- stream_set_code(pid, exit_code);
- stream_set_return(pid, rv);
+ stream_set_code(tid, exit_code);
+ stream_set_return(tid, rv);
}
}
}
@@ -630,10 +631,10 @@ _mlog_exit( const char *file, int line, int exit_code, rv_t rv )
void
_mlog_exit_hint( const char *file, int line, rv_t rv )
{
- pid_t pid;
+ pthread_t tid;
const struct rv_map *rvp;
- pid = getpid();
+ tid = pthread_self();
rvp = rv_getdesc(rv);
mlog( MLOG_DEBUG | MLOG_NOLOCK,
@@ -655,10 +656,10 @@ _mlog_exit_hint( const char *file, int line, rv_t rv )
* information about the termination condition.
*/
- if (pid == parentpid)
+ if ( pthread_equal( tid, parenttid ) )
mlog_main_exit_hint = rv;
else
- stream_set_hint( pid, rv );
+ stream_set_hint( tid, rv );
}
@@ -670,10 +671,10 @@ mlog_get_hint( void )
bool_t ok;
rv_t hint;
- if (getpid() == parentpid)
+ if ( pthread_equal( pthread_self(), parenttid ) )
return mlog_main_exit_hint;
- ok = stream_get_exit_status(getpid(), states, N(states),
+ ok = stream_get_exit_status(pthread_self(), states, N(states),
NULL, NULL, NULL, NULL, &hint);
ASSERT(ok);
return hint;
@@ -697,8 +698,8 @@ mlog_get_hint( void )
void
mlog_exit_flush(void)
{
- pid_t pids[STREAM_SIMMAX];
- int i, npids;
+ pthread_t tids[STREAM_SIMMAX];
+ int i, ntids;
const struct rv_map *rvp;
stream_state_t states[] = { S_RUNNING, S_ZOMBIE };
bool_t incomplete = BOOL_FALSE;
@@ -713,13 +714,13 @@ mlog_exit_flush(void)
if (mlog_main_exit_hint == RV_USAGE)
return;
- npids = stream_find_all(states, N(states), pids, STREAM_SIMMAX);
- if (npids > 0) {
+ ntids = stream_find_all(states, N(states), tids, STREAM_SIMMAX);
+ if (ntids > 0) {
/* print the state of all the streams */
fprintf(mlog_fp, _("%s: %s Summary:\n"), progname, PROGSTR_CAPS );
- for (i = 0; i < npids; i++) {
+ for (i = 0; i < ntids; i++) {
stream_state_t state;
intgen_t streamix;
int exit_code;
@@ -727,7 +728,7 @@ mlog_exit_flush(void)
/* REFERENCED */
bool_t ok;
- ok = stream_get_exit_status(pids[i],
+ ok = stream_get_exit_status(tids[i],
states,
N(states),
&state,
@@ -743,11 +744,10 @@ mlog_exit_flush(void)
/* print status of this stream */
rvp = rv_getdesc(rv);
fprintf(mlog_fp,
- _("%s: stream %d (pid %d) %s "
+ _("%s: stream %d %s "
"%s (%s)\n"),
progname,
streamix,
- pids[i],
drivepp[streamix]->d_pathname,
rvp->rv_string,
rvp->rv_desc);
diff --git a/common/ring.c b/common/ring.c
index e1dddba..42bd7d7 100644
--- a/common/ring.c
+++ b/common/ring.c
@@ -123,7 +123,6 @@ ring_create( size_t ringlen,
/* kick off the slave thread
*/
ok = cldmgr_create( ring_slave_entry,
- CLONE_VM,
drive_index,
_("slave"),
ringp );
@@ -421,11 +420,7 @@ ring_slave_entry( void *ringctxp )
sigaddset( &blocked_set, SIGTERM );
sigaddset( &blocked_set, SIGQUIT );
sigaddset( &blocked_set, SIGALRM );
- sigprocmask( SIG_SETMASK, &blocked_set, NULL );
-
- /* record slave pid to be used to kill slave
- */
- ringp->r_slavepid = getpid( );
+ pthread_sigmask( SIG_SETMASK, &blocked_set, NULL );
/* loop reading and precessing messages until told to die
*/
@@ -498,5 +493,5 @@ ring_slave_entry( void *ringctxp )
ring_slave_put( ringp, msgp );
}
- exit( 0 );
+ return 0;
}
diff --git a/common/ring.h b/common/ring.h
index 56e3924..caa505c 100644
--- a/common/ring.h
+++ b/common/ring.h
@@ -134,7 +134,6 @@ struct ring {
time32_t r_first_io_time;
off64_t r_all_io_cnt;
/* ALL BELOW PRIVATE!!! */
- pid_t r_slavepid;
size_t r_len;
ring_msg_t *r_msgp;
size_t r_ready_in_ix;
diff --git a/common/sproc.c b/common/sproc.c
deleted file mode 100644
index 3b1425a..0000000
--- a/common/sproc.c
+++ /dev/null
@@ -1,42 +0,0 @@
-/*
- * Copyright (c) 2000-2001 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-
-#include <stdlib.h>
-#include <signal.h>
-#include <sched.h>
-
-#define STACKSIZE 65536
-
-int
-sproc (int (*entry) (void *), int flags, void *arg)
-{
- int retval = -1;
-#ifdef HIDDEN
- void *newstack;
-
- if ( (newstack = calloc (1, STACKSIZE)) != NULL ) {
- void ** stackp = ((void **)newstack)+(STACKSIZE -1)/sizeof(void*);
-
- flags |= SIGCHLD;
-
- retval = clone (entry, stackp, flags, arg);
- }
-#endif
-
- return retval;
-}
diff --git a/common/sproc.h b/common/sproc.h
deleted file mode 100644
index 46111ff..0000000
--- a/common/sproc.h
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Copyright (c) 2000-2001 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef SPROC_H
-#define SPROC_H
-
-int sproc (int (*) (void *), int, void *);
-
-#endif /* SPROC_H */
diff --git a/common/stream.c b/common/stream.c
index adaf7c0..48e25ee 100644
--- a/common/stream.c
+++ b/common/stream.c
@@ -19,18 +19,19 @@
#include <xfs/xfs.h>
#include <xfs/jdm.h>
+#include <pthread.h>
+
#include "types.h"
#include "exit.h"
#include "stream.h"
#include "lock.h"
#include "mlog.h"
-#define PROCMAX ( STREAM_SIMMAX * 2 + 1 )
#define N(a) (sizeof((a)) / sizeof((a)[0]))
struct spm {
stream_state_t s_state;
- pid_t s_pid;
+ pthread_t s_tid;
intgen_t s_ix;
int s_exit_code;
rv_t s_exit_return;
@@ -38,37 +39,28 @@ struct spm {
};
typedef struct spm spm_t;
-extern pid_t parentpid;
static spm_t spm[ STREAM_SIMMAX * 3 ];
static bool_t initialized = BOOL_FALSE;
void
stream_init( void )
{
-#ifdef HIDDEN
- /* REFERENCED */
- intgen_t rval;
-
- rval = ( intgen_t )usconfig( CONF_INITUSERS, PROCMAX );
- ASSERT( rval >= 0 );
-#endif /* HIDDEN */
-
( void )memset( ( void * )spm, 0, sizeof( spm ));
initialized = BOOL_TRUE;
}
/*
* Note that the stream list structure (updated via the stream_* functions)
- * is indexed by pid. Multiple processes can be registered against the same
- * stream index, typically: the primary content process that does the work;
- * and the drive slave process, which just processes stuff off the ring buffer.
- * In general having multiple pids registered per stream is not an issue for
- * termination status reporting, as the mlog_exit* logging functions only
+ * is indexed by pthread_t (tid). Multiple processes can be registered against
+ * the same stream index, typically: the primary content process that does the
+ * work; and the drive slave process, which just processes stuff off the ring
+ * buffer. In general having multiple tids registered per stream is not an issue
+ * for termination status reporting, as the mlog_exit* logging functions only
* ever get called out of the primary content process.
*/
void
-stream_register( pid_t pid, intgen_t streamix )
+stream_register( pthread_t tid, intgen_t streamix )
{
spm_t *p = spm;
spm_t *ep = spm + N(spm);
@@ -87,7 +79,7 @@ stream_register( pid_t pid, intgen_t streamix )
if ( p >= ep ) return;
- p->s_pid = pid;
+ p->s_tid = tid;
p->s_ix = streamix;
p->s_exit_code = -1;
p->s_exit_return = RV_NONE;
@@ -95,14 +87,14 @@ stream_register( pid_t pid, intgen_t streamix )
}
void
-stream_dead( pid_t pid )
+stream_dead( pthread_t tid )
{
spm_t *p = spm;
spm_t *ep = spm + N(spm);
lock();
for ( ; p < ep ; p++ )
- if ( p->s_pid == pid ) {
+ if ( pthread_equal( p->s_tid, tid ) ) {
p->s_state = S_ZOMBIE;
break;
}
@@ -111,14 +103,14 @@ stream_dead( pid_t pid )
}
void
-stream_free( pid_t pid )
+stream_free( pthread_t tid )
{
spm_t *p = spm;
spm_t *ep = spm + N(spm);
lock();
for ( ; p < ep ; p++ ) {
- if ( p->s_pid == pid ) {
+ if ( pthread_equal( p->s_tid, tid ) ) {
(void) memset( (void *) p, 0, sizeof(spm_t) );
p->s_state = S_FREE;
break;
@@ -130,22 +122,22 @@ stream_free( pid_t pid )
int
stream_find_all( stream_state_t states[], int nstates,
- pid_t pids[], int npids )
+ pthread_t tids[], int ntids )
{
int i, count = 0;
spm_t *p = spm;
spm_t *ep = spm + N(spm);
- ASSERT(nstates > 0 && npids > 0);
+ ASSERT(nstates > 0 && ntids > 0);
if (!initialized)
return 0;
/* lock - make sure we get a consistent snapshot of the stream status */
lock();
- for ( ; p < ep && count < npids; p++ )
+ for ( ; p < ep && count < ntids; p++ )
for (i = 0; i < nstates; i++)
if (p->s_state == states[i]) {
- pids[count++] = p->s_pid;
+ tids[count++] = p->s_tid;
break;
}
unlock();
@@ -153,7 +145,7 @@ stream_find_all( stream_state_t states[], int nstates,
}
static spm_t *
-stream_find( pid_t pid, stream_state_t s[], int nstates )
+stream_find( pthread_t tid, stream_state_t s[], int nstates )
{
int i;
spm_t *p = spm;
@@ -163,7 +155,7 @@ stream_find( pid_t pid, stream_state_t s[], int nstates )
/* note we don't lock the stream array in this function */
for ( ; p < ep ; p++ )
- if ( p->s_pid == pid ) {
+ if ( pthread_equal( p->s_tid, tid ) ) {
/* check state */
for (i = 0; i < nstates; i++)
if (p->s_state == s[i])
@@ -174,8 +166,8 @@ stream_find( pid_t pid, stream_state_t s[], int nstates )
{
static const char *state_strings[] = { "S_FREE", "S_RUNNING", "S_ZOMBIE" };
mlog( MLOG_DEBUG | MLOG_ERROR | MLOG_NOLOCK | MLOG_BARE,
- "stream_find(): no stream with pid: %d and state%s:",
- pid, nstates == 1 ? "" : "s" );
+ "stream_find(): no stream with tid: %lu and state%s:",
+ tid, nstates == 1 ? "" : "s" );
for (i = 0; i < nstates; i++)
mlog( MLOG_DEBUG | MLOG_ERROR | MLOG_NOLOCK | MLOG_BARE,
" %s", state_strings[s[i]]);
@@ -187,20 +179,18 @@ stream_find( pid_t pid, stream_state_t s[], int nstates )
}
/*
- * Note, the following function is called from two places:
- * main.c:sighandler(), and mlog.c:mlog_va() in the first case we
- * aren't allowed to take locks, and in the second locking may be
- * disabled and we are already protected by another lock. So no
- * locking is done in this function.
+ * Note, the following function is called from mlog.c:mlog_va(),
+ * where locking may be disabled and we are already protected by
+ * another lock. So no locking is done in this function.
*/
intgen_t
-stream_getix( pid_t pid )
+stream_getix( pthread_t tid )
{
stream_state_t states[] = { S_RUNNING };
spm_t *p;
intgen_t ix;
- p = stream_find( pid, states, N(states) );
+ p = stream_find( tid, states, N(states) );
ix = p ? p->s_ix : -1;
return ix;
}
@@ -213,43 +203,43 @@ stream_getix( pid_t pid )
* streams.
*/
-#define stream_set(field_name, pid, value) \
+#define stream_set(field_name, tid, value) \
stream_state_t states[] = { S_RUNNING }; \
spm_t *p; \
- pid_t mypid = getpid(); \
+ pthread_t mytid = pthread_self(); \
\
- if (mypid != (pid)) { \
+ if ( !pthread_equal(mytid, (tid))) { \
mlog( MLOG_DEBUG | MLOG_ERROR | MLOG_NOLOCK, \
"stream_set_" #field_name "(): " \
- "foreign stream (pid %d) " \
- "not permitted to update this stream (pid %d)\n", \
- mypid, (pid)); \
+ "foreign stream (tid %lu) " \
+ "not permitted to update this stream (tid %lu)\n",\
+ mytid, (tid)); \
return; \
} \
\
lock(); \
- p = stream_find( (pid), states, N(states) ); \
+ p = stream_find( (tid), states, N(states) ); \
if (p) p->s_exit_ ## field_name = (value); \
unlock();
-void stream_set_code( pid_t pid, int exit_code )
+void stream_set_code( pthread_t tid, int exit_code )
{
- stream_set( code, pid, exit_code );
+ stream_set( code, tid, exit_code );
}
-void stream_set_return( pid_t pid, rv_t rv )
+void stream_set_return( pthread_t tid, rv_t rv )
{
- stream_set( return, pid, rv );
+ stream_set( return, tid, rv );
}
-void stream_set_hint( pid_t pid, rv_t rv )
+void stream_set_hint( pthread_t tid, rv_t rv )
{
- stream_set( hint, pid, rv );
+ stream_set( hint, tid, rv );
}
bool_t
-stream_get_exit_status( pid_t pid,
+stream_get_exit_status( pthread_t tid,
stream_state_t states[],
int nstates,
stream_state_t *state,
@@ -262,7 +252,7 @@ stream_get_exit_status( pid_t pid,
spm_t *p;
lock();
- p = stream_find( pid, states, nstates );
+ p = stream_find( tid, states, nstates );
if (! p) goto unlock;
if (state) *state = p->s_state;
diff --git a/common/stream.h b/common/stream.h
index 984634c..292792e 100644
--- a/common/stream.h
+++ b/common/stream.h
@@ -42,19 +42,19 @@
typedef enum { S_FREE, S_RUNNING, S_ZOMBIE } stream_state_t;
extern void stream_init( void );
-extern void stream_register( pid_t pid, intgen_t streamix );
-extern void stream_dead( pid_t pid );
-extern void stream_free( pid_t pid );
+extern void stream_register( pthread_t tid, intgen_t streamix );
+extern void stream_dead( pthread_t tid );
+extern void stream_free( pthread_t tid );
extern int stream_find_all( stream_state_t states[],
int nstates,
- pid_t pids[],
- int npids );
-extern intgen_t stream_getix( pid_t pid );
-extern void stream_set_code( pid_t pid, int code );
-extern void stream_set_return( pid_t pid, rv_t rv );
-extern void stream_set_hint( pid_t pid, rv_t rv );
-extern bool_t stream_exists( pid_t pid );
-extern bool_t stream_get_exit_status( pid_t pid,
+ pthread_t tids[],
+ int ntids );
+extern intgen_t stream_getix( pthread_t tid );
+extern void stream_set_code( pthread_t tid, int code );
+extern void stream_set_return( pthread_t tid, rv_t rv );
+extern void stream_set_hint( pthread_t tid, rv_t rv );
+extern bool_t stream_exists( pthread_t tid );
+extern bool_t stream_get_exit_status( pthread_t tid,
stream_state_t states[],
int nstates,
stream_state_t *state,
diff --git a/dump/Makefile b/dump/Makefile
index d193f68..97879fa 100644
--- a/dump/Makefile
+++ b/dump/Makefile
@@ -30,7 +30,6 @@ COMMINCL = \
ts_mtio.h \
types.h \
util.h \
- sproc.h \
rec_hdr.h
INVINCL = \
@@ -68,8 +67,7 @@ COMMON = \
ring.c \
stream.c \
timeutil.c \
- util.c \
- sproc.c
+ util.c
LOCALS = \
content.c \
diff --git a/dump/content.c b/dump/content.c
index 64bfe54..33f1166 100644
--- a/dump/content.c
+++ b/dump/content.c
@@ -1666,12 +1666,12 @@ baseuuidbypass:
sigaddset( &tty_set, SIGINT );
sigaddset( &tty_set, SIGQUIT );
sigaddset( &tty_set, SIGHUP );
- sigprocmask( SIG_BLOCK, &tty_set, &orig_set );
+ pthread_sigmask( SIG_BLOCK, &tty_set, &orig_set );
result = create_inv_session( gwhdrtemplatep, &fsid, mntpnt,
fsdevice, subtreecnt, strmix );
- sigprocmask( SIG_SETMASK, &orig_set, NULL );
+ pthread_sigmask( SIG_SETMASK, &orig_set, NULL );
if ( !result ) {
return BOOL_FALSE;
diff --git a/restore/Makefile b/restore/Makefile
index ce3d6b4..c6f3f25 100644
--- a/restore/Makefile
+++ b/restore/Makefile
@@ -25,7 +25,6 @@ COMMINCL = \
qlock.h \
rec_hdr.h \
ring.h \
- sproc.h \
stream.h \
timeutil.h \
ts_mtio.h \
@@ -64,7 +63,6 @@ COMMON = \
path.c \
qlock.c \
ring.c \
- sproc.c \
stream.c \
timeutil.c \
util.c
diff --git a/restore/content.c b/restore/content.c
index 8dfa456..0108a40 100644
--- a/restore/content.c
+++ b/restore/content.c
@@ -33,6 +33,7 @@
#include <dirent.h>
#include <utime.h>
#include <malloc.h>
+#include <pthread.h>
#include "types.h"
#include "timeutil.h"
@@ -2227,7 +2228,7 @@ content_stream_restore( ix_t thrdix )
#if DEBUG_DUMPSTREAMS
{
static int count[STREAM_MAX] = {0};
- intgen_t streamix = stream_getix( getpid() );
+ intgen_t streamix = stream_getix( pthread_self() );
if (++(count[streamix]) == 30) {
mlog( MLOG_TRACE,
"still waiting for dirs to be restored\n");
@@ -2390,7 +2391,7 @@ content_stream_restore( ix_t thrdix )
#if DEBUG_DUMPSTREAMS
{
static int count[STREAM_MAX] = {0};
- intgen_t streamix = stream_getix( getpid() );
+ intgen_t streamix = stream_getix( pthread_self() );
if (++(count[streamix]) == 30) {
mlog( MLOG_NORMAL,
"still waiting for dirs post-processing\n");
--
1.7.0.4
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 6/8] xfsdump: process thread exit status
2011-11-02 21:10 [PATCH 0/8] xfsdump: enable support for multiple streams Bill Kendall
` (4 preceding siblings ...)
2011-11-02 21:10 ` [PATCH 5/8] xfsdump: convert IRIX sproc threads to pthreads Bill Kendall
@ 2011-11-02 21:10 ` Bill Kendall
2011-11-03 8:32 ` Christoph Hellwig
2011-11-02 21:10 ` [PATCH 7/8] xfsdump: path lookup cache must be thread specific Bill Kendall
` (2 subsequent siblings)
8 siblings, 1 reply; 19+ messages in thread
From: Bill Kendall @ 2011-11-02 21:10 UTC (permalink / raw)
To: xfs
When IRIX sprocs were in use, the main thread was notified of a thread
exit just as if a child process exited -- it received SIGCHLD. The main
thread would grab the pid and exit status, then call cldmgr_died() to
inform it that the child was gone so the slot in the child array could
be freed up for reuse.
This patch implements a similar mechanism for pthreads. The "c_busy"
field in struct cld has been replaced with a "c_state" field that
indicates whether the array slot is free (C_AVAIL), in use (C_ALIVE), or
is waiting to be joined (C_EXITED). Additionally a "c_exit_code" field
has been added to store the thread's exit value. Normally this is set
when the thread entry function returns, but it is initialized to
EXIT_INTERRUPT in case the thread is cancelled or calls pthread_exit()
rather than returning (neither of which happens in the code today).
When the child thread starts, it registers a pthread cleanup handler
which takes care of marking the child as C_EXITED and notifies the main
thread that a child is gone. Doing this in a cleanup handler ensures
that it's done regardless of how the thread exits. The main thread's
loop is based around sigsuspsend(), so the notification is done by
sending SIGUSR1. The main thread will then call cldmgr_join() to join
all exited threads and obtain their exit status.
Additional changes:
* cldmgr_findbypid() has been removed, it's no longer referenced.
* stream_dead() no longer grabs the lock(), because it's called
only from cldmgr_join() which already holds the lock().
Signed-off-by: Bill Kendall <wkendall@sgi.com>
---
common/cldmgr.c | 88 +++++++++++++++++++++++++++++++++++-------------------
common/cldmgr.h | 7 +++-
common/main.c | 33 +++++---------------
common/stream.c | 3 +-
common/stream.h | 1 +
5 files changed, 73 insertions(+), 59 deletions(-)
diff --git a/common/cldmgr.c b/common/cldmgr.c
index 4574834..be7de34 100644
--- a/common/cldmgr.c
+++ b/common/cldmgr.c
@@ -26,6 +26,7 @@
#include <errno.h>
#include <pthread.h>
+#include "exit.h"
#include "types.h"
#include "lock.h"
#include "qlock.h"
@@ -36,8 +37,12 @@
extern size_t pgsz;
#define CLD_MAX ( STREAM_SIMMAX * 2 )
+
+typedef enum { C_AVAIL, C_ALIVE, C_EXITED } state_t;
+
struct cld {
- bool_t c_busy;
+ state_t c_state;
+ intgen_t c_exit_code;
pthread_t c_tid;
ix_t c_streamix;
int ( * c_entry )( void *arg1 );
@@ -50,8 +55,8 @@ static cld_t cld[ CLD_MAX ];
static bool_t cldmgr_stopflag;
static cld_t *cldmgr_getcld( void );
-static cld_t * cldmgr_findbytid( pthread_t );
static void *cldmgr_entry( void * );
+static void cldmgr_cleanup( void * );
/* REFERENCED */
static pthread_t cldmgr_parenttid;
@@ -87,6 +92,7 @@ cldmgr_create( int ( * entry )( void *arg1 ),
return BOOL_FALSE;
}
+ cldp->c_exit_code = EXIT_INTERRUPT;
cldp->c_streamix = streamix;
cldp->c_entry = entry;
cldp->c_arg1 = arg1;
@@ -117,18 +123,37 @@ cldmgr_stop( void )
cldmgr_stopflag = BOOL_TRUE;
}
-void
-cldmgr_died( pthread_t tid )
+intgen_t
+cldmgr_join( void )
{
- cld_t *cldp = cldmgr_findbytid( tid );
+ cld_t *p = cld;
+ cld_t *ep = cld + sizeof( cld ) / sizeof( cld[ 0 ] );
+ intgen_t xc = EXIT_NORMAL;
- if ( ! cldp ) {
- return;
- }
- cldp->c_busy = BOOL_FALSE;
- if ( ( intgen_t )( cldp->c_streamix ) >= 0 ) {
- stream_dead( tid );
+ lock();
+ for ( ; p < ep ; p++ ) {
+ if ( p->c_state == C_EXITED ) {
+ if ( ( intgen_t )( p->c_streamix ) >= 0 ) {
+ stream_dead( p->c_tid );
+ }
+ pthread_join( p->c_tid, NULL );
+ if ( p->c_exit_code != EXIT_NORMAL && xc != EXIT_FAULT )
+ xc = p->c_exit_code;
+ if ( p->c_exit_code != EXIT_NORMAL ) {
+ mlog( MLOG_DEBUG | MLOG_PROC | MLOG_NOLOCK,
+ "child (thread %lu) requested stop: "
+ "exit code %d (%s)\n",
+ p->c_tid, p->c_exit_code,
+ exit_codestring( p->c_exit_code ));
+ }
+
+ // reinit this child for reuse
+ memset( ( void * )p, 0, sizeof( cld_t ));
+ }
}
+ unlock();
+
+ return xc;
}
bool_t
@@ -147,7 +172,7 @@ cldmgr_remainingcnt( void )
cnt = 0;
lock( );
for ( ; p < ep ; p++ ) {
- if ( p->c_busy ) {
+ if ( p->c_state == C_ALIVE ) {
cnt++;
}
}
@@ -164,7 +189,7 @@ cldmgr_otherstreamsremain( ix_t streamix )
lock( );
for ( ; p < ep ; p++ ) {
- if ( p->c_busy && p->c_streamix != streamix ) {
+ if ( p->c_state == C_ALIVE && p->c_streamix != streamix ) {
unlock( );
return BOOL_TRUE;
}
@@ -182,8 +207,8 @@ cldmgr_getcld( void )
lock();
for ( ; p < ep ; p++ ) {
- if ( ! p->c_busy ) {
- p->c_busy = BOOL_TRUE;
+ if ( p->c_state == C_AVAIL ) {
+ p->c_state = C_ALIVE;
break;
}
}
@@ -192,27 +217,14 @@ cldmgr_getcld( void )
return ( p < ep ) ? p : 0;
}
-static cld_t *
-cldmgr_findbytid( pthread_t tid )
-{
- cld_t *p = cld;
- cld_t *ep = cld + sizeof( cld ) / sizeof( cld[ 0 ] );
-
- for ( ; p < ep ; p++ ) {
- if ( p->c_busy && pthread_equal( p->c_tid, tid )) {
- break;
- }
- }
-
- return ( p < ep ) ? p : 0;
-}
-
static void *
cldmgr_entry( void *arg1 )
{
cld_t *cldp = ( cld_t * )arg1;
pthread_t tid = pthread_self( );
+ pthread_cleanup_push( cldmgr_cleanup, arg1 );
+
if ( ( intgen_t )( cldp->c_streamix ) >= 0 ) {
stream_register( tid, ( intgen_t )cldp->c_streamix );
}
@@ -220,7 +232,21 @@ cldmgr_entry( void *arg1 )
"thread %lu created for stream %d\n",
tid,
cldp->c_streamix );
+ cldp->c_exit_code = ( * cldp->c_entry )( cldp->c_arg1 );
+
+ pthread_cleanup_pop( 1 );
- ( * cldp->c_entry )( cldp->c_arg1 );
return NULL;
}
+
+static void
+cldmgr_cleanup( void *arg1 )
+{
+ cld_t *cldp = ( cld_t * )arg1;
+
+ lock();
+ cldp->c_state = C_EXITED;
+ // signal the main thread to look for exited threads
+ kill( getpid( ), SIGUSR1 );
+ unlock();
+}
diff --git a/common/cldmgr.h b/common/cldmgr.h
index e393b82..1df0c0c 100644
--- a/common/cldmgr.h
+++ b/common/cldmgr.h
@@ -39,9 +39,12 @@ extern bool_t cldmgr_create( int ( * entry )( void *arg1 ),
*/
extern void cldmgr_stop( void );
-/* cldmgr_died - tells the child manager that the child died
+/* cldmgr_join - join child threads that have exited.
+ * returns EXIT_NORMAL if all exited normally (or no threads have exited),
+ * EXIT_FAULT if any threads requested a core dump, or another EXIT_*
+ * value if any threads exited abnormally.
*/
-extern void cldmgr_died( pthread_t tid );
+extern intgen_t cldmgr_join( void );
/* cldmgr_stop_requested - returns TRUE if the child should gracefully
* terminate.
diff --git a/common/main.c b/common/main.c
index d4dbe28..38b3889 100644
--- a/common/main.c
+++ b/common/main.c
@@ -137,10 +137,6 @@ static bool_t sighup_received;
static bool_t sigterm_received;
static bool_t sigquit_received;
static bool_t sigint_received;
-static size_t prbcld_cnt;
-static pid_t prbcld_pid;
-static intgen_t prbcld_xc;
-static intgen_t prbcld_signo;
/* REFERENCED */
static intgen_t sigstray_received;
static bool_t progrpt_enabledpr;
@@ -168,6 +164,8 @@ main( int argc, char *argv[] )
intgen_t exitcode;
rlim64_t tmpstacksz;
struct sigaction sa;
+ intgen_t prbcld_xc = EXIT_NORMAL;
+ intgen_t xc;
bool_t ok;
/* REFERENCED */
int rval;
@@ -563,7 +561,6 @@ main( int argc, char *argv[] )
sigint_received = BOOL_FALSE;
sigquit_received = BOOL_FALSE;
sigstray_received = BOOL_FALSE;
- prbcld_cnt = 0;
alarm( 0 );
@@ -573,6 +570,7 @@ main( int argc, char *argv[] )
sigaddset( &blocked_set, SIGTERM );
sigaddset( &blocked_set, SIGQUIT );
sigaddset( &blocked_set, SIGALRM );
+ sigaddset( &blocked_set, SIGUSR1 );
pthread_sigmask( SIG_SETMASK, &blocked_set, NULL );
sa.sa_handler = sighandler;
@@ -581,6 +579,7 @@ main( int argc, char *argv[] )
sigaction( SIGTERM, &sa, NULL );
sigaction( SIGQUIT, &sa, NULL );
sigaction( SIGALRM, &sa, NULL );
+ sigaction( SIGUSR1, &sa, NULL );
}
/* do content initialization.
@@ -710,31 +709,16 @@ main( int argc, char *argv[] )
* stop. furthermore, note that core should be dumped if
* the child explicitly exited with EXIT_FAULT.
*/
- if ( prbcld_cnt ) {
- if ( prbcld_xc == EXIT_FAULT || prbcld_signo != 0 ) {
+ xc = cldmgr_join( );
+ if ( xc ) {
+ if ( xc == EXIT_FAULT ) {
coredump_requested = BOOL_TRUE;
stop_timeout = ABORT_TIMEOUT;
} else {
stop_timeout = STOP_TIMEOUT;
}
+ prbcld_xc = xc;
stop_requested = BOOL_TRUE;
- if ( prbcld_xc != EXIT_NORMAL ) {
- mlog( MLOG_DEBUG | MLOG_PROC,
- "child (pid %d) requested stop: "
- "exit code %d (%s)\n",
- prbcld_pid,
- prbcld_xc,
- exit_codestring( prbcld_xc ));
- } else if ( prbcld_signo ) {
- ASSERT( prbcld_signo );
- mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_PROC,
- _("child (pid %d) faulted: "
- "signal number %d (%s)\n"),
- prbcld_pid,
- prbcld_signo,
- sig_numstring( prbcld_signo ));
- }
- prbcld_cnt = 0;
}
/* all children died normally. break out.
@@ -1528,6 +1512,7 @@ sighandler( int signo )
sigquit_received = BOOL_TRUE;
break;
case SIGALRM:
+ case SIGUSR1:
break;
default:
sigstray_received = signo;
diff --git a/common/stream.c b/common/stream.c
index 48e25ee..6704661 100644
--- a/common/stream.c
+++ b/common/stream.c
@@ -86,19 +86,18 @@ stream_register( pthread_t tid, intgen_t streamix )
p->s_exit_hint = RV_NONE;
}
+/* NOTE: lock() must be held when calling stream_dead() */
void
stream_dead( pthread_t tid )
{
spm_t *p = spm;
spm_t *ep = spm + N(spm);
- lock();
for ( ; p < ep ; p++ )
if ( pthread_equal( p->s_tid, tid ) ) {
p->s_state = S_ZOMBIE;
break;
}
- unlock();
ASSERT( p < ep );
}
diff --git a/common/stream.h b/common/stream.h
index 292792e..4b3799f 100644
--- a/common/stream.h
+++ b/common/stream.h
@@ -43,6 +43,7 @@ typedef enum { S_FREE, S_RUNNING, S_ZOMBIE } stream_state_t;
extern void stream_init( void );
extern void stream_register( pthread_t tid, intgen_t streamix );
+/* NOTE: lock() must be held when calling stream_dead */
extern void stream_dead( pthread_t tid );
extern void stream_free( pthread_t tid );
extern int stream_find_all( stream_state_t states[],
--
1.7.0.4
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 7/8] xfsdump: path lookup cache must be thread specific
2011-11-02 21:10 [PATCH 0/8] xfsdump: enable support for multiple streams Bill Kendall
` (5 preceding siblings ...)
2011-11-02 21:10 ` [PATCH 6/8] xfsdump: process thread exit status Bill Kendall
@ 2011-11-02 21:10 ` Bill Kendall
2011-11-03 8:33 ` Christoph Hellwig
2011-11-02 21:10 ` [PATCH 8/8] xfsdump: enable multiple streams Bill Kendall
2011-11-03 10:19 ` [PATCH 0/8] xfsdump: enable support for " Christoph Hellwig
8 siblings, 1 reply; 19+ messages in thread
From: Bill Kendall @ 2011-11-02 21:10 UTC (permalink / raw)
To: xfs
The xfsrestore path lookup routine (maps an ino + generation number to a
pathname) contains a cache for fast lookups of consecutive files in the
same parent directory. This cache either needs to be protected by a lock
or a cache is needed per thread. I took the latter approach since it
doesn't add lock contention and it increases the cache hit rate (it's
relatively unlikely that each stream is processing files in the same
directory).
Signed-off-by: Bill Kendall <wkendall@sgi.com>
---
restore/tree.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/restore/tree.c b/restore/tree.c
index 7bddddf..9e4e83c 100644
--- a/restore/tree.c
+++ b/restore/tree.c
@@ -3441,7 +3441,7 @@ Node2path( nh_t nh, char *path, char *errmsg )
static intgen_t
Node2path_recurse( nh_t nh, char *buf, intgen_t bufsz, intgen_t level )
{
- static path_cache_t cache = { NH_NULL, 0, "" };
+ static __thread path_cache_t cache = { NH_NULL, 0, "" };
node_t *np;
nh_t parh;
xfs_ino_t ino;
--
1.7.0.4
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH 7/8] xfsdump: path lookup cache must be thread specific
2011-11-02 21:10 ` [PATCH 7/8] xfsdump: path lookup cache must be thread specific Bill Kendall
@ 2011-11-03 8:33 ` Christoph Hellwig
0 siblings, 0 replies; 19+ messages in thread
From: Christoph Hellwig @ 2011-11-03 8:33 UTC (permalink / raw)
To: Bill Kendall; +Cc: xfs
On Wed, Nov 02, 2011 at 04:10:53PM -0500, Bill Kendall wrote:
> The xfsrestore path lookup routine (maps an ino + generation number to a
> pathname) contains a cache for fast lookups of consecutive files in the
> same parent directory. This cache either needs to be protected by a lock
> or a cache is needed per thread. I took the latter approach since it
> doesn't add lock contention and it increases the cache hit rate (it's
> relatively unlikely that each stream is processing files in the same
> directory).
>
> Signed-off-by: Bill Kendall <wkendall@sgi.com>
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH 8/8] xfsdump: enable multiple streams
2011-11-02 21:10 [PATCH 0/8] xfsdump: enable support for multiple streams Bill Kendall
` (6 preceding siblings ...)
2011-11-02 21:10 ` [PATCH 7/8] xfsdump: path lookup cache must be thread specific Bill Kendall
@ 2011-11-02 21:10 ` Bill Kendall
2011-11-03 8:36 ` Christoph Hellwig
2011-11-03 10:19 ` [PATCH 0/8] xfsdump: enable support for " Christoph Hellwig
8 siblings, 1 reply; 19+ messages in thread
From: Bill Kendall @ 2011-11-02 21:10 UTC (permalink / raw)
To: xfs
IRIX contained an environment referred to as "miniroot" where
sproc threads were either not available, or at least not used
in xfsdump. Throughout xfsdump there's a "miniroot" variable
which indicates whether or not thread support is enabled. On
Linux this variable has always been false in order to disable
support for multiple streams.
Now that the threading infracstructure has been converted over
to pthreads, this patch removes the "miniroot" variable and
enables the option of using multiple streams.
Note that another feature in xfsdump, using a ring buffer for
I/O to tapes, also depends on thread support. I'm leaving that
disabled for now until more testing has been done.
Signed-off-by: Bill Kendall <wkendall@sgi.com>
---
common/drive.c | 20 +------
common/drive.h | 8 +--
common/drive_minrmt.c | 15 +++---
common/drive_scsitape.c | 15 +++---
common/drive_simple.c | 8 ++--
common/main.c | 124 +++++-----------------------------------------
dump/content.c | 35 ++++++-------
dump/getopt.h | 4 +-
man/man8/xfsdump.8 | 8 +--
man/man8/xfsrestore.8 | 4 +-
restore/content.c | 23 ++++-----
restore/getopt.h | 4 +-
12 files changed, 74 insertions(+), 194 deletions(-)
diff --git a/common/drive.c b/common/drive.c
index 227fecd..32a7191 100644
--- a/common/drive.c
+++ b/common/drive.c
@@ -81,7 +81,7 @@ static drive_strategy_t *strategypp[] = {
* specified on the command line.
*/
bool_t
-drive_init1( int argc, char *argv[ ], bool_t singlethreaded )
+drive_init1( int argc, char *argv[ ] )
{
intgen_t c;
ix_t driveix;
@@ -103,18 +103,6 @@ drive_init1( int argc, char *argv[ ], bool_t singlethreaded )
}
}
- /* validate drive count
- */
- if ( singlethreaded && drivecnt > 1 ) {
- mlog( MLOG_NORMAL, _(
- "too many -%c arguments: "
- "maximum is %d when running in miniroot\n"),
- GETOPT_DUMPDEST,
- 1 );
- usage( );
- return BOOL_FALSE;
- }
-
/* allocate an array to hold ptrs to drive descriptors
*/
if (drivecnt > 0) {
@@ -221,8 +209,7 @@ drive_init1( int argc, char *argv[ ], bool_t singlethreaded )
intgen_t score;
score = ( * sp->ds_match )( argc,
argv,
- drivep,
- singlethreaded );
+ drivep );
if ( ! bestsp || score > bestscore ) {
bestsp = sp;
bestscore = score;
@@ -237,8 +224,7 @@ drive_init1( int argc, char *argv[ ], bool_t singlethreaded )
bestsp->ds_description );
ok = ( * bestsp->ds_instantiate )( argc,
argv,
- drivep,
- singlethreaded );
+ drivep );
if ( ! ok ) {
return BOOL_FALSE;
}
diff --git a/common/drive.h b/common/drive.h
index 4dc4270..f693976 100644
--- a/common/drive.h
+++ b/common/drive.h
@@ -140,15 +140,13 @@ struct drive_strategy {
*/
intgen_t ( * ds_match )( intgen_t argc,
char *argv[ ],
- struct drive *drivep,
- bool_t singlethreaded );
+ struct drive *drivep );
/* returns degree of match. drivep has been pre-allocated
* and initialized with generic info.
*/
bool_t ( * ds_instantiate )( intgen_t argc,
char *argv[ ],
- struct drive *drivep,
- bool_t singlethreaded );
+ struct drive *drivep );
/* creates a drive manager instance, by filling in the
* blanks of the pre-allocated drive descriptor
* returns FALSE on failure.
@@ -648,7 +646,7 @@ extern size_t partialmax;
*
* Returns FALSE if utility should be aborted.
*/
-extern bool_t drive_init1( int argc, char *argv[], bool_t singlethreaded );
+extern bool_t drive_init1( int argc, char *argv[] );
/* drive_init2 - allocate and initialize read and write hdr buffers,
diff --git a/common/drive_minrmt.c b/common/drive_minrmt.c
index 3ff4d0f..5f5f96c 100644
--- a/common/drive_minrmt.c
+++ b/common/drive_minrmt.c
@@ -257,8 +257,8 @@ extern int rmtwrite( int, const void *, uint);
/* strategy functions
*/
-static intgen_t ds_match( int, char *[], drive_t *, bool_t );
-static intgen_t ds_instantiate( int, char *[], drive_t *, bool_t );
+static intgen_t ds_match( int, char *[], drive_t * );
+static intgen_t ds_instantiate( int, char *[], drive_t * );
/* manager operations
*/
@@ -404,7 +404,7 @@ static u_int32_t cmdlineblksize = 0;
*/
/* ARGSUSED */
static intgen_t
-ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded )
+ds_match( int argc, char *argv[], drive_t *drivep )
{
intgen_t fd;
intgen_t c;
@@ -464,7 +464,7 @@ ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded )
*/
/*ARGSUSED*/
static bool_t
-ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded )
+ds_instantiate( int argc, char *argv[], drive_t *drivep )
{
drive_context_t *contextp;
intgen_t c;
@@ -487,9 +487,10 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded )
ASSERT( contextp );
memset( ( void * )contextp, 0, sizeof( *contextp ));
- /* transfer indication of singlethreadedness to context
+ /* do not enable a separate I/O thread,
+ * more testing to be done first...
*/
- contextp->dc_singlethreadedpr = singlethreaded;
+ contextp->dc_singlethreadedpr = BOOL_TRUE;
/* scan the command line for the I/O buffer ring length
* and record checksum request
@@ -580,7 +581,7 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded )
/* if threads not allowed, allocate a record buffer. otherwise
* create a ring, from which buffers will be taken.
*/
- if ( singlethreaded ) {
+ if ( contextp->dc_singlethreadedpr ) {
contextp->dc_bufp = ( char * )memalign( PGSZ, STAPE_MAX_RECSZ );
ASSERT( contextp->dc_bufp );
} else {
diff --git a/common/drive_scsitape.c b/common/drive_scsitape.c
index f24d604..71b809f 100644
--- a/common/drive_scsitape.c
+++ b/common/drive_scsitape.c
@@ -299,8 +299,8 @@ extern int rmtwrite( int, const void *, uint);
/* strategy functions
*/
-static intgen_t ds_match( int, char *[], drive_t *, bool_t );
-static intgen_t ds_instantiate( int, char *[], drive_t *, bool_t );
+static intgen_t ds_match( int, char *[], drive_t * );
+static intgen_t ds_instantiate( int, char *[], drive_t * );
/* manager operations
*/
@@ -506,7 +506,7 @@ is_scsi_driver(char *pathname)
*/
/* ARGSUSED */
static intgen_t
-ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded )
+ds_match( int argc, char *argv[], drive_t *drivep )
{
struct mtget mt_stat;
intgen_t fd;
@@ -544,7 +544,7 @@ ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded )
*/
/*ARGSUSED*/
static bool_t
-ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded )
+ds_instantiate( int argc, char *argv[], drive_t *drivep )
{
drive_context_t *contextp;
intgen_t c;
@@ -567,9 +567,10 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded )
ASSERT( contextp );
memset( ( void * )contextp, 0, sizeof( *contextp ));
- /* transfer indication of singlethreadedness to context
+ /* do not enable a separate I/O thread,
+ * more testing to be done first...
*/
- contextp->dc_singlethreadedpr = singlethreaded;
+ contextp->dc_singlethreadedpr = BOOL_TRUE;
/* scan the command line for the I/O buffer ring length
* and record checksum request
@@ -670,7 +671,7 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded )
/* if threads not allowed, allocate a record buffer. otherwise
* create a ring, from which buffers will be taken.
*/
- if ( singlethreaded ) {
+ if ( contextp->dc_singlethreadedpr ) {
contextp->dc_bufp = ( char * )memalign( PGSZ, STAPE_MAX_RECSZ );
ASSERT( contextp->dc_bufp );
} else {
diff --git a/common/drive_simple.c b/common/drive_simple.c
index cb88aa8..086d162 100644
--- a/common/drive_simple.c
+++ b/common/drive_simple.c
@@ -98,8 +98,8 @@ extern size_t pgsz;
/* strategy functions
*/
-static intgen_t ds_match( int, char *[], drive_t *, bool_t );
-static intgen_t ds_instantiate( int, char *[], drive_t *, bool_t );
+static intgen_t ds_match( int, char *[], drive_t * );
+static intgen_t ds_instantiate( int, char *[], drive_t * );
/* declare manager operators
*/
@@ -178,7 +178,7 @@ static drive_ops_t drive_ops = {
*/
/* ARGSUSED */
static intgen_t
-ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded )
+ds_match( int argc, char *argv[], drive_t *drivep )
{
bool_t isrmtpr;
struct stat64 statbuf;
@@ -221,7 +221,7 @@ ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded )
*/
/*ARGSUSED*/
static bool_t
-ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded )
+ds_instantiate( int argc, char *argv[], drive_t *drivep )
{
drive_context_t *contextp;
diff --git a/common/main.c b/common/main.c
index 38b3889..5880723 100644
--- a/common/main.c
+++ b/common/main.c
@@ -87,10 +87,6 @@ bool_t preemptchk( int );
static bool_t loadoptfile( int *argcp, char ***argvp );
static char * stripquotes( char *p );
static void shiftleftby1( char *p, char *endp );
-static bool_t in_miniroot_heuristic( void );
-#ifdef HIDDEN
-static void mrh_sighandler( int );
-#endif
static void sighandler( int );
static int childmain( void * );
static bool_t sigint_dialog( void );
@@ -111,11 +107,6 @@ intgen_t version = 3;
intgen_t subversion = 0;
char *progname = 0; /* used in all error output */
char *homedir = 0; /* directory invoked from */
-#ifdef HIDDEN
-bool_t miniroot = BOOL_FALSE;
-#else
-bool_t miniroot = BOOL_TRUE;
-#endif /* HIDDEN */
bool_t pipeline = BOOL_FALSE;
bool_t stdoutpiped = BOOL_FALSE;
pthread_t parenttid;
@@ -215,16 +206,11 @@ main( int argc, char *argv[] )
if ( ! ok ) {
return mlog_exit(EXIT_ERROR, RV_INIT);
}
- /* scan the command line for the miniroot, info, progress
+ /* scan the command line for the info, progress
* report options, and stacksz.
*/
minstacksz = MINSTACKSZ;
maxstacksz = MAXSTACKSZ;
-#ifdef HIDDEN
- miniroot = BOOL_FALSE;
-#else
- miniroot = BOOL_TRUE;
-#endif /* HIDDEN */
infoonly = BOOL_FALSE;
progrpt_enabledpr = BOOL_FALSE;
optind = 1;
@@ -275,9 +261,6 @@ main( int argc, char *argv[] )
}
maxstacksz = tmpstacksz;
break;
- case GETOPT_MINIROOT:
- miniroot = BOOL_TRUE;
- break;
case GETOPT_HELP:
infoonly = BOOL_TRUE;
mlog_exit_hint(RV_USAGE);
@@ -349,13 +332,6 @@ main( int argc, char *argv[] )
return mlog_exit(EXIT_ERROR, RV_INIT);
}
- /* perform an experiment to determine if we are in the miniroot.
- * various features will be disallowed if in miniroot.
- */
- if ( ! miniroot && in_miniroot_heuristic( )) {
- miniroot = BOOL_TRUE;
- }
-
/* initialize message logging (stage 2) - allocate the message lock
*/
ok = mlog_init2( );
@@ -473,7 +449,7 @@ main( int argc, char *argv[] )
* terribly time-consuming here. A second initialization pass
* will be done shortly.
*/
- ok = drive_init1( argc, argv, miniroot );
+ ok = drive_init1( argc, argv );
if ( ! ok ) {
return mlog_exit(EXIT_ERROR, RV_INIT);
}
@@ -503,11 +479,7 @@ main( int argc, char *argv[] )
VERSION,
version,
subversion );
- if ( miniroot ) {
- mlog( MLOG_VERBOSE | MLOG_BARE, _(
- " - "
- "Running single-threaded\n") );
- } else if ( ! pipeline && ! stdoutpiped && sistr && dlog_allowed( )) {
+ if ( ! pipeline && ! stdoutpiped && sistr && dlog_allowed( )) {
mlog( MLOG_VERBOSE | MLOG_BARE, _(
" - "
"type %s for status and control\n"),
@@ -531,11 +503,10 @@ main( int argc, char *argv[] )
*/
mlog_tell_streamcnt( drivecnt );
- /* initialize the state of signal processing. if miniroot or
- * pipeline, just want to exit when a signal is received. otherwise,
- * hold signals so they don't interfere with sys calls; they will
- * be released at pre-emption points and upon pausing in the main
- * loop.
+ /* initialize the state of signal processing. if in a pipeline, just
+ * want to exit when a signal is received. otherwise, hold signals so
+ * they don't interfere with sys calls; they will be released at
+ * pre-emption points and upon pausing in the main loop.
*/
sigfillset(&sa.sa_mask);
@@ -551,7 +522,7 @@ main( int argc, char *argv[] )
sigaction( SIGPIPE, &sa, NULL );
sigaction( SIGCHLD, &sa, NULL );
- if ( ! miniroot && ! pipeline ) {
+ if ( ! pipeline ) {
sigset_t blocked_set;
stop_in_progress = BOOL_FALSE;
@@ -594,10 +565,9 @@ main( int argc, char *argv[] )
return mlog_exit(EXIT_ERROR, RV_INIT);
}
- /* if miniroot or a pipeline, go single-threaded
- * with just one stream.
+ /* if in a pipeline, go single-threaded with just one stream.
*/
- if ( miniroot || pipeline ) {
+ if ( pipeline ) {
intgen_t exitcode;
sa.sa_handler = sighandler;
@@ -977,9 +947,6 @@ usage( void )
ULO(_("(show verbosity in messages)"), GETOPT_SHOWLOGLEVEL );
#endif /* REVEAL */
ULO(_("<I/O buffer ring length>"), GETOPT_RINGLEN );
-#ifdef REVEAL
- ULO(_("(miniroot restrictions)"), GETOPT_MINIROOT );
-#endif /* REVEAL */
ULN(_("- (stdout)") );
ULN(_("<source (mntpnt|device)>") );
#endif /* DUMP */
@@ -1030,9 +997,6 @@ usage( void )
#endif /* REVEAL */
ULO(_("<excluded subtree> ..."), GETOPT_NOSUBTREE );
ULO(_("<I/O buffer ring length>"), GETOPT_RINGLEN );
-#ifdef REVEAL
- ULO(_("(miniroot restrictions)"), GETOPT_MINIROOT );
-#endif /* REVEAL */
ULN(_("- (stdin)") );
ULN(_("<destination>") );
#endif /* RESTORE */
@@ -1081,9 +1045,9 @@ preemptchk( int flg )
return BOOL_FALSE;
}
- /* signals not caught in these cases
+ /* signals not caught if in a pipeline
*/
- if ( miniroot || pipeline ) {
+ if ( pipeline ) {
return BOOL_FALSE;
}
@@ -1400,66 +1364,6 @@ loadoptfile( intgen_t *argcp, char ***argvp )
return BOOL_TRUE;
}
-#ifdef HIDDEN
-static pid_t mrh_cid;
-#endif
-
-static bool_t
-in_miniroot_heuristic( void )
-{
- return BOOL_TRUE;
-
-#ifdef HIDDEN
- SIG_PF prev_handler_hup;
- SIG_PF prev_handler_term;
- SIG_PF prev_handler_int;
- SIG_PF prev_handler_quit;
- SIG_PF prev_handler_cld;
- bool_t in_miniroot;
-
- /* attempt to call sproc.
- */
- prev_handler_hup = sigset( SIGHUP, SIG_IGN );
- prev_handler_term = sigset( SIGTERM, SIG_IGN );
- prev_handler_int = sigset( SIGINT, SIG_IGN );
- prev_handler_quit = sigset( SIGQUIT, SIG_IGN );
- prev_handler_cld = sigset( SIGCLD, mrh_sighandler );
- ( void )sighold( SIGCLD );
- mrh_cid = ( pid_t )sproc( ( void ( * )( void * ))exit, PR_SALL, 0 );
- if ( mrh_cid < 0 ) {
- in_miniroot = BOOL_TRUE;
- } else {
- while ( mrh_cid >= 0 ) {
- ( void )sigpause( SIGCLD );
- }
- in_miniroot = BOOL_FALSE;
- }
- ( void )sigset( SIGHUP, prev_handler_hup );
- ( void )sigset( SIGTERM, prev_handler_term );
- ( void )sigset( SIGINT, prev_handler_int );
- ( void )sigset( SIGQUIT, prev_handler_quit );
- ( void )sigset( SIGCLD, prev_handler_cld );
-
- return in_miniroot;
-#endif /* HIDDEN */
-}
-
-#ifdef HIDDEN
-static void
-mrh_sighandler( int signo )
-{
- if ( signo == SIGCLD ) {
- pid_t cid;
- intgen_t stat;
-
- cid = wait( &stat );
- if ( cid == mrh_cid ) {
- mrh_cid = -1;
- }
- }
-}
-#endif
-
/* parent and children share this handler.
*/
static void
@@ -1470,9 +1374,9 @@ sighandler( int signo )
if ( dlog_sighandler( signo ) )
return;
- /* if in miniroot, don't do anything risky. just quit.
+ /* if in pipeline, don't do anything risky. just quit.
*/
- if ( miniroot || pipeline ) {
+ if ( pipeline ) {
intgen_t rval;
mlog( MLOG_TRACE | MLOG_NOTE | MLOG_NOLOCK | MLOG_PROC,
diff --git a/dump/content.c b/dump/content.c
index 33f1166..3a7f508 100644
--- a/dump/content.c
+++ b/dump/content.c
@@ -244,7 +244,6 @@ typedef struct pds pds_t;
extern void usage( void );
extern bool_t preemptchk( int );
extern char *homedir;
-extern bool_t miniroot;
extern bool_t pipeline;
extern bool_t stdoutpiped;
extern char *sistr;
@@ -2352,7 +2351,7 @@ content_stream_dump( ix_t strmix )
inomap_next_nondir,
inomap_contextp,
( intgen_t * )&rv,
- ( miniroot || pipeline ) ?
+ pipeline ?
(bool_t (*)(int))preemptchk : 0,
bstatbufp,
bstatbuflen );
@@ -2586,24 +2585,22 @@ decision_more:
* from all streams have been registered.
*/
if ( drivep->d_capabilities & DRIVE_CAP_FILES ) {
- if ( ! miniroot ) {
- if ( stream_cnt( ) > 1 ) {
- mlog( MLOG_VERBOSE, _(
- "waiting for synchronized "
- "session inventory dump\n") );
- sc_stat_pds[ strmix ].pds_phase = PDS_INVSYNC;
- }
+ if ( stream_cnt( ) > 1 ) {
+ mlog( MLOG_VERBOSE, _(
+ "waiting for synchronized "
+ "session inventory dump\n") );
+ sc_stat_pds[ strmix ].pds_phase = PDS_INVSYNC;
+ }
- /* first be sure all threads have begun
- */
- while ( sc_thrdsarrivedcnt < drivecnt ) {
- sleep( 1 );
- }
- /* now wait for survivors to checkin
- */
- while ( sc_thrdsdonecnt < stream_cnt( )) {
- sleep( 1 );
- }
+ /* first be sure all threads have begun
+ */
+ while ( sc_thrdsarrivedcnt < drivecnt ) {
+ sleep( 1 );
+ }
+ /* now wait for survivors to checkin
+ */
+ while ( sc_thrdsdonecnt < stream_cnt( )) {
+ sleep( 1 );
}
/* proceeed
*/
diff --git a/dump/getopt.h b/dump/getopt.h
index 8a55631..ba26c93 100644
--- a/dump/getopt.h
+++ b/dump/getopt.h
@@ -27,7 +27,7 @@
* facilitating easy changes.
*/
-#define GETOPT_CMDSTRING "ab:c:d:ef:hl:mop:qs:t:v:z:AB:CDEFG:H:I:JL:M:NO:PRSTUVWY:Z"
+#define GETOPT_CMDSTRING "ab:c:d:ef:hl:mop:qs:t:v:z:AB:CDEFG:H:I:JL:M:NO:PRSTUVWY:"
#define GETOPT_DUMPASOFFLINE 'a' /* dump DMF dualstate files as offline */
#define GETOPT_BLOCKSIZE 'b' /* blocksize for rmt */
@@ -80,6 +80,6 @@
#define GETOPT_SHOWLOGLEVEL 'W' /* show level of log messages */
/* 'X' */
#define GETOPT_RINGLEN 'Y' /* specify I/O buffer ring length */
-#define GETOPT_MINIROOT 'Z' /* apply miniroot restrictions */
+/* 'Z' used on irix for miniroot */
#endif /* GETOPT_H */
diff --git a/man/man8/xfsdump.8 b/man/man8/xfsdump.8
index 46e6bd5..5360378 100644
--- a/man/man8/xfsdump.8
+++ b/man/man8/xfsdump.8
@@ -66,10 +66,6 @@ option. Other techniques, such as making a second copy of the dump
image, provide more protection against media failures than multiple
media files will.
.P
-However, the current implementation in Linux only supports one destination
-and running single threaded. Therefore, the above comments
-regarding multiple streams describe the possible future capabilities.
-.P
.I xfsdump
maintains an online dump inventory in \f2/var/lib/xfsdump/inventory\f1.
The
@@ -391,8 +387,8 @@ Specify I/O buffer ring length.
uses a ring of output buffers to achieve maximum throughput
when dumping to tape drives.
The default ring length is 3.
-However, this is only supported when running multi-threaded
-which has not been done for Linux yet - making this option benign.
+However, this is not currently enabled on
+Linux yet, making this option benign.
.TP 5
.B \-
A lone
diff --git a/man/man8/xfsrestore.8 b/man/man8/xfsrestore.8
index 8e33ee9..aad97fa 100644
--- a/man/man8/xfsrestore.8
+++ b/man/man8/xfsrestore.8
@@ -426,8 +426,8 @@ Specify I/O buffer ring length.
uses a ring of input buffers to achieve maximum throughput
when restoring from tape drives.
The default ring length is 3.
-However, this is only supported when running multi-threaded
-which has not been done for Linux yet - making this option benign.
+However, this is not currently enabled on
+Linux yet, making this option benign.
.TP 5
.B \-
A lone
diff --git a/restore/content.c b/restore/content.c
index 0108a40..a9e0b20 100644
--- a/restore/content.c
+++ b/restore/content.c
@@ -660,7 +660,6 @@ typedef struct tran tran_t;
extern void usage( void );
extern bool_t preemptchk( void );
extern char *homedir;
-extern bool_t miniroot;
extern bool_t pipeline;
extern bool_t stdoutpiped;
extern char *sistr;
@@ -2544,14 +2543,12 @@ content_stream_restore( ix_t thrdix )
}
tranp->t_sync5 = SYNC_BUSY;
unlock( );
- if ( ! miniroot ) {
- if ( drivecnt > 1 ) {
- mlog( MLOG_TRACE,
- "waiting for other streams to exit\n" );
- }
- while ( cldmgr_otherstreamsremain( thrdix )) {
- sleep( 1 );
- }
+ if ( drivecnt > 1 ) {
+ mlog( MLOG_TRACE,
+ "waiting for other streams to exit\n" );
+ }
+ while ( cldmgr_otherstreamsremain( thrdix )) {
+ sleep( 1 );
}
mlog( MLOG_DEBUG,
@@ -3006,10 +3003,10 @@ applydirdump( drive_t *drivep,
return RV_INTR;
}
- /* if miniroot or pipeline , call preemptchk( ) to
+ /* if in a pipeline , call preemptchk( ) to
* print status reports
*/
- if ( miniroot || pipeline )
+ if ( pipeline )
{
mlog( MLOG_DEBUG ,
"preemptchk( )\n");
@@ -3543,10 +3540,10 @@ applynondirdump( drive_t *drivep,
fhdrp->fh_offset );
}
- /* if miniroot or pipeline , call preemptchk( ) to
+ /* if in a pipeline , call preemptchk( ) to
* print status reports
*/
- if ( miniroot || pipeline )
+ if ( pipeline )
{
mlog( MLOG_DEBUG ,
"preemptchk( )\n");
diff --git a/restore/getopt.h b/restore/getopt.h
index 4ee865d..63568de 100644
--- a/restore/getopt.h
+++ b/restore/getopt.h
@@ -26,7 +26,7 @@
* purpose is to contain that command string.
*/
-#define GETOPT_CMDSTRING "a:b:c:def:himn:op:qrs:tv:wABCDEFG:H:I:JL:M:NO:PQRS:TUVWX:Y:Z"
+#define GETOPT_CMDSTRING "a:b:c:def:himn:op:qrs:tv:wABCDEFG:H:I:JL:M:NO:PQRS:TUVWX:Y:"
#define GETOPT_WORKSPACE 'a' /* workspace dir (content.c) */
#define GETOPT_BLOCKSIZE 'b' /* blocksize for rmt */
@@ -79,6 +79,6 @@
#define GETOPT_SHOWLOGLEVEL 'W' /* show level of log messages */
#define GETOPT_NOSUBTREE 'X' /* subtree to exclude */
#define GETOPT_RINGLEN 'Y' /* specify I/O buffer ring length */
-#define GETOPT_MINIROOT 'Z' /* apply miniroot restrictions */
+/* 'Z' used on irix for miniroot */
#endif /* GETOPT_H */
--
1.7.0.4
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH 0/8] xfsdump: enable support for multiple streams
2011-11-02 21:10 [PATCH 0/8] xfsdump: enable support for multiple streams Bill Kendall
` (7 preceding siblings ...)
2011-11-02 21:10 ` [PATCH 8/8] xfsdump: enable multiple streams Bill Kendall
@ 2011-11-03 10:19 ` Christoph Hellwig
2011-11-03 12:11 ` Bill Kendall
8 siblings, 1 reply; 19+ messages in thread
From: Christoph Hellwig @ 2011-11-03 10:19 UTC (permalink / raw)
To: Bill Kendall; +Cc: xfs
On Wed, Nov 02, 2011 at 04:10:46PM -0500, Bill Kendall wrote:
> This series resurrects the IRIX multi-stream support for splitting a
> backup among several output files/tapes. This offers some nice
> performance improvements, particularly in xfsrestore where a single
> stream often cannot keep the filesystem/disks busy. I've observed
> a 1.7x improvement on a backup and a 5x improvement on restore.
>
> I have a couple of xfstests for this, and will submit those once
> a few outstanding xfsdump test patches have been reviewed.
I've put both your two previous xfsdump patches and the all your
xfstests changes into the kernel.org repositories. They all look
fine to me.
Btw, one thing I'd love to see is a script that uses the userspace
iscsi target (tgt) infrastructure to emulate a tape, and thus allow to
run the tape tests in any VM without requiring a real tape device.
Maybe there is someone on the list feels challenged by this?
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PATCH 0/8] xfsdump: enable support for multiple streams
2011-11-03 10:19 ` [PATCH 0/8] xfsdump: enable support for " Christoph Hellwig
@ 2011-11-03 12:11 ` Bill Kendall
0 siblings, 0 replies; 19+ messages in thread
From: Bill Kendall @ 2011-11-03 12:11 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: xfs
On 11/03/2011 05:19 AM, Christoph Hellwig wrote:
> On Wed, Nov 02, 2011 at 04:10:46PM -0500, Bill Kendall wrote:
>> This series resurrects the IRIX multi-stream support for splitting a
>> backup among several output files/tapes. This offers some nice
>> performance improvements, particularly in xfsrestore where a single
>> stream often cannot keep the filesystem/disks busy. I've observed
>> a 1.7x improvement on a backup and a 5x improvement on restore.
>>
>> I have a couple of xfstests for this, and will submit those once
>> a few outstanding xfsdump test patches have been reviewed.
>
> I've put both your two previous xfsdump patches and the all your
> xfstests changes into the kernel.org repositories. They all look
> fine to me.
Thanks for the reviews. I'm going to be out of town for a few days
but will look at your suggestion to use POSIX semaphores when I
return.
>
> Btw, one thing I'd love to see is a script that uses the userspace
> iscsi target (tgt) infrastructure to emulate a tape, and thus allow to
> run the tape tests in any VM without requiring a real tape device.
>
> Maybe there is someone on the list feels challenged by this?
I've been using an open source vtl to achieve the same goal, and have
given some thought to integrating (or at a minimum documenting) how
to use it with the xfsdump tests.
https://github.com/markh794/mhvtl
Bill
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 19+ messages in thread