multipathd.init

multipathd.init

[Alasdair G Kergon]

[-- Attachment #1: Type: text/plain, Size: 1587 bytes --]

Attached is a cleaned-up version of multipathd init script - at least 
for Red Hat based systems.

While it uses the standard daemon-handling functions, it's still only 
suitable for a few situations though.

If you have / on dm-multipath, the daemon has to be started from
the initrd - so you don't want it stopping/starting from init.d,
but you might want to HUP it.

If you have other system parts of your filesystem (e.g. /usr) you 
want the daemon starting from rc.sysinit.

If you have data filesystems over multipath, you want something
to mount them after multipathd starts.

And you might also have cryptsetup, md, lvm2, kpartx involved
- in various combinations...

So you could have a multi-level startup with multiple instances
of multipathd each configured to manage only a subset of devices.
Then the init.d multipathd script would only kill the daemon
instance started during by the init.d script, and would leave
alone the instances managing / and /usr.

Or you could forget completely about init.d and /var/lock/subsys
and /var/run and just launch the daemon from the initrd or
failing that from rc.sysinit.

Then you would need a new mechanism to manage it.
(ie a dedicated client program that communicates with it e.g. via
shared memory)
I anticipate that something like this is the right way to go.

That client program could even be 'dmsetup' if the daemon functionality
could be incorporated into plug-ins: The various types of mirroring 
have similar requirements, so it's sensible to have a single
daemon infrastructure for them all.

Alasdair
-- 
agk@redhat.com

[-- Attachment #2: multipathd --]
[-- Type: text/plain, Size: 1196 bytes --]

#!/bin/bash

#
#	/etc/rc.d/init.d/multipathd
#
# Starts the multipath daemon
#
# chkconfig: 2345 13 87
# description: Manage device-mapper multipath devices
# processname: multipathd

DAEMON=/sbin/multipathd
prog=`basename $DAEMON`
initdir=/etc/rc.d/init.d
lockdir=/var/lock/subsys
sysconfig=/etc/sysconfig

 
system=redhat
 
if [ $system = redhat ]; then
	# Source function library.
	. $initdir/functions
fi
 
test -x $DAEMON || exit 0
test -r $sysconfig/$prog && . $sysconfig/$prog

RETVAL=0

#
# See how we were called.
#

start() {
	echo -n $"Starting $prog daemon: "
	daemon $DAEMON
	RETVAL=$?
	[ $RETVAL -eq 0 ] && touch $lockdir/$prog
	echo
}

stop() {
	echo -n $"Stopping $prog daemon: "
	killproc $DAEMON
	RETVAL=$?
	[ $RETVAL -eq 0 ] && rm -f $lockdir/$prog
	echo
}

restart() {
	stop
	start
}	

reload() {
	echo -n "Reloading $prog: "
	trap "" SIGHUP
	killproc $DAEMON -HUP
	RETVAL=$?
	echo
}	

case "$1" in
start)
	start
	;;
stop)
	stop
	;;
reload)
	reload
	;;
restart)
	restart
	;;
condrestart)
	if [ -f $lockdir/$prog ]; then
	    restart
	fi
	;;
status)
	status $prog
	RETVAL=$?
	;;
*)
	echo $"Usage: $0 {start|stop|status|restart|condrestart|reload}"
	RETVAL=1
esac

exit $RETVAL

[-- Attachment #3: Type: text/plain, Size: 0 bytes --]

Re: multipathd.init

[christophe varoqui]

On ven, 2005-04-01 at 21:40 +0100, Alasdair G Kergon wrote:
> Attached is a cleaned-up version of multipathd init script - at least 
> for Red Hat based systems.
> 
> While it uses the standard daemon-handling functions, it's still only 
> suitable for a few situations though.
> 
> If you have / on dm-multipath, the daemon has to be started from
> the initrd - so you don't want it stopping/starting from init.d,
> but you might want to HUP it.
> 
Not agreed.

The daemon is only needed for the multipathed FS to survive PG failover
-> PG failback cycles.

Hence, the vulnerability window is between "/ mounted in intrd" and
"multipathd started in rc?.d/".

Do we really want to do complicated things to close that window ? I
would guess not.

I insist : we do survive simple PG failover during that risky window,
with multipathd running. The failing scenario probability is so thin ...

Regards,
-- 
christophe varoqui <christophe.varoqui@free.fr>

Re: multipathd.init

[Igor Feoktistov]

Does it have to be running from initrd in case of root on dm-multipath?
Would not it be sufficient just run multipath from initrd and then run
multipathd from init scripts? This is exactly what I'm doing in the lab
so far...

>From: Alasdair G Kergon <agk@redhat.com>
>To: device-mapper development <dm-devel@redhat.com>
>Mail-Followup-To: device-mapper development <dm-devel@redhat.com>
>Content-Disposition: inline
>User-Agent: Mutt/1.4.1i
>X-loop: dm-devel@redhat.com
>Subject: [dm-devel] multipathd.init
>
>Attached is a cleaned-up version of multipathd init script - at least 
>for Red Hat based systems.
>
>While it uses the standard daemon-handling functions, it's still only 
>suitable for a few situations though.
>
>If you have / on dm-multipath, the daemon has to be started from
>the initrd - so you don't want it stopping/starting from init.d,
>but you might want to HUP it.
>
>If you have other system parts of your filesystem (e.g. /usr) you 
>want the daemon starting from rc.sysinit.
>
>If you have data filesystems over multipath, you want something
>to mount them after multipathd starts.
>
>And you might also have cryptsetup, md, lvm2, kpartx involved
>- in various combinations...
>
>So you could have a multi-level startup with multiple instances
>of multipathd each configured to manage only a subset of devices.
>Then the init.d multipathd script would only kill the daemon
>instance started during by the init.d script, and would leave
>alone the instances managing / and /usr.
>
>Or you could forget completely about init.d and /var/lock/subsys
>and /var/run and just launch the daemon from the initrd or
>failing that from rc.sysinit.
>
>Then you would need a new mechanism to manage it.
>(ie a dedicated client program that communicates with it e.g. via
>shared memory)
>I anticipate that something like this is the right way to go.
>
>That client program could even be 'dmsetup' if the daemon functionality
>could be incorporated into plug-ins: The various types of mirroring 
>have similar requirements, so it's sensible to have a single
>daemon infrastructure for them all.
>
>Alasdair
>-- 
>agk@redhat.com

Re: multipathd.init

[Alasdair G Kergon]

On Fri, Apr 01, 2005 at 11:09:04PM +0200, christophe varoqui wrote:
> Do we really want to do complicated things to close that window ? I
> would guess not.
 
In other words, if the paths can't survive the window between initrd 
running and init scripts completing, the machine doesn't deserve to boot.

And if you want the daemon in single-user mode, you can start it by hand.

Fair point.

So is there ever a requirement to run the daemon while /var is read-only?

Alasdair
-- 
agk@redhat.com

Re: multipathd.init

[christophe varoqui]

On ven, 2005-04-01 at 22:19 +0100, Alasdair G Kergon wrote:
> On Fri, Apr 01, 2005 at 11:09:04PM +0200, christophe varoqui wrote:
> > Do we really want to do complicated things to close that window ? I
> > would guess not.
>  
> In other words, if the paths can't survive the window between initrd 
> running and init scripts completing, the machine doesn't deserve to boot.
> 
> And if you want the daemon in single-user mode, you can start it by hand.
> 
> Fair point.
> 
> So is there ever a requirement to run the daemon while /var is read-only?
>
I personnality guess not, but Debian people already got annoyed by that.
Don't how they handled it in the end.

Regards,
-- 
christophe varoqui <christophe.varoqui@free.fr>

Re: multipathd.init

[christophe varoqui]

On ven, 2005-04-01 at 23:25 +0200, christophe varoqui wrote:
> On ven, 2005-04-01 at 22:19 +0100, Alasdair G Kergon wrote:
> > On Fri, Apr 01, 2005 at 11:09:04PM +0200, christophe varoqui wrote:
> > > Do we really want to do complicated things to close that window ? I
> > > would guess not.
> >  
> > In other words, if the paths can't survive the window between initrd 
> > running and init scripts completing, the machine doesn't deserve to boot.
> > 
> > And if you want the daemon in single-user mode, you can start it by hand.
> > 
> > Fair point.
> > 
> > So is there ever a requirement to run the daemon while /var is read-only?
> >
> I personnality guess not, but Debian people already got annoyed by that.
> Don't how they handled it in the end.
> 
Alasdair,

Would it ease your packaging work if I put this init script as
multipathd/multipathd.init.redhat ?

I could also rename the current multipathd/multipathd.init to
multipathd/multipathd.init.debian and stop installing it in the
'install' make target.

Debian people, and other distro packagers, may express their views too.

I also would like to kick off the debate on libsysfs and libdevmapper
klibc versions packaging, as I removed these libs from the
multipath-tools package and thus broke the klibc building framework.

What needs to be done to have libdevmapper and libsysfs packages include
their respective klibc build version ? What about klibc shared objects
vs static ?

Regards,
-- 
christophe varoqui <christophe.varoqui@free.fr>

Re: multipathd.init

[christophe varoqui]

> 
> What needs to be done to have libdevmapper and libsysfs packages include
> their respective klibc build version ? What about klibc shared objects
> vs static ?
> 

As a kick-off, here is a little clean up patch.
I guess it's harmless anyway.

diff -ur devmapper-1.01.00/lib/libdm-file.c
devmapper-1.01.00.klibcbuild/lib/libdm-file.c
--- devmapper-1.01.00/lib/libdm-file.c  2005-01-06 19:22:44.000000000
+0100
+++ devmapper-1.01.00.klibcbuild/lib/libdm-file.c       2005-04-03
21:26:13.000000000 +0200
@@ -16,7 +16,6 @@
 #include "lib.h"
 #include "libdm-file.h"
 -#include <sys/file.h>
 #include <fcntl.h>
 #include <dirent.h>
 #include <malloc.h>

-- 
christophe varoqui <christophe.varoqui@free.fr>

Re: multipathd.init

[christophe varoqui]

On dim, 2005-04-03 at 21:59 +0200, christophe varoqui wrote:
> > 
> > What needs to be done to have libdevmapper and libsysfs packages include
> > their respective klibc build version ? What about klibc shared objects
> > vs static ?
> > 
> 
> As a kick-off, here is a little clean up patch.
> I guess it's harmless anyway.
> 
> diff -ur devmapper-1.01.00/lib/libdm-file.c
> devmapper-1.01.00.klibcbuild/lib/libdm-file.c
> --- devmapper-1.01.00/lib/libdm-file.c  2005-01-06 19:22:44.000000000
> +0100
> +++ devmapper-1.01.00.klibcbuild/lib/libdm-file.c       2005-04-03
> 21:26:13.000000000 +0200
> @@ -16,7 +16,6 @@
>  #include "lib.h"
>  #include "libdm-file.h"
>  -#include <sys/file.h>
>  #include <fcntl.h>
>  #include <dirent.h>
>  #include <malloc.h>
> 

Nothing to be proud off, but has the merit to compile a klibc static
archive and installs it as /usr/lib/klibc/lib/libdevmapper.a :

--- devmapper-1.01.00/lib/Makefile.in   2005-04-03 20:48:28.000000000
+0200
+++ devmapper-1.01.00.klibcbuild/lib/Makefile.in        2005-04-03
21:13:43.000000000 +0200
@@ -23,6 +23,7 @@

 LIB_STATIC = $(interface)/libdevmapper.a
 LIB_SHARED = $(interface)/libdevmapper.so
+KLIB_STATIC = $(interface)/libdevmapper-klibc.a

 CFLAGS += -DDEVICE_UID=@DEVICE_UID@ -DDEVICE_GID=@DEVICE_GID@ \
          -DDEVICE_MODE=@DEVICE_MODE@
@@ -30,7 +31,8 @@
 include ../make.tmpl

 .PHONY: install_dynamic install_static \
-       install_fs install_ioctl install_ioctl_static
+       install_fs install_ioctl install_ioctl_static \
+       install_ioctl_klibc_static

 INSTALL_TYPE = install_dynamic install_static

@@ -41,7 +43,7 @@
        $(INSTALL) -D $(OWNER) $(GROUP) -m 444 libdevmapper.h \
                $(includedir)/libdevmapper.h 
-install_static: install_@interface@_static +install_static:
install_@interface@_static install_@interface@_klibc_static
        $(INSTALL) -D $(OWNER) $(GROUP) -m 444 libdevmapper.h \
                $(includedir)/libdevmapper.h 
@@ -57,6 +59,10 @@         $(INSTALL) -D $(OWNER) $(GROUP) -m 755
$(STRIP) $< \
                $(libdir)/libdevmapper.a

+install_ioctl_klibc_static: ioctl/libdevmapper-klibc.a
+       $(INSTALL) -D $(OWNER) $(GROUP) -m 755 $(STRIP) $< \
+               $(klibdir)/libdevmapper.a
+
 .PHONY: distclean_lib distclean

 distclean_lib:
diff -ur devmapper-1.01.00/make.tmpl.in
devmapper-1.01.00.klibcbuild/make.tmpl.in---
devmapper-1.01.00/make.tmpl.in      2004-07-03 20:17:32.000000000 +0200
+++ devmapper-1.01.00.klibcbuild/make.tmpl.in   2005-04-03
21:24:38.000000000 +0200
@@ -18,13 +18,13 @@
 @SET_MAKE@

 CC = @CC@
+KLCC = klcc
 RANLIB = @RANLIB@
 SHELL = /bin/sh
 INSTALL = @INSTALL@
 MSGFMT = @MSGFMT@
 LN_S = @LN_S@
 LIBS = @LIBS@
-CFLAGS += @CFLAGS@
 CLDFLAGS += @CLDFLAGS@
 LDDEPS += @LDDEPS@
 LDFLAGS += @LDFLAGS@
@@ -36,6 +36,7 @@
 bindir = $(DESTDIR)@bindir@
 includedir = $(DESTDIR)@includedir@
 libdir = $(DESTDIR)@libdir@
+klibdir = $(DESTDIR)/usr/lib/klibc/lib
 sbindir = $(DESTDIR)@sbindir@
 infodir = $(DESTDIR)@infodir@
 mandir = $(DESTDIR)@mandir@
@@ -54,14 +55,16 @@
 MAKEFLAGS = @JOBS@
 endif

-SUFFIXES = .c .d .o .so .a .po .pot .mo
+SUFFIXES = .c .d .o .so .a .po .pot .mo .ko

-CFLAGS += -fPIC -Wall -Wundef -Wshadow -Wcast-align -Wwrite-strings
-Wmissing-prototypes -Wmissing-declarations -Wnested-externs -Winline
+CFLAGS2 = -fPIC -Wall -Wundef -Wshadow -Wcast-align -Wwrite-strings
-Wmissing-prototypes -Wmissing-declarations -Wnested-externs -Winline

-#CFLAGS += -W -Wconversion -Wpointer-arith -Wredundant-decls
-Wbad-function-cast -Wcast-qual -Wmissing-noreturn
+#CFLAGS2 = -W -Wconversion -Wpointer-arith -Wredundant-decls
-Wbad-function-cast -Wcast-qual -Wmissing-noreturn

 CFLAGS += @COPTIMISE_FLAG@

+CFLAGS2 += @CFLAGS@
+
 ifeq ("@DEBUG@", "yes")
   CFLAGS += -g -fno-omit-frame-pointer -DDEBUG
 endif
@@ -100,6 +103,7 @@
 DEPS = $(top_srcdir)/make.tmpl $(top_srcdir)/VERSION Makefile
$(INC_LNS)

 OBJECTS = $(SOURCES:%.c=%.o)
+KOBJECTS = $(SOURCES:%.c=%.ko)
 POTFILES = $(SOURCES:%.c=%.pot)

 SUBDIRS.install := $(SUBDIRS:=.install)
@@ -111,7 +115,7 @@
 .PHONY: $(SUBDIRS) $(SUBDIRS.install) $(SUBDIRS.clean)
$(SUBDIRS.distclean)
 .PHONY: $(SUBDIRS.pofile)

-TARGETS += $(LIB_SHARED) $(LIB_STATIC)
+TARGETS += $(LIB_SHARED) $(LIB_STATIC) $(KLIB_STATIC)

 all: $(SUBDIRS) $(TARGETS)

@@ -136,17 +140,20 @@
        $(MAKE) -C $(@:.pofile=) pofile
 endif

-$(TARGETS): $(OBJECTS)
+$(TARGETS): $(OBJECTS) $(KOBJECTS)
+
+%.ko: %.c
+       $(KLCC) -c $(INCLUDES) $(CFLAGS) $< -o $@

 %.o: %.c
-       $(CC) -c $(INCLUDES) $(CFLAGS) $< -o $@
+       $(CC) -c $(INCLUDES) $(CFLAGS) $(CFLAGS2) $< -o $@

 %.pot: %.c Makefile
        $(CC) -E $(INCLUDES) -include $(top_srcdir)/include/pogen.h \
-               $(CFLAGS) $< > $@
+               $(CFLAGS) $(CFLAGS2) $< > $@

 %.so: %.o
-       $(CC) -c $(INCLUDES) $(CFLAGS) $< -o $@
+       $(CC) -c $(INCLUDES) $(CFLAGS) $(CFLAGS2) $< -o $@

 $(LIB_SHARED): $(OBJECTS) $(LDDEPS)
        $(CC) $(SOFLAG) -Wl,-soname,$(notdir $@).$(LIB_VERSION) \
@@ -156,11 +163,15 @@
        $(RM) $@
        $(AR) rs $@ $(OBJECTS)

+$(KLIB_STATIC): $(KOBJECTS)
+       $(RM) $@
+       $(AR) rs $@ $(KOBJECTS)
+
 %.d: %.c
        set -e; \
        FILE=`echo $@ | sed 's/\\//\\\\\\//g;s/\\.d//g'`; \
        DEPS=`echo $(DEPS) | sed -e 's/\\//\\\\\\//g'`; \
-       $(CC) -MM $(INCLUDES) $(CFLAGS) $< | \
+       $(CC) -MM $(INCLUDES) $(CFLAGS) $(CFLAGS2) $< | \
        sed "s/\(.*\)\.o[ :]*/$$FILE.o $$FILE.d $$FILE.pot: $$DEPS /g" >
$@; \
        [ -s $@ ] || $(RM) $@


-- 
christophe varoqui <christophe.varoqui@free.fr>

Re: multipathd.init

[Alasdair G Kergon]

On Sun, Apr 03, 2005 at 11:43:37AM +0200, christophe varoqui wrote:
> Would it ease your packaging work if I put this init script as
> multipathd/multipathd.init.redhat ?
 
Yes - that's the line I've been taking with other packages,
including multiple init scripts and letting people choose
the one they need.

Alasdair
-- 
agk@redhat.com

Re: multipathd.init

[Alasdair G Kergon]

On Sun, Apr 03, 2005 at 11:43:37AM +0200, christophe varoqui wrote:
> I also would like to kick off the debate on libsysfs and libdevmapper
> klibc versions packaging, 

For Red Hat distributions, I intend to use glibc static builds for 
the multipath-tools, same as for lvm2 & dmsetup.

I don't want the worry of having to support more than one libc.
(All tests have to be done twice then, for starters, and you lose
use of glibc features the other libraries don't support.)

So much is getting pushed into the initrd these days, that there's 
even the prospect of shared object support being added one day.

Alasdair
-- 
agk@redhat.com

Re: multipathd.init

[christophe varoqui]

On lun, 2005-04-04 at 21:52 +0100, Alasdair G Kergon wrote:
> On Sun, Apr 03, 2005 at 11:43:37AM +0200, christophe varoqui wrote:
> > Would it ease your packaging work if I put this init script as
> > multipathd/multipathd.init.redhat ?
>  
> Yes - that's the line I've been taking with other packages,
> including multiple init scripts and letting people choose
> the one they need.
> 
ok, merged
-- 
christophe varoqui <christophe.varoqui@free.fr>

Re: multipathd.init

[christophe varoqui]

On lun, 2005-04-04 at 22:15 +0100, Alasdair G Kergon wrote:
> On Sun, Apr 03, 2005 at 11:43:37AM +0200, christophe varoqui wrote:
> > I also would like to kick off the debate on libsysfs and libdevmapper
> > klibc versions packaging, 
> 
> For Red Hat distributions, I intend to use glibc static builds for 
> the multipath-tools, same as for lvm2 & dmsetup.
> 
> I don't want the worry of having to support more than one libc.
> (All tests have to be done twice then, for starters, and you lose
> use of glibc features the other libraries don't support.)
> 
> So much is getting pushed into the initrd these days, that there's 
> even the prospect of shared object support being added one day.
> 
So be it, for now.

The place is cleaned from libsysfs/ and libdevmapper/ starting from
0.4.4-pre6.

I modified the Makefiles to still compile properly with BUILD=klibc on
systems with /usr/lib/klibc/ properly stuffed with klibc-compiled
versions of these libs. A little more work for people interested in this
setup, but still fairly manageable.

Hope it eases the packaging for everyone.
Please tell if some more work needs to be merged in this area.

Regards,
-- 
christophe varoqui <christophe.varoqui@free.fr>

Re: multipathd.init

[christophe varoqui]

I pushed 0.4.4-pre8 online.
It contains all fixes discussed here today and the day before.

I hope I'll have positive feedback to release 0.4.4 soon.

Remember the blacklist format changed to regex, as demonstrated in the
example config files in the tarball.

Regards,
cvaroqui

On mar, 2005-04-05 at 00:14 +0200, christophe varoqui wrote:
> On lun, 2005-04-04 at 22:15 +0100, Alasdair G Kergon wrote:
> > On Sun, Apr 03, 2005 at 11:43:37AM +0200, christophe varoqui wrote:
> > > I also would like to kick off the debate on libsysfs and libdevmapper
> > > klibc versions packaging, 
> > 
> > For Red Hat distributions, I intend to use glibc static builds for 
> > the multipath-tools, same as for lvm2 & dmsetup.
> > 
> > I don't want the worry of having to support more than one libc.
> > (All tests have to be done twice then, for starters, and you lose
> > use of glibc features the other libraries don't support.)
> > 
> > So much is getting pushed into the initrd these days, that there's 
> > even the prospect of shared object support being added one day.
> > 
> So be it, for now.
> 
> The place is cleaned from libsysfs/ and libdevmapper/ starting from
> 0.4.4-pre6.
> 
> I modified the Makefiles to still compile properly with BUILD=klibc on
> systems with /usr/lib/klibc/ properly stuffed with klibc-compiled
> versions of these libs. A little more work for people interested in this
> setup, but still fairly manageable.
> 
> Hope it eases the packaging for everyone.
> Please tell if some more work needs to be merged in this area.
> 
> Regards,
-- 
christophe varoqui <christophe.varoqui@free.fr>

Re: multipathd.init

[Lars Marowsky-Bree]

On 2005-04-01T22:19:02, Alasdair G Kergon <agk@redhat.com> wrote:

> On Fri, Apr 01, 2005 at 11:09:04PM +0200, christophe varoqui wrote:
> > Do we really want to do complicated things to close that window ? I
> > would guess not.
>  
> In other words, if the paths can't survive the window between initrd 
> running and init scripts completing, the machine doesn't deserve to boot.
> 
> And if you want the daemon in single-user mode, you can start it by hand.
> 
> Fair point.
> 
> So is there ever a requirement to run the daemon while /var is read-only?

We ran into the same problem, and just decided to invoke /sbin/multipath
once during the boot/ sequence, and then start the daemon in the
runlevel itself.

Note that LSB apparently also suggests that no unneeded daemons are
running in single-user mode; but I couldn't find that detailed anywhere
else.


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
High Availability & Clustering
SUSE Labs, Research and Development
SUSE LINUX Products GmbH - A Novell Business

Re: multipathd.init

[Lars Marowsky-Bree]

On 2005-04-04T22:15:20, Alasdair G Kergon <agk@redhat.com> wrote:

> For Red Hat distributions, I intend to use glibc static builds for 
> the multipath-tools, same as for lvm2 & dmsetup.
> 
> I don't want the worry of having to support more than one libc.
> (All tests have to be done twice then, for starters, and you lose
> use of glibc features the other libraries don't support.)
> 
> So much is getting pushed into the initrd these days, that there's 
> even the prospect of shared object support being added one day.

Again, same here. Though I'm not even sure our initrd doesn't have the
glibc.so already ;-)


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
High Availability & Clustering
SUSE Labs, Research and Development
SUSE LINUX Products GmbH - A Novell Business