mpath_ctl observation + some novice questions

mpath_ctl observation + some novice questions

[Ray Stell]

I'm not a developer, but I thought I'd throw out this observation.

environment:
 - rhas4 U2 Beta - Linux pecan 2.6.9-16.EL #1 Mon Aug 15 19:58:49 EDT 2005 i686 i686 i386 GNU/Linux
 - multipath-tools-0.4.5.49.  
 - 2 QLogic PCI to Fibre Channel Host Adapters for QLA2340:
   Firmware version 3.03.15 IPX, Driver version 8.01.00b5-rh2
 - IBM ESS 2105F20 4 paths

When I used this command:  multipath -v3  
The cpu got eaten up by mpath_ctl and remains that way until reboot:

top - 21:14:55 up  3:51,  2 users,  load average: 4.18, 2.54, 1.06
Tasks:  68 total,   3 running,  65 sleeping,   0 stopped,   0 zombie
Cpu(s): 15.9% us, 84.1% sy,  0.0% ni,  0.0% id,  0.0% wa,  0.0% hi,  0.0% si
Mem:   1554428k total,   244244k used,  1310184k free,    34268k buffers
Swap:  5116620k total,        0k used,  5116620k free,   119464k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND 
 4202 root      15 -10  2324   92   72 R 99.7  0.0   4:34.19 mpath_ctl 
 4255 root      16   0  3344  928  744 R  0.3  0.1   0:00.05 top 
...

I was able to repeat this twice, but not a the third time.  I may
have something misconfigured since this is my first foray into
dm.  I'll be glad to supply any info of interest.  Thanks.

Next topic:  The number of scsi devices I'm seeing in my 
sand box is wrong.  I have 4 luns with 4 paths.  I'm only
seeing 14 devices, after one of the reboots, I only had 13.
Any clue why?

[root@pecan ~]# multipath -v2 -d
create: 1IBM_____2105____________50A22206
[size=19 GB][features="0"][hwhandler="0"]
\_ round-robin 0
  \_ 0:0:0:0 sda  8:0
  \_ 0:0:1:0 sde  8:64
  \_ 1:0:0:0 sdi  8:128
  \_ 1:0:1:0 sdm  8:192

create: 1IBM_____2105____________30A22206
[size=19 GB][features="0"][hwhandler="0"]
\_ round-robin 0
  \_ 0:0:0:1 sdb  8:16
  \_ 1:0:0:1 sdj  8:144
  \_ 1:0:1:1 sdn  8:208

create: 1IBM_____2105____________72622206
[size=19 GB][features="0"][hwhandler="0"]
\_ round-robin 0
  \_ 0:0:0:2 sdc  8:32
  \_ 0:0:1:2 sdg  8:96
  \_ 1:0:0:2 sdk  8:160
  \_ 1:0:1:2 sdo  8:224

create: 1IBM_____2105____________72722206
[size=19 GB][features="0"][hwhandler="0"]
\_ round-robin 0
  \_ 0:0:0:3 sdd  8:48
  \_ 1:0:0:3 sdl  8:176
  \_ 1:0:1:3 sdp  8:240

Next topic.  I'm a real dm novice user.  I have not
figured out how to create the disk devices and lvm fs.  Is there 
a doc on how to do this?  My guess is that I need 
to map the wwid in multipath.conf.  Is this demonstrated
somewhere?  Thanks.
============================================================
Ray Stell  stellr@vt.edu  (540) 231-4109  Tempus fugit  28^D

Re: mpath_ctl observation + some novice questions

[Alasdair G Kergon]

On Thu, Sep 15, 2005 at 09:25:38AM -0400, Ray Stell wrote:
> I'm not a developer, but I thought I'd throw out this observation.
> 
> environment:
>  - rhas4 U2 Beta - Linux pecan 2.6.9-16.EL #1 Mon Aug 15 19:58:49 EDT 2005 i686 i686 i386 GNU/Linux
>  - multipath-tools-0.4.5.49.  

RPM (which?) or built from source tarball?

> The cpu got eaten up by mpath_ctl and remains that way until reboot:

The latest RPM is device-mapper-multipath-0.4.5-6.0.RHEL4
which contains a patch (attached) that should fix this.

(uxsock.c where this code came from may also have similar issues)

Alasdair
-- 
agk@redhat.com


--- multipath-tools-0.4.5.49/multipath/mpath_faker.c	2005-08-01 22:07:57.000000000 +0100
+++ multipath-tools-0.4.5.49.1/multipath/mpath_faker.c	2005-09-02 22:55:55.000000000 +0100
@@ -8,17 +8,22 @@
 #include "uxsock.h"
 
 /*
- * keep writing until its all sent
+ * keep writing until it's all sent
  */
-int write_all(int fd, const void *buf, size_t len)
+size_t write_all(int fd, const void *buf, size_t len)
 {
 	size_t total = 0;
+
 	while (len) {
-		int n = write(fd, buf, len);
+		ssize_t n = write(fd, buf, len);
 		if (n < 0) {
+			if ((errno == EINTR) || (errno == EAGAIN))
+				continue;
 			perror("failed to send message to multipathd");
 			return total;
 		}
+		if (!n)
+			return total;
 		buf = n + (char *)buf;
 		len -= n;
 		total += n;
@@ -30,19 +35,25 @@ int write_all(int fd, const void *buf, s
 /*
  * keep reading until its all read
  */
-int read_all(int fd, void *buf, size_t len)
+size_t read_all(int fd, void *buf, size_t len)
 {
 	size_t total = 0;
+
 	while (len) {
-		int n = read(fd, buf, len);
+		ssize_t n = read(fd, buf, len);
 		if (n < 0) {
+			if ((errno == EINTR) || (errno == EAGAIN))
+				continue;
 			perror("failed to receive message from multipathd");
 			return total;
 		}
+		if (!n)
+			return total;
 		buf = n + (char *)buf;
 		len -= n;
 		total += n;
 	}
+
 	return total;
 }
 
@@ -57,7 +68,7 @@ int main(int argc, char **argv)
 {
 	int sock;
 	struct sockaddr_un sun;
-	int len;
+	size_t len;
 	char reply[6];
 
 	if (argc == 1 || (!strcmp(argv[1], "-h")) ||
@@ -103,6 +114,6 @@ int main(int argc, char **argv)
 		exit(1);
 	}
 	close(sock);
+
 	return 0;
 }
-	

Re: mpath_ctl observation + some novice questions

[Ray Stell]

+Alasdair G Kergon <agk redhat com> wrote:

>RPM (which?) or built from source tarball?

source, from:

ftp://sources.redhat.com/pub/dm/multipath-tools/

> The latest RPM is device-mapper-multipath-0.4.5-6.0.RHEL4
> which contains a patch (attached) that should fix this.

where are these located?
============================================================
Ray Stell  stellr@vt.edu  (540) 231-4109  Tempus fugit  28^D

Re: mpath_ctl observation + some novice questions

[gistolero]

Hi,

> Next topic.  I'm a real dm novice user.  I have not
> figured out how to create the disk devices and lvm fs.  Is there 
> a doc on how to do this?  My guess is that I need 
> to map the wwid in multipath.conf.  Is this demonstrated
> somewhere?  Thanks.

use a test environment, no production server!


startup-configuration:

1.) load dm-mapper and HBA kernel modules


2.) create partitions on each lun (e.g.: lun0 has two paths sda and sdb - 
"fdisk sda" creates the partitions for sda _and_ sdb) - you have to do this 
step _before_ creating multipath devices (kpartx in 
/etc/dev.d/block/multipath.dev reads this partition information)


3.) create /etc/multipath.conf

- check multipath.conf.annotated and multipath.conf.synthetic in the 
multipath-tools tarball
- get lun wwid's with "scsi_id -g -u -s /block/sd?"
- create a "multipath { ... }" section for each lun (define "wwid" and "alias")
- create a "device { ... }" section for your storage (see "TestedEnvironments" 
located at the multipath-tools homepage) - set "path_grouping_policy" and 
"prio_callout" (setting "prio_callout" is important. if you choose a wrong 
value, you will see nothing after starting multipath)


4.) start multipathd (debug with "-d" or "-v4" (syslog))


5.) run "multipath /dev/sd?" for each sd? device

multipath should create device-mapper table entries (see "dmsetup table"), 
block-device-files in /dev/mapper/ and links in /dev/ for the lun and for each 
partition

e.g.: you have created two partitions and your lun alias is "yellow" (defined 
in /etc/multipath.conf):

# dmsetup table
yellow1: ...
yellow: ...
yellow2: ...

# ls /dev/mapper/
yellow yellow1 yellow2 control

# ls -F /dev/yellow*
/dev/yellow@  /dev/yellow1@  /dev/yellow2@

and check "multipath [-v3] -ll"


6.) create filesystems and mount /dev/yellow1 and /dev/yellow2


you can delete the current configuration with "dmsetup remove yellow2; dmsetup 
remove yellow1; dmsetup remove yellow"

hope that helps
simon

Re: mpath_ctl observation + some novice questions

[christophe varoqui]

Applied upstream, thanks.

On jeu, 2005-09-15 at 15:42 +0100, Alasdair G Kergon wrote:
> On Thu, Sep 15, 2005 at 09:25:38AM -0400, Ray Stell wrote:
> > I'm not a developer, but I thought I'd throw out this observation.
> > 
> > environment:
> >  - rhas4 U2 Beta - Linux pecan 2.6.9-16.EL #1 Mon Aug 15 19:58:49 EDT 2005 i686 i686 i386 GNU/Linux
> >  - multipath-tools-0.4.5.49.  
> 
> RPM (which?) or built from source tarball?
> 
> > The cpu got eaten up by mpath_ctl and remains that way until reboot:
> 
> The latest RPM is device-mapper-multipath-0.4.5-6.0.RHEL4
> which contains a patch (attached) that should fix this.
> 
> (uxsock.c where this code came from may also have similar issues)
> 
> Alasdair
-- 
christophe varoqui <christophe.varoqui@free.fr>