Setting up a multipath device

Setting up a multipath device

[Erez Zilber]

Hi,

I'm new to device mapper and trying to set a multipath target:

I'm connected to the same LUN over iSCSI in 2 paths:

Here's the output of sg_map:
sg_map -i -x
/dev/sg0 3 0 0 0 0 /dev/sda DotHill SANnet II FC 411I
/dev/sg1 4 0 0 0 0 /dev/sdb DotHill SANnet II FC 411I

I ran the following command:
multipath 2 round-robin 1 0 /dev/sda round-robin 1 0 /dev/sdb

If I understand correctly, the 1st priority group contains /dev/sda. The 
2nd priority group contains /dev/sdb. If /dev/sda fails, traffic should 
automatically move to /dev/sdb. Is that right?

Here's the out put of multipath -l:

3600c0ff00000000007c1d121c397f503
[size=10 GB][features=0][hwhandler=0]
\_ round-robin 0 [prio=0][active]
\_ 3:0:0:0 sda 8:0 [active][undef]
\_ round-robin 0 [prio=0][enabled]
\_ 4:0:0:0 sdb 8:16 [active][undef]

What does 8:0 & 8:16 mean? What is 3600c0ff00000000007c1d121c397f503?

Now, assuming that what I did so far is ok, how do I use the multipath 
device? Do I use /dev/sda (or /dev/sdb)?

Thanks in advance
-- 

____________________________________________________________

Erez Zilber | 972-9-971-7689

Software Engineer, Storage Team

Voltaire – _The Grid Backbone_

__

www.voltaire.com <http://www.voltaire.com/>

Re: Setting up a multipath device

[Christophe Varoqui]

On Thu, May 04, 2006 at 05:08:41PM +0300, Erez Zilber wrote:
> Hi,
> 
> I'm new to device mapper and trying to set a multipath target:
> 
> I'm connected to the same LUN over iSCSI in 2 paths:
> 
> Here's the output of sg_map:
> sg_map -i -x
> /dev/sg0 3 0 0 0 0 /dev/sda DotHill SANnet II FC 411I
> /dev/sg1 4 0 0 0 0 /dev/sdb DotHill SANnet II FC 411I
> 
> I ran the following command:
> multipath 2 round-robin 1 0 /dev/sda round-robin 1 0 /dev/sdb
> 
Just start multipathd
or run multipath without params

> If I understand correctly, the 1st priority group contains /dev/sda. The 
> 2nd priority group contains /dev/sdb. If /dev/sda fails, traffic should 
> automatically move to /dev/sdb. Is that right?
> 
> Here's the out put of multipath -l:
> 
> 3600c0ff00000000007c1d121c397f503
> [size=10 GB][features=0][hwhandler=0]
> \_ round-robin 0 [prio=0][active]
> \_ 3:0:0:0 sda 8:0 [active][undef]
> \_ round-robin 0 [prio=0][enabled]
> \_ 4:0:0:0 sdb 8:16 [active][undef]
> 
> What does 8:0 & 8:16 mean? What is 3600c0ff00000000007c1d121c397f503?
>
8:0 & 8:16 are paths major:minor numbers
3600c0ff00000000007c1d121c397f503 is the multipath name.
consider activating the fancy naming option in the config file.
 
> Now, assuming that what I did so far is ok, how do I use the multipath 
> device? Do I use /dev/sda (or /dev/sdb)?
>
You should use /dev/mapper/3600c0ff00000000007c1d121c397f503
 
Regards,
cvaroqui

Re: Setting up a multipath device

[Erez Zilber]

Christophe Varoqui wrote:

>On Thu, May 04, 2006 at 05:08:41PM +0300, Erez Zilber wrote:
>  
>
>>Hi,
>>
>>I'm new to device mapper and trying to set a multipath target:
>>
>>I'm connected to the same LUN over iSCSI in 2 paths:
>>
>>Here's the output of sg_map:
>>sg_map -i -x
>>/dev/sg0 3 0 0 0 0 /dev/sda DotHill SANnet II FC 411I
>>/dev/sg1 4 0 0 0 0 /dev/sdb DotHill SANnet II FC 411I
>>
>>I ran the following command:
>>multipath 2 round-robin 1 0 /dev/sda round-robin 1 0 /dev/sdb
>>
>>    
>>
>Just start multipathd
>or run multipath without params
>
>  
>
>>If I understand correctly, the 1st priority group contains /dev/sda. The 
>>2nd priority group contains /dev/sdb. If /dev/sda fails, traffic should 
>>automatically move to /dev/sdb. Is that right?
>>
>>Here's the out put of multipath -l:
>>
>>3600c0ff00000000007c1d121c397f503
>>[size=10 GB][features=0][hwhandler=0]
>>\_ round-robin 0 [prio=0][active]
>>\_ 3:0:0:0 sda 8:0 [active][undef]
>>\_ round-robin 0 [prio=0][enabled]
>>\_ 4:0:0:0 sdb 8:16 [active][undef]
>>
>>What does 8:0 & 8:16 mean? What is 3600c0ff00000000007c1d121c397f503?
>>
>>    
>>
>8:0 & 8:16 are paths major:minor numbers
>3600c0ff00000000007c1d121c397f503 is the multipath name.
>consider activating the fancy naming option in the config file.
> 
>  
>
>>Now, assuming that what I did so far is ok, how do I use the multipath 
>>device? Do I use /dev/sda (or /dev/sdb)?
>>
>>    
>>
>You should use /dev/mapper/3600c0ff00000000007c1d121c397f503
> 
>Regards,
>cvaroqui
>
>--
>dm-devel mailing list
>dm-devel@redhat.com
>https://www.redhat.com/mailman/listinfo/dm-devel
>  
>
Thanks. It seems to work now. Anyway, I have some more questions:

   1. Was it wrong to run 'multipath 2 round-robin 1 0 /dev/sda
      round-robin 1 0 /dev/sdb'?
   2. If I don't supply any parameters, how does the dm know which
      devices are the same physical device (that /dev/sda & /dev/sdb are
      the same physical device)?
   3. How are the priority groups defined if no parameters are supplied?
   4. What is the timeout for a device that has failed? When does the dm
      move to the next priority group?


Thanks
-- 

____________________________________________________________

Erez Zilber | 972-9-971-7689

Software Engineer, Storage Team

Voltaire – _The Grid Backbone_

__

www.voltaire.com <http://www.voltaire.com/>

Re: Setting up a multipath device

[Christophe Varoqui]

> 
>    1. Was it wrong to run 'multipath 2 round-robin 1 0 /dev/sda
>       round-robin 1 0 /dev/sdb'?

The multipath command just don't understand these parameters.
This is a map as seen by the DM kernel driver. You can feed it to
dmsetup(8), not to multipath(8).

>    2. If I don't supply any parameters, how does the dm know which
>       devices are the same physical device (that /dev/sda & /dev/sdb are
>       the same physical device)?

Because it scans the paths for unique LU ids, then coalesce.

>    3. How are the priority groups defined if no parameters are supplied?

User defines, through config file.

>    4. What is the timeout for a device that has failed? When does the dm
>       move to the next priority group?
> 
When secondary PG priority becomes higher than the active PG priority,
modulo user-define delay or user-define disabling.

Or when all paths in the active PG have failed.

Regards,
cvaroqui

Re: Setting up a multipath device

[Erez Zilber]

Some more questions:

   1. I couldn't find the config file. Where is it located? Is there any
      documentation about it?
   2. How can I configure fancy naming for devices (instaed of using
      something like /dev/mapper/3600c0ff00000000007c1d121c397f503) in
      the config file?
   3. How can I config the delay from the time that a device has failed
      until the dm moves to the next active device? Again, I understand
      that I should use the config file somehow.

I tried to run the following test: I created a multipath device with 2 
physical devices (/dev/sda & /dev/sdb). I started running traffic on the 
multipath device. All traffic ran through /dev/sda. Then, I disconnected 
the iSCSI target that was represented by /dev/sda. After some time, the 
traffic was directed to /dev/sdb. I stopped the traffic, disconnected 
the iSCSI target that was represented by /dev/sdb and reconnected 
/dev/sda. Now, if I run `multipath -l`, I see the following output:

3600c0ff00000000007c1d121c397f503
[size=10 GB][features=0][hwhandler=0]
\_ round-robin 0 [prio=0][enabled]
 \_ 0:0:0:0 sda 8:0   [failed][undef]
\_ round-robin 0 [prio=0][enabled]
 \_ 1:0:0:0 sdb 8:16  [failed][faulty]

/dev/sdb is dead and I'm ok with it. However, /dev/sda is alive. I was 
able to run the following command:

dd if=/dev/sda of=/dev/null count=100000

However, if I run the following command:

dd if=/dev/mapper/3600c0ff00000000007c1d121c397f503 of=/dev/null 
count=100000

This is what I get:

dd: reading `/dev/mapper/3600c0ff00000000007c1d121c397f503': 
Input/output error
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00067 seconds, 0.0 kB/s

Can't it recongnize that a device that has failed (/dev/sda) is back to 
life?

Thanks
Erez

Re: Setting up a multipath device

[Christophe Varoqui]

Le dimanche 07 mai 2006 à 17:01 +0300, Erez Zilber a écrit :
> Some more questions:
> 
>    1. I couldn't find the config file. Where is it located? Is there any
>       documentation about it?

The config file normal location is /etc/multipath.conf

>    2. How can I configure fancy naming for devices (instaed of using
>       something like /dev/mapper/3600c0ff00000000007c1d121c397f503) in
>       the config file?

Have a look at the templates provided with the upstream package.

>    3. How can I config the delay from the time that a device has failed
>       until the dm moves to the next active device? Again, I understand
>       that I should use the config file somehow.
> 
Ditto.

> I tried to run the following test: I created a multipath device with 2 
> physical devices (/dev/sda & /dev/sdb). I started running traffic on the 
> multipath device. All traffic ran through /dev/sda. Then, I disconnected 
> the iSCSI target that was represented by /dev/sda. After some time, the 
> traffic was directed to /dev/sdb. I stopped the traffic, disconnected 
> the iSCSI target that was represented by /dev/sdb and reconnected 
> /dev/sda. Now, if I run `multipath -l`, I see the following output:
> 
> 3600c0ff00000000007c1d121c397f503
> [size=10 GB][features=0][hwhandler=0]
> \_ round-robin 0 [prio=0][enabled]
>  \_ 0:0:0:0 sda 8:0   [failed][undef]
> \_ round-robin 0 [prio=0][enabled]
>  \_ 1:0:0:0 sdb 8:16  [failed][faulty]
> 
> /dev/sdb is dead and I'm ok with it. However, /dev/sda is alive. I was 
> able to run the following command:
> 
> dd if=/dev/sda of=/dev/null count=100000
> 
> However, if I run the following command:
> 
> dd if=/dev/mapper/3600c0ff00000000007c1d121c397f503 of=/dev/null 
> count=100000
> 
> This is what I get:
> 
> dd: reading `/dev/mapper/3600c0ff00000000007c1d121c397f503': 
> Input/output error
> 0+0 records in
> 0+0 records out
> 0 bytes (0 B) copied, 0.00067 seconds, 0.0 kB/s
> 
> Can't it recongnize that a device that has failed (/dev/sda) is back to 
> life?
> 
Is the daemon (multipathd) running ? The daemon takes care of path
revalidation.

Regards,
cvaroqui