Re: Multipath not using multiple NICs at once

[Hannes Reinecke <hare@suse.de>]

On 03/23/2014 05:08 AM, Eric wrote:
> Hello,
>
> I'm fairly new to multipath and I am having an issue with it not using
> all of my NICs. Currently, my node has 4 gigE NICs to my storage network
> and the SAN has 8 gigE NICs to the same network and I am attempting to
> setup multipath with ISCSI in order to utilize more than 1 gigabit
> connection. However, when I use nload to check the network usage, you
> can see the traffic hop around the NICs. For example, data would send
> for 2-3 seconds on eth1, then stops and starts on eth2, then stops and
> starts back up on eth3. All perfectly distributed, but in this setup,
> unable to reach beyond the capacity of a 1 gigabit connection.
>
> I have each NIC on a different network (e.g. 10.1.1.0/24 for eth1,
> 10.1.2.0/24 for eth2, etc.). Netstat shows that the connections are
> being made each to different IPs:
>
> tcp        0      0 10.1.3.8:35493          10.1.5.241:3260 ESTABLISHED
> tcp        0      0 10.1.3.8:53972          10.1.3.241:3260 ESTABLISHED
> tcp        0      0 10.1.6.8:41090          10.1.4.241:3260 ESTABLISHED
> tcp        0      0 10.1.1.8:50754          10.1.1.241:3260 ESTABLISHED
> tcp        0      0 10.1.6.8:49780          10.1.5.241:3260 ESTABLISHED
> tcp        0      0 10.1.1.8:36938          10.1.6.241:3260 ESTABLISHED
> tcp        0      0 10.1.6.8:52009          10.1.6.241:3260 ESTABLISHED
> tcp        0      0 10.1.5.8:51630          10.1.1.241:3260 ESTABLISHED
> tcp        0      0 10.1.5.8:54481          10.1.4.241:3260 ESTABLISHED
> tcp        0      0 10.1.1.8:54504          10.1.5.241:3260 ESTABLISHED
> tcp        0      0 10.1.5.8:58229          10.1.3.241:3260 ESTABLISHED
> tcp        0      0 10.1.3.8:49031          10.1.1.241:3260 ESTABLISHED
> tcp        0      0 10.1.5.8:40551          10.1.6.241:3260 ESTABLISHED
> tcp        0      0 10.1.4.8:45016          10.1.5.241:3260 ESTABLISHED
> tcp        0      0 10.1.4.8:55665          10.1.4.241:3260 ESTABLISHED
> tcp        0      0 10.1.6.8:57472          10.1.3.241:3260 ESTABLISHED
> tcp        0      0 10.1.6.8:39278          10.1.1.241:3260 ESTABLISHED
> tcp        0      0 10.1.4.8:41329          10.1.6.241:3260 ESTABLISHED
> tcp        0      0 10.1.5.8:33553          10.1.5.241:3260 ESTABLISHED
> tcp        0      0 10.1.3.8:48950          10.1.6.241:3260 ESTABLISHED
> tcp        0      0 10.1.4.8:54752          10.1.1.241:3260 ESTABLISHED
> tcp        0      0 10.1.1.8:40911          10.1.4.241:3260 ESTABLISHED
> tcp        0      0 10.1.4.8:41135          10.1.3.241:3260 ESTABLISHED
> tcp        0      0 10.1.3.8:44606          10.1.4.241:3260 ESTABLISHED
> tcp        0      0 10.1.1.8:54677          10.1.3.241:3260 ESTABLISHED
>
> (10.1.*.8 is the node and 10.1.*.241 is the SAN)
>
> Here is my /etc/multipath.conf:
>
> defaults {
>          path_grouping_policy    multibus
>          path_selector           readsector0
>          polling_interval        3
>          path_selector           "round-robin 0"
>          failback                immediate
>          features                "0"
>          no_path_retry           1
>          rr_weight               uniform
>          rr_min_io               100
> #       user_friendly_names     yes
> }
>
> Both servers are running Ubuntu 12.04LTS.
>
> Any ideas?
>
Probably a routing issue. What is the routing table?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare@suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg)