From: Eric <iggiggitoo@gmail.com>
To: device-mapper development <dm-devel@redhat.com>
Subject: Re: Multipath not using multiple NICs at once
Date: Tue, 25 Mar 2014 09:55:48 -0400 [thread overview]
Message-ID: <53318AE4.2000308@gmail.com> (raw)
In-Reply-To: <532EEF01.3070400@suse.de>
On 3/23/2014 10:26 AM, Hannes Reinecke wrote:
> On 03/23/2014 05:08 AM, Eric wrote:
>> Hello,
>>
>> I'm fairly new to multipath and I am having an issue with it not using
>> all of my NICs. Currently, my node has 4 gigE NICs to my storage network
>> and the SAN has 8 gigE NICs to the same network and I am attempting to
>> setup multipath with ISCSI in order to utilize more than 1 gigabit
>> connection. However, when I use nload to check the network usage, you
>> can see the traffic hop around the NICs. For example, data would send
>> for 2-3 seconds on eth1, then stops and starts on eth2, then stops and
>> starts back up on eth3. All perfectly distributed, but in this setup,
>> unable to reach beyond the capacity of a 1 gigabit connection.
>>
>> I have each NIC on a different network (e.g. 10.1.1.0/24 for eth1,
>> 10.1.2.0/24 for eth2, etc.). Netstat shows that the connections are
>> being made each to different IPs:
>>
>> tcp 0 0 10.1.3.8:35493 10.1.5.241:3260 ESTABLISHED
>> tcp 0 0 10.1.3.8:53972 10.1.3.241:3260 ESTABLISHED
>> tcp 0 0 10.1.6.8:41090 10.1.4.241:3260 ESTABLISHED
>> tcp 0 0 10.1.1.8:50754 10.1.1.241:3260 ESTABLISHED
>> tcp 0 0 10.1.6.8:49780 10.1.5.241:3260 ESTABLISHED
>> tcp 0 0 10.1.1.8:36938 10.1.6.241:3260 ESTABLISHED
>> tcp 0 0 10.1.6.8:52009 10.1.6.241:3260 ESTABLISHED
>> tcp 0 0 10.1.5.8:51630 10.1.1.241:3260 ESTABLISHED
>> tcp 0 0 10.1.5.8:54481 10.1.4.241:3260 ESTABLISHED
>> tcp 0 0 10.1.1.8:54504 10.1.5.241:3260 ESTABLISHED
>> tcp 0 0 10.1.5.8:58229 10.1.3.241:3260 ESTABLISHED
>> tcp 0 0 10.1.3.8:49031 10.1.1.241:3260 ESTABLISHED
>> tcp 0 0 10.1.5.8:40551 10.1.6.241:3260 ESTABLISHED
>> tcp 0 0 10.1.4.8:45016 10.1.5.241:3260 ESTABLISHED
>> tcp 0 0 10.1.4.8:55665 10.1.4.241:3260 ESTABLISHED
>> tcp 0 0 10.1.6.8:57472 10.1.3.241:3260 ESTABLISHED
>> tcp 0 0 10.1.6.8:39278 10.1.1.241:3260 ESTABLISHED
>> tcp 0 0 10.1.4.8:41329 10.1.6.241:3260 ESTABLISHED
>> tcp 0 0 10.1.5.8:33553 10.1.5.241:3260 ESTABLISHED
>> tcp 0 0 10.1.3.8:48950 10.1.6.241:3260 ESTABLISHED
>> tcp 0 0 10.1.4.8:54752 10.1.1.241:3260 ESTABLISHED
>> tcp 0 0 10.1.1.8:40911 10.1.4.241:3260 ESTABLISHED
>> tcp 0 0 10.1.4.8:41135 10.1.3.241:3260 ESTABLISHED
>> tcp 0 0 10.1.3.8:44606 10.1.4.241:3260 ESTABLISHED
>> tcp 0 0 10.1.1.8:54677 10.1.3.241:3260 ESTABLISHED
>>
>> (10.1.*.8 is the node and 10.1.*.241 is the SAN)
>>
>> Here is my /etc/multipath.conf:
>>
>> defaults {
>> path_grouping_policy multibus
>> path_selector readsector0
>> polling_interval 3
>> path_selector "round-robin 0"
>> failback immediate
>> features "0"
>> no_path_retry 1
>> rr_weight uniform
>> rr_min_io 100
>> # user_friendly_names yes
>> }
>>
>> Both servers are running Ubuntu 12.04LTS.
>>
>> Any ideas?
>>
> Probably a routing issue. What is the routing table?
>
> Cheers,
>
> Hannes
Hannes,
Here's the output of "route -n":
0.0.0.0 209.124.44.1 0.0.0.0 UG 100 0 0 br0
10.1.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br1
10.1.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
10.1.4.0 0.0.0.0 255.255.255.0 U 0 0 0 eth3
10.1.5.0 0.0.0.0 255.255.255.0 U 0 0 0 eth4
10.1.6.0 0.0.0.0 255.255.255.0 U 0 0 0 eth5
10.2.5.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr2
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
209.124.44.0 0.0.0.0 255.255.255.0 U 0 0 0 br0
br0 is a bridge to eth0 and br1 is a bridge to eth1. eth1-5 are on the
network with the SAN.
Regards,
Eric
next prev parent reply other threads:[~2014-03-25 13:55 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-03-23 4:08 Multipath not using multiple NICs at once Eric
2014-03-23 13:52 ` urgrue
2014-03-25 13:59 ` Eric
2014-03-23 14:26 ` Hannes Reinecke
2014-03-25 13:55 ` Eric [this message]
-- strict thread matches above, loose matches on Subject: below --
2014-03-22 7:03 Eric
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=53318AE4.2000308@gmail.com \
--to=iggiggitoo@gmail.com \
--cc=dm-devel@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.