How to configure if there are tow network cards in Client

How to configure if there are tow network cards in Client

[蔡毅]

Hi all,
    When we read the code, we haven’t find the function that the client can bind a specific IP. In Ceph’s configuration, we could only find the parameter “public network”, but it seems acts on the OSD but not the client.
    There is a scenario that the client has two network cards named NIC1 and NIC2. The NIC1 is responsible for communicating with cluster (monitor and RADOS) and the NIC2 has other services except Ceph’s client. So   we need the client can bind specific IP in order to differentiate the IP communicating with cluster from another IP serving other applications. We want to know is there any configuration in Ceph to achieve this function? If there is, how could we configure the IP? if not, could we add this function in Ceph? Thank you so much.
Best regards,
Cai Yi

Re: How to configure if there are tow network cards in Client

[Sage Weil]

On Fri, 25 Dec 2015, ?? wrote:
> Hi all,
>     When we read the code, we haven?t find the function that the client can bind a specific IP. In Ceph?s configuration, we could only find the parameter ?public network?, but it seems acts on the OSD but not the client.
>     There is a scenario that the client has two network cards named NIC1 and NIC2. The NIC1 is responsible for communicating with cluster (monitor and RADOS) and the NIC2 has other services except Ceph?s client. So   we need the client can bind specific IP in order to differentiate the IP communicating with cluster from another IP serving other applications. We want to know is there any configuration in Ceph to achieve this function? If there is, how could we configure the IP? if not, could we add this function in Ceph? Thank you so much.

Right.  There isn't a configurable to do this now--we've always just let 
the kernel network layer sort it out. Is this just a matter of calling 
bind on the socket before connecting? I've never done this before..

sage

Re: How to configure if there are tow network cards in Client

[Linux Chips]

On 12/28/2015 07:47 PM, Sage Weil wrote:
> On Fri, 25 Dec 2015, ?? wrote:
>> Hi all,
>>      When we read the code, we haven?t find the function that the client can bind a specific IP. In Ceph?s configuration, we could only find the parameter ?public network?, but it seems acts on the OSD but not the client.
>>      There is a scenario that the client has two network cards named NIC1 and NIC2. The NIC1 is responsible for communicating with cluster (monitor and RADOS) and the NIC2 has other services except Ceph?s client. So   we need the client can bind specific IP in order to differentiate the IP communicating with cluster from another IP serving other applications. We want to know is there any configuration in Ceph to achieve this function? If there is, how could we configure the IP? if not, could we add this function in Ceph? Thank you so much.
you can use routing tables plus routing rules. otherwise linux will just 
use the default gateway.
or you can put the second interface on the same public net of ceph. 
though that would break if you have multiple external nets.
> Right.  There isn't a configurable to do this now--we've always just let
> the kernel network layer sort it out. Is this just a matter of calling
> bind on the socket before connecting? I've never done this before..
linux will send all packets to the default gateway event if an 
application binds to an ip on different interface, the packet will go 
out with the source address as the binded one but through your router. 
the only solution, even if the bind function exists is to use the 
routing tables and rules.
>
> sage
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re:Re: How to configure if there are tow network cards in Client

[蔡毅]

      Thank for your replies.  
      So is it reasonable that we could write a file such as shell script to bind one process with a specific IP and modify the routing tables and rules
as one of Ceph’s tools? So that the users is convenient when they want to change the NIC connecting with the OSD.



At 2015-12-29 18:21:21, "Linux Chips" <linux.chips@gmail.com> wrote:
>On 12/28/2015 07:47 PM, Sage Weil wrote:
>> On Fri, 25 Dec 2015, ?? wrote:
>>> Hi all,
>>>      When we read the code, we haven?t find the function that the client can bind a specific IP. In Ceph?s configuration, we could only find the parameter ?public network?, but it seems acts on the OSD but not the client.
>>>      There is a scenario that the client has two network cards named NIC1 and NIC2. The NIC1 is responsible for communicating with cluster (monitor and RADOS) and the NIC2 has other services except Ceph?s client. So   we need the client can bind specific IP in order to differentiate the IP communicating with cluster from another IP serving other applications. We want to know is there any configuration in Ceph to achieve this function? If there is, how could we configure the IP? if not, could we add this function in Ceph? Thank you so much.
>you can use routing tables plus routing rules. otherwise linux will just 
>use the default gateway.
>or you can put the second interface on the same public net of ceph. 
>though that would break if you have multiple external nets.
>> Right.  There isn't a configurable to do this now--we've always just let
>> the kernel network layer sort it out. Is this just a matter of calling
>> bind on the socket before connecting? I've never done this before..
>linux will send all packets to the default gateway event if an 
>application binds to an ip on different interface, the packet will go 
>out with the source address as the binded one but through your router. 
>the only solution, even if the bind function exists is to use the 
>routing tables and rules.
>>
>> sage
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>

Re: How to configure if there are tow network cards in Client

[Linux Chips]

it would certainly help those with less knowledge about networking in 
linux, though i do not know how many people using ceph are in this 
category. Sage and the others here may have a better idea about its 
feasibility.
but i usually use rule-* and route-* (in CentOS) files, they work with 
networkmanager, and very easy to configure. in ubuntu you can put them 
in interfaces file, and they are as easy. if such a tool is made, i 
think it should understand the ceph.conf file, but i doubt it can figure 
out the routes correctly without you putting them in.

On 12/29/2015 03:58 PM, 蔡毅 wrote:
>        Thank for your replies.
>        So is it reasonable that we could write a file such as shell script to bind one process with a specific IP and modify the routing tables and rules
> as one of Ceph’s tools? So that the users is convenient when they want to change the NIC connecting with the OSD.
>
>
>
> At 2015-12-29 18:21:21, "Linux Chips"<linux.chips@gmail.com>  wrote:
>> On 12/28/2015 07:47 PM, Sage Weil wrote:
>>> On Fri, 25 Dec 2015, ?? wrote:
>>>> Hi all,
>>>>       When we read the code, we haven?t find the function that the client can bind a specific IP. In Ceph?s configuration, we could only find the parameter ?public network?, but it seems acts on the OSD but not the client.
>>>>       There is a scenario that the client has two network cards named NIC1 and NIC2. The NIC1 is responsible for communicating with cluster (monitor and RADOS) and the NIC2 has other services except Ceph?s client. So   we need the client can bind specific IP in order to differentiate the IP communicating with cluster from another IP serving other applications. We want to know is there any configuration in Ceph to achieve this function? If there is, how could we configure the IP? if not, could we add this function in Ceph? Thank you so much.
>> you can use routing tables plus routing rules. otherwise linux will just
>> use the default gateway.
>> or you can put the second interface on the same public net of ceph.
>> though that would break if you have multiple external nets.
>>> Right.  There isn't a configurable to do this now--we've always just let
>>> the kernel network layer sort it out. Is this just a matter of calling
>>> bind on the socket before connecting? I've never done this before..
>> linux will send all packets to the default gateway event if an
>> application binds to an ip on different interface, the packet will go
>> out with the source address as the binded one but through your router.
>> the only solution, even if the bind function exists is to use the
>> routing tables and rules.
>>> sage
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>> the body of a message tomajordomo@vger.kernel.org
>>> More majordomo info athttp://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html