[LARTC] urgent TEQL problem

[LARTC] urgent TEQL problem

[Li, Ji]

OK, I spammed the mailing list recently, but I will be fired if I can't
solve the problem today. (just kidding, but I did waste lots of time on
it  :(  )

The common configuration for teql is for two computers connected
directly with two links. My topology is a little different: one link is
connected directly, but the other is connected through a gateway. My
problem is teql can't send packets through both links at the same time.
When I don't configure a gateway for teql0 in the routing table, all
packets go to the direct link; when I do add the gateway, all packets go
to the link with the gateway.

It seems to me that the problem is that when packets are forwarded by
the teql0 interface to each local Ethernet cards, the routing is still
based on the routing entry for the teql0 IP address in the routing
table, instead of being based on the IP of the local Ethernet interface
that packets are passed to. Since I can't both specify the gateway
address and not specify at the same time, there is only one link that
teql finds working and pass packets to at each time. In the common case
when both links are directly connected, there is no such problem (I
tried two directed links, and teql just works). Am I correct? Is there
any way to solve the problem, besides changing the gateway to a bridge?
(If it's not true, there must be something wrong with my routing tables.
One thing that I am not sure if it's OK is that I put teql0 and eth1
into the same subnet.)

Thanks a lot!

-Ji
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] urgent TEQL problem

[Pan'ko Alexander]

I advice you to do not discribe your problem, but give your routing rules on both sides. I think here is your problem.

-- 
With best regards, Pan'ko Alexander.
pankoAA@yandex.ru
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

RE: [LARTC] urgent TEQL problem

[Li, Ji]

Thanks for your advice. The routing tables are shown below. Your help is
highly appreciated!


Topology: Node1.eth0 and Node2.eth0 are directly connected. Node1.eth1
is connected to Gateway.eth0, and Node2.eth1 is connected to
Gateway.eth1.
The problem is: When I "ping 16.119.144.66" from Node1, all ping request
go to Node1.eth0, no packet goes to Node1.eth1.

Node 1:
- eth0:  16.119.144.21
- eth1:  16.119.144.33
- teql0: 16.119.144.34

Destination     Gateway         Genmask         Flags Metric Ref    Use
Iface
16.119.144.66   0.0.0.0         255.255.255.255 UH    0      0        0
teql0
16.119.144.27   0.0.0.0         255.255.255.255 UH    0      0        0
eth0
16.119.144.32   0.0.0.0         255.255.255.224 U     0      0        0
eth1
16.119.144.64   16.119.144.35   255.255.255.224 UG    0      0        0
eth1
0.0.0.0         16.119.144.35   0.0.0.0         UG    0      0        0
eth1


Node 2:
- eth0:  16.119.144.27
- eth1:  16.119.144.65
- teql0: 16.119.144.66

Destination     Gateway         Genmask         Flags Metric Ref    Use
Iface
16.119.144.21   0.0.0.0         255.255.255.255 UH    0      0        0
eth0
16.119.144.34   0.0.0.0         255.255.255.255 UH    0      0        0
teql0
16.119.144.32   16.119.144.67   255.255.255.224 UG    0      0        0
eth1
16.119.144.64   0.0.0.0         255.255.255.224 U     0      0        0
eth1
0.0.0.0         16.119.144.67   0.0.0.0         UG    0      0        0
eth1


Gateway:
- eth0: 16.119.144.35
- eth1: 16.119.144.67

Destination     Gateway         Genmask         Flags Metric Ref    Use
Iface
16.119.144.32   0.0.0.0         255.255.255.224 U     0      0        0
eth0
16.119.144.64   0.0.0.0         255.255.255.224 U     0      0        0
eth1
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0
lo 


Best regards,
-Ji

-----Original Message-----
From: lartc-bounces@mailman.ds9a.nl
[mailto:lartc-bounces@mailman.ds9a.nl] On Behalf Of Pan'ko Alexander
Sent: Thursday, June 23, 2005 2:51 PM
To: lartc@mailman.ds9a.nl
Subject: Re: [LARTC] urgent TEQL problem


I advice you to do not discribe your problem, but give your routing
rules on both sides. I think here is your problem.

--
With best regards, Pan'ko Alexander.
pankoAA@yandex.ru
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

RE: [LARTC] urgent TEQL problem

[Li, Ji]

One more thing to add is that when I "ping 16.119.144.66" from Node1
(16.119.144.66 is the teql0 of Node2), all messages I saw from
Node1.eth1 are ARP messages "Who has 16.119.144.66? Tell 16.119.144.33".

Thanks,
-Ji

-----Original Message-----
From: lartc-bounces@mailman.ds9a.nl
[mailto:lartc-bounces@mailman.ds9a.nl] On Behalf Of Li, Ji
Sent: Thursday, June 23, 2005 3:45 PM
To: Pan'ko Alexander; lartc@mailman.ds9a.nl
Cc: jli@mit.edu
Subject: RE: [LARTC] urgent TEQL problem

Thanks for your advice. The routing tables are shown below. Your help is
highly appreciated!


Topology: Node1.eth0 and Node2.eth0 are directly connected. Node1.eth1
is connected to Gateway.eth0, and Node2.eth1 is connected to
Gateway.eth1.
The problem is: When I "ping 16.119.144.66" from Node1, all ping request
go to Node1.eth0, no packet goes to Node1.eth1.

Node 1:
- eth0:  16.119.144.21
- eth1:  16.119.144.33
- teql0: 16.119.144.34

Destination     Gateway         Genmask         Flags Metric Ref    Use
Iface
16.119.144.66   0.0.0.0         255.255.255.255 UH    0      0        0
teql0
16.119.144.27   0.0.0.0         255.255.255.255 UH    0      0        0
eth0
16.119.144.32   0.0.0.0         255.255.255.224 U     0      0        0
eth1
16.119.144.64   16.119.144.35   255.255.255.224 UG    0      0        0
eth1
0.0.0.0         16.119.144.35   0.0.0.0         UG    0      0        0
eth1


Node 2:
- eth0:  16.119.144.27
- eth1:  16.119.144.65
- teql0: 16.119.144.66

Destination     Gateway         Genmask         Flags Metric Ref    Use
Iface
16.119.144.21   0.0.0.0         255.255.255.255 UH    0      0        0
eth0
16.119.144.34   0.0.0.0         255.255.255.255 UH    0      0        0
teql0
16.119.144.32   16.119.144.67   255.255.255.224 UG    0      0        0
eth1
16.119.144.64   0.0.0.0         255.255.255.224 U     0      0        0
eth1
0.0.0.0         16.119.144.67   0.0.0.0         UG    0      0        0
eth1


Gateway:
- eth0: 16.119.144.35
- eth1: 16.119.144.67

Destination     Gateway         Genmask         Flags Metric Ref    Use
Iface
16.119.144.32   0.0.0.0         255.255.255.224 U     0      0        0
eth0
16.119.144.64   0.0.0.0         255.255.255.224 U     0      0        0
eth1
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0
lo 


Best regards,
-Ji

-----Original Message-----
From: lartc-bounces@mailman.ds9a.nl
[mailto:lartc-bounces@mailman.ds9a.nl] On Behalf Of Pan'ko Alexander
Sent: Thursday, June 23, 2005 2:51 PM
To: lartc@mailman.ds9a.nl
Subject: Re: [LARTC] urgent TEQL problem


I advice you to do not discribe your problem, but give your routing
rules on both sides. I think here is your problem.

--
With best regards, Pan'ko Alexander.
pankoAA@yandex.ru
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] urgent TEQL problem

[Taylor, Grant]

Based on your previous email and this email I see a couple problems with your situation.

1)  Your ""Gateway will not be able to ""route correctly between two interfaces on the same subnet.
2)  Node1 is ARPing out eth1 to try to find 16.119.114.66 on it's local subnet as it should be there but it is not as it is on the other side of a gateway.

If you do not want to do this on layer 2 and turn your gateway in to a bridge I think you will have to do all of this on layer 3 and route *BOTH* sides of teql0, how to do this I'm not sure of at the moment.  I have a feeling you will need to ultimately try to reach an IP that is not on any of your physical network cards but rather something on a dummy interface that is accessible via a route using either interface to get to.  I'll try to describe such a setup below.

Node 1:
- eth0:  <subnet 1>.1
- eth1:  <subnet 2>.1
- teql0: <subnet ?>.1
- dumy0: <subnet 3>.1

Node 2:
- eth0:  <subnet 1>.2
- eth1:  <subnet 4>.2
- teql0: <subnet ?>.2
- dump0: <subnet 5>.2

Gateway:
- eth0:  <subnet 2>.254
- eth1:  <subnet 4>.254

(Sitting her looking at this I'm not entirely sure that you even need teql0 but rather ECMP routing.)

Node 1 routing table:
<subnet 3> via <subnet 1>.2   metric 2
<subnet 3> via <subnet 2>.254 metric 1

Node 2 routing table:
<subnet 5> via <subnet 1>.1   metric 2
<subnet 5> via <subnet 4>.254 metric 1

Gateway routing table:
<subnet 3> via <subnet 2>.1 metric 1
<subnet 5> via <subnet 4>.2 metric 1

This *SHOULD* (if I have things correct in my head) establish two routes from <subnet 3> to <subnet 5> with the same overall metric of 2.  However to use ECMP you will need the metrics for both routes from <subnet 3 or 5> to <subnet 5 or 3> to be the same on Node 1 and Node 2.  Thus I might modify the routing tables as such.

Node 1 routing table:
<subnet 3> via <subnet 1>.2   metric 2
<subnet 3> via <subnet 2>.254 metric 2

Node 2 routing table:
<subnet 5> via <subnet 1>.1   metric 2
<subnet 5> via <subnet 4>.254 metric 2

Gateway routing table:
<subnet 3> via <subnet 2>.1 metric 0
<subnet 5> via <subnet 4>.2 metric 0

As my pager goes off I realize that I have to submit a partial post back to the list, but hopefully there is enough here to get a couple of points across and enough for someone else to work with to help flesh out this idea.



Grant. . . .

Li, Ji wrote:
> One more thing to add is that when I "ping 16.119.144.66" from Node1
> (16.119.144.66 is the teql0 of Node2), all messages I saw from
> Node1.eth1 are ARP messages "Who has 16.119.144.66? Tell 16.119.144.33".
> 
> Thanks,
> -Ji
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

RE: [LARTC] urgent TEQL problem

[Li, Ji]

Thanks a lot, Grant!

I don't understand problem (1). Do you mean the routing between the two
subnets connected by the gateway? I think the gateway can route
correctly since I have 
"16.119.144.64   16.119.144.35   255.255.255.224 UG    0      0        0
eth1" in Node1's routing table, and a similar entry in Node2's routing
table. 

Problem (2) seems to be caused by "16.119.144.66   0.0.0.0
255.255.255.255 UH    0      0        0   teql0" in Node1's routing
table. There are two paths to 16.119.144.66, one with direct connection,
and one with the gateway. I don't know how to specify both cases at the
same time. As I said in my original post, It seems that the problem is
that when packets are forwarded by the teql0 interface to each local
Ethernet cards, the routing is still based on the routing entry for the
teql0_IP address in the routing table, instead of being based on the IP
of the local Ethernet interface that packets are passed to. Since I
can't both specify the gateway address and not specify at the same time,
there is only one link that teql finds working and pass packets to at
each time. But it sounds unreasonable for TEQL to be implemented that
way. Probably my configuration is wrong somewhere.

I think your idea of dummy interfaces will probably work, but I'll try
to change the gateway into a bridge first. Not sure which one is simpler
yet. 

Thanks a lot again!

-Ji

-----Original Message-----
From: lartc-bounces@mailman.ds9a.nl
[mailto:lartc-bounces@mailman.ds9a.nl] On Behalf Of Taylor, Grant
Sent: Thursday, June 23, 2005 5:03 PM
To: lartc@mailman.ds9a.nl
Subject: Re: [LARTC] urgent TEQL problem

Based on your previous email and this email I see a couple problems with
your situation.

1)  Your ""Gateway will not be able to ""route correctly between two
interfaces on the same subnet.
2)  Node1 is ARPing out eth1 to try to find 16.119.114.66 on it's local
subnet as it should be there but it is not as it is on the other side of
a gateway.

If you do not want to do this on layer 2 and turn your gateway in to a
bridge I think you will have to do all of this on layer 3 and route
*BOTH* sides of teql0, how to do this I'm not sure of at the moment.  I
have a feeling you will need to ultimately try to reach an IP that is
not on any of your physical network cards but rather something on a
dummy interface that is accessible via a route using either interface to
get to.  I'll try to describe such a setup below.

Node 1:
- eth0:  <subnet 1>.1
- eth1:  <subnet 2>.1
- teql0: <subnet ?>.1
- dumy0: <subnet 3>.1

Node 2:
- eth0:  <subnet 1>.2
- eth1:  <subnet 4>.2
- teql0: <subnet ?>.2
- dump0: <subnet 5>.2

Gateway:
- eth0:  <subnet 2>.254
- eth1:  <subnet 4>.254

(Sitting her looking at this I'm not entirely sure that you even need
teql0 but rather ECMP routing.)

Node 1 routing table:
<subnet 3> via <subnet 1>.2   metric 2
<subnet 3> via <subnet 2>.254 metric 1

Node 2 routing table:
<subnet 5> via <subnet 1>.1   metric 2
<subnet 5> via <subnet 4>.254 metric 1

Gateway routing table:
<subnet 3> via <subnet 2>.1 metric 1
<subnet 5> via <subnet 4>.2 metric 1

This *SHOULD* (if I have things correct in my head) establish two routes
from <subnet 3> to <subnet 5> with the same overall metric of 2.
However to use ECMP you will need the metrics for both routes from
<subnet 3 or 5> to <subnet 5 or 3> to be the same on Node 1 and Node 2.
Thus I might modify the routing tables as such.

Node 1 routing table:
<subnet 3> via <subnet 1>.2   metric 2
<subnet 3> via <subnet 2>.254 metric 2

Node 2 routing table:
<subnet 5> via <subnet 1>.1   metric 2
<subnet 5> via <subnet 4>.254 metric 2

Gateway routing table:
<subnet 3> via <subnet 2>.1 metric 0
<subnet 5> via <subnet 4>.2 metric 0

As my pager goes off I realize that I have to submit a partial post back
to the list, but hopefully there is enough here to get a couple of
points across and enough for someone else to work with to help flesh out
this idea.



Grant. . . .

Li, Ji wrote:
> One more thing to add is that when I "ping 16.119.144.66" from Node1
> (16.119.144.66 is the teql0 of Node2), all messages I saw from
> Node1.eth1 are ARP messages "Who has 16.119.144.66? Tell
16.119.144.33".
> 
> Thanks,
> -Ji
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

RE: [LARTC] urgent TEQL problem

[Li, Ji]

Dear Pan'ko,

Thanks for your suggestion and sorry to reply late!

I did read that chapter 10 before I start doing this, and I have tried your suggested configuration, but the same problem remains. 

The difference between my configuration and chapter 10 is that my two connections are different: one is direct connection, and the other is through a gateway. (I did try the configuration in Chapter 10, and it works) 

It seems to me that when packets are forwarded by the teql0 interface to each local Ethernet cards, the routing is still based on the routing entry for the teql0_IP address in the routing table, instead of being based on the IP of the local Ethernet interface. Since I can't both specify the gateway address and not specify it for the teql0 at the same time, there is only one link that teql finds working and pass packets to at each time. But it sounds unreasonable for TEQL to be implemented that way. 

Now I don't use a gateway since teql doesn't work in this heteogeneous case. I configured the machine in the middle as a bridge to avoid the routing problem. (And I switched to NetEm instead of NIST Net, because NetEm can run on a bridge)

Please let me know if you have any other suggestions! Thanks again for your kind help!

Best,
-Ji


-----Original Message-----
From: Pan'ko Alexander [mailto:pankoAA@yandex.ru] 
Sent: Saturday, June 25, 2005 1:36 AM
To: Li, Ji
Subject: Re: [LARTC] urgent TEQL problem

On Thu, 23 Jun 2005 16:01:50 -0400
"Li, Ji" <ji.li3@hp.com> wrote:

Read chapter 10 of http://lartc.org/lartc.html, please.

As for me, you have very strange routing and subnets configuration.
Try to do teql on else subnet /31

My variant:
 Node 1:
 - eth0:  16.119.144.21
 - eth1:  16.119.144.33
 - teql0: 192.168.1.0
 
 Destination     Gateway         Genmask         Flags Metric Ref    Use  Iface
 192.168.1.0   0.0.0.0         255.255.255.254 UH    0      0        0  teql0
 16.119.144.27   0.0.0.0         255.255.255.255 UH    0      0        0  eth0
 16.119.144.32   0.0.0.0         255.255.255.224 U     0      0        0  eth1
 16.119.144.64   16.119.144.35   255.255.255.224 UG    0      0        0  eth1
 0.0.0.0         16.119.144.35   0.0.0.0         UG    0      0        0  eth1


 Node 2:
 - eth0:  16.119.144.27
 - eth1:  16.119.144.65
 - teql0: 192.168.1.1
 
 Destination     Gateway         Genmask         Flags Metric Ref    Use
 Iface
 16.119.144.21   0.0.0.0         255.255.255.255 UH    0      0        0  eth0
 192.168.1.0   0.0.0.0         255.255.255.254 UH    0      0        0   teql0
 16.119.144.32   16.119.144.67   255.255.255.224 UG    0      0        0  eth1
 16.119.144.64   0.0.0.0         255.255.255.224 U     0      0        0  eth1
 0.0.0.0         16.119.144.67   0.0.0.0         UG    0      0        0  eth1


Try this. Maybe when ping goes to 16.119.144.66 it do not undestand clearly if need to use gw. Now you will be able ping 192.168.19.1 See example http://lartc.org/lartc.html Take in maind that icmp may have a special status, try other protocols, check firewall rules.

Please reply.
Good luck.


> One more thing to add is that when I "ping 16.119.144.66" from Node1
> (16.119.144.66 is the teql0 of Node2), all messages I saw from
> Node1.eth1 are ARP messages "Who has 16.119.144.66? Tell 16.119.144.33".
> 
> Thanks,
> -Ji
> 
> -----Original Message-----
> From: lartc-bounces@mailman.ds9a.nl
> [mailto:lartc-bounces@mailman.ds9a.nl] On Behalf Of Li, Ji
> Sent: Thursday, June 23, 2005 3:45 PM
> To: Pan'ko Alexander; lartc@mailman.ds9a.nl
> Cc: jli@mit.edu
> Subject: RE: [LARTC] urgent TEQL problem
> 
> Thanks for your advice. The routing tables are shown below. Your help 
> is highly appreciated!
> 
> 
> Topology: Node1.eth0 and Node2.eth0 are directly connected. Node1.eth1 
> is connected to Gateway.eth0, and Node2.eth1 is connected to 
> Gateway.eth1.
> The problem is: When I "ping 16.119.144.66" from Node1, all ping 
> request go to Node1.eth0, no packet goes to Node1.eth1.
> 
> Node 1:
> - eth0:  16.119.144.21
> - eth1:  16.119.144.33
> - teql0: 16.119.144.34
> 
> Destination     Gateway         Genmask         Flags Metric Ref    Use
> Iface
> 16.119.144.66   0.0.0.0         255.255.255.255 UH    0      0        0
> teql0
> 16.119.144.27   0.0.0.0         255.255.255.255 UH    0      0        0
> eth0
> 16.119.144.32   0.0.0.0         255.255.255.224 U     0      0        0
> eth1
> 16.119.144.64   16.119.144.35   255.255.255.224 UG    0      0        0
> eth1
> 0.0.0.0         16.119.144.35   0.0.0.0         UG    0      0        0
> eth1
> 
> 
> Node 2:
> - eth0:  16.119.144.27
> - eth1:  16.119.144.65
> - teql0: 16.119.144.66
> 
> Destination     Gateway         Genmask         Flags Metric Ref    Use
> Iface
> 16.119.144.21   0.0.0.0         255.255.255.255 UH    0      0        0
> eth0
> 16.119.144.34   0.0.0.0         255.255.255.255 UH    0      0        0
> teql0
> 16.119.144.32   16.119.144.67   255.255.255.224 UG    0      0        0
> eth1
> 16.119.144.64   0.0.0.0         255.255.255.224 U     0      0        0
> eth1
> 0.0.0.0         16.119.144.67   0.0.0.0         UG    0      0        0
> eth1
> 
> 
> Gateway:
> - eth0: 16.119.144.35
> - eth1: 16.119.144.67
> 
> Destination     Gateway         Genmask         Flags Metric Ref    Use
> Iface
> 16.119.144.32   0.0.0.0         255.255.255.224 U     0      0        0
> eth0
> 16.119.144.64   0.0.0.0         255.255.255.224 U     0      0        0
> eth1
> 127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0
> lo
> 
> 
> Best regards,
> -Ji
> 
> -----Original Message-----
> From: lartc-bounces@mailman.ds9a.nl
> [mailto:lartc-bounces@mailman.ds9a.nl] On Behalf Of Pan'ko Alexander
> Sent: Thursday, June 23, 2005 2:51 PM
> To: lartc@mailman.ds9a.nl
> Subject: Re: [LARTC] urgent TEQL problem
> 
> 
> I advice you to do not discribe your problem, but give your routing 
> rules on both sides. I think here is your problem.
> 
> --
> With best regards, Pan'ko Alexander.
> pankoAA@yandex.ru
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc


--
С наилучшими пожеланиями, Панько Александр.
With best regards, Pan'ko Alexander.
pankoAA@yandex.ru
http://interdon.net/~panko/
ICQ 231647363
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc