netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Do you know the TCP stack? (127.x.x.x routing)
@ 2005-03-06  2:20 Zdenek Radouch
  2005-03-06  9:56 ` Martin Mares
  0 siblings, 1 reply; 52+ messages in thread
From: Zdenek Radouch @ 2005-03-06  2:20 UTC (permalink / raw)
  To: netdev, linux-net


How can I disable the stack processing for the 127 net?
Can someone estimate the amount of work needed to do that,
and/or point me to the relevant piece of code?

That is, I'd like to treat the 127 net the same as all other network
numbers. Since that is not the case in the current stack,
I want to remove or disable whatever processing there is.
I am using 2.4 kernel.



Here is a long version and a rationale.
I need a truly private network on a device that serves as a router
in someone else's network.  (The device itself has an internal network).
As far as I know, there is no provision for this within the existing network
numbering scheme.  Obviously, the architects of the current  numbering
scheme did not think one could build a router with more than a single
card. Unfortunately, routers are being built today with intelligent
line cards, and there is nothing simpler that the IP/socket based IPC
between Ethernet-connected cards.  The problem is immediately obvious:
one can't use any legal address for the internal network, since it may
collide with an external network the device is handling.  And since
the device can be routing non-Internet addresses, the "reserved"
numbers are as unusable as the normal ones. The only solution I've seen
on routers running BSD stack is to subnet the 127 net, and use one
of the subnets for the internal network.

Unfortunately, this does not work with the Linux stack, because the
127 net is treated (for good reasons I suppose) as a special net.
What I need is to remove whatever special processing there is,
so that the net can be treated as any other net.  Then I could, for
example, attach 127.0.0.1/16 to the "lo" device, and 127.1.0.0/16
would be my internal net, thus keeping the standard 127.0.0.1
address for the localhost, and having a truly private internal network.

So, that's all fine, except for the fact that I am not familiar with the
Linux stack code.  I do need this done, so as a first step I'd like
to get a feeling for the scope of the required modification and
an estimated effort to do this.  As with my previous problem,
if it turns out that this is a non-trivial effort,  I will gladly arrange
a short-term contract for someone in order to be adequately
compensated for the work.

Thanks.
-Zdenek

^ permalink raw reply	[flat|nested] 52+ messages in thread
* RE: Do you know the TCP stack? (127.x.x.x routing)
@ 2005-03-08 15:07 Steve Iribarne
  0 siblings, 0 replies; 52+ messages in thread
From: Steve Iribarne @ 2005-03-08 15:07 UTC (permalink / raw)
  To: hadi
  Cc: Eran Mann, Zdenek Radouch, Thomas Graf, Andi Kleen, Martin Mares,
	netdev, linux-net

-> BTW, please cc netdev or myself if you are addressing me. This email
was
-> just forwarde by someone else to me - I am not on linux-net. You seem
to
-> have trimmed down the CC list.
-> 

You should join the list and the quit when you are done.  Otherwise,
like with this email I get multiple copies of it.

-> I read this a few times and still didnt get it:
-> Why is it that people using 1918 addresses are affecting you?
-> Does using 127.x help you because you assume _nobody_ else would be
using
-> 127.x addresses?

I am in a chassis.  I need a way to do interface card communication.
Even if those cards are exposed to the outside world.  

-> I am assuming you want this address for some internal network whereas
the
-> external contains some routable addresses?
->

Yep.

-> > So I have this working in my products now.  I had to do something a
bit
-> > different in that I want a "special" 127.xx.xx.xx range to be sent
out
-> > on the wire.  So here is what I did.
-> 
-> [..]
-> 
-> Seems you did too much. Look at the 2 liner patch posted by Eran Mann

Right. That works too.  But what I did was about 10 lines of code.  And
I refined it a bit better I believe.  That way packets destined for "my"
internal network got out the appropriate interface.  The rest go on
their merry way to the loopback world.




^ permalink raw reply	[flat|nested] 52+ messages in thread
* RE: Do you know the TCP stack? (127.x.x.x routing)
@ 2005-03-09 15:01 Steve Iribarne
  2005-03-09 16:00 ` jamal
  2005-03-10  6:48 ` Catalin(ux aka Dino) BOIE
  0 siblings, 2 replies; 52+ messages in thread
From: Steve Iribarne @ 2005-03-09 15:01 UTC (permalink / raw)
  To: hadi, Henrik Nordstrom
  Cc: Martin Mares, Zdenek Radouch, Eran Mann, Thomas Graf, Andi Kleen,
	netdev, linux-net

First off, apologies for the all the cc's on this.  I hate doing it, but
I will only do it for this post!

-> 1) Addresses for intra-chasis communication.
-> The addresses used by the blades are intrachasis relevant only and
the
-> packets never leave the box. The blades are interconnected via some
-> L2/VLAN/bridge within the chasis.
-> 

Big assumption here.  The VLAN/Bridge/Router that I have in my chassis
is hooked up to a switch.  The switch will NOT send the packets on my
mgmt VLAN out over the network.  

(see below for more details on this.. in the "what am I missing" section
)


-> Conclusion:
-> If these packets never leave the box - no ARP will ever see them and
no
-> dynamic routing protocol will ever advertise them - therefore no IP
-> address collision. You can use _whatever_ address you want, private
-> public, IBMs, intels etc. Do we agree on this? In other words hack
not
-> needed here.

Wrong.  Packets need to leave each blade.  You cannot treat the blades
as a private entity.  You must ARP to find out the other blades MAC
address.

-> 
-> 2) The addresses for chasis-outside world communication. You have one
or
-> more dedicated gateways to connect between the outside of the chasis
to
-> inside.
-> There are many tricks you could use to somehow get the packets
to/from
-> the internal blades: NAT, forward, have aliases inside the chasis
which
-> get forwarded etc. Lets not discuss about how the the packets finaly
-> make it outside, rather just assume these packets make it outside the
-> chasis then lets explore using either 127.x or RFC1918 addresses.
-> 
-> a) using private addresses implies possibility of conflict of
addresses
-> within customer's  network. To quote Zdenek:
-> You couldn't walk in the NOC and tell them: "You can't use the 10.x
-> net to manage your equipment - my box is already using that net".
-> Conclusion:
-> You walk into the NOC and say "can i use 10.0.0.x/22 subnet" they say
"no
-> thats going to collide use 10.0.0.0/28"
-> Summary: You may need to go to your box and reconfigure its external
-> looking
-> addresses.
-> 

I _use_ to do exactly what you stated above.  When RFC 1918 first came
out I used the 10 net.  

1st bug:  Customer had the same 10.100.xx.xx/24 net that I had and my
inter-system communication wouldn't work, because all my routes got
screwed up.  (i.e the SNMP sub-agents couldn't talk with the master). 

1st response to bug:  Well can you use another network address range?
Customer response:  Hell no.

Solution to bug1:  Easy, let the user configure the mgmt network ip
address.
Customers answer to bug1 solution:  Get the hell out of here; you don't
do out-of-band mgmt.  Do you know what a security risk this is for me?
Blah blah blah....  Even though all inter-chassis communication was done
securely, I couldn't convince them. I had a customer boot me out of his
office and boot our company out **because** of my design.  Not a good
feeling.


-> a') Using 127.x addresses. You -> NOC "can i use 127.0.0.x/22 subnet"
-> they say either "sorry, our routers cant route 127.x" or "no Zdenek
-> was here before you, thats going to collide use 127.0.0.0/28"
-> 

This is __EXACTLY__ the behavior we want.  I want routers to drop those
packets.  My inter-chassis communication better NOT go through a router.

-> So tell me what i am missing!
-> 

Experience.  You are missing a big key factor.  The routing part of what
you are saying is sound.  The big thing you are not getting is how the
"applications", telnet, snmp, ssh, Linux-HA, etc.. will interact with
your system.  You do NOT want to rewrite those applications to have some
knowledge of your system.  They want to connect to an IP address and
that better work (off-the-shelf).  

Therefore,

As a kernel programmer, it's easier for me to make sure the 127.xx net
works and sends the 127.xx packets to the proper network.

In conclusion:

It seems that Zdenek and I have been down this road _many_ times.  I
have shipped over 10 different routers/chassis systems.  I speak from
experience and experience alone.  I don't claim to be the smartest
person in the world, but I know what works.

This post started with a simple question of "can I do this".  The answer
I believe has been posted a long time ago.  I am not about to change the
way that I do my inter-chassis communications until the IEEE or RFC
community give me an address change for inter-chassis communication.
(which I believe is coming down the road).

And would you please subscribe to the list so I don't have the cc the
world every time?

Thanks.

-stv


^ permalink raw reply	[flat|nested] 52+ messages in thread
* RE: Do you know the TCP stack? (127.x.x.x routing)
@ 2005-03-09 17:33 Steve Iribarne
  2005-03-09 19:40 ` jamal
  0 siblings, 1 reply; 52+ messages in thread
From: Steve Iribarne @ 2005-03-09 17:33 UTC (permalink / raw)
  To: hadi
  Cc: Henrik Nordstrom, Martin Mares, Zdenek Radouch, Eran Mann,
	Thomas Graf, Andi Kleen, netdev, linux-net


-> Your blades --> VLANX/SubnetX
->      --> [some L3 switch]

umm.. I have a L2 switch... not L3 switch.


-> Read what i wrote again and cross reference with the diagram. ARP is
-> only L2 switched. It would be wise to configure the blade IP
addresses
-> to be within the same subnet - in which case the only route you need
on
-> your blades is a link scope one and perhaps a default GW pointing to
-> your L3 device.
-> 

L2 device.

> 
-> Its just a matter of time before you say "oh, thats what i do now for
-> 127.x". This is the point i have been trying to make all along.
-> 
-> > -> So tell me what i am missing!
-> > ->
-> >
-> > Experience.
-> 
-> I think you are making some very big assumption ;-> Please dont go
this
-> path unless you wish to end this thread.
-> 

Ok.  I won't.


-> Btw, i do believe what you and Zdenek are trying to solve are _very_
-> different problems. He is trying to build a distributed router of
some
-> form; i.e his blades are infact line-cards where traffic comes in.
-> You on the other hand seem to have the blades doing computes (i.e
they
-> are not router line cards).

Nope.  I'm in a very similar situation.  I don't have "line-cards"
per-se.  Not like xDSL or the type of line cards I've worked on in the
past, but my boards **DO** have both mgmt traffic and network traffic
coming into them.  
I have signaling traffic, bearer traffic and network mgmt traffic.

Very much the same.  I have basically 5 VLANS setup.  4 of which get
tagged at the switch so that when the packets come inside the chassis, I
know how to handle them.  1 of the VLANS is for mgmt communication.
Like I said, it works great.  The 127.xx net I will NEVER need to talk
to outside of my chassis and when I do chassis to chassis redundancy I
use a different scheme.

So I will never run into the "gee, someone else is using the 127.xx net"
because as long as my applications know how to get to the 127.xx net the
traffic will be sent to the proper ports and get tagged with the proper
vlan ID.


I wish my setup was easy enough to just draw a quick picture.  But it
aint, sorry.


-> 
-> The point is this: Whatever you folks are doing, probably inherited
from
-> some other projects more than likely using some other OS is not
-> necessary in Linux. I respect your desire to use those addresses if
it
-> makes you comfortable - I just vehemently disagree it is needed.


Nope.  Wrote the project from scratch using Embedded Linux.

Again, if you can show me a way of doing this, I'm all ears, but so far,
you haven't shown me any other way around it.  Believe me.  I've tried
and tried to find another solution to this problem.  

And I can't emphasis enough that telling a customer, "Pick a network
address range that I can use" is NOT, I have to repeat, NOT a solution.
The will NEVER NEVER NEVER go for it.  Maybe your customers will but
mine wont.  My customers are RBOCs and the like.  Anyone dealing with
them knows what I am talking about.


-> So i hope you dont show up with the patch and ask for its inclusion.
-> 

Nope.  Never will.  I don't think the big boys (Alan Cox, Linus and the
like) would never let it happen because there is no RFC on it.  There
will be someday.  I do know that there are ideas floating around right
now that will soon become the beginnings of the RFC.  

-> cheers,
-> jamal

^ permalink raw reply	[flat|nested] 52+ messages in thread
* RE: Do you know the TCP stack? (127.x.x.x routing)
@ 2005-03-09 21:57 Steve Iribarne
  2005-03-10  0:11 ` jamal
  0 siblings, 1 reply; 52+ messages in thread
From: Steve Iribarne @ 2005-03-09 21:57 UTC (permalink / raw)
  To: hadi
  Cc: Henrik Nordstrom, Martin Mares, Zdenek Radouch, Eran Mann,
	Thomas Graf, Andi Kleen, netdev, linux-net

It's not the routing of the packet that gets screwed up.  It's the
applications that my "intra" communication use.  

I do this...

I have a redundant system.  So two Ethernet switches that go to either a
switch/hubbed/routed network.  Not controlled by me, but by my
customers.

So you have duplicate three wire coming into both ends of my chassis.

------ net 1 --------|                   | ---- net 1 ----
                     |                   | 
------- net 2 -------|  Chassis 21 slots | ----- net 2 ----
                     |                   |
------- net 3 -------|                   | ---- net 3 ----


all three of those "outside" nets get to me by either a bridge, router
or hub.

My 19 boards internal need to talk to each other ALONG with talking to
the outside world.

Boards in slot 1/21 are the switches.
so boards 2-20 are my linux blades that talk to each other.

The switch is configured to have the VLANS.  Management traffic I tag on
a VLAN.  So when my host controller or any of the other linux blades
need to do host communication, they talk to ip address 127.100.xx.xx
which is associated with a VLAN tagged interface.

Traffic being sent to the outside world is tagged as it comes in from
the outside world (so I know where it came from), and sent to the proper
board.  
L2 switching stuff.

Traffic that I send back out to the outside world is tagged when it
leaves one of my blades so the switch knows which port to send it back
out on.  (net 1, net 2 or net 3)


Ok.. that being said...

The _only_ way I can have normal applications (ie. ping, telnet, nfs) to
work and _guarantee_ not intra communication problems is if I use the
127.xx net.  

I'm not sure what you are not getting.  I'm not talking about basic
routing.  I'm talking about getting applications not to collide.

Let me give you and example.

If I were to use the 10.100.xx.xx network for example.  I have an snmp
master-agent/sub-agent configuration.  So I have a host controller with
the 10.100.0.1 address and my subagents are 10.100.0.<slotnum>

Everything works great, everyone is happy, until someone from the
outside world (say net 2) tries to telnet to me with a host address of
say 10.100.0.73.  Well, my host controller will route that packet onto
my private network.  So when I go to respond to the telnet request I
will tag it for my internal network because that is what the FIB routing
tells it to do.

There it is.  I'm not going to spend anymore time on this, and neither
should you.  Like I said, I've been doing this for a darn long time, and
I have, as yet, to see anyone who can make this problem just work.
Other than the way I did it.  (I along with many others)

have a happy day.

-stv
 

-> -----Original Message-----
-> From: jamal [mailto:hadi@cyberus.ca]
-> Sent: Wednesday, March 09, 2005 11:41 AM
-> To: Steve Iribarne
-> Cc: Henrik Nordstrom; Martin Mares; Zdenek Radouch; Eran Mann; Thomas
-> Graf; Andi Kleen; netdev@oss.sgi.com; linux-net@vger.kernel.org
-> Subject: RE: Do you know the TCP stack? (127.x.x.x routing)
-> 
-> On Wed, 2005-03-09 at 12:33, Steve Iribarne wrote:
-> > -> Your blades --> VLANX/SubnetX
-> > ->      --> [some L3 switch]
-> >
-> > umm.. I have a L2 switch... not L3 switch.
-> >
-> 
-> Lets go over this slowly so we can hopefully resolve why we dont see
eye
-> to eye. I am not sure why i am spending all this energy on this.
-> 
-> Lets get the diagrams better:
-> 
-> 1) your case:
->     Your blades <--> VLANX/SubnetX
->       <--> [some L2 switch]
->            <--> VLANY/SubnetY <--> outside world
-> 
-> You probably have redundancy etc in some ATCA||2.16 setup with links
-> going to
-> two internal switches - but lets also ignore that - just assume the
-> simple
-> switch for now for sake of clarity. You may also have many VLANs
in/out
-> like you
-> said "signaling traffic, bearer traffic  and network mgmt traffic",
but
-> the
-> two internal vs external interfaces  i showed above should suffice to
-> indicate
-> the general picture. Agreed?
-> 
-> To sumarize, for you to get to/from the outside world to your blades
you
-> go
-> via L2 switch with a "few" interfaces to the ouside world.
-> In your case the "internal" interface is the VLANX port(s) facing the
-> switch.
-> The "external" interface is the port(s) on VLANY facing the outside.
-> 
-> 2) Note this is slightly different from Zdenek, which is:
-> 
->   Outside <->one or more interfaces <->  [LinecardX]
<-->[swicth/fabric]
->   Outside <->one or more interfaces <->  [LinecardY]
<-->[swicth/fabric]
->   .
->   .
-> 
-> In other words each line card  has many interfaces that come into the
-> box.
-> It is not unusual to find 12-48 interface line cards.  The "switch"
aka
-> "fabric"
-> connects these line cards (and perhaps some  control plane blade(s)).
-> Typically such
-> a switch will not run IP but rather some other internal thing like
CSIX
-> or SPI-x etc.
-> 
-> In both setups, if you do run IP internally, it does make sense not
-> "leak" internal
-> traffic to the outside world with such addresses.
-> In both cases you both try hard (and i am sure succeed) to not leak
those
-> packets
-> out - In your case its a simple separation of collision domains. The
only
-> way you can
-> get from internal to external is if infact you have L2 connectivity
-> between the two
-> (since you said you dont have L3 switching in your chasis).
-> 
-> By making the 127.x routable in linux of all places - which is where
i
-> started
-> disagreeing, you introduce some challenges with hope that 127.x
obscurity
-> is
-> going to help.
-> 
-> To avoid confusion and have Zdenek respond when i am talking to you
or
-> viceversa
-> lets make the two as separate issues:
-> 
-> 1) In your case i saw no reason for you to use 127.x - you could have
-> achieved the
-> same with 10.x. Your internal packets will never leak out. You say
you
-> will have collisions
-> with customer; but then if i understood correctly you said these
internal
-> packets never
-> the box.  So my conclusion was you didnt need the hack.
-> 
-> 2)Zdenek's case
-> 
-> Just avoiding the leak is not good enough if the 127.x is routable
and
-> someone else
-> is using it and he has to route such packets. In such a case, even if
-> Zdenek  hides
-> the internal network at some point  he will have to route a  packet
-> coming into
-> linecardX, port A to linecardY, port B.
-> And for this reason he cant totaly avoid collision. This is why i
called
-> it survival
-> via obscurity.
-> 
-> Note: I am not questioning his technique but i would never use it
-> myself. Lets say we can achieve the same goal in a different way.
-> 
-> > Again, if you can show me a way of doing this, I'm all ears, but so
-> far,
-> > you haven't shown me any other way around it.  Believe me.  I've
tried
-> > and tried to find another solution to this problem.
-> >
-> 
-> Lets talk about this when we are clear what the problem is. Fix up
the
-> diagram above if it is wrong, then we can talk.
-> 
-> cheers,
-> jamal

^ permalink raw reply	[flat|nested] 52+ messages in thread
* Re: Do you know the TCP stack? (127.x.x.x routing)
@ 2005-03-09 23:51 Boian Bonev
  2005-03-10  0:23 ` Jason Lunz
  0 siblings, 1 reply; 52+ messages in thread
From: Boian Bonev @ 2005-03-09 23:51 UTC (permalink / raw)
  To: Jason Lunz, netdev; +Cc: linux-net

[-- Attachment #1: Type: text/plain, Size: 1913 bytes --]

> The question, though, is: *How* do you configure the nodes within the
> chassis such that the internal IPs (whatever they are) _stay_ internal,
> and any non-127/8 addressing can be used for the external interfaces?
> 
> I've done something similar, for example, using policy routing and the
> arp sysctls. Suppose you have a machine with 2 interfaces, and you want
> IP routing to happen on each of the two interfaces as independently as
> possible. My solution involves using the "iif" modifier in your routing
> rules ("ip rule" rules) to send packets to two completely different
> routing tables, and making sure arp doesn't bleed across the two
> interfaces. I don't know whether policy routing gives enough control to
> do this in a general fashion; i did it only for very specific types of
> traffic. But I suspect you could come up with something workable.

you can do that but you omit the interface addresses - suppose ext net is 10.20.10.1/24,
internal is 10.10.10.1/24, no matter what routing policies and rules you put, both interface
ips will be visible from both interfaces. now imagine you have another external net 10.30.10.1/24
and customer wants to route e.g. 10.10.0.0/16 from 10.20.10.1/24 via 10.30.10.5...
at least host 10.10.10.1 will not route but arrive locally to your blade host

btw. i have seen recently on iptables' patch-o-matic some module that could by condition route
traffic to local addresses to another host. anyway the whole thing is doable with any kind of 
addresses but just imagine what nightmare startup ruleset you will have on each box; then
modify your custom rules to conform that hell ruleset and... imho it will be much more easy to
create a custom transport over ethernet (in case your ext network could share addresses form
that protocol also) and forget about ipv4 for internal implementation. thus you'd have at least
better security between worlds ;-)

b.

^ permalink raw reply	[flat|nested] 52+ messages in thread
* RE: Do you know the TCP stack? (127.x.x.x routing)
@ 2005-03-10 14:35 Steve Iribarne
  2005-03-10 14:49 ` Dmitry Torokhov
  0 siblings, 1 reply; 52+ messages in thread
From: Steve Iribarne @ 2005-03-10 14:35 UTC (permalink / raw)
  To: Catalin(ux aka Dino) BOIE
  Cc: hadi, Henrik Nordstrom, Martin Mares, Zdenek Radouch, Eran Mann,
	Thomas Graf, Andi Kleen, netdev, linux-net

-> You say that a client will not allow you to use net 10.
-> OK, but, the same client would not allow you to use 127/8 because
they
-> use
-> it!
-> What I'm saying is that 10.0.0.0/8 and 127.0.0.0/8 are the same. The
-> customer can use them.
-> You assume that the client will not use 127/8. Why? This is wrong.
-> You can use it, the client can use it.
-> 

No.  If the client uses 127/8 on a linux box, it is just a loopback and
will never go out on the wire and the applications (i.e. telnet, ftp,
ping, whatever) will just loopback.


^ permalink raw reply	[flat|nested] 52+ messages in thread
* RE: Do you know the TCP stack? (127.x.x.x routing)
@ 2005-03-10 15:04 Steve Iribarne
  2005-03-10 15:25 ` Catalin(ux aka Dino) BOIE
  0 siblings, 1 reply; 52+ messages in thread
From: Steve Iribarne @ 2005-03-10 15:04 UTC (permalink / raw)
  To: dtor_core
  Cc: Catalin(ux aka Dino) BOIE, hadi, Henrik Nordstrom, Martin Mares,
	Zdenek Radouch, Eran Mann, Thomas Graf, Andi Kleen, netdev,
	linux-net


-> This assumes that the client did not apply the same hack you did.
-> 

They wouldn't do it on the same machine or system. I control my internal
system.  It's an embedded system which means I have complete control.
If they applied it somewhere else on the network, I wouldn't receive any
of those packets because the router would drop it and my patch drops the
127.xxx rcv'd packets if not received on the proper VLAN and my system
doesn't accept tagged packets from the outside world.

I don't think I'm the only one who has done this.  As a matter of fact,
I KNOW I'm not.  When I worked on BSD years ago (10+), I worked for a
company who did the same sort of hack.

-stv

^ permalink raw reply	[flat|nested] 52+ messages in thread
end of thread, other threads:[~2005-03-10 15:25 UTC | newest]

Thread overview: 52+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2005-03-06  2:20 Do you know the TCP stack? (127.x.x.x routing) Zdenek Radouch
2005-03-06  9:56 ` Martin Mares
2005-03-06 17:01   ` Zdenek Radouch
2005-03-06 17:12     ` alex
2005-03-06 17:31     ` Thomas Graf
2005-03-06 19:48       ` Zdenek Radouch
2005-03-06 20:19         ` alex
2005-03-06 20:19         ` Andi Kleen
2005-03-06 20:45           ` Thomas Graf
2005-03-06 21:30             ` Andi Kleen
2005-03-06 21:50               ` Thomas Graf
2005-03-06 21:50             ` Zdenek Radouch
2005-03-07  7:01               ` Sumit Pandya
2005-03-07  8:05               ` Eran Mann
2005-03-07 12:14                 ` jamal
2005-03-07 23:50                 ` jamal
2005-03-08  3:15                   ` Zdenek Radouch
2005-03-08 13:34                     ` jamal
2005-03-08 13:51                       ` Martin Mares
2005-03-08 13:58                         ` jamal
2005-03-08 14:03                           ` Martin Mares
2005-03-08 14:17                             ` jamal
2005-03-08 14:20                               ` Martin Mares
2005-03-08 18:40                               ` Henrik Nordstrom
2005-03-08 21:17                                 ` jamal
2005-03-09  9:09                                   ` Henrik Nordstrom
2005-03-09 12:39                                     ` jamal
2005-03-09 13:39                                       ` Zdenek Radouch
2005-03-09 14:18                                         ` jamal
2005-03-09 16:46                                           ` Jason Lunz
2005-03-10 10:10                                             ` Henrik Nordstrom
2005-03-09 17:52                                           ` Matt Mackall
2005-03-10  6:57                                             ` Catalin(ux aka Dino) BOIE
2005-03-09 22:34                                       ` Henrik Nordstrom
2005-03-10  1:47                                         ` Jamie Lokier
2005-03-08 18:34                       ` Henrik Nordstrom
2005-03-09  5:33                       ` Zdenek Radouch
2005-03-08 14:02                     ` Thomas Graf
  -- strict thread matches above, loose matches on Subject: below --
2005-03-08 15:07 Steve Iribarne
2005-03-09 15:01 Steve Iribarne
2005-03-09 16:00 ` jamal
2005-03-10  6:48 ` Catalin(ux aka Dino) BOIE
2005-03-09 17:33 Steve Iribarne
2005-03-09 19:40 ` jamal
2005-03-09 21:57 Steve Iribarne
2005-03-10  0:11 ` jamal
2005-03-09 23:51 Boian Bonev
2005-03-10  0:23 ` Jason Lunz
2005-03-10 14:35 Steve Iribarne
2005-03-10 14:49 ` Dmitry Torokhov
2005-03-10 15:04 Steve Iribarne
2005-03-10 15:25 ` Catalin(ux aka Dino) BOIE

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).