strange kernel message when hacking the NIC driver

strange kernel message when hacking the NIC driver

[Pei Zheng]

Hi, 

I've done some simple hackings to the network device driver. Basically
i want to connect two NICs as a point-to-point link,ie, packets sending
out from one NIC will be redirected to another NIC, sort of direct-link
cable between these two NICs.
So in the drivers of these two NICs, before hart_start_xmit(), the skb's 
header will be forcibly set to its peer's MAC address, thus each packey
sending out from one NIC will be directly diverted to its peer NIC. This 
works fine most of the case since tcpdump shows that packets traverse in
this way. However, i found errors in system log:

Jan 10 18:02:40 em2 kernel: 1130: 09 => cb
Jan 10 18:02:40 em2 kernel: 388: f4 => 45
Jan 10 18:02:40 em2 kernel: 26: c0 => 7d
Jan 10 18:02:40 em2 kernel: 149: 1f => 98
Jan 10 18:02:40 em2 kernel: After error applied:
Jan 10 18:02:40 em2 kernel: 00 80 c8 b9 6b b6 00 80 c8 b9 6b b6 08 00
45 00
Jan 10 18:02:40 em2 kernel: 05 dc 73 cd 20 b9 3f 11 44 26 7d a8 0c 0a
c0 a8
Jan 10 18:02:40 em2 kernel: 10 0a e3 c1 4c 1f 6f 83 61 dc fc 5e 5c f4
66 17
Jan 10 18:02:40 em2 kernel: 2c 73 19 ce cc 2d 69 69 bd 43 33 6d 82 de
4a 87
Jan 10 18:02:40 em2 kernel: 8d 9d 81 f6 2e b2 8c 6a d5 e4 f2 e6 bc ab
5a 01
Jan 10 18:02:40 em2 kernel: 34 d5 39 f8 d9 5b 16 bc dc 95 bd b4 08 a9
5e 11
Jan 10 18:02:40 em2 kernel: df 38 80 8a ca d8 1c 53 65 97 91 4c 84 5a
a1 0e
Jan 10 18:02:40 em2 kernel: c2 5f d4 02 a1 9e 1c 35 d4 95 08 44 81 16
a3 e0

there are many this kind of messages. Don't understand what it is. the
beginning part of the data seems to be an icmp packet(00 80 c8 b9 6b
b6 is the MAC of one NIC). For some reason kernel thinks that this
packet is not correct. Any idea about this? If i want to modify a
packet's header before it goes to hard_strat_xmit(), what else should
i do except setting the skb header to the MAC of the NIC's peer only? 
checksum stuff?

Any helps will be highly appreciated.

-Pei

Re: strange kernel message when hacking the NIC driver

[Timothy Covell]

On Thursday 10 January 2002 22:19, Pei Zheng wrote:
> Hi,
>
> I've done some simple hackings to the network device driver. Basically
> i want to connect two NICs as a point-to-point link,ie, packets sending
> out from one NIC will be redirected to another NIC, sort of direct-link
> cable between these two NICs.
> So in the drivers of these two NICs, before hart_start_xmit(), the skb's
> header will be forcibly set to its peer's MAC address, thus each packey
> sending out from one NIC will be directly diverted to its peer NIC. This
> works fine most of the case since tcpdump shows that packets traverse in
> this way. However, i found errors in system log:
>
> Jan 10 18:02:40 em2 kernel: 1130: 09 => cb
> Jan 10 18:02:40 em2 kernel: 388: f4 => 45
> Jan 10 18:02:40 em2 kernel: 26: c0 => 7d
> Jan 10 18:02:40 em2 kernel: 149: 1f => 98
> Jan 10 18:02:40 em2 kernel: After error applied:
> Jan 10 18:02:40 em2 kernel: 00 80 c8 b9 6b b6 00 80 c8 b9 6b b6 08 00
> 45 00
> Jan 10 18:02:40 em2 kernel: 05 dc 73 cd 20 b9 3f 11 44 26 7d a8 0c 0a
> c0 a8
> Jan 10 18:02:40 em2 kernel: 10 0a e3 c1 4c 1f 6f 83 61 dc fc 5e 5c f4
> 66 17
> Jan 10 18:02:40 em2 kernel: 2c 73 19 ce cc 2d 69 69 bd 43 33 6d 82 de
> 4a 87
> Jan 10 18:02:40 em2 kernel: 8d 9d 81 f6 2e b2 8c 6a d5 e4 f2 e6 bc ab
> 5a 01
> Jan 10 18:02:40 em2 kernel: 34 d5 39 f8 d9 5b 16 bc dc 95 bd b4 08 a9
> 5e 11
> Jan 10 18:02:40 em2 kernel: df 38 80 8a ca d8 1c 53 65 97 91 4c 84 5a
> a1 0e
> Jan 10 18:02:40 em2 kernel: c2 5f d4 02 a1 9e 1c 35 d4 95 08 44 81 16
> a3 e0
>
> there are many this kind of messages. Don't understand what it is. the
> beginning part of the data seems to be an icmp packet(00 80 c8 b9 6b
> b6 is the MAC of one NIC). For some reason kernel thinks that this
> packet is not correct. Any idea about this? If i want to modify a
> packet's header before it goes to hard_strat_xmit(), what else should
> i do except setting the skb header to the MAC of the NIC's peer only?
> checksum stuff?
>
> Any helps will be highly appreciated.
>
> -Pei


You have me confused about whether your trying to build some kind of
ethernet bridge or some kind of secret packet sniffer or some other
such thing.   It looks like you're mixing up TCP/IP and Ethernet.

Ethernet is the link level protocol that addresses NICs by the MAC
address.    There can only be one NIC on the network with the same
MAC address.   TCP/IP uses IP addresses, not MAC addresses.  And you
cannot have two IP addresses on the same link if they have the same
network and broadcast address unless you do multipathing.  


I guess that I don't understand what you are trying to achieve.



-- 
timothy.covell@ashavan.org.

Re: strange kernel message when hacking the NIC driver

[Timothy Covell]

On Thursday 10 January 2002 23:20, Timothy Covell wrote:


Let me clarify what I said earlier.  You cannot have
identical MAC addresses on two different NICs.   Indeed,
it is impossible w/o trying to fool the kernel into
redefining the NICs hardware based MAC address. 

As concerns TCP/IP, you can define two NICs to have the
same IP address, but you will only end up confusing
your switch/HUB router which assumes a 1 to 1 mapping
of MACs to IPs.   The whole point of ICMP is to
discover this mapping via ARP requests.

If you want to increase your aggregate bandwidth, then
there are several methods.  One is to use the bonding 
driver to make the NICs look like one but it needs to 
be supported by the switch that you use or for point-to-point 
connections, the other box. Another way is to use something 
like OSPF or advanced iptable routing to effect multipathing.  
Finally, a crude way to increase your effective bandwidth is 
to run several instances of your server(s) on different IP 
addresses/NICS but they cannot all have default routes on 
Linux without resorting to OSPF and/or iptables.


-- 
timothy.covell@ashavan.org.

Re: strange kernel message when hacking the NIC driver

[David S. Miller]

   From: Timothy Covell <timothy.covell@ashavan.org>
   Date: Fri, 11 Jan 2002 05:55:20 -0600

   Let me clarify what I said earlier.  You cannot have
   identical MAC addresses on two different NICs.

There is nothing illegal about that at all.  As long at
the NICs live on different subnets, it is perfectly fine.
In fact this is pretty common on Sun machines.

Re: strange kernel message when hacking the NIC driver

[Timothy Covell]

On Friday 11 January 2002 06:07, David S. Miller wrote:
>    From: Timothy Covell <timothy.covell@ashavan.org>
>    Date: Fri, 11 Jan 2002 05:55:20 -0600
>
>    Let me clarify what I said earlier.  You cannot have
>    identical MAC addresses on two different NICs.
>
> There is nothing illegal about that at all.  As long at
> the NICs live on different subnets, it is perfectly fine.
> In fact this is pretty common on Sun machines.

True.  I was assuming that the context of the post was
that the NICs were on the same network link.    

Solaris _defaults_ to using the MAC address from the 
primary (hostname) NIC for the rest of them.   IMHO, this 
is a really stupid thing to do, and  I disable it tout de suite
when given a choice.   Of course, if you like it, then
why don't you try to convince Linus to change his mind 
about it?

-- 
Surah II 120, Surah II 191-193, Surah V 45, Surah V 51
Surah VIII 12-18, Surah VIII 39-40, Surah VIII 65-69, etc.
--
timothy.covell@ashavan.org.

Re: strange kernel message when hacking the NIC driver

[David S. Miller]

   From: Timothy Covell <timothy.covell@ashavan.org>
   Date: Fri, 11 Jan 2002 06:20:42 -0600
   
   True.  I was assuming that the context of the post was
   that the NICs were on the same network link.    
   
I didn't catch that bit.

   Solaris _defaults_ to using the MAC address from the 
   primary (hostname) NIC for the rest of them. 

This has nothing to do with Solaris, it has to do with
open firmware variable settings.  If there is a 'local-mac-address'
property in the device node for the ethernet card, that is what
the drivers use.  Otherwise they use the "host MAC".

If you look, this is what we do under Linux on Sparc too.

Re: strange kernel message when hacking the NIC driver

[Prof. Brand]

Timothy Covell <timothy.covell@ashavan.org> said:
> On Friday 11 January 2002 06:07, David S. Miller wrote:
> >    From: Timothy Covell <timothy.covell@ashavan.org>
> >    Date: Fri, 11 Jan 2002 05:55:20 -0600
> >
> >    Let me clarify what I said earlier.  You cannot have
> >    identical MAC addresses on two different NICs.
> >
> > There is nothing illegal about that at all.  As long at
> > the NICs live on different subnets, it is perfectly fine.
> > In fact this is pretty common on Sun machines.
> 
> True.  I was assuming that the context of the post was
> that the NICs were on the same network link.

This is not the typical setup...

> Solaris _defaults_ to using the MAC address from the 
> primary (hostname) NIC for the rest of them.

Sun sets the MAC from the machine ID on all network interfaces by
default. Not Solaris, AFAIU it is done by the PROM.

>                                              IMHO, this 
> is a really stupid thing to do, and  I disable it tout de suite
> when given a choice.   Of course, if you like it, then
> why don't you try to convince Linus to change his mind 
> about it?

Why should DaveM convince Linus to get Sun to change their mind on NIC
setup?

Especially if it works just fine for 99.95% of Suns, and has the bonus that
you can track each machine by a _single_ MAC, even if you change NICs or
add more?
-- 
Horst von Brand			     http://counter.li.org # 22616

Re: strange kernel message when hacking the NIC driver

[Timothy Covell]

On Saturday 12 January 2002 15:37, Prof. Brand  wrote:
> Timothy Covell <timothy.covell@ashavan.org> said:
> > On Friday 11 January 2002 06:07, David S. Miller wrote:
> > >    From: Timothy Covell <timothy.covell@ashavan.org>
> > >    Date: Fri, 11 Jan 2002 05:55:20 -0600
> > >
> > >    Let me clarify what I said earlier.  You cannot have
> > >    identical MAC addresses on two different NICs.
> > >
> > > There is nothing illegal about that at all.  As long at
> > > the NICs live on different subnets, it is perfectly fine.
> > > In fact this is pretty common on Sun machines.
> >
> > True.  I was assuming that the context of the post was
> > that the NICs were on the same network link.
>
> This is not the typical setup...
>
> > Solaris _defaults_ to using the MAC address from the
> > primary (hostname) NIC for the rest of them.
>
> Sun sets the MAC from the machine ID on all network interfaces by
> default. Not Solaris, AFAIU it is done by the PROM.
>
> >                                              IMHO, this
> > is a really stupid thing to do, and  I disable it tout de suite
> > when given a choice.   Of course, if you like it, then
> > why don't you try to convince Linus to change his mind
> > about it?
>
> Why should DaveM convince Linus to get Sun to change their mind on NIC
> setup?

You have it backwards.  If you all seem to like the Sun way of doing things,
then Linus should change Linux to do likewise.

>
> Especially if it works just fine for 99.95% of Suns, and has the bonus that
> you can track each machine by a _single_ MAC, even if you change NICs or
> add more?

If you are already keeping a MAC <--> Host table, then it's not very hard to
expand it to keep track of other NICs.   That is, of course, if you care about
tracking your inventory.   

-- 
timothy.covell@ashavan.org.

Re: strange kernel message when hacking the NIC driver

[Alan Cox]

> Let me clarify what I said earlier.  You cannot have
> identical MAC addresses on two different NICs.   Indeed,
> it is impossible w/o trying to fool the kernel into
> redefining the NICs hardware based MAC address. 

Wrong

A mac address is per system. Now in fact almost all systems do it per ethernet
card but that is not what the specifications guarantee. There are machines
out there and cards out there which use the same MAC on all interfaces.

Re: strange kernel message when hacking the NIC driver

[Timothy Covell]

On Friday 11 January 2002 07:11, Alan Cox wrote:
> > Let me clarify what I said earlier.  You cannot have
> > identical MAC addresses on two different NICs.   Indeed,
> > it is impossible w/o trying to fool the kernel into
> > redefining the NICs hardware based MAC address.
>
> Wrong
>
> A mac address is per system. Now in fact almost all systems do it per
> ethernet card but that is not what the specifications guarantee. There are
> machines out there and cards out there which use the same MAC on all
> interfaces. -

IMHO, they _should_ be unique except for multicast addressing and uses
such as in HSRP/VRRP and such.   Network admins usually like to have
a firm grip concerning how traffic is routed.  They don't want traffic on
one segment/subnet getting routed to another.  IHMO, this is a vector
for viruses proliferation because the host vulnerable thinks that all
segments/subnets are the same.


I don't have the money to shell out for a copy of the IEEE 802.x standards, 
but I can quote some other folks on this:


RFC 826:

However, 48.bit Ethernet addresses are supposed to be unique and fixed for 
all time, so they shouldn't
change.


>From the Ethernet FAQ:

02.10Q: What is a MAC address?
A: It is  the unique hexadecimal serial number assigned to each Ether-
net  network device to identify it  on  the network.  With Ethernet
devices  (as  with  most other  network  types),  this  address  is
permanently set at  the time of manufacturer, though it can usually
be changed  through  software (though this is  generally a Very
Bad Thing to do).

02.11Q: Why must the MAC address to be unique?

A: Each card  has  a  unique MAC address,  so  that
it will be able to exclusively  grab  packets  off the wire
meant  for  it.   If  MAC addresses are not unique,  there is
no  way  to distinguish between two  stations.  Devices on the
network  watch  network traffic and look for their own MAC address
in each packet to determine whether they should decode  it or  not.
Special circumstances exist  for broadcasting to every device.

04.01Q: What is a "segment"?
A: A  piece of network wire bounded by bridges,  routers, repeaters or
terminators.

04.02Q: What is a "subnet"?
A: Another overloaded  term. It can  mean, depending on  the usage,  a
segment, a set of machines  grouped together by a specific protocol
feature  (note  that  these machines do not have to be on  the same
segment, but they could be) or a big  nylon  thing used  to capture
enemy subs.







-- 
timothy.covell@ashavan.org.

Re: strange kernel message when hacking the NIC driver

[Paul Walmsley]

On Fri, 11 Jan 2002, Timothy Covell wrote:

> I don't have the money to shell out for a copy of the IEEE 802.x standards,

The IEEE 802 standards can be downloaded for free from the IEEE.
<http://standards.ieee.org/getieee802/>


- Paul

Re: strange kernel message when hacking the NIC driver

[Timothy Covell]

On Friday 11 January 2002 13:02, Timothy Covell wrote:
> On Friday 11 January 2002 07:11, Alan Cox wrote:
> > > Let me clarify what I said earlier.  You cannot have
> > > identical MAC addresses on two different NICs.   Indeed,
> > > it is impossible w/o trying to fool the kernel into
> > > redefining the NICs hardware based MAC address.
> >
> > Wrong
> >
> > A mac address is per system. Now in fact almost all systems do it per
> > ethernet card but that is not what the specifications guarantee. There
> > are machines out there and cards out there which use the same MAC on all
> > interfaces. -
>
> IMHO, they _should_ be unique except for multicast addressing and uses
> such as in HSRP/VRRP and such.   Network admins usually like to have
> a firm grip concerning how traffic is routed.  They don't want traffic on
> one segment/subnet getting routed to another.  IHMO, this is a vector
> for viruses proliferation because the host vulnerable thinks that all
> segments/subnets are the same.
>
>

OK, here are some snippets which I found from reading the IEEE 802.x
standards.   As far as I can see, the only justification for your and Sun's
(OpenBootProm) method is to assume that all ports will be aggregated.

IEEE 802:
---------
However, due to the shared-medium nature of the IEEE 802 LANs, there is 
always a MAC Sublayer.


IEEE 802:
---------

5.1 Organizationally Unique Identifier 
5.1.1 Concept. 

Organizationally Unique Identifiers allow a general means of assuring unique 
identifiers for a 
number of purposes. Currently, the IEEE assigns Organizationally Unique 
Identifiers to be used 
for generating LAN MAC addresses and protocol identifiers. Assuming correct 
administration by 
the assignee, the LAN MAC addresses and protocol identifiers will be 
universally unique.



IEEE 802.1D
-----------

a) A Bridge is not directly addressed by communicating end stations, except 
as an end station for
management purposes: frames transmitted between end stations carry the MAC 
Address of the peer-end
station in their Destination Address field, not the MAC Address, if any, of 
the Bridge. 

b) All MAC Addresses must be unique within the Bridged LAN. 

c) The MAC Addresses of end stations are not restricted by the topology and 
configuration of the Bridged LAN.


IEEE 802.1G 
-------------

Preservation of the MAC service The MAC service offered by a remotely bridged 
LAN is 
similar (see 5.3) to that offered by a single LAN. In consequence 

a) A bridge is not directly addressed by communicating end stations, except
as an end station for management purposes: frames transmitted between end 
stations carry the MAC address of the peer end station in their Destination 
Address fields, 
not the MAC address, if any, of a bridge. 

b) All MAC addresses are unique and addressable within the bridged LAN. 

c) The MAC addresses of end stations are not restricted by the topology and 
configuration of the bridged LAN.


IEEE 802.1Q
----------------
B.1.2 Duplicate MAC Addresses 

The simplest example of a need for Independent VLAN Learning occurs where
two (or more) distinct devices in different parts of the network reuse 
the same individual MAC Address, or where a single device is connected to 
multiple LAN segments, and all of its LAN interfaces use the same individual 
MAC Address. This is shown in Figure B-3.


[picture ommitted for obvious reasons]

The example shows two clients with access to the same server; both clients 
are using the same individual MAC Address, X. If the Bridge shares learning 
between VLAN Red (which serves Client A) and VLAN Blue (which serves Client 
B), 
i.e., the Bridge uses the same FID for both VLANs, then Address X will appear 
to 
move between Ports 1 and 2 of the Bridge, depending upon which client has 
most recently 
trans-mitted a frame. Communication between these Clients and the server will 
therefore 
be seriously disrupted. Assignment of distinct FIDs for Red and Blue ensures 
that 
communication can take place correctly.




IEEE 802.3
----------  
The only thing which I could find that supported you and Sun's
position is in using LACP.  Then, one uses the SystemID 
(the hostid from eeprom, or one of the port's MAC).  So, if you are
assuming that all ports should belong to an aggregate group by
default, then I could understand this point.  However, the vast
majority of system's NICs (aside from the QFE cards) are never
aggregated.  So, it's a bit of a stretch to using LACP as a default.

Quotations:


i)Each port is assigned a unique,globally administered MAC address.This MAC 
address 
is used as the source address for frame exchanges that are initiated by 
entities within 
the Link Aggregation sublayer itself (i.e.,LACP and Marker protocol 
exchanges). NOTE  
The LACP and Marker protocols use a multicast destination address for all 
exchanges,
and do not impose any requirement for a port to recognize more than one 
unicast address 
on received frames. 

j)Each Aggregator is assigned a unique,globally administered MAC address;this 
address is 
used as the MAC address of the aggregation from the perspective of the MAC 
Client, both as 
a source address for transmitted frames and as the destination address for 
received frames.
The MAC address of the Aggregator may be one of the MAC addresses of a port 
in the associated 
Link Aggregation Group (see 43.2.10).



43.2.8 Aggregator An Aggregator comprises an instance of a Frame Collection 
function,
an instance of a Frame Distribution function and one or more instances of the 
Aggregator 
Parser/Multiplexer function for a Link Aggregation Group.A single Aggregator 
is associated 
with each Link Aggregation Group.  An Aggregator offers a standard IEEE 802.3 
MAC service 
interface to its associated MAC Client;access to the MAC service by a MAC 
Client is always 
achieved via an Aggregator. An Aggregator can therefore be considered to be a 
logical MAC ,
bound to one or more ports,through which the MAC client is provided access to 
the MAC service. 
A single,individual MAC address is associated with each Aggregator (see 
43.2.10). An Aggregator is 
available for use by the MAC Client if the following are all true: 

a)It has one or more attached ports. 
b)The Aggregator has not been set to a disabled state by administrative 
action (see 30.7.1.1.13).
c)The collection and/or distribution function associated with one or more of 
the attached ports is 
enabled (see 30.7.1.1.14). 

NOTE  To simplify the modeling and description of the operation of Link 
Aggregation,it is assumed that 
there are as many Aggregators as there are ports in a given System; however, 
this is not a requirement 
of this standard.Aggregation of two or more ports consists of changing the 
bindings between ports and 
Aggregators such that more than one port is bound to a single Aggregator.The 
creation of any aggregations 
of two or more links will therefore result in one or more Aggregators that 
are bound to more than one port,
and one or more Aggregators that are not bound to any port.An Aggregator that 
is not bound to any port appears 
to a MAC Client as a MAC interface to an inactive port.During times when the 
bindings between ports and Aggregators a
re changing,or as a consequence of particular con  guration choices, there 
may be occasions when one or more ports 
are not bound to any Aggregator.

43.2.10 Addressing Each IEEE 802.3 MAC has an associated globally-unique 
individual MAC address,whether that MAC 
is used for Link Aggregation or not (see Clause 4). Each Aggregator to which 
one or more ports are attached has 
an associated globally-unique individual MAC address (see 43.3.3).The MAC 
address of the Aggregator may be the 
globally-unique individual MAC addresses of one of the MACs in the associated 
Link Aggregation Group,or it may 
be a distinct MAC address.The manner in which such addresses are chosen is 
not otherwise constrained by this standard. 
Protocol entities sourcing frames from within the Link Aggregation sublayer 
(e.g.,LACP and the Marker protocol)use the 
MAC address of the MAC within an underlying port as the source address in 
frames transmitted through that port.The MAC 
Client sees only the Aggregator and not the underlying MACs,and therefore 
uses the Aggregator  s MAC address as the source 
address in transmitted frames.If a MAC Client submits a frame to the 
Aggregator for transmission without specifying a 
source address,the Aggregator inserts its own MAC address as the source 
address for transmitted frames.



RFC 826:
--------

However, 48.bit Ethernet addresses are supposed to be unique and fixed for 
all time, so they shouldn't
change.


>From the Ethernet FAQ:
----------------------

02.10Q: What is a MAC address?
A: It is  the unique hexadecimal serial number assigned to each Ether-
net  network device to identify it  on  the network.  With Ethernet
devices  (as  with  most other  network  types),  this  address  is
permanently set at  the time of manufacturer, though it can usually
be changed  through  software (though this is  generally a Very
Bad Thing to do). 

02.11Q: Why must the MAC address to be unique?  

A: Each card  has  a  unique MAC address,  so  that
it will be able to exclusively  grab  packets  off the wire
meant  for  it.   If  MAC addresses are not unique,  there is 
no  way  to distinguish between two  stations.  Devices on the 
network  watch  network traffic and look for their own MAC address 
in each packet to determine whether they should decode  it or  not.
Special circumstances exist  for broadcasting to every device.

04.01Q: What is a "segment"?
A: A  piece of network wire bounded by bridges,  routers, repeaters or
terminators. 

04.02Q: What is a "subnet"?
A: Another overloaded  term. It can  mean, depending on  the usage,  a
segment, a set of machines  grouped together by a specific protocol
feature  (note  that  these machines do not have to be on  the same
segment, but they could be) or a big  nylon  thing used  to capture 
enemy subs.


-- 
timothy.covell@ashavan.org.

Re: strange kernel message when hacking the NIC driver

[Prof. Brand]

Timothy Covell <timothy.covell@ashavan.org> said:
> On Thursday 10 January 2002 23:20, Timothy Covell wrote:

> Let me clarify what I said earlier.  You cannot have
> identical MAC addresses on two different NICs.

You sure can. Look at your nearest Sun machine with two NICs for an
example.

>                                                Indeed,
> it is impossible w/o trying to fool the kernel into
> redefining the NICs hardware based MAC address. 

It is not "fooling", you can set the MAC address on some cards by software.
Kernel has nothing to do with it.

> As concerns TCP/IP, you can define two NICs to have the
> same IP address, but you will only end up confusing
> your switch/HUB router which assumes a 1 to 1 mapping
> of MACs to IPs.   The whole point of ICMP is to
> discover this mapping via ARP requests.

TCP/IP is a 4(5) level protocol stack:

0 Hardware [Cribbed from ISO]
1 Device driver
2 IP [Includes ICMP]
3 TCP and UDP
4 Applications

A switch/hub works on Ethernet frames, i.e., at the hardware level. It has
absolutely no idea of mappings of MACs to IPs.

ICMP is part of the IP layer, the whole of the mapping of IPs to MACs is
(conceptually at least) part of the device driver layer. This includes ARP,
DHCP et al. ICMP has nothing to do with ARP.
-- 
Horst von Brand			     http://counter.li.org # 22616

Re: strange kernel message when hacking the NIC driver

[Timothy Covell]

On Saturday 12 January 2002 15:37, Prof. Brand  wrote:
> Timothy Covell <timothy.covell@ashavan.org> said:
> > On Thursday 10 January 2002 23:20, Timothy Covell wrote:
> >
> > Let me clarify what I said earlier.  You cannot have
> > identical MAC addresses on two different NICs.
>
> You sure can. Look at your nearest Sun machine with two NICs for an
> example.

And why don't you connect those two NICs with the same MACs to
the same HUB and see how well that works.


>
> >                                                Indeed,
> > it is impossible w/o trying to fool the kernel into
> > redefining the NICs hardware based MAC address.
>
> It is not "fooling", you can set the MAC address on some cards by software.
> Kernel has nothing to do with it.
>
> > As concerns TCP/IP, you can define two NICs to have the
> > same IP address, but you will only end up confusing
> > your switch/HUB router which assumes a 1 to 1 mapping
> > of MACs to IPs.   The whole point of ICMP is to
> > discover this mapping via ARP requests.
>
> TCP/IP is a 4(5) level protocol stack:
>
> 0 Hardware [Cribbed from ISO]
> 1 Device driver
> 2 IP [Includes ICMP]
> 3 TCP and UDP
> 4 Applications
>
> A switch/hub works on Ethernet frames, i.e., at the hardware level. It has
> absolutely no idea of mappings of MACs to IPs.
>
> ICMP is part of the IP layer, the whole of the mapping of IPs to MACs is
> (conceptually at least) part of the device driver layer. This includes ARP,
> DHCP et al. ICMP has nothing to do with ARP.


If you have two NICs with the identical MAC address on the same segment,
your ethernet routing will become confused.  The ARP table keeps a map
of MAC to IP address.   This table assumes a 1 to 1 relationship.   If you
try to reach a host but fail due to routing issues, then ICMP sends out 
DESTINATION HOST UNKNOWN/HOST UNREACHABLE messages.  
So, ICMP and very much to do with ARP.

The whole ISO Model is flawed because there really is not
a clean delineation among the layers.   For example, BGP uses TCP
to route IP, so it's an ISO layer 3 process, but it makes use of TCP
which is layer 4.  So, you need layer 4 to define layer 3.   Crazy huh.
The same argument can be shifted to the Internet Model, which is
IHMO, an after-the-fact idea that came from trying to apply ISO to
Internet.


-- 
timothy.covell@ashavan.org.

RE: strange kernel message when hacking the NIC driver

[Pei Zheng]

Thank you for your reply. Sorry for bring confusion.
Let me clarify what I am trying to do. I am now using same MAC for two NICs.
Each NIC has a unique MAC in the context of my work.

To put it simple, routing simulation,ie.
to simulate(or emulate) 2 "routers" in one kernel. For example, 6 NICs exist
on a machine, 3 of them(eth1-eth3) consist "router A", the others(eth4-eth6)
"router B". Each router has a routing table associated with it. We are
trying to
set a route like entering eth1->kernel->out from eth2->ethernet switch->back
to eth4->kernel->out from eth5. We cannot setup a route directly like this
because the kernel will not send out packets whose nexthop is one of its
network
interfaces. So "ip route add from host_a to host_b via ip_of_eth4 dev eth2"
does
not work, packets never get out of the box before sending out from eth5.

Using iproutes, we can configure that packets entering eth1 is routed to
sending out from eth2 in "router A", and packets entering eth4 is routed to
sending out from eth5 in "router B". No next hop is specified here. Now
packets entering the box through eth2 first go through "router A", then
physically
get out of the box from eth2. We need to redirect them back to eth4 to go
through "router B".  That is why I forcibly modify each packet's ethernet
frame
header and change the destination MAC addr to be the other end of NIC of
the link. For example, the dest_MAC_addr of a packet sending out from eth2
is set to MAC of eth4. This hack works. We can see packets come and go
like what we want. But I got some errors in system log like what i post
before.
It seems to be that some of the packets are "wrong" from the kernel's point
of view.

Please correct me if i misunderstand some points of ethernet, the kernel
and IP stack. Any comment to this approach, and how to do "router simulation
in one kernel" are highly appreciated. Thank you!

-Pei

> -----Original Message-----
> From: Timothy Covell [mailto:timothy.covell@ashavan.org]
> Sent: Thursday, January 10, 2002 9:21 PM
> To: Pei Zheng; linux-kernel@vger.kernel.org
> Subject: Re: strange kernel message when hacking the NIC driver
>
>
> On Thursday 10 January 2002 22:19, Pei Zheng wrote:
> > Hi,
> >
> > I've done some simple hackings to the network device driver. Basically
> > i want to connect two NICs as a point-to-point link,ie, packets sending
> > out from one NIC will be redirected to another NIC, sort of direct-link
> > cable between these two NICs.
> > So in the drivers of these two NICs, before hart_start_xmit(), the skb's
> > header will be forcibly set to its peer's MAC address, thus each packey
> > sending out from one NIC will be directly diverted to its peer NIC. This
> > works fine most of the case since tcpdump shows that packets traverse in
> > this way. However, i found errors in system log:
> >
> > Jan 10 18:02:40 em2 kernel: 1130: 09 => cb
> > Jan 10 18:02:40 em2 kernel: 388: f4 => 45
> > Jan 10 18:02:40 em2 kernel: 26: c0 => 7d
> > Jan 10 18:02:40 em2 kernel: 149: 1f => 98
> > Jan 10 18:02:40 em2 kernel: After error applied:
> > Jan 10 18:02:40 em2 kernel: 00 80 c8 b9 6b b6 00 80 c8 b9 6b b6 08 00
> > 45 00
> > Jan 10 18:02:40 em2 kernel: 05 dc 73 cd 20 b9 3f 11 44 26 7d a8 0c 0a
> > c0 a8
> > Jan 10 18:02:40 em2 kernel: 10 0a e3 c1 4c 1f 6f 83 61 dc fc 5e 5c f4
> > 66 17
> > Jan 10 18:02:40 em2 kernel: 2c 73 19 ce cc 2d 69 69 bd 43 33 6d 82 de
> > 4a 87
> > Jan 10 18:02:40 em2 kernel: 8d 9d 81 f6 2e b2 8c 6a d5 e4 f2 e6 bc ab
> > 5a 01
> > Jan 10 18:02:40 em2 kernel: 34 d5 39 f8 d9 5b 16 bc dc 95 bd b4 08 a9
> > 5e 11
> > Jan 10 18:02:40 em2 kernel: df 38 80 8a ca d8 1c 53 65 97 91 4c 84 5a
> > a1 0e
> > Jan 10 18:02:40 em2 kernel: c2 5f d4 02 a1 9e 1c 35 d4 95 08 44 81 16
> > a3 e0
> >
> > there are many this kind of messages. Don't understand what it is. the
> > beginning part of the data seems to be an icmp packet(00 80 c8 b9 6b
> > b6 is the MAC of one NIC). For some reason kernel thinks that this
> > packet is not correct. Any idea about this? If i want to modify a
> > packet's header before it goes to hard_strat_xmit(), what else should
> > i do except setting the skb header to the MAC of the NIC's peer only?
> > checksum stuff?
> >
> > Any helps will be highly appreciated.
> >
> > -Pei
>
>
> You have me confused about whether your trying to build some kind of
> ethernet bridge or some kind of secret packet sniffer or some other
> such thing.   It looks like you're mixing up TCP/IP and Ethernet.
>
> Ethernet is the link level protocol that addresses NICs by the MAC
> address.    There can only be one NIC on the network with the same
> MAC address.   TCP/IP uses IP addresses, not MAC addresses.  And you
> cannot have two IP addresses on the same link if they have the same
> network and broadcast address unless you do multipathing.
>
>
> I guess that I don't understand what you are trying to achieve.
>
>
>
> --
> timothy.covell@ashavan.org.

Re: strange kernel message when hacking the NIC driver

[Zwane Mwaikambo]

Timothy Covell <timothy.covell@ashavan.org> said:
>If you try to reach a host but fail due to routing issues, then ICMP sends out
>DESTINATION HOST UNKNOWN/HOST UNREACHABLE messages.
>So, ICMP and very much to do with ARP.

Using your argument... Everytime my feline decides to chew my ethernet
cable i lose my network connection, hence i get ICMP HOST UNREACHABLE ergo
my cat has very much to do with ARP.

Those are errors just filtering up the layers, same way an app doesn't
know anything about what device you're writing to, but will complain about
incomplete data writes if you run out of space.

Regards,
	Zwane Mwaikambo

Re: strange kernel message when hacking the NIC driver

[Bernd Eckenfels]

In article <Pine.LNX.4.33.0201140831240.28735-100000@netfinity.realnet.co.sz> you wrote:
> Using your argument... Everytime my feline decides to chew my ethernet
> cable i lose my network connection, hence i get ICMP HOST UNREACHABLE ergo
> my cat has very much to do with ARP.

> Those are errors just filtering up the layers, same way an app doesn't
> know anything about what device you're writing to, but will complain about
> incomplete data writes if you run out of space.

Well actually, the neighbour alive discovery done by arp influences the
routing cache very much. Therefore recent Linux networking is actually able to
send "Host unreachable" messages based on the fact, that an host is down.

You are right, that it is not closely related, but it is something one should
not forget, since it is a behaviour introduced in 2.2 kernels, I think.

You can check it with "ip neig". Entries which have an unreachable host are
marked with "nud failed".

Greetings
Bernd