2.6.8.1: ip auto-config accepts wrong packages

2.6.8.1: ip auto-config accepts wrong packages

[Frank Steiner]

Hi,

we are running a diskless client system using kernel ip autoconfiguration.
The problem is that sometimes clients accept packages that are meant
for other clients.


All clients load the kernel via pxeboot. The pxelinux.cfg contains
the following append line:

append ip=dhcp nfsroot=/,tcp,nfsvers...

Usually everything goes fine. But if two clients send the dhcp request
in the kernel at almost the same time, sometimes they accept the answer
for the other client and then two clients end up with the same IP.

E.g., we have two clients:

   host turan {
     hardware ethernet  ...:02:11:84;
     fixed-address      ....1.167;
  }

   host rado {
     hardware ethernet  ...:02:11:A5;
     fixed-address      ....1.168;
  }

And sometimes we see this output from the kernel on the boot screen:

--
Sending dhcp request. <6>tg3: eth0: Link is up at 100Mbps, full duplex.
tg3: eth0: FLow control is on for TX and on for RX
,.,., OK
IP-Config: Got DHCP answer from ...1.132, my address is ...1.167
--

But this *identical* output appears on *both* screens, i.e., also on the
screen of the host with the MAC :11:A5 that shouldget .168 as IP.
In the server log I find this:

Aug 25 10:03:26 curry dhcpd: DHCPDISCOVER from ..:11:84 via eth0
Aug 25 10:03:26 curry dhcpd: DHCPOFFER on ...1.167 to ...:11:84 via eth0
Aug 25 10:03:26 curry dhcpd: DHCPREQUEST for ...1.167 (...1.132) from ...:11:a5 via eth0: lease ....1.167 unavailable.
Aug 25 10:03:26 curry dhcpd: DHCPNAK on ...1.167 to ...:11:a5 via eth0
Aug 25 10:03:28 curry dhcpd: DHCPDISCOVER from ...:11:a5 via eth0
Aug 25 10:03:28 curry dhcpd: DHCPOFFER on ...1.168 to ...:11:a5 via eth0
Aug 25 10:03:28 curry dhcpd: DHCPREQUEST for ...1.168 (...1.132) from ...:11:84 via eth0: lease ...1.168 unavailable.
Aug 25 10:03:28 curry dhcpd: DHCPNAK on ...1.168 to ...:11:84 via eth0
Aug 25 10:03:28 curry dhcpd: DHCPREQUEST for ...1.168 (...1.132) from ...:11:a5 via eth0
Aug 25 10:03:28 curry dhcpd: DHCPACK on ...1.168 to ...:11:a5 via eth0

So it looks like the first client request is detected at 10:03:26 and the
answer is sent, but likely the second client has also sent its request in
the meantime, and now both clients accept the answer meant only for
the first client.
Then, when the second request is detected at 10:03:28 and answered,
again both clients answer to it.

 From tcpdump on the server I can indeed see that the server gets a
BOOTPREQUEST from :11:84 asking for everything, and it sends a
BOOTREPLY with the IP .1.167.

Next it gets a BOOTREQUEST from the :11:a5 address asking for the
usual parameters (Mask etc.), but it has option 50 set:
OPTION:  50 (  4) Request IP address        141.84.1.167

Then the answer from the server is of course the DHCPNAK.

Thus, it seems that the clients accept any answer without checking if
the packet is really for them during the IP autoconfig in the kernel.
Can/should they do that?

I wasn't able to reproduce the problem from the command line using
dhclient, so I'm not sure if this is a general problem with dhcp or
a problem in the ip-autoconfig code of the client.

Server and Clients both run 2.6.8.1 and the dhcp server is the ISC DHCP
server version 3.0.1rc12.

cu,
Frank

-- 
Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049

Re: 2.6.8.1: ip auto-config accepts wrong packages

[Mark Broadbent]

On Wed, 2004-08-25 at 10:40, Frank Steiner wrote:
> Hi,
> 
> we are running a diskless client system using kernel ip autoconfiguration.
> The problem is that sometimes clients accept packages that are meant
> for other clients.
> 
> ...
>
> I wasn't able to reproduce the problem from the command line using
> dhclient, so I'm not sure if this is a general problem with dhcp or
> a problem in the ip-autoconfig code of the client.

Try enabling the debug messaging in the ipconfig code.  Edit the file
net/ipv4/ipconfig.c just below the #include's and change the line 

#undef IPCONFIG_DEBUG

to

#define IPCONFIG_DEBUG

and rebuild.  Post the outputs from both machines when you see this
effect again.

> Server and Clients both run 2.6.8.1 and the dhcp server is the ISC DHCP
> server version 3.0.1rc12.


Thanks
Mark

> cu,
> Frank

Re: 2.6.8.1: ip auto-config accepts wrong packages

[Frank Steiner]

Hi Mark,

Mark Broadbent wrote:
> On Wed, 2004-08-25 at 10:40, Frank Steiner wrote:

> Try enabling the debug messaging in the ipconfig code.  Edit the file
> net/ipv4/ipconfig.c just below the #include's and change the line 
> 
> #undef IPCONFIG_DEBUG
> 
> to
> 
> #define IPCONFIG_DEBUG
> 
> and rebuild.  Post the outputs from both machines when you see this
> effect again.

the output is exactly the same on both machines! On the one machine that
went through (because it got the IP first) I got the output from dmesg,
printed it and compared it to the both boot screens. And they are really
identical.

I didn't see any option (or "extension") that contains the mac address.
Could that be the problem? Do I have to tell the server that the mac
address must be in the request-list?

cu,
Frank



IP-Config: Entered.
IP-Config: eth0 UP (able=1, xid=07196018)
Sending DHCP requests .DHCP: Sending message type 1
tg3: eth0: Link is up at 100 Mbps, full duplex.
tg3: eth0: Flow control is on for TX and on for RX.
DHCP: Got message type 2
DHCP: Offered address 141.84.1.168 by server 141.84.1.132
DHCP/BOOTP: Got extension 53: 02
DHCP/BOOTP: Got extension 54: 8d 54 01 84
DHCP/BOOTP: Got extension 51: 00 00 1c 20
DHCP/BOOTP: Got extension 1: ff ff ff 80
DHCP/BOOTP: Got extension 3: 8d 54 01 82
DHCP/BOOTP: Got extension 6: 81 bb d6 87
DHCP/BOOTP: Got extension 12: 72 61 64 6f
DHCP/BOOTP: Got extension 15: 62 69 6f 2e 69 66 69 2e 6c 6d 75 2e 64 65
DHCP/BOOTP: Got extension 17: 2f
DHCP/BOOTP: Got extension 28: 8d 54 01 ff
DHCP/BOOTP: Got extension 208: f1 00 74 7e
DHCP/BOOTP: Got extension 209: 70 78 65 6c 69 6e 75 78 2e 63 66 67 2e 72 61 64 6f
DHCP/BOOTP: Got extension 210: 68 6f 73 74 73 2f
,DHCP: Sending message type 3
DHCP: Got message type 5
DHCP/BOOTP: Got extension 53: 05
DHCP/BOOTP: Got extension 54: 8d 54 01 84
DHCP/BOOTP: Got extension 51: 00 00 1c 20
DHCP/BOOTP: Got extension 1: ff ff ff 80
DHCP/BOOTP: Got extension 3: 8d 54 01 82
DHCP/BOOTP: Got extension 6: 81 bb d6 87
DHCP/BOOTP: Got extension 12: 72 61 64 6f
DHCP/BOOTP: Got extension 15: 62 69 6f 2e 69 66 69 2e 6c 6d 75 2e 64 65
DHCP/BOOTP: Got extension 17: 2f
DHCP/BOOTP: Got extension 28: 8d 54 01 ff
DHCP/BOOTP: Got extension 208: f1 00 74 7e
DHCP/BOOTP: Got extension 209: 70 78 65 6c 69 6e 75 78 2e 63 66 67 2e 72 61 64 6f
DHCP/BOOTP: Got extension 210: 68 6f 73 74 73 2f
  OK
IP-Config: Got DHCP answer from xxx.xxx.1.132, my address is xxx.xxx.1.168
IP-Config: Complete:
       device=eth0, addr=xxx.xxx.1.168, mask=255.255.255.128, gw=xxx.xxx.1.130,
      host=rado, domain=xxx nis-domain=(none),
      bootserver=xxx.xxx.1.132, rootserver=xxx.xxx.1.132, rootpath=/



-- 
Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049

Re: 2.6.8.1: ip auto-config accepts wrong packages

[Mark Broadbent]

Hi Frank,

On Wed, 2004-08-25 at 15:41, Frank Steiner wrote:
>
> the output is exactly the same on both machines! On the one machine that
> went through (because it got the IP first) I got the output from dmesg,
> printed it and compared it to the both boot screens. And they are really
> identical.

Stupid question but the MAC addresses on the cards are different aren't
they?  Could you also double check that the xid values differ on both
machines, this is shown on the line:

IP-Config: eth0 UP (able=1, xid=07196018)
                                ^^^^^^^^

Both MAC and xid should be different.

> I didn't see any option (or "extension") that contains the mac address.
> Could that be the problem? Do I have to tell the server that the mac
> address must be in the request-list?

The MAC address is embedded in the chaddr field of the request packets.

This trace does not contain any DHCP NACK responses that were shown in
the DHCP server log (annotated in the log below).  Are you sure it is
behaving exactly as before (i.e. is the DHCP server showing the same
messages being transmitted, including the NACKs)?

The kernel appears to be working correctly in this case.

Thanks
Mark

> IP-Config: Entered.
> IP-Config: eth0 UP (able=1, xid=07196018)
> Sending DHCP requests .DHCP: Sending message type 1
[DHCPDISCOVER]
> tg3: eth0: Link is up at 100 Mbps, full duplex.
> tg3: eth0: Flow control is on for TX and on for RX.
> DHCP: Got message type 2
[DHCPOFFER]
> DHCP: Offered address 141.84.1.168 by server 141.84.1.132
> DHCP/BOOTP: Got extension 53: 02
> DHCP/BOOTP: Got extension 54: 8d 54 01 84
> DHCP/BOOTP: Got extension 51: 00 00 1c 20
> DHCP/BOOTP: Got extension 1: ff ff ff 80
> DHCP/BOOTP: Got extension 3: 8d 54 01 82
> DHCP/BOOTP: Got extension 6: 81 bb d6 87
> DHCP/BOOTP: Got extension 12: 72 61 64 6f
> DHCP/BOOTP: Got extension 15: 62 69 6f 2e 69 66 69 2e 6c 6d 75 2e 64 65
> DHCP/BOOTP: Got extension 17: 2f
> DHCP/BOOTP: Got extension 28: 8d 54 01 ff
> DHCP/BOOTP: Got extension 208: f1 00 74 7e
> DHCP/BOOTP: Got extension 209: 70 78 65 6c 69 6e 75 78 2e 63 66 67 2e 72 61 64 6f
> DHCP/BOOTP: Got extension 210: 68 6f 73 74 73 2f
> ,DHCP: Sending message type 3
[DHCPREQUEST]
> DHCP: Got message type 5
[DHCPACK]
> DHCP/BOOTP: Got extension 53: 05
> DHCP/BOOTP: Got extension 54: 8d 54 01 84
> DHCP/BOOTP: Got extension 51: 00 00 1c 20
> DHCP/BOOTP: Got extension 1: ff ff ff 80
> DHCP/BOOTP: Got extension 3: 8d 54 01 82
> DHCP/BOOTP: Got extension 6: 81 bb d6 87
> DHCP/BOOTP: Got extension 12: 72 61 64 6f
> DHCP/BOOTP: Got extension 15: 62 69 6f 2e 69 66 69 2e 6c 6d 75 2e 64 65
> DHCP/BOOTP: Got extension 17: 2f
> DHCP/BOOTP: Got extension 28: 8d 54 01 ff
> DHCP/BOOTP: Got extension 208: f1 00 74 7e
> DHCP/BOOTP: Got extension 209: 70 78 65 6c 69 6e 75 78 2e 63 66 67 2e 72 61 64 6f
> DHCP/BOOTP: Got extension 210: 68 6f 73 74 73 2f
>   OK
> IP-Config: Got DHCP answer from xxx.xxx.1.132, my address is xxx.xxx.1.168
> IP-Config: Complete:
>        device=eth0, addr=xxx.xxx.1.168, mask=255.255.255.128, gw=xxx.xxx.1.130,
>       host=rado, domain=xxx nis-domain=(none),
>       bootserver=xxx.xxx.1.132, rootserver=xxx.xxx.1.132, rootpath=/

Re: 2.6.8.1: ip auto-config accepts wrong packages

[Frank Steiner]

Hi Mark,

Mark Broadbent wrote:

> Stupid question but the MAC addresses on the cards are different aren't
> they?  Could you also double check that the xid values differ on both
> machines, this is shown on the line:
> 
> IP-Config: eth0 UP (able=1, xid=07196018)
>                                 ^^^^^^^^
> 
> Both MAC and xid should be different.

obiously, the problem is here. The xid is the same on 5 different hosts
I tested with the debugging-enabled kernel. Looking it ipconfig.c I now
understand that with several hosts using the same xid it must go wrong.
I'm currently compiling a new kernel with debugging enabled in rancom.c
to see what's going wrong here. Looks like random_bytes is returning
the same value on all hosts :-(

Can anyone else maybe do a quick check with "ip=dhcp" on two hosts
just to see if the problem with the identical xids just show up here?

I will send the result of the debugging output from random.c.

cu,
Frank

-- 
Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049

Re: 2.6.8.1: ip auto-config accepts wrong packages

[Frank Steiner]

Hi Mark,

in addition to the DEBUG flags I added two little debugging lines to make
sure the right code is executed.

ipconfig.c:
			if (able & IC_BOOTP){
			  DBG(("calling get_random_btyes"));
				get_random_bytes(&d->xid, sizeof(u32));

random.c:

	void get_random_bytes(void *buf, int nbytes)
	{
	  DEBUG_ENT("get_random_bytes entered\n");
	  if (sec_random_state) {
	    DEBUG_ENT("sec_random_state\n");
	    extract_entropy(sec_random_state, (char *) buf, nbytes,
			    EXTRACT_ENTROPY_SECONDARY);
	  }


And here is the output from boot.msg:


<4>IP-Config: Entered.
<4>calling get_random_btyes<7>random: get_random_bytes entered
<7>random: sec_random_state
<7>random: 0000 0000 : going to reseed secondary with 64 bits (32 of 0 requested)
<7>random: 0000 0000 : trying to extract 64 bits from primary
<7>random: 0000 0000 : debiting 0 bits from primary
<7>random: 0000 0000 : trying to extract 32 bits from secondary
<7>random: 0000 0000 : debiting 32 bits from secondary (unlimited)
<4>IP-Config: eth0 UP (able=1, xid=07196018)
<5>Sending BOOTP requests .<7>random: get_random_bytes entered
<7>random: sec_random_state
<7>random: 0000 0000 : going to reseed secondary with 64 bits (32 of 0 requested)
<7>random: 0000 0000 : trying to extract 64 bits from primary
<7>random: 0000 0000 : debiting 0 bits from primary
<7>random: 0000 0000 : trying to extract 32 bits from secondary
<7>random: 0000 0000 : debiting 32 bits from secondary (unlimited)
<6>tg3: eth0: Link is up at 100 Mbps, full duplex.
<6>tg3: eth0: Flow control is on for TX and on for RX.
<4>.DHCP/BOOTP: Got extension 1: ff ff ff 80
<4>DHCP/BOOTP: Got extension 3: 8d 54 01 82
<4>DHCP/BOOTP: Got extension 6: 81 bb d6 87
<4>DHCP/BOOTP: Got extension 17: 2f
<4>DHCP/BOOTP: Got extension 28: 8d 54 01 ff
<4>DHCP/BOOTP: Got extension 15: 62 69 6f 2e 69 66 69 2e 6c 6d 75 2e 64 65
<4> OK
<4>IP-Config: Got BOOTP answer from 141.84.1.132, my address is 141.84.1.167
...

So it looks like the code is executed, but the xid is always the
same. Just like the random module would not create a random number
but a constant :-) I'm not good enough in C to really understand
and further debug the code in random.c or ipconfig.c :-((

cu,
Frank

-- 
Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049
* Rekursion kann man erst verstehen, wenn man Rekursion verstanden hat. *

Re: 2.6.8.1: ip auto-config accepts wrong packages

[Randy.Dunlap]

On Thu, 26 Aug 2004 12:31:12 +0200 Frank Steiner wrote:

| Hi Mark,
| 
| in addition to the DEBUG flags I added two little debugging lines to make
| sure the right code is executed.
| 
| ipconfig.c:
| 			if (able & IC_BOOTP){
| 			  DBG(("calling get_random_btyes"));
| 				get_random_bytes(&d->xid, sizeof(u32));
| 
| random.c:
| 
| 	void get_random_bytes(void *buf, int nbytes)
| 	{
| 	  DEBUG_ENT("get_random_bytes entered\n");
| 	  if (sec_random_state) {
| 	    DEBUG_ENT("sec_random_state\n");
| 	    extract_entropy(sec_random_state, (char *) buf, nbytes,
| 			    EXTRACT_ENTROPY_SECONDARY);
| 	  }
| 
| 
| And here is the output from boot.msg:
| 
| 
| <4>IP-Config: Entered.
| <4>calling get_random_btyes<7>random: get_random_bytes entered
| <7>random: sec_random_state
| <7>random: 0000 0000 : going to reseed secondary with 64 bits (32 of 0 requested)
| <7>random: 0000 0000 : trying to extract 64 bits from primary
| <7>random: 0000 0000 : debiting 0 bits from primary
| <7>random: 0000 0000 : trying to extract 32 bits from secondary
| <7>random: 0000 0000 : debiting 32 bits from secondary (unlimited)
| <4>IP-Config: eth0 UP (able=1, xid=07196018)
| <5>Sending BOOTP requests .<7>random: get_random_bytes entered
| <7>random: sec_random_state
| <7>random: 0000 0000 : going to reseed secondary with 64 bits (32 of 0 requested)
| <7>random: 0000 0000 : trying to extract 64 bits from primary
| <7>random: 0000 0000 : debiting 0 bits from primary
| <7>random: 0000 0000 : trying to extract 32 bits from secondary
| <7>random: 0000 0000 : debiting 32 bits from secondary (unlimited)
| <6>tg3: eth0: Link is up at 100 Mbps, full duplex.
| <6>tg3: eth0: Flow control is on for TX and on for RX.
| <4>.DHCP/BOOTP: Got extension 1: ff ff ff 80
| <4>DHCP/BOOTP: Got extension 3: 8d 54 01 82
| <4>DHCP/BOOTP: Got extension 6: 81 bb d6 87
| <4>DHCP/BOOTP: Got extension 17: 2f
| <4>DHCP/BOOTP: Got extension 28: 8d 54 01 ff
| <4>DHCP/BOOTP: Got extension 15: 62 69 6f 2e 69 66 69 2e 6c 6d 75 2e 64 65
| <4> OK
| <4>IP-Config: Got BOOTP answer from 141.84.1.132, my address is 141.84.1.167
| ...
| 
| So it looks like the code is executed, but the xid is always the
| same. Just like the random module would not create a random number
| but a constant :-) I'm not good enough in C to really understand
| and further debug the code in random.c or ipconfig.c :-((

Maybe fixed by
http://linux.bkbits.net:8080/linux-2.5/cset@412a4a00MfXRfzWB5kTFo9NXM1Q3hw?nav=index.html|ChangeSet@-7d

i.e., fix is already merged, I think.

--
~Randy

Re: 2.6.8.1: ip auto-config accepts wrong packages

[Frank Steiner]

Randy.Dunlap wrote:

> Maybe fixed by
> http://linux.bkbits.net:8080/linux-2.5/cset@412a4a00MfXRfzWB5kTFo9NXM1Q3hw?nav=index.html|ChangeSet@-7d
> 
> i.e., fix is already merged, I think.

Yes, that's exactly describing the problem I encountered, and indeed
fixes it :-) I have to admit that I didn't know this page yet... I will
check there first before reporting bugs in the future!

Thanks a lot!

cu,
Frank
-- 
Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049