[PATCH] [XM-TEST] Use explicit (IP id==packet size) on hping2 tests to avoid problems with IP id==0

[PATCH] [XM-TEST] Use explicit (IP id==packet size) on hping2 tests to avoid problems with IP id==0

[James Dykman]

[-- Attachment #1: Type: text/plain, Size: 5192 bytes --]

We have run into some xm-test network test failures in the tests that use 
hping2 that are caused by an IP id of 0:

- hping2 uses a packet socket, so must do its own fragmentation when 
sending messages >1500 bytes.
- When fragmenting, hping2 uses the low byte of the PID as the IP id in 
each of the fragmented IP headers.
- The kernel will assign an IP id if it sees the IP id ==  0 in the header 
that hping2 passes down 
   (net/ipv4/raw.c:raw_send_hdrinc())
So if the low byte of the PID is 0, every fragment will get assigned a 
unique IP id by the kernel, and 
reassembly will fail at the destination.

Jim
----------------------------------
Use explicit (IP id==packet size) on hping2 tests to avoid problems with 
IP id==0

Signed-off-by: Jim Dykman <dykman@us.ibm.com>

diff -r dc213d745642 
tools/xm-test/tests/network/03_network_local_tcp_pos.py
--- a/tools/xm-test/tests/network/03_network_local_tcp_pos.py   Mon May 15 
15:32:09 2006
+++ b/tools/xm-test/tests/network/03_network_local_tcp_pos.py   Thu May 18 
14:19:01 2006
@@ -44,7 +44,7 @@
     lofails=""
     for size in trysizes:
         out = console.runCmd("hping2 127.0.0.1 -E /dev/urandom -q -c 20 "
-              + "--fast -d " + str(size))
+              + "--fast -d " + str(size) + " -N " + str(size))
         if out["return"]:
             lofails += " " + str(size)

@@ -54,7 +54,7 @@
     ip = netdev.getNetDevIP()
     for size in trysizes:
         out = console.runCmd("hping2 " + ip + " -E /dev/urandom -q -c 20 
"
-              + "--fast -d "+ str(size))
+              + "--fast -d "+ str(size) + " -N " + str(size))
         if out["return"]:
             eth0fails += " " + str(size)
 except ConsoleError, e:
diff -r dc213d745642 
tools/xm-test/tests/network/04_network_local_udp_pos.py
--- a/tools/xm-test/tests/network/04_network_local_udp_pos.py   Mon May 15 
15:32:09 2006
+++ b/tools/xm-test/tests/network/04_network_local_udp_pos.py   Thu May 18 
14:19:01 2006
@@ -43,7 +43,7 @@
     lofails=""
     for size in trysizes:
         out = console.runCmd("hping2 127.0.0.1 -E /dev/urandom -2 -q -c 
20 "
-              + "--fast -d " + str(size))
+              + "--fast -d " + str(size) + " -N " + str(size))
         if out["return"]:
             lofails += " " + str(size)
             print out["output"]
@@ -54,7 +54,7 @@
     ip = netdev.getNetDevIP()
     for size in trysizes:
         out = console.runCmd("hping2 " + ip + " -E /dev/urandom -2 -q -c 
20 "
-              + "--fast -d " + str(size))
+              + "--fast -d " + str(size) + " -N " + str(size))
         if out["return"]:
             eth0fails += " " + str(size)
             print out["output"]
diff -r dc213d745642 
tools/xm-test/tests/network/06_network_dom0_tcp_pos.py 
--- a/tools/xm-test/tests/network/06_network_dom0_tcp_pos.py    Mon May 15 
15:32:09 2006
+++ b/tools/xm-test/tests/network/06_network_dom0_tcp_pos.py    Thu May 18 
14:19:01 2006
@@ -44,7 +44,7 @@
     dom0ip = netdev.getDom0AliasIP()
     for size in trysizes:
         out = console.runCmd("hping2 " + dom0ip + " -E /dev/urandom -q -c 
20 "
-              + "--fast -d " + str(size))
+              + "--fast -d " + str(size) + " -N " + str(size))
         if out["return"]:
             fails += " " + str(size)
             print out["output"]
diff -r dc213d745642 
tools/xm-test/tests/network/07_network_dom0_udp_pos.py
--- a/tools/xm-test/tests/network/07_network_dom0_udp_pos.py    Mon May 15 
15:32:09 2006
+++ b/tools/xm-test/tests/network/07_network_dom0_udp_pos.py    Thu May 18 
14:19:01 2006
@@ -43,7 +43,7 @@
     dom0ip = netdev.getDom0AliasIP()
     for size in trysizes:
         out = console.runCmd("hping2 " + dom0ip + " -E /dev/urandom -2 -q 
-c 20"
-             + " --fast -d " + str(size))
+             + " --fast -d " + str(size) + " -N " + str(size))
         if out["return"]:
             fails += " " + str(size)
             print out["output"]
diff -r dc213d745642 
tools/xm-test/tests/network/12_network_domU_tcp_pos.py
--- a/tools/xm-test/tests/network/12_network_domU_tcp_pos.py    Mon May 15 
15:32:09 2006
+++ b/tools/xm-test/tests/network/12_network_domU_tcp_pos.py    Thu May 18 
14:19:01 2006
@@ -50,7 +50,7 @@
     ip2 = dst_netdev.getNetDevIP()
     for size in pingsizes:
         out = src_console.runCmd("hping2 " + ip2 + " -E /dev/urandom -q 
-c 20 "
-              + "--fast -d " + str(size))
+              + "--fast -d " + str(size) + " -N " + str(size))
         if out["return"]:
             fails += " " + str(size)
             print out["output"]
diff -r dc213d745642 
tools/xm-test/tests/network/13_network_domU_udp_pos.py
--- a/tools/xm-test/tests/network/13_network_domU_udp_pos.py    Mon May 15 
15:32:09 2006
+++ b/tools/xm-test/tests/network/13_network_domU_udp_pos.py    Thu May 18 
14:19:01 2006
@@ -50,7 +50,7 @@
     ip2 = dst_netdev.getNetDevIP()
     for size in pingsizes:
         out = src_console.runCmd("hping2 " + ip2 + " -E /dev/urandom -2 
-q "
-              + "-c 20 --fast -d " + str(size))
+              + "-c 20 --fast -d " + str(size) + " -N " + str(size))
         if out["return"]:
             fails += " " + str(size)
             print out["output"]





[-- Attachment #2: ipid.patch --]
[-- Type: application/octet-stream, Size: 4357 bytes --]

diff -r dc213d745642 tools/xm-test/tests/network/03_network_local_tcp_pos.py
--- a/tools/xm-test/tests/network/03_network_local_tcp_pos.py	Mon May 15 15:32:09 2006
+++ b/tools/xm-test/tests/network/03_network_local_tcp_pos.py	Thu May 18 14:19:01 2006
@@ -44,7 +44,7 @@
     lofails=""
     for size in trysizes:
         out = console.runCmd("hping2 127.0.0.1 -E /dev/urandom -q -c 20 " 
-              + "--fast -d " + str(size))
+              + "--fast -d " + str(size) + " -N " + str(size))
         if out["return"]:
             lofails += " " + str(size)
 
@@ -54,7 +54,7 @@
     ip = netdev.getNetDevIP()
     for size in trysizes:
         out = console.runCmd("hping2 " + ip + " -E /dev/urandom -q -c 20 "
-              + "--fast -d "+ str(size))
+              + "--fast -d "+ str(size) + " -N " + str(size))
         if out["return"]:
             eth0fails += " " + str(size) 
 except ConsoleError, e:
diff -r dc213d745642 tools/xm-test/tests/network/04_network_local_udp_pos.py
--- a/tools/xm-test/tests/network/04_network_local_udp_pos.py	Mon May 15 15:32:09 2006
+++ b/tools/xm-test/tests/network/04_network_local_udp_pos.py	Thu May 18 14:19:01 2006
@@ -43,7 +43,7 @@
     lofails=""
     for size in trysizes:
         out = console.runCmd("hping2 127.0.0.1 -E /dev/urandom -2 -q -c 20 "
-              + "--fast -d " + str(size))
+              + "--fast -d " + str(size) + " -N " + str(size))
         if out["return"]:
             lofails += " " + str(size)
             print out["output"]
@@ -54,7 +54,7 @@
     ip = netdev.getNetDevIP()
     for size in trysizes:
         out = console.runCmd("hping2 " + ip + " -E /dev/urandom -2 -q -c 20 "
-              + "--fast -d " + str(size))
+              + "--fast -d " + str(size) + " -N " + str(size))
         if out["return"]:
             eth0fails += " " + str(size) 
             print out["output"]
diff -r dc213d745642 tools/xm-test/tests/network/06_network_dom0_tcp_pos.py
--- a/tools/xm-test/tests/network/06_network_dom0_tcp_pos.py	Mon May 15 15:32:09 2006
+++ b/tools/xm-test/tests/network/06_network_dom0_tcp_pos.py	Thu May 18 14:19:01 2006
@@ -44,7 +44,7 @@
     dom0ip = netdev.getDom0AliasIP()
     for size in trysizes:
         out = console.runCmd("hping2 " + dom0ip + " -E /dev/urandom -q -c 20 "
-              + "--fast -d " + str(size))
+              + "--fast -d " + str(size) + " -N " + str(size))
         if out["return"]:
             fails += " " + str(size) 
             print out["output"]
diff -r dc213d745642 tools/xm-test/tests/network/07_network_dom0_udp_pos.py
--- a/tools/xm-test/tests/network/07_network_dom0_udp_pos.py	Mon May 15 15:32:09 2006
+++ b/tools/xm-test/tests/network/07_network_dom0_udp_pos.py	Thu May 18 14:19:01 2006
@@ -43,7 +43,7 @@
     dom0ip = netdev.getDom0AliasIP()
     for size in trysizes:
         out = console.runCmd("hping2 " + dom0ip + " -E /dev/urandom -2 -q -c 20"
-             + " --fast -d " + str(size))
+             + " --fast -d " + str(size) + " -N " + str(size))
         if out["return"]:
             fails += " " + str(size) 
             print out["output"]
diff -r dc213d745642 tools/xm-test/tests/network/12_network_domU_tcp_pos.py
--- a/tools/xm-test/tests/network/12_network_domU_tcp_pos.py	Mon May 15 15:32:09 2006
+++ b/tools/xm-test/tests/network/12_network_domU_tcp_pos.py	Thu May 18 14:19:01 2006
@@ -50,7 +50,7 @@
     ip2 = dst_netdev.getNetDevIP()
     for size in pingsizes:
         out = src_console.runCmd("hping2 " + ip2 + " -E /dev/urandom -q -c 20 "
-              + "--fast -d " + str(size))
+              + "--fast -d " + str(size) + " -N " + str(size))
         if out["return"]:
             fails += " " + str(size) 
             print out["output"]
diff -r dc213d745642 tools/xm-test/tests/network/13_network_domU_udp_pos.py
--- a/tools/xm-test/tests/network/13_network_domU_udp_pos.py	Mon May 15 15:32:09 2006
+++ b/tools/xm-test/tests/network/13_network_domU_udp_pos.py	Thu May 18 14:19:01 2006
@@ -50,7 +50,7 @@
     ip2 = dst_netdev.getNetDevIP()
     for size in pingsizes:
         out = src_console.runCmd("hping2 " + ip2 + " -E /dev/urandom -2 -q "
-              + "-c 20 --fast -d " + str(size))
+              + "-c 20 --fast -d " + str(size) + " -N " + str(size))
         if out["return"]:
             fails += " " + str(size) 
             print out["output"]

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH] [XM-TEST] Use explicit (IP id==packet size) on hping2 tests to avoid problems with IP id==0

[Nivedita Singhvi]

James Dykman wrote:
> We have run into some xm-test network test failures in the tests that use 
> hping2 that are caused by an IP id of 0:
> 
> - hping2 uses a packet socket, so must do its own fragmentation when 
> sending messages >1500 bytes.
> - When fragmenting, hping2 uses the low byte of the PID as the IP id in 
> each of the fragmented IP headers.
> - The kernel will assign an IP id if it sees the IP id ==  0 in the header 
> that hping2 passes down 
>    (net/ipv4/raw.c:raw_send_hdrinc())
> So if the low byte of the PID is 0, every fragment will get assigned a 
> unique IP id by the kernel, and 
> reassembly will fail at the destination.
> 
> Jim
> ----------------------------------
> Use explicit (IP id==packet size) on hping2 tests to avoid problems with 
> IP id==0

Jim, will this not be a cause potential corruption - if we have incoming
fragments belonging to different packets with the same ipid
there is no way for the reassembly code to distinguish them.
Checksum failures might catch some, but that is no guarantee of
safe delivery.

I realize the stuff may be getting serialized enough for now
that we don't run into the problem while testing - but a globally
unique ipid is necessary to avoid corruption in an
environment where fragments can get dropped or arrive out
of order. What am I missing?

thanks,
Nivedita


> Signed-off-by: Jim Dykman <dykman@us.ibm.com>
> 
> diff -r dc213d745642 
> tools/xm-test/tests/network/03_network_local_tcp_pos.py
> --- a/tools/xm-test/tests/network/03_network_local_tcp_pos.py   Mon May 15 
> 15:32:09 2006
> +++ b/tools/xm-test/tests/network/03_network_local_tcp_pos.py   Thu May 18 
> 14:19:01 2006
> @@ -44,7 +44,7 @@
>      lofails=""
>      for size in trysizes:
>          out = console.runCmd("hping2 127.0.0.1 -E /dev/urandom -q -c 20 "
> -              + "--fast -d " + str(size))
> +              + "--fast -d " + str(size) + " -N " + str(size))
>          if out["return"]:
>              lofails += " " + str(size)
> 
> @@ -54,7 +54,7 @@
>      ip = netdev.getNetDevIP()
>      for size in trysizes:
>          out = console.runCmd("hping2 " + ip + " -E /dev/urandom -q -c 20 
> "
> -              + "--fast -d "+ str(size))
> +              + "--fast -d "+ str(size) + " -N " + str(size))
>          if out["return"]:
>              eth0fails += " " + str(size)
>  except ConsoleError, e:
> diff -r dc213d745642 
> tools/xm-test/tests/network/04_network_local_udp_pos.py
> --- a/tools/xm-test/tests/network/04_network_local_udp_pos.py   Mon May 15 
> 15:32:09 2006
> +++ b/tools/xm-test/tests/network/04_network_local_udp_pos.py   Thu May 18 
> 14:19:01 2006
> @@ -43,7 +43,7 @@
>      lofails=""
>      for size in trysizes:
>          out = console.runCmd("hping2 127.0.0.1 -E /dev/urandom -2 -q -c 
> 20 "
> -              + "--fast -d " + str(size))
> +              + "--fast -d " + str(size) + " -N " + str(size))
>          if out["return"]:
>              lofails += " " + str(size)
>              print out["output"]
> @@ -54,7 +54,7 @@
>      ip = netdev.getNetDevIP()
>      for size in trysizes:
>          out = console.runCmd("hping2 " + ip + " -E /dev/urandom -2 -q -c 
> 20 "
> -              + "--fast -d " + str(size))
> +              + "--fast -d " + str(size) + " -N " + str(size))
>          if out["return"]:
>              eth0fails += " " + str(size)
>              print out["output"]
> diff -r dc213d745642 
> tools/xm-test/tests/network/06_network_dom0_tcp_pos.py 
> --- a/tools/xm-test/tests/network/06_network_dom0_tcp_pos.py    Mon May 15 
> 15:32:09 2006
> +++ b/tools/xm-test/tests/network/06_network_dom0_tcp_pos.py    Thu May 18 
> 14:19:01 2006
> @@ -44,7 +44,7 @@
>      dom0ip = netdev.getDom0AliasIP()
>      for size in trysizes:
>          out = console.runCmd("hping2 " + dom0ip + " -E /dev/urandom -q -c 
> 20 "
> -              + "--fast -d " + str(size))
> +              + "--fast -d " + str(size) + " -N " + str(size))
>          if out["return"]:
>              fails += " " + str(size)
>              print out["output"]
> diff -r dc213d745642 
> tools/xm-test/tests/network/07_network_dom0_udp_pos.py
> --- a/tools/xm-test/tests/network/07_network_dom0_udp_pos.py    Mon May 15 
> 15:32:09 2006
> +++ b/tools/xm-test/tests/network/07_network_dom0_udp_pos.py    Thu May 18 
> 14:19:01 2006
> @@ -43,7 +43,7 @@
>      dom0ip = netdev.getDom0AliasIP()
>      for size in trysizes:
>          out = console.runCmd("hping2 " + dom0ip + " -E /dev/urandom -2 -q 
> -c 20"
> -             + " --fast -d " + str(size))
> +             + " --fast -d " + str(size) + " -N " + str(size))
>          if out["return"]:
>              fails += " " + str(size)
>              print out["output"]
> diff -r dc213d745642 
> tools/xm-test/tests/network/12_network_domU_tcp_pos.py
> --- a/tools/xm-test/tests/network/12_network_domU_tcp_pos.py    Mon May 15 
> 15:32:09 2006
> +++ b/tools/xm-test/tests/network/12_network_domU_tcp_pos.py    Thu May 18 
> 14:19:01 2006
> @@ -50,7 +50,7 @@
>      ip2 = dst_netdev.getNetDevIP()
>      for size in pingsizes:
>          out = src_console.runCmd("hping2 " + ip2 + " -E /dev/urandom -q 
> -c 20 "
> -              + "--fast -d " + str(size))
> +              + "--fast -d " + str(size) + " -N " + str(size))
>          if out["return"]:
>              fails += " " + str(size)
>              print out["output"]
> diff -r dc213d745642 
> tools/xm-test/tests/network/13_network_domU_udp_pos.py
> --- a/tools/xm-test/tests/network/13_network_domU_udp_pos.py    Mon May 15 
> 15:32:09 2006
> +++ b/tools/xm-test/tests/network/13_network_domU_udp_pos.py    Thu May 18 
> 14:19:01 2006
> @@ -50,7 +50,7 @@
>      ip2 = dst_netdev.getNetDevIP()
>      for size in pingsizes:
>          out = src_console.runCmd("hping2 " + ip2 + " -E /dev/urandom -2 
> -q "
> -              + "-c 20 --fast -d " + str(size))
> +              + "-c 20 --fast -d " + str(size) + " -N " + str(size))
>          if out["return"]:
>              fails += " " + str(size)
>              print out["output"]
> 
> 
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: [PATCH] [XM-TEST] Use explicit (IP id==packet size) on hping2 tests to avoid problems with IP id==0

[James Dykman]

niv@us.ltcfwd.linux.ibm.com wrote on 05/18/2006 05:32:26 PM:

> James Dykman wrote:
> > We have run into some xm-test network test failures in the tests that 
use 
> > hping2 that are caused by an IP id of 0:
> > 
> > - hping2 uses a packet socket, so must do its own fragmentation when 
> > sending messages >1500 bytes.
> > - When fragmenting, hping2 uses the low byte of the PID as the IP id 
in 
> > each of the fragmented IP headers.
> > - The kernel will assign an IP id if it sees the IP id ==  0 in the 
header 
> > that hping2 passes down 
> >    (net/ipv4/raw.c:raw_send_hdrinc())
> > So if the low byte of the PID is 0, every fragment will get assigned a 

> > unique IP id by the kernel, and 
> > reassembly will fail at the destination.
> > 
> > Jim
> > ----------------------------------
> > Use explicit (IP id==packet size) on hping2 tests to avoid problems 
with 
> > IP id==0
> 
> Jim, will this not be a cause potential corruption - if we have incoming
> fragments belonging to different packets with the same ipid
> there is no way for the reassembly code to distinguish them.
> Checksum failures might catch some, but that is no guarantee of
> safe delivery.
> 

That possibility exists in the tests as they are today... hping2 is 
assigning IP ids based on
its PID, completely independent of the kernel.

If you want to solve that problem TOO, then we need to replace hping2. Any 
suggestions?

Jim

Re: [PATCH] [XM-TEST] Use explicit (IP id==packet size) on hping2 tests to avoid problems with IP id==0

[Nivedita Singhvi]

James Dykman wrote:

>>Jim, will this not be a cause potential corruption - if we have incoming
>>fragments belonging to different packets with the same ipid
>>there is no way for the reassembly code to distinguish them.
>>Checksum failures might catch some, but that is no guarantee of
>>safe delivery.
>>
> 
> 
> That possibility exists in the tests as they are today... hping2 is 
> assigning IP ids based on
> its PID, completely independent of the kernel.

1. This might be fine for temporary purposes (although
    we send multiple packets of the same size in consecutive
    order, so only serialization is helping us in those
    tests, correct? Perhaps something that would be unique
    at least across the xmtest set of hpings for sure?)

> If you want to solve that problem TOO, then we need to replace hping2. Any 
> suggestions?

How about for now removing the hping tests which involve
sends of > MTU size, and using ping and netperf udp/tcp tests
to replace those (underway)?

This is just to avoid another round of bugs further down
the road when we get to SMP testing in a serious way.

thanks,
Nivedita