* Response error to fragmented ICMP echo request
@ 2023-03-07 12:11 Song, Xiongwei
2023-03-07 12:35 ` Vladimir Oltean
0 siblings, 1 reply; 5+ messages in thread
From: Song, Xiongwei @ 2023-03-07 12:11 UTC (permalink / raw)
To: vladimir.oltean@nxp.com, claudiu.manoil@nxp.com,
alexandre.belloni@bootlin.com, UNGLinuxDriver@microchip.com,
davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
pabeni@redhat.com, richardcochran@gmail.com
Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org
Hi Experts,
We are using a NXP LS1028A board and facing an issue on icmp:
1. Scenario
Our test sends 2 fragments from VM to our board and expects echo reply but
with no response is present. send a fragment first, 2nd fragment is sent
with delay and test expects "ICMP ip reassembly time exceeded" but there
is also no response.
Detailed description of the testcase:
* Action Send ICMP Echo request packet VM -> device. The packet contains: IP Source Address
* field set to address of VM, IP Destination Address field set to address of the device
* Identification field set to 1, Fragmentation offset field set to 0, MF flags
* set
* Action Wait 100ms
*
* Action Send ICMP Echo request packet VM -> device. The packet contains: IP Source Address
* field set to address of VM, IP Destination Address field set to address of the device
* Identification field set to 1, Fragmentation offset field set to 1, MF flags
* cleared
*
* Result The ICMP echo response is received
2. We are using scapy to do test, the following is scapy packet configuration
f1=Ether(src="26:84:d5:7f:7d:be", dst="7C:72:6E:D4:44:C1")/Dot1Q(prio=0, vlan=984)/IP(src="10.225.32.20", dst="10.225.32.21")/'\x08\x00\xF7\xFF\x00\x00\x00\x00\x00'
f1.flags='MF'
f1.frag=0
f1.id=1
f1.proto='icmp'
// result of f1.show()
###[ Ethernet ]###
dst = 7C:72:6E:D4:44:C1
src = 26:84:d5:7f:7d:be
type = 0x8100
###[ 802.1Q ]###
prio = 0
id = 1
vlan = 984
type = 0x800
###[ IP ]###
version = 4
ihl = None
tos = 0x0
len = None
id = 1
flags = MF
frag = 0
ttl = 64
proto = icmp
chksum = None
src = 10.225.32.20
dst = 10.225.32.21
\options \
###[ Raw ]###
load = '\x08\x00\xf7\xff\x00\x00\x00\x00\x00'
f2=Ether(src="26:84:d5:7f:7d:be", dst="7C:72:6E:D4:44:C1")/Dot1Q(prio=0, vlan=984)/IP(src="10.225.32.20", dst="10.225.32.21")/'\x00\x00\x00\x00\x00\x00\x00\x00'
f2.frag=1
f2.id=1
f2.proto='icmp'
// result of f2.show()
###[ Ethernet ]###
dst = 7C:72:6E:D4:44:C1
src = 26:84:d5:7f:7d:be
type = 0x8100
###[ 802.1Q ]###
prio = 0
id = 1
vlan = 984
type = 0x800
###[ IP ]###
version = 4
ihl = None
tos = 0x0
len = None
id = 1
flags =
frag = 1
ttl = 64
proto = icmp
chksum = None
src = 10.225.32.20
dst = 10.225.32.21
\options \
###[ Raw ]###
load = '\x00\x00\x00\x00\x00\x00\x00\x00'
3. Logs for this issue:
1). From ethtool
We use ethtool to check the hw status, found packet was dropped which pointed out
By drop_yellow_prio_0 field :
......snip......
failing SW:
rx_octets +64
rx_unicast +1
rx_frames_below_65_octets +1
rx_yellow_prio_0 +1
*drop_yellow_prio_0 +1
......snip......
2). From netstat
The outputs of netstat for failing and working cases:
// failing logs
///before test
root@xxx:~# netstat -s
Ip:
Forwarding: 1
3 total packets received
0 forwarded
0 incoming packets discarded
3 incoming packets delivered
3 requests sent out
Icmp:
3 ICMP messages received
0 input ICMP message failed
ICMP input histogram:
echo requests: 3
3 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
echo replies: 3
IcmpMsg:
InType8: 3
OutType0: 3
Tcp:
0 active connection openings
0 passive connection openings
0 failed connection attempts
0 connection resets received
0 connections established
0 segments received
0 segments sent out
0 segments retransmitted
0 bad segments received
0 resets sent
Udp:
0 packets received
0 packets to unknown port received
0 packet receive errors
0 packets sent
0 receive buffer errors
0 send buffer errors
UdpLite:
TcpExt:
0 packet headers predicted
IpExt:
InOctets: 252
OutOctets: 252
InNoECTPkts: 3
///after test
root@xxx:~# netstat -s
Ip:
Forwarding: 1
3 total packets received
0 forwarded
0 incoming packets discarded
3 incoming packets delivered
3 requests sent out
Icmp:
3 ICMP messages received
0 input ICMP message failed
ICMP input histogram:
echo requests: 3
3 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
echo replies: 3
IcmpMsg:
InType8: 3
OutType0: 3
Tcp:
0 active connection openings
0 passive connection openings
0 failed connection attempts
0 connection resets received
0 connections established
0 segments received
0 segments sent out
0 segments retransmitted
0 bad segments received
0 resets sent
Udp:
0 packets received
0 packets to unknown port received
0 packet receive errors
0 packets sent
0 receive buffer errors
0 send buffer errors
UdpLite:
TcpExt:
0 packet headers predicted
IpExt:
InOctets: 252
OutOctets: 252
InNoECTPkts: 3
// working case
///before test
root@xxx:~#netstat -s
Ip:
Forwarding: 1
3 total packets received
0 forwarded
0 incoming packets discarded
3 incoming packets delivered
3 requests sent out
Icmp:
3 ICMP messages received
0 input ICMP message failed
ICMP input histogram:
echo requests: 3
3 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
echo replies: 3
IcmpMsg:
InType8: 3
OutType0: 3
Tcp:
0 active connection openings
0 passive connection openings
0 failed connection attempts
0 connection resets received
0 connections established
0 segments received
0 segments sent out
0 segments retransmitted
0 bad segments received
0 resets sent
Udp:
0 packets received
0 packets to unknown port received
0 packet receive errors
0 packets sent
0 receive buffer errors
0 send buffer errors
UdpLite:
TcpExt:
0 packet headers predicted
IpExt:
InOctets: 252
OutOctets: 252
InNoECTPkts: 3
///after test
root@xxx:~#netstat -s
Ip:
Forwarding: 1
4 total packets received
0 forwarded
0 incoming packets discarded
3 incoming packets delivered
4 requests sent out
******difference******
1 fragments dropped after timeout
1 reassemblies required
1 packet reassemblies failed
Icmp:
3 ICMP messages received
0 input ICMP message failed
ICMP input histogram:
echo requests: 3
4 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
time exceeded: 1
echo replies: 3
IcmpMsg:
InType8: 3
OutType0: 3
OutType11: 1
******difference******
Tcp:
0 active connection openings
0 passive connection openings
0 failed connection attempts
0 connection resets received
0 connections established
0 segments received
0 segments sent out
0 segments retransmitted
0 bad segments received
0 resets sent
Udp:
0 packets received
0 packets to unknown port received
0 packet receive errors
0 packets sent
0 receive buffer errors
0 send buffer errors
UdpLite:
TcpExt:
0 packet headers predicted
IpExt:
InOctets: 294
OutOctets: 308
InNoECTPkts: 4
3). From pcap file(the pcap was collected on the senderside (VM))
Frame 1: 64 bytes on wire (512 bits), 64 bytes captured (512 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Dec 16, 2022 16:36:42.803640000 CST
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1671179802.803640000 seconds
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 64 bytes (512 bits)
Capture Length: 64 bytes (512 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:vlan:ethertype:arp]
Ethernet II, Src: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f), Dst: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
Destination: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
Address: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Address: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: 802.1Q Virtual LAN (0x8100)
802.1Q Virtual LAN, PRI: 6, DEI: 0, ID: 981
110. .... .... .... = Priority: Internetwork Control (6)
...0 .... .... .... = DEI: Ineligible
.... 0011 1101 0101 = ID: 981
Type: ARP (0x0806)
Padding: 0000000000000000000000000000
Trailer: 00000000
Address Resolution Protocol (request)
Hardware type: Ethernet (1)
Protocol type: IPv4 (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: request (1)
Sender MAC address: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Sender IP address: 10.225.32.5
Target MAC address: 00:00:00_00:00:00 (00:00:00:00:00:00)
Target IP address: 10.225.32.4
Frame 2: 46 bytes on wire (368 bits), 46 bytes captured (368 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Dec 16, 2022 16:36:42.803664000 CST
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1671179802.803664000 seconds
[Time delta from previous captured frame: 0.000024000 seconds]
[Time delta from previous displayed frame: 0.000024000 seconds]
[Time since reference or first frame: 0.000024000 seconds]
Frame Number: 2
Frame Length: 46 bytes (368 bits)
Capture Length: 46 bytes (368 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:vlan:ethertype:arp]
Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Destination: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Address: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
Address: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: 802.1Q Virtual LAN (0x8100)
802.1Q Virtual LAN, PRI: 0, DEI: 0, ID: 981
000. .... .... .... = Priority: Best Effort (default) (0)
...0 .... .... .... = DEI: Ineligible
.... 0011 1101 0101 = ID: 981
Type: ARP (0x0806)
Address Resolution Protocol (reply)
Hardware type: Ethernet (1)
Protocol type: IPv4 (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: reply (2)
Sender MAC address: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
Sender IP address: 10.225.32.4
Target MAC address: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Target IP address: 10.225.32.5
Frame 3: 47 bytes on wire (376 bits), 47 bytes captured (376 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Dec 16, 2022 16:36:49.915062000 CST
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1671179809.915062000 seconds
[Time delta from previous captured frame: 7.111398000 seconds]
[Time delta from previous displayed frame: 7.111398000 seconds]
[Time since reference or first frame: 7.111422000 seconds]
Frame Number: 3
Frame Length: 47 bytes (376 bits)
Capture Length: 47 bytes (376 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:vlan:ethertype:ip:data]
Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Destination: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Address: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
Address: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: 802.1Q Virtual LAN (0x8100)
802.1Q Virtual LAN, PRI: 0, DEI: 1, ID: 981
000. .... .... .... = Priority: Best Effort (default) (0)
...1 .... .... .... = DEI: Eligible
.... 0011 1101 0101 = ID: 981
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.225.32.4, Dst: 10.225.32.5
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 29
Identification: 0x0001 (1)
Flags: 0x2000, More fragments
0... .... .... .... = Reserved bit: Not set
.0.. .... .... .... = Don't fragment: Not set
..1. .... .... .... = More fragments: Set
Fragment offset: 0
Time to live: 64
Protocol: ICMP (1)
Header checksum: 0x0515 [validation disabled]
[Header checksum status: Unverified]
Source: 10.225.32.4
Destination: 10.225.32.5
Data (9 bytes)
0000 08 00 f7 ff 00 00 00 00 00 .........
Data: 0800f7ff0000000000
[Length: 9]
Frame 4: 46 bytes on wire (368 bits), 46 bytes captured (368 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Dec 16, 2022 16:36:50.063589000 CST
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1671179810.063589000 seconds
[Time delta from previous captured frame: 0.148527000 seconds]
[Time delta from previous displayed frame: 0.148527000 seconds]
[Time since reference or first frame: 7.259949000 seconds]
Frame Number: 4
Frame Length: 46 bytes (368 bits)
Capture Length: 46 bytes (368 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:vlan:ethertype:ip:icmp:data]
Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Destination: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Address: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
Address: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: 802.1Q Virtual LAN (0x8100)
802.1Q Virtual LAN, PRI: 0, DEI: 1, ID: 981
000. .... .... .... = Priority: Best Effort (default) (0)
...1 .... .... .... = DEI: Eligible
.... 0011 1101 0101 = ID: 981
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.225.32.4, Dst: 10.225.32.5
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 28
Identification: 0x0001 (1)
Flags: 0x0001
0... .... .... .... = Reserved bit: Not set
.0.. .... .... .... = Don't fragment: Not set
..0. .... .... .... = More fragments: Not set
Fragment offset: 8
Time to live: 64
Protocol: ICMP (1)
Header checksum: 0x2515 [validation disabled]
[Header checksum status: Unverified]
Source: 10.225.32.4
Destination: 10.225.32.5
[2 IPv4 Fragments (16 bytes): #3(9), #4(8)]
[Frame: 3, payload: 0-8 (9 bytes)]
[Frame: 4, payload: 8-15 (8 bytes)]
[Fragment overlap: True]
[Fragment count: 2]
[Reassembled IPv4 length: 16]
[Reassembled IPv4 data: 0800f7ff000000000000000000000000]
Internet Control Message Protocol
Type: 8 (Echo (ping) request)
Code: 0
Checksum: 0xf7ff [correct]
[Checksum Status: Good]
Identifier (BE): 0 (0x0000)
Identifier (LE): 0 (0x0000)
Sequence number (BE): 0 (0x0000)
Sequence number (LE): 0 (0x0000)
Data (8 bytes)
0000 00 00 00 00 00 00 00 00 ........
Data: 0000000000000000
[Length: 8]
4). What we've found so far
According binary search, we found out the following commit causes this issue:
a4ae997adcbd("net: mscc: ocelot: initialize watermarks to sane defaults").
Without this commit the test case was passed.
Could you please take a look? Please let me know if you need more debug info.
Regards,
Xiongwei
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Response error to fragmented ICMP echo request
2023-03-07 12:11 Response error to fragmented ICMP echo request Song, Xiongwei
@ 2023-03-07 12:35 ` Vladimir Oltean
2023-03-07 12:56 ` Song, Xiongwei
0 siblings, 1 reply; 5+ messages in thread
From: Vladimir Oltean @ 2023-03-07 12:35 UTC (permalink / raw)
To: Song, Xiongwei
Cc: claudiu.manoil@nxp.com, alexandre.belloni@bootlin.com,
UNGLinuxDriver@microchip.com, davem@davemloft.net,
edumazet@google.com, kuba@kernel.org, pabeni@redhat.com,
richardcochran@gmail.com, linux-kernel@vger.kernel.org,
netdev@vger.kernel.org
Hello Xiongwei,
On Tue, Mar 07, 2023 at 12:11:52PM +0000, Song, Xiongwei wrote:
> ......snip......
> failing SW:
> rx_octets +64
> rx_unicast +1
> rx_frames_below_65_octets +1
> rx_yellow_prio_0 +1
> *drop_yellow_prio_0 +1
> ......snip......
>
> 3). From pcap file(the pcap was collected on the senderside (VM))
>
> Frame 1: 64 bytes on wire (512 bits), 64 bytes captured (512 bits)
> Ethernet II, Src: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f), Dst: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
> Destination: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
> Source: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
> Type: 802.1Q Virtual LAN (0x8100)
> 802.1Q Virtual LAN, PRI: 6, DEI: 0, ID: 981
> 110. .... .... .... = Priority: Internetwork Control (6)
> ...0 .... .... .... = DEI: Ineligible
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> .... 0011 1101 0101 = ID: 981
> Type: ARP (0x0806)
> Padding: 0000000000000000000000000000
> Trailer: 00000000
>
> Frame 2: 46 bytes on wire (368 bits), 46 bytes captured (368 bits)
> Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
> 802.1Q Virtual LAN, PRI: 0, DEI: 0, ID: 981
> 000. .... .... .... = Priority: Best Effort (default) (0)
> ...0 .... .... .... = DEI: Ineligible
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> .... 0011 1101 0101 = ID: 981
> Type: ARP (0x0806)
>
> Frame 3: 47 bytes on wire (376 bits), 47 bytes captured (376 bits)
> Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
> 802.1Q Virtual LAN, PRI: 0, DEI: 1, ID: 981
> 000. .... .... .... = Priority: Best Effort (default) (0)
> ...1 .... .... .... = DEI: Eligible
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> .... 0011 1101 0101 = ID: 981
> Type: IPv4 (0x0800)
>
> Frame 4: 46 bytes on wire (368 bits), 46 bytes captured (368 bits)
> Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
> 802.1Q Virtual LAN, PRI: 0, DEI: 1, ID: 981
> 000. .... .... .... = Priority: Best Effort (default) (0)
> ...1 .... .... .... = DEI: Eligible
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> .... 0011 1101 0101 = ID: 981
> Type: IPv4 (0x0800)
>
> 4). What we've found so far
>
> According binary search, we found out the following commit causes this issue:
> a4ae997adcbd("net: mscc: ocelot: initialize watermarks to sane defaults").
> Without this commit the test case was passed.
>
> Could you please take a look? Please let me know if you need more debug info.
I've marked the DEI values in the message you posted above.
Commit a4ae997adcbd ("net: mscc: ocelot: initialize watermarks to sane defaults")
tells the hardware to not allow frames with DEI=1 consume from the shared switch
resources (buffers / frame references) by default. Drop Eligible Indicator = 1
means "eligible for dropping". The only chance for DEI=1 frames to not be dropped
is to set up a resource reservation for that stream, via the devlink-sb command.
Frames 3 and 4 are sent with DEI=1 and are dropped, frames 1 and 2 are
sent with DEI=0 and are not dropped. I'm not sure if varying the DEI
field is part of the intentions of the test? Is there any RFC which says
that IP fragments over VLAN should use DEI=1, or some other reason?
^ permalink raw reply [flat|nested] 5+ messages in thread
* RE: Response error to fragmented ICMP echo request
2023-03-07 12:35 ` Vladimir Oltean
@ 2023-03-07 12:56 ` Song, Xiongwei
2023-03-07 13:18 ` Vladimir Oltean
0 siblings, 1 reply; 5+ messages in thread
From: Song, Xiongwei @ 2023-03-07 12:56 UTC (permalink / raw)
To: Vladimir Oltean
Cc: claudiu.manoil@nxp.com, alexandre.belloni@bootlin.com,
UNGLinuxDriver@microchip.com, davem@davemloft.net,
edumazet@google.com, kuba@kernel.org, pabeni@redhat.com,
richardcochran@gmail.com, linux-kernel@vger.kernel.org,
netdev@vger.kernel.org
Hello Vladimir,
Thanks for the quick response.
> -----Original Message-----
> From: Vladimir Oltean <vladimir.oltean@nxp.com>
> Sent: Tuesday, March 7, 2023 8:35 PM
> To: Song, Xiongwei <Xiongwei.Song@windriver.com>
> Cc: claudiu.manoil@nxp.com; alexandre.belloni@bootlin.com;
> UNGLinuxDriver@microchip.com; davem@davemloft.net; edumazet@google.com;
> kuba@kernel.org; pabeni@redhat.com; richardcochran@gmail.com; linux-
> kernel@vger.kernel.org; netdev@vger.kernel.org
> Subject: Re: Response error to fragmented ICMP echo request
>
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the
> content is safe.
>
> Hello Xiongwei,
>
> On Tue, Mar 07, 2023 at 12:11:52PM +0000, Song, Xiongwei wrote:
> > ......snip......
> > failing SW:
> > rx_octets +64
> > rx_unicast +1
> > rx_frames_below_65_octets +1
> > rx_yellow_prio_0 +1
> > *drop_yellow_prio_0 +1
> > ......snip......
> >
> > 3). From pcap file(the pcap was collected on the senderside (VM))
> >
> > Frame 1: 64 bytes on wire (512 bits), 64 bytes captured (512 bits)
> > Ethernet II, Src: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f), Dst: aa:3a:b3:e7:67:5c
> (aa:3a:b3:e7:67:5c)
> > Destination: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
> > Source: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
> > Type: 802.1Q Virtual LAN (0x8100)
> > 802.1Q Virtual LAN, PRI: 6, DEI: 0, ID: 981
> > 110. .... .... .... = Priority: Internetwork Control (6)
> > ...0 .... .... .... = DEI: Ineligible
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> > .... 0011 1101 0101 = ID: 981
> > Type: ARP (0x0806)
> > Padding: 0000000000000000000000000000
> > Trailer: 00000000
> >
> > Frame 2: 46 bytes on wire (368 bits), 46 bytes captured (368 bits)
> > Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f
> (7c:72:6e:d4:44:5f)
> > 802.1Q Virtual LAN, PRI: 0, DEI: 0, ID: 981
> > 000. .... .... .... = Priority: Best Effort (default) (0)
> > ...0 .... .... .... = DEI: Ineligible
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> > .... 0011 1101 0101 = ID: 981
> > Type: ARP (0x0806)
> >
> > Frame 3: 47 bytes on wire (376 bits), 47 bytes captured (376 bits)
> > Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f
> (7c:72:6e:d4:44:5f)
> > 802.1Q Virtual LAN, PRI: 0, DEI: 1, ID: 981
> > 000. .... .... .... = Priority: Best Effort (default) (0)
> > ...1 .... .... .... = DEI: Eligible
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> > .... 0011 1101 0101 = ID: 981
> > Type: IPv4 (0x0800)
> >
> > Frame 4: 46 bytes on wire (368 bits), 46 bytes captured (368 bits)
> > Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f
> (7c:72:6e:d4:44:5f)
> > 802.1Q Virtual LAN, PRI: 0, DEI: 1, ID: 981
> > 000. .... .... .... = Priority: Best Effort (default) (0)
> > ...1 .... .... .... = DEI: Eligible
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> > .... 0011 1101 0101 = ID: 981
> > Type: IPv4 (0x0800)
> >
> > 4). What we've found so far
> >
> > According binary search, we found out the following commit causes this issue:
> > a4ae997adcbd("net: mscc: ocelot: initialize watermarks to sane defaults").
> > Without this commit the test case was passed.
> >
> > Could you please take a look? Please let me know if you need more debug info.
>
> I've marked the DEI values in the message you posted above.
>
> Commit a4ae997adcbd ("net: mscc: ocelot: initialize watermarks to sane defaults")
> tells the hardware to not allow frames with DEI=1 consume from the shared switch
> resources (buffers / frame references) by default. Drop Eligible Indicator = 1
> means "eligible for dropping". The only chance for DEI=1 frames to not be dropped
> is to set up a resource reservation for that stream, via the devlink-sb command.
Oh.., thanks for the detailed explanation.
>
> Frames 3 and 4 are sent with DEI=1 and are dropped, frames 1 and 2 are
> sent with DEI=0 and are not dropped. I'm not sure if varying the DEI
> field is part of the intentions of the test? Is there any RFC which says
> that IP fragments over VLAN should use DEI=1, or some other reason?
I didn't notice that. Let me check the test why set DEI=1.
Have a good day.
Regards,
Xiongwei
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Response error to fragmented ICMP echo request
2023-03-07 12:56 ` Song, Xiongwei
@ 2023-03-07 13:18 ` Vladimir Oltean
2023-03-08 6:00 ` Song, Xiongwei
0 siblings, 1 reply; 5+ messages in thread
From: Vladimir Oltean @ 2023-03-07 13:18 UTC (permalink / raw)
To: Song, Xiongwei
Cc: claudiu.manoil@nxp.com, alexandre.belloni@bootlin.com,
UNGLinuxDriver@microchip.com, davem@davemloft.net,
edumazet@google.com, kuba@kernel.org, pabeni@redhat.com,
richardcochran@gmail.com, linux-kernel@vger.kernel.org,
netdev@vger.kernel.org
On Tue, Mar 07, 2023 at 12:56:17PM +0000, Song, Xiongwei wrote:
> > Frames 3 and 4 are sent with DEI=1 and are dropped, frames 1 and 2 are
> > sent with DEI=0 and are not dropped. I'm not sure if varying the DEI
> > field is part of the intentions of the test? Is there any RFC which says
> > that IP fragments over VLAN should use DEI=1, or some other reason?
>
> I didn't notice that. Let me check the test why set DEI=1.
Ok. It would be good to have an answer to this, because one of the
assumptions of that patch was that whomever sets DEI=1 doesn't get to
complain that their packets are *actually* dropped :)
FWIW, if you do need to set up a reservation for traffic received on a
port, section 8.6.3.3.15 Buffer reservation watermarks (page 817) should
help with this:
https://www.nxp.com/docs/en/user-guide/LLDPUG_RevL5.15.71-2.2.0.pdf
^ permalink raw reply [flat|nested] 5+ messages in thread
* RE: Response error to fragmented ICMP echo request
2023-03-07 13:18 ` Vladimir Oltean
@ 2023-03-08 6:00 ` Song, Xiongwei
0 siblings, 0 replies; 5+ messages in thread
From: Song, Xiongwei @ 2023-03-08 6:00 UTC (permalink / raw)
To: Vladimir Oltean
Cc: claudiu.manoil@nxp.com, alexandre.belloni@bootlin.com,
UNGLinuxDriver@microchip.com, davem@davemloft.net,
edumazet@google.com, kuba@kernel.org, pabeni@redhat.com,
richardcochran@gmail.com, linux-kernel@vger.kernel.org,
netdev@vger.kernel.org
Hi Vladimir,
> On Tue, Mar 07, 2023 at 12:56:17PM +0000, Song, Xiongwei wrote:
> > > Frames 3 and 4 are sent with DEI=1 and are dropped, frames 1 and 2 are
> > > sent with DEI=0 and are not dropped. I'm not sure if varying the DEI
> > > field is part of the intentions of the test? Is there any RFC which says
> > > that IP fragments over VLAN should use DEI=1, or some other reason?
> >
> > I didn't notice that. Let me check the test why set DEI=1.
>
> Ok. It would be good to have an answer to this, because one of the
> assumptions of that patch was that whomever sets DEI=1 doesn't get to
> complain that their packets are *actually* dropped :)
We found the problem in the test:
=====================
f2=Ether(src="26:84:d5:7f:7d:be", dst="7C:72:6E:D4:44:C1")/Dot1Q(prio=0, vlan=984)/IP(src="10.225.32.20", dst="10.225.32.21")/'\x00\x00\x00\x00\x00\x00\x00\x00'
f2.frag=1
===>f2.id=1
f2.proto='icmp'
// result of f2.show()
###[ Ethernet ]###
dst = 7C:72:6E:D4:44:C1
src = 26:84:d5:7f:7d:be
type = 0x8100
###[ 802.1Q ]###
prio = 0
id = 1
vlan = 984
type = 0x800
###[ IP ]###
version = 4
ihl = None
tos = 0x0
len = None
id = 1
flags =
frag = 1
ttl = 64
proto = icmp
chksum = None
src = 10.225.32.20
dst = 10.225.32.21
\options \
###[ Raw ]###
load = '\x00\x00\x00\x00\x00\x00\x00\x00'
=================
This sets both the L2 & L3 id. L2 id is actually DEI.
We updated the script to only set L3 id and now the test passes.
>
> FWIW, if you do need to set up a reservation for traffic received on a
> port, section 8.6.3.3.15 Buffer reservation watermarks (page 817) should
> help with this:
> https://www.nxp.com/docs/en/user-guide/LLDPUG_RevL5.15.71-2.2.0.pdf
Thank you so much. We don't need to set up a reservation for now, but good
to know this.
Regards,
Xiongwei
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-03-08 6:00 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-03-07 12:11 Response error to fragmented ICMP echo request Song, Xiongwei
2023-03-07 12:35 ` Vladimir Oltean
2023-03-07 12:56 ` Song, Xiongwei
2023-03-07 13:18 ` Vladimir Oltean
2023-03-08 6:00 ` Song, Xiongwei
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).