[Qemu-devel] slirp, incoming packets get truncated

[Qemu-devel] slirp, incoming packets get truncated

[Chris Heinze]

with slirp, incoming packets to the guest get truncated (outgoing seem to work correctly).

this has been observed on a x86_64 machine with devuan2 and qemu 4.1.0.

e.g. on the host:
root@host:~# qemu-system-x86_64 -net nic,model=virtio -net user,hostfwd=udp::19003-:19003
root@host:~# head -c 9000 /dev/zero | nc -u 127.0.0.1 19003

on the guest:
root@guest:~# tcpdump -ni eth0 port 19003
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
16:49:39.430959 IP 10.0.2.2.33294 > 10.0.2.15.19003: UDP, bad length 9000 > 1472

these packets obviously are truncated to 1472 bytes.

i tried to change slirp/src/if.h to: 

#define IF_MTU 9000
#define IF_MRU 9000

but the resulting qemu-system-x86_64 binary did not behave differently.

-- 

Mit freundlichen Grüßen / Kind regards,

Chris Heinze
*Network-/System-/DevOps-Engineer*

*PreciBake GmbH*
Gollierstr. 70
80339 München
Deutschland / Germany
*P:* +49-(0)-89-2154895-30 <+49.89.2154895.30>
*F:* +49-(0)-89-2154895-99
*E:* c.heinze@precibake.com
*W:* www.precibake.com

<http://www.precibake.com/>

Handelsregister / Commercial Register:
Amtsgericht München, HRB 206356
Steuer-Nr. / Tax-ID143/172/11644
Umsatzsteuer-ID / VAT registration number: DE290040302
Geschäftsführer / Managing Director: Dr. Ingo Stork-Wersborg

Der Inhalt dieser E-Mail ist ausschliesslich fuer den/die bezeichneten Empfaenger bestimmt. Wenn Sie nicht der vorgesehene Adressat dieser E-Mail oder dessen Vertreter sein sollten, so beachten Sie bitte, dass jede Form der Kenntnisnahme, Veroeffentlichung, Vervielfaeltigung oder Weitergabe des Inhalts dieser E-Mail unzulaessig ist. Wir bitten Sie, sich in diesem Fall mit dem Absender der E-Mail in Verbindung zu setzen. Wir moechten Sie ausserdem darauf hinweisen, dass die Kommunikation per E-Mail ueber das Internet unsicher ist, da fuer unberechtigte Dritte grundsaetzlich die Moeglichkeit der Kenntnisnahme und Manipulation besteht.
The information contained in this email is intended solely for the recipient(s). Access to this email by anyone else is unauthorized. If you are not the intended recipient, any form of disclosure, reproduction, distribution or any action taken or refrained from in reliance on it, is prohibited. Please notify the sender of this email immediately. We also like to inform you that communication via email over the internet is insecure because third parties may have the possibility to access and manipulate emails.

Re: [Qemu-devel] slirp, incoming packets get truncated

[Samuel Thibault]

Hello,

Chris Heinze, le mar. 03 sept. 2019 17:02:15 +0200, a ecrit:
> on the guest:
> root@guest:~# tcpdump -ni eth0 port 19003
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
> 16:49:39.430959 IP 10.0.2.2.33294 > 10.0.2.15.19003: UDP, bad length 9000 > 1472

Just to be sure: did you configure your guest mru to 9000 ? Does the
emulated network hardware card support jumbo frames?

Samuel

Re: [Qemu-devel] slirp, incoming packets get truncated

[Chris Heinze]

hi,

On 9/6/19 11:59 AM, Samuel Thibault wrote:
[...]
>> 16:49:39.430959 IP 10.0.2.2.33294 > 10.0.2.15.19003: UDP, bad length 9000 > 1472
> 
> Just to be sure: did you configure your guest mru to 9000 ? Does the
> emulated network hardware card support jumbo frames?

hm! all interfaces (host and guest) are configured to a mtu of 9000 (except for lo which is always 64k). there's no serial/ppp involved, so i can not set a mru (for non-p2p links the 'mru' kindof should be the mtu of the respective other interface of traffic?). i found no way to configure slirp (in qemu) itself except for the DEFINEs in the if.h.
i tried all available nic-emulations, then continued with virtio, which is one that accepts the mtu settings (and sending jumbo frames out does work). the other nics where jumbo frames can be configured do not behave differently.
i'm not aware of any (canonical) parameters to set a max size for received packets - i guess when implementing an interface one could decide to truncate according to the mtu (but i guess i would not and i would also argue against it, i guess when i need a mru size for fixed receive-buffers (which i'd try to avoid) i'd assume the mtu). afaik even with ppp mru is just used to signal a max mtu to set to the peer.

-- 

Mit freundlichen Grüßen / Kind regards,

Chris Heinze
*Network-/System-/DevOps-Engineer*

*PreciBake GmbH*
Gollierstr. 70
80339 München
Deutschland / Germany
*P:* +49-(0)-89-2154895-30 <+49.89.2154895.30>
*F:* +49-(0)-89-2154895-99
*E:* c.heinze@precibake.com
*W:* www.precibake.com

<http://www.precibake.com/>

Handelsregister / Commercial Register:
Amtsgericht München, HRB 206356
Steuer-Nr. / Tax-ID143/172/11644
Umsatzsteuer-ID / VAT registration number: DE290040302
Geschäftsführer / Managing Director: Dr. Ingo Stork-Wersborg

Der Inhalt dieser E-Mail ist ausschliesslich fuer den/die bezeichneten Empfaenger bestimmt. Wenn Sie nicht der vorgesehene Adressat dieser E-Mail oder dessen Vertreter sein sollten, so beachten Sie bitte, dass jede Form der Kenntnisnahme, Veroeffentlichung, Vervielfaeltigung oder Weitergabe des Inhalts dieser E-Mail unzulaessig ist. Wir bitten Sie, sich in diesem Fall mit dem Absender der E-Mail in Verbindung zu setzen. Wir moechten Sie ausserdem darauf hinweisen, dass die Kommunikation per E-Mail ueber das Internet unsicher ist, da fuer unberechtigte Dritte grundsaetzlich die Moeglichkeit der Kenntnisnahme und Manipulation besteht.
The information contained in this email is intended solely for the recipient(s). Access to this email by anyone else is unauthorized. If you are not the intended recipient, any form of disclosure, reproduction, distribution or any action taken or refrained from in reliance on it, is prohibited. Please notify the sender of this email immediately. We also like to inform you that communication via email over the internet is insecure because third parties may have the possibility to access and manipulate emails.

Re: [Qemu-devel] slirp, incoming packets get truncated

[Samuel Thibault]

Chris Heinze, le ven. 06 sept. 2019 12:54:24 +0200, a ecrit:
> i'm not aware of any (canonical) parameters to set a max size for received packets

Ah, right.

> i found no way to configure slirp (in qemu) itself except for the DEFINEs in the if.h.

Yes, that's where it is to be configured atm. IF_MRU is however only
used for setting the TCP mss, there is no code that is supposed to
truncate packets etc., that's why I'm really surprised that there is
such truncation happening.

Samuel

Re: [Qemu-devel] slirp, incoming packets get truncated

[Samuel Thibault]

[-- Attachment #1: Type: text/plain, Size: 1207 bytes --]

Hello,

As usual, several things here.

Chris Heinze, le mar. 03 sept. 2019 17:02:15 +0200, a ecrit:
> root@guest:~# tcpdump -ni eth0 port 19003
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
> 16:49:39.430959 IP 10.0.2.2.33294 > 10.0.2.15.19003: UDP, bad length 9000 > 1472

tcpdump seems to be showing dumb output here. The packet is fragmented
by slirp, which makes tcpdump confused and show only the first
fragment. If you let tcpdump print everything, you will see the other
fragments. In reality, everything is going fine here.

> i tried to change slirp/src/if.h to: 
> 
> #define IF_MTU 9000
> #define IF_MRU 9000
> 
> but the resulting qemu-system-x86_64 binary did not behave differently.

Did you explicitly remove the qemu-system-x86_64 binary? As mentioned
previously on the list, there seems to be a missing Makefile dependency.

Now, with MTU set to 9000, the packets just don't go at all. Could you
try the attached patch? The lowest layer of slirp was indeed limited to
1600-byte frames for no good reason. With this and the virtio driver, I
could exchange 9000-byte packets.

Samuel

[-- Attachment #2: patch --]
[-- Type: text/plain, Size: 1365 bytes --]

diff --git a/src/slirp.c b/src/slirp.c
index b0194cb..3fd6f68 100644
--- a/src/slirp.c
+++ b/src/slirp.c
@@ -890,20 +890,22 @@ static int if_encap6(Slirp *slirp, struct mbuf *ifm, struct ethhdr *eh,
  */
 int if_encap(Slirp *slirp, struct mbuf *ifm)
 {
-    uint8_t buf[1600];
-    struct ethhdr *eh = (struct ethhdr *)buf;
+    uint8_t *buf;
+    struct ethhdr *eh;
     uint8_t ethaddr[ETH_ALEN];
     const struct ip *iph = (const struct ip *)ifm->m_data;
     int ret;
 
-    if (ifm->m_len + ETH_HLEN > sizeof(buf)) {
-        return 1;
-    }
+    buf = g_malloc(ifm->m_len + ETH_HLEN);
+    if (!buf)
+        return 0;
+    eh = (struct ethhdr *)buf;
 
     switch (iph->ip_v) {
     case IPVERSION:
         ret = if_encap4(slirp, ifm, eh, ethaddr);
         if (ret < 2) {
+            g_free(buf);
             return ret;
         }
         break;
@@ -911,6 +913,7 @@ int if_encap(Slirp *slirp, struct mbuf *ifm)
     case IP6VERSION:
         ret = if_encap6(slirp, ifm, eh, ethaddr);
         if (ret < 2) {
+            g_free(buf);
             return ret;
         }
         break;
@@ -929,6 +932,7 @@ int if_encap(Slirp *slirp, struct mbuf *ifm)
               eh->h_dest[5]);
     memcpy(buf + sizeof(struct ethhdr), ifm->m_data, ifm->m_len);
     slirp_send_packet_all(slirp, buf, ifm->m_len + ETH_HLEN);
+    g_free(buf);
     return 1;
 }
 

Re: [Qemu-devel] slirp, incoming packets get truncated

[Chris Heinze]

hi,

On 9/8/19 1:29 AM, Samuel Thibault wrote:
>> 16:49:39.430959 IP 10.0.2.2.33294 > 10.0.2.15.19003: UDP, bad length 9000 > 1472
> 
> tcpdump seems to be showing dumb output here. The packet is fragmented
> by slirp, which makes tcpdump confused and show only the first
> fragment. If you let tcpdump print everything, you will see the other
> fragments. In reality, everything is going fine here.

oh ok. for some reason the receiving process is not happy with these packets anyway. thanks for the hint, i'll have to look into that deeper then.

>> #define IF_MRU 9000
>>
>> but the resulting qemu-system-x86_64 binary did not behave differently.
> 
> Did you explicitly remove the qemu-system-x86_64 binary? As mentioned

yup, i mean no, but i initially tested by copying binary and executing it directly, so it was the patched binary.

> try the attached patch? The lowest layer of slirp was indeed limited to
> 1600-byte frames for no good reason. With this and the virtio driver, I
> could exchange 9000-byte packets.

ah, thanks! looks good:

root@test-host:~# head -c 9000 /dev/zero | nc -u 127.0.0.1 19003
root@test-guest:~# tcpdump -ni eth0 port 19003
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
12:20:56.484044 IP 10.0.2.2.37331 > 10.0.2.15.19003: UDP, bad length 9000 > 8968

root@test-host:~# head -c 8000 /dev/zero | nc -u 127.0.0.1 19003
root@test-guest:~# tcpdump -ni eth0 port 19003
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
12:25:43.967476 IP 10.0.2.2.47243 > 10.0.2.15.19003: UDP, length 8000

...with mtu 9000 everywhere. mtu is l2, and complete frame size i assume, so: ip header = 20B, dst-mac + src-mac = 12B

great, thanx 1e6!

-- 

Mit freundlichen Grüßen / Kind regards,

Chris Heinze
*Network-/System-/DevOps-Engineer*

*PreciBake GmbH*
Gollierstr. 70
80339 München
Deutschland / Germany
*P:* +49-(0)-89-2154895-30 <+49.89.2154895.30>
*F:* +49-(0)-89-2154895-99
*E:* c.heinze@precibake.com
*W:* www.precibake.com

<http://www.precibake.com/>

Handelsregister / Commercial Register:
Amtsgericht München, HRB 206356
Steuer-Nr. / Tax-ID143/172/11644
Umsatzsteuer-ID / VAT registration number: DE290040302
Geschäftsführer / Managing Director: Dr. Ingo Stork-Wersborg

Der Inhalt dieser E-Mail ist ausschliesslich fuer den/die bezeichneten Empfaenger bestimmt. Wenn Sie nicht der vorgesehene Adressat dieser E-Mail oder dessen Vertreter sein sollten, so beachten Sie bitte, dass jede Form der Kenntnisnahme, Veroeffentlichung, Vervielfaeltigung oder Weitergabe des Inhalts dieser E-Mail unzulaessig ist. Wir bitten Sie, sich in diesem Fall mit dem Absender der E-Mail in Verbindung zu setzen. Wir moechten Sie ausserdem darauf hinweisen, dass die Kommunikation per E-Mail ueber das Internet unsicher ist, da fuer unberechtigte Dritte grundsaetzlich die Moeglichkeit der Kenntnisnahme und Manipulation besteht.
The information contained in this email is intended solely for the recipient(s). Access to this email by anyone else is unauthorized. If you are not the intended recipient, any form of disclosure, reproduction, distribution or any action taken or refrained from in reliance on it, is prohibited. Please notify the sender of this email immediately. We also like to inform you that communication via email over the internet is insecure because third parties may have the possibility to access and manipulate emails.

Re: [Qemu-devel] slirp, incoming packets get truncated

[Philippe Mathieu-Daudé]

Hi Samuel,

On 9/8/19 1:29 AM, Samuel Thibault wrote:
>
> Now, with MTU set to 9000, the packets just don't go at all. Could you
> try the attached patch? The lowest layer of slirp was indeed limited to
> 1600-byte frames for no good reason. With this and the virtio driver, I
> could exchange 9000-byte packets.
> 
> Samuel
> 
> diff --git a/src/slirp.c b/src/slirp.c
> index b0194cb..3fd6f68 100644
> --- a/src/slirp.c
> +++ b/src/slirp.c
> @@ -890,20 +890,22 @@ static int if_encap6(Slirp *slirp, struct mbuf
*ifm, struct ethhdr *eh,
>   */
>  int if_encap(Slirp *slirp, struct mbuf *ifm)
>  {
> -    uint8_t buf[1600];
> -    struct ethhdr *eh = (struct ethhdr *)buf;
> +    uint8_t *buf;
> +    struct ethhdr *eh;
>      uint8_t ethaddr[ETH_ALEN];
>      const struct ip *iph = (const struct ip *)ifm->m_data;
>      int ret;
>
> -    if (ifm->m_len + ETH_HLEN > sizeof(buf)) {
> -        return 1;
> -    }
> +    buf = g_malloc(ifm->m_len + ETH_HLEN);

Since g_malloc() aborts on failure, you want g_try_malloc() here.

> +    if (!buf)
> +        return 0;
> +    eh = (struct ethhdr *)buf;
>
>      switch (iph->ip_v) {
>      case IPVERSION:
>          ret = if_encap4(slirp, ifm, eh, ethaddr);
>          if (ret < 2) {
> +            g_free(buf);
>              return ret;
>          }
>          break;
> @@ -911,6 +913,7 @@ int if_encap(Slirp *slirp, struct mbuf *ifm)
>      case IP6VERSION:
>          ret = if_encap6(slirp, ifm, eh, ethaddr);
>          if (ret < 2) {
> +            g_free(buf);
>              return ret;
>          }
>          break;
> @@ -929,6 +932,7 @@ int if_encap(Slirp *slirp, struct mbuf *ifm)
>                eh->h_dest[5]);
>      memcpy(buf + sizeof(struct ethhdr), ifm->m_data, ifm->m_len);
>      slirp_send_packet_all(slirp, buf, ifm->m_len + ETH_HLEN);
> +    g_free(buf);
>      return 1;
>  }

Eventually easier to review using less exit points, i.e.:

-- >8 --
@@ -903,16 +903,10 @@ int if_encap(Slirp *slirp, struct mbuf *ifm)
     switch (iph->ip_v) {
     case IPVERSION:
         ret = if_encap4(slirp, ifm, eh, ethaddr);
-        if (ret < 2) {
-            return ret;
-        }
         break;

     case IP6VERSION:
         ret = if_encap6(slirp, ifm, eh, ethaddr);
-        if (ret < 2) {
-            return ret;
-        }
         break;

     default:
@@ -920,16 +914,21 @@ int if_encap(Slirp *slirp, struct mbuf *ifm)
         break;
     }

-    memcpy(eh->h_dest, ethaddr, ETH_ALEN);
-    DEBUG_ARG("src = %02x:%02x:%02x:%02x:%02x:%02x", eh->h_source[0],
-              eh->h_source[1], eh->h_source[2], eh->h_source[3],
-              eh->h_source[4], eh->h_source[5]);
-    DEBUG_ARG("dst = %02x:%02x:%02x:%02x:%02x:%02x", eh->h_dest[0],
-              eh->h_dest[1], eh->h_dest[2], eh->h_dest[3], eh->h_dest[4],
-              eh->h_dest[5]);
-    memcpy(buf + sizeof(struct ethhdr), ifm->m_data, ifm->m_len);
-    slirp_send_packet_all(slirp, buf, ifm->m_len + ETH_HLEN);
-    return 1;
+    if (ret >= 2) {
+        memcpy(eh->h_dest, ethaddr, ETH_ALEN);
+        DEBUG_ARG("src = %02x:%02x:%02x:%02x:%02x:%02x", eh->h_source[0],
+                  eh->h_source[1], eh->h_source[2], eh->h_source[3],
+                  eh->h_source[4], eh->h_source[5]);
+        DEBUG_ARG("dst = %02x:%02x:%02x:%02x:%02x:%02x", eh->h_dest[0],
+                  eh->h_dest[1], eh->h_dest[2], eh->h_dest[3],
eh->h_dest[4],
+                  eh->h_dest[5]);
+        memcpy(buf + sizeof(struct ethhdr), ifm->m_data, ifm->m_len);
+        slirp_send_packet_all(slirp, buf, ifm->m_len + ETH_HLEN);
+        ret = 1;
+    }
+
+    g_free(buf);
+    return ret;
 }
---

Anyhow, if you plan to properly (with your S-o-b) commit your patch to
the libslirp repository, using g_try_malloc() you can add:

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Regards,

Phil.

Re: [Qemu-devel] slirp, incoming packets get truncated

[Samuel Thibault]

Hello,

Philippe Mathieu-Daudé, le lun. 09 sept. 2019 12:51:23 +0200, a ecrit:
> Anyhow, if you plan to properly (with your S-o-b) commit your patch to
> the libslirp repository,

Actually the libslirp repository already has something there. Since
IPv4/6 have a maximum of 64KB packets, it just allocates that size
on the stack, whîch is simpler in the end. Also it makes mru/mtu
configurable, we will be able to plug that into qemu once released.

Samuel