From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 169F8C433DB for ; Tue, 26 Jan 2021 14:17:16 +0000 (UTC) Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A58C6206CA for ; Tue, 26 Jan 2021 14:17:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A58C6206CA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernelnewbies-bounces@kernelnewbies.org Received: from localhost ([::1] helo=shelob.surriel.com) by shelob.surriel.com with esmtp (Exim 4.94) (envelope-from ) id 1l4P9Q-0001BI-UD; Tue, 26 Jan 2021 09:17:00 -0500 Received: from mail-ej1-x633.google.com ([2a00:1450:4864:20::633]) by shelob.surriel.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94) (envelope-from ) id 1l4P9P-0001Ai-Hx for kernelnewbies@kernelnewbies.org; Tue, 26 Jan 2021 09:16:59 -0500 Received: by mail-ej1-x633.google.com with SMTP id ox12so23218556ejb.2 for ; Tue, 26 Jan 2021 06:16:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:mail-followup-to:mime-version :content-disposition:user-agent; bh=QKucVU+UOcfFyl7BB9PcuZRXR5t3NHOsqDsthUZXp+E=; b=I45mAryyqQdYYB3DbW+UzuCGT2IjWNSuRlyjhQPfbIJVCby9AlnQGbTTlcJoSn5L14 rgR2Vr4W47GGseLIGYv17ToAlGdZuvqfrt1H7nda7FgAM8rMRGY0nmi+bU7amdW6ykzP F4IMIYUxNfgJAiF53OLsJ+nCpXHVPFJu3/9gULyO3VsRwdj5OutbajXdbv+6XIPsVYsE VAcmSkN9+0U5bj0YnO7V7QXMXOfk85OO3OoU2XuHBULL8dlR+nA7L4uYx8rmfh1Z+kbe 2TzzIn+xFfgr2vxpe877eFY1I77KKC6I0zb6KHqArT10R/+pkcazd+I2MOBhqIWBAySn 1TZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:mime-version:content-disposition:user-agent; bh=QKucVU+UOcfFyl7BB9PcuZRXR5t3NHOsqDsthUZXp+E=; b=uitcdz+5yxEKki6SfK+NbWV8GxLOoC7uuzkN94ydjnoiRWboetLuyDzjRruTi4XTMt 6uGS5qztVQ3WR3ewWg0oFp0M2tsF+I4BSdH919pgY96Fx0Q/67aU7x8HUD/AuENJv0HQ qtg+Ea9yVYguh4f9UsQWohlFKpNtWpZLMQ1M+/8rOC9kSsdtKzaoO15fLSxPHjW/qvms C/EqU/pJR79YcNutjFxXwkD1yyNVJ5Tg6XVANTkN4o2Qpk5D7I5VZlLzajiBO3+wkLbi Sl5UfOP1Q2oCmAxGIWKpLWxMXuflgibXPdR8ovewmZPXIbvIPYDiIGTYND/SyI6DLPcd kO+g== X-Gm-Message-State: AOAM530vC54Lyacd/z1U8hkp8lFtx5GcIiR5Xpx2VRWknzhYo6+Odv0q msRIHNLAqQrfu0SWNZ4L5MjWB6ruaqi9TQ== X-Google-Smtp-Source: ABdhPJwi5cHzpKsFy2OaeKI8j5JUBfi3YFoRkLLptcDV3zY05gIp0MmexjTvylJtV/m1NmNaNvza8g== X-Received: by 2002:a17:907:3f9e:: with SMTP id hr30mr3511686ejc.445.1611670617877; Tue, 26 Jan 2021 06:16:57 -0800 (PST) Received: from localhost (dslb-002-207-138-002.002.207.pools.vodafone-ip.de. [2.207.138.2]) by smtp.gmail.com with ESMTPSA id c14sm12865559edr.46.2021.01.26.06.16.56 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 26 Jan 2021 06:16:57 -0800 (PST) Date: Tue, 26 Jan 2021 15:12:49 +0100 From: Oliver Graute To: kernelnewbies@kernelnewbies.org, netdev@vger.kernel.org Subject: UDP implementation and the MSG_MORE flag Message-ID: <20210126141248.GA27281@optiplex> Mail-Followup-To: kernelnewbies@kernelnewbies.org, netdev@vger.kernel.org, kuznet@ms2.inr.ac.ru, yoshfuji@linux-ipv6.org, jakub@cloudflare.com, pabeni@redhat.com MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.9.4 (2018-02-28) Cc: yoshfuji@linux-ipv6.org, kuznet@ms2.inr.ac.ru, pabeni@redhat.com, jakub@cloudflare.com X-BeenThere: kernelnewbies@kernelnewbies.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Learn about the Linux kernel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kernelnewbies-bounces@kernelnewbies.org Hello, we observe some unexpected behavior in the UDP implementation of the linux kernel. Some UDP packets send via the loopback interface are dropped in the kernel on the receive side when using sendto with the MSG_MORE flag. Every drop increases the InCsumErrors in /proc/self/net/snmp. Some example code to reproduce it is appended below. In the code we tracked it down to this code section. ( Even a little further but its unclear to me wy the csum() is wrong in the bad case) udpv6_recvmsg() ... if (checksum_valid || udp_skb_csum_unnecessary(skb)) { if (udp_skb_is_linear(skb)) err = copy_linear_skb(skb, copied, off, &msg->msg_iter); else err = skb_copy_datagram_msg(skb, off, msg, copied); } else { err = skb_copy_and_csum_datagram_msg(skb, off, msg); if (err == -EINVAL) { goto csum_copy_err; } } ... Perhaps someone with deeper knowledge can comment on this and can explain us the reason of this behavior. Best regards, Oliver udp-send.c #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #define BUFFSIZE 512*1024 int main(int argc, char** argv) { int fd = 0; int port = 0; char *buffer; struct sockaddr_in addr; ssize_t addrlen = 0; if(argc == 2) { port = atoi(argv[1]); } else { port = 4711; } fd = socket(PF_INET, SOCK_DGRAM, 0); addr.sin_family = AF_INET; addr.sin_port = htons(port); addr.sin_addr.s_addr = inet_addr("127.0.0.1"); addrlen = sizeof(addr); buffer = malloc(BUFFSIZE); if (!buffer) { return 0; } printf("\nsending BROKEN segmented testdata on local port %i \n", port); snprintf(buffer, BUFFSIZE, "start-data {\n"); sendto(fd, buffer, strlen(buffer), MSG_MORE, (struct sockaddr *) &addr, addrlen); snprintf(buffer, BUFFSIZE, "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n"); sendto(fd, buffer, strlen(buffer), MSG_MORE, (struct sockaddr *) &addr, addrlen); snprintf(buffer, BUFFSIZE, "}\n"); sendto(fd, buffer, strlen(buffer), 0, (struct sockaddr *) &addr, addrlen); printf("\nsending VALID segmented testdata on local port %i \n", port); snprintf(buffer, BUFFSIZE, "start-data {\n"); sendto(fd, buffer, strlen(buffer), MSG_MORE, (struct sockaddr *) &addr, addrlen); snprintf(buffer, BUFFSIZE, "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n"); sendto(fd, buffer, strlen(buffer), MSG_MORE, (struct sockaddr *) &addr, addrlen); snprintf(buffer, BUFFSIZE, "}\n"); sendto(fd, buffer, strlen(buffer), 0, (struct sockaddr *) &addr, addrlen); printf("\nsending VALID unsegmented testdata on local port %i \n", port); snprintf(buffer, BUFFSIZE, "start-data {\n"); snprintf(buffer + strlen(buffer), BUFFSIZE - strlen(buffer), "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n"); snprintf(buffer+ strlen(buffer), BUFFSIZE - strlen(buffer), "}\n"); sendto(fd, buffer, strlen(buffer), 0, (struct sockaddr *) &addr, addrlen); free(buffer); return 0; } ------- udp-receive.c #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include int main(int argc, char** argv) { int fd = 0; int arg = 0; int ret = 0; struct sockaddr_in addr; ssize_t addrlen = 0; int port = 0; char *buffer; char *printbuffer; int recvlen = 0; if(argc == 2) { port = atoi(argv[1]); } else { port = 4711; } fd = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP); addr.sin_family = AF_INET; addr.sin_port = htons(port); addr.sin_addr.s_addr = inet_addr("0.0.0.0"); addrlen = sizeof(addr); buffer = malloc(65536); if (!buffer) { return 0; } printbuffer = malloc(65537); if (!printbuffer) { return 0; } if(fd) { printf("\nbinding to local port %i \n", port); //bind ret = bind(fd, (struct sockaddr *)&addr, addrlen); printf("result error %i, errno %i\n", ret, errno); do { recvlen = recvfrom(fd, buffer, 65536, 0, NULL, NULL); if (recvlen >0) { printf("\nreceived %i bytes of data:\n", recvlen); memset(printbuffer, 0, 65537); memcpy(printbuffer, buffer, recvlen); printf("%s\n", printbuffer); } else if(recvlen < 0) { printf("\n receive error %i, errno %i\n", recvlen, errno); } } while(1); close(fd); } else { printf("\nerror creating socket\n"); } free(buffer); free(printbuffer); return 0; } _______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies