From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8568FC28B2C for ; Mon, 15 Aug 2022 07:17:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241679AbiHOHRG (ORCPT ); Mon, 15 Aug 2022 03:17:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233043AbiHOHPw (ORCPT ); Mon, 15 Aug 2022 03:15:52 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 4B77A1EAE5 for ; Mon, 15 Aug 2022 00:14:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1660547695; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=uunb/SmxieTcIyt41wPcw20H+0i6av2iZEDGKTJdPks=; b=ZLUiVF/kHCjb2aGhYp2giiDXHVCw71D/dpP6aPSfMyRKgWQiHVOB8tGiGhy9C1ZODohdWZ c9qDj8VKSE8cf/Ksfz0CDefZaFFdNdfNr7+fxt7WfHr3rWmUrR7wD3qN/OAa22k/voFhCJ i3sSUO69bTShyNTvXn8fEGUgGH//Ing= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-7-5Zy8ZhBfNM6dY3vxqVGDeQ-1; Mon, 15 Aug 2022 03:14:54 -0400 X-MC-Unique: 5Zy8ZhBfNM6dY3vxqVGDeQ-1 Received: by mail-ed1-f71.google.com with SMTP id b6-20020a056402278600b0043e686058feso4253392ede.10 for ; Mon, 15 Aug 2022 00:14:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc; bh=uunb/SmxieTcIyt41wPcw20H+0i6av2iZEDGKTJdPks=; b=d9U+ld5NM7re0U9gmVDT34VXij3qd0pOrspW5tghhzuPVChMBf3H4wVl6rqh1Hc3EL YhhryTXCABcZ5W4Kd1ITJKVMOWhxRX8QWq9r56nkUMaAYDZ1z8wDJBw/HyG1VE0jM6S5 oFmz2Jg4vsv69rsETtKyjuTLGpxAypjCxO8i70RQTv/YDCyjtQ4g3YXT8vVUHVk9fZYo 0AE3+Nq9p05zQXoP/TQCDTqRttQ4vi0zeRXNVR0D6pOzziy0DQWlh8rtYg5QUEdz5Gvi OxJ+0PwFL2+XGVRyWmNwxTSEP5vJV+/64EayLWUgyqWh/wJ3kj/Y4HgBQy2c+5GtohAl PJng== X-Gm-Message-State: ACgBeo2jzwZSKe6gTRKC9y/vVmbqvUjK7Z+fW9qGCDKHPylP5m4yx6P6 k/fA/6a8H9fB9cSivCTLJ1L+qE4IIMpaCrwfJuSJwUOtgUSCvKo679vqvk6ugrEcRTFl9j0bN00 tWrOs2EFYc9s1k0lK X-Received: by 2002:a05:6402:a47:b0:43d:17a0:fdc9 with SMTP id bt7-20020a0564020a4700b0043d17a0fdc9mr13334569edb.41.1660547693092; Mon, 15 Aug 2022 00:14:53 -0700 (PDT) X-Google-Smtp-Source: AA6agR4YoHheG5W/eaV8BZh8HsvP1ibrkGNG7x2cY+yPrQSEoDviob3P1rxs8nDl9bBsoZz0emD9gQ== X-Received: by 2002:a05:6402:a47:b0:43d:17a0:fdc9 with SMTP id bt7-20020a0564020a4700b0043d17a0fdc9mr13334553edb.41.1660547692848; Mon, 15 Aug 2022 00:14:52 -0700 (PDT) Received: from redhat.com ([2.54.169.49]) by smtp.gmail.com with ESMTPSA id f1-20020a1709067f8100b007310a9a65cbsm3880906ejr.16.2022.08.15.00.14.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Aug 2022 00:14:52 -0700 (PDT) Date: Mon, 15 Aug 2022 03:14:43 -0400 From: "Michael S. Tsirkin" To: Xuan Zhuo Cc: virtualization@lists.linux-foundation.org, Richard Weinberger , Anton Ivanov , Johannes Berg , Jason Wang , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , Mark Gross , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Vincent Whitchurch , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org, kangjie.xu@linux.alibaba.com Subject: Re: [PATCH v14 37/42] virtio_net: set the default max ring size by find_vqs() Message-ID: <20220815031022-mutt-send-email-mst@kernel.org> References: <20220801063902.129329-1-xuanzhuo@linux.alibaba.com> <20220801063902.129329-38-xuanzhuo@linux.alibaba.com> <20220815015405-mutt-send-email-mst@kernel.org> <1660545303.436073-9-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1660545303.436073-9-xuanzhuo@linux.alibaba.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Mon, Aug 15, 2022 at 02:35:03PM +0800, Xuan Zhuo wrote: > On Mon, 15 Aug 2022 02:00:16 -0400, "Michael S. Tsirkin" wrote: > > On Mon, Aug 01, 2022 at 02:38:57PM +0800, Xuan Zhuo wrote: > > > Use virtio_find_vqs_ctx_size() to specify the maximum ring size of tx, > > > rx at the same time. > > > > > > | rx/tx ring size > > > ------------------------------------------- > > > speed == UNKNOWN or < 10G| 1024 > > > speed < 40G | 4096 > > > speed >= 40G | 8192 > > > > > > Call virtnet_update_settings() once before calling init_vqs() to update > > > speed. > > > > > > Signed-off-by: Xuan Zhuo > > > Acked-by: Jason Wang > > > > I've been looking at this patchset because of the resent > > reported crashes, and I'm having second thoughts about this. > > > > Do we really want to second-guess the device supplied > > max ring size? If yes why? > > > > Could you please share some performance data that motivated this > > specific set of numbers? > > > The impact of this value on performance is as follows. The larger the value, the > throughput can be increased, but the delay will also increase accordingly. It is > a maximum limit for the ring size under the corresponding speed. The purpose of > this limitation is not to improve performance, but more to reduce memory usage. > > These data come from many other network cards and some network optimization > experience. > > For example, in the case of speed = 20G, the impact of ring size greater > than 4096 on performance has no meaning. At this time, if the device supports > 8192, we limit it to 4096 through this, the real meaning is to reduce the memory > usage. > > > > > > Also why do we intepret UNKNOWN as "very low"? > > I'm thinking that should definitely be "don't change anything". > > > > Generally speaking, for a network card with a high speed, it will return a > correct speed. But I think it is a good idea to do nothing. > > > Finally if all this makes sense then shouldn't we react when > > speed changes? > > This is the feedback of the network card when it is started, and theoretically > it should not change in the future. Yes it should: Both \field{speed} and \field{duplex} can change, thus the driver is expected to re-read these values after receiving a configuration change notification. Moreover, during probe link can quite reasonably be down. If it is, then speed and duplex might not be correct. > > > > Could you try reverting this and showing performance results > > before and after please? Thanks! > > I hope the above reply can help you, if there is anything else you need me to > cooperate with, I am very happy. > > If you think it's ok, I can resubmit a commit with 'UNKNOW' set to unlimited. I > can submit it with the issue of #30. > > Thanks. > > > > > > > --- > > > drivers/net/virtio_net.c | 42 ++++++++++++++++++++++++++++++++++++---- > > > 1 file changed, 38 insertions(+), 4 deletions(-) > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > > index 8a5810bcb839..40532ecbe7fc 100644 > > > --- a/drivers/net/virtio_net.c > > > +++ b/drivers/net/virtio_net.c > > > @@ -3208,6 +3208,29 @@ static unsigned int mergeable_min_buf_len(struct virtnet_info *vi, struct virtqu > > > (unsigned int)GOOD_PACKET_LEN); > > > } > > > > > > +static void virtnet_config_sizes(struct virtnet_info *vi, u32 *sizes) > > > +{ > > > + u32 i, rx_size, tx_size; > > > + > > > + if (vi->speed == SPEED_UNKNOWN || vi->speed < SPEED_10000) { > > > + rx_size = 1024; > > > + tx_size = 1024; > > > + > > > + } else if (vi->speed < SPEED_40000) { > > > + rx_size = 1024 * 4; > > > + tx_size = 1024 * 4; > > > + > > > + } else { > > > + rx_size = 1024 * 8; > > > + tx_size = 1024 * 8; > > > + } > > > + > > > + for (i = 0; i < vi->max_queue_pairs; i++) { > > > + sizes[rxq2vq(i)] = rx_size; > > > + sizes[txq2vq(i)] = tx_size; > > > + } > > > +} > > > + > > > static int virtnet_find_vqs(struct virtnet_info *vi) > > > { > > > vq_callback_t **callbacks; > > > @@ -3215,6 +3238,7 @@ static int virtnet_find_vqs(struct virtnet_info *vi) > > > int ret = -ENOMEM; > > > int i, total_vqs; > > > const char **names; > > > + u32 *sizes; > > > bool *ctx; > > > > > > /* We expect 1 RX virtqueue followed by 1 TX virtqueue, followed by > > > @@ -3242,10 +3266,15 @@ static int virtnet_find_vqs(struct virtnet_info *vi) > > > ctx = NULL; > > > } > > > > > > + sizes = kmalloc_array(total_vqs, sizeof(*sizes), GFP_KERNEL); > > > + if (!sizes) > > > + goto err_sizes; > > > + > > > /* Parameters for control virtqueue, if any */ > > > if (vi->has_cvq) { > > > callbacks[total_vqs - 1] = NULL; > > > names[total_vqs - 1] = "control"; > > > + sizes[total_vqs - 1] = 64; > > > } > > > > > > /* Allocate/initialize parameters for send/receive virtqueues */ > > > @@ -3260,8 +3289,10 @@ static int virtnet_find_vqs(struct virtnet_info *vi) > > > ctx[rxq2vq(i)] = true; > > > } > > > > > > - ret = virtio_find_vqs_ctx(vi->vdev, total_vqs, vqs, callbacks, > > > - names, ctx, NULL); > > > + virtnet_config_sizes(vi, sizes); > > > + > > > + ret = virtio_find_vqs_ctx_size(vi->vdev, total_vqs, vqs, callbacks, > > > + names, sizes, ctx, NULL); > > > if (ret) > > > goto err_find; > > > > > > @@ -3281,6 +3312,8 @@ static int virtnet_find_vqs(struct virtnet_info *vi) > > > > > > > > > err_find: > > > + kfree(sizes); > > > +err_sizes: > > > kfree(ctx); > > > err_ctx: > > > kfree(names); > > > @@ -3630,6 +3663,9 @@ static int virtnet_probe(struct virtio_device *vdev) > > > vi->curr_queue_pairs = num_online_cpus(); > > > vi->max_queue_pairs = max_queue_pairs; > > > > > > + virtnet_init_settings(dev); > > > + virtnet_update_settings(vi); > > > + > > > /* Allocate/initialize the rx/tx queues, and invoke find_vqs */ > > > err = init_vqs(vi); > > > if (err) > > > @@ -3642,8 +3678,6 @@ static int virtnet_probe(struct virtio_device *vdev) > > > netif_set_real_num_tx_queues(dev, vi->curr_queue_pairs); > > > netif_set_real_num_rx_queues(dev, vi->curr_queue_pairs); > > > > > > - virtnet_init_settings(dev); > > > - > > > if (virtio_has_feature(vdev, VIRTIO_NET_F_STANDBY)) { > > > vi->failover = net_failover_create(vi->dev); > > > if (IS_ERR(vi->failover)) { > > > -- > > > 2.31.0 > >