From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5708FC282E1 for ; Thu, 25 Apr 2019 02:55:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 26E0D21773 for ; Thu, 25 Apr 2019 02:55:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388659AbfDYCza (ORCPT ); Wed, 24 Apr 2019 22:55:30 -0400 Received: from mx1.redhat.com ([209.132.183.28]:63968 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387859AbfDYCza (ORCPT ); Wed, 24 Apr 2019 22:55:30 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C67A82DA99F; Thu, 25 Apr 2019 02:55:29 +0000 (UTC) Received: from [10.72.12.167] (ovpn-12-167.pek2.redhat.com [10.72.12.167]) by smtp.corp.redhat.com (Postfix) with ESMTP id C6FD61001DDE; Thu, 25 Apr 2019 02:55:22 +0000 (UTC) Subject: Re: virtio_net: suspicious RCU usage with xdp To: "Michael S. Tsirkin" , David Ahern Cc: makita.toshiaki@lab.ntt.co.jp, Jesper Dangaard Brouer , =?UTF-8?Q?Toke_H=c3=b8iland-J=c3=b8rgensen?= , "netdev@vger.kernel.org" References: <20190424132533-mutt-send-email-mst@kernel.org> From: Jason Wang Message-ID: Date: Thu, 25 Apr 2019 10:55:21 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <20190424132533-mutt-send-email-mst@kernel.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Thu, 25 Apr 2019 02:55:29 +0000 (UTC) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 2019/4/25 上午1:37, Michael S. Tsirkin wrote: > On Wed, Apr 24, 2019 at 11:13:42AM -0600, David Ahern wrote: >> seeing an RCU warning testing xdp with virtio net. net-next as of commit >> b2f97f7de2f6a4df8e431330cf467576486651c5. No obvious changes so hoping >> this rings a bell with someone else. >> >> >> [ 121.990304] ============================= >> [ 121.991488] WARNING: suspicious RCU usage >> [ 121.992392] 5.1.0-rc5+ #60 Not tainted >> [ 121.993220] ----------------------------- >> [ 121.994158] /home/dsa/kernel-3.git/drivers/net/virtio_net.c:516 >> suspicious rcu_dereference_check() usage! >> [ 121.996284] >> other info that might help us debug this: >> >> [ 121.997988] >> rcu_scheduler_active = 2, debug_locks = 1 >> [ 121.999321] no locks held by swapper/1/0. >> [ 122.000328] >> stack backtrace: >> [ 122.001253] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.1.0-rc5+ #60 >> [ 122.002474] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), >> BIOS 1.11.1-1 04/01/2014 >> [ 122.004141] Call Trace: >> [ 122.004651] >> [ 122.005082] dump_stack+0x7e/0xbb >> [ 122.005757] lockdep_rcu_suspicious+0x102/0x10b >> [ 122.006654] virtnet_xdp_xmit+0x104/0x4fe >> [ 122.007447] ? kasan_check_read+0x11/0x13 >> [ 122.008267] ? mergeable_rx_buffer_size_show+0x163/0x163 >> [ 122.009299] ? __asan_loadN+0xf/0x11 >> [ 122.010010] ? pvclock_clocksource_read+0xfa/0x189 >> [ 122.010975] bq_xmit_all+0xdc/0x358 >> [ 122.011699] __dev_map_flush+0xc2/0xef >> [ 122.012472] xdp_do_flush_map+0x5b/0x74 >> [ 122.013238] virtnet_poll+0x58f/0x679 > Well virtnet_xdp_xmit seems to be called from .ndo_xdp_xmit > and that isn't in an RCU read-side critical section. > > Looks like we just need to add RCU read lock/unlock. > Like the below perhaps? > > This issue was introduced by 8dcc5b0ab0 however I find it > inelegant that we need to do checks in each driver, > and add RCU locks just for a startup initialization issue. > Can't XDP core make sure the callback isn't invoked > at an inappropriate time instead? > > ---> > > virtio_net: call virtnet_xdp_xmit with RCU lock > > This functions uses rcu_dereference so it needs to be > called in an RCU read-side critical section. > > Fixes: 8dcc5b0ab0 ("virtio_net: fix ndo_xdp_xmit crash towards dev not ready for XDP") > Signed-off-by: Michael S. Tsirkin > > -- > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > index 9010938e2d71..ccc1bdd1bb1f 100644 > --- a/drivers/net/virtio_net.c > +++ b/drivers/net/virtio_net.c > @@ -495,8 +495,8 @@ static struct send_queue *virtnet_xdp_sq(struct virtnet_info *vi) > return &vi->sq[qp]; > } > > -static int virtnet_xdp_xmit(struct net_device *dev, > - int n, struct xdp_frame **frames, u32 flags) > +static int __virtnet_xdp_xmit(struct net_device *dev, > + int n, struct xdp_frame **frames, u32 flags) > { > struct virtnet_info *vi = netdev_priv(dev); > struct receive_queue *rq = vi->rq; > @@ -569,6 +569,17 @@ static int virtnet_xdp_xmit(struct net_device *dev, > return ret; > } > > +static int virtnet_xdp_xmit(struct net_device *dev, > + int n, struct xdp_frame **frames, u32 flags) > +{ > + int r; > + > + rcu_read_lock_bh(); > + r = __virtnet_xdp_xmit(dev, n, frames, flags); > + rcu_read_unlock_bh(); > + return r; > +} > + > static unsigned int virtnet_get_headroom(struct virtnet_info *vi) > { > return vi->xdp_queue_pairs ? VIRTIO_XDP_HEADROOM : 0; > @@ -714,7 +725,7 @@ static struct sk_buff *receive_small(struct net_device *dev, > xdpf = convert_to_xdp_frame(&xdp); > if (unlikely(!xdpf)) > goto err_xdp; > - err = virtnet_xdp_xmit(dev, 1, &xdpf, 0); > + err = __virtnet_xdp_xmit(dev, 1, &xdpf, 0); > if (unlikely(err < 0)) { > trace_xdp_exception(vi->dev, xdp_prog, act); > goto err_xdp; > @@ -887,7 +898,7 @@ static struct sk_buff *receive_mergeable(struct net_device *dev, > xdpf = convert_to_xdp_frame(&xdp); > if (unlikely(!xdpf)) > goto err_xdp; > - err = virtnet_xdp_xmit(dev, 1, &xdpf, 0); > + err = __virtnet_xdp_xmit(dev, 1, &xdpf, 0); > if (unlikely(err < 0)) { > trace_xdp_exception(vi->dev, xdp_prog, act); > if (unlikely(xdp_page != page)) > > Acked-by: Jason Wang