From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB154C282E1 for ; Wed, 24 Apr 2019 17:37:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7A6E821907 for ; Wed, 24 Apr 2019 17:37:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392230AbfDXRhf (ORCPT ); Wed, 24 Apr 2019 13:37:35 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:38872 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392217AbfDXRhe (ORCPT ); Wed, 24 Apr 2019 13:37:34 -0400 Received: by mail-qk1-f196.google.com with SMTP id g1so11367137qki.5 for ; Wed, 24 Apr 2019 10:37:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=o4cNVCLa3OdVPTkXpCkCnBxCu1qNhWjqcPhf34fL7lM=; b=JYpn6QhXFAUrSAHGXIKb0ThIittrRSbyCO4zHo0j2rPFjgIUBFnuphYqTiebS5QFGw Aij5o/gagtkWLRRx3VbDH0nnYnbxTMnLI5I+UySIzQw2kiSY+QIMHJJYm8eo+GWRW40M 9arSLsa9QzAXTx94O8hHnMwOm6xcYOo5mHWEyYVwfieIHWpAjCcbVke+YUXcFYoFNgz/ rstQYb1Q1Kd6w9ASizsXVzhe0Wckip4J+TB/8S0XFh5puiNj13yKeizqtRkBXY4NjJ4g 5AODFOQYbYJeayA0897wQTTpOthlWM+VnSJ+SU5Y0MXxA7p8YpwB9d8bkRguXIwudsoQ DdLw== X-Gm-Message-State: APjAAAX9bFNVrf371cRTBTxM0GF+fI/0JovCV3eFgSsgJ/blY5B0fNoi ZsUxuDizHw3hZU1W0Kz3vMpR8A== X-Google-Smtp-Source: APXvYqyQdhNCQYWfmNaOBIPMTrqYQiZ3tV8iH7eWY59JFufC3LbPPrYW7M+esRBBu06vmZ+UEC8AcA== X-Received: by 2002:a37:5ac4:: with SMTP id o187mr26155733qkb.356.1556127452902; Wed, 24 Apr 2019 10:37:32 -0700 (PDT) Received: from redhat.com (pool-173-76-105-71.bstnma.fios.verizon.net. [173.76.105.71]) by smtp.gmail.com with ESMTPSA id w40sm3467883qth.35.2019.04.24.10.37.31 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Apr 2019 10:37:32 -0700 (PDT) Date: Wed, 24 Apr 2019 13:37:29 -0400 From: "Michael S. Tsirkin" To: David Ahern Cc: makita.toshiaki@lab.ntt.co.jp, Jesper Dangaard Brouer , Toke =?iso-8859-1?Q?H=F8iland-J=F8rgensen?= , "netdev@vger.kernel.org" , Jason Wang Subject: Re: virtio_net: suspicious RCU usage with xdp Message-ID: <20190424132533-mutt-send-email-mst@kernel.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Wed, Apr 24, 2019 at 11:13:42AM -0600, David Ahern wrote: > seeing an RCU warning testing xdp with virtio net. net-next as of commit > b2f97f7de2f6a4df8e431330cf467576486651c5. No obvious changes so hoping > this rings a bell with someone else. > > > [ 121.990304] ============================= > [ 121.991488] WARNING: suspicious RCU usage > [ 121.992392] 5.1.0-rc5+ #60 Not tainted > [ 121.993220] ----------------------------- > [ 121.994158] /home/dsa/kernel-3.git/drivers/net/virtio_net.c:516 > suspicious rcu_dereference_check() usage! > [ 121.996284] > other info that might help us debug this: > > [ 121.997988] > rcu_scheduler_active = 2, debug_locks = 1 > [ 121.999321] no locks held by swapper/1/0. > [ 122.000328] > stack backtrace: > [ 122.001253] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.1.0-rc5+ #60 > [ 122.002474] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), > BIOS 1.11.1-1 04/01/2014 > [ 122.004141] Call Trace: > [ 122.004651] > [ 122.005082] dump_stack+0x7e/0xbb > [ 122.005757] lockdep_rcu_suspicious+0x102/0x10b > [ 122.006654] virtnet_xdp_xmit+0x104/0x4fe > [ 122.007447] ? kasan_check_read+0x11/0x13 > [ 122.008267] ? mergeable_rx_buffer_size_show+0x163/0x163 > [ 122.009299] ? __asan_loadN+0xf/0x11 > [ 122.010010] ? pvclock_clocksource_read+0xfa/0x189 > [ 122.010975] bq_xmit_all+0xdc/0x358 > [ 122.011699] __dev_map_flush+0xc2/0xef > [ 122.012472] xdp_do_flush_map+0x5b/0x74 > [ 122.013238] virtnet_poll+0x58f/0x679 Well virtnet_xdp_xmit seems to be called from .ndo_xdp_xmit and that isn't in an RCU read-side critical section. Looks like we just need to add RCU read lock/unlock. Like the below perhaps? This issue was introduced by 8dcc5b0ab0 however I find it inelegant that we need to do checks in each driver, and add RCU locks just for a startup initialization issue. Can't XDP core make sure the callback isn't invoked at an inappropriate time instead? ---> virtio_net: call virtnet_xdp_xmit with RCU lock This functions uses rcu_dereference so it needs to be called in an RCU read-side critical section. Fixes: 8dcc5b0ab0 ("virtio_net: fix ndo_xdp_xmit crash towards dev not ready for XDP") Signed-off-by: Michael S. Tsirkin -- diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 9010938e2d71..ccc1bdd1bb1f 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -495,8 +495,8 @@ static struct send_queue *virtnet_xdp_sq(struct virtnet_info *vi) return &vi->sq[qp]; } -static int virtnet_xdp_xmit(struct net_device *dev, - int n, struct xdp_frame **frames, u32 flags) +static int __virtnet_xdp_xmit(struct net_device *dev, + int n, struct xdp_frame **frames, u32 flags) { struct virtnet_info *vi = netdev_priv(dev); struct receive_queue *rq = vi->rq; @@ -569,6 +569,17 @@ static int virtnet_xdp_xmit(struct net_device *dev, return ret; } +static int virtnet_xdp_xmit(struct net_device *dev, + int n, struct xdp_frame **frames, u32 flags) +{ + int r; + + rcu_read_lock_bh(); + r = __virtnet_xdp_xmit(dev, n, frames, flags); + rcu_read_unlock_bh(); + return r; +} + static unsigned int virtnet_get_headroom(struct virtnet_info *vi) { return vi->xdp_queue_pairs ? VIRTIO_XDP_HEADROOM : 0; @@ -714,7 +725,7 @@ static struct sk_buff *receive_small(struct net_device *dev, xdpf = convert_to_xdp_frame(&xdp); if (unlikely(!xdpf)) goto err_xdp; - err = virtnet_xdp_xmit(dev, 1, &xdpf, 0); + err = __virtnet_xdp_xmit(dev, 1, &xdpf, 0); if (unlikely(err < 0)) { trace_xdp_exception(vi->dev, xdp_prog, act); goto err_xdp; @@ -887,7 +898,7 @@ static struct sk_buff *receive_mergeable(struct net_device *dev, xdpf = convert_to_xdp_frame(&xdp); if (unlikely(!xdpf)) goto err_xdp; - err = virtnet_xdp_xmit(dev, 1, &xdpf, 0); + err = __virtnet_xdp_xmit(dev, 1, &xdpf, 0); if (unlikely(err < 0)) { trace_xdp_exception(vi->dev, xdp_prog, act); if (unlikely(xdp_page != page)) -- MST