From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11DF5C43382 for ; Thu, 27 Sep 2018 08:57:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B842B2156B for ; Thu, 27 Sep 2018 08:57:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B842B2156B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727201AbeI0POX (ORCPT ); Thu, 27 Sep 2018 11:14:23 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53066 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726929AbeI0POX (ORCPT ); Thu, 27 Sep 2018 11:14:23 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A2DE3792C0; Thu, 27 Sep 2018 08:57:12 +0000 (UTC) Received: from ovpn-112-72.ams2.redhat.com (ovpn-112-72.ams2.redhat.com [10.36.112.72]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2EEF160D62; Thu, 27 Sep 2018 08:57:06 +0000 (UTC) Message-ID: <1538038625.19334.2.camel@redhat.com> Subject: Re: [PATCH] hv_netvsc: Make sure out channel is fully opened on send From: Mohammed Gamal Reply-To: mgamal@redhat.com To: Haiyang Zhang , Stephen Hemminger , "netdev@vger.kernel.org" Cc: KY Srinivasan , vkuznets , "otubo@redhat.com" , cavery , "linux-kernel@vger.kernel.org" , "devel@linuxdriverproject.org" Date: Thu, 27 Sep 2018 10:57:05 +0200 In-Reply-To: References: <1537979659-26979-1-git-send-email-mgamal@redhat.com> Organization: Red Hat Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Thu, 27 Sep 2018 08:57:12 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 2018-09-26 at 17:13 +0000, Haiyang Zhang wrote: > > -----Original Message----- > > From: Mohammed Gamal > > Sent: Wednesday, September 26, 2018 12:34 PM > > To: Stephen Hemminger ; netdev@vger.kernel. > > org > > Cc: KY Srinivasan ; Haiyang Zhang > > ; vkuznets ; > > otubo@redhat.com; cavery ; linux- > > kernel@vger.kernel.org; devel@linuxdriverproject.org; Mohammed > > Gamal > > > > Subject: [PATCH] hv_netvsc: Make sure out channel is fully opened > > on send > > > > Dring high network traffic changes to network interface parameters > > such as > > number of channels or MTU can cause a kernel panic with a NULL > > pointer > > dereference. This is due to netvsc_device_remove() being called and > > deallocating the channel ring buffers, which can then be accessed > > by > > netvsc_send_pkt() before they're allocated on calling > > netvsc_device_add() > > > > The patch fixes this problem by checking the channel state and > > returning > > ENODEV if not yet opened. We also move the call to > > hv_ringbuf_avail_percent() > > which may access the uninitialized ring buffer. > > > > Signed-off-by: Mohammed Gamal > > --- > >  drivers/net/hyperv/netvsc.c | 7 ++++++- > >  1 file changed, 6 insertions(+), 1 deletion(-) > > > > diff --git a/drivers/net/hyperv/netvsc.c > > b/drivers/net/hyperv/netvsc.c index > > fe01e14..75f1b31 100644 > > --- a/drivers/net/hyperv/netvsc.c > > +++ b/drivers/net/hyperv/netvsc.c > > @@ -825,7 +825,12 @@ static inline int netvsc_send_pkt( > >   struct netdev_queue *txq = netdev_get_tx_queue(ndev, > > packet->q_idx); > >   u64 req_id; > >   int ret; > > - u32 ring_avail = > > hv_get_avail_to_write_percent(&out_channel- > > > outbound); > > > > + u32 ring_avail; > > + > > + if (out_channel->state != CHANNEL_OPENED_STATE) > > + return -ENODEV; > > + > > + ring_avail = hv_get_avail_to_write_percent(&out_channel- > > >outbound); > > When you reproducing the NULL ptr panic, does your kernel include the > following patch? > hv_netvsc: common detach logic > https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/c > ommit/?id=7b2ee50c0cd513a176a26a71f2989facdd75bfea > Yes it is included. And the commit did reduce the occurrence of this race condition, but it still nevertheless occurs albeit rarely. > We call netif_tx_disable(ndev) and netif_device_detach(ndev) before > doing the changes  > on MTU or #channels. So there should be no call to start_xmit() when > channel is not ready. > > If you see the check for CHANNEL_OPENED_STATE is still necessary on > upstream kernel (including  > the patch " common detach logic "), we should debug further on the > code and find out the  > root cause. > > Thanks, > - Haiyang > From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mohammed Gamal Subject: Re: [PATCH] hv_netvsc: Make sure out channel is fully opened on send Date: Thu, 27 Sep 2018 10:57:05 +0200 Message-ID: <1538038625.19334.2.camel@redhat.com> References: <1537979659-26979-1-git-send-email-mgamal@redhat.com> Reply-To: mgamal@redhat.com Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Cc: "otubo@redhat.com" , "linux-kernel@vger.kernel.org" , "devel@linuxdriverproject.org" , vkuznets To: Haiyang Zhang , Stephen Hemminger , "netdev@vger.kernel.org" Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: driverdev-devel-bounces@linuxdriverproject.org Sender: "devel" List-Id: netdev.vger.kernel.org T24gV2VkLCAyMDE4LTA5LTI2IGF0IDE3OjEzICswMDAwLCBIYWl5YW5nIFpoYW5nIHdyb3RlOgo+ ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0KPiA+IEZyb206IE1vaGFtbWVkIEdhbWFsIDxt Z2FtYWxAcmVkaGF0LmNvbT4KPiA+IFNlbnQ6IFdlZG5lc2RheSwgU2VwdGVtYmVyIDI2LCAyMDE4 IDEyOjM0IFBNCj4gPiBUbzogU3RlcGhlbiBIZW1taW5nZXIgPHN0aGVtbWluQG1pY3Jvc29mdC5j b20+OyBuZXRkZXZAdmdlci5rZXJuZWwuCj4gPiBvcmcKPiA+IENjOiBLWSBTcmluaXZhc2FuIDxr eXNAbWljcm9zb2Z0LmNvbT47IEhhaXlhbmcgWmhhbmcKPiA+IDxoYWl5YW5nekBtaWNyb3NvZnQu Y29tPjsgdmt1em5ldHMgPHZrdXpuZXRzQHJlZGhhdC5jb20+Owo+ID4gb3R1Ym9AcmVkaGF0LmNv bTsgY2F2ZXJ5IDxjYXZlcnlAcmVkaGF0LmNvbT47IGxpbnV4LQo+ID4ga2VybmVsQHZnZXIua2Vy bmVsLm9yZzsgZGV2ZWxAbGludXhkcml2ZXJwcm9qZWN0Lm9yZzsgTW9oYW1tZWQKPiA+IEdhbWFs Cj4gPiA8bWdhbWFsQHJlZGhhdC5jb20+Cj4gPiBTdWJqZWN0OiBbUEFUQ0hdIGh2X25ldHZzYzog TWFrZSBzdXJlIG91dCBjaGFubmVsIGlzIGZ1bGx5IG9wZW5lZAo+ID4gb24gc2VuZAo+ID4gCj4g PiBEcmluZyBoaWdoIG5ldHdvcmsgdHJhZmZpYyBjaGFuZ2VzIHRvIG5ldHdvcmsgaW50ZXJmYWNl IHBhcmFtZXRlcnMKPiA+IHN1Y2ggYXMKPiA+IG51bWJlciBvZiBjaGFubmVscyBvciBNVFUgY2Fu IGNhdXNlIGEga2VybmVsIHBhbmljIHdpdGggYSBOVUxMCj4gPiBwb2ludGVyCj4gPiBkZXJlZmVy ZW5jZS4gVGhpcyBpcyBkdWUgdG8gbmV0dnNjX2RldmljZV9yZW1vdmUoKSBiZWluZyBjYWxsZWQg YW5kCj4gPiBkZWFsbG9jYXRpbmcgdGhlIGNoYW5uZWwgcmluZyBidWZmZXJzLCB3aGljaCBjYW4g dGhlbiBiZSBhY2Nlc3NlZAo+ID4gYnkKPiA+IG5ldHZzY19zZW5kX3BrdCgpIGJlZm9yZSB0aGV5 J3JlIGFsbG9jYXRlZCBvbiBjYWxsaW5nCj4gPiBuZXR2c2NfZGV2aWNlX2FkZCgpCj4gPiAKPiA+ IFRoZSBwYXRjaCBmaXhlcyB0aGlzIHByb2JsZW0gYnkgY2hlY2tpbmcgdGhlIGNoYW5uZWwgc3Rh dGUgYW5kCj4gPiByZXR1cm5pbmcKPiA+IEVOT0RFViBpZiBub3QgeWV0IG9wZW5lZC4gV2UgYWxz byBtb3ZlIHRoZSBjYWxsIHRvCj4gPiBodl9yaW5nYnVmX2F2YWlsX3BlcmNlbnQoKQo+ID4gd2hp Y2ggbWF5IGFjY2VzcyB0aGUgdW5pbml0aWFsaXplZCByaW5nIGJ1ZmZlci4KPiA+IAo+ID4gU2ln bmVkLW9mZi1ieTogTW9oYW1tZWQgR2FtYWwgPG1nYW1hbEByZWRoYXQuY29tPgo+ID4gLS0tCj4g PiDCoGRyaXZlcnMvbmV0L2h5cGVydi9uZXR2c2MuYyB8IDcgKysrKysrLQo+ID4gwqAxIGZpbGUg Y2hhbmdlZCwgNiBpbnNlcnRpb25zKCspLCAxIGRlbGV0aW9uKC0pCj4gPiAKPiA+IGRpZmYgLS1n aXQgYS9kcml2ZXJzL25ldC9oeXBlcnYvbmV0dnNjLmMKPiA+IGIvZHJpdmVycy9uZXQvaHlwZXJ2 L25ldHZzYy5jIGluZGV4Cj4gPiBmZTAxZTE0Li43NWYxYjMxIDEwMDY0NAo+ID4gLS0tIGEvZHJp dmVycy9uZXQvaHlwZXJ2L25ldHZzYy5jCj4gPiArKysgYi9kcml2ZXJzL25ldC9oeXBlcnYvbmV0 dnNjLmMKPiA+IEBAIC04MjUsNyArODI1LDEyIEBAIHN0YXRpYyBpbmxpbmUgaW50IG5ldHZzY19z ZW5kX3BrdCgKPiA+IMKgCXN0cnVjdCBuZXRkZXZfcXVldWUgKnR4cSA9IG5ldGRldl9nZXRfdHhf cXVldWUobmRldiwKPiA+IHBhY2tldC0+cV9pZHgpOwo+ID4gwqAJdTY0IHJlcV9pZDsKPiA+IMKg CWludCByZXQ7Cj4gPiAtCXUzMiByaW5nX2F2YWlsID0KPiA+IGh2X2dldF9hdmFpbF90b193cml0 ZV9wZXJjZW50KCZvdXRfY2hhbm5lbC0KPiA+ID4gb3V0Ym91bmQpOwo+ID4gCj4gPiArCXUzMiBy aW5nX2F2YWlsOwo+ID4gKwo+ID4gKwlpZiAob3V0X2NoYW5uZWwtPnN0YXRlICE9IENIQU5ORUxf T1BFTkVEX1NUQVRFKQo+ID4gKwkJcmV0dXJuIC1FTk9ERVY7Cj4gPiArCj4gPiArCXJpbmdfYXZh aWwgPSBodl9nZXRfYXZhaWxfdG9fd3JpdGVfcGVyY2VudCgmb3V0X2NoYW5uZWwtCj4gPiA+b3V0 Ym91bmQpOwo+IAo+IFdoZW4geW91IHJlcHJvZHVjaW5nIHRoZSBOVUxMIHB0ciBwYW5pYywgZG9l cyB5b3VyIGtlcm5lbCBpbmNsdWRlIHRoZQo+IGZvbGxvd2luZyBwYXRjaD8KPiBodl9uZXR2c2M6 IGNvbW1vbiBkZXRhY2ggbG9naWMKPiBodHRwczovL2dpdC5rZXJuZWwub3JnL3B1Yi9zY20vbGlu dXgva2VybmVsL2dpdC9uZXh0L2xpbnV4LW5leHQuZ2l0L2MKPiBvbW1pdC8/aWQ9N2IyZWU1MGMw Y2Q1MTNhMTc2YTI2YTcxZjI5ODlmYWNkZDc1YmZlYQo+IApZZXMgaXQgaXMgaW5jbHVkZWQuIEFu ZCB0aGUgY29tbWl0IGRpZCByZWR1Y2UgdGhlIG9jY3VycmVuY2Ugb2YgdGhpcwpyYWNlIGNvbmRp dGlvbiwgYnV0IGl0IHN0aWxsIG5ldmVydGhlbGVzcyBvY2N1cnMgYWxiZWl0IHJhcmVseS4KCj4g V2UgY2FsbCBuZXRpZl90eF9kaXNhYmxlKG5kZXYpIGFuZCBuZXRpZl9kZXZpY2VfZGV0YWNoKG5k ZXYpIGJlZm9yZQo+IGRvaW5nIHRoZSBjaGFuZ2VzwqAKPiBvbiBNVFUgb3IgI2NoYW5uZWxzLiBT byB0aGVyZSBzaG91bGQgYmUgbm8gY2FsbCB0byBzdGFydF94bWl0KCkgd2hlbgo+IGNoYW5uZWwg aXMgbm90IHJlYWR5Lgo+IAo+IElmIHlvdSBzZWUgdGhlIGNoZWNrIGZvciBDSEFOTkVMX09QRU5F RF9TVEFURSBpcyBzdGlsbCBuZWNlc3Nhcnkgb24KPiB1cHN0cmVhbSBrZXJuZWwgKGluY2x1ZGlu Z8KgCj4gdGhlIHBhdGNoICIgY29tbW9uIGRldGFjaCBsb2dpYyAiKSwgd2Ugc2hvdWxkIGRlYnVn IGZ1cnRoZXIgb24gdGhlCj4gY29kZSBhbmQgZmluZCBvdXQgdGhlwqAKPiByb290IGNhdXNlLgo+ IAo+IFRoYW5rcywKPiAtIEhhaXlhbmcKPiAKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX18KZGV2ZWwgbWFpbGluZyBsaXN0CmRldmVsQGxpbnV4ZHJpdmVycHJv amVjdC5vcmcKaHR0cDovL2RyaXZlcmRldi5saW51eGRyaXZlcnByb2plY3Qub3JnL21haWxtYW4v bGlzdGluZm8vZHJpdmVyZGV2LWRldmVsCg==