From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95620C4360F for ; Wed, 3 Apr 2019 08:03:07 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 625CA20830 for ; Wed, 3 Apr 2019 08:03:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="R7HD9fHw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 625CA20830 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:References: To:Subject:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=8cek90vTZ9BHlB9fTVQQN687rb/5ZEpte6Egs8Ce8XE=; b=R7HD9fHwwAJel8g8JOqS2tY40 4cZVZPx/QiY9QLXObqHk4icLE6ninzL3zjgo8wbeu8HB0POP1b35xBTeC98GG2h25tYRV26LigVVk IFQs1Mtr3zSOZYvXGhqkSzJ/nySm/EnxzysjhXss8IPAO3RX+9KaAJRqCIiR2NZ4AM7qEO7Fcyvmk QtXFbuYWn+wGt52+o1Ff2O7LXjcZachDUbMAHp3bwtQB07efMrWS6klJugdcR2CzMHl9oukcjdnza AKp1Ql0qGpvg9Qx8AU0HOjqDcouQtvfpz5NX029TeczsNS6csYKWKSrYlmrLaIfQJjMExbVP+0VMH Nqct9e+7g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hBarP-0008Sz-1A; Wed, 03 Apr 2019 08:03:03 +0000 Received: from szxga06-in.huawei.com ([45.249.212.32] helo=huawei.com) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1hBarK-0008SG-SX for linux-arm-kernel@lists.infradead.org; Wed, 03 Apr 2019 08:03:00 +0000 Received: from DGGEMS407-HUB.china.huawei.com (unknown [10.3.19.207]) by Forcepoint Email with ESMTP id DBFF020F5E0B15893976; Wed, 3 Apr 2019 16:02:53 +0800 (CST) Received: from [127.0.0.1] (10.202.227.238) by DGGEMS407-HUB.china.huawei.com (10.3.19.207) with Microsoft SMTP Server id 14.3.408.0; Wed, 3 Apr 2019 16:02:42 +0800 From: John Garry Subject: Re: [PATCH] driver core: Postpone DMA tear-down until after devres release for probe failure To: , References: <1553767685-27077-1-git-send-email-john.garry@huawei.com> Message-ID: <703aca1f-9475-b50d-624a-5f1ceea2c3b2@huawei.com> Date: Wed, 3 Apr 2019 09:02:36 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <1553767685-27077-1-git-send-email-john.garry@huawei.com> X-Originating-IP: [10.202.227.238] X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190403_010259_278279_C06CFAC8 X-CRM114-Status: GOOD ( 22.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: robh@kernel.org, chenxiang66@hisilicon.com, linuxarm@huawei.com, linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, geert@linux-m68k.org, robin.murphy@arm.com, hch@lst.de, linux-arm-kernel@lists.infradead.org Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 28/03/2019 10:08, John Garry wrote: > In commit 376991db4b64 ("driver core: Postpone DMA tear-down until after > devres release"), we changed the ordering of tearing down the device DMA > ops and releasing all the device's resources; this was because the DMA ops > should be maintained until we release the device's managed DMA memories. > Hi all, A friendly reminder on this patch... I didn't see any update. I thought that it had some importance. Thanks, John > However, we have seen another crash on an arm64 system when a > device driver probe fails: > > hisi_sas_v3_hw 0000:74:02.0: Adding to iommu group 2 > scsi host1: hisi_sas_v3_hw > BUG: Bad page state in process swapper/0 pfn:313f5 > page:ffff7e0000c4fd40 count:1 mapcount:0 > mapping:0000000000000000 index:0x0 > flags: 0xfffe00000001000(reserved) > raw: 0fffe00000001000 ffff7e0000c4fd48 ffff7e0000c4fd48 > 0000000000000000 > raw: 0000000000000000 0000000000000000 00000001ffffffff > 0000000000000000 > page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set > bad because of flags: 0x1000(reserved) > Modules linked in: > CPU: 49 PID: 1 Comm: swapper/0 Not tainted > 5.1.0-rc1-43081-g22d97fd-dirty #1433 > Hardware name: Huawei D06/D06, BIOS Hisilicon D06 UEFI > RC0 - V1.12.01 01/29/2019 > Call trace: > dump_backtrace+0x0/0x118 > show_stack+0x14/0x1c > dump_stack+0xa4/0xc8 > bad_page+0xe4/0x13c > free_pages_check_bad+0x4c/0xc0 > __free_pages_ok+0x30c/0x340 > __free_pages+0x30/0x44 > __dma_direct_free_pages+0x30/0x38 > dma_direct_free+0x24/0x38 > dma_free_attrs+0x9c/0xd8 > dmam_release+0x20/0x28 > release_nodes+0x17c/0x220 > devres_release_all+0x34/0x54 > really_probe+0xc4/0x2c8 > driver_probe_device+0x58/0xfc > device_driver_attach+0x68/0x70 > __driver_attach+0x94/0xdc > bus_for_each_dev+0x5c/0xb4 > driver_attach+0x20/0x28 > bus_add_driver+0x14c/0x200 > driver_register+0x6c/0x124 > __pci_register_driver+0x48/0x50 > sas_v3_pci_driver_init+0x20/0x28 > do_one_initcall+0x40/0x25c > kernel_init_freeable+0x2b8/0x3c0 > kernel_init+0x10/0x100 > ret_from_fork+0x10/0x18 > Disabling lock debugging due to kernel taint > BUG: Bad page state in process swapper/0 pfn:313f6 > page:ffff7e0000c4fd80 count:1 mapcount:0 > mapping:0000000000000000 index:0x0 > [ 89.322983] flags: 0xfffe00000001000(reserved) > raw: 0fffe00000001000 ffff7e0000c4fd88 ffff7e0000c4fd88 > 0000000000000000 > raw: 0000000000000000 0000000000000000 00000001ffffffff > 0000000000000000 > > The crash occurs for the same reason. > > In this case, on the really_probe() failure path, we are still clearing > the DMA ops prior to releasing the device's managed memories. > > This patch fixes this issue by reordering the DMA ops teardown and the > call to devres_release_all() on the failure path. > > Reported-by: Xiang Chen > Tested-by: Xiang Chen > Signed-off-by: John Garry > --- > > For convenience, here is the 2nd half of really_probe() now: > > atomic_inc(&probe_count); > pr_debug("bus: '%s': %s: probing driver %s with device %s\n", > drv->bus->name, __func__, drv->name, dev_name(dev)); > WARN_ON(!list_empty(&dev->devres_head)); > > re_probe: > dev->driver = drv; > > /* If using pinctrl, bind pins now before probing */ > ret = pinctrl_bind_pins(dev); > if (ret) > goto pinctrl_bind_failed; > > if (dev->bus->dma_configure) { > ret = dev->bus->dma_configure(dev); > if (ret) > goto probe_failed; > } > > if (driver_sysfs_add(dev)) { > printk(KERN_ERR "%s: driver_sysfs_add(%s) failed\n", > __func__, dev_name(dev)); > goto probe_failed; > } > > if (dev->pm_domain && dev->pm_domain->activate) { > ret = dev->pm_domain->activate(dev); > if (ret) > goto probe_failed; > } > > if (dev->bus->probe) { > ret = dev->bus->probe(dev); > if (ret) > goto probe_failed; > } else if (drv->probe) { > ret = drv->probe(dev); > if (ret) > goto probe_failed; > } > > if (test_remove) { > test_remove = false; > > if (dev->bus->remove) > dev->bus->remove(dev); > else if (drv->remove) > drv->remove(dev); > > devres_release_all(dev); > driver_sysfs_remove(dev); > dev->driver = NULL; > dev_set_drvdata(dev, NULL); > if (dev->pm_domain && dev->pm_domain->dismiss) > dev->pm_domain->dismiss(dev); > pm_runtime_reinit(dev); > > goto re_probe; > } > > pinctrl_init_done(dev); > > if (dev->pm_domain && dev->pm_domain->sync) > dev->pm_domain->sync(dev); > > driver_bound(dev); > ret = 1; > pr_debug("bus: '%s': %s: bound device %s to driver %s\n", > drv->bus->name, __func__, dev_name(dev), drv->name); > goto done; > > probe_failed: > if (dev->bus) > blocking_notifier_call_chain(&dev->bus->p->bus_notifier, > BUS_NOTIFY_DRIVER_NOT_BOUND, dev); > pinctrl_bind_failed: > device_links_no_driver(dev); > devres_release_all(dev); > arch_teardown_dma_ops(dev); > driver_sysfs_remove(dev); > dev->driver = NULL; > dev_set_drvdata(dev, NULL); > if (dev->pm_domain && dev->pm_domain->dismiss) > dev->pm_domain->dismiss(dev); > pm_runtime_reinit(dev); > dev_pm_set_driver_flags(dev, 0); > > switch (ret) { > case -EPROBE_DEFER: > /* Driver requested deferred probing */ > dev_dbg(dev, "Driver %s requests probe deferral\n", drv->name); > driver_deferred_probe_add_trigger(dev, local_trigger_count); > break; > case -ENODEV: > case -ENXIO: > pr_debug("%s: probe of %s rejects match %d\n", > drv->name, dev_name(dev), ret); > break; > default: > /* driver matched but the probe failed */ > printk(KERN_WARNING > "%s: probe of %s failed with error %d\n", > drv->name, dev_name(dev), ret); > } > /* > * Ignore errors returned by ->probe so that the next driver can try > * its luck. > */ > ret = 0; > done: > atomic_dec(&probe_count); > wake_up(&probe_waitqueue); > return ret; > } > > diff --git a/drivers/base/dd.c b/drivers/base/dd.c > index a823f469e53f..0df9b4461766 100644 > --- a/drivers/base/dd.c > +++ b/drivers/base/dd.c > @@ -490,7 +490,7 @@ static int really_probe(struct device *dev, struct device_driver *drv) > if (dev->bus->dma_configure) { > ret = dev->bus->dma_configure(dev); > if (ret) > - goto dma_failed; > + goto probe_failed; > } > > if (driver_sysfs_add(dev)) { > @@ -546,14 +546,13 @@ static int really_probe(struct device *dev, struct device_driver *drv) > goto done; > > probe_failed: > - arch_teardown_dma_ops(dev); > -dma_failed: > if (dev->bus) > blocking_notifier_call_chain(&dev->bus->p->bus_notifier, > BUS_NOTIFY_DRIVER_NOT_BOUND, dev); > pinctrl_bind_failed: > device_links_no_driver(dev); > devres_release_all(dev); > + arch_teardown_dma_ops(dev); > driver_sysfs_remove(dev); > dev->driver = NULL; > dev_set_drvdata(dev, NULL); > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel