From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23D54C43613 for ; Mon, 24 Jun 2019 07:21:41 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E60482083D for ; Mon, 24 Jun 2019 07:21:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ozlabs-ru.20150623.gappssmtp.com header.i=@ozlabs-ru.20150623.gappssmtp.com header.b="PFzbwHHo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E60482083D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ozlabs.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:48466 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hfJIK-0006Ww-01 for qemu-devel@archiver.kernel.org; Mon, 24 Jun 2019 03:21:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52769) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hfJHO-0005tI-75 for qemu-devel@nongnu.org; Mon, 24 Jun 2019 03:20:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hfJHM-0001vb-PN for qemu-devel@nongnu.org; Mon, 24 Jun 2019 03:20:42 -0400 Received: from mail-pf1-x444.google.com ([2607:f8b0:4864:20::444]:45343) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hfJHM-0001sd-7g for qemu-devel@nongnu.org; Mon, 24 Jun 2019 03:20:40 -0400 Received: by mail-pf1-x444.google.com with SMTP id r1so6966848pfq.12 for ; Mon, 24 Jun 2019 00:20:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ozlabs-ru.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:openpgp:autocrypt:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=CjkC61nGTV4FF48qyFjx2B2JbbIELIAcRpJMgO8KnGo=; b=PFzbwHHoNgOAPyZ9Nk5SX2rN1My35jOTMsF2a85bLuBIidvBESYqDBJuwQiEHptbsJ Uv+StDUXR98j9mEoYUV8dgkNKJXGP3GtOSeIpIb4sBrpSZkjQIIZ1RyMtHe3CRmN+2L4 MQiCQlWLyJLR93D9RPqU8QYfeoqwjqAbyEJBNUknLK28FLPUD74enjYfy/eu6FCccWxh 4jekiN4ztnzpMNXXnDdrzjJnqRc+RIbvCe8RAMzn+CafjytiZbzOTFsHqie4br3rQg8b g3ns1ZAQ32aUsGXAVleHW1VIU5LfvTGdtLMho3yRhUpBgH8Cgj2qG8LiPjeH/67ug695 aU+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=CjkC61nGTV4FF48qyFjx2B2JbbIELIAcRpJMgO8KnGo=; b=YUGvjlrrAGcw/vK1+KdbUiDcN/PZDmRyzLrBWiRllrb2l7l3qdyY7IGLfQDzOxVcRE VRuMrZo/x/Td+G0ydZcw/XyKgKzKJFCv2m0cLD12VZpTFJ9hbKEM/s4To/o36Uds1tvd FLKNZxkVJGShEqkTXo3N3LmVUvXRzsKyWvrzLEOLnTjtBpB965DHqukvZdgd22bA/Uay IJRkube6ZDfCEOV6BCKYDVAl5HGP+dbdVqxibYWkxXBlBwm77PvBEALc/xmtlACWoY0j uwRCwNtaU0f9udH/qYrpWPInoHDvTIHPymkE7uUBGxNL/sgNdChYgGH9VrwqZJA4tKu9 C9fA== X-Gm-Message-State: APjAAAUm3nzhu2hOg2phvm1wXGt1PqIUrCmaTBtAfV0a4LhheDL/qb7c 77UQ6YpSxmfcz0zXTc4+UEwpAg== X-Google-Smtp-Source: APXvYqxTbvuUjQMBBabX0bwUYF9bYDsDMnpHrNTrgnkXLCLIbJR/7l3pwW1emCAUCnT4WyGbCN58UQ== X-Received: by 2002:a17:90a:e397:: with SMTP id b23mr22781682pjz.140.1561360838543; Mon, 24 Jun 2019 00:20:38 -0700 (PDT) Received: from [10.61.2.175] ([122.99.82.10]) by smtp.gmail.com with ESMTPSA id u23sm12156197pfn.140.2019.06.24.00.20.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 00:20:37 -0700 (PDT) To: Greg Kurz , David Gibson , Paolo Bonzini References: <156110925375.92514.11649846071216864570.stgit@bahia.lan> From: Alexey Kardashevskiy Openpgp: preference=signencrypt Autocrypt: addr=aik@ozlabs.ru; keydata= mQINBE+rT0sBEADFEI2UtPRsLLvnRf+tI9nA8T91+jDK3NLkqV+2DKHkTGPP5qzDZpRSH6mD EePO1JqpVuIow/wGud9xaPA5uvuVgRS1q7RU8otD+7VLDFzPRiRE4Jfr2CW89Ox6BF+q5ZPV /pS4v4G9eOrw1v09lEKHB9WtiBVhhxKK1LnUjPEH3ifkOkgW7jFfoYgTdtB3XaXVgYnNPDFo PTBYsJy+wr89XfyHr2Ev7BB3Xaf7qICXdBF8MEVY8t/UFsesg4wFWOuzCfqxFmKEaPDZlTuR tfLAeVpslNfWCi5ybPlowLx6KJqOsI9R2a9o4qRXWGP7IwiMRAC3iiPyk9cknt8ee6EUIxI6 t847eFaVKI/6WcxhszI0R6Cj+N4y+1rHfkGWYWupCiHwj9DjILW9iEAncVgQmkNPpUsZECLT WQzMuVSxjuXW4nJ6f4OFHqL2dU//qR+BM/eJ0TT3OnfLcPqfucGxubhT7n/CXUxEy+mvWwnm s9p4uqVpTfEuzQ0/bE6t7dZdPBua7eYox1AQnk8JQDwC3Rn9kZq2O7u5KuJP5MfludMmQevm pHYEMF4vZuIpWcOrrSctJfIIEyhDoDmR34bCXAZfNJ4p4H6TPqPh671uMQV82CfTxTrMhGFq 8WYU2AH86FrVQfWoH09z1WqhlOm/KZhAV5FndwVjQJs1MRXD8QARAQABtCRBbGV4ZXkgS2Fy ZGFzaGV2c2tpeSA8YWlrQG96bGFicy5ydT6JAjgEEwECACIFAk+rT0sCGwMGCwkIBwMCBhUI AgkKCwQWAgMBAh4BAheAAAoJEIYTPdgrwSC5fAIP/0wf/oSYaCq9PhO0UP9zLSEz66SSZUf7 AM9O1rau1lJpT8RoNa0hXFXIVbqPPKPZgorQV8SVmYRLr0oSmPnTiZC82x2dJGOR8x4E01gK TanY53J/Z6+CpYykqcIpOlGsytUTBA+AFOpdaFxnJ9a8p2wA586fhCZHVpV7W6EtUPH1SFTQ q5xvBmr3KkWGjz1FSLH4FeB70zP6uyuf/B2KPmdlPkyuoafl2UrU8LBADi/efc53PZUAREih sm3ch4AxaL4QIWOmlE93S+9nHZSRo9jgGXB1LzAiMRII3/2Leg7O4hBHZ9Nki8/fbDo5///+ kD4L7UNbSUM/ACWHhd4m1zkzTbyRzvL8NAVQ3rckLOmju7Eu9whiPueGMi5sihy9VQKHmEOx OMEhxLRQbzj4ypRLS9a+oxk1BMMu9cd/TccNy0uwx2UUjDQw/cXw2rRWTRCxoKmUsQ+eNWEd iYLW6TCfl9CfHlT6A7Zmeqx2DCeFafqEd69DqR9A8W5rx6LQcl0iOlkNqJxxbbW3ddDsLU/Y r4cY20++WwOhSNghhtrroP+gouTOIrNE/tvG16jHs8nrYBZuc02nfX1/gd8eguNfVX/ZTHiR gHBWe40xBKwBEK2UeqSpeVTohYWGBkcd64naGtK9qHdo1zY1P55lHEc5Uhlk743PgAnOi27Q ns5zuQINBE+rT0sBEACnV6GBSm+25ACT+XAE0t6HHAwDy+UKfPNaQBNTTt31GIk5aXb2Kl/p AgwZhQFEjZwDbl9D/f2GtmUHWKcCmWsYd5M/6Ljnbp0Ti5/xi6FyfqnO+G/wD2VhGcKBId1X Em/B5y1kZVbzcGVjgD3HiRTqE63UPld45bgK2XVbi2+x8lFvzuFq56E3ZsJZ+WrXpArQXib2 hzNFwQleq/KLBDOqTT7H+NpjPFR09Qzfa7wIU6pMNF2uFg5ihb+KatxgRDHg70+BzQfa6PPA o1xioKXW1eHeRGMmULM0Eweuvpc7/STD3K7EJ5bBq8svoXKuRxoWRkAp9Ll65KTUXgfS+c0x gkzJAn8aTG0z/oEJCKPJ08CtYQ5j7AgWJBIqG+PpYrEkhjzSn+DZ5Yl8r+JnZ2cJlYsUHAB9 jwBnWmLCR3gfop65q84zLXRQKWkASRhBp4JK3IS2Zz7Nd/Sqsowwh8x+3/IUxVEIMaVoUaxk Wt8kx40h3VrnLTFRQwQChm/TBtXqVFIuv7/Mhvvcq11xnzKjm2FCnTvCh6T2wJw3de6kYjCO 7wsaQ2y3i1Gkad45S0hzag/AuhQJbieowKecuI7WSeV8AOFVHmgfhKti8t4Ff758Z0tw5Fpc BFDngh6Lty9yR/fKrbkkp6ux1gJ2QncwK1v5kFks82Cgj+DSXK6GUQARAQABiQIfBBgBAgAJ BQJPq09LAhsMAAoJEIYTPdgrwSC5NYEP/2DmcEa7K9A+BT2+G5GXaaiFa098DeDrnjmRvumJ BhA1UdZRdfqICBADmKHlJjj2xYo387sZpS6ABbhrFxM6s37g/pGPvFUFn49C47SqkoGcbeDz Ha7JHyYUC+Tz1dpB8EQDh5xHMXj7t59mRDgsZ2uVBKtXj2ZkbizSHlyoeCfs1gZKQgQE8Ffc F8eWKoqAQtn3j4nE3RXbxzTJJfExjFB53vy2wV48fUBdyoXKwE85fiPglQ8bU++0XdOr9oyy j1llZlB9t3tKVv401JAdX8EN0++ETiOovQdzE1m+6ioDCtKEx84ObZJM0yGSEGEanrWjiwsa nzeK0pJQM9EwoEYi8TBGhHC9ksaAAQipSH7F2OHSYIlYtd91QoiemgclZcSgrxKSJhyFhmLr QEiEILTKn/pqJfhHU/7R7UtlDAmFMUp7ByywB4JLcyD10lTmrEJ0iyRRTVfDrfVP82aMBXgF tKQaCxcmLCaEtrSrYGzd1sSPwJne9ssfq0SE/LM1J7VdCjm6OWV33SwKrfd6rOtvOzgadrG6 3bgUVBw+bsXhWDd8tvuCXmdY4bnUblxF2B6GOwSY43v6suugBttIyW5Bl2tXSTwP+zQisOJo +dpVG2pRr39h+buHB3NY83NEPXm1kUOhduJUA17XUY6QQCAaN4sdwPqHq938S3EmtVhsuQIN BFq54uIBEACtPWrRdrvqfwQF+KMieDAMGdWKGSYSfoEGGJ+iNR8v255IyCMkty+yaHafvzpl PFtBQ/D7Fjv+PoHdFq1BnNTk8u2ngfbre9wd9MvTDsyP/TmpF0wyyTXhhtYvE267Av4X/BQT lT9IXKyAf1fP4BGYdTNgQZmAjrRsVUW0j6gFDrN0rq2J9emkGIPvt9rQt6xGzrd6aXonbg5V j6Uac1F42ESOZkIh5cN6cgnGdqAQb8CgLK92Yc8eiCVCH3cGowtzQ2m6U32qf30cBWmzfSH0 HeYmTP9+5L8qSTA9s3z0228vlaY0cFGcXjdodBeVbhqQYseMF9FXiEyRs28uHAJEyvVZwI49 CnAgVV/n1eZa5qOBpBL+ZSURm8Ii0vgfvGSijPGbvc32UAeAmBWISm7QOmc6sWa1tobCiVmY SNzj5MCNk8z4cddoKIc7Wt197+X/X5JPUF5nQRvg3SEHvfjkS4uEst9GwQBpsbQYH9MYWq2P PdxZ+xQE6v7cNB/pGGyXqKjYCm6v70JOzJFmheuUq0Ljnfhfs15DmZaLCGSMC0Amr+rtefpA y9FO5KaARgdhVjP2svc1F9KmTUGinSfuFm3quadGcQbJw+lJNYIfM7PMS9fftq6vCUBoGu3L j4xlgA/uQl/LPneu9mcvit8JqcWGS3fO+YeagUOon1TRqQARAQABiQRsBBgBCAAgFiEEZSrP ibrORRTHQ99dhhM92CvBILkFAlq54uICGwICQAkQhhM92CvBILnBdCAEGQEIAB0WIQQIhvWx rCU+BGX+nH3N7sq0YorTbQUCWrni4gAKCRDN7sq0YorTbVVSD/9V1xkVFyUCZfWlRuryBRZm S4GVaNtiV2nfUfcThQBfF0sSW/aFkLP6y+35wlOGJE65Riw1C2Ca9WQYk0xKvcZrmuYkK3DZ 0M9/Ikkj5/2v0vxz5Z5w/9+IaCrnk7pTnHZuZqOh23NeVZGBls/IDIvvLEjpD5UYicH0wxv+ X6cl1RoP2Kiyvenf0cS73O22qSEw0Qb9SId8wh0+ClWet2E7hkjWFkQfgJ3hujR/JtwDT/8h 3oCZFR0KuMPHRDsCepaqb/k7VSGTLBjVDOmr6/C9FHSjq0WrVB9LGOkdnr/xcISDZcMIpbRm EkIQ91LkT/HYIImL33ynPB0SmA+1TyMgOMZ4bakFCEn1vxB8Ir8qx5O0lHMOiWMJAp/PAZB2 r4XSSHNlXUaWUg1w3SG2CQKMFX7vzA31ZeEiWO8tj/c2ZjQmYjTLlfDK04WpOy1vTeP45LG2 wwtMA1pKvQ9UdbYbovz92oyZXHq81+k5Fj/YA1y2PI4MdHO4QobzgREoPGDkn6QlbJUBf4To pEbIGgW5LRPLuFlOPWHmIS/sdXDrllPc29aX2P7zdD/ivHABslHmt7vN3QY+hG0xgsCO1JG5 pLORF2N5XpM95zxkZqvYfC5tS/qhKyMcn1kC0fcRySVVeR3tUkU8/caCqxOqeMe2B6yTiU1P aNDq25qYFLeYxg67D/4w/P6BvNxNxk8hx6oQ10TOlnmeWp1q0cuutccblU3ryRFLDJSngTEu ZgnOt5dUFuOZxmMkqXGPHP1iOb+YDznHmC0FYZFG2KAc9pO0WuO7uT70lL6larTQrEneTDxQ CMQLP3qAJ/2aBH6SzHIQ7sfbsxy/63jAiHiT3cOaxAKsWkoV2HQpnmPOJ9u02TPjYmdpeIfa X2tXyeBixa3i/6dWJ4nIp3vGQicQkut1YBwR7dJq67/FCV3Mlj94jI0myHT5PIrCS2S8LtWX ikTJSxWUKmh7OP5mrqhwNe0ezgGiWxxvyNwThOHc5JvpzJLd32VDFilbxgu4Hhnf6LcgZJ2c Zd44XWqUu7FzVOYaSgIvTP0hNrBYm/E6M7yrLbs3JY74fGzPWGRbBUHTZXQEqQnZglXaVB5V ZhSFtHopZnBSCUSNDbB+QGy4B/E++Bb02IBTGl/JxmOwG+kZUnymsPvTtnNIeTLHxN/H/ae0 c7E5M+/NpslPCmYnDjs5qg0/3ihh6XuOGggZQOqrYPC3PnsNs3NxirwOkVPQgO6mXxpuifvJ DG9EMkK8IBXnLulqVk54kf7fE0jT/d8RTtJIA92GzsgdK2rpT1MBKKVffjRFGwN7nQVOzi4T XrB5p+6ML7Bd84xOEGsj/vdaXmz1esuH7BOZAGEZfLRCHJ0GVCSssg== Message-ID: Date: Mon, 24 Jun 2019 17:20:32 +1000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 In-Reply-To: <156110925375.92514.11649846071216864570.stgit@bahia.lan> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::444 Subject: Re: [Qemu-devel] [PATCH] spapr_pci: Unregister listeners before destroying the IOMMU address space X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alex Williamson , qemu-ppc@nongnu.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On 21/06/2019 19:27, Greg Kurz wrote: > Hot-unplugging a PHB with a VFIO device connected to it crashes QEMU: > > -device spapr-pci-host-bridge,index=1,id=phb1 \ > -device vfio-pci,host=0034:01:00.3,id=vfio0 > > (qemu) device_del phb1 > [ 357.207183] iommu: Removing device 0001:00:00.0 from group 1 > [ 360.375523] rpadlpar_io: slot PHB 1 removed > qemu-system-ppc64: memory.c:2742: > do_address_space_destroy: Assertion `QTAILQ_EMPTY(&as->listeners)' failed. > > 'as' is the IOMMU address space, which indeed has a listener registered > to by vfio_connect_container() when the VFIO device is realized. This > listener is supposed to be unregistered by vfio_disconnect_container() > when the VFIO device is finalized. Unfortunately, the VFIO device hasn't > reached finalize yet at the time the PHB unrealize function is called, > and address_space_destroy() gets called with the VFIO listener still > being registered. > > All regions have just been unmapped from the address space. Listeners > aren't needed anymore at this point. Remove them before destroying the > address space. > > The VFIO code will try to remove them _again_ at device finalize, > but it is okay since memory_listener_unregister() is idempotent. > > Signed-off-by: Greg Kurz > --- > hw/ppc/spapr_pci.c | 6 ++++++ > include/exec/memory.h | 10 ++++++++++ > memory.c | 7 +++++++ > 3 files changed, 23 insertions(+) > > diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c > index 2dca1e57f36c..eee92b102d5c 100644 > --- a/hw/ppc/spapr_pci.c > +++ b/hw/ppc/spapr_pci.c > @@ -1788,6 +1788,12 @@ static void spapr_phb_unrealize(DeviceState *dev, Error **errp) > > memory_region_del_subregion(&sphb->iommu_root, &sphb->msiwindow); > > + /* > + * An attached PCI device may have memory listeners, eg. VFIO PCI. We have > + * unmapped all sections. Remove the listeners now, before destroying the > + * address space. > + */ The code around the comment (which is slightly incorrect as containers have listeners, not devices) looks self-explanatory imho. > + address_space_remove_listeners(&sphb->iommu_as); > address_space_destroy(&sphb->iommu_as); > > qbus_set_hotplug_handler(BUS(phb->bus), NULL, &error_abort); > diff --git a/include/exec/memory.h b/include/exec/memory.h > index e6140e8a0489..1ba2e89aa8ce 100644 > --- a/include/exec/memory.h > +++ b/include/exec/memory.h > @@ -1757,6 +1757,16 @@ void address_space_init(AddressSpace *as, MemoryRegion *root, const char *name); > */ > void address_space_destroy(AddressSpace *as); > > +/** > + * address_space_remove_listeners: unregister all listerners of an address space s/listerners/listeners/g Otherwise, fixes the bug and Reviewed-by: Alexey Kardashevskiy > + * > + * Removes all callbacks previously registered with memory_listener_register() > + * for @as. > + * > + * @as: an initialized #AddressSpace > + */ > +void address_space_remove_listeners(AddressSpace *as); > + > /** > * address_space_rw: read from or write to an address space. > * > diff --git a/memory.c b/memory.c > index 0a089a73ae1a..480f3d989b4f 100644 > --- a/memory.c > +++ b/memory.c > @@ -2723,6 +2723,13 @@ void memory_listener_unregister(MemoryListener *listener) > listener->address_space = NULL; > } > > +void address_space_remove_listeners(AddressSpace *as) > +{ > + while (!QTAILQ_EMPTY(&as->listeners)) { > + memory_listener_unregister(QTAILQ_FIRST(&as->listeners)); > + } > +} > + > void address_space_init(AddressSpace *as, MemoryRegion *root, const char *name) > { > memory_region_ref(root); > -- Alexey