From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1AD4547C9E for ; Wed, 27 Sep 2023 15:43:53 +0000 (UTC) Received: from mail-qv1-xf29.google.com (mail-qv1-xf29.google.com [IPv6:2607:f8b0:4864:20::f29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B98849D3 for ; Wed, 27 Sep 2023 08:40:12 -0700 (PDT) Received: by mail-qv1-xf29.google.com with SMTP id 6a1803df08f44-65afae9e51fso46927076d6.0 for ; Wed, 27 Sep 2023 08:40:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1695829211; x=1696434011; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=t9oIEXPNWkoSP0yICfThZznT48YRRSxNiQbO+UpaNXk=; b=REPvhS0od8YGeIuF8pH6xO7Z6Nm5hZjrPlyVdh7k+7W57+wslImmwmFDpr9Si1InPQ v9MKY4koSdpGP9mGG1NiWxhdKQqDUG7D5oDSypk78gCbbiG0P67vj0AzPT/AxcKejuoG 8jdjFvm4LXJ4bZqorrR3X66+ui3ZaiaEsEa96mHzBIl5EzTTpygaN1VywN/JTi966tvF RFZZ4jSkT71tem9hdvKNonJ5b/2pZXxsTznO30rtUPo5hqep55nUvnsMjhyl8hI/Iy5n jPHsWwKXRTzW/KOFjoC9Y7qzBFFdHd1ktdahzs6YfoT5jKaqsI7/Bh5cHg5CzMLkx6Y9 ItgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695829211; x=1696434011; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=t9oIEXPNWkoSP0yICfThZznT48YRRSxNiQbO+UpaNXk=; b=my+3GPvlbtcUS/eFe260mqnl0F4NxCc+Ai+oHt/7LsiOwTaPWkIZHJtJXG1nrPWoJM 71x0dN5o60ySNxz8Bbf0w0SqQAFQLf76EvwpQ9HjvmVzUBsfUnwb/kpZ+NbUMEEiCgHi Mg0j1awms0Qy9IPNJq00fGuZQxUZj2qHlmHq8l0v+Uz2ED6+hA27K5J1KNOT/63qxW5U RtujUehzh4M0/znnXPU3SCc3QzknIAYVCFAoaQwZ40H2zPrUlMzpI0XKD7yMG5HMyCR1 ptgWL0iIrbREhHgf3GewCP2jDmfgVIs20b/HyrbNaExrX4twb993d0efEZtjCAkZ8DKx m6Yw== X-Gm-Message-State: AOJu0YyTfi9ZwTRg7XRmP2gjl8vcovyz3kLpUskYvCmKXJ/n0YdWCMA8 Pgx7lWn141Kn+89y87SX5vKIAQ== X-Google-Smtp-Source: AGHT+IEBIEK8bL4oC9/JbUXAJTo9b7yotL8eHtI4zGCh9Ar0J6VaF2Vk0l+zBuzNRh2jU30sxMvEWQ== X-Received: by 2002:a0c:de03:0:b0:64f:8d4c:1c0b with SMTP id t3-20020a0cde03000000b0064f8d4c1c0bmr2581322qvk.43.1695829210841; Wed, 27 Sep 2023 08:40:10 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-26-201.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.26.201]) by smtp.gmail.com with ESMTPSA id i11-20020a0cab4b000000b00655e4f57732sm3474144qvb.35.2023.09.27.08.40.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 08:40:10 -0700 (PDT) Received: from jgg by wakko with local (Exim 4.95) (envelope-from ) id 1qlWe1-001Qtr-JG; Wed, 27 Sep 2023 12:40:09 -0300 Date: Wed, 27 Sep 2023 12:40:09 -0300 From: Jason Gunthorpe To: Niklas Schnelle Cc: Joerg Roedel , Matthew Rosato , Will Deacon , Wenjia Zhang , Robin Murphy , Gerd Bayer , Julian Ruess , Pierre Morel , Alexandra Winter , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Suravee Suthikulpanit , Hector Martin , Sven Peter , Alyssa Rosenzweig , David Woodhouse , Lu Baolu , Andy Gross , Bjorn Andersson , Konrad Dybcio , Yong Wu , Matthias Brugger , AngeloGioacchino Del Regno , Gerald Schaefer , Orson Zhai , Baolin Wang , Chunyan Zhang , Chen-Yu Tsai , Jernej Skrabec , Samuel Holland , Thierry Reding , Krishna Reddy , Jonathan Hunter , Jonathan Corbet , linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-arm-msm@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, linux-doc@vger.kernel.org Subject: Re: [PATCH v12 0/6] iommu/dma: s390 DMA API conversion and optimized IOTLB flushing Message-ID: <20230927154009.GN13795@ziepe.ca> References: <20230825-dma_iommu-v12-0-4134455994a7@linux.ibm.com> <20230926160832.GM13795@ziepe.ca> <6dab29f58ac1ccd58caaee031f98f4d0d382cbcd.camel@linux.ibm.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net On Wed, Sep 27, 2023 at 05:24:20PM +0200, Niklas Schnelle wrote: > Ok, another update. On trying it out again this problem actually also > occurs when applying this v12 on top of v6.6-rc3 too. Also I guess > unlike my prior thinking it probably doesn't occur with > iommu.forcedac=1 since that still allows IOVAs below 4 GiB and we might > be the only ones who don't support those. From my point of view this > sounds like a mlx5_core issue they really should call > dma_set_mask_and_coherent() before their first call to > dma_alloc_coherent() not after. So I guess I'll send a v13 of this > series rebased on iommu/core and with an additional mlx5 patch and then > let's hope we can get that merged in a way that doesn't leave us with > broken ConnectX VFs for too long. Yes, OK. It definitely sounds wrong that mlx5 is doing dma allocations before setting it's dma_set_mask_and_coherent(). Please link to this thread and we can get Leon or Saeed to ack it for Joerg. (though wondering why s390 is the only case that ever hit this?) Jason