From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 968211A9FA0 for ; Tue, 30 Sep 2025 16:01:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759248082; cv=none; b=V2lIMw9L3y7tyvlnYIAQNaDlHE58EKtN2jqajNoe9g4xa6q4rJt7QFlqkyE7JyUlLsj8ItHREIyuSvh1jP2Z6ouYC1pofCATvbOukF8St0Xlifc3PYXnXYIQBw9LIAMn08f83MEXbz2kSDMmG6K7zG2NkpuLw/ox7Hetxmsg18s= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759248082; c=relaxed/simple; bh=Bn7zEJ911XHzxxR8rR96F8Z2uaJ/NUbD+h2AbVhFNRg=; h=Date:From:To:Cc:Subject:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=T6iXSuavu4vvHQj2clSul3oVHa2osyka1at01M43YePM2d1IInWY88fL2JzX0yqtQQyCFj+r+ySYo4MM/yOuAjIBRrezRp0wWcepVAEL2mBGoeJATuFNH/kwou/FH4fRsaG2Vf4yHrNVNhmalfrld50W+H12rrd99VAeSaNojY8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Rwg0TymT; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Rwg0TymT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1759248078; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HOYVDlXcgikSCUwOdxOfSsjt8Nh4FIwdpCdV/JBPVPE=; b=Rwg0TymTof2GChCfuZLJYnFJ5RK2ZAdw4yisax3/tLWEyG3H/OEqAgt4rU8OYzy4bplGgl JyxSL+dJ2qK7nNDg8vk7K0bkbVzUyv6MAdG1Lb1okeuTxXU0WfmTn/Q8AwH6Xw17gDP442 clCJV8veraieu2+hxfzWPOrRASmkCSA= Received: from mail-il1-f200.google.com (mail-il1-f200.google.com [209.85.166.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-628-FH0DfcGnN1iDadTEDT-M2Q-1; Tue, 30 Sep 2025 12:01:16 -0400 X-MC-Unique: FH0DfcGnN1iDadTEDT-M2Q-1 X-Mimecast-MFC-AGG-ID: FH0DfcGnN1iDadTEDT-M2Q_1759248075 Received: by mail-il1-f200.google.com with SMTP id e9e14a558f8ab-427a3ef578dso8004435ab.2 for ; Tue, 30 Sep 2025 09:01:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759248075; x=1759852875; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HOYVDlXcgikSCUwOdxOfSsjt8Nh4FIwdpCdV/JBPVPE=; b=OugHUK1aLpj6YEB4TJk103C6Zux//Rc43Fp09FG+4Yp6j1tr5TzYCYLwkHnePxkkwH FfF2CgTI9cQZQspqGnPpWYsz49XEckuoey+weSwWQCFNSJhXAdn1HUqHsTWx5VXD9UWT aMn2qdfLCoQJ6/o7BGa9ZY/3FhSyYyhZYF+NgZ9Kaz09ksDOmS+nISJE7f4oxkqO8m3R FvDuccOL/7E397UJH8jVOo8Uyuu0lhNkcqg06eGEINh39VQiR+PxWTNsQ+7sVnsMFkXY 79kKs8gUSwzJjDEBvdpzvlDwYMzDlbA//GclC6LrnWILLikH6SnIPc4ymv41K8Z/T9HA nMLg== X-Forwarded-Encrypted: i=1; AJvYcCV1Qm8MqjwB6Os2G5/BCVQ9mFMUVgVv1Cn04PuX0Bpqdk8lKRTKAboCQVxsu2vzzcvyvGs9xEHcr2E=@vger.kernel.org X-Gm-Message-State: AOJu0YyH0MjvcUMPhLEJ2iMBV+DPBNUAcwZozWVwA9oFPzzXj57yIwjO QF9vr/2oc3Yy2V6aH9j+uQDA4PVduUVGiBRauI+k/boG4tz4wFMsftc4H6WdVjNm9+Y+HLB/0H6 8bIh8CgcgieFwCV4KVxMVqZTajhjMBkEdpAiIXQwQw9RiwiLJzECDoThBQL45PQ== X-Gm-Gg: ASbGncunFxxXEbS3xDibiTDzuprfdOF7AdNOWsOJrym1YyE7l5M6tNeU3QiHVESquN2 9zF8TH14FUbGlHiF2xuoNBoeAw+74THlIYzrW8HDX/g8TaJZoPJEEC7Tm2cLYReQhhXtjs+NgPN gU9N+G4XbQ2fXMs+fNNgtA738WKYAbeSrSOyDqbwn5XGSvcO/UzjIwuzFzaBlUqDRos6aksgERe suh8ONoQGPzkupldpFWrz7Ek8y+LsnogFoUZJoG0GkKt739K8M9PwBEH6gAJVqhTJGeyyh/ba9F 0t6N8WL5bqx48vAnAomXwUaw6bE6vsRBVMNKEXa12QhdFADX X-Received: by 2002:a05:6e02:164b:b0:42b:1763:5796 with SMTP id e9e14a558f8ab-42d81635257mr2672765ab.7.1759248075077; Tue, 30 Sep 2025 09:01:15 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFa3YWCojRcq8FvhA8oy0OghueS9jHVQ9BVoBDjYigh74bKjlyhu7iHP48jxbQj2TC5kTDAzg== X-Received: by 2002:a05:6e02:164b:b0:42b:1763:5796 with SMTP id e9e14a558f8ab-42d81635257mr2672385ab.7.1759248074259; Tue, 30 Sep 2025 09:01:14 -0700 (PDT) Received: from redhat.com ([38.15.36.11]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-425bfba6242sm68758215ab.27.2025.09.30.09.01.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Sep 2025 09:01:13 -0700 (PDT) Date: Tue, 30 Sep 2025 10:01:10 -0600 From: Alex Williamson To: Leon Romanovsky , Marek Szyprowski Cc: Jason Gunthorpe , Andrew Morton , Bjorn Helgaas , Christian =?UTF-8?B?S8O2bmln?= , dri-devel@lists.freedesktop.org, iommu@lists.linux.dev, Jens Axboe , Joerg Roedel , kvm@vger.kernel.org, linaro-mm-sig@lists.linaro.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Logan Gunthorpe , Robin Murphy , Sumit Semwal , Vivek Kasireddy , Will Deacon Subject: Re: [PATCH v4 08/10] vfio/pci: Enable peer-to-peer DMA transactions by default Message-ID: <20250930100110.6ec5b8a1.alex.williamson@redhat.com> In-Reply-To: <20250930073053.GE324804@unreal> References: <20250929151745.439be1ec.alex.williamson@redhat.com> <20250930073053.GE324804@unreal> X-Mailer: Claws Mail 4.3.1 (GTK 3.24.43; x86_64-redhat-linux-gnu) Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit On Tue, 30 Sep 2025 10:30:53 +0300 Leon Romanovsky wrote: > On Mon, Sep 29, 2025 at 03:17:45PM -0600, Alex Williamson wrote: > > On Sun, 28 Sep 2025 17:50:18 +0300 > > Leon Romanovsky wrote: > > > > > From: Leon Romanovsky > > > > > > Make sure that all VFIO PCI devices have peer-to-peer capabilities > > > enables, so we would be able to export their MMIO memory through DMABUF, > > > > > > Signed-off-by: Leon Romanovsky > > > --- > > > drivers/vfio/pci/vfio_pci_core.c | 9 +++++++++ > > > 1 file changed, 9 insertions(+) > > > > > > diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c > > > index 7dcf5439dedc..608af135308e 100644 > > > --- a/drivers/vfio/pci/vfio_pci_core.c > > > +++ b/drivers/vfio/pci/vfio_pci_core.c > > > @@ -28,6 +28,9 @@ > > > #include > > > #include > > > #include > > > +#ifdef CONFIG_VFIO_PCI_DMABUF > > > +#include > > > +#endif > > > #if IS_ENABLED(CONFIG_EEH) > > > #include > > > #endif > > > @@ -2085,6 +2088,7 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev) > > > { > > > struct vfio_pci_core_device *vdev = > > > container_of(core_vdev, struct vfio_pci_core_device, vdev); > > > + int __maybe_unused ret; > > > > > > vdev->pdev = to_pci_dev(core_vdev->dev); > > > vdev->irq_type = VFIO_PCI_NUM_IRQS; > > > @@ -2094,6 +2098,11 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev) > > > INIT_LIST_HEAD(&vdev->dummy_resources_list); > > > INIT_LIST_HEAD(&vdev->ioeventfds_list); > > > INIT_LIST_HEAD(&vdev->sriov_pfs_item); > > > +#ifdef CONFIG_VFIO_PCI_DMABUF > > > + ret = pcim_p2pdma_init(vdev->pdev); > > > + if (ret) > > > + return ret; > > > +#endif > > > init_rwsem(&vdev->memory_lock); > > > xa_init(&vdev->ctx); > > > > > > > What breaks if we don't test the return value and remove all the > > #ifdefs? The feature call should fail if we don't have a provider but > > that seems more robust than failing to register the device. Thanks, > > pcim_p2pdma_init() fails if memory allocation fails, which is worth to check. > Such failure will most likely cause to non-working vfio-pci module anyway, > as failure in pcim_p2pdma_init() will trigger OOM. It is better to fail early > and help for the system to recover from OOM, instead of delaying to the > next failure while trying to load vfio-pci. > > CONFIG_VFIO_PCI_DMABUF is mostly for next line "INIT_LIST_HEAD(&vdev->dmabufs);" > from the following patch. Because that pcim_p2pdma_init() and dmabufs list are > coupled, I put CONFIG_VFIO_PCI_DMABUF on both of them. Maybe it would remove my hang-up on the #ifdefs if we were to unconditionally include the header and move everything below that into a 'if (IS_ENABLED(CONFIG_VFIO_PCI_DMA)) {}' block. I think that would be statically evaluated by the compiler so we can still conditionalize the list_head in the vfio_pci_core_device struct via #ifdef, though I'm not super concerned about that since I'm expecting this will eventually be necessary for p2p DMA with IOMMUFD. That's also my basis for questioning why we think this needs a user visible kconfig option. I don't see a lot of value in enabling P2PDMA, DMABUF, and VFIO_PCI, but not VFIO_PCI_DMABUF. Thanks, Alex