From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 35F53221F3C for ; Fri, 23 May 2025 23:30:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748043060; cv=none; b=g/waNsdtjwpGr0Z7Igfdnq4YdvZD6fPzc+UZ1iqY2QaFtoeDpSUnFYyXSp+30f9FCRVu5CQ6hfqRjP7eacLLfBuV9YPbi+z/Qanz0hfvVgVVPXPD0BA40rdPtC4FZOO5dTess9V2GhryB+OLHwkqYRKjZJiifx2qqww8LMAXvcs= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748043060; c=relaxed/simple; bh=g+E8JXWKncGSLKydmpWC/uNckMj76ox67ykM4jULD04=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WVJZo26as3DRQHsbDNy8/M0tBZo7PgwIlRwz8RpMEaTYBr9OeciM3dFdcEeXYQsUiW/l8HWE4nTAbAVaAH7eQm16Uy23gXbVLjTo0gzgD1o3WhSoMJUlcrP82KkGukKKnI0sO0V3//XVzGYHzNIIXZZV+1+Mq8C5VGhwvCW8Csc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=4DimUTrk; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dmatlack.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="4DimUTrk" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-30e896e116fso392667a91.2 for ; Fri, 23 May 2025 16:30:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1748043058; x=1748647858; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=M56wBrhTere3v95HCeXBYP3UY5DColz6PlhgTZ75vTg=; b=4DimUTrkhJmlb3eb4H3DiBwrMTmQnZ88F1Q/eqL+TDdabvh7hkAI199JX8HpSswgUL jrHVjvXHYhusjrOXb4r7Bk1B8bWVBrPs9tYp8VkcG1hMET4yLzvzJxa4bmpcsGqJm8RL SNgryQULra6BWXnsDYnWfTtp/PsvcJsD0zKXMmafh6K/mT8o2moTvYofGBbfr7AYukcq qVQBdIRxKV4NY74RSWE8BU8ZZ2NbaOxoDsNy3XULXPLD32LALftcbMkLaRYkVu1J5WdS vIsJaAFHC3gfuMdFKFYetilP+5vJqf/92RA76Mqg6ru6CB9iHeROvZPt5cfeiikvt3o5 soUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748043058; x=1748647858; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=M56wBrhTere3v95HCeXBYP3UY5DColz6PlhgTZ75vTg=; b=aklPwc8rrbZ0xQQlsEH75WwB39x2w/W3adzn31cq4HCZnJTTtHvAQesRYOvM8YJto0 C+VG3yDdbMaWssHtF0cqyj6zTHH9RdLl7CKVTTQck0epBYU3QYjscx5D7pmaEKIjM0nU g2BT0fYD9h4yRLM5JV6kMY5YnHpyqlFRBq8h5MVGAzmsEjKASveokZqM+KABg/m+Te4I 3V1wIaPTNqa6C5z1LVh7tUmmm1TABtBeDnc7vPRbZDoVaoFtV1Z7U+O1MULd1YAOINV6 p8zA8CSoJIlpFiZ5UwW/avFMdViSqYCkv8xOAuwnckxR5EXY6bS/z/TFlQ+nYHZkp3zZ PURg== X-Forwarded-Encrypted: i=1; AJvYcCWNoU8r7ydP8RcN66KQTZdrG32Vol0btbK0cuwafm+4wrnoRv7vhIQr7/HqldVtZlH4vFY=@vger.kernel.org X-Gm-Message-State: AOJu0YyZsbqL4VyGeqW0OrAEzeWduykbZI0C6HOy2V8FCmE/M3PVjhQe 83uk9yVHx3HlwD9rr8igRb/7kouSQzGypS+jS0FXcsAX2hsR8Hc5Mzr84Y7iS1ZptPW+mfu2z2e LXV20Qu6c1+8hnw== X-Google-Smtp-Source: AGHT+IEv8h8XeAJL1DmmIRW0ElKkUDYQycR8zF4ioQ5lpDDrsb/RHwypOkptwRczYrb10lureb4Y0tVgvEmdqA== X-Received: from pji12.prod.google.com ([2002:a17:90b:3fcc:b0:2f5:63a:4513]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3bd0:b0:30e:823f:ef21 with SMTP id 98e67ed59e1d1-31110d933e0mr1170568a91.32.1748043058586; Fri, 23 May 2025 16:30:58 -0700 (PDT) Date: Fri, 23 May 2025 23:30:05 +0000 In-Reply-To: <20250523233018.1702151-1-dmatlack@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250523233018.1702151-1-dmatlack@google.com> X-Mailer: git-send-email 2.49.0.1151.ga128411c76-goog Message-ID: <20250523233018.1702151-21-dmatlack@google.com> Subject: [RFC PATCH 20/33] vfio: sefltests: Add vfio_pci_driver_test From: David Matlack To: Alex Williamson Cc: Shuah Khan , Paolo Bonzini , David Matlack , Vinod Koul , Fenghua Yu , "Masami Hiramatsu (Google)" , Adhemerval Zanella , Jiri Olsa , Andrii Nakryiko , Wei Yang , Bjorn Helgaas , Takashi Iwai , Greg Kroah-Hartman , Pierre-Louis Bossart , Andy Shevchenko , FUJITA Tomonori , WangYuli , Sean Christopherson , Andrew Jones , Claudio Imbrenda , Eric Auger , Josh Hilke , linux-kselftest@vger.kernel.org, kvm@vger.kernel.org, Jason Gunthorpe , Kevin Tian , Vipin Sharma , Pasha Tatashin , Saeed Mahameed , Adithya Jayachandran , Parav Pandit , Leon Romanovsky , Vinicius Costa Gomes , Dave Jiang , Dan Williams Content-Type: text/plain; charset="UTF-8" Add a new selftest that tests all driver operations. This test serves both as a demonstration of the driver framework, and also as a correctness test for future drivers. Signed-off-by: David Matlack --- tools/testing/selftests/vfio/Makefile | 1 + .../selftests/vfio/vfio_pci_driver_test.c | 239 ++++++++++++++++++ 2 files changed, 240 insertions(+) create mode 100644 tools/testing/selftests/vfio/vfio_pci_driver_test.c diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftests/vfio/Makefile index 2cc97176c6be..21fb1809035e 100644 --- a/tools/testing/selftests/vfio/Makefile +++ b/tools/testing/selftests/vfio/Makefile @@ -2,6 +2,7 @@ CFLAGS = $(KHDR_INCLUDES) TEST_GEN_PROGS_EXTENDED += vfio_dma_mapping_test TEST_GEN_PROGS_EXTENDED += vfio_iommufd_setup_test TEST_GEN_PROGS_EXTENDED += vfio_pci_device_test +TEST_GEN_PROGS_EXTENDED += vfio_pci_driver_test include ../lib.mk include lib/libvfio.mk diff --git a/tools/testing/selftests/vfio/vfio_pci_driver_test.c b/tools/testing/selftests/vfio/vfio_pci_driver_test.c new file mode 100644 index 000000000000..5d14d892b796 --- /dev/null +++ b/tools/testing/selftests/vfio/vfio_pci_driver_test.c @@ -0,0 +1,239 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include + +#include +#include +#include +#include + +#include + +#include "../kselftest_harness.h" + +const char *device_bdf; + +#define ASSERT_NO_MSI(_eventfd) do { \ + u64 __value; \ + \ + ASSERT_EQ(-1, read(_eventfd, &__value, 8)); \ + ASSERT_EQ(EAGAIN, errno); \ +} while (0) + +static void region_setup(struct vfio_pci_device *device, + struct vfio_dma_region *region, + iova_t iova, u64 size) +{ + const int flags = MAP_SHARED | MAP_ANONYMOUS; + const int prot = PROT_READ | PROT_WRITE; + void *vaddr; + + vaddr = mmap(NULL, size, prot, flags, -1, 0); + VFIO_ASSERT_NE(vaddr, MAP_FAILED); + + region->vaddr = vaddr; + region->iova = iova; + region->size = size; + + vfio_pci_dma_map(device, region); +} + +static void region_teardown(struct vfio_pci_device *device, + struct vfio_dma_region *region) +{ + vfio_pci_dma_unmap(device, region); + VFIO_ASSERT_EQ(munmap(region->vaddr, region->size), 0); +} + +FIXTURE(vfio_pci_driver_test) { + struct vfio_pci_device *device; + struct vfio_dma_region memcpy_region; + void *vaddr; + int msi_fd; + + u64 size; + void *src; + void *dst; + iova_t src_iova; + iova_t dst_iova; + iova_t unmapped_iova; +}; + +FIXTURE_SETUP(vfio_pci_driver_test) { + struct vfio_pci_driver *driver; + + self->device = vfio_pci_device_init(device_bdf, VFIO_TYPE1_IOMMU); + + driver = &self->device->driver; + + region_setup(self->device, &self->memcpy_region, SZ_1G, SZ_1G); + region_setup(self->device, &driver->region, SZ_4G, SZ_2M); + + /* Any IOVA that doesn't overlap memcpy_region and driver->region. */ + self->unmapped_iova = 8UL * SZ_1G; + + vfio_pci_driver_init(self->device); + self->msi_fd = self->device->msi_eventfds[driver->msi]; + + /* + * Use the maximum size supported by the device for memcpy operations, + * slimmed down to fit into the memcpy region (divided by 2 so src and + * dst regions do not overlap). + */ + self->size = self->device->driver.max_memcpy_size; + self->size = min(self->size, self->memcpy_region.size / 2); + + self->src = self->memcpy_region.vaddr; + self->dst = self->src + self->size; + + self->src_iova = to_iova(self->device, self->src); + self->dst_iova = to_iova(self->device, self->dst); +} + +FIXTURE_TEARDOWN(vfio_pci_driver_test) { + struct vfio_pci_driver *driver = &self->device->driver; + + vfio_pci_driver_remove(self->device); + + region_teardown(self->device, &self->memcpy_region); + region_teardown(self->device, &driver->region); + + vfio_pci_device_cleanup(self->device); +} + +TEST_F(vfio_pci_driver_test, init_remove) +{ + int i; + + for (i = 0; i < 10; i++) { + vfio_pci_driver_remove(self->device); + vfio_pci_driver_init(self->device); + } +} + +TEST_F(vfio_pci_driver_test, memcpy_success) +{ + fcntl_set_nonblock(self->msi_fd); + + memset(self->src, 'x', self->size); + memset(self->dst, 'y', self->size); + + ASSERT_EQ(0, vfio_pci_driver_memcpy(self->device, + self->src_iova, + self->dst_iova, + self->size)); + + ASSERT_EQ(0, memcmp(self->src, self->dst, self->size)); + ASSERT_NO_MSI(self->msi_fd); +} + +TEST_F(vfio_pci_driver_test, memcpy_from_unmapped_iova) +{ + fcntl_set_nonblock(self->msi_fd); + + /* + * Ignore the return value since not all devices will detect and report + * accesses to unmapped IOVAs as errors. + */ + vfio_pci_driver_memcpy(self->device, self->unmapped_iova, + self->dst_iova, self->size); + + ASSERT_NO_MSI(self->msi_fd); +} + +TEST_F(vfio_pci_driver_test, memcpy_to_unmapped_iova) +{ + fcntl_set_nonblock(self->msi_fd); + + /* + * Ignore the return value since not all devices will detect and report + * accesses to unmapped IOVAs as errors. + */ + vfio_pci_driver_memcpy(self->device, self->src_iova, + self->unmapped_iova, self->size); + + ASSERT_NO_MSI(self->msi_fd); +} + +TEST_F(vfio_pci_driver_test, send_msi) +{ + u64 value; + + vfio_pci_driver_send_msi(self->device); + ASSERT_EQ(8, read(self->msi_fd, &value, 8)); + ASSERT_EQ(1, value); +} + +TEST_F(vfio_pci_driver_test, mix_and_match) +{ + u64 value; + int i; + + for (i = 0; i < 10; i++) { + memset(self->src, 'x', self->size); + memset(self->dst, 'y', self->size); + + ASSERT_EQ(0, vfio_pci_driver_memcpy(self->device, + self->src_iova, + self->dst_iova, + self->size)); + + ASSERT_EQ(0, memcmp(self->src, self->dst, self->size)); + + vfio_pci_driver_memcpy(self->device, + self->unmapped_iova, + self->dst_iova, + self->size); + + vfio_pci_driver_send_msi(self->device); + ASSERT_EQ(8, read(self->msi_fd, &value, 8)); + ASSERT_EQ(1, value); + } +} + +TEST_F_TIMEOUT(vfio_pci_driver_test, memcpy_storm, 60) +{ + struct vfio_pci_driver *driver = &self->device->driver; + u64 total_size; + u64 count; + + fcntl_set_nonblock(self->msi_fd); + + /* + * Perform up to 250GiB worth of DMA reads and writes across several + * memcpy operations. Some devices can support even more but the test + * will take too long. + */ + total_size = 250UL * SZ_1G; + count = min(total_size / self->size, driver->max_memcpy_count); + + printf("Kicking off %lu memcpys of size 0x%lx\n", count, self->size); + vfio_pci_driver_memcpy_start(self->device, + self->src_iova, + self->dst_iova, + self->size, count); + + ASSERT_EQ(0, vfio_pci_driver_memcpy_wait(self->device)); + ASSERT_NO_MSI(self->msi_fd); +} + +int main(int argc, char *argv[]) +{ + struct vfio_pci_device *device; + + if (argc != 2) { + fprintf(stderr, "usage: %s segment:bus:device.function\n", argv[0]); + return KSFT_FAIL; + } + + device_bdf = argv[1]; + + device = vfio_pci_device_init(device_bdf, VFIO_TYPE1_IOMMU); + if (!device->driver.ops) { + fprintf(stderr, "No driver found for device %s\n", device_bdf); + return KSFT_SKIP; + } + vfio_pci_device_cleanup(device); + + return test_harness_run(1, argv); +} -- 2.49.0.1151.ga128411c76-goog