Re: [PATCH] nvmet: make nvmet_wq visible in sysfs

[Chaitanya Kulkarni <chaitanyak@nvidia.com>]

On 10/30/24 19:45, Chaitanya Kulkarni wrote:
>> I will send the v2 with our usecase to expain why we should restrict the cpumask,
>>
>> I'm concerned whether blktests can handle such complex tests, as it relies on deploying
>>
>> many Docker containers and services. Should it only test the case of setting the cpumask
>>
>> with fio?
>>
>> Best Regards,
>>
>> Guixin Liu
>>
> For now just cpumask and fio is sufficient so that when we upstream this patch we have some sort of testing done via sysfs.
>
>
> -ck

based on my very limited understanding I've written a rough blktest
for your patch see if this helps, it's bit rough and totally
untested  :-

#!/bin/bash
# SPDX-License-Identifier: GPL-3.0+
# Description: Test `nvmet_wq` cpumask sysfs attribute with NVMe-oF and 
fio workload

. tests/nvme/rc

DESCRIPTION="Test nvmet_wq cpumask sysfs attribute and verify with fio 
on NVMe-oF device"

requires() {
     _nvme_requires
     _have_fio
     _require_nvme_trtype_is_fabrics
}

test() {
     local cpumask_path="/sys/devices/virtual/workqueue/nvmet_wq/cpumask"

     # Check if cpumask attribute exists
     if [[ ! -f "$cpumask_path" ]]; then
         SKIP_REASONS+=("nvmet_wq cpumask sysfs attribute not found.")
         return 1
     fi

     # Save original cpumask value
     local original_cpumask
     original_cpumask=$(cat "$cpumask_path")
     echo "Original cpumask: $original_cpumask"

     # Set a new cpumask (e.g., CPU 0)
     echo 1 | tee "$cpumask_path" > /dev/null
     local new_cpumask
     new_cpumask=$(cat "$cpumask_path")

     if [[ "$new_cpumask" != "1" ]]; then
         echo "Test Failed: cpumask was not set correctly"
         return 1
     else
         echo "Test Passed: cpumask set to $new_cpumask"
     fi

     # Set up NVMe-over-Fabrics target
     echo "Setting up NVMe-oF target"
     _setup_nvmet
     _nvmet_target_setup
     _nvme_connect_subsys

     # Locate the NVMe-oF namespace
     local ns
     ns=$(_find_nvme_ns "${def_subsys_uuid}")

     # Run fio with data verification on NVMe-oF device
     echo "Starting fio workload with verification on NVMe-oF device"
     fio --name=nvmet_wq_test --filename="/dev/$ns" --direct=1 
--rw=randwrite \
         --bs=4k --size=100M --numjobs=1 --verify=crc32c --verify_fatal=1 \
         --time_based --runtime=30s --iodepth=16 --ioengine=libaio 
--group_reporting

     # Disconnect and clean up NVMe-oF target
     echo "Cleaning up NVMe-oF setup"
     _nvme_disconnect_subsys
     _nvmet_target_cleanup

     # Restore original cpumask
     echo "$original_cpumask" | tee "$cpumask_path" > /dev/null
     restored_cpumask=$(cat "$cpumask_path")

     if [[ "$restored_cpumask" != "$original_cpumask" ]]; then
         echo "Failed to restore original cpumask."
         return 1
     else
         echo "Original cpumask restored successfully."
     fi
}