Re: [PATCH RFC] bringup: Add soft iWARP bring-up option

[Jeff Layton <jlayton@kernel.org>]

On Wed, 2024-01-24 at 21:25 -0500, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> An iWARP device on the target hosts enables testing RDMA-based
> storage protocols such as SMB Direct, NVMeoF, or NFS/RDMA.
> 
> "rdma link" has to be done after every boot; it's not sticky, since
> there's no physical RDMA device involved. Thus a udev rule is
> inserted on each host to instantiate the siw device for the local
> eth0, whenever it is rebooted.
> 
> Note that the Linux kernel Kconfig option
> 
>   CONFIG_RDMA_SIW
> 
> must be set to M or Y in order for the siw.ko driver to be built for
> the target hosts.
> 
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> ---
>  Makefile                                   |    4 ++++
>  kconfigs/Kconfig.bringup.goals             |   11 +++++++++++
>  playbooks/roles/siw/tasks/main.yml         |   23 +++++++++++++++++++++++
>  playbooks/roles/siw/templates/99-siw.rules |    2 ++
>  playbooks/siw.yml                          |    4 ++++
>  scripts/bringup.Makefile                   |    4 ++++
>  scripts/siw.Makefile                       |    5 +++++
>  7 files changed, 53 insertions(+)
>  create mode 100644 playbooks/roles/siw/tasks/main.yml
>  create mode 100644 playbooks/roles/siw/templates/99-siw.rules
>  create mode 100644 playbooks/siw.yml
>  create mode 100644 scripts/siw.Makefile
> 

Looks good to me. I had started some work on a similar option, but yours
is definitely farther along. I gave this a spin and it brought up the
interfaces properly.

It would be nice to add a new client for the fstests and gitr workflows.
It should be pretty simple to add that on top of this, I think. Nice
work!

Tested-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Jeff Layton <jlayton@kernel.org>

> diff --git a/Makefile b/Makefile
> index c9ee45a7a639..ad34743e3e53 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -118,6 +118,10 @@ ifeq (y,$(CONFIG_WORKFLOWS))
>  include workflows/Makefile
>  endif # CONFIG_WORKFLOWS
>  
> +ifeq (y,$(CONFIG_KDEVOPS_SETUP_SIW))
> +include scripts/siw.Makefile
> +endif # CONFIG_KDEVOPS_SETUP_SIW
> +
>  ifeq (y,$(CONFIG_KDEVOPS_SETUP_KTLS))
>  include scripts/ktls.Makefile
>  endif # CONFIG_KDEVOPS_SETUP_KTLS
> diff --git a/kconfigs/Kconfig.bringup.goals b/kconfigs/Kconfig.bringup.goals
> index 8e4a8735f8e0..71948e9b65c6 100644
> --- a/kconfigs/Kconfig.bringup.goals
> +++ b/kconfigs/Kconfig.bringup.goals
> @@ -77,6 +77,17 @@ config KDEVOPS_TRY_INSTALL_KDEV_TOOLS
>  	  most kernel developers might prefer to have installed on target
>  	  systems.
>  
> +config KDEVOPS_SETUP_SIW
> +	bool "Configure soft iWARP on all test systems"
> +	default n
> +	help
> +	  Configure software-emulated RDMA on all target hosts via the
> +	  kernel siw.ko driver. siw can be used with any standard
> +	  Ethernet device, including virtio-net devices.
> +
> +	  Because siw is currently not enabled in most Linux distro
> +	  kernels, this option defaults to N.
> +
>  config KDEVOPS_SETUP_KTLS
>  	bool "Configure ktls on the hosts with self-signed CA"
>  	default n
> diff --git a/playbooks/roles/siw/tasks/main.yml b/playbooks/roles/siw/tasks/main.yml
> new file mode 100644
> index 000000000000..6b0ab7ac02c0
> --- /dev/null
> +++ b/playbooks/roles/siw/tasks/main.yml
> @@ -0,0 +1,23 @@
> +- name: Import optional extra_args file
> +  include_vars: "{{ item }}"
> +  ignore_errors: yes
> +  with_first_found:
> +    - files:
> +      - "../extra_vars.yml"
> +      - "../extra_vars.yaml"
> +      - "../extra_vars.json"
> +      skip: true
> +
> +- name: Insert udev rule to create siw device
> +  become: yes
> +  become_method: sudo
> +  template:
> +    src: "99-siw.rules"
> +    dest: "/usr/lib/udev/rules.d/"
> +    owner: "root"
> +    group: "root"
> +
> +- name: Reload udev rules
> +  become: yes
> +  become_method: sudo
> +  ansible.builtin.command: "udevadm control --reload"
> diff --git a/playbooks/roles/siw/templates/99-siw.rules b/playbooks/roles/siw/templates/99-siw.rules
> new file mode 100644
> index 000000000000..6c0cc7810e8e
> --- /dev/null
> +++ b/playbooks/roles/siw/templates/99-siw.rules
> @@ -0,0 +1,2 @@
> +ACTION=="add", SUBSYSTEM=="net", KERNEL=="eth0", \
> +   PROGRAM="/usr/sbin/rdma link add siw0 type siw netdev eth0"
> diff --git a/playbooks/siw.yml b/playbooks/siw.yml
> new file mode 100644
> index 000000000000..5258a5af9dca
> --- /dev/null
> +++ b/playbooks/siw.yml
> @@ -0,0 +1,4 @@
> +---
> +- hosts: all
> +  roles:
> +    - role: siw
> diff --git a/scripts/bringup.Makefile b/scripts/bringup.Makefile
> index dcab51556ddb..6e64b6c12172 100644
> --- a/scripts/bringup.Makefile
> +++ b/scripts/bringup.Makefile
> @@ -29,6 +29,10 @@ KDEVOPS_BRING_UP_DEPS += ktls
>  KDEVOPS_DESTROY_DEPS += ktls-destroy
>  endif # KDEVOPS_SETUP_KTLS
>  
> +ifeq (y,$(CONFIG_KDEVOPS_SETUP_SIW))
> +KDEVOPS_BRING_UP_DEPS += siw
> +endif # KDEVOPS_SETUP_SIW
> +
>  ifeq (y,$(CONFIG_DEVCONFIG_ENABLE_SYSTEMD_JOURNAL_REMOTE))
>  
>  JOURNAL_REMOTE:=$(subst ",,$(CONFIG_DEVCONFIG_SYSTEMD_JOURNAL_REMOTE_URL))
> diff --git a/scripts/siw.Makefile b/scripts/siw.Makefile
> new file mode 100644
> index 000000000000..80f963092b11
> --- /dev/null
> +++ b/scripts/siw.Makefile
> @@ -0,0 +1,5 @@
> +siw:
> +	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --extra-vars=@./extra_vars.yaml \
> +		-f 30 -i hosts playbooks/siw.yml
> +
> +PHONY += siw
> 
> 
>