[PATCH v11 0/3] Switchtec Switch DMA Engine Driver

Logan Gunthorpe
2025-12-15T18:40:49Z

This is v11 of the Switchtec Switch DMA Engine Driver. We sent v10 in the
previous cycle with almost no feedback.

To reiterate the response to the feedback we keep getting: virt-dma in
not appropriate for this driver because the hardware has a submission
queue itself. There's no sense adding a software queue when entries can
just be placed on the hardware queue. This is the same as similar dma
engine drivers which came before it: ioat and plx.

The patchset has been rebased on v6.19-rc1.

Hoping to get this reviewed and in this cycle.

Thanks,

Logan

--

v11 changes:
- Rebased onto v6.19-rc1

v10 changes:
- Rebased onto v6.18-rc1
- Some systems were seeing the warning message: "WARN: Device release
is not defined so it is not safe to unbind this driver while in use"
Adjusted the copy_align parameter to align to 8 bytes to fix this
issue.

v9 changes:
- Rebase onto v6.17-rc2
- Fix multiple RCU derference warnings
- Fix the spinlocks being used before they were initialized causing
an INFO message when built with lockdep enabled. (noticed by Maciej
Grochowski)
- Add additional entries to the PCI table which use an EFAR vendor ID.
- Remove some enums that were not used by the code
- Update copyright date to 2025 (as requested by Vinod)
- Drop the default values enum and just set the defaults
directly. (to address some confusing code noticed by Vinod)
- Use GENMASK and FIELD_GET/PREP macros instead of direct bit
operations (suggested by Vinod)
- Use lower case hex numbers (requested by Vinod)
- Don't mangle the return value of readl_poll_timeout_atomic()
and return it's error directly (per Vinod)
- Split the patch into a couple patches (per Vinod)
- Fix some missing new line characters on numerous printks
- Rename switchtec_dma_process_desc() function to
switchtec_dma_cleanup_completed() for clarity.
- Open code a few short, unnecessary, helper functions.
- Drop the tasklet used for the chan_status interrupt. It is only
really used when the device is paused so performance is not
critical and thus it can be a threaded interrupt instead of
a tasklet.
- Fixed an uninitialized symbol ('i') error (caught by Dan Carpenter)

v8 changes:
- Rebase onto kernel 6.8
- Add Gen5 device IDs

v7 changes:
- Remove implementation of device_prep_dma_imm_data

v6 changes:
- Fix './scripts/checkpatch.pl --strict' warnings
- Use readl_poll_timeout_atomic for status checking with timeout
- Wrap enable_channel/disable_channel over channel_op
- Use flag GFP_NOWAIT for mem allocation in switchtec_dma_alloc_desc
- Use proper comment for macro SWITCHTEC_DMA_DEVICE

v5 changes:
- Remove unnecessary structure modifier '__packed'
- Remove the use of union of identical data types in a structure
- Remove unnecessary call sites of synchronize_irq
- Remove unnecessary rcu lock for pdev during device initialization
- Use pci_request_irq/pci_free_irq to replace request_irq/free_irq
- Add mailing list info in file MAINTAINERS
- Miscellaneous cleanups

v4 changes:
- Sort driver entry in drivers/dma/Kconfig and drivers/dma/Makefile
alphabetically
- Fix miscellaneous style issues
- Correct year in copyright
- Add function and call sites to flush PCIe MMIO Write
- Add a helper to wait for status register update
- Move synchronize_irq out of RCU critical section
- Remove unnecessary endianness conversion for register access
- Remove some unused code
- Use pci_enable_device/pci_request_mem_regions instead of
pcim_enable_device/pcim_iomap_regions to make the resource lifetime
management more understandable
- Use offset macros instead of memory mapped structures when accessing
some registers
- Remove the attempt to set DMA mask with smaller number as it would
never succeed if the first attempt with bigger number fails
- Use PCI_VENDOR_ID_MICROSEMI in include/linux/pci_ids.h as device ID

v3 changes:
- Remove some unnecessary memory/variable zeroing

v2 changes:
- Move put_device(dma_dev->dev) before kfree(swdma_dev) as dma_dev is
part of swdma_dev.
- Convert dev_ print calls to pci_ print calls to make the use of
print functions consistent within switchtec_dma_create().
- Remove some dev_ print calls, which use device pointer as handles,
to ensure there's no reference issue when the device is unbound.
- Remove unused .driver_data from pci_device_id structure.

v1:
The following patch implements a DMAEngine driver to use the DMA
controller in Switchtec PSX/PFX switchtes. The DMA controller
appears as a PCI function on the switch upstream port. The DMA
function can include one or more DMA channels.
Switchtec Switch DMA Engine Driver

This is v9 of the Switchtec Switch DMA Engine Driver. It's been over a
year since the last posting, appologies for that. Kelvin has asked
me to take over getting the work upstream.

I reviewed the feedback and addressed most of it in this posting. I've
also caught additional issues during my testing.

The remaining disagreement is on how the completions are cleaned up. I
maintain[1] that it's best to clean them up in the status() call back
in addition to the interrupt as this allows for transfers that don't
use interrupts. Vinod has argued that this is wrong and the other
drivers that do this should be changed as well (IOAT and PLX). In this
poisting, I have left it the original way with cleanups in the status()
call back.

This patch set is based on v6.17-rc2.

Thanks,

Logan

[1] https://lore.kernel.org/r/e759d483-e303-421a-b674-72fd9121750d@deltatee.com

--

v9 changes:
- Rebase onto v6.17-rc2
- Fix multiple RCU derference warnings
- Fix the spinlocks being used before they were initialized causing
an INFO message when built with lockdep enabled. (noticed by Maciej
Grochowski)
- Add additional entries to the PCI table which use an EFAR vendor ID.
- Remove some enums that were not used by the code
- Update copyright date to 2025 (as requested by Vinod)
- Drop the default values enum and just set the defaults
directly. (to address some confusing code noticed by Vinod)
- Use GENMASK and FIELD_GET/PREP macros instead of direct bit
operations (suggested by Vinod)
- Use lower case hex numbers (requested by Vinod)
- Don't mangle the return value of readl_poll_timeout_atomic()
and return it's error directly (per Vinod)
- Split the patch into a couple patches (per Vinod)
- Fix some missing new line characters on numerous printks
- Rename switchtec_dma_process_desc() function to
switchtec_dma_cleanup_completed() for clarity.
- Open code a few short, unnecessary, helper functions.
- Drop the tasklet used for the chan_status interrupt. It is only
really used when the device is paused so performance is not
critical and thus it can be a threaded interrupt instead of
a tasklet.
- Fixed an uninitialized symbol ('i') error (caught by Dan Carpenter)

v8 changes:
- Rebase onto kernel 6.8
- Add Gen5 device IDs

v7 changes:
- Remove implementation of device_prep_dma_imm_data

v6 changes:
- Fix './scripts/checkpatch.pl --strict' warnings
- Use readl_poll_timeout_atomic for status checking with timeout
- Wrap enable_channel/disable_channel over channel_op
- Use flag GFP_NOWAIT for mem allocation in switchtec_dma_alloc_desc
- Use proper comment for macro SWITCHTEC_DMA_DEVICE

v5 changes:
- Remove unnecessary structure modifier '__packed'
- Remove the use of union of identical data types in a structure
- Remove unnecessary call sites of synchronize_irq
- Remove unnecessary rcu lock for pdev during device initialization
- Use pci_request_irq/pci_free_irq to replace request_irq/free_irq
- Add mailing list info in file MAINTAINERS
- Miscellaneous cleanups

v4 changes:
- Sort driver entry in drivers/dma/Kconfig and drivers/dma/Makefile
alphabetically
- Fix miscellaneous style issues
- Correct year in copyright
- Add function and call sites to flush PCIe MMIO Write
- Add a helper to wait for status register update
- Move synchronize_irq out of RCU critical section
- Remove unnecessary endianness conversion for register access
- Remove some unused code
- Use pci_enable_device/pci_request_mem_regions instead of
pcim_enable_device/pcim_iomap_regions to make the resource lifetime
management more understandable
- Use offset macros instead of memory mapped structures when accessing
some registers
- Remove the attempt to set DMA mask with smaller number as it would
never succeed if the first attempt with bigger number fails
- Use PCI_VENDOR_ID_MICROSEMI in include/linux/pci_ids.h as device ID

v3 changes:
- Remove some unnecessary memory/variable zeroing

v2 changes:
- Move put_device(dma_dev->dev) before kfree(swdma_dev) as dma_dev is
part of swdma_dev.
- Convert dev_ print calls to pci_ print calls to make the use of
print functions consistent within switchtec_dma_create().
- Remove some dev_ print calls, which use device pointer as handles,
to ensure there's no reference issue when the device is unbound.
- Remove unused .driver_data from pci_device_id structure.

v1:
The following patch implements a DMAEngine driver to use the DMA
controller in Switchtec PSX/PFX switchtes. The DMA controller
appears as a PCI function on the switch upstream port. The DMA
function can include one or more DMA channels.

Kelvin Cao (3):
dmaengine: switchtec-dma: Introduce Switchtec DMA engine skeleton
dmaengine: switchtec-dma: Implement hardware initialization and
cleanup
dmaengine: switchtec-dma: Implement descriptor submission

MAINTAINERS | 7 +
drivers/dma/Kconfig | 9 +
drivers/dma/Makefile | 1 +
drivers/dma/switchtec_dma.c | 1456 +++++++++++++++++++++++++++++++++++
4 files changed, 1473 insertions(+)
create mode 100644 drivers/dma/switchtec_dma.c

base-commit: 8f0b4cce4481fb22653697cced8d0d04027cb1e8
--
2.47.3