[PATCH 0/4] crypto: qat - enable configuration for 4xxx

[PATCH 0/4] crypto: qat - enable configuration for 4xxx

[Giovanni Cabiddu]

qat_4xxx devices can be configured to allow either crypto or compression
operations. By default, devices are configured statically according
to following rule:
  - odd numbered devices assigned to compression services
  - even numbered devices assigned to crypto services

This set exposes two attributes in sysfs that allow to report and change
the state and the configuration of a QAT 4xxx device.
The first, /sys/bus/pci/devices/<BDF>/qat/state, allows to bring a
device down in order to change the configuration, and bring it up again.
The second, /sys/bus/pci/devices/<BDF>/qat/cfg_services, allows to
inspect the current configuration of a device (i.e. crypto or
compression) and change it.

Giovanni Cabiddu (4):
  crypto: qat - expose device state through sysfs for 4xxx
  crypto: qat - change behaviour of adf_cfg_add_key_value_param()
  crypto: qat - relocate and rename adf_sriov_prepare_restart()
  crypto: qat - expose device config through sysfs for 4xxx

 Documentation/ABI/testing/sysfs-driver-qat    |  58 ++++++
 .../crypto/qat/qat_4xxx/adf_4xxx_hw_data.c    |   1 +
 .../crypto/qat/qat_4xxx/adf_4xxx_hw_data.h    |   1 +
 drivers/crypto/qat/qat_4xxx/adf_drv.c         |   6 +-
 drivers/crypto/qat/qat_common/Makefile        |   1 +
 .../crypto/qat/qat_common/adf_accel_devices.h |   1 +
 drivers/crypto/qat/qat_common/adf_cfg.c       |  41 +++-
 .../crypto/qat/qat_common/adf_common_drv.h    |   3 +
 drivers/crypto/qat/qat_common/adf_init.c      |  26 +++
 drivers/crypto/qat/qat_common/adf_sriov.c     |  28 +--
 drivers/crypto/qat/qat_common/adf_sysfs.c     | 191 ++++++++++++++++++
 11 files changed, 328 insertions(+), 29 deletions(-)
 create mode 100644 Documentation/ABI/testing/sysfs-driver-qat
 create mode 100644 drivers/crypto/qat/qat_common/adf_sysfs.c

-- 
2.36.1

[PATCH 1/4] crypto: qat - expose device state through sysfs for 4xxx

[Giovanni Cabiddu]

Expose the device state through an attribute in sysfs and allow to
change it. This is to stop and shutdown a QAT device in order to change
its configuration.

The state attribute has been added to a newly created `qat` attribute
group which will contain all _QAT specific_ attributes.

The logic that implements the sysfs entries is part of a new file,
adf_sysfs.c. This exposes an entry point to allow the driver to create
attributes.

The function that creates the sysfs attributes is called from the probe
function of the driver and not in the state machine init function to
allow the change of states even if the device is the down state.

In order to restore the device configuration between a transition from
down to up, the function that configures the devices has been abstracted
into the HW data structure.

The `state` attribute is only exposed for qat_4xxx devices.

Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Co-developed-by: Tomasz Kowallik <tomaszx.kowalik@intel.com>
Signed-off-by: Tomasz Kowallik <tomaszx.kowalik@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Reviewed-by: Fiona Trahe <fiona.trahe@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
---
 Documentation/ABI/testing/sysfs-driver-qat    |  21 ++++
 .../crypto/qat/qat_4xxx/adf_4xxx_hw_data.c    |   1 +
 .../crypto/qat/qat_4xxx/adf_4xxx_hw_data.h    |   1 +
 drivers/crypto/qat/qat_4xxx/adf_drv.c         |   6 +-
 drivers/crypto/qat/qat_common/Makefile        |   1 +
 .../crypto/qat/qat_common/adf_accel_devices.h |   1 +
 .../crypto/qat/qat_common/adf_common_drv.h    |   2 +
 drivers/crypto/qat/qat_common/adf_sysfs.c     | 119 ++++++++++++++++++
 8 files changed, 151 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/ABI/testing/sysfs-driver-qat
 create mode 100644 drivers/crypto/qat/qat_common/adf_sysfs.c

diff --git a/Documentation/ABI/testing/sysfs-driver-qat b/Documentation/ABI/testing/sysfs-driver-qat
new file mode 100644
index 000000000000..0915253efaa3
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-driver-qat
@@ -0,0 +1,21 @@
+What:		/sys/bus/pci/devices/<BDF>/qat/state
+Date:		June 2022
+KernelVersion:	5.19
+Contact:	qat-linux@intel.com
+Description:	Reports the current state of the QAT device and allows to
+		change it.
+
+		This attribute is RW.
+
+		Returned values:
+			up: the device is up and running
+			down: the device is down
+
+		Allowed values:
+			up: initialize and start the device
+			down: stop the device and bring it down
+
+		It is possible to transition the device from up to down only
+		if the device is up and vice versa.
+
+		This attribute is only available for qat_4xxx devices.
diff --git a/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c b/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c
index fb5970a68484..fda5f699ff57 100644
--- a/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c
+++ b/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c
@@ -357,6 +357,7 @@ void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data)
 	hw_data->ring_pair_reset = adf_gen4_ring_pair_reset;
 	hw_data->enable_pm = adf_gen4_enable_pm;
 	hw_data->handle_pm_interrupt = adf_gen4_handle_pm_interrupt;
+	hw_data->dev_config = adf_crypto_dev_config;
 
 	adf_gen4_init_hw_csr_ops(&hw_data->csr_ops);
 	adf_gen4_init_pf_pfvf_ops(&hw_data->pfvf_ops);
diff --git a/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.h b/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.h
index 1034752845ca..9d49248931f6 100644
--- a/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.h
+++ b/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.h
@@ -70,5 +70,6 @@ enum icp_qat_4xxx_slice_mask {
 
 void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data);
 void adf_clean_hw_data_4xxx(struct adf_hw_device_data *hw_data);
+int adf_crypto_dev_config(struct adf_accel_dev *accel_dev);
 
 #endif
diff --git a/drivers/crypto/qat/qat_4xxx/adf_drv.c b/drivers/crypto/qat/qat_4xxx/adf_drv.c
index 181fa1c8b3c7..2f212561acc4 100644
--- a/drivers/crypto/qat/qat_4xxx/adf_drv.c
+++ b/drivers/crypto/qat/qat_4xxx/adf_drv.c
@@ -53,7 +53,7 @@ static int adf_cfg_dev_init(struct adf_accel_dev *accel_dev)
 	return 0;
 }
 
-static int adf_crypto_dev_config(struct adf_accel_dev *accel_dev)
+int adf_crypto_dev_config(struct adf_accel_dev *accel_dev)
 {
 	char key[ADF_CFG_MAX_KEY_LEN_IN_BYTES];
 	int banks = GET_MAX_BANKS(accel_dev);
@@ -289,6 +289,10 @@ static int adf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 		goto out_err_disable_aer;
 	}
 
+	ret = adf_sysfs_init(accel_dev);
+	if (ret)
+		goto out_err_disable_aer;
+
 	ret = adf_crypto_dev_config(accel_dev);
 	if (ret)
 		goto out_err_disable_aer;
diff --git a/drivers/crypto/qat/qat_common/Makefile b/drivers/crypto/qat/qat_common/Makefile
index 04f058acc4d3..80919cfcc29d 100644
--- a/drivers/crypto/qat/qat_common/Makefile
+++ b/drivers/crypto/qat/qat_common/Makefile
@@ -10,6 +10,7 @@ intel_qat-objs := adf_cfg.o \
 	adf_transport.o \
 	adf_admin.o \
 	adf_hw_arbiter.o \
+	adf_sysfs.o \
 	adf_gen2_hw_data.o \
 	adf_gen4_hw_data.o \
 	adf_gen4_pm.o \
diff --git a/drivers/crypto/qat/qat_common/adf_accel_devices.h b/drivers/crypto/qat/qat_common/adf_accel_devices.h
index ede6458c9dbf..0a55a4f34dcf 100644
--- a/drivers/crypto/qat/qat_common/adf_accel_devices.h
+++ b/drivers/crypto/qat/qat_common/adf_accel_devices.h
@@ -199,6 +199,7 @@ struct adf_hw_device_data {
 	char *(*uof_get_name)(struct adf_accel_dev *accel_dev, u32 obj_num);
 	u32 (*uof_get_num_objs)(void);
 	u32 (*uof_get_ae_mask)(struct adf_accel_dev *accel_dev, u32 obj_num);
+	int (*dev_config)(struct adf_accel_dev *accel_dev);
 	struct adf_pfvf_ops pfvf_ops;
 	struct adf_hw_csr_ops csr_ops;
 	const char *fw_name;
diff --git a/drivers/crypto/qat/qat_common/adf_common_drv.h b/drivers/crypto/qat/qat_common/adf_common_drv.h
index 0464fa257929..0f3031f9055d 100644
--- a/drivers/crypto/qat/qat_common/adf_common_drv.h
+++ b/drivers/crypto/qat/qat_common/adf_common_drv.h
@@ -132,6 +132,8 @@ void adf_vf_isr_resource_free(struct adf_accel_dev *accel_dev);
 
 int adf_pfvf_comms_disabled(struct adf_accel_dev *accel_dev);
 
+int adf_sysfs_init(struct adf_accel_dev *accel_dev);
+
 int qat_hal_init(struct adf_accel_dev *accel_dev);
 void qat_hal_deinit(struct icp_qat_fw_loader_handle *handle);
 int qat_hal_start(struct icp_qat_fw_loader_handle *handle);
diff --git a/drivers/crypto/qat/qat_common/adf_sysfs.c b/drivers/crypto/qat/qat_common/adf_sysfs.c
new file mode 100644
index 000000000000..8f47a5694dd7
--- /dev/null
+++ b/drivers/crypto/qat/qat_common/adf_sysfs.c
@@ -0,0 +1,119 @@
+// SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0-only)
+/* Copyright(c) 2022 Intel Corporation */
+#include <linux/device.h>
+#include <linux/errno.h>
+#include <linux/pci.h>
+#include "adf_accel_devices.h"
+#include "adf_cfg.h"
+#include "adf_common_drv.h"
+
+static const char * const state_operations[] = {
+	[DEV_DOWN] = "down",
+	[DEV_UP] = "up",
+};
+
+static ssize_t state_show(struct device *dev, struct device_attribute *attr,
+			  char *buf)
+{
+	struct adf_accel_dev *accel_dev;
+	char *state;
+
+	accel_dev = adf_devmgr_pci_to_accel_dev(to_pci_dev(dev));
+	if (!accel_dev)
+		return -EINVAL;
+
+	state = adf_dev_started(accel_dev) ? "up" : "down";
+	return sysfs_emit(buf, "%s\n", state);
+}
+
+static ssize_t state_store(struct device *dev, struct device_attribute *attr,
+			   const char *buf, size_t count)
+{
+	struct adf_accel_dev *accel_dev;
+	u32 accel_id;
+	int ret;
+
+	accel_dev = adf_devmgr_pci_to_accel_dev(to_pci_dev(dev));
+	if (!accel_dev)
+		return -EINVAL;
+
+	accel_id = accel_dev->accel_id;
+
+	if (adf_devmgr_in_reset(accel_dev) || adf_dev_in_use(accel_dev)) {
+		dev_info(dev, "Device qat_dev%d is busy\n", accel_id);
+		return -EBUSY;
+	}
+
+	ret = sysfs_match_string(state_operations, buf);
+	if (ret < 0)
+		return ret;
+
+	switch (ret) {
+	case DEV_DOWN:
+		if (!adf_dev_started(accel_dev)) {
+			dev_info(dev, "Device qat_dev%d already down\n",
+				 accel_id);
+			return -EINVAL;
+		}
+
+		dev_info(dev, "Stopping device qat_dev%d\n", accel_id);
+
+		adf_dev_stop(accel_dev);
+		adf_dev_shutdown(accel_dev);
+
+		break;
+	case DEV_UP:
+		if (adf_dev_started(accel_dev)) {
+			dev_info(dev, "Device qat_dev%d already up\n",
+				 accel_id);
+			return -EINVAL;
+		}
+
+		dev_info(dev, "Starting device qat_dev%d\n", accel_id);
+
+		ret = GET_HW_DATA(accel_dev)->dev_config(accel_dev);
+		if (!ret)
+			ret = adf_dev_init(accel_dev);
+		if (!ret)
+			ret = adf_dev_start(accel_dev);
+
+		if (ret < 0) {
+			dev_err(dev, "Failed to start device qat_dev%d\n",
+				accel_id);
+			adf_dev_stop(accel_dev);
+			adf_dev_shutdown(accel_dev);
+			return ret;
+		}
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return count;
+}
+
+static DEVICE_ATTR_RW(state);
+
+static struct attribute *qat_attrs[] = {
+	&dev_attr_state.attr,
+	NULL,
+};
+
+static struct attribute_group qat_group = {
+	.attrs = qat_attrs,
+	.name = "qat",
+};
+
+int adf_sysfs_init(struct adf_accel_dev *accel_dev)
+{
+	int ret;
+
+	ret = devm_device_add_group(&GET_DEV(accel_dev), &qat_group);
+	if (ret) {
+		dev_err(&GET_DEV(accel_dev),
+			"Failed to create qat attribute group: %d\n", ret);
+	}
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(adf_sysfs_init);
-- 
2.36.1

[PATCH 2/4] crypto: qat - change behaviour of adf_cfg_add_key_value_param()

[Giovanni Cabiddu]

The function adf_cfg_add_key_value_param() allows to insert duplicates
entries in the key value store of the driver.

Change the behaviour of that function to the following policy:
- if the key doesn't exist, add it;
- if the key already exists with a different value, then delete it and
  replace it with a new one containing the new value;
- if the key exists with the same value, then return without doing
  anything.

The behaviour of this function has been changed in order to easily
update key-values in the driver database. In particular this is required
to update the value of the ServiceEnables key used to change the service
loaded on a device.

Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Reviewed-by: Fiona Trahe <fiona.trahe@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
---
 drivers/crypto/qat/qat_common/adf_cfg.c | 41 ++++++++++++++++++++++++-
 1 file changed, 40 insertions(+), 1 deletion(-)

diff --git a/drivers/crypto/qat/qat_common/adf_cfg.c b/drivers/crypto/qat/qat_common/adf_cfg.c
index b5b208cbe5a1..e61b3e13db3b 100644
--- a/drivers/crypto/qat/qat_common/adf_cfg.c
+++ b/drivers/crypto/qat/qat_common/adf_cfg.c
@@ -128,6 +128,24 @@ static void adf_cfg_keyval_add(struct adf_cfg_key_val *new,
 	list_add_tail(&new->list, &sec->param_head);
 }
 
+static void adf_cfg_keyval_remove(const char *key, struct adf_cfg_section *sec)
+{
+	struct list_head *head = &sec->param_head;
+	struct list_head *list_ptr, *tmp;
+
+	list_for_each_prev_safe(list_ptr, tmp, head) {
+		struct adf_cfg_key_val *ptr =
+			list_entry(list_ptr, struct adf_cfg_key_val, list);
+
+		if (strncmp(ptr->key, key, sizeof(ptr->key)))
+			continue;
+
+		list_del(list_ptr);
+		kfree(ptr);
+		break;
+	}
+}
+
 static void adf_cfg_keyval_del_all(struct list_head *head)
 {
 	struct list_head *list_ptr, *tmp;
@@ -208,7 +226,8 @@ static int adf_cfg_key_val_get(struct adf_accel_dev *accel_dev,
  * @type: Type - string, int or address
  *
  * Function adds configuration key - value entry in the appropriate section
- * in the given acceleration device
+ * in the given acceleration device. If the key exists already, the value
+ * is updated.
  * To be used by QAT device specific drivers.
  *
  * Return: 0 on success, error code otherwise.
@@ -222,6 +241,8 @@ int adf_cfg_add_key_value_param(struct adf_accel_dev *accel_dev,
 	struct adf_cfg_key_val *key_val;
 	struct adf_cfg_section *section = adf_cfg_sec_find(accel_dev,
 							   section_name);
+	char temp_val[ADF_CFG_MAX_VAL_LEN_IN_BYTES];
+
 	if (!section)
 		return -EFAULT;
 
@@ -246,6 +267,24 @@ int adf_cfg_add_key_value_param(struct adf_accel_dev *accel_dev,
 		return -EINVAL;
 	}
 	key_val->type = type;
+
+	/* Add the key-value pair as below policy:
+	 * 1. if the key doesn't exist, add it;
+	 * 2. if the key already exists with a different value then update it
+	 *    to the new value (the key is deleted and the newly created
+	 *    key_val containing the new value is added to the database);
+	 * 3. if the key exists with the same value, then return without doing
+	 *    anything (the newly created key_val is freed).
+	 */
+	if (!adf_cfg_key_val_get(accel_dev, section_name, key, temp_val)) {
+		if (strncmp(temp_val, key_val->val, sizeof(temp_val))) {
+			adf_cfg_keyval_remove(key, section);
+		} else {
+			kfree(key_val);
+			return 0;
+		}
+	}
+
 	down_write(&cfg->lock);
 	adf_cfg_keyval_add(key_val, section);
 	up_write(&cfg->lock);
-- 
2.36.1

[PATCH 3/4] crypto: qat - relocate and rename adf_sriov_prepare_restart()

[Giovanni Cabiddu]

The function adf_sriov_prepare_restart() is used in adf_sriov.c to stop
and shutdown a device preserving its configuration.

Since this function will be re-used by the logic that allows to
reconfigure the device through sysfs, move it to adf_init.c and rename
it as adf_dev_shutdown_cache_cfg();

Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Reviewed-by: Fiona Trahe <fiona.trahe@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
---
 .../crypto/qat/qat_common/adf_common_drv.h    |  1 +
 drivers/crypto/qat/qat_common/adf_init.c      | 26 +++++++++++++++++
 drivers/crypto/qat/qat_common/adf_sriov.c     | 28 +------------------
 3 files changed, 28 insertions(+), 27 deletions(-)

diff --git a/drivers/crypto/qat/qat_common/adf_common_drv.h b/drivers/crypto/qat/qat_common/adf_common_drv.h
index 0f3031f9055d..b6104f8c3571 100644
--- a/drivers/crypto/qat/qat_common/adf_common_drv.h
+++ b/drivers/crypto/qat/qat_common/adf_common_drv.h
@@ -61,6 +61,7 @@ int adf_dev_init(struct adf_accel_dev *accel_dev);
 int adf_dev_start(struct adf_accel_dev *accel_dev);
 void adf_dev_stop(struct adf_accel_dev *accel_dev);
 void adf_dev_shutdown(struct adf_accel_dev *accel_dev);
+int adf_dev_shutdown_cache_cfg(struct adf_accel_dev *accel_dev);
 
 void adf_devmgr_update_class_index(struct adf_hw_device_data *hw_data);
 void adf_clean_vf_map(bool);
diff --git a/drivers/crypto/qat/qat_common/adf_init.c b/drivers/crypto/qat/qat_common/adf_init.c
index c2c718f1b489..33a9a46d6949 100644
--- a/drivers/crypto/qat/qat_common/adf_init.c
+++ b/drivers/crypto/qat/qat_common/adf_init.c
@@ -363,3 +363,29 @@ int adf_dev_restarted_notify(struct adf_accel_dev *accel_dev)
 	}
 	return 0;
 }
+
+int adf_dev_shutdown_cache_cfg(struct adf_accel_dev *accel_dev)
+{
+	char services[ADF_CFG_MAX_VAL_LEN_IN_BYTES] = {0};
+	int ret;
+
+	ret = adf_cfg_get_param_value(accel_dev, ADF_GENERAL_SEC,
+				      ADF_SERVICES_ENABLED, services);
+
+	adf_dev_stop(accel_dev);
+	adf_dev_shutdown(accel_dev);
+
+	if (!ret) {
+		ret = adf_cfg_section_add(accel_dev, ADF_GENERAL_SEC);
+		if (ret)
+			return ret;
+
+		ret = adf_cfg_add_key_value_param(accel_dev, ADF_GENERAL_SEC,
+						  ADF_SERVICES_ENABLED,
+						  services, ADF_STR);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
diff --git a/drivers/crypto/qat/qat_common/adf_sriov.c b/drivers/crypto/qat/qat_common/adf_sriov.c
index f38b2ffde146..b2db1d70d71f 100644
--- a/drivers/crypto/qat/qat_common/adf_sriov.c
+++ b/drivers/crypto/qat/qat_common/adf_sriov.c
@@ -120,32 +120,6 @@ void adf_disable_sriov(struct adf_accel_dev *accel_dev)
 }
 EXPORT_SYMBOL_GPL(adf_disable_sriov);
 
-static int adf_sriov_prepare_restart(struct adf_accel_dev *accel_dev)
-{
-	char services[ADF_CFG_MAX_VAL_LEN_IN_BYTES] = {0};
-	int ret;
-
-	ret = adf_cfg_get_param_value(accel_dev, ADF_GENERAL_SEC,
-				      ADF_SERVICES_ENABLED, services);
-
-	adf_dev_stop(accel_dev);
-	adf_dev_shutdown(accel_dev);
-
-	if (!ret) {
-		ret = adf_cfg_section_add(accel_dev, ADF_GENERAL_SEC);
-		if (ret)
-			return ret;
-
-		ret = adf_cfg_add_key_value_param(accel_dev, ADF_GENERAL_SEC,
-						  ADF_SERVICES_ENABLED,
-						  services, ADF_STR);
-		if (ret)
-			return ret;
-	}
-
-	return 0;
-}
-
 /**
  * adf_sriov_configure() - Enable SRIOV for the device
  * @pdev:  Pointer to PCI device.
@@ -185,7 +159,7 @@ int adf_sriov_configure(struct pci_dev *pdev, int numvfs)
 			return -EBUSY;
 		}
 
-		ret = adf_sriov_prepare_restart(accel_dev);
+		ret = adf_dev_shutdown_cache_cfg(accel_dev);
 		if (ret)
 			return ret;
 	}
-- 
2.36.1

[PATCH 4/4] crypto: qat - expose device config through sysfs for 4xxx

[Giovanni Cabiddu]

qat_4xxx devices can be configured to allow either crypto or compression
operations. At the moment, devices are configured statically according to
the following rule:
- odd numbered devices assigned to compression services
- even numbered devices assigned to crypto services

Expose the sysfs attribute /sys/bus/pci/devices/<BDF>/qat/cfg_services
to allow to detect the configuration of a device and to change it.

The `cfg_service` attribute is only exposed for qat_4xxx devices and it
is limited to two configurations: (1) "sym;asym" for crypto services and
"dc" for compression services.

Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Co-developed-by: Tomasz Kowallik <tomaszx.kowalik@intel.com>
Signed-off-by: Tomasz Kowallik <tomaszx.kowalik@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Reviewed-by: Fiona Trahe <fiona.trahe@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
---
 Documentation/ABI/testing/sysfs-driver-qat | 37 ++++++++++
 drivers/crypto/qat/qat_common/adf_sysfs.c  | 80 ++++++++++++++++++++--
 2 files changed, 113 insertions(+), 4 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-driver-qat b/Documentation/ABI/testing/sysfs-driver-qat
index 0915253efaa3..58a4152f5180 100644
--- a/Documentation/ABI/testing/sysfs-driver-qat
+++ b/Documentation/ABI/testing/sysfs-driver-qat
@@ -19,3 +19,40 @@ Description:	Reports the current state of the QAT device and allows to
 		if the device is up and vice versa.
 
 		This attribute is only available for qat_4xxx devices.
+
+What:		/sys/bus/pci/devices/<BDF>/qat/cfg_services
+Date:		June 2022
+KernelVersion:	5.19
+Contact:	qat-linux@intel.com
+Description:	Reports the current configuration of the QAT device and allows
+		to change it.
+
+		This attribute is RW.
+
+		Returned values:
+			sym;asym:	the device is configured for running
+					crypto services
+			dc:		the device is configured for running
+					compression services
+
+		Allowed values:
+			sym;asym:	configure the device for running
+					crypto services
+			dc:		configure the device for running
+					compression services
+
+		It is possible to set the configuration only if the device
+		is in the `down` state (see /sys/bus/pci/devices/<BDF>/qat/state)
+
+		The following example shows how to change the configuration of
+		a device configured for running crypto services in order to
+		run data compression:
+			# cat /sys/bus/pci/devices/<BDF>/qat/state
+			up
+			# cat /sys/bus/pci/devices/<BDF>/qat/cfg_services
+			sym;asym
+			# echo down > /sys/bus/pci/devices/<BDF>/qat/state
+			# echo dc > /sys/bus/pci/devices/<BDF>/qat/cfg_services
+			# echo up > /sys/bus/pci/devices/<BDF>/qat/state
+
+		This attribute is only available for qat_4xxx devices.
diff --git a/drivers/crypto/qat/qat_common/adf_sysfs.c b/drivers/crypto/qat/qat_common/adf_sysfs.c
index 8f47a5694dd7..e8b078e719c2 100644
--- a/drivers/crypto/qat/qat_common/adf_sysfs.c
+++ b/drivers/crypto/qat/qat_common/adf_sysfs.c
@@ -58,8 +58,9 @@ static ssize_t state_store(struct device *dev, struct device_attribute *attr,
 
 		dev_info(dev, "Stopping device qat_dev%d\n", accel_id);
 
-		adf_dev_stop(accel_dev);
-		adf_dev_shutdown(accel_dev);
+		ret = adf_dev_shutdown_cache_cfg(accel_dev);
+		if (ret < 0)
+			return -EINVAL;
 
 		break;
 	case DEV_UP:
@@ -80,8 +81,7 @@ static ssize_t state_store(struct device *dev, struct device_attribute *attr,
 		if (ret < 0) {
 			dev_err(dev, "Failed to start device qat_dev%d\n",
 				accel_id);
-			adf_dev_stop(accel_dev);
-			adf_dev_shutdown(accel_dev);
+			adf_dev_shutdown_cache_cfg(accel_dev);
 			return ret;
 		}
 		break;
@@ -92,10 +92,82 @@ static ssize_t state_store(struct device *dev, struct device_attribute *attr,
 	return count;
 }
 
+static const char * const services_operations[] = {
+	ADF_CFG_CY,
+	ADF_CFG_DC,
+};
+
+static ssize_t cfg_services_show(struct device *dev, struct device_attribute *attr,
+				 char *buf)
+{
+	char services[ADF_CFG_MAX_VAL_LEN_IN_BYTES] = {0};
+	struct adf_accel_dev *accel_dev;
+	int ret;
+
+	accel_dev = adf_devmgr_pci_to_accel_dev(to_pci_dev(dev));
+	if (!accel_dev)
+		return -EINVAL;
+
+	ret = adf_cfg_get_param_value(accel_dev, ADF_GENERAL_SEC,
+				      ADF_SERVICES_ENABLED, services);
+	if (ret)
+		return ret;
+
+	return sysfs_emit(buf, "%s\n", services);
+}
+
+static int adf_sysfs_update_dev_config(struct adf_accel_dev *accel_dev,
+				       const char *services)
+{
+	return adf_cfg_add_key_value_param(accel_dev, ADF_GENERAL_SEC,
+					   ADF_SERVICES_ENABLED, services,
+					   ADF_STR);
+}
+
+static ssize_t cfg_services_store(struct device *dev, struct device_attribute *attr,
+				  const char *buf, size_t count)
+{
+	struct adf_hw_device_data *hw_data;
+	struct adf_accel_dev *accel_dev;
+	int ret;
+
+	ret = sysfs_match_string(services_operations, buf);
+	if (ret < 0)
+		return ret;
+
+	accel_dev = adf_devmgr_pci_to_accel_dev(to_pci_dev(dev));
+	if (!accel_dev)
+		return -EINVAL;
+
+	if (adf_dev_started(accel_dev)) {
+		dev_info(dev, "Device qat_dev%d must be down to reconfigure the service.\n",
+			 accel_dev->accel_id);
+		return -EINVAL;
+	}
+
+	ret = adf_sysfs_update_dev_config(accel_dev, services_operations[ret]);
+	if (ret < 0)
+		return ret;
+
+	hw_data = GET_HW_DATA(accel_dev);
+
+	/* Update capabilities mask after change in configuration.
+	 * A call to this function is required as capabilities are, at the
+	 * moment, tied to configuration
+	 */
+	hw_data->accel_capabilities_mask = hw_data->get_accel_cap(accel_dev);
+	if (!hw_data->accel_capabilities_mask)
+		return -EINVAL;
+
+	return count;
+}
+
 static DEVICE_ATTR_RW(state);
+static DEVICE_ATTR_RW(cfg_services);
 
 static struct attribute *qat_attrs[] = {
 	&dev_attr_state.attr,
+	&dev_attr_cfg_services.attr,
 	NULL,
 };
 
-- 
2.36.1

Re: [PATCH 0/4] crypto: qat - enable configuration for 4xxx

[Vlad Dronov]

Hi, Giovanni, all,

I've read through the patchset and it looks good to me. I would have made
the following two minor (really minor) changes.

Add a missing "in" to the patch message:

@@ [PATCH 1/4] crypto: qat - expose device state through sysfs for 4xxx
-allow the change of states even if the device is the down state.
+allow the change of states even if the device is in the down state.

Probably add an indication that cfg_services has been really changed to
the sysfs-driver-qat doc for clarity:

@@ [PATCH 4/4] crypto: qat - expose device config through sysfs for 4xxx
diff --git a/Documentation/ABI/testing/sysfs-driver-qat
b/Documentation/ABI/testing/sysfs-driver-qat
 +                       # echo dc > /sys/bus/pci/devices/<BDF>/qat/cfg_services
 +                       # echo up > /sys/bus/pci/devices/<BDF>/qat/state
++                       # cat /sys/bus/pci/devices/<BDF>/qat/cfg_services
++                       dc

Anyway, please feel free to use:

Reviewed-by: Vladis Dronov <vdronov@redhat.com>

Best regards,
Vladis Dronov | Red Hat, Inc. | The Core Kernel | Senior Software Engineer

On Tue, May 17, 2022 at 4:10 PM Giovanni Cabiddu
<giovanni.cabiddu@intel.com> wrote:
>
> qat_4xxx devices can be configured to allow either crypto or compression
> operations. By default, devices are configured statically according
> to following rule:
>   - odd numbered devices assigned to compression services
>   - even numbered devices assigned to crypto services
>
> This set exposes two attributes in sysfs that allow to report and change
> the state and the configuration of a QAT 4xxx device.
> The first, /sys/bus/pci/devices/<BDF>/qat/state, allows to bring a
> device down in order to change the configuration, and bring it up again.
> The second, /sys/bus/pci/devices/<BDF>/qat/cfg_services, allows to
> inspect the current configuration of a device (i.e. crypto or
> compression) and change it.
>
> Giovanni Cabiddu (4):
>   crypto: qat - expose device state through sysfs for 4xxx
>   crypto: qat - change behaviour of adf_cfg_add_key_value_param()
>   crypto: qat - relocate and rename adf_sriov_prepare_restart()
>   crypto: qat - expose device config through sysfs for 4xxx
>
>  Documentation/ABI/testing/sysfs-driver-qat    |  58 ++++++
>  .../crypto/qat/qat_4xxx/adf_4xxx_hw_data.c    |   1 +
>  .../crypto/qat/qat_4xxx/adf_4xxx_hw_data.h    |   1 +
>  drivers/crypto/qat/qat_4xxx/adf_drv.c         |   6 +-
>  drivers/crypto/qat/qat_common/Makefile        |   1 +
>  .../crypto/qat/qat_common/adf_accel_devices.h |   1 +
>  drivers/crypto/qat/qat_common/adf_cfg.c       |  41 +++-
>  .../crypto/qat/qat_common/adf_common_drv.h    |   3 +
>  drivers/crypto/qat/qat_common/adf_init.c      |  26 +++
>  drivers/crypto/qat/qat_common/adf_sriov.c     |  28 +--
>  drivers/crypto/qat/qat_common/adf_sysfs.c     | 191 ++++++++++++++++++
>  11 files changed, 328 insertions(+), 29 deletions(-)
>  create mode 100644 Documentation/ABI/testing/sysfs-driver-qat
>  create mode 100644 drivers/crypto/qat/qat_common/adf_sysfs.c
>
> --
> 2.36.1
>

Re: [PATCH 0/4] crypto: qat - enable configuration for 4xxx

[Giovanni Cabiddu]

On Thu, Jun 02, 2022 at 03:21:57PM +0200, Vlad Dronov wrote:
> Hi, Giovanni, all,
> 
> I've read through the patchset and it looks good to me. I would have made
> the following two minor (really minor) changes.
> 
> Add a missing "in" to the patch message:
> 
> @@ [PATCH 1/4] crypto: qat - expose device state through sysfs for 4xxx
> -allow the change of states even if the device is the down state.
> +allow the change of states even if the device is in the down state.
> 
> Probably add an indication that cfg_services has been really changed to
> the sysfs-driver-qat doc for clarity:
> 
> @@ [PATCH 4/4] crypto: qat - expose device config through sysfs for 4xxx
> diff --git a/Documentation/ABI/testing/sysfs-driver-qat
> b/Documentation/ABI/testing/sysfs-driver-qat
>  +                       # echo dc > /sys/bus/pci/devices/<BDF>/qat/cfg_services
>  +                       # echo up > /sys/bus/pci/devices/<BDF>/qat/state
> ++                       # cat /sys/bus/pci/devices/<BDF>/qat/cfg_services
> ++                       dc
> 
> Anyway, please feel free to use:
> 
> Reviewed-by: Vladis Dronov <vdronov@redhat.com>
Thanks Vladis!
I'm going to rework and re-send.
Also I'm going to update the KernelVersion attribute in the
documentation as this patch didn't make kernel 5.19.

-- 
Giovanni