[PATCH 0/3] Add DDR sleep stats

[PATCH 0/3] Add DDR sleep stats

[Konrad Dybcio]

RPMh LPM stats are quite extensive, but we can extract even more data by
poking the AOSS QMP. This series adds DDR stats using this approach,
setting it up on 8280 as an example.

Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
---
Konrad Dybcio (3):
      dt-bindings: soc: qcom: stats: Add QMP handle
      soc: qcom: stats: Add DDR sleep stats
      arm64: dts: qcom: sc8280xp: Add QMP handle to RPMh stats

 .../devicetree/bindings/soc/qcom/qcom-stats.yaml   |  14 ++
 arch/arm64/boot/dts/qcom/sc8280xp.dtsi             |   1 +
 drivers/soc/qcom/qcom_stats.c                      | 186 ++++++++++++++++++++-
 3 files changed, 200 insertions(+), 1 deletion(-)
---
base-commit: 1f5c003694fab4b1ba6cbdcc417488b975c088d0
change-id: 20231130-topic-ddr_sleep_stats-d968e2e28083

Best regards,
-- 
Konrad Dybcio <konrad.dybcio@linaro.org>

[PATCH 1/3] dt-bindings: soc: qcom: stats: Add QMP handle

[Konrad Dybcio]

The stats can be expanded by poking the Always-On Subsystem through QMP.
Allow passing a QMP handle for configurations that support it.

Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
---
 Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml b/Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml
index 96a7f1822022..686a7ef2f48a 100644
--- a/Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml
+++ b/Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml
@@ -31,10 +31,24 @@ properties:
   reg:
     maxItems: 1
 
+  qcom,qmp:
+    $ref: /schemas/types.yaml#/definitions/phandle
+    description: Reference to the AOSS side-channel message RAM
+
 required:
   - compatible
   - reg
 
+allOf:
+  - if:
+      not:
+        properties:
+          compatible:
+            const: qcom,rpmh-stats
+    then:
+      properties:
+        qcom,qmp: false
+
 additionalProperties: false
 
 examples:

-- 
2.43.0

Re: [PATCH 1/3] dt-bindings: soc: qcom: stats: Add QMP handle

[Krzysztof Kozlowski]

On 30/11/2023 15:58, Konrad Dybcio wrote:
> The stats can be expanded by poking the Always-On Subsystem through QMP.
> Allow passing a QMP handle for configurations that support it.
> 
> Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
> ---
>  Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)

Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>

Best regards,
Krzysztof

Re: [PATCH 1/3] dt-bindings: soc: qcom: stats: Add QMP handle

[Johan Hovold]

On Thu, Nov 30, 2023 at 03:58:21PM +0100, Konrad Dybcio wrote:
> The stats can be expanded by poking the Always-On Subsystem through QMP.
> Allow passing a QMP handle for configurations that support it.

I know Qualcomm likes their opaque acronyms, but how about mentioning
what QMP stands for here and in the binding below?

Especially since we also have QMP PHYs (Qualcomm Multi PHY?), which I
doubt this is related to.

> Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
> ---
>  Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml b/Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml
> index 96a7f1822022..686a7ef2f48a 100644
> --- a/Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml
> +++ b/Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml
> @@ -31,10 +31,24 @@ properties:
>    reg:
>      maxItems: 1
>  
> +  qcom,qmp:
> +    $ref: /schemas/types.yaml#/definitions/phandle
> +    description: Reference to the AOSS side-channel message RAM

Johan

Re: [PATCH 1/3] dt-bindings: soc: qcom: stats: Add QMP handle

[Konrad Dybcio]

On 1.12.2023 10:44, Johan Hovold wrote:
> On Thu, Nov 30, 2023 at 03:58:21PM +0100, Konrad Dybcio wrote:
>> The stats can be expanded by poking the Always-On Subsystem through QMP.
>> Allow passing a QMP handle for configurations that support it.
> 
> I know Qualcomm likes their opaque acronyms, but how about mentioning
> what QMP stands for here and in the binding below?
I'll amend the commit message for v2.


> Especially since we also have QMP PHYs (Qualcomm Multi PHY?), which I
> doubt this is related to.
There's only so many three-letter acronyms that begin with a Q ;)

QMP stands for Qualcomm Messaging Protocol, which is a JSON-like way
to communicate with the Always-On Subsystem (which is responsible for
things such as managing low-power states and general SoC building blocks
housekeeping)

Konrad

[PATCH 2/3] soc: qcom: stats: Add DDR sleep stats

[Konrad Dybcio]

Add DDR sleep stats that include:

- the available RAM low power states
- per-state residency information
- per-frequency residency information (for some freqs only, it seems)
- DDR vote information (AB/IB)

and some magic thing that we're yet to decode.

Based on the msm-5.4 downstream implementation, debugged with some help
from Qualcomm's Maulik Shah.

Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
---
 drivers/soc/qcom/qcom_stats.c | 186 +++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 185 insertions(+), 1 deletion(-)

diff --git a/drivers/soc/qcom/qcom_stats.c b/drivers/soc/qcom/qcom_stats.c
index 0216fc24f2ca..94ee543d2e8e 100644
--- a/drivers/soc/qcom/qcom_stats.c
+++ b/drivers/soc/qcom/qcom_stats.c
@@ -11,6 +11,7 @@
 #include <linux/platform_device.h>
 #include <linux/seq_file.h>
 
+#include <linux/soc/qcom/qcom_aoss.h>
 #include <linux/soc/qcom/smem.h>
 #include <clocksource/arm_arch_timer.h>
 
@@ -22,8 +23,20 @@
 #define LAST_ENTERED_AT_OFFSET	0x8
 #define LAST_EXITED_AT_OFFSET	0x10
 #define ACCUMULATED_OFFSET	0x18
+#define DDR_DYNAMIC_OFFSET	0x1c
+ #define DDR_OFFSET_MASK	GENMASK(9, 0)
 #define CLIENT_VOTES_OFFSET	0x20
 
+#define ARCH_TIMER_FREQ		19200000
+#define DDR_MAGIC_KEY1		0xA1157A75 /* leetspeak "ALLSTATS" */
+#define DDR_MAX_NUM_ENTRIES	20
+
+#define DDR_VOTE_DRV_MAX	18
+#define DDR_VOTE_DRV_ABSENT	0xdeaddead
+#define DDR_VOTE_DRV_INVALID	0xffffdead
+#define DDR_VOTE_X		GENMASK(27, 14)
+#define DDR_VOTE_Y		GENMASK(13, 0)
+
 struct subsystem_data {
 	const char *name;
 	u32 smem_item;
@@ -48,6 +61,7 @@ struct stats_config {
 	bool appended_stats_avail;
 	bool dynamic_offset;
 	bool subsystem_stats_in_smem;
+	bool ddr_stats;
 };
 
 struct stats_data {
@@ -68,6 +82,25 @@ struct appended_stats {
 	u32 reserved[3];
 };
 
+struct ddr_stats_entry {
+	u32 name;
+	u32 count;
+	u64 dur;
+} __packed;
+
+struct ddr_stats {
+	u32 key;
+	u32 entry_count;
+#define MAX_DDR_STAT_ENTRIES	20
+	struct ddr_stats_entry entry[MAX_DDR_STAT_ENTRIES];
+} __packed;
+
+struct ddr_stats_data {
+	struct device *dev;
+	void __iomem *base;
+	struct qmp *qmp;
+};
+
 static void qcom_print_stats(struct seq_file *s, const struct sleep_stats *stat)
 {
 	u64 accumulated = stat->accumulated;
@@ -118,6 +151,108 @@ static int qcom_soc_sleep_stats_show(struct seq_file *s, void *unused)
 	return 0;
 }
 
+#define DDR_NAME_TYPE		GENMASK(15, 8)
+ #define DDR_NAME_TYPE_LPM	0
+ #define DDR_NAME_TYPE_FREQ	1
+
+#define DDR_NAME_LPM_NAME	GENMASK(7, 0)
+
+#define DDR_NAME_FREQ_MHZ	GENMASK(31, 16)
+#define DDR_NAME_FREQ_CP_IDX	GENMASK(4, 0)
+static void qcom_ddr_stats_print(struct seq_file *s, struct ddr_stats_entry *entry)
+{
+	u32 cp_idx, name;
+	u8 type;
+
+	type = FIELD_GET(DDR_NAME_TYPE, entry->name);
+
+	switch (type) {
+	case DDR_NAME_TYPE_LPM:
+		name = FIELD_GET(DDR_NAME_LPM_NAME, entry->name);
+
+		seq_printf(s, "LPM  | Type 0x%2x\tcount: %u\ttime: %llums\n",
+			   name, entry->count, entry->dur);
+		break;
+	case DDR_NAME_TYPE_FREQ:
+		cp_idx = FIELD_GET(DDR_NAME_FREQ_CP_IDX, entry->name);
+		name = FIELD_GET(DDR_NAME_FREQ_MHZ, entry->name);
+
+		/* Neither 0Mhz nor 0 votes is very interesting */
+		if (!name || !entry->count)
+			return;
+
+		seq_printf(s, "Freq | %dMHz (idx %u)\tcount: %u\ttime: %llums\n",
+			   name, cp_idx, entry->count, entry->dur);
+		break;
+	default:
+		seq_printf(s, "Unknown data chunk (type = 0x%x count = 0x%x dur = 0x%llx)\n",
+			   type, entry->count, entry->dur);
+	}
+}
+
+static int qcom_ddr_stats_show(struct seq_file *s, void *unused)
+{
+	struct ddr_stats_data *ddrd = s->private;
+	struct ddr_stats ddr;
+	struct ddr_stats_entry *entry = ddr.entry;
+	u32 entry_count, stats_size;
+	u32 votes[DDR_VOTE_DRV_MAX];
+	int i, ret;
+
+	/* Request a stats sync, it may take some time to update though.. */
+	ret = qmp_send(ddrd->qmp, "{class: ddr, action: freqsync}");
+	if (ret) {
+		dev_err(ddrd->dev, "failed to send QMP message\n");
+		return ret;
+	}
+
+	entry_count = readl(ddrd->base + offsetof(struct ddr_stats, entry_count));
+	if (entry_count > DDR_MAX_NUM_ENTRIES)
+		return -EINVAL;
+
+	/* We're not guaranteed to have DDR_MAX_NUM_ENTRIES */
+	stats_size = sizeof(ddr);
+	stats_size -= DDR_MAX_NUM_ENTRIES * sizeof(*entry);
+	stats_size += entry_count * sizeof(*entry);
+
+	/* Copy and process the stats */
+	memcpy_fromio(&ddr, ddrd->base, stats_size);
+
+	for (i = 0; i < ddr.entry_count; i++) {
+		/* Convert the period to ms */
+		entry[i].dur = mult_frac(MSEC_PER_SEC, entry[i].dur, ARCH_TIMER_FREQ);
+	}
+
+	for (i = 0; i < ddr.entry_count; i++)
+		qcom_ddr_stats_print(s, &entry[i]);
+
+	/* Ask AOSS to dump DDR votes */
+	ret = qmp_send(ddrd->qmp, "{class: ddr, res: drvs_ddr_votes}");
+	if (ret) {
+		dev_err(ddrd->dev, "failed to send QMP message\n");
+		return ret;
+	}
+
+	/* Subsystem votes */
+	memcpy_fromio(votes, ddrd->base + stats_size, sizeof(u32) * DDR_VOTE_DRV_MAX);
+
+	for (i = 0; i < DDR_VOTE_DRV_MAX; i++) {
+		u32 ab, ib;
+
+		if (votes[i] == DDR_VOTE_DRV_ABSENT || votes[i] == DDR_VOTE_DRV_INVALID)
+			ab = ib = votes[i];
+		else {
+			ab = FIELD_GET(DDR_VOTE_X, votes[i]);
+			ib = FIELD_GET(DDR_VOTE_Y, votes[i]);
+		}
+
+		seq_printf(s, "Vote | AB = %5u\tIB = %5u\n", ab, ib);
+	}
+
+	return 0;
+}
+
+DEFINE_SHOW_ATTRIBUTE(qcom_ddr_stats);
 DEFINE_SHOW_ATTRIBUTE(qcom_soc_sleep_stats);
 DEFINE_SHOW_ATTRIBUTE(qcom_subsystem_sleep_stats);
 
@@ -180,13 +315,56 @@ static void qcom_create_subsystem_stat_files(struct dentry *root,
 				    &qcom_subsystem_sleep_stats_fops);
 }
 
+static int qcom_create_ddr_stats_files(struct device *dev,
+				       struct dentry *root,
+				       void __iomem *reg,
+				       const struct stats_config *config)
+{
+	struct ddr_stats_data *ddrd;
+	u32 key, stats_offset;
+	struct dentry *dent;
+
+	/* Nothing to do */
+	if (!config->ddr_stats)
+		return 0;
+
+	ddrd = devm_kzalloc(dev, sizeof(*ddrd), GFP_KERNEL);
+	if (!ddrd)
+		return dev_err_probe(dev, -ENOMEM, "Couldn't allocate DDR stats data\n");
+
+	ddrd->dev = dev;
+
+	/* Get the offset of DDR stats */
+	stats_offset = readl(reg + DDR_DYNAMIC_OFFSET) & DDR_OFFSET_MASK;
+	ddrd->base = reg + stats_offset;
+
+	/* Check if DDR stats are present */
+	key = readl(ddrd->base);
+	if (key != DDR_MAGIC_KEY1)
+		return 0;
+
+	dent = debugfs_create_file("ddr_sleep_stats", 0400, root, ddrd, &qcom_ddr_stats_fops);
+	if (IS_ERR(dent))
+		return PTR_ERR(dent);
+
+	/* QMP is only necessary for DDR votes */
+	ddrd->qmp = qmp_get(dev);
+	if (IS_ERR(ddrd->qmp)) {
+		dev_err(dev, "Couldn't get QMP mailbox: %ld. DDR votes won't be available.\n",
+			PTR_ERR(ddrd->qmp));
+		debugfs_remove(dent);
+	}
+
+	return 0;
+}
+
 static int qcom_stats_probe(struct platform_device *pdev)
 {
 	void __iomem *reg;
 	struct dentry *root;
 	const struct stats_config *config;
 	struct stats_data *d;
-	int i;
+	int i, ret;
 
 	config = device_get_match_data(&pdev->dev);
 	if (!config)
@@ -208,6 +386,11 @@ static int qcom_stats_probe(struct platform_device *pdev)
 
 	qcom_create_subsystem_stat_files(root, config);
 	qcom_create_soc_sleep_stat_files(root, reg, d, config);
+	ret = qcom_create_ddr_stats_files(&pdev->dev, root, reg, config);
+	if (ret) {
+		debugfs_remove_recursive(root);
+		return ret;
+	};
 
 	platform_set_drvdata(pdev, root);
 
@@ -254,6 +437,7 @@ static const struct stats_config rpmh_data = {
 	.appended_stats_avail = false,
 	.dynamic_offset = false,
 	.subsystem_stats_in_smem = true,
+	.ddr_stats = true,
 };
 
 static const struct of_device_id qcom_stats_table[] = {

-- 
2.43.0

[PATCH 3/3] arm64: dts: qcom: sc8280xp: Add QMP handle to RPMh stats

[Konrad Dybcio]

When a handle to QMP is accessible, we can query even more internal
power management stats. Add it.

Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
---
 arch/arm64/boot/dts/qcom/sc8280xp.dtsi | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/boot/dts/qcom/sc8280xp.dtsi b/arch/arm64/boot/dts/qcom/sc8280xp.dtsi
index cad59af7ccef..38cc823c9c87 100644
--- a/arch/arm64/boot/dts/qcom/sc8280xp.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc8280xp.dtsi
@@ -4030,6 +4030,7 @@ aoss_qmp: power-management@c300000 {
 		sram@c3f0000 {
 			compatible = "qcom,rpmh-stats";
 			reg = <0 0x0c3f0000 0 0x400>;
+			qcom,qmp = <&aoss_qmp>;
 		};
 
 		spmi_bus: spmi@c440000 {

-- 
2.43.0

Re: (subset) [PATCH 0/3] Add DDR sleep stats

[Bjorn Andersson]

On Thu, 30 Nov 2023 15:58:20 +0100, Konrad Dybcio wrote:
> RPMh LPM stats are quite extensive, but we can extract even more data by
> poking the AOSS QMP. This series adds DDR stats using this approach,
> setting it up on 8280 as an example.
> 
> 

Applied, thanks!

[3/3] arm64: dts: qcom: sc8280xp: Add QMP handle to RPMh stats
      commit: bcaa71f13cc7160f2d5f9a401e9c58215612f79e

Best regards,
-- 
Bjorn Andersson <andersson@kernel.org>