[PATCH] genirq: move clear of kstat_irqs to free_desc()

[Luigi Rizzo <lrizzo@google.com>]

desc_set_defaults() has a loop to clear the per-cpu counters kstats_irq.

This is only needed in free_desc(), which is used with non-sparse IRQs
so that the irq_desc can be recycled. For newly allocated irq_desc,
the memory comes from alloc_percpu() and is already zeroed out.

Move the loop to free_desc() to avoid wasting time unnecessarily.

This is especially important on large servers with 100+ CPUs, because
each write results in a cache miss, and the write buffer can only have
so many outstanding transactions.

Below is an example of cost on a host with 480 CPUs, taken with
local_irq_save()/restore() around the code to avoid interference.
Measurements taken with kstats
https://github.com/luigirizzo/lr-cstats/tree/main/kstats

BUCKET 	SAMPLES	 AVG TIME(ns)  PERCENTILE

 40           3         2432   0.000366
 41           3         3000   0.000732
 42          24         3241   0.003662
 43          33         3971   0.007690
 44         963         4742   0.125244
 45        1071         5545   0.255981
 46         494         6644   0.316284
 47         352         7661   0.359252
 48         816         9447   0.458862
 49        2214        11493   0.729125
 50        1440        13027   0.904907
 51         428        15219   0.957153
 52         275        18211   0.990722
 53          69        21396   0.999145
 54           4        26125   0.999633
 55           1        28996   0.999755
 56           2        37253   1.000000

Signed-off-by: Luigi Rizzo <lrizzo@google.com>
---
 kernel/irq/irqdesc.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c
index f8e4e13dbe339..fa1e3c7838aa7 100644
--- a/kernel/irq/irqdesc.c
+++ b/kernel/irq/irqdesc.c
@@ -134,8 +134,6 @@ static void desc_set_defaults(unsigned int irq, struct irq_desc *desc, int node,
 	desc->tot_count = 0;
 	desc->name = NULL;
 	desc->owner = owner;
-	for_each_possible_cpu(cpu)
-		*per_cpu_ptr(desc->kstat_irqs, cpu) = (struct irqstat) { };
 	desc_smp_init(desc, node, affinity);
 }
 
@@ -622,8 +620,11 @@ static void free_desc(unsigned int irq)
 {
 	struct irq_desc *desc = irq_to_desc(irq);
 
-	scoped_guard(raw_spinlock_irqsave, &desc->lock)
+	scoped_guard(raw_spinlock_irqsave, &desc->lock) {
 		desc_set_defaults(irq, desc, irq_desc_get_node(desc), NULL, NULL);
+		for_each_possible_cpu(cpu)
+			*per_cpu_ptr(desc->kstat_irqs, cpu) = (struct irqstat) { };
+	}
 	delete_irq_desc(irq);
 }
 
-- 
2.52.0.457.g6b5491de43-goog