* [GIT pull] core/rseq for v7.3-rc1
2026-08-17 11:23 [GIT pull] core/entry for v7.3-rc1 Thomas Gleixner
@ 2026-08-17 11:23 ` Thomas Gleixner
2026-08-19 1:10 ` pr-tracker-bot
2026-08-17 11:23 ` [GIT pull] irq/core " Thomas Gleixner
` (8 subsequent siblings)
9 siblings, 1 reply; 26+ messages in thread
From: Thomas Gleixner @ 2026-08-17 11:23 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest core/rseq branch from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git core-rseq-2026-08-17
up to: 2a98ffac7733: selftests/rseq: Add missing test binaries to .gitignore
Two trivial updates for RSEQ:
- Add the recently added new test binaries to .gitignore
- Fix a trivial typo in a comment.
Thanks,
tglx
------------------>
Cihan Karadag (1):
selftests/rseq: Add missing test binaries to .gitignore
Ding Junlin (1):
selftests/rseq: Fix spelling of accommodate
tools/testing/selftests/rseq/.gitignore | 2 ++
tools/testing/selftests/rseq/rseq.c | 2 +-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/rseq/.gitignore b/tools/testing/selftests/rseq/.gitignore
index ec01d164c1f0..9b6eb6c3922f 100644
--- a/tools/testing/selftests/rseq/.gitignore
+++ b/tools/testing/selftests/rseq/.gitignore
@@ -11,3 +11,5 @@ param_test_mm_cid_benchmark
param_test_mm_cid_compare_twice
syscall_errors_test
slice_test
+legacy_check
+check_optimized
diff --git a/tools/testing/selftests/rseq/rseq.c b/tools/testing/selftests/rseq/rseq.c
index be0d0a97031e..1cd633923b1d 100644
--- a/tools/testing/selftests/rseq/rseq.c
+++ b/tools/testing/selftests/rseq/rseq.c
@@ -73,7 +73,7 @@ static int rseq_ownership;
#define ORIG_RSEQ_ALLOC_SIZE 32
/*
- * Use a union to ensure we allocate a TLS area of 1024 bytes to accomodate an
+ * Use a union to ensure we allocate a TLS area of 1024 bytes to accommodate an
* rseq registration that is larger than the current rseq ABI.
*/
union rseq_tls {
^ permalink raw reply related [flat|nested] 26+ messages in thread* [GIT pull] irq/core for v7.3-rc1
2026-08-17 11:23 [GIT pull] core/entry for v7.3-rc1 Thomas Gleixner
2026-08-17 11:23 ` [GIT pull] core/rseq " Thomas Gleixner
@ 2026-08-17 11:23 ` Thomas Gleixner
2026-08-19 1:10 ` pr-tracker-bot
2026-08-17 11:23 ` [GIT pull] irq/drivers " Thomas Gleixner
` (7 subsequent siblings)
9 siblings, 1 reply; 26+ messages in thread
From: Thomas Gleixner @ 2026-08-17 11:23 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest irq/core branch from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq-core-2026-08-17
up to: 1ad6d4a722f5: MAINTAINERS: Add Radu Rendec as reviewer for the interrupt subsystem
Generic interrupt subsystem updates:
- Remove pointless NULL checks of the kstats_irqs field. That's a
historical left over and not longer required.
- Add Radu Rendec as reviewer. Radu thankfully stepped up to help
reviewing the interrupt core and the related drivers code.
- The usual small improvements and fixes
Thanks,
tglx
------------------>
Kemeng Shi (3):
irqdomain: Plug leak in irq_domain_alloc_irqs_locked() error path
genirq/manage: Use irqd_get_parent_data() helper in __irq_get_irqchip_state()
irqdomain: Remove unnedded NULL check in __irq_domain_[de]activate_irq()
Li RongQing (1):
genirq/msi: Move misplaced EXPORT_SYMBOL_GPL for msi_domain_free_irqs_all()
Radu Rendec (2):
genirq: Remove unnecessary NULL check of the kstat_irqs field
parisc: Remove unnecessary NULL check of the kstat_irqs field
Thomas Gleixner (1):
MAINTAINERS: Add Radu Rendec as reviewer for the interrupt subsystem
MAINTAINERS | 3 +++
arch/parisc/kernel/smp.c | 2 +-
include/linux/irqdesc.h | 2 +-
kernel/irq/irqdesc.c | 11 ++++-------
kernel/irq/irqdomain.c | 8 +++++---
kernel/irq/manage.c | 6 +-----
kernel/irq/msi.c | 2 +-
7 files changed, 16 insertions(+), 18 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index 15011f5752a9..ebf9d388e939 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -13712,6 +13712,7 @@ F: sound/soc/codecs/sma*
IRQ DOMAINS (IRQ NUMBER MAPPING LIBRARY)
M: Thomas Gleixner <tglx@kernel.org>
+R: Radu Rendec <radu@rendec.net>
S: Maintained
P: Documentation/process/maintainer-tip.rst
T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/core
@@ -13723,6 +13724,7 @@ F: kernel/irq/msi.c
IRQ SUBSYSTEM
M: Thomas Gleixner <tglx@kernel.org>
+R: Radu Rendec <radu@rendec.net>
L: linux-kernel@vger.kernel.org
S: Maintained
P: Documentation/process/maintainer-tip.rst
@@ -13737,6 +13739,7 @@ F: lib/group_cpus.c
IRQCHIP DRIVERS
M: Thomas Gleixner <tglx@kernel.org>
+R: Radu Rendec <radu@rendec.net>
L: linux-kernel@vger.kernel.org
S: Maintained
P: Documentation/process/maintainer-tip.rst
diff --git a/arch/parisc/kernel/smp.c b/arch/parisc/kernel/smp.c
index b2d12ab728b1..36be17e32948 100644
--- a/arch/parisc/kernel/smp.c
+++ b/arch/parisc/kernel/smp.c
@@ -343,7 +343,7 @@ static int smp_boot_one_cpu(int cpuid, struct task_struct *idle)
for (i = 0; i < NR_IRQS; i++) {
struct irq_desc *desc = irq_to_desc(i);
- if (desc && desc->kstat_irqs)
+ if (desc)
*per_cpu_ptr(desc->kstat_irqs, cpuid) = (struct irqstat) { };
}
#endif
diff --git a/include/linux/irqdesc.h b/include/linux/irqdesc.h
index 8080db17c1b1..779d6023c5c2 100644
--- a/include/linux/irqdesc.h
+++ b/include/linux/irqdesc.h
@@ -146,7 +146,7 @@ extern struct irq_desc irq_desc[NR_IRQS];
static inline unsigned int irq_desc_kstat_cpu(struct irq_desc *desc,
unsigned int cpu)
{
- return desc->kstat_irqs ? per_cpu(desc->kstat_irqs->cnt, cpu) : 0;
+ return per_cpu(desc->kstat_irqs->cnt, cpu);
}
static inline struct irq_desc *irq_data_to_desc(struct irq_data *data)
diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c
index 80ef4e27dcf4..3a818f07a101 100644
--- a/kernel/irq/irqdesc.c
+++ b/kernel/irq/irqdesc.c
@@ -1004,7 +1004,7 @@ unsigned int kstat_irqs_cpu(unsigned int irq, int cpu)
{
struct irq_desc *desc = irq_to_desc(irq);
- return desc && desc->kstat_irqs ? per_cpu(desc->kstat_irqs->cnt, cpu) : 0;
+ return desc ? irq_desc_kstat_cpu(desc, cpu) : 0;
}
static unsigned int kstat_irqs_desc(struct irq_desc *desc, const struct cpumask *cpumask)
@@ -1026,7 +1026,7 @@ static unsigned int kstat_irqs(unsigned int irq)
{
struct irq_desc *desc = irq_to_desc(irq);
- if (!desc || !desc->kstat_irqs)
+ if (!desc)
return 0;
return kstat_irqs_desc(desc, cpu_possible_mask);
}
@@ -1038,18 +1038,15 @@ void kstat_snapshot_irqs(void)
struct irq_desc *desc;
unsigned int irq;
- for_each_irq_desc(irq, desc) {
- if (!desc->kstat_irqs)
- continue;
+ for_each_irq_desc(irq, desc)
this_cpu_write(desc->kstat_irqs->ref, this_cpu_read(desc->kstat_irqs->cnt));
- }
}
unsigned int kstat_get_irq_since_snapshot(unsigned int irq)
{
struct irq_desc *desc = irq_to_desc(irq);
- if (!desc || !desc->kstat_irqs)
+ if (!desc)
return 0;
return this_cpu_read(desc->kstat_irqs->cnt) - this_cpu_read(desc->kstat_irqs->ref);
}
diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
index f15c9f1223bb..57c819da30c2 100644
--- a/kernel/irq/irqdomain.c
+++ b/kernel/irq/irqdomain.c
@@ -1658,7 +1658,7 @@ static int irq_domain_alloc_irqs_locked(struct irq_domain *domain, int irq_base,
for (i = 0; i < nr_irqs; i++) {
ret = irq_domain_trim_hierarchy(virq + i);
if (ret)
- goto out_free_irq_data;
+ goto out_free_irqs;
}
for (i = 0; i < nr_irqs; i++)
@@ -1666,6 +1666,8 @@ static int irq_domain_alloc_irqs_locked(struct irq_domain *domain, int irq_base,
return virq;
+out_free_irqs:
+ irq_domain_free_irqs_hierarchy(domain, virq, nr_irqs);
out_free_irq_data:
irq_domain_free_irq_data(virq, nr_irqs);
out_free_desc:
@@ -1963,7 +1965,7 @@ EXPORT_SYMBOL_GPL(irq_domain_free_irqs_parent);
static void __irq_domain_deactivate_irq(struct irq_data *irq_data)
{
- if (irq_data && irq_data->domain) {
+ if (irq_data->domain) {
struct irq_domain *domain = irq_data->domain;
if (domain->ops->deactivate)
@@ -1977,7 +1979,7 @@ static int __irq_domain_activate_irq(struct irq_data *irqd, bool reserve)
{
int ret = 0;
- if (irqd && irqd->domain) {
+ if (irqd->domain) {
struct irq_domain *domain = irqd->domain;
if (irqd->parent_data)
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index 7eb07e3bdb4c..4fa569db8ec7 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -2703,11 +2703,7 @@ static int __irq_get_irqchip_state(struct irq_data *data, enum irqchip_irq_state
return -ENODEV;
if (chip->irq_get_irqchip_state)
break;
-#ifdef CONFIG_IRQ_DOMAIN_HIERARCHY
- data = data->parent_data;
-#else
- data = NULL;
-#endif
+ data = irqd_get_parent_data(data);
} while (data);
if (data)
diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c
index 903be7289c53..fb5f372215bf 100644
--- a/kernel/irq/msi.c
+++ b/kernel/irq/msi.c
@@ -1683,7 +1683,6 @@ void msi_domain_free_irqs_range(struct device *dev, unsigned int domid,
guard(msi_descs_lock)(dev);
msi_domain_free_irqs_range_locked(dev, domid, first, last);
}
-EXPORT_SYMBOL_GPL(msi_domain_free_irqs_all);
/**
* msi_domain_free_irqs_all_locked - Free all interrupts from a MSI interrupt domain
@@ -1714,6 +1713,7 @@ void msi_domain_free_irqs_all(struct device *dev, unsigned int domid)
guard(msi_descs_lock)(dev);
msi_domain_free_irqs_all_locked(dev, domid);
}
+EXPORT_SYMBOL_GPL(msi_domain_free_irqs_all);
/**
* msi_device_domain_free_wired - Free a wired interrupt in @domain
^ permalink raw reply related [flat|nested] 26+ messages in thread* [GIT pull] irq/drivers for v7.3-rc1
2026-08-17 11:23 [GIT pull] core/entry for v7.3-rc1 Thomas Gleixner
2026-08-17 11:23 ` [GIT pull] core/rseq " Thomas Gleixner
2026-08-17 11:23 ` [GIT pull] irq/core " Thomas Gleixner
@ 2026-08-17 11:23 ` Thomas Gleixner
2026-08-19 1:10 ` pr-tracker-bot
2026-08-17 11:23 ` [GIT pull] locking/futex " Thomas Gleixner
` (6 subsequent siblings)
9 siblings, 1 reply; 26+ messages in thread
From: Thomas Gleixner @ 2026-08-17 11:23 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest irq/drivers branch from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq-drivers-2026-08-17
up to: 7f328162a98e: irqchip/gic-v3-its: Fix grammar and replace a bit number with its symbol
Interrupt chip driver updates:
- Update the realtek-rtl driver to support multiple interrupt domains and
remove hardcoded assumptions about the underlying demultiplex interrupt
- Remove the unused IMGPDC driver, which got orphaned when the metag
architecture was removed long ago.
- Update the Qualcom PDC interrupt chip driver to support differnent
versions of the IP block, to properly differentiable between direct and
GPIO based interrupts and to utilize pass through mode to the
underlying GIC interrupt.
- Remove redundant dev_err() and dev_err_probe() invocations in various
drivers as that error logging has been moved to devm_request_*_irq().
- Support dynamic MSI-X allocations in the GIC-V3-ITS MSI parent domain
driver. That's important for VFIO pass through devices to avoid
reinitialization of the MSI-X space when extending the vector
range. Such a reinitialization can result is lost interrupts and stale
devices.
- Fix a bunch of resource leaks in various driver error paths/
- The usual small improvements all over the place
Thanks,
tglx
------------------>
Ben Hutchings (1):
irqchip/irq-imgpdc: Remove unused driver
Jinqian Yang (1):
irqchip/gic-v3-its: Enable dynamic MSI-X allocation
Kemeng Shi (4):
irqchip/gic-v3-its: Fix memleak in its_probe_one()
irqchip/gic-v3-its: Fix its node leak in gic_acpi_parse_madt_its()
irqchip/gic-v3-its: Prevent leak in its_vpe_irq_domain_alloc()
irqchip/gic-v3-its: Fix grammar and replace a bit number with its symbol
Lorenzo Pieralisi (1):
irqchip/gic-v5/iwb: Fix stray verb in comment
Markus Stockhausen (7):
dt-bindings: interrupt-controller: realtek,rtl-intc: Allow 2 interrupt cells
irqchip/irq-realtek-rtl: Split out parent setup code
irqchip/irq-realtek-rtl: Add interrupt data structure
irqchip/irq-realtek-rtl: Add mask for interrupt handling
irqchip/irq-realtek-rtl: Add a select function
irqchip/irq-realtek-rtl: Allow shuffled interrupt order
irqchip/irq-realtek-rtl: Activate multiple parents
Maulik Shah (4):
irqchip/qcom-pdc: Restructure version support
irqchip/qcom-pdc: Move all static variables to struct pdc_desc
irqchip/qcom-pdc: Differentiate between direct SPI and GPIO as SPI
irqchip/qcom-pdc: Configure PDC to pass through mode
Pan Chuang (1):
irqchip: Remove redundant dev_err()/dev_err_probe()
Qingshuang Fu (1):
irqchip/renesas-irqc: Fix generic interrupt chip leak on remove
Thomas Gleixner (1):
irqchip/qcom-pdc: Fix kernel doc for qcom_pdc_gic_secondary_set_type()
.../interrupt-controller/realtek,rtl-intc.yaml | 5 +-
drivers/irqchip/Kconfig | 5 -
drivers/irqchip/Makefile | 1 -
drivers/irqchip/irq-gic-its-msi-parent.c | 3 +-
drivers/irqchip/irq-gic-v3-its.c | 30 +-
drivers/irqchip/irq-gic-v5-iwb.c | 2 +-
drivers/irqchip/irq-imgpdc.c | 495 -------------------
drivers/irqchip/irq-qcom-mpm.c | 4 +-
drivers/irqchip/irq-realtek-rtl.c | 152 ++++--
drivers/irqchip/irq-renesas-irqc.c | 1 +
drivers/irqchip/irq-renesas-rzt2h.c | 2 +-
drivers/irqchip/irq-renesas-rzv2h.c | 4 +-
drivers/irqchip/qcom-pdc.c | 540 ++++++++++++++++-----
13 files changed, 568 insertions(+), 676 deletions(-)
delete mode 100644 drivers/irqchip/irq-imgpdc.c
diff --git a/Documentation/devicetree/bindings/interrupt-controller/realtek,rtl-intc.yaml b/Documentation/devicetree/bindings/interrupt-controller/realtek,rtl-intc.yaml
index 833a01cdd1b1..5c52e57c6647 100644
--- a/Documentation/devicetree/bindings/interrupt-controller/realtek,rtl-intc.yaml
+++ b/Documentation/devicetree/bindings/interrupt-controller/realtek,rtl-intc.yaml
@@ -32,8 +32,9 @@ properties:
"#interrupt-cells":
description:
- SoC interrupt line index.
- const: 1
+ First cell is the SoC interrupt line index. Optional second cell
+ specifies the parent interrupt index to route to.
+ enum: [1, 2]
reg:
minItems: 1
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index 42f2278a702d..20b77fbc51ee 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -212,11 +212,6 @@ config HISILICON_IRQ_MBIGEN
select ARM_GIC_V3
select ARM_GIC_V3_ITS
-config IMGPDC_IRQ
- bool
- select GENERIC_IRQ_CHIP
- select IRQ_DOMAIN
-
config IXP4XX_IRQ
bool
select IRQ_DOMAIN
diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
index 74912429c59f..ab33cccd8471 100644
--- a/drivers/irqchip/Makefile
+++ b/drivers/irqchip/Makefile
@@ -44,7 +44,6 @@ obj-$(CONFIG_ARMADA_370_XP_IRQ) += irq-armada-370-xp.o
obj-$(CONFIG_ATMEL_AIC_IRQ) += irq-atmel-aic-common.o irq-atmel-aic.o
obj-$(CONFIG_ATMEL_AIC5_IRQ) += irq-atmel-aic-common.o irq-atmel-aic5.o
obj-$(CONFIG_I8259) += irq-i8259.o
-obj-$(CONFIG_IMGPDC_IRQ) += irq-imgpdc.o
obj-$(CONFIG_IRQ_MIPS_CPU) += irq-mips-cpu.o
obj-$(CONFIG_IXP4XX_IRQ) += irq-ixp4xx.o
obj-$(CONFIG_JCORE_AIC) += irq-jcore-aic.o
diff --git a/drivers/irqchip/irq-gic-its-msi-parent.c b/drivers/irqchip/irq-gic-its-msi-parent.c
index b9257103a999..b2b9d2068bb1 100644
--- a/drivers/irqchip/irq-gic-its-msi-parent.c
+++ b/drivers/irqchip/irq-gic-its-msi-parent.c
@@ -18,7 +18,8 @@
#define ITS_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK | \
MSI_FLAG_PCI_MSIX | \
- MSI_FLAG_MULTI_PCI_MSI)
+ MSI_FLAG_MULTI_PCI_MSI | \
+ MSI_FLAG_PCI_MSIX_ALLOC_DYN)
static int its_translate_frame_address(struct fwnode_handle *msi_node, phys_addr_t *pa)
{
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index b57d81ad33a0..2c43b4ab5b70 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -163,7 +163,7 @@ struct event_lpi_map {
/*
* The ITS view of a device - belongs to an ITS, owns an interrupt
- * translation table, and a list of interrupts. If it some of its
+ * translation table, and a list of interrupts. If some of its
* LPIs are injected into a guest (GICv4), the event_map.vm field
* indicates which one.
*/
@@ -2501,10 +2501,7 @@ static bool its_parse_indirect_baser(struct its_node *its,
/* No need to enable Indirection if memory requirement < (psz*2)bytes */
if ((esz << ids) > (psz * 2)) {
- /*
- * Find out whether hw supports a single or two-level table by
- * table by reading bit at offset '62' after writing '1' to it.
- */
+ /* Find out whether the hardware supports a single or two-level table */
its_write_baser(its, baser, val | GITS_BASER_INDIRECT);
indirect = !!(baser->val & GITS_BASER_INDIRECT);
@@ -4594,6 +4591,13 @@ static int its_vpe_init(struct its_vpe *vpe)
static void its_vpe_teardown(struct its_vpe *vpe)
{
+ /*
+ * If vpt_page is NULL, then its_vpe_init() has failed, and
+ * there is nothing to do as no resource has been allocated.
+ */
+ if (vpe->vpt_page == NULL)
+ return;
+
its_vpe_db_proxy_unmap(vpe);
its_vpe_id_free(vpe->vpe_id);
its_free_pending_table(vpe->vpt_page);
@@ -4674,8 +4678,10 @@ static int its_vpe_irq_domain_alloc(struct irq_domain *domain, unsigned int virq
irqd_set_resend_when_in_progress(irq_get_irq_data(virq + i));
}
- if (err)
+ if (err) {
+ its_vpe_teardown(vm->vpes[i]);
its_vpe_irq_domain_free(domain, virq, i);
+ }
return err;
}
@@ -5322,7 +5328,7 @@ static int __init its_probe_one(struct its_node *its)
err = its_init_domain(its);
if (err)
- goto out_free_tables;
+ goto out_free_collection;
raw_spin_lock(&its_lock);
list_add(&its->entry, &its_nodes);
@@ -5330,6 +5336,8 @@ static int __init its_probe_one(struct its_node *its)
return 0;
+out_free_collection:
+ kfree(its->collections);
out_free_tables:
its_free_tables(its);
out_free_cmd:
@@ -5741,9 +5749,13 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header,
its->flags |= ITS_FLAGS_FORCE_NON_SHAREABLE;
err = its_probe_one(its);
- if (!err)
- return 0;
+ if (err)
+ goto probe_err;
+
+ return 0;
+probe_err:
+ its_node_destroy(its);
node_err:
iort_deregister_domain_token(its_entry->translation_id);
dom_err:
diff --git a/drivers/irqchip/irq-gic-v5-iwb.c b/drivers/irqchip/irq-gic-v5-iwb.c
index 9103feb70ce8..2cf0df53e9dd 100644
--- a/drivers/irqchip/irq-gic-v5-iwb.c
+++ b/drivers/irqchip/irq-gic-v5-iwb.c
@@ -143,7 +143,7 @@ static int gicv5_iwb_irq_domain_translate(struct irq_domain *d, struct irq_fwspe
return -EINVAL;
/*
- * param[0] is be the wire
+ * param[0] is the wire interrupt number
* param[1] is the interrupt type
*/
*hwirq = fwspec->param[0];
diff --git a/drivers/irqchip/irq-imgpdc.c b/drivers/irqchip/irq-imgpdc.c
deleted file mode 100644
index 4feef4ab5fec..000000000000
--- a/drivers/irqchip/irq-imgpdc.c
+++ /dev/null
@@ -1,495 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * IMG PowerDown Controller (PDC)
- *
- * Copyright 2010-2013 Imagination Technologies Ltd.
- *
- * Exposes the syswake and PDC peripheral wake interrupts to the system.
- *
- */
-
-#include <linux/bitops.h>
-#include <linux/interrupt.h>
-#include <linux/irqdomain.h>
-#include <linux/io.h>
-#include <linux/kernel.h>
-#include <linux/of.h>
-#include <linux/platform_device.h>
-#include <linux/spinlock.h>
-
-/* PDC interrupt register numbers */
-
-#define PDC_IRQ_STATUS 0x310
-#define PDC_IRQ_ENABLE 0x314
-#define PDC_IRQ_CLEAR 0x318
-#define PDC_IRQ_ROUTE 0x31c
-#define PDC_SYS_WAKE_BASE 0x330
-#define PDC_SYS_WAKE_STRIDE 0x8
-#define PDC_SYS_WAKE_CONFIG_BASE 0x334
-#define PDC_SYS_WAKE_CONFIG_STRIDE 0x8
-
-/* PDC interrupt register field masks */
-
-#define PDC_IRQ_SYS3 0x08
-#define PDC_IRQ_SYS2 0x04
-#define PDC_IRQ_SYS1 0x02
-#define PDC_IRQ_SYS0 0x01
-#define PDC_IRQ_ROUTE_WU_EN_SYS3 0x08000000
-#define PDC_IRQ_ROUTE_WU_EN_SYS2 0x04000000
-#define PDC_IRQ_ROUTE_WU_EN_SYS1 0x02000000
-#define PDC_IRQ_ROUTE_WU_EN_SYS0 0x01000000
-#define PDC_IRQ_ROUTE_WU_EN_WD 0x00040000
-#define PDC_IRQ_ROUTE_WU_EN_IR 0x00020000
-#define PDC_IRQ_ROUTE_WU_EN_RTC 0x00010000
-#define PDC_IRQ_ROUTE_EXT_EN_SYS3 0x00000800
-#define PDC_IRQ_ROUTE_EXT_EN_SYS2 0x00000400
-#define PDC_IRQ_ROUTE_EXT_EN_SYS1 0x00000200
-#define PDC_IRQ_ROUTE_EXT_EN_SYS0 0x00000100
-#define PDC_IRQ_ROUTE_EXT_EN_WD 0x00000004
-#define PDC_IRQ_ROUTE_EXT_EN_IR 0x00000002
-#define PDC_IRQ_ROUTE_EXT_EN_RTC 0x00000001
-#define PDC_SYS_WAKE_RESET 0x00000010
-#define PDC_SYS_WAKE_INT_MODE 0x0000000e
-#define PDC_SYS_WAKE_INT_MODE_SHIFT 1
-#define PDC_SYS_WAKE_PIN_VAL 0x00000001
-
-/* PDC interrupt constants */
-
-#define PDC_SYS_WAKE_INT_LOW 0x0
-#define PDC_SYS_WAKE_INT_HIGH 0x1
-#define PDC_SYS_WAKE_INT_DOWN 0x2
-#define PDC_SYS_WAKE_INT_UP 0x3
-#define PDC_SYS_WAKE_INT_CHANGE 0x6
-#define PDC_SYS_WAKE_INT_NONE 0x4
-
-/**
- * struct pdc_intc_priv - private pdc interrupt data.
- * @nr_perips: Number of peripheral interrupt signals.
- * @nr_syswakes: Number of syswake signals.
- * @perip_irqs: List of peripheral IRQ numbers handled.
- * @syswake_irq: Shared PDC syswake IRQ number.
- * @domain: IRQ domain for PDC peripheral and syswake IRQs.
- * @pdc_base: Base of PDC registers.
- * @irq_route: Cached version of PDC_IRQ_ROUTE register.
- * @lock: Lock to protect the PDC syswake registers and the cached
- * values of those registers in this struct.
- */
-struct pdc_intc_priv {
- unsigned int nr_perips;
- unsigned int nr_syswakes;
- unsigned int *perip_irqs;
- unsigned int syswake_irq;
- struct irq_domain *domain;
- void __iomem *pdc_base;
-
- u32 irq_route;
- raw_spinlock_t lock;
-};
-
-static void pdc_write(struct pdc_intc_priv *priv, unsigned int reg_offs,
- unsigned int data)
-{
- iowrite32(data, priv->pdc_base + reg_offs);
-}
-
-static unsigned int pdc_read(struct pdc_intc_priv *priv,
- unsigned int reg_offs)
-{
- return ioread32(priv->pdc_base + reg_offs);
-}
-
-/* Generic IRQ callbacks */
-
-#define SYS0_HWIRQ 8
-
-static unsigned int hwirq_is_syswake(irq_hw_number_t hw)
-{
- return hw >= SYS0_HWIRQ;
-}
-
-static unsigned int hwirq_to_syswake(irq_hw_number_t hw)
-{
- return hw - SYS0_HWIRQ;
-}
-
-static irq_hw_number_t syswake_to_hwirq(unsigned int syswake)
-{
- return SYS0_HWIRQ + syswake;
-}
-
-static struct pdc_intc_priv *irqd_to_priv(struct irq_data *data)
-{
- return (struct pdc_intc_priv *)data->domain->host_data;
-}
-
-/*
- * perip_irq_mask() and perip_irq_unmask() use IRQ_ROUTE which also contains
- * wake bits, therefore we cannot use the generic irqchip mask callbacks as they
- * cache the mask.
- */
-
-static void perip_irq_mask(struct irq_data *data)
-{
- struct pdc_intc_priv *priv = irqd_to_priv(data);
-
- raw_spin_lock(&priv->lock);
- priv->irq_route &= ~data->mask;
- pdc_write(priv, PDC_IRQ_ROUTE, priv->irq_route);
- raw_spin_unlock(&priv->lock);
-}
-
-static void perip_irq_unmask(struct irq_data *data)
-{
- struct pdc_intc_priv *priv = irqd_to_priv(data);
-
- raw_spin_lock(&priv->lock);
- priv->irq_route |= data->mask;
- pdc_write(priv, PDC_IRQ_ROUTE, priv->irq_route);
- raw_spin_unlock(&priv->lock);
-}
-
-static int syswake_irq_set_type(struct irq_data *data, unsigned int flow_type)
-{
- struct pdc_intc_priv *priv = irqd_to_priv(data);
- unsigned int syswake = hwirq_to_syswake(data->hwirq);
- unsigned int irq_mode;
- unsigned int soc_sys_wake_regoff, soc_sys_wake;
-
- /* translate to syswake IRQ mode */
- switch (flow_type) {
- case IRQ_TYPE_EDGE_BOTH:
- irq_mode = PDC_SYS_WAKE_INT_CHANGE;
- break;
- case IRQ_TYPE_EDGE_RISING:
- irq_mode = PDC_SYS_WAKE_INT_UP;
- break;
- case IRQ_TYPE_EDGE_FALLING:
- irq_mode = PDC_SYS_WAKE_INT_DOWN;
- break;
- case IRQ_TYPE_LEVEL_HIGH:
- irq_mode = PDC_SYS_WAKE_INT_HIGH;
- break;
- case IRQ_TYPE_LEVEL_LOW:
- irq_mode = PDC_SYS_WAKE_INT_LOW;
- break;
- default:
- return -EINVAL;
- }
-
- raw_spin_lock(&priv->lock);
-
- /* set the IRQ mode */
- soc_sys_wake_regoff = PDC_SYS_WAKE_BASE + syswake*PDC_SYS_WAKE_STRIDE;
- soc_sys_wake = pdc_read(priv, soc_sys_wake_regoff);
- soc_sys_wake &= ~PDC_SYS_WAKE_INT_MODE;
- soc_sys_wake |= irq_mode << PDC_SYS_WAKE_INT_MODE_SHIFT;
- pdc_write(priv, soc_sys_wake_regoff, soc_sys_wake);
-
- /* and update the handler */
- irq_setup_alt_chip(data, flow_type);
-
- raw_spin_unlock(&priv->lock);
-
- return 0;
-}
-
-/* applies to both peripheral and syswake interrupts */
-static int pdc_irq_set_wake(struct irq_data *data, unsigned int on)
-{
- struct pdc_intc_priv *priv = irqd_to_priv(data);
- irq_hw_number_t hw = data->hwirq;
- unsigned int mask = (1 << 16) << hw;
- unsigned int dst_irq;
-
- raw_spin_lock(&priv->lock);
- if (on)
- priv->irq_route |= mask;
- else
- priv->irq_route &= ~mask;
- pdc_write(priv, PDC_IRQ_ROUTE, priv->irq_route);
- raw_spin_unlock(&priv->lock);
-
- /* control the destination IRQ wakeup too for standby mode */
- if (hwirq_is_syswake(hw))
- dst_irq = priv->syswake_irq;
- else
- dst_irq = priv->perip_irqs[hw];
- irq_set_irq_wake(dst_irq, on);
-
- return 0;
-}
-
-static void pdc_intc_perip_isr(struct irq_desc *desc)
-{
- unsigned int irq = irq_desc_get_irq(desc);
- struct pdc_intc_priv *priv;
- unsigned int i;
-
- priv = (struct pdc_intc_priv *)irq_desc_get_handler_data(desc);
-
- /* find the peripheral number */
- for (i = 0; i < priv->nr_perips; ++i)
- if (irq == priv->perip_irqs[i])
- goto found;
-
- /* should never get here */
- return;
-found:
-
- /* pass on the interrupt */
- generic_handle_domain_irq(priv->domain, i);
-}
-
-static void pdc_intc_syswake_isr(struct irq_desc *desc)
-{
- struct pdc_intc_priv *priv;
- unsigned int syswake;
- unsigned int status;
-
- priv = (struct pdc_intc_priv *)irq_desc_get_handler_data(desc);
-
- status = pdc_read(priv, PDC_IRQ_STATUS) &
- pdc_read(priv, PDC_IRQ_ENABLE);
- status &= (1 << priv->nr_syswakes) - 1;
-
- for (syswake = 0; status; status >>= 1, ++syswake) {
- /* Has this sys_wake triggered? */
- if (!(status & 1))
- continue;
-
- generic_handle_domain_irq(priv->domain, syswake_to_hwirq(syswake));
- }
-}
-
-static void pdc_intc_setup(struct pdc_intc_priv *priv)
-{
- int i;
- unsigned int soc_sys_wake_regoff;
- unsigned int soc_sys_wake;
-
- /*
- * Mask all syswake interrupts before routing, or we could receive an
- * interrupt before we're ready to handle it.
- */
- pdc_write(priv, PDC_IRQ_ENABLE, 0);
-
- /*
- * Enable routing of all syswakes
- * Disable all wake sources
- */
- priv->irq_route = ((PDC_IRQ_ROUTE_EXT_EN_SYS0 << priv->nr_syswakes) -
- PDC_IRQ_ROUTE_EXT_EN_SYS0);
- pdc_write(priv, PDC_IRQ_ROUTE, priv->irq_route);
-
- /* Initialise syswake IRQ */
- for (i = 0; i < priv->nr_syswakes; ++i) {
- /* set the IRQ mode to none */
- soc_sys_wake_regoff = PDC_SYS_WAKE_BASE + i*PDC_SYS_WAKE_STRIDE;
- soc_sys_wake = PDC_SYS_WAKE_INT_NONE
- << PDC_SYS_WAKE_INT_MODE_SHIFT;
- pdc_write(priv, soc_sys_wake_regoff, soc_sys_wake);
- }
-}
-
-static int pdc_intc_probe(struct platform_device *pdev)
-{
- struct pdc_intc_priv *priv;
- struct device_node *node = pdev->dev.of_node;
- struct resource *res_regs;
- struct irq_chip_generic *gc;
- unsigned int i;
- int irq, ret;
- u32 val;
-
- if (!node)
- return -ENOENT;
-
- /* Get registers */
- res_regs = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (res_regs == NULL) {
- dev_err(&pdev->dev, "cannot find registers resource\n");
- return -ENOENT;
- }
-
- /* Allocate driver data */
- priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
- if (!priv)
- return -ENOMEM;
- raw_spin_lock_init(&priv->lock);
- platform_set_drvdata(pdev, priv);
-
- /* Ioremap the registers */
- priv->pdc_base = devm_ioremap(&pdev->dev, res_regs->start,
- resource_size(res_regs));
- if (!priv->pdc_base)
- return -EIO;
-
- /* Get number of peripherals */
- ret = of_property_read_u32(node, "num-perips", &val);
- if (ret) {
- dev_err(&pdev->dev, "No num-perips node property found\n");
- return -EINVAL;
- }
- if (val > SYS0_HWIRQ) {
- dev_err(&pdev->dev, "num-perips (%u) out of range\n", val);
- return -EINVAL;
- }
- priv->nr_perips = val;
-
- /* Get number of syswakes */
- ret = of_property_read_u32(node, "num-syswakes", &val);
- if (ret) {
- dev_err(&pdev->dev, "No num-syswakes node property found\n");
- return -EINVAL;
- }
- if (val > SYS0_HWIRQ) {
- dev_err(&pdev->dev, "num-syswakes (%u) out of range\n", val);
- return -EINVAL;
- }
- priv->nr_syswakes = val;
-
- /* Get peripheral IRQ numbers */
- priv->perip_irqs = devm_kcalloc(&pdev->dev, 4, priv->nr_perips,
- GFP_KERNEL);
- if (!priv->perip_irqs)
- return -ENOMEM;
- for (i = 0; i < priv->nr_perips; ++i) {
- irq = platform_get_irq(pdev, 1 + i);
- if (irq < 0)
- return irq;
- priv->perip_irqs[i] = irq;
- }
- /* check if too many were provided */
- if (platform_get_irq(pdev, 1 + i) >= 0) {
- dev_err(&pdev->dev, "surplus perip IRQs detected\n");
- return -EINVAL;
- }
-
- /* Get syswake IRQ number */
- irq = platform_get_irq(pdev, 0);
- if (irq < 0)
- return irq;
- priv->syswake_irq = irq;
-
- /* Set up an IRQ domain */
- priv->domain = irq_domain_create_linear(dev_fwnode(&pdev->dev), 16, &irq_generic_chip_ops,
- priv);
- if (unlikely(!priv->domain)) {
- dev_err(&pdev->dev, "cannot add IRQ domain\n");
- return -ENOMEM;
- }
- priv->domain->flags |= IRQ_DOMAIN_FLAG_DESTROY_GC;
-
- /*
- * Set up 2 generic irq chips with 2 chip types.
- * The first one for peripheral irqs (only 1 chip type used)
- * The second one for syswake irqs (edge and level chip types)
- */
- ret = irq_alloc_domain_generic_chips(priv->domain, 8, 2, "pdc",
- handle_level_irq, 0, 0,
- IRQ_GC_INIT_NESTED_LOCK);
- if (ret)
- goto err_generic;
-
- /* peripheral interrupt chip */
-
- gc = irq_get_domain_generic_chip(priv->domain, 0);
- gc->unused = ~(BIT(priv->nr_perips) - 1);
- gc->reg_base = priv->pdc_base;
- /*
- * IRQ_ROUTE contains wake bits, so we can't use the generic versions as
- * they cache the mask
- */
- gc->chip_types[0].regs.mask = PDC_IRQ_ROUTE;
- gc->chip_types[0].chip.irq_mask = perip_irq_mask;
- gc->chip_types[0].chip.irq_unmask = perip_irq_unmask;
- gc->chip_types[0].chip.irq_set_wake = pdc_irq_set_wake;
-
- /* syswake interrupt chip */
-
- gc = irq_get_domain_generic_chip(priv->domain, 8);
- gc->unused = ~(BIT(priv->nr_syswakes) - 1);
- gc->reg_base = priv->pdc_base;
-
- /* edge interrupts */
- gc->chip_types[0].type = IRQ_TYPE_EDGE_BOTH;
- gc->chip_types[0].handler = handle_edge_irq;
- gc->chip_types[0].regs.ack = PDC_IRQ_CLEAR;
- gc->chip_types[0].regs.mask = PDC_IRQ_ENABLE;
- gc->chip_types[0].chip.irq_ack = irq_gc_ack_set_bit;
- gc->chip_types[0].chip.irq_mask = irq_gc_mask_clr_bit;
- gc->chip_types[0].chip.irq_unmask = irq_gc_mask_set_bit;
- gc->chip_types[0].chip.irq_set_type = syswake_irq_set_type;
- gc->chip_types[0].chip.irq_set_wake = pdc_irq_set_wake;
- /* for standby we pass on to the shared syswake IRQ */
- gc->chip_types[0].chip.flags = IRQCHIP_MASK_ON_SUSPEND;
-
- /* level interrupts */
- gc->chip_types[1].type = IRQ_TYPE_LEVEL_MASK;
- gc->chip_types[1].handler = handle_level_irq;
- gc->chip_types[1].regs.ack = PDC_IRQ_CLEAR;
- gc->chip_types[1].regs.mask = PDC_IRQ_ENABLE;
- gc->chip_types[1].chip.irq_ack = irq_gc_ack_set_bit;
- gc->chip_types[1].chip.irq_mask = irq_gc_mask_clr_bit;
- gc->chip_types[1].chip.irq_unmask = irq_gc_mask_set_bit;
- gc->chip_types[1].chip.irq_set_type = syswake_irq_set_type;
- gc->chip_types[1].chip.irq_set_wake = pdc_irq_set_wake;
- /* for standby we pass on to the shared syswake IRQ */
- gc->chip_types[1].chip.flags = IRQCHIP_MASK_ON_SUSPEND;
-
- /* Set up the hardware to enable interrupt routing */
- pdc_intc_setup(priv);
-
- /* Setup chained handlers for the peripheral IRQs */
- for (i = 0; i < priv->nr_perips; ++i) {
- irq = priv->perip_irqs[i];
- irq_set_chained_handler_and_data(irq, pdc_intc_perip_isr,
- priv);
- }
-
- /* Setup chained handler for the syswake IRQ */
- irq_set_chained_handler_and_data(priv->syswake_irq,
- pdc_intc_syswake_isr, priv);
-
- dev_info(&pdev->dev,
- "PDC IRQ controller initialised (%u perip IRQs, %u syswake IRQs)\n",
- priv->nr_perips,
- priv->nr_syswakes);
-
- return 0;
-err_generic:
- irq_domain_remove(priv->domain);
- return ret;
-}
-
-static void pdc_intc_remove(struct platform_device *pdev)
-{
- struct pdc_intc_priv *priv = platform_get_drvdata(pdev);
-
- for (unsigned int i = 0; i < priv->nr_perips; ++i)
- irq_set_chained_handler_and_data(priv->perip_irqs[i], NULL, NULL);
-
- irq_set_chained_handler_and_data(priv->syswake_irq, NULL, NULL);
-
- irq_domain_remove(priv->domain);
-}
-
-static const struct of_device_id pdc_intc_match[] = {
- { .compatible = "img,pdc-intc" },
- {}
-};
-
-static struct platform_driver pdc_intc_driver = {
- .driver = {
- .name = "pdc-intc",
- .of_match_table = pdc_intc_match,
- },
- .probe = pdc_intc_probe,
- .remove = pdc_intc_remove,
-};
-
-static int __init pdc_intc_init(void)
-{
- return platform_driver_register(&pdc_intc_driver);
-}
-core_initcall(pdc_intc_init);
diff --git a/drivers/irqchip/irq-qcom-mpm.c b/drivers/irqchip/irq-qcom-mpm.c
index 181320528a47..8c575d7ce8a1 100644
--- a/drivers/irqchip/irq-qcom-mpm.c
+++ b/drivers/irqchip/irq-qcom-mpm.c
@@ -464,10 +464,8 @@ static int qcom_mpm_probe(struct platform_device *pdev, struct device_node *pare
ret = devm_request_irq(dev, irq, qcom_mpm_handler, IRQF_NO_SUSPEND,
"qcom_mpm", priv);
- if (ret) {
- dev_err(dev, "failed to request irq: %d\n", ret);
+ if (ret)
goto remove_domain;
- }
return 0;
diff --git a/drivers/irqchip/irq-realtek-rtl.c b/drivers/irqchip/irq-realtek-rtl.c
index 2ae3be7fa633..c8becb458da2 100644
--- a/drivers/irqchip/irq-realtek-rtl.c
+++ b/drivers/irqchip/irq-realtek-rtl.c
@@ -25,6 +25,15 @@
#define REG(cpu, x) (realtek_ictl_base[cpu] + x)
+struct realtek_ictl_output {
+ struct fwnode_handle *fwnode;
+ struct irq_domain *domain;
+ unsigned int parent_irq;
+ unsigned int parent_hwirq;
+ unsigned int index;
+ u32 mask;
+};
+
static DEFINE_RAW_SPINLOCK(irq_lock);
static void __iomem *realtek_ictl_base[NR_CPUS];
@@ -105,53 +114,141 @@ static struct irq_chip realtek_ictl_irq = {
.irq_set_affinity = realtek_ictl_irq_affinity,
};
-static int intc_map(struct irq_domain *d, unsigned int irq, irq_hw_number_t hw)
+static int intc_map(struct irq_domain *d, unsigned int irq, irq_hw_number_t hw_irq)
{
+ struct realtek_ictl_output *output = d->host_data;
unsigned int cpu;
irq_set_chip_and_handler(irq, &realtek_ictl_irq, handle_level_irq);
guard(raw_spinlock_irqsave)(&irq_lock);
+ output->mask |= BIT(hw_irq);
for_each_present_cpu(cpu)
- write_irr(cpu, hw, 1);
+ write_irr(cpu, hw_irq, output->parent_hwirq - 1);
return 0;
}
+static int intc_select(struct irq_domain *d, struct irq_fwspec *fwspec,
+ enum irq_domain_bus_token bus_token)
+{
+ struct realtek_ictl_output *output = d->host_data;
+ unsigned int index = 0;
+
+ if (fwspec->fwnode != output->fwnode)
+ return false;
+
+ if (fwspec->param_count == 2)
+ index = fwspec->param[1];
+
+ return index == output->index;
+}
+
static const struct irq_domain_ops irq_domain_ops = {
- .map = intc_map,
- .xlate = irq_domain_xlate_onecell,
+ .map = intc_map,
+ .select = intc_select,
+ .xlate = irq_domain_xlate_onecell,
};
static void realtek_irq_dispatch(struct irq_desc *desc)
{
+ struct realtek_ictl_output *output = irq_desc_get_handler_data(desc);
struct irq_chip *chip = irq_desc_get_chip(desc);
unsigned int cpu = smp_processor_id();
- struct irq_domain *domain;
unsigned long pending;
- unsigned int soc_int;
+ unsigned int hw_irq;
chained_irq_enter(chip, desc);
- pending = readl(REG(cpu, RTL_ICTL_GIMR)) & readl(REG(cpu, RTL_ICTL_GISR));
+ pending = readl(REG(cpu, RTL_ICTL_GIMR)) & readl(REG(cpu, RTL_ICTL_GISR)) & output->mask;
if (unlikely(!pending)) {
spurious_interrupt();
goto out;
}
- domain = irq_desc_get_handler_data(desc);
- for_each_set_bit(soc_int, &pending, RTL_ICTL_NUM_INPUTS)
- generic_handle_domain_irq(domain, soc_int);
+ for_each_set_bit(hw_irq, &pending, RTL_ICTL_NUM_INPUTS)
+ generic_handle_domain_irq(output->domain, hw_irq);
out:
chained_irq_exit(chip, desc);
}
-static int __init realtek_rtl_of_init(struct device_node *node, struct device_node *parent)
+static int __init realtek_setup_parents(struct device_node *node)
{
+ int p, cnt, err, parent_irq, num_parents = of_irq_count(node);
+ struct realtek_ictl_output *output;
+ struct irq_data *parent_data;
struct of_phandle_args oirq;
struct irq_domain *domain;
- int cpu, parent_irq;
+
+ cnt = max(1, num_parents);
+ output = kcalloc(cnt, sizeof(*output), GFP_KERNEL);
+ if (!output)
+ return -ENOMEM;
+
+ for (p = 0; p < cnt; p++) {
+ if (WARN_ON(!num_parents)) {
+ /*
+ * If DT contains no parent interrupts, assume MIPS IRQ 2 (HW0) is
+ * connected to the first output. This is the case for all known hardware.
+ */
+ oirq.np = of_find_compatible_node(NULL, NULL,
+ "mti,cpu-interrupt-controller");
+ if (!oirq.np) {
+ err = -EINVAL;
+ goto err_out;
+ }
+
+ oirq.args_count = 1;
+ oirq.args[0] = 2;
+ parent_irq = irq_create_of_mapping(&oirq);
+ of_node_put(oirq.np);
+ } else {
+ parent_irq = of_irq_get(node, p);
+ }
+
+ if (parent_irq <= 0) {
+ err = parent_irq ? parent_irq : -ENODEV;
+ goto err_out;
+ }
+
+ parent_data = irq_get_irq_data(parent_irq);
+ if (!parent_data) {
+ err = -EINVAL;
+ goto err_out;
+ }
+
+ domain = irq_domain_create_linear(of_fwnode_handle(node), RTL_ICTL_NUM_INPUTS,
+ &irq_domain_ops, &output[p]);
+ if (!domain) {
+ err = -ENOMEM;
+ goto err_out;
+ }
+
+ output[p].domain = domain;
+ output[p].fwnode = of_fwnode_handle(node);
+ output[p].index = p;
+ output[p].parent_irq = parent_irq;
+ output[p].parent_hwirq = irqd_to_hwirq(parent_data);
+ irq_set_chained_handler_and_data(parent_irq, realtek_irq_dispatch, &output[p]);
+ }
+
+ return 0;
+
+err_out:
+ while (p--) {
+ irq_set_chained_handler_and_data(output[p].parent_irq, NULL, NULL);
+ irq_domain_remove(output[p].domain);
+ }
+
+ kfree(output);
+
+ return err;
+}
+
+static int __init realtek_rtl_of_init(struct device_node *node, struct device_node *parent)
+{
+ unsigned int cpu;
for_each_present_cpu(cpu) {
realtek_ictl_base[cpu] = of_iomap(node, cpu);
@@ -165,36 +262,7 @@ static int __init realtek_rtl_of_init(struct device_node *node, struct device_no
}
}
- if (WARN_ON(!of_irq_count(node))) {
- /*
- * If DT contains no parent interrupts, assume MIPS CPU IRQ 2
- * (HW0) is connected to the first output. This is the case for
- * all known hardware anyway. "interrupt-map" is deprecated, so
- * don't bother trying to parse that.
- */
- oirq.np = of_find_compatible_node(NULL, NULL, "mti,cpu-interrupt-controller");
- oirq.args_count = 1;
- oirq.args[0] = 2;
-
- parent_irq = irq_create_of_mapping(&oirq);
-
- of_node_put(oirq.np);
- } else {
- parent_irq = of_irq_get(node, 0);
- }
-
- if (parent_irq < 0)
- return parent_irq;
- else if (!parent_irq)
- return -ENODEV;
-
- domain = irq_domain_create_linear(of_fwnode_handle(node), RTL_ICTL_NUM_INPUTS, &irq_domain_ops, NULL);
- if (!domain)
- return -ENOMEM;
-
- irq_set_chained_handler_and_data(parent_irq, realtek_irq_dispatch, domain);
-
- return 0;
+ return realtek_setup_parents(node);
}
IRQCHIP_DECLARE(realtek_rtl_intc, "realtek,rtl-intc", realtek_rtl_of_init);
diff --git a/drivers/irqchip/irq-renesas-irqc.c b/drivers/irqchip/irq-renesas-irqc.c
index a20a6471b0e4..1ff3535a4617 100644
--- a/drivers/irqchip/irq-renesas-irqc.c
+++ b/drivers/irqchip/irq-renesas-irqc.c
@@ -176,6 +176,7 @@ static int irqc_probe(struct platform_device *pdev)
goto err_runtime_pm_disable;
}
+ p->irq_domain->flags |= IRQ_DOMAIN_FLAG_DESTROY_GC;
ret = irq_alloc_domain_generic_chips(p->irq_domain, p->number_of_irqs,
1, "irqc", handle_level_irq,
0, 0, IRQ_GC_INIT_NESTED_LOCK);
diff --git a/drivers/irqchip/irq-renesas-rzt2h.c b/drivers/irqchip/irq-renesas-rzt2h.c
index e06264add3cc..9086fc3a2043 100644
--- a/drivers/irqchip/irq-renesas-rzt2h.c
+++ b/drivers/irqchip/irq-renesas-rzt2h.c
@@ -398,7 +398,7 @@ static int rzt2h_icu_request_irqs(struct platform_device *pdev, struct irq_domai
ret = devm_request_irq(dev, virq, handler, 0, dev_name(dev),
data ?: (void *)(uintptr_t)offset);
if (ret)
- return dev_err_probe(dev, ret, "Failed to request IRQ %u\n", offset);
+ return ret;
}
return 0;
diff --git a/drivers/irqchip/irq-renesas-rzv2h.c b/drivers/irqchip/irq-renesas-rzv2h.c
index 971ac83eee90..1a4703a353e9 100644
--- a/drivers/irqchip/irq-renesas-rzv2h.c
+++ b/drivers/irqchip/irq-renesas-rzv2h.c
@@ -816,7 +816,7 @@ static int rzv2h_icu_setup_irqs(struct platform_device *pdev, struct irq_domain
ret = devm_request_irq(dev, virq, rzv2h_icu_swint_irq, 0, dev_name(dev),
(void *)(uintptr_t)i);
if (ret)
- return dev_err_probe(dev, ret, "Failed to request int-ca55-%u IRQ\n", i);
+ return ret;
}
/* Unmask and clear all IP/CA55 error interrupts */
@@ -844,7 +844,7 @@ static int rzv2h_icu_setup_irqs(struct platform_device *pdev, struct irq_domain
ret = devm_request_irq(dev, virq, rzv2h_icu_error_irq, 0, dev_name(dev), rzv2h_icu_data);
if (ret)
- return dev_err_probe(dev, ret, "Failed to request icu-error-ca55 IRQ\n");
+ return ret;
return 0;
}
diff --git a/drivers/irqchip/qcom-pdc.c b/drivers/irqchip/qcom-pdc.c
index 2014dbb0bc43..ce6d80c7f17a 100644
--- a/drivers/irqchip/qcom-pdc.c
+++ b/drivers/irqchip/qcom-pdc.c
@@ -20,22 +20,19 @@
#include <linux/spinlock.h>
#include <linux/slab.h>
#include <linux/types.h>
+#include <linux/firmware/qcom/qcom_scm.h>
-#define PDC_MAX_GPIO_IRQS 256
-#define PDC_DRV_SIZE 0x10000
-
-/* Valid only on HW version < 3.2 */
-#define IRQ_ENABLE_BANK 0x10
-#define IRQ_ENABLE_BANK_MAX (IRQ_ENABLE_BANK + BITS_TO_BYTES(PDC_MAX_GPIO_IRQS))
+#define PDC_MAX_IRQS 256
+#define IRQ_ENABLE_BANK_MAX BITS_TO_BYTES(PDC_MAX_IRQS)
#define IRQ_ENABLE_BANK_INDEX_MASK GENMASK(31, 5)
#define IRQ_ENABLE_BANK_BIT_MASK GENMASK(4, 0)
-#define IRQ_i_CFG 0x110
-
-/* Valid only on HW version >= 3.2 */
-#define IRQ_i_CFG_IRQ_ENABLE 3
-#define IRQ_i_CFG_TYPE_MASK GENMASK(2, 0)
+/* Secure DRV register to configure the PDC mode via qcom_scm_io_writel() */
+#define PDC_GPIO_INT_CTL_ENABLE 0xb2045e8
+#define PDC_PASS_THROUGH_MODE 0x0
+#define PDC_SECONDARY_MODE 0x1
+#define PDC_DRV_SIZE 0x10000
#define PDC_VERSION_REG 0x1000
#define PDC_VERSION_MAJOR GENMASK(23, 16)
#define PDC_VERSION_MINOR GENMASK(15, 8)
@@ -46,22 +43,149 @@
/* Notable PDC versions */
#define PDC_VERSION_3_2 PDC_VERSION(3, 2, 0)
+#define PDC_VERSION_3_0 PDC_VERSION(3, 0, 0)
+#define PDC_VERSION_2_7 PDC_VERSION(2, 7, 0)
+
+/*
+ * PDC Hardware registers layout per version:
+ *
+ * IRQ_ENABLE_BANK[b], b = 0....BITS_TO_BYTES(PDC_MAX_IRQS)
+ * IRQ_CFG[n], n = 0....PDC_MAX_IRQS
+ *
+ * +---------------------------------------------------------------+
+ * | v2.7 | v3.0 | v3.2 |
+ * |---------------------------------------------------------------|
+ * | BASE | BASE | BASE |
+ * |---------------------------------------------------------------|
+ * | |
+ * | IRQ_ENABLE_BANK | IRQ_ENABLE_BANK | NA |
+ * |---------------------------------------------------------------|
+ * | IRQ_CFG | IRQ_CFG | IRQ_CFG |
+ * | | | |
+ * | | | [31:6] Unused |
+ * | | [31:5] Unused | [5] GPIO_STATUS |
+ * | | [4] GPIO_STATUS| [4] GPIO_MASK |
+ * | [31:3] Unused | [3] GPIO_MASK | [3] IRQ_ENABLE |
+ * | [0:2] Type | [0:2] Type | [0:2] Type |
+ * |---------------------------------------------------------------|
+ * | IRQ_PARAM | IRQ_PARAM | IRQ_PARAM |
+ * | | |
+ * | [15:8] NUM_GPIO | [15:8] NUM_GPIO | [15:8] NUM_GPIO |
+ * | [7:0] NUM_SPI | [7:0] NUM_SPI | [7:0] NUM_SPI |
+ * +---------------------------------------------------------------+
+ */
+
+/**
+ * struct pdc_regs: PDC registers location
+ *
+ * @irq_en_reg: IRQ_ENABLE_BANK register location
+ * @irq_cfg_reg: IRQ_CFG register location
+ * @irq_param_reg: IRQ_PARAM register location
+ */
+struct pdc_regs {
+ u32 irq_en_reg;
+ u32 irq_cfg_reg;
+ u32 irq_param_reg;
+};
+
+/**
+ * struct pdc_irq_cfg: bit fields for PDC IRQ_CFG register
+ *
+ * @gpio_irq_sts: bit number for GPIO_STATUS field
+ * @gpio_irq_mask: bit number for GPIO_MASK field
+ * @irq_enable: bit number for IRQ_ENABLE field
+ * @irq_type: GENMASK for IRQ_TYPE field
+ */
+struct pdc_irq_cfg {
+ u32 gpio_irq_sts;
+ u32 gpio_irq_mask;
+ u32 irq_enable;
+ u32 irq_type;
+};
+
+/**
+ * struct pdc_desc: PDC driver state
+ *
+ * @base: PDC base register for DRV2 / HLOS
+ * @prev_base: PDC DRV1 base, applicable only for x1e RTL bug.
+ * @version: PDC version
+ * @num_spis: Total number of direct SPI interrupts
+ * @region: PDC interrupt continuous range
+ * @region_cnt: Total PDC ranges
+ * @mode: PDC_PASS_THROUGH_MODE or PDC_SECONDARY_MODE
+ * @x1e_quirk: x1e H/W Bug handling
+ * @lock: lock for IRQ_ENABLE_BANK protection
+ * @regs: PDC regs (IRQ_ENABLE_BANK and IRQ_CFG)
+ * @cfg_fields: Fields of IRQ_CFG reg
+ * @enable_intr: pointer to enable function based on PDC version
+ * @unmask_gpio: pointer to GPIO irq unmask function
+ * @clear_gpio: pointer to GPIO irq clear function
+ */
+struct pdc_desc {
+ void __iomem *base;
+ void __iomem *prev_base;
+ u32 version;
+ u32 num_spis;
+
+ struct pdc_pin_region *region;
+ int region_cnt;
+
+ bool x1e_quirk;
+
+ u8 mode;
+ raw_spinlock_t lock;
+
+ const struct pdc_regs *regs;
+ const struct pdc_irq_cfg *cfg_fields;
+
+ void (*enable_intr)(int pin_out, bool on);
+ void (*unmask_gpio)(int pin_out, bool on);
+ void (*clear_gpio)(int pin_out);
+};
+
+static const struct pdc_regs pdc_v3_2 = {
+ .irq_cfg_reg = 0x110,
+ .irq_param_reg = 0x100c,
+};
+
+static const struct pdc_irq_cfg pdc_cfg_v3_2 = {
+ .gpio_irq_sts = 5,
+ .gpio_irq_mask = 4,
+ .irq_enable = 3,
+ .irq_type = GENMASK(2, 0),
+};
+
+static const struct pdc_regs pdc_v3_0 = {
+ .irq_en_reg = 0x10,
+ .irq_cfg_reg = 0x110,
+ .irq_param_reg = 0x100c,
+};
+
+static const struct pdc_irq_cfg pdc_cfg_v3_0 = {
+ .gpio_irq_sts = 4,
+ .gpio_irq_mask = 3,
+ .irq_type = GENMASK(2, 0),
+};
+
+static const struct pdc_regs pdc_v2_7 = {
+ .irq_en_reg = 0x10,
+ .irq_cfg_reg = 0x110,
+ .irq_param_reg = 0x100c,
+};
+
+static const struct pdc_irq_cfg pdc_cfg_v2_7 = {
+ .irq_type = GENMASK(2, 0),
+};
struct pdc_pin_region {
- u32 pin_base;
- u32 parent_base;
- u32 cnt;
+ u32 pin_base;
+ u32 parent_base;
+ u32 cnt;
};
#define pin_to_hwirq(r, p) ((r)->parent_base + (p) - (r)->pin_base)
-static DEFINE_RAW_SPINLOCK(pdc_lock);
-static void __iomem *pdc_base;
-static void __iomem *pdc_prev_base;
-static struct pdc_pin_region *pdc_region;
-static int pdc_region_cnt;
-static unsigned int pdc_version;
-static bool pdc_x1e_quirk;
+static struct pdc_desc *pdc;
static void pdc_base_reg_write(void __iomem *base, int reg, u32 i, u32 val)
{
@@ -70,12 +194,21 @@ static void pdc_base_reg_write(void __iomem *base, int reg, u32 i, u32 val)
static void pdc_reg_write(int reg, u32 i, u32 val)
{
- pdc_base_reg_write(pdc_base, reg, i, val);
+ pdc_base_reg_write(pdc->base, reg, i, val);
}
static u32 pdc_reg_read(int reg, u32 i)
{
- return readl_relaxed(pdc_base + reg + i * sizeof(u32));
+ return readl_relaxed(pdc->base + reg + i * sizeof(u32));
+}
+
+static inline bool pdc_pin_is_gpio(int pin_out)
+{
+ /*
+ * PDC allocates direct SPIs at the beginning and
+ * all GPIOs as SPIs are allocated after direct SPIs.
+ */
+ return pin_out >= pdc->num_spis;
}
static void pdc_x1e_irq_enable_write(u32 bank, u32 enable)
@@ -86,24 +219,24 @@ static void pdc_x1e_irq_enable_write(u32 bank, u32 enable)
switch (bank) {
case 0 ... 1:
/* Use previous DRV (client) region and shift to bank 3-4 */
- base = pdc_prev_base;
+ base = pdc->prev_base;
bank += 3;
break;
case 2 ... 4:
/* Use our own region and shift to bank 0-2 */
- base = pdc_base;
+ base = pdc->base;
bank -= 2;
break;
case 5:
/* No fixup required for bank 5 */
- base = pdc_base;
+ base = pdc->base;
break;
default:
WARN_ON(1);
return;
}
- pdc_base_reg_write(base, IRQ_ENABLE_BANK, bank, enable);
+ pdc_base_reg_write(base, pdc->regs->irq_en_reg, bank, enable);
}
static void pdc_enable_intr_bank(int pin_out, bool on)
@@ -114,52 +247,97 @@ static void pdc_enable_intr_bank(int pin_out, bool on)
index = FIELD_GET(IRQ_ENABLE_BANK_INDEX_MASK, pin_out);
mask = FIELD_GET(IRQ_ENABLE_BANK_BIT_MASK, pin_out);
- enable = pdc_reg_read(IRQ_ENABLE_BANK, index);
+ guard(raw_spinlock_irqsave)(&pdc->lock);
+
+ enable = pdc_reg_read(pdc->regs->irq_en_reg, index);
__assign_bit(mask, &enable, on);
- if (pdc_x1e_quirk)
+ if (pdc->x1e_quirk)
pdc_x1e_irq_enable_write(index, enable);
else
- pdc_reg_write(IRQ_ENABLE_BANK, index, enable);
+ pdc_reg_write(pdc->regs->irq_en_reg, index, enable);
}
-static void pdc_enable_intr_cfg(int pin_out, bool on)
+static void pdc_clear_gpio_cfg(int pin_out)
{
- unsigned long enable = pdc_reg_read(IRQ_i_CFG, pin_out);
+ unsigned long gpio_sts;
- __assign_bit(IRQ_i_CFG_IRQ_ENABLE, &enable, on);
- pdc_reg_write(IRQ_i_CFG, pin_out, enable);
+ gpio_sts = pdc_reg_read(pdc->regs->irq_cfg_reg, pin_out);
+ __clear_bit(pdc->cfg_fields->gpio_irq_sts, &gpio_sts);
+ pdc_reg_write(pdc->regs->irq_cfg_reg, pin_out, gpio_sts);
}
-static void __pdc_enable_intr(int pin_out, bool on)
+static void pdc_unmask_gpio_cfg(int pin_out, bool unmask)
{
- if (pdc_version < PDC_VERSION_3_2)
- pdc_enable_intr_bank(pin_out, on);
- else
- pdc_enable_intr_cfg(pin_out, on);
+ unsigned long gpio_mask;
+
+ gpio_mask = pdc_reg_read(pdc->regs->irq_cfg_reg, pin_out);
+ __assign_bit(pdc->cfg_fields->gpio_irq_mask, &gpio_mask, !unmask);
+ pdc_reg_write(pdc->regs->irq_cfg_reg, pin_out, gpio_mask);
}
-static void pdc_enable_intr(struct irq_data *d, bool on)
+static void pdc_enable_intr_cfg(int pin_out, bool on)
{
- unsigned long flags;
+ unsigned long enable = pdc_reg_read(pdc->regs->irq_cfg_reg, pin_out);
- raw_spin_lock_irqsave(&pdc_lock, flags);
- __pdc_enable_intr(d->hwirq, on);
- raw_spin_unlock_irqrestore(&pdc_lock, flags);
+ __assign_bit(pdc->cfg_fields->irq_enable, &enable, on);
+ pdc_reg_write(pdc->regs->irq_cfg_reg, pin_out, enable);
+}
+
+static void qcom_pdc_gic_secondary_disable(struct irq_data *d)
+{
+ pdc->enable_intr(d->hwirq, false);
+ pdc->unmask_gpio(d->hwirq, false);
+ irq_chip_disable_parent(d);
}
static void qcom_pdc_gic_disable(struct irq_data *d)
{
- pdc_enable_intr(d, false);
+ pdc->enable_intr(d->hwirq, false);
irq_chip_disable_parent(d);
}
static void qcom_pdc_gic_enable(struct irq_data *d)
{
- pdc_enable_intr(d, true);
+ pdc->enable_intr(d->hwirq, true);
+ irq_chip_enable_parent(d);
+}
+
+static void qcom_pdc_gic_secondary_enable(struct irq_data *d)
+{
+ pdc->enable_intr(d->hwirq, true);
+ pdc->unmask_gpio(d->hwirq, true);
irq_chip_enable_parent(d);
}
+static void qcom_pdc_secondary_ack(struct irq_data *d)
+{
+ if (!irqd_is_level_type(d))
+ pdc->clear_gpio(d->hwirq);
+}
+
+static void qcom_pdc_gic_secondary_eoi(struct irq_data *d)
+{
+ if (irqd_is_level_type(d))
+ pdc->clear_gpio(d->hwirq);
+
+ irq_chip_eoi_parent(d);
+}
+
+static void qcom_pdc_secondary_mask(struct irq_data *d)
+{
+ pdc->enable_intr(d->hwirq, false);
+ pdc->unmask_gpio(d->hwirq, false);
+ irq_chip_mask_parent(d);
+}
+
+static void qcom_pdc_secondary_unmask(struct irq_data *d)
+{
+ pdc->enable_intr(d->hwirq, true);
+ pdc->unmask_gpio(d->hwirq, true);
+ irq_chip_unmask_parent(d);
+}
+
/*
* GIC does not handle falling edge or active low. To allow falling edge and
* active low interrupts to be handled at GIC, PDC has an inverter that inverts
@@ -186,18 +364,18 @@ enum pdc_irq_config_bits {
/**
* qcom_pdc_gic_set_type: Configure PDC for the interrupt
*
- * @d: the interrupt data
+ * @d: the interrupt data
* @type: the interrupt type
*
- * If @type is edge triggered, forward that as Rising edge as PDC
- * takes care of converting falling edge to rising edge signal
+ * If @type is edge triggered, forward that as rising edge as PDC
+ * takes care of converting all edge types to rising edge signal
* If @type is level, then forward that as level high as PDC
- * takes care of converting falling edge to rising edge signal
+ * takes care of converting all level types to level high signal
*/
static int qcom_pdc_gic_set_type(struct irq_data *d, unsigned int type)
{
- enum pdc_irq_config_bits pdc_type;
enum pdc_irq_config_bits old_pdc_type;
+ enum pdc_irq_config_bits pdc_type;
int ret;
switch (type) {
@@ -224,9 +402,75 @@ static int qcom_pdc_gic_set_type(struct irq_data *d, unsigned int type)
return -EINVAL;
}
- old_pdc_type = pdc_reg_read(IRQ_i_CFG, d->hwirq);
- pdc_type |= (old_pdc_type & ~IRQ_i_CFG_TYPE_MASK);
- pdc_reg_write(IRQ_i_CFG, d->hwirq, pdc_type);
+ old_pdc_type = pdc_reg_read(pdc->regs->irq_cfg_reg, d->hwirq);
+ pdc_type |= (old_pdc_type & ~pdc->cfg_fields->irq_type);
+ pdc_reg_write(pdc->regs->irq_cfg_reg, d->hwirq, pdc_type);
+
+ ret = irq_chip_set_type_parent(d, type);
+ if (ret)
+ return ret;
+
+ /*
+ * When we change types the PDC can give a phantom interrupt.
+ * Clear it. Specifically the phantom shows up when reconfiguring
+ * polarity of interrupt without changing the state of the signal
+ * but let's be consistent and clear it always.
+ *
+ * Doing this works because we have IRQCHIP_SET_TYPE_MASKED so the
+ * interrupt will be cleared before the rest of the system sees it.
+ */
+ if (old_pdc_type != pdc_type)
+ irq_chip_set_parent_state(d, IRQCHIP_STATE_PENDING, false);
+
+ return 0;
+}
+
+/**
+ * qcom_pdc_gic_secondary_set_type: Configure PDC for the interrupt in secondary mode
+ *
+ * @d: the interrupt data
+ * @type: the interrupt type
+ *
+ * All @type are forwarded as level high type to parent GIC
+ */
+static int qcom_pdc_gic_secondary_set_type(struct irq_data *d, unsigned int type)
+{
+ enum pdc_irq_config_bits old_pdc_type;
+ enum pdc_irq_config_bits pdc_type;
+ int ret;
+
+ switch (type) {
+ case IRQ_TYPE_EDGE_RISING:
+ pdc_type = PDC_EDGE_RISING;
+ break;
+ case IRQ_TYPE_EDGE_FALLING:
+ pdc_type = PDC_EDGE_FALLING;
+ break;
+ case IRQ_TYPE_EDGE_BOTH:
+ pdc_type = PDC_EDGE_DUAL;
+ break;
+ case IRQ_TYPE_LEVEL_HIGH:
+ pdc_type = PDC_LEVEL_HIGH;
+ break;
+ case IRQ_TYPE_LEVEL_LOW:
+ pdc_type = PDC_LEVEL_LOW;
+ break;
+ default:
+ WARN_ON(1);
+ return -EINVAL;
+ }
+
+ old_pdc_type = pdc_reg_read(pdc->regs->irq_cfg_reg, d->hwirq);
+ pdc_type |= (old_pdc_type & ~pdc->cfg_fields->irq_type);
+ pdc_reg_write(pdc->regs->irq_cfg_reg, d->hwirq, pdc_type);
+
+ /*
+ * PDC forwards GPIOs as level high to GIC in secondary
+ * mode. Update the type and clear any previously latched
+ * phantom interrupt at PDC.
+ */
+ type = IRQ_TYPE_LEVEL_HIGH;
+ pdc->clear_gpio(d->hwirq);
ret = irq_chip_set_type_parent(d, type);
if (ret)
@@ -266,14 +510,32 @@ static struct irq_chip qcom_pdc_gic_chip = {
.irq_set_affinity = irq_chip_set_affinity_parent,
};
+static struct irq_chip qcom_pdc_gic_secondary_chip = {
+ .name = "PDC",
+ .irq_ack = qcom_pdc_secondary_ack,
+ .irq_eoi = qcom_pdc_gic_secondary_eoi,
+ .irq_mask = qcom_pdc_secondary_mask,
+ .irq_unmask = qcom_pdc_secondary_unmask,
+ .irq_disable = qcom_pdc_gic_secondary_disable,
+ .irq_enable = qcom_pdc_gic_secondary_enable,
+ .irq_get_irqchip_state = irq_chip_get_parent_state,
+ .irq_set_irqchip_state = irq_chip_set_parent_state,
+ .irq_retrigger = irq_chip_retrigger_hierarchy,
+ .irq_set_type = qcom_pdc_gic_secondary_set_type,
+ .flags = IRQCHIP_MASK_ON_SUSPEND |
+ IRQCHIP_SET_TYPE_MASKED |
+ IRQCHIP_SKIP_SET_WAKE |
+ IRQCHIP_ENABLE_WAKEUP_ON_SUSPEND,
+ .irq_set_vcpu_affinity = irq_chip_set_vcpu_affinity_parent,
+ .irq_set_affinity = irq_chip_set_affinity_parent,
+};
+
static struct pdc_pin_region *get_pin_region(int pin)
{
- int i;
-
- for (i = 0; i < pdc_region_cnt; i++) {
- if (pin >= pdc_region[i].pin_base &&
- pin < pdc_region[i].pin_base + pdc_region[i].cnt)
- return &pdc_region[i];
+ for (int i = 0; i < pdc->region_cnt; i++) {
+ if (pin >= pdc->region[i].pin_base &&
+ pin < pdc->region[i].pin_base + pdc->region[i].cnt)
+ return &pdc->region[i];
}
return NULL;
@@ -301,24 +563,45 @@ static int qcom_pdc_alloc(struct irq_domain *domain, unsigned int virq,
if (ret)
return ret;
+ /*
+ * PDC secondary chip is only set for the GPIO interrupts as SPIs.
+ * Direct SPI interrupts are still in pass through mode (no latching
+ * at PDC).
+ */
+ if (pdc->mode == PDC_SECONDARY_MODE && pdc_pin_is_gpio(hwirq)) {
+ ret = irq_domain_set_hwirq_and_chip(domain, virq, hwirq,
+ &qcom_pdc_gic_secondary_chip,
+ NULL);
+ if (ret)
+ return ret;
+
+ /* Secondary mode converts all interrupts to LEVEL HIGH type */
+ type = IRQ_TYPE_LEVEL_HIGH;
+ } else {
+ ret = irq_domain_set_hwirq_and_chip(domain, virq, hwirq,
+ &qcom_pdc_gic_chip,
+ NULL);
+ if (ret)
+ return ret;
+
+ if (type & IRQ_TYPE_EDGE_BOTH)
+ type = IRQ_TYPE_EDGE_RISING;
+
+ if (type & IRQ_TYPE_LEVEL_MASK)
+ type = IRQ_TYPE_LEVEL_HIGH;
+ }
+
region = get_pin_region(hwirq);
if (!region)
return irq_domain_disconnect_hierarchy(domain->parent, virq);
- if (type & IRQ_TYPE_EDGE_BOTH)
- type = IRQ_TYPE_EDGE_RISING;
-
- if (type & IRQ_TYPE_LEVEL_MASK)
- type = IRQ_TYPE_LEVEL_HIGH;
-
parent_fwspec.fwnode = domain->parent->fwnode;
parent_fwspec.param_count = 3;
parent_fwspec.param[0] = 0;
parent_fwspec.param[1] = pin_to_hwirq(region, hwirq);
parent_fwspec.param[2] = type;
- return irq_domain_alloc_irqs_parent(domain, virq, nr_irqs,
- &parent_fwspec);
+ return irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, &parent_fwspec);
}
static const struct irq_domain_ops qcom_pdc_ops = {
@@ -327,52 +610,56 @@ static const struct irq_domain_ops qcom_pdc_ops = {
.free = irq_domain_free_irqs_common,
};
-static int pdc_setup_pin_mapping(struct device_node *np)
+static int pdc_setup_pin_mapping(struct device *dev)
{
- int ret, n, i;
+ struct device_node *np = dev->of_node;
+ int ret, n;
n = of_property_count_elems_of_size(np, "qcom,pdc-ranges", sizeof(u32));
if (n <= 0 || n % 3)
return -EINVAL;
- pdc_region_cnt = n / 3;
- pdc_region = kzalloc_objs(*pdc_region, pdc_region_cnt);
- if (!pdc_region) {
- pdc_region_cnt = 0;
+ pdc->region_cnt = n / 3;
+ pdc->region = devm_kcalloc(dev, pdc->region_cnt, sizeof(*pdc->region), GFP_KERNEL);
+ if (!pdc->region) {
+ pdc->region_cnt = 0;
return -ENOMEM;
}
- for (n = 0; n < pdc_region_cnt; n++) {
- ret = of_property_read_u32_index(np, "qcom,pdc-ranges",
- n * 3 + 0,
- &pdc_region[n].pin_base);
+ for (n = 0; n < pdc->region_cnt; n++) {
+ ret = of_property_read_u32_index(np, "qcom,pdc-ranges", n * 3 + 0,
+ &pdc->region[n].pin_base);
if (ret)
return ret;
- ret = of_property_read_u32_index(np, "qcom,pdc-ranges",
- n * 3 + 1,
- &pdc_region[n].parent_base);
+ ret = of_property_read_u32_index(np, "qcom,pdc-ranges", n * 3 + 1,
+ &pdc->region[n].parent_base);
if (ret)
return ret;
- ret = of_property_read_u32_index(np, "qcom,pdc-ranges",
- n * 3 + 2,
- &pdc_region[n].cnt);
+ ret = of_property_read_u32_index(np, "qcom,pdc-ranges", n * 3 + 2,
+ &pdc->region[n].cnt);
if (ret)
return ret;
- for (i = 0; i < pdc_region[n].cnt; i++)
- __pdc_enable_intr(i + pdc_region[n].pin_base, 0);
+ for (int i = 0; i < pdc->region[n].cnt; i++) {
+ if (pdc_pin_is_gpio(i + pdc->region[n].pin_base) &&
+ pdc->mode == PDC_SECONDARY_MODE)
+ pdc->clear_gpio(i + pdc->region[n].pin_base);
+
+ pdc->enable_intr(i + pdc->region[n].pin_base, false);
+ }
}
return 0;
}
-
static int qcom_pdc_probe(struct platform_device *pdev, struct device_node *parent)
{
struct irq_domain *parent_domain, *pdc_domain;
struct device_node *node = pdev->dev.of_node;
+ struct device *dev = &pdev->dev;
resource_size_t res_size;
struct resource res;
+ u32 irq_param;
int ret;
/* compat with old sm8150 DT which had very small region for PDC */
@@ -383,6 +670,34 @@ static int qcom_pdc_probe(struct platform_device *pdev, struct device_node *pare
if (res_size > resource_size(&res))
pr_warn("%pOF: invalid reg size, please fix DT\n", node);
+ pdc = devm_kzalloc(dev, sizeof(*pdc), GFP_KERNEL);
+ if (!pdc)
+ return -ENOMEM;
+
+ pdc->base = devm_ioremap(dev, res.start, res_size);
+ if (!pdc->base) {
+ pr_err("%pOF: unable to map PDC registers\n", node);
+ return -ENXIO;
+ }
+
+ pdc->version = pdc_reg_read(PDC_VERSION_REG, 0);
+
+ if (pdc->version >= PDC_VERSION_3_2) {
+ pdc->cfg_fields = &pdc_cfg_v3_2;
+ pdc->regs = &pdc_v3_2;
+ pdc->enable_intr = pdc_enable_intr_cfg;
+ } else if (pdc->version >= PDC_VERSION_3_0) {
+ pdc->cfg_fields = &pdc_cfg_v3_0;
+ pdc->regs = &pdc_v3_0;
+ pdc->enable_intr = pdc_enable_intr_bank;
+ } else {
+ pdc->cfg_fields = &pdc_cfg_v2_7;
+ pdc->regs = &pdc_v2_7;
+ pdc->enable_intr = pdc_enable_intr_bank;
+ }
+
+ pdc->mode = PDC_PASS_THROUGH_MODE;
+
/*
* PDC has multiple DRV regions, each one provides the same set of
* registers for a particular client in the system. Due to a hardware
@@ -392,57 +707,54 @@ static int qcom_pdc_probe(struct platform_device *pdev, struct device_node *pare
* region with the expected offset to preserve support for old DTs.
*/
if (of_device_is_compatible(node, "qcom,x1e80100-pdc")) {
- pdc_prev_base = ioremap(res.start - PDC_DRV_SIZE, IRQ_ENABLE_BANK_MAX);
- if (!pdc_prev_base) {
+ pdc->prev_base = devm_ioremap(dev, res.start - PDC_DRV_SIZE,
+ pdc->regs->irq_en_reg + IRQ_ENABLE_BANK_MAX);
+ if (!pdc->prev_base) {
pr_err("%pOF: unable to map previous PDC DRV region\n", node);
return -ENXIO;
}
- pdc_x1e_quirk = true;
- }
+ pdc->x1e_quirk = true;
- pdc_base = ioremap(res.start, res_size);
- if (!pdc_base) {
- pr_err("%pOF: unable to map PDC registers\n", node);
- ret = -ENXIO;
- goto fail;
+ if (!qcom_scm_is_available())
+ return -EPROBE_DEFER;
+
+ ret = qcom_scm_io_writel(PDC_GPIO_INT_CTL_ENABLE, PDC_PASS_THROUGH_MODE);
+ if (ret) {
+ pdc->mode = PDC_SECONDARY_MODE;
+ pdc->unmask_gpio = pdc_unmask_gpio_cfg;
+ pdc->clear_gpio = pdc_clear_gpio_cfg;
+ }
}
- pdc_version = pdc_reg_read(PDC_VERSION_REG, 0);
+ irq_param = pdc_reg_read(pdc->regs->irq_param_reg, 0);
+ pdc->num_spis = FIELD_GET(GENMASK(7, 0), irq_param);
parent_domain = irq_find_host(parent);
if (!parent_domain) {
pr_err("%pOF: unable to find PDC's parent domain\n", node);
- ret = -ENXIO;
- goto fail;
+ return -ENXIO;
}
- ret = pdc_setup_pin_mapping(node);
+ raw_spin_lock_init(&pdc->lock);
+
+ ret = pdc_setup_pin_mapping(dev);
if (ret) {
pr_err("%pOF: failed to init PDC pin-hwirq mapping\n", node);
- goto fail;
+ return ret;
}
- pdc_domain = irq_domain_create_hierarchy(parent_domain,
- IRQ_DOMAIN_FLAG_QCOM_PDC_WAKEUP,
- PDC_MAX_GPIO_IRQS,
- of_fwnode_handle(node),
- &qcom_pdc_ops, NULL);
+ pdc_domain = irq_domain_create_hierarchy(parent_domain, IRQ_DOMAIN_FLAG_QCOM_PDC_WAKEUP,
+ PDC_MAX_IRQS, of_fwnode_handle(node),
+ &qcom_pdc_ops, NULL);
if (!pdc_domain) {
pr_err("%pOF: PDC domain add failed\n", node);
- ret = -ENOMEM;
- goto fail;
+ return -ENOMEM;
}
irq_domain_update_bus_token(pdc_domain, DOMAIN_BUS_WAKEUP);
return 0;
-
-fail:
- kfree(pdc_region);
- iounmap(pdc_base);
- iounmap(pdc_prev_base);
- return ret;
}
IRQCHIP_PLATFORM_DRIVER_BEGIN(qcom_pdc)
^ permalink raw reply related [flat|nested] 26+ messages in thread* [GIT pull] locking/futex for v7.3-rc1
2026-08-17 11:23 [GIT pull] core/entry for v7.3-rc1 Thomas Gleixner
` (2 preceding siblings ...)
2026-08-17 11:23 ` [GIT pull] irq/drivers " Thomas Gleixner
@ 2026-08-17 11:23 ` Thomas Gleixner
2026-08-19 1:10 ` pr-tracker-bot
2026-08-17 11:23 ` [GIT pull] smp/core " Thomas Gleixner
` (5 subsequent siblings)
9 siblings, 1 reply; 26+ messages in thread
From: Thomas Gleixner @ 2026-08-17 11:23 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest locking/futex branch from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git locking-futex-2026-08-17
up to: d1c656df36d0: selftests/futex: Give circular-list nodes static storage
Futex updates:
- A set of improvements to various futex self tests
- Conversion to the selftest harness
- Provide and use thread creation and synchronization helpers to reduce
the dependency on delays, which tend to fail on loaded test systems
- New tests for validating owner exit scenarios for robust and PI futexes
- Runtime detect supported features and skip the tests if the kernel
has no support
- A few minor fixes
Thanks,
tglx
------------------>
Ankit Khushwaha (1):
selftests/futex: Remove static keyword from 'head'
Hisam Mehboob (1):
selftests/rseq: Replace glibc-specific __GNUC_PREREQ with portable check
Miles Krause (1):
selftests/futex: Give circular-list nodes static storage
Valery Borovsky (1):
selftests/futex: Add FUTEX_LOCK_PI owner-exiting coverage
Vasileios Almpanis (1):
futex: Remove unnecessary NULL check before kvfree()
Wake Liu (13):
selftests/futex: Migrate functional tests to harness
selftests/futex: Correct validation logic in waitv
selftests/futex: Migrate futex_wait to harness
selftests/futex: Migrate futex_wait_private_mapped_file to harness
selftests/futex: Migrate futex_wait_uninitialized_heap to harness
selftests/futex: Migrate futex_requeue to harness
selftests/futex: Migrate futex_requeue_pi to harness
selftests/futex: Migrate futex_requeue_pi_mismatched_ops to harness
selftests/futex: Migrate futex_requeue_pi_signal_restart to harness
selftests/futex: Migrate futex_numa_mpol to harness
selftests/futex: Migrate futex_priv_hash to harness
selftests/futex: Migrate robust_list to harness
selftests/futex: Dynamically skip unsupported tests
Yuwen Chen (2):
selftests/futex: Provide thread creation and synchronization helpers
selftests/futex: Use thread synchronization helpers instead of usleep()
kernel/futex/core.c | 3 +-
.../testing/selftests/futex/functional/.gitignore | 1 +
tools/testing/selftests/futex/functional/Makefile | 6 +-
.../futex/functional/futex_lock_pi_exiting.c | 263 +++++++++++++++++++++
.../selftests/futex/functional/futex_numa_mpol.c | 127 +++++-----
.../selftests/futex/functional/futex_priv_hash.c | 163 ++++++-------
.../selftests/futex/functional/futex_requeue.c | 67 ++++--
.../selftests/futex/functional/futex_requeue_pi.c | 157 ++++++------
.../functional/futex_requeue_pi_mismatched_ops.c | 41 ++--
.../functional/futex_requeue_pi_signal_restart.c | 74 +++---
.../selftests/futex/functional/futex_wait.c | 127 ++++++----
.../functional/futex_wait_private_mapped_file.c | 36 +--
.../futex/functional/futex_wait_timeout.c | 108 ++++-----
.../functional/futex_wait_uninitialized_heap.c | 35 +--
.../futex/functional/futex_wait_wouldblock.c | 34 +--
.../selftests/futex/functional/futex_waitv.c | 146 ++++++------
.../selftests/futex/functional/robust_list.c | 183 +++++++-------
tools/testing/selftests/futex/functional/run.sh | 3 +
tools/testing/selftests/futex/include/futex2test.h | 10 +
.../testing/selftests/futex/include/futex_thread.h | 117 +++++++++
.../selftests/rseq/rseq-x86-thread-pointer.h | 4 +-
21 files changed, 1097 insertions(+), 608 deletions(-)
create mode 100644 tools/testing/selftests/futex/functional/futex_lock_pi_exiting.c
create mode 100644 tools/testing/selftests/futex/include/futex_thread.h
diff --git a/kernel/futex/core.c b/kernel/futex/core.c
index 179b26e9c934..233b926f5624 100644
--- a/kernel/futex/core.c
+++ b/kernel/futex/core.c
@@ -1746,8 +1746,7 @@ void futex_hash_free(struct mm_struct *mm)
free_percpu(mm->futex.phash.ref);
kvfree(mm->futex.phash.hash_new);
fph = rcu_dereference_raw(mm->futex.phash.hash);
- if (fph)
- kvfree(fph);
+ kvfree(fph);
}
static bool futex_pivot_pending(struct mm_struct *mm)
diff --git a/tools/testing/selftests/futex/functional/.gitignore b/tools/testing/selftests/futex/functional/.gitignore
index 23b9fea8d190..7c39d10b38e4 100644
--- a/tools/testing/selftests/futex/functional/.gitignore
+++ b/tools/testing/selftests/futex/functional/.gitignore
@@ -1,4 +1,5 @@
# SPDX-License-Identifier: GPL-2.0-only
+futex_lock_pi_exiting
futex_numa_mpol
futex_priv_hash
futex_requeue
diff --git a/tools/testing/selftests/futex/functional/Makefile b/tools/testing/selftests/futex/functional/Makefile
index 5c1c824f9740..a03bd5acba50 100644
--- a/tools/testing/selftests/futex/functional/Makefile
+++ b/tools/testing/selftests/futex/functional/Makefile
@@ -11,7 +11,8 @@ endif
LOCAL_HDRS := \
../include/futextest.h \
- ../include/atomic.h
+ ../include/atomic.h \
+ ../include/futex_thread.h
TEST_GEN_PROGS := \
futex_wait_timeout \
futex_wait_wouldblock \
@@ -26,7 +27,8 @@ TEST_GEN_PROGS := \
futex_numa_mpol \
futex_waitv \
futex_numa \
- robust_list
+ robust_list \
+ futex_lock_pi_exiting
TEST_PROGS := run.sh
diff --git a/tools/testing/selftests/futex/functional/futex_lock_pi_exiting.c b/tools/testing/selftests/futex/functional/futex_lock_pi_exiting.c
new file mode 100644
index 000000000000..623c5f3a1836
--- /dev/null
+++ b/tools/testing/selftests/futex/functional/futex_lock_pi_exiting.c
@@ -0,0 +1,263 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/******************************************************************************
+ *
+ * futex_lock_pi_exiting.c
+ *
+ * Coverage for the FUTEX_LOCK_PI owner-exiting path. futex_wait_timeout.c
+ * already covers FUTEX_LOCK_PI timeout semantics and robust_list.c covers
+ * owner death via the robust list, but nothing exercises FUTEX_LOCK_PI when a
+ * non-robust PI owner exits while holding the lock, nor the basic ownership /
+ * EDEADLK / unlock word semantics.
+ *
+ * DESCRIPTION
+ * Three tests:
+ *
+ * 1. lock_unlock_basic - uncontended FUTEX_LOCK_PI semantics: the futex
+ * word carries the owner TID, a recursive lock by the owner returns
+ * EDEADLK, and FUTEX_UNLOCK_PI clears the word.
+ *
+ * 2. owner_dies_with_blocked_waiter - a thread acquires a PI futex and
+ * exits while holding it. do_exit() runs futex_cleanup_begin() (which
+ * flips the task's futex state to FUTEX_STATE_EXITING) and
+ * exit_pi_state_list() (which hands off / tears down the pi_state). A
+ * contending FUTEX_LOCK_PI waiter must end up in one of:
+ *
+ * 0 - ownership was transferred to / acquired by the waiter
+ * EOWNERDEAD - previous owner died holding the lock; the caller is
+ * now the owner and must acknowledge by unlocking
+ * ESRCH - the owner encoded in the futex word is already gone
+ *
+ * and on the first two it must actually own the lock afterwards.
+ *
+ * 3. stress_owner_exits - hammer that same exiting-owner path. This is
+ * where the following bug lived: the 'exiting' task pointer was not
+ * reset at the retry label, so after wait_for_owner_exiting() dropped
+ * its reference a subsequent retry that returned a non-EBUSY error fed
+ * the stale pointer back in and tripped WARN_ON_ONCE(exiting). That
+ * warning is invisible to user space, so this test cannot observe it
+ * through a syscall return value; it only becomes a visible failure
+ * (crash) on a kernel booted with panic_on_warn=1 (or built with
+ * CONFIG_BUG_ON_DATA_CORRUPTION). The loop drives the path so that
+ * such a kernel trips on it - the canonical way fuzz/CI catch these.
+ *
+ * Fix: 210d36d892de ("futex: Clear stale exiting pointer in
+ * futex_lock_pi() retry path")
+ * Fixes: 3ef240eaff36 ("futex: Prevent exit livelock")
+ *
+ * AUTHOR
+ * Based on futex test boilerplate by Darren Hart <dvhart@linux.intel.com>
+ *
+ *****************************************************************************/
+
+#define _GNU_SOURCE
+
+#include <errno.h>
+#include <pthread.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/syscall.h>
+
+#include "futextest.h"
+#include "kselftest_harness.h"
+
+/*
+ * Iterations for the stress variant. Enough to repeatedly land in the narrow
+ * EXITING window while keeping the test fast.
+ */
+#define STRESS_ITERS 1000
+
+static futex_t pi_futex;
+static pthread_barrier_t locked_barrier;
+static pthread_barrier_t release_barrier;
+
+static pid_t sys_gettid(void)
+{
+ return syscall(SYS_gettid);
+}
+
+/*
+ * Owner thread: acquire the PI futex and exit while still holding it. Two
+ * modes:
+ * park == 0: signal that we hold the lock, then exit immediately (racy; the
+ * waiter races against our exit path).
+ * park == 1: signal that we hold the lock and keep holding until released
+ * via release_barrier, so a waiter has time to contend as a real
+ * PI waiter before we die.
+ */
+static void *owner_thread(void *arg)
+{
+ long park = (long)arg;
+
+ if (futex_lock_pi(&pi_futex, NULL, 0, FUTEX_PRIVATE_FLAG) != 0)
+ return (void *)(intptr_t)-errno;
+
+ pthread_barrier_wait(&locked_barrier);
+
+ if (park)
+ pthread_barrier_wait(&release_barrier);
+
+ /* Die while still holding the lock. */
+ pthread_exit((void *)0);
+}
+
+/*
+ * Block on the PI futex as a waiter. Returns 0 on acquisition, otherwise the
+ * positive errno.
+ */
+static int waiter_lock_pi(void)
+{
+ int ret = futex_lock_pi(&pi_futex, NULL, 0, FUTEX_PRIVATE_FLAG);
+
+ return ret == 0 ? 0 : errno;
+}
+
+static int outcome_ok(int outcome)
+{
+ return outcome == 0 || outcome == EOWNERDEAD || outcome == ESRCH;
+}
+
+/* Results published by waiter_thread() for the owning thread to assert on. */
+static int waiter_outcome;
+static int waiter_owns;
+
+/*
+ * Waiter thread for the blocked-waiter test. Contends for the lock and, when
+ * it acquires, records whether the futex word actually carries its TID and
+ * releases the lock itself (FUTEX_UNLOCK_PI must run in the owning thread).
+ */
+static void *waiter_thread(void *arg)
+{
+ pid_t tid = sys_gettid();
+
+ waiter_outcome = waiter_lock_pi();
+ if (waiter_outcome == 0 || waiter_outcome == EOWNERDEAD) {
+ waiter_owns = (pi_futex & FUTEX_TID_MASK) == (futex_t)tid;
+ futex_unlock_pi(&pi_futex, FUTEX_PRIVATE_FLAG);
+ }
+ return NULL;
+}
+
+FIXTURE(lock_pi_exiting) {
+};
+
+FIXTURE_SETUP(lock_pi_exiting) {
+}
+
+FIXTURE_TEARDOWN(lock_pi_exiting) {
+}
+
+/*
+ * Uncontended FUTEX_LOCK_PI semantics, fully deterministic.
+ */
+TEST_F(lock_pi_exiting, lock_unlock_basic)
+{
+ pid_t tid = sys_gettid();
+ int ret;
+
+ pi_futex = FUTEX_INITIALIZER;
+
+ /* Acquire: we become the owner, our TID lands in the futex word. */
+ ret = futex_lock_pi(&pi_futex, NULL, 0, FUTEX_PRIVATE_FLAG);
+ ASSERT_EQ(ret, 0)
+ TH_LOG("lock failed: errno=%d (%s)", errno, strerror(errno));
+ ASSERT_EQ(pi_futex & FUTEX_TID_MASK, (futex_t)tid)
+ TH_LOG("owner TID not in futex word: 0x%08x", pi_futex);
+
+ /* A recursive lock by the owner must be refused, not deadlock. */
+ errno = 0;
+ ret = futex_lock_pi(&pi_futex, NULL, 0, FUTEX_PRIVATE_FLAG);
+ ASSERT_EQ(ret, -1);
+ ASSERT_EQ(errno, EDEADLK)
+ TH_LOG("recursive lock: expected EDEADLK, got errno=%d", errno);
+
+ /* Release: the futex word is handed back clean. */
+ ret = futex_unlock_pi(&pi_futex, FUTEX_PRIVATE_FLAG);
+ ASSERT_EQ(ret, 0)
+ TH_LOG("unlock failed: errno=%d", errno);
+ ASSERT_EQ(pi_futex, (futex_t)0)
+ TH_LOG("futex word not cleared after unlock: 0x%08x", pi_futex);
+}
+
+/*
+ * A PI waiter inherits the lock when the owner dies holding it.
+ *
+ * The owner parks while holding the lock, this thread contends for it, then
+ * the owner exits. The waiter must come out cleanly (no hang, no unexpected
+ * error) and, when it acquires, must actually own the lock.
+ */
+TEST_F(lock_pi_exiting, owner_dies_with_blocked_waiter)
+{
+ pthread_t owner, waiter;
+
+ pthread_barrier_init(&locked_barrier, NULL, 2);
+ pthread_barrier_init(&release_barrier, NULL, 2);
+ pi_futex = FUTEX_INITIALIZER;
+ waiter_outcome = -1;
+ waiter_owns = 0;
+
+ ASSERT_EQ(pthread_create(&owner, NULL, owner_thread, (void *)1), 0);
+
+ /* Wait until the owner actually holds the lock. */
+ pthread_barrier_wait(&locked_barrier);
+
+ /* Start the waiter and give it time to block as a real PI waiter. */
+ ASSERT_EQ(pthread_create(&waiter, NULL, waiter_thread, NULL), 0);
+ usleep(1000);
+
+ /* Release the owner so it dies while the waiter is queued on it. */
+ pthread_barrier_wait(&release_barrier);
+
+ pthread_join(waiter, NULL);
+ pthread_join(owner, NULL);
+
+ ASSERT_TRUE(outcome_ok(waiter_outcome)) {
+ TH_LOG("unexpected FUTEX_LOCK_PI outcome: %d (%s)",
+ waiter_outcome, strerror(waiter_outcome));
+ }
+ if (waiter_outcome == 0 || waiter_outcome == EOWNERDEAD) {
+ ASSERT_TRUE(waiter_owns)
+ TH_LOG("waiter acquired but futex word lacks its TID");
+ }
+
+ pthread_barrier_destroy(&locked_barrier);
+ pthread_barrier_destroy(&release_barrier);
+}
+
+/*
+ * Stress: repeatedly let an owner exit while a waiter contends for the lock.
+ *
+ * Each iteration drives the FUTEX_STATE_EXITING -> -EBUSY -> retry path that
+ * the stale-'exiting'-pointer bug lived on (210d36d892de). The warning it
+ * fixed is invisible to user space, so on a normally-configured kernel both
+ * the buggy and fixed kernels pass here; the point is to make a kernel booted
+ * with panic_on_warn=1 trip during one of these iterations.
+ */
+TEST_F(lock_pi_exiting, stress_owner_exits)
+{
+ for (int i = 0; i < STRESS_ITERS; i++) {
+ pthread_t owner;
+ int outcome;
+
+ pthread_barrier_init(&locked_barrier, NULL, 2);
+ pi_futex = FUTEX_INITIALIZER;
+
+ ASSERT_EQ(pthread_create(&owner, NULL, owner_thread, (void *)0), 0);
+
+ /* Owner holds the lock; race FUTEX_LOCK_PI against its exit. */
+ pthread_barrier_wait(&locked_barrier);
+
+ outcome = waiter_lock_pi();
+ ASSERT_TRUE(outcome_ok(outcome)) {
+ TH_LOG("iter %d: unexpected outcome %d (%s)",
+ i, outcome, strerror(outcome));
+ }
+ if (outcome == 0 || outcome == EOWNERDEAD)
+ futex_unlock_pi(&pi_futex, FUTEX_PRIVATE_FLAG);
+
+ pthread_join(owner, NULL);
+ pthread_barrier_destroy(&locked_barrier);
+ }
+}
+
+TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/futex/functional/futex_numa_mpol.c b/tools/testing/selftests/futex/functional/futex_numa_mpol.c
index 78c0f7a59e17..4ffcf41efe1f 100644
--- a/tools/testing/selftests/futex/functional/futex_numa_mpol.c
+++ b/tools/testing/selftests/futex/functional/futex_numa_mpol.c
@@ -9,6 +9,7 @@
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
+#include <string.h>
#include <unistd.h>
#ifdef LIBNUMA_VER_SUFFICIENT
#include <numa.h>
@@ -28,9 +29,9 @@ static pthread_barrier_t barrier_main;
static pthread_t threads[MAX_THREADS];
struct thread_args {
- void *futex_ptr;
- unsigned int flags;
- int result;
+ void *futex_ptr;
+ unsigned int flags;
+ int result;
};
static struct thread_args thread_args[MAX_THREADS];
@@ -54,7 +55,7 @@ static void *thread_lock_fn(void *arg)
return NULL;
}
-static void create_max_threads(void *futex_ptr)
+static void create_max_threads(struct __test_metadata *_metadata, void *futex_ptr)
{
int i, ret;
@@ -63,28 +64,29 @@ static void create_max_threads(void *futex_ptr)
thread_args[i].flags = FUTEX2_SIZE_U32 | FUTEX_PRIVATE_FLAG | FUTEX2_NUMA;
thread_args[i].result = 0;
ret = pthread_create(&threads[i], NULL, thread_lock_fn, &thread_args[i]);
- if (ret)
- ksft_exit_fail_msg("pthread_create failed\n");
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_create failed");
}
}
-static void join_max_threads(void)
+static void join_max_threads(struct __test_metadata *_metadata)
{
int i, ret;
for (i = 0; i < MAX_THREADS; i++) {
ret = pthread_join(threads[i], NULL);
- if (ret)
- ksft_exit_fail_msg("pthread_join failed for thread %d\n", i);
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_join failed for thread %d", i);
}
}
-static void __test_futex(void *futex_ptr, int err_value, unsigned int futex_flags)
+static void __test_futex(struct __test_metadata *_metadata, void *futex_ptr, int err_value,
+ unsigned int futex_flags)
{
- int to_wake, ret, i, need_exit = 0;
+ int to_wake, ret, i;
pthread_barrier_init(&barrier_main, NULL, MAX_THREADS + 1);
- create_max_threads(futex_ptr);
+ create_max_threads(_metadata, futex_ptr);
pthread_barrier_wait(&barrier_main);
to_wake = MAX_THREADS;
@@ -92,45 +94,50 @@ static void __test_futex(void *futex_ptr, int err_value, unsigned int futex_flag
ret = futex2_wake(futex_ptr, to_wake, futex_flags);
if (err_value) {
- if (ret >= 0)
- ksft_exit_fail_msg("futex2_wake(%d, 0x%x) should fail, but didn't\n",
- to_wake, futex_flags);
+ EXPECT_LT(ret, 0) {
+ TH_LOG("futex2_wake(%d, 0x%x) should fail, but didn't",
+ to_wake, futex_flags);
+ }
- if (errno != err_value)
- ksft_exit_fail_msg("futex2_wake(%d, 0x%x) expected error was %d, but returned %d (%s)\n",
- to_wake, futex_flags, err_value, errno, strerror(errno));
+ EXPECT_EQ(errno, err_value) {
+ TH_LOG("futex2_wake(%d, 0x%x) expected error was %d, but returned %d (%s)",
+ to_wake, futex_flags, err_value, errno, strerror(errno));
+ }
break;
}
if (ret < 0) {
- ksft_exit_fail_msg("Failed futex2_wake(%d, 0x%x): %m\n",
- to_wake, futex_flags);
+ if (errno == ENOSYS || (errno == EINVAL && (futex_flags & FUTEX2_NUMA)))
+ SKIP(return, "futex2 or FUTEX2_NUMA not supported by kernel");
+
+ ASSERT_GE(ret, 0) {
+ TH_LOG("Failed futex2_wake(%d, 0x%x): %s",
+ to_wake, futex_flags, strerror(errno));
+ }
}
if (!ret)
usleep(50);
to_wake -= ret;
} while (to_wake);
- join_max_threads();
+ join_max_threads(_metadata);
for (i = 0; i < MAX_THREADS; i++) {
- if (err_value && thread_args[i].result != -1) {
- ksft_print_msg("Thread %d should fail but succeeded (%d)\n",
+ if (err_value) {
+ EXPECT_EQ(thread_args[i].result, -1) {
+ TH_LOG("Thread %d should fail but succeeded (%d)",
i, thread_args[i].result);
- need_exit = 1;
- }
- if (!err_value && thread_args[i].result != 0) {
- ksft_print_msg("Thread %d failed (%d)\n", i, thread_args[i].result);
- need_exit = 1;
+ }
+ } else {
+ EXPECT_EQ(thread_args[i].result, 0)
+ TH_LOG("Thread %d failed (%d)", i, thread_args[i].result);
}
}
- if (need_exit)
- ksft_exit_fail_msg("Aborting due to earlier errors.\n");
}
-static void test_futex(void *futex_ptr, int err_value)
+static void test_futex(struct __test_metadata *_metadata, void *futex_ptr, int err_value)
{
- __test_futex(futex_ptr, err_value, FUTEX2_SIZE_U32 | FUTEX_PRIVATE_FLAG | FUTEX2_NUMA);
+ __test_futex(_metadata, futex_ptr, err_value, FUTEX2_SIZE_U32 | FUTEX_PRIVATE_FLAG | FUTEX2_NUMA);
}
TEST(futex_numa_mpol)
@@ -141,43 +148,41 @@ TEST(futex_numa_mpol)
mem_size = sysconf(_SC_PAGE_SIZE);
futex_ptr = mmap(NULL, mem_size * 2, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, 0, 0);
- if (futex_ptr == MAP_FAILED)
- ksft_exit_fail_msg("mmap() for %d bytes failed\n", mem_size);
+ ASSERT_NE(futex_ptr, MAP_FAILED)
+ TH_LOG("mmap() for %d bytes failed: %s", mem_size, strerror(errno));
/* Create an invalid memory region for the "Memory out of range" test */
mprotect(futex_ptr + mem_size, mem_size, PROT_NONE);
futex_numa = futex_ptr;
- ksft_print_msg("Regular test\n");
+ TH_LOG("Regular test");
futex_numa->futex = 0;
futex_numa->numa = FUTEX_NO_NODE;
- test_futex(futex_ptr, 0);
+ test_futex(_metadata, futex_ptr, 0);
- if (futex_numa->numa == FUTEX_NO_NODE)
- ksft_exit_fail_msg("NUMA node is left uninitialized\n");
+ EXPECT_NE(futex_numa->numa, FUTEX_NO_NODE)
+ TH_LOG("NUMA node is left uninitialized");
/* FUTEX2_NUMA futex must be 8-byte aligned */
- ksft_print_msg("Mis-aligned futex\n");
- test_futex(futex_ptr + mem_size - 4, EINVAL);
+ TH_LOG("Mis-aligned futex");
+ test_futex(_metadata, futex_ptr + mem_size - 4, EINVAL);
- ksft_print_msg("Memory out of range\n");
- test_futex(futex_ptr + mem_size, EFAULT);
+ TH_LOG("Memory out of range");
+ test_futex(_metadata, futex_ptr + mem_size, EFAULT);
futex_numa->numa = FUTEX_NO_NODE;
mprotect(futex_ptr, mem_size, PROT_READ);
- ksft_print_msg("Memory, RO\n");
- test_futex(futex_ptr, EFAULT);
+ TH_LOG("Memory, RO");
+ test_futex(_metadata, futex_ptr, EFAULT);
mprotect(futex_ptr, mem_size, PROT_NONE);
- ksft_print_msg("Memory, no access\n");
- test_futex(futex_ptr, EFAULT);
+ TH_LOG("Memory, no access");
+ test_futex(_metadata, futex_ptr, EFAULT);
mprotect(futex_ptr, mem_size, PROT_READ | PROT_WRITE);
- ksft_print_msg("Memory back to RW\n");
- test_futex(futex_ptr, 0);
-
- ksft_test_result_pass("futex2 memory boundary tests passed\n");
+ TH_LOG("Memory back to RW");
+ test_futex(_metadata, futex_ptr, 0);
/* MPOL test. Does not work as expected */
#ifdef LIBNUMA_VER_SUFFICIENT
@@ -190,25 +195,23 @@ TEST(futex_numa_mpol)
sizeof(nodemask) * 8, 0);
if (ret == 0) {
ret = numa_set_mempolicy_home_node(futex_ptr, mem_size, i, 0);
- if (ret != 0)
- ksft_exit_fail_msg("Failed to set home node: %m, %d\n", errno);
+ ASSERT_EQ(ret, 0)
+ TH_LOG("Failed to set home node: %s, %d", strerror(errno), errno);
- ksft_print_msg("Node %d test\n", i);
+ TH_LOG("Node %d test", i);
futex_numa->futex = 0;
futex_numa->numa = FUTEX_NO_NODE;
- ret = futex2_wake(futex_ptr, 0, FUTEX2_SIZE_U32 | FUTEX_PRIVATE_FLAG | FUTEX2_NUMA | FUTEX2_MPOL);
- if (ret < 0)
- ksft_test_result_fail("Failed to wake 0 with MPOL: %m\n");
- if (futex_numa->numa != i) {
- ksft_exit_fail_msg("Returned NUMA node is %d expected %d\n",
- futex_numa->numa, i);
- }
+ ret = futex2_wake(futex_ptr, 0, FUTEX2_SIZE_U32 | FUTEX_PRIVATE_FLAG |
+ FUTEX2_NUMA | FUTEX2_MPOL);
+ EXPECT_GE(ret, 0)
+ TH_LOG("Failed to wake 0 with MPOL: %s", strerror(errno));
+ EXPECT_EQ(futex_numa->numa, i)
+ TH_LOG("Returned NUMA node is %d expected %d", futex_numa->numa, i);
}
}
- ksft_test_result_pass("futex2 MPOL hints test passed\n");
#else
- ksft_test_result_skip("futex2 MPOL hints test requires libnuma 2.0.18+\n");
+ SKIP(return, "futex2 MPOL hints test requires libnuma 2.0.18+");
#endif
munmap(futex_ptr, mem_size * 2);
}
diff --git a/tools/testing/selftests/futex/functional/futex_priv_hash.c b/tools/testing/selftests/futex/functional/futex_priv_hash.c
index e8079d7c65e8..a8742e204540 100644
--- a/tools/testing/selftests/futex/functional/futex_priv_hash.c
+++ b/tools/testing/selftests/futex/functional/futex_priv_hash.c
@@ -9,6 +9,7 @@
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
+#include <string.h>
#include <unistd.h>
#include <linux/prctl.h>
@@ -39,31 +40,28 @@ static int futex_hash_slots_get(void)
return prctl(PR_FUTEX_HASH, PR_FUTEX_HASH_GET_SLOTS);
}
-static void futex_hash_slots_set_verify(int slots)
+static void futex_hash_slots_set_verify(struct __test_metadata *_metadata, int slots)
{
int ret;
ret = futex_hash_slots_set(slots);
- if (ret != 0) {
- ksft_test_result_fail("Failed to set slots to %d: %m\n", slots);
- ksft_finished();
- }
+ ASSERT_EQ(ret, 0)
+ TH_LOG("Failed to set slots to %d: %s", slots, strerror(errno));
+
ret = futex_hash_slots_get();
- if (ret != slots) {
- ksft_test_result_fail("Set %d slots but PR_FUTEX_HASH_GET_SLOTS returns: %d, %m\n",
- slots, ret);
- ksft_finished();
+ ASSERT_EQ(ret, slots) {
+ TH_LOG("Set %d slots but PR_FUTEX_HASH_GET_SLOTS returns: %d, %s",
+ slots, ret, strerror(errno));
}
- ksft_test_result_pass("SET and GET slots %d passed\n", slots);
}
-static void futex_hash_slots_set_must_fail(int slots)
+static void futex_hash_slots_set_must_fail(struct __test_metadata *_metadata, int slots)
{
int ret;
ret = futex_hash_slots_set(slots);
- ksft_test_result(ret < 0, "futex_hash_slots_set(%d)\n",
- slots);
+ EXPECT_LT(ret, 0)
+ TH_LOG("futex_hash_slots_set(%d) should fail but succeeded", slots);
}
static void *thread_return_fn(void *arg)
@@ -82,32 +80,32 @@ static void *thread_lock_fn(void *arg)
return NULL;
}
-static void create_max_threads(void *(*thread_fn)(void *))
+static void create_max_threads(struct __test_metadata *_metadata, void *(*thread_fn)(void *))
{
int i, ret;
for (i = 0; i < MAX_THREADS; i++) {
ret = pthread_create(&threads[i], NULL, thread_fn, NULL);
- if (ret)
- ksft_exit_fail_msg("pthread_create failed: %m\n");
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_create failed: %s", strerror(errno));
}
}
-static void join_max_threads(void)
+static void join_max_threads(struct __test_metadata *_metadata)
{
int i, ret;
for (i = 0; i < MAX_THREADS; i++) {
ret = pthread_join(threads[i], NULL);
- if (ret)
- ksft_exit_fail_msg("pthread_join failed for thread %d\n", i);
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_join failed for thread %d: %s", i, strerror(errno));
}
}
#define SEC_IN_NSEC 1000000000
#define MSEC_IN_NSEC 1000000
-static void futex_dummy_op(void)
+static void futex_dummy_op(struct __test_metadata *_metadata)
{
pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER;
struct timespec timeout;
@@ -121,11 +119,11 @@ static void futex_dummy_op(void)
timeout.tv_sec++;
}
ret = pthread_mutex_timedlock(&lock, &timeout);
- if (ret == 0)
- ksft_exit_fail_msg("Successfully locked an already locked mutex.\n");
+ ASSERT_NE(ret, 0)
+ TH_LOG("Successfully locked an already locked mutex");
- if (ret != ETIMEDOUT)
- ksft_exit_fail_msg("pthread_mutex_timedlock() did not timeout: %d.\n", ret);
+ ASSERT_EQ(ret, ETIMEDOUT)
+ TH_LOG("pthread_mutex_timedlock() did not timeout: %d", ret);
}
static const char *test_msg_auto_create = "Automatic hash bucket init on thread creation.\n";
@@ -140,50 +138,48 @@ TEST(priv_hash)
ret = pthread_mutexattr_init(&mutex_attr_pi);
ret |= pthread_mutexattr_setprotocol(&mutex_attr_pi, PTHREAD_PRIO_INHERIT);
ret |= pthread_mutex_init(&global_lock, &mutex_attr_pi);
- if (ret != 0) {
- ksft_exit_fail_msg("Failed to initialize pthread mutex.\n");
- }
+ ASSERT_EQ(ret, 0)
+ TH_LOG("Failed to initialize pthread mutex");
+
/* First thread, expect to be 0, not yet initialized */
ret = futex_hash_slots_get();
- if (ret != 0)
- ksft_exit_fail_msg("futex_hash_slots_get() failed: %d, %m\n", ret);
+ if (ret < 0 && errno == EINVAL)
+ SKIP(return, "PR_FUTEX_HASH not supported by kernel");
+
+ ASSERT_EQ(ret, 0)
+ TH_LOG("futex_hash_slots_get() failed: %d, %s", ret, strerror(errno));
- ksft_test_result_pass("Basic get slots and immutable status.\n");
ret = pthread_create(&threads[0], NULL, thread_return_fn, NULL);
- if (ret != 0)
- ksft_exit_fail_msg("pthread_create() failed: %d, %m\n", ret);
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_create() failed: %d, %s", ret, strerror(errno));
ret = pthread_join(threads[0], NULL);
- if (ret != 0)
- ksft_exit_fail_msg("pthread_join() failed: %d, %m\n", ret);
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_join() failed: %d, %s", ret, strerror(errno));
/* First thread, has to initialize private hash */
futex_slots1 = futex_hash_slots_get();
- if (futex_slots1 <= 0) {
- ksft_print_msg("Current hash buckets: %d\n", futex_slots1);
- ksft_exit_fail_msg("%s", test_msg_auto_create);
- }
-
- ksft_test_result_pass("%s", test_msg_auto_create);
+ EXPECT_GT(futex_slots1, 0)
+ TH_LOG("Current hash buckets: %d. %s", futex_slots1, test_msg_auto_create);
online_cpus = sysconf(_SC_NPROCESSORS_ONLN);
ret = pthread_barrier_init(&barrier_main, NULL, MAX_THREADS + 1);
- if (ret != 0)
- ksft_exit_fail_msg("pthread_barrier_init failed: %m.\n");
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_barrier_init failed: %s", strerror(errno));
ret = pthread_mutex_lock(&global_lock);
- if (ret != 0)
- ksft_exit_fail_msg("pthread_mutex_lock failed: %m.\n");
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_mutex_lock failed: %s", strerror(errno));
counter = 0;
- create_max_threads(thread_lock_fn);
+ create_max_threads(_metadata, thread_lock_fn);
pthread_barrier_wait(&barrier_main);
/*
* The current default size of hash buckets is 16. The auto increase
* works only if more than 16 CPUs are available.
*/
- ksft_print_msg("Online CPUs: %d\n", online_cpus);
+ TH_LOG("Online CPUs: %d", online_cpus);
if (online_cpus > 16) {
retry_getslots:
futex_slotsn = futex_hash_slots_get();
@@ -200,71 +196,76 @@ TEST(priv_hash)
* sleep for 100ms and issue a futex operation.
*/
if (retry > 0) {
- futex_dummy_op();
+ futex_dummy_op(_metadata);
goto retry_getslots;
}
- ksft_print_msg("Expected increase of hash buckets but got: %d -> %d\n",
- futex_slots1, futex_slotsn);
- ksft_exit_fail_msg("%s", test_msg_auto_inc);
+ EXPECT_NE(futex_slots1, futex_slotsn) {
+ TH_LOG("Expected increase of hash buckets but got: %d -> %d. %s",
+ futex_slots1, futex_slotsn, test_msg_auto_inc);
+ }
}
- ksft_test_result_pass("%s", test_msg_auto_inc);
} else {
- ksft_test_result_skip("%s", test_msg_auto_inc);
+ SKIP(return, "Automatic increase with more than 16 CPUs (only %d online)", online_cpus);
}
ret = pthread_mutex_unlock(&global_lock);
/* Once the user changes it, it has to be what is set */
- futex_hash_slots_set_verify(2);
- futex_hash_slots_set_verify(4);
- futex_hash_slots_set_verify(8);
- futex_hash_slots_set_verify(32);
- futex_hash_slots_set_verify(16);
+ futex_hash_slots_set_verify(_metadata, 2);
+ futex_hash_slots_set_verify(_metadata, 4);
+ futex_hash_slots_set_verify(_metadata, 8);
+ futex_hash_slots_set_verify(_metadata, 32);
+ futex_hash_slots_set_verify(_metadata, 16);
ret = futex_hash_slots_set(15);
- ksft_test_result(ret < 0, "Use 15 slots\n");
+ EXPECT_LT(ret, 0)
+ TH_LOG("Use 15 slots should fail but succeeded");
+
+ futex_hash_slots_set_verify(_metadata, 2);
+ join_max_threads(_metadata);
+
+ EXPECT_EQ(counter, MAX_THREADS)
+ TH_LOG("Created and waited for %d of %d threads", counter, MAX_THREADS);
- futex_hash_slots_set_verify(2);
- join_max_threads();
- ksft_test_result(counter == MAX_THREADS, "Created and waited for %d of %d threads\n",
- counter, MAX_THREADS);
counter = 0;
/* Once the user set something, auto resize must be disabled */
ret = pthread_barrier_init(&barrier_main, NULL, MAX_THREADS);
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_barrier_init failed: %s", strerror(errno));
- create_max_threads(thread_lock_fn);
- join_max_threads();
+ create_max_threads(_metadata, thread_lock_fn);
+ join_max_threads(_metadata);
ret = futex_hash_slots_get();
- ksft_test_result(ret == 2, "No more auto-resize after manual setting, got %d\n",
- ret);
+ EXPECT_EQ(ret, 2)
+ TH_LOG("No more auto-resize after manual setting, got %d", ret);
- futex_hash_slots_set_must_fail(1 << 29);
- futex_hash_slots_set_verify(4);
+ futex_hash_slots_set_must_fail(_metadata, 1 << 29);
+ futex_hash_slots_set_verify(_metadata, 4);
/*
* Once the global hash has been requested, then this requested can not
* be undone.
*/
ret = futex_hash_slots_set(0);
- ksft_test_result(ret == 0, "Global hash request\n");
- if (ret != 0)
- return;
+ ASSERT_EQ(ret, 0)
+ TH_LOG("Global hash request failed: %s", strerror(errno));
- futex_hash_slots_set_must_fail(4);
- futex_hash_slots_set_must_fail(8);
- futex_hash_slots_set_must_fail(8);
- futex_hash_slots_set_must_fail(0);
- futex_hash_slots_set_must_fail(6);
+ futex_hash_slots_set_must_fail(_metadata, 4);
+ futex_hash_slots_set_must_fail(_metadata, 8);
+ futex_hash_slots_set_must_fail(_metadata, 8);
+ futex_hash_slots_set_must_fail(_metadata, 0);
+ futex_hash_slots_set_must_fail(_metadata, 6);
ret = pthread_barrier_init(&barrier_main, NULL, MAX_THREADS);
- if (ret != 0)
- ksft_exit_fail_msg("pthread_barrier_init failed: %m\n");
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_barrier_init failed: %s", strerror(errno));
- create_max_threads(thread_lock_fn);
- join_max_threads();
+ create_max_threads(_metadata, thread_lock_fn);
+ join_max_threads(_metadata);
ret = futex_hash_slots_get();
- ksft_test_result(ret == 0, "Continue to use global hash\n");
+ EXPECT_EQ(ret, 0)
+ TH_LOG("Continue to use global hash failed");
}
TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/futex/functional/futex_requeue.c b/tools/testing/selftests/futex/functional/futex_requeue.c
index dcf0d5f2f312..cc31f051765d 100644
--- a/tools/testing/selftests/futex/functional/futex_requeue.c
+++ b/tools/testing/selftests/futex/functional/futex_requeue.c
@@ -5,55 +5,70 @@
* futex cmp requeue test by André Almeida <andrealmeid@collabora.com>
*/
-#include <pthread.h>
#include <limits.h>
+#include <pthread.h>
+#include <string.h>
#include "futextest.h"
+#include "futex_thread.h"
#include "kselftest_harness.h"
-#define timeout_ns 30000000
-#define WAKE_WAIT_US 10000
+struct waiter_args {
+ struct __test_metadata *_metadata;
+ unsigned int n_threads;
+};
volatile futex_t *f1;
-void *waiterfn(void *arg)
+static int waiterfn(void *arg)
{
- struct timespec to;
+ struct __test_metadata *_metadata;
+ struct waiter_args *wargs = arg;
+ struct timespec to = { };
+ int res;
- to.tv_sec = 0;
- to.tv_nsec = timeout_ns;
+ _metadata = wargs->_metadata;
+ to.tv_sec = (wargs->n_threads + 1) * WAIT_FOR_THREAD_SECS;
- if (futex_wait(f1, *f1, &to, 0))
- printf("waiter failed errno %d\n", errno);
+ res = futex_wait(f1, *f1, &to, 0);
+ if (res) {
+ EXPECT_EQ(res, 0)
+ TH_LOG("waiter failed errno %d: %s", errno, strerror(errno));
+ }
- return NULL;
+ return 0;
}
TEST(requeue_single)
{
+ struct waiter_args wargs = { ._metadata = _metadata, .n_threads = 1 };
+ struct futex_thread waiter;
volatile futex_t _f1 = 0;
volatile futex_t f2 = 0;
- pthread_t waiter[10];
f1 = &_f1;
/*
* Requeue a waiter from f1 to f2, and wake f2.
*/
- ASSERT_EQ(0, pthread_create(&waiter[0], NULL, waiterfn, NULL));
+ ASSERT_EQ(futex_thread_create(&waiter, waiterfn, &wargs), 0)
+ TH_LOG("pthread_create failed");
- usleep(WAKE_WAIT_US);
+ ASSERT_EQ(futex_wait_for_thread(&waiter, _metadata), 0)
+ TH_LOG("Wait for thread failed");
- EXPECT_EQ(1, futex_cmp_requeue(f1, 0, &f2, 0, 1, 0));
- EXPECT_EQ(1, futex_wake(&f2, 1, 0));
+ EXPECT_EQ(futex_cmp_requeue(f1, 0, &f2, 0, 1, 0), 1);
+ EXPECT_EQ(futex_wake(&f2, 1, 0), 1);
+
+ EXPECT_EQ(futex_thread_destroy(&waiter), 0);
}
TEST(requeue_multiple)
{
+ struct waiter_args wargs = { ._metadata = _metadata, .n_threads = 10 };
+ struct futex_thread waiter[10];
volatile futex_t _f1 = 0;
volatile futex_t f2 = 0;
- pthread_t waiter[10];
- int i;
f1 = &_f1;
@@ -61,13 +76,21 @@ TEST(requeue_multiple)
* Create 10 waiters at f1. At futex_requeue, wake 3 and requeue 7.
* At futex_wake, wake INT_MAX (should be exactly 7).
*/
- for (i = 0; i < 10; i++)
- ASSERT_EQ(0, pthread_create(&waiter[i], NULL, waiterfn, NULL));
+ for (int i = 0; i < 10; i++) {
+ ASSERT_EQ(futex_thread_create(&waiter[i], waiterfn, &wargs), 0)
+ TH_LOG("pthread_create failed for waiter %d", i);
+ }
+
+ for (int i = 0; i < 10; i++) {
+ ASSERT_EQ(futex_wait_for_thread(&waiter[i], _metadata), 0)
+ TH_LOG("Wait for waiter thread %d failed", i);
+ }
- usleep(WAKE_WAIT_US);
+ EXPECT_EQ(futex_cmp_requeue(f1, 0, &f2, 3, 7, 0), 10);
+ EXPECT_EQ(futex_wake(&f2, INT_MAX, 0), 7);
- EXPECT_EQ(10, futex_cmp_requeue(f1, 0, &f2, 3, 7, 0));
- EXPECT_EQ(7, futex_wake(&f2, INT_MAX, 0));
+ for (int i = 0; i < 10; i++)
+ EXPECT_EQ(futex_thread_destroy(&waiter[i]), 0);
}
TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/futex/functional/futex_requeue_pi.c b/tools/testing/selftests/futex/functional/futex_requeue_pi.c
index 46d2858e15a8..65326cbef3a4 100644
--- a/tools/testing/selftests/futex/functional/futex_requeue_pi.c
+++ b/tools/testing/selftests/futex/functional/futex_requeue_pi.c
@@ -43,12 +43,13 @@ futex_t f2 = FUTEX_INITIALIZER;
futex_t wake_complete = FUTEX_INITIALIZER;
struct thread_arg {
- long id;
- struct timespec *timeout;
- int lock;
- int ret;
+ struct __test_metadata *_metadata;
+ long id;
+ struct timespec *timeout;
+ int lock;
+ int ret;
};
-#define THREAD_ARG_INITIALIZER { 0, NULL, 0, 0 }
+#define THREAD_ARG_INITIALIZER { NULL, 0, NULL, 0, 0 }
FIXTURE(args)
{
@@ -118,7 +119,7 @@ FIXTURE_VARIANT_ADD_TIMEOUT(5000);
FIXTURE_VARIANT_ADD_TIMEOUT(500000);
FIXTURE_VARIANT_ADD_TIMEOUT(2000000000);
-int create_rt_thread(pthread_t *pth, void*(*func)(void *), void *arg,
+int create_rt_thread(struct __test_metadata *_metadata, pthread_t *pth, void*(*func)(void *), void *arg,
int policy, int prio)
{
int ret;
@@ -129,29 +130,22 @@ int create_rt_thread(pthread_t *pth, void*(*func)(void *), void *arg,
memset(&schedp, 0, sizeof(schedp));
ret = pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED);
- if (ret) {
- ksft_exit_fail_msg("pthread_attr_setinheritsched\n");
- return -1;
- }
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_attr_setinheritsched failed");
ret = pthread_attr_setschedpolicy(&attr, policy);
- if (ret) {
- ksft_exit_fail_msg("pthread_attr_setschedpolicy\n");
- return -1;
- }
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_attr_setschedpolicy failed");
schedp.sched_priority = prio;
ret = pthread_attr_setschedparam(&attr, &schedp);
- if (ret) {
- ksft_exit_fail_msg("pthread_attr_setschedparam\n");
- return -1;
- }
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_attr_setschedparam failed");
ret = pthread_create(pth, &attr, func, arg);
- if (ret) {
- ksft_exit_fail_msg("pthread_create\n");
- return -1;
- }
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_create failed");
+
return 0;
}
@@ -159,70 +153,76 @@ int create_rt_thread(pthread_t *pth, void*(*func)(void *), void *arg,
void *waiterfn(void *arg)
{
struct thread_arg *args = (struct thread_arg *)arg;
+ struct __test_metadata *_metadata = args->_metadata;
futex_t old_val;
- ksft_print_dbg_msg("Waiter %ld: running\n", args->id);
+ TH_LOG("Waiter %ld: running", args->id);
/* Each thread sleeps for a different amount of time
* This is to avoid races, because we don't lock the
- * external mutex here */
+ * external mutex here
+ */
usleep(1000 * (long)args->id);
old_val = f1;
atomic_inc(&waiters_blocked);
- ksft_print_dbg_msg("Calling futex_wait_requeue_pi: %p (%u) -> %p\n",
- &f1, f1, &f2);
+ TH_LOG("Calling futex_wait_requeue_pi: %p (%u) -> %p", &f1, f1, &f2);
args->ret = futex_wait_requeue_pi(&f1, old_val, &f2, args->timeout,
FUTEX_PRIVATE_FLAG);
- ksft_print_dbg_msg("waiter %ld woke with %d %s\n", args->id, args->ret,
- args->ret < 0 ? strerror(errno) : "");
+ TH_LOG("waiter %ld woke with %d %s", args->id, args->ret,
+ args->ret < 0 ? strerror(errno) : "");
atomic_inc(&waiters_woken);
if (args->ret < 0) {
- if (args->timeout && errno == ETIMEDOUT)
+ if (args->timeout && errno == ETIMEDOUT) {
args->ret = 0;
- else {
- ksft_exit_fail_msg("futex_wait_requeue_pi\n");
+ } else {
+ ASSERT_EQ(args->ret, 0)
+ TH_LOG("futex_wait_requeue_pi failed: %s", strerror(errno));
}
futex_lock_pi(&f2, NULL, 0, FUTEX_PRIVATE_FLAG);
}
futex_unlock_pi(&f2, FUTEX_PRIVATE_FLAG);
- ksft_print_dbg_msg("Waiter %ld: exiting with %d\n", args->id, args->ret);
+ TH_LOG("Waiter %ld: exiting with %d", args->id, args->ret);
pthread_exit((void *)&args->ret);
}
void *broadcast_wakerfn(void *arg)
{
struct thread_arg *args = (struct thread_arg *)arg;
+ struct __test_metadata *_metadata = args->_metadata;
int nr_requeue = INT_MAX;
int task_count = 0;
futex_t old_val;
int nr_wake = 1;
int i = 0;
- ksft_print_dbg_msg("Waker: waiting for waiters to block\n");
+ TH_LOG("Waker: waiting for waiters to block");
while (waiters_blocked.val < THREAD_MAX)
usleep(1000);
usleep(1000);
- ksft_print_dbg_msg("Waker: Calling broadcast\n");
+ TH_LOG("Waker: Calling broadcast");
if (args->lock) {
- ksft_print_dbg_msg("Calling FUTEX_LOCK_PI on mutex=%x @ %p\n", f2, &f2);
+ TH_LOG("Calling FUTEX_LOCK_PI on mutex=%x @ %p", f2, &f2);
futex_lock_pi(&f2, NULL, 0, FUTEX_PRIVATE_FLAG);
}
continue_requeue:
old_val = f1;
args->ret = futex_cmp_requeue_pi(&f1, old_val, &f2, nr_wake, nr_requeue,
- FUTEX_PRIVATE_FLAG);
+ FUTEX_PRIVATE_FLAG);
if (args->ret < 0) {
- ksft_exit_fail_msg("FUTEX_CMP_REQUEUE_PI failed\n");
+ ASSERT_GE(args->ret, 0)
+ TH_LOG("FUTEX_CMP_REQUEUE_PI failed: %s", strerror(errno));
} else if (++i < MAX_WAKE_ITERS) {
task_count += args->ret;
if (task_count < THREAD_MAX - waiters_woken.val)
goto continue_requeue;
} else {
- ksft_exit_fail_msg("max broadcast iterations (%d) reached with %d/%d tasks woken or requeued\n",
- MAX_WAKE_ITERS, task_count, THREAD_MAX);
+ ASSERT_TRUE(0) {
+ TH_LOG("max broadcast iterations (%d) reached with %d/%d tasks woken or requeued",
+ MAX_WAKE_ITERS, task_count, THREAD_MAX);
+ }
}
futex_wake(&wake_complete, 1, FUTEX_PRIVATE_FLAG);
@@ -233,33 +233,33 @@ void *broadcast_wakerfn(void *arg)
if (args->ret > 0)
args->ret = task_count;
- ksft_print_dbg_msg("Waker: exiting with %d\n", args->ret);
+ TH_LOG("Waker: exiting with %d", args->ret);
pthread_exit((void *)&args->ret);
}
void *signal_wakerfn(void *arg)
{
struct thread_arg *args = (struct thread_arg *)arg;
+ struct __test_metadata *_metadata = args->_metadata;
unsigned int old_val;
int nr_requeue = 0;
int task_count = 0;
int nr_wake = 1;
int i = 0;
- ksft_print_dbg_msg("Waker: waiting for waiters to block\n");
+ TH_LOG("Waker: waiting for waiters to block");
while (waiters_blocked.val < THREAD_MAX)
usleep(1000);
usleep(1000);
while (task_count < THREAD_MAX && waiters_woken.val < THREAD_MAX) {
- ksft_print_dbg_msg("task_count: %d, waiters_woken: %d\n",
+ TH_LOG("task_count: %d, waiters_woken: %d",
task_count, waiters_woken.val);
if (args->lock) {
- ksft_print_dbg_msg("Calling FUTEX_LOCK_PI on mutex=%x @ %p\n",
- f2, &f2);
+ TH_LOG("Calling FUTEX_LOCK_PI on mutex=%x @ %p", f2, &f2);
futex_lock_pi(&f2, NULL, 0, FUTEX_PRIVATE_FLAG);
}
- ksft_print_dbg_msg("Waker: Calling signal\n");
+ TH_LOG("Waker: Calling signal");
/* cond_signal */
old_val = f1;
args->ret = futex_cmp_requeue_pi(&f1, old_val, &f2,
@@ -267,23 +267,27 @@ void *signal_wakerfn(void *arg)
FUTEX_PRIVATE_FLAG);
if (args->ret < 0)
args->ret = -errno;
- ksft_print_dbg_msg("futex: %x\n", f2);
+ TH_LOG("futex: %x", f2);
if (args->lock) {
- ksft_print_dbg_msg("Calling FUTEX_UNLOCK_PI on mutex=%x @ %p\n",
+ TH_LOG("Calling FUTEX_UNLOCK_PI on mutex=%x @ %p",
f2, &f2);
futex_unlock_pi(&f2, FUTEX_PRIVATE_FLAG);
}
- ksft_print_dbg_msg("futex: %x\n", f2);
- if (args->ret < 0)
- ksft_exit_fail_msg("FUTEX_CMP_REQUEUE_PI failed\n");
+ TH_LOG("futex: %x", f2);
+ if (args->ret < 0) {
+ ASSERT_GE(args->ret, 0)
+ TH_LOG("FUTEX_CMP_REQUEUE_PI failed: %s", strerror(-args->ret));
+ }
task_count += args->ret;
usleep(SIGNAL_PERIOD_US);
i++;
/* we have to loop at least THREAD_MAX times */
if (i > MAX_WAKE_ITERS + THREAD_MAX) {
- ksft_exit_fail_msg("max signaling iterations (%d) reached, giving up on pending waiters.\n",
- MAX_WAKE_ITERS + THREAD_MAX);
+ ASSERT_TRUE(0) {
+ TH_LOG("max signaling iterations (%d) reached, giving up on pending waiters.",
+ MAX_WAKE_ITERS + THREAD_MAX);
+ }
}
}
@@ -292,14 +296,15 @@ void *signal_wakerfn(void *arg)
if (args->ret >= 0)
args->ret = task_count;
- ksft_print_dbg_msg("Waker: exiting with %d\n", args->ret);
- ksft_print_dbg_msg("Waker: waiters_woken: %d\n", waiters_woken.val);
+ TH_LOG("Waker: exiting with %d", args->ret);
+ TH_LOG("Waker: waiters_woken: %d", waiters_woken.val);
pthread_exit((void *)&args->ret);
}
void *third_party_blocker(void *arg)
{
struct thread_arg *args = (struct thread_arg *)arg;
+ struct __test_metadata *_metadata = args->_metadata;
int ret2 = 0;
args->ret = futex_lock_pi(&f2, NULL, 0, FUTEX_PRIVATE_FLAG);
@@ -310,8 +315,10 @@ void *third_party_blocker(void *arg)
ret2 = futex_unlock_pi(&f2, FUTEX_PRIVATE_FLAG);
out:
- if (args->ret || ret2)
- ksft_exit_fail_msg("third_party_blocker() futex error");
+ if (args->ret || ret2) {
+ ASSERT_TRUE(0)
+ TH_LOG("%s() futex error", __func__);
+ }
pthread_exit((void *)&args->ret);
}
@@ -330,20 +337,19 @@ TEST_F(args, futex_requeue_pi)
bool lock = variant->locked;
int *waiter_ret, i, ret = 0;
- ksft_print_msg(
- "\tArguments: broadcast=%d locked=%d owner=%d timeout=%ldns\n",
+ TH_LOG("Arguments: broadcast=%d locked=%d owner=%d timeout=%ldns",
broadcast, lock, third_party_owner, timeout_ns);
if (timeout_ns) {
time_t secs;
- ksft_print_dbg_msg("timeout_ns = %ld\n", timeout_ns);
+ TH_LOG("timeout_ns = %ld", timeout_ns);
ret = clock_gettime(CLOCK_MONOTONIC, &ts);
secs = (ts.tv_nsec + timeout_ns) / 1000000000;
ts.tv_nsec = ((int64_t)ts.tv_nsec + timeout_ns) % 1000000000;
ts.tv_sec += secs;
- ksft_print_dbg_msg("ts.tv_sec = %ld\n", ts.tv_sec);
- ksft_print_dbg_msg("ts.tv_nsec = %ld\n", ts.tv_nsec);
+ TH_LOG("ts.tv_sec = %ld", ts.tv_sec);
+ TH_LOG("ts.tv_nsec = %ld", ts.tv_nsec);
tsp = &ts;
}
@@ -351,34 +357,29 @@ TEST_F(args, futex_requeue_pi)
wakerfn = broadcast_wakerfn;
if (third_party_owner) {
- if (create_rt_thread(&blocker, third_party_blocker,
- (void *)&blocker_arg, SCHED_FIFO, 1)) {
- ksft_exit_fail_msg("Creating third party blocker thread failed\n");
- }
+ blocker_arg._metadata = _metadata;
+ create_rt_thread(_metadata, &blocker, third_party_blocker,
+ (void *)&blocker_arg, SCHED_FIFO, 1);
}
atomic_set(&waiters_woken, 0);
for (i = 0; i < THREAD_MAX; i++) {
+ args[i]._metadata = _metadata;
args[i].id = i;
args[i].timeout = tsp;
- ksft_print_dbg_msg("Starting thread %d\n", i);
- if (create_rt_thread(&waiter[i], waiterfn, (void *)&args[i],
- SCHED_FIFO, 1)) {
- ksft_exit_fail_msg("Creating waiting thread failed\n");
- }
+ TH_LOG("Starting thread %d", i);
+ create_rt_thread(_metadata, &waiter[i], waiterfn, (void *)&args[i],
+ SCHED_FIFO, 1);
}
+ waker_arg._metadata = _metadata;
waker_arg.lock = lock;
- if (create_rt_thread(&waker, wakerfn, (void *)&waker_arg,
- SCHED_FIFO, 1)) {
- ksft_exit_fail_msg("Creating waker thread failed\n");
- }
+ create_rt_thread(_metadata, &waker, wakerfn, (void *)&waker_arg, SCHED_FIFO, 1);
/* Wait for threads to finish */
/* Store the first error or failure encountered in waiter_ret */
waiter_ret = &args[0].ret;
for (i = 0; i < THREAD_MAX; i++)
- pthread_join(waiter[i],
- *waiter_ret ? NULL : (void **)&waiter_ret);
+ pthread_join(waiter[i], *waiter_ret ? NULL : (void **)&waiter_ret);
if (third_party_owner)
pthread_join(blocker, NULL);
@@ -393,8 +394,8 @@ TEST_F(args, futex_requeue_pi)
ret = blocker_arg.ret;
}
- if (ret)
- ksft_test_result_fail("fail");
+ EXPECT_EQ(ret, 0)
+ TH_LOG("Test failed with error code: %d", ret);
}
TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/futex/functional/futex_requeue_pi_mismatched_ops.c b/tools/testing/selftests/futex/functional/futex_requeue_pi_mismatched_ops.c
index f686e605359c..35bb8a807bb9 100644
--- a/tools/testing/selftests/futex/functional/futex_requeue_pi_mismatched_ops.c
+++ b/tools/testing/selftests/futex/functional/futex_requeue_pi_mismatched_ops.c
@@ -29,14 +29,17 @@
futex_t f1 = FUTEX_INITIALIZER;
futex_t f2 = FUTEX_INITIALIZER;
-int child_ret = 0;
+int child_ret;
void *blocking_child(void *arg)
{
+ struct __test_metadata *_metadata = (struct __test_metadata *)arg;
+
child_ret = futex_wait(&f1, f1, NULL, FUTEX_PRIVATE_FLAG);
if (child_ret < 0) {
child_ret = -errno;
- ksft_exit_fail_msg("futex_wait\n");
+ ASSERT_EQ(child_ret, 0)
+ TH_LOG("futex_wait failed: %s", strerror(errno));
}
return (void *)&child_ret;
}
@@ -46,8 +49,8 @@ TEST(requeue_pi_mismatched_ops)
pthread_t child;
int ret;
- if (pthread_create(&child, NULL, blocking_child, NULL))
- ksft_exit_fail_msg("pthread_create\n");
+ ASSERT_EQ(pthread_create(&child, NULL, blocking_child, _metadata), 0)
+ TH_LOG("pthread_create failed");
/* Allow the child to block in the kernel. */
sleep(1);
@@ -67,27 +70,33 @@ TEST(requeue_pi_mismatched_ops)
* FUTEX_WAKE.
*/
ret = futex_wake(&f1, 1, FUTEX_PRIVATE_FLAG);
- if (ret == 1)
+ if (ret == 1) {
ret = 0;
- else if (ret < 0)
- ksft_exit_fail_msg("futex_wake\n");
- else
- ksft_exit_fail_msg("futex_wake did not wake the child\n");
+ } else if (ret < 0) {
+ ASSERT_GE(ret, 0)
+ TH_LOG("futex_wake failed: %s", strerror(errno));
+ } else {
+ ASSERT_TRUE(0)
+ TH_LOG("futex_wake did not wake the child");
+ }
} else {
- ksft_exit_fail_msg("futex_cmp_requeue_pi\n");
+ ASSERT_TRUE(0)
+ TH_LOG("futex_cmp_requeue_pi failed with unexpected errno: %s", strerror(errno));
}
} else if (ret > 0) {
- ksft_test_result_fail("futex_cmp_requeue_pi failed to detect the mismatch\n");
+ EXPECT_EQ(ret, 0)
+ TH_LOG("futex_cmp_requeue_pi failed to detect the mismatch");
} else {
- ksft_exit_fail_msg("futex_cmp_requeue_pi found no waiters\n");
+ ASSERT_TRUE(0)
+ TH_LOG("futex_cmp_requeue_pi found no waiters");
}
pthread_join(child, NULL);
- if (!ret && !child_ret)
- ksft_test_result_pass("futex_requeue_pi_mismatched_ops passed\n");
- else
- ksft_test_result_pass("futex_requeue_pi_mismatched_ops failed\n");
+ EXPECT_EQ(ret, 0)
+ TH_LOG("Test failed: ret=%d", ret);
+ EXPECT_EQ(child_ret, 0)
+ TH_LOG("Child failed: child_ret=%d", child_ret);
}
TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/futex/functional/futex_requeue_pi_signal_restart.c b/tools/testing/selftests/futex/functional/futex_requeue_pi_signal_restart.c
index a18ccae73eb1..4933612a7b55 100644
--- a/tools/testing/selftests/futex/functional/futex_requeue_pi_signal_restart.c
+++ b/tools/testing/selftests/futex/functional/futex_requeue_pi_signal_restart.c
@@ -35,10 +35,10 @@ futex_t f1 = FUTEX_INITIALIZER;
futex_t f2 = FUTEX_INITIALIZER;
atomic_t requeued = ATOMIC_INITIALIZER;
-int waiter_ret = 0;
+int waiter_ret;
-int create_rt_thread(pthread_t *pth, void*(*func)(void *), void *arg,
- int policy, int prio)
+int create_rt_thread(struct __test_metadata *_metadata, pthread_t *pth, void*(*func)(void *),
+ void *arg, int policy, int prio)
{
struct sched_param schedp;
pthread_attr_t attr;
@@ -48,45 +48,47 @@ int create_rt_thread(pthread_t *pth, void*(*func)(void *), void *arg,
memset(&schedp, 0, sizeof(schedp));
ret = pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED);
- if (ret)
- ksft_exit_fail_msg("pthread_attr_setinheritsched\n");
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_attr_setinheritsched failed");
ret = pthread_attr_setschedpolicy(&attr, policy);
- if (ret)
- ksft_exit_fail_msg("pthread_attr_setschedpolicy\n");
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_attr_setschedpolicy failed");
schedp.sched_priority = prio;
ret = pthread_attr_setschedparam(&attr, &schedp);
- if (ret)
- ksft_exit_fail_msg("pthread_attr_setschedparam\n");
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_attr_setschedparam failed");
ret = pthread_create(pth, &attr, func, arg);
- if (ret)
- ksft_exit_fail_msg("pthread_create\n");
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_create failed");
return 0;
}
void handle_signal(int signo)
{
- ksft_print_dbg_msg("signal received %s requeue\n",
- requeued.val ? "after" : "prior to");
+ printf("INFO: signal received %s requeue\n", requeued.val ? "after" : "prior to");
}
void *waiterfn(void *arg)
{
+ struct __test_metadata *_metadata = (struct __test_metadata *)arg;
unsigned int old_val;
int res;
- ksft_print_dbg_msg("Waiter running\n");
- ksft_print_dbg_msg("Calling FUTEX_LOCK_PI on f2=%x @ %p\n", f2, &f2);
+ TH_LOG("Waiter running");
+ TH_LOG("Calling FUTEX_LOCK_PI on f2=%x @ %p", f2, &f2);
old_val = f1;
res = futex_wait_requeue_pi(&f1, old_val, &(f2), NULL,
FUTEX_PRIVATE_FLAG);
if (!requeued.val || errno != EWOULDBLOCK) {
- ksft_test_result_fail("unexpected return from futex_wait_requeue_pi: %d (%s)\n",
- res, strerror(errno));
- ksft_print_dbg_msg("w2:futex: %x\n", f2);
+ EXPECT_TRUE(0) {
+ TH_LOG("unexpected return from futex_wait_requeue_pi: %d (%s)",
+ res, strerror(errno));
+ }
+ TH_LOG("w2:futex: %x", f2);
if (!res)
futex_unlock_pi(&f2, FUTEX_PRIVATE_FLAG);
}
@@ -94,7 +96,6 @@ void *waiterfn(void *arg)
pthread_exit(NULL);
}
-
TEST(futex_requeue_pi_signal_restart)
{
unsigned int old_val;
@@ -105,19 +106,17 @@ TEST(futex_requeue_pi_signal_restart)
sa.sa_handler = handle_signal;
sigemptyset(&sa.sa_mask);
sa.sa_flags = 0;
- if (sigaction(SIGUSR1, &sa, NULL))
- ksft_exit_fail_msg("sigaction\n");
+ ASSERT_EQ(sigaction(SIGUSR1, &sa, NULL), 0)
+ TH_LOG("sigaction failed");
- ksft_print_dbg_msg("m1:f2: %x\n", f2);
- ksft_print_dbg_msg("Creating waiter\n");
- res = create_rt_thread(&waiter, waiterfn, NULL, SCHED_FIFO, 1);
- if (res)
- ksft_exit_fail_msg("Creating waiting thread failed");
+ TH_LOG("m1:f2: %x", f2);
+ TH_LOG("Creating waiter");
+ create_rt_thread(_metadata, &waiter, waiterfn, _metadata, SCHED_FIFO, 1);
- ksft_print_dbg_msg("Calling FUTEX_LOCK_PI on f2=%x @ %p\n", f2, &f2);
- ksft_print_dbg_msg("m2:f2: %x\n", f2);
+ TH_LOG("Calling FUTEX_LOCK_PI on f2=%x @ %p", f2, &f2);
+ TH_LOG("m2:f2: %x", f2);
futex_lock_pi(&f2, 0, 0, FUTEX_PRIVATE_FLAG);
- ksft_print_dbg_msg("m3:f2: %x\n", f2);
+ TH_LOG("m3:f2: %x", f2);
while (1) {
/*
@@ -125,11 +124,11 @@ TEST(futex_requeue_pi_signal_restart)
* restart futex_wait_requeue_pi() in the kernel. Wait for the
* waiter to block on f1 again.
*/
- ksft_print_dbg_msg("Issuing SIGUSR1 to waiter\n");
+ TH_LOG("Issuing SIGUSR1 to waiter");
pthread_kill(waiter, SIGUSR1);
usleep(DELAY_US);
- ksft_print_dbg_msg("Requeueing waiter via FUTEX_CMP_REQUEUE_PI\n");
+ TH_LOG("Requeueing waiter via FUTEX_CMP_REQUEUE_PI");
old_val = f1;
res = futex_cmp_requeue_pi(&f1, old_val, &(f2), 1, 0,
FUTEX_PRIVATE_FLAG);
@@ -143,10 +142,11 @@ TEST(futex_requeue_pi_signal_restart)
atomic_set(&requeued, 1);
break;
} else if (res < 0) {
- ksft_exit_fail_msg("FUTEX_CMP_REQUEUE_PI failed\n");
+ ASSERT_GE(res, 0)
+ TH_LOG("FUTEX_CMP_REQUEUE_PI failed: %s", strerror(errno));
}
}
- ksft_print_dbg_msg("m4:f2: %x\n", f2);
+ TH_LOG("m4:f2: %x", f2);
/*
* Signal the waiter after requeue, waiter should return from
@@ -154,14 +154,14 @@ TEST(futex_requeue_pi_signal_restart)
* futex_unlock_pi() can't happen before the signal wakeup is detected
* in the kernel.
*/
- ksft_print_dbg_msg("Issuing SIGUSR1 to waiter\n");
+ TH_LOG("Issuing SIGUSR1 to waiter");
pthread_kill(waiter, SIGUSR1);
- ksft_print_dbg_msg("Waiting for waiter to return\n");
+ TH_LOG("Waiting for waiter to return");
pthread_join(waiter, NULL);
- ksft_print_dbg_msg("Calling FUTEX_UNLOCK_PI on mutex=%x @ %p\n", f2, &f2);
+ TH_LOG("Calling FUTEX_UNLOCK_PI on mutex=%x @ %p", f2, &f2);
futex_unlock_pi(&f2, FUTEX_PRIVATE_FLAG);
- ksft_print_dbg_msg("m5:f2: %x\n", f2);
+ TH_LOG("m5:f2: %x", f2);
}
TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/futex/functional/futex_wait.c b/tools/testing/selftests/futex/functional/futex_wait.c
index 7b8879409007..5292f5d2c3b5 100644
--- a/tools/testing/selftests/futex/functional/futex_wait.c
+++ b/tools/testing/selftests/futex/functional/futex_wait.c
@@ -5,10 +5,11 @@
* futex cmp requeue test by André Almeida <andrealmeid@collabora.com>
*/
+#include <fcntl.h>
#include <pthread.h>
+#include <stdlib.h>
#include <sys/shm.h>
#include <sys/mman.h>
-#include <fcntl.h>
#include "futextest.h"
#include "kselftest_harness.h"
@@ -19,125 +20,157 @@
void *futex;
+struct waiter_args {
+ struct __test_metadata *_metadata;
+ unsigned int flags;
+};
+
static void *waiterfn(void *arg)
{
+ struct waiter_args *args = (struct waiter_args *)arg;
+ struct __test_metadata *_metadata = args->_metadata;
struct timespec to;
- unsigned int flags = 0;
-
- if (arg)
- flags = *((unsigned int *) arg);
+ int res;
to.tv_sec = 0;
to.tv_nsec = timeout_ns;
- if (futex_wait(futex, 0, &to, flags))
- printf("waiter failed errno %d\n", errno);
+ res = futex_wait(futex, 0, &to, args->flags);
+ if (res) {
+ EXPECT_EQ(res, 0)
+ TH_LOG("waiter failed errno %d: %s", errno, strerror(errno));
+ }
+ free(args);
return NULL;
}
TEST(private_futex)
{
- unsigned int flags = FUTEX_PRIVATE_FLAG;
+ struct waiter_args *args = malloc(sizeof(*args));
u_int32_t f_private = 0;
pthread_t waiter;
int res;
+ args->_metadata = _metadata;
+ args->flags = FUTEX_PRIVATE_FLAG;
futex = &f_private;
/* Testing a private futex */
- ksft_print_dbg_msg("Calling private futex_wait on futex: %p\n", futex);
- if (pthread_create(&waiter, NULL, waiterfn, (void *) &flags))
- ksft_exit_fail_msg("pthread_create failed\n");
+ TH_LOG("Calling private futex_wait on futex: %p", futex);
+ ASSERT_EQ(pthread_create(&waiter, NULL, waiterfn, args), 0)
+ TH_LOG("pthread_create failed");
usleep(WAKE_WAIT_US);
- ksft_print_dbg_msg("Calling private futex_wake on futex: %p\n", futex);
+ TH_LOG("Calling private futex_wake on futex: %p", futex);
res = futex_wake(futex, 1, FUTEX_PRIVATE_FLAG);
- if (res != 1) {
- ksft_test_result_fail("futex_wake private returned: %d %s\n",
- errno, strerror(errno));
- } else {
- ksft_test_result_pass("futex_wake private succeeds\n");
- }
+ EXPECT_EQ(res, 1)
+ TH_LOG("futex_wake private returned: %d %s", res, res < 0 ? strerror(errno) : "");
+
+ pthread_join(waiter, NULL);
}
TEST(anon_page)
{
+ struct waiter_args *args = malloc(sizeof(*args));
u_int32_t *shared_data;
pthread_t waiter;
int res, shm_id;
+ args->_metadata = _metadata;
+ args->flags = 0;
+
/* Testing an anon page shared memory */
shm_id = shmget(IPC_PRIVATE, 4096, IPC_CREAT | 0666);
if (shm_id < 0) {
- if (errno == ENOSYS)
- ksft_exit_skip("shmget syscall not supported\n");
- perror("shmget");
- exit(1);
+ if (errno == ENOSYS) {
+ free(args);
+ SKIP(return, "shmget syscall not supported");
+ }
+ ASSERT_GE(shm_id, 0)
+ TH_LOG("shmget failed: %s", strerror(errno));
}
shared_data = shmat(shm_id, NULL, 0);
+ if (shared_data == (void *)-1) {
+ free(args);
+ ASSERT_NE(shared_data, (void *)-1)
+ TH_LOG("shmat failed: %s", strerror(errno));
+ }
*shared_data = 0;
futex = shared_data;
- ksft_print_dbg_msg("Calling shared (page anon) futex_wait on futex: %p\n", futex);
- if (pthread_create(&waiter, NULL, waiterfn, NULL))
- ksft_exit_fail_msg("pthread_create failed\n");
+ TH_LOG("Calling shared (page anon) futex_wait on futex: %p", futex);
+ ASSERT_EQ(pthread_create(&waiter, NULL, waiterfn, args), 0)
+ TH_LOG("pthread_create failed");
usleep(WAKE_WAIT_US);
- ksft_print_dbg_msg("Calling shared (page anon) futex_wake on futex: %p\n", futex);
+ TH_LOG("Calling shared (page anon) futex_wake on futex: %p", futex);
res = futex_wake(futex, 1, 0);
- if (res != 1) {
- ksft_test_result_fail("futex_wake shared (page anon) returned: %d %s\n",
- errno, strerror(errno));
- } else {
- ksft_test_result_pass("futex_wake shared (page anon) succeeds\n");
+ EXPECT_EQ(res, 1) {
+ TH_LOG("futex_wake shared (page anon) returned: %d %s",
+ res, res < 0 ? strerror(errno) : "");
}
+ pthread_join(waiter, NULL);
shmdt(shared_data);
}
TEST(file_backed)
{
+ struct waiter_args *args = malloc(sizeof(*args));
u_int32_t f_private = 0;
pthread_t waiter;
int res, fd;
void *shm;
+ args->_metadata = _metadata;
+ args->flags = 0;
+
/* Testing a file backed shared memory */
- fd = open(SHM_PATH, O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
- if (fd < 0)
- ksft_exit_fail_msg("open\n");
+ fd = open(SHM_PATH, O_RDWR | O_CREAT, 0600);
+ if (fd < 0) {
+ free(args);
+ ASSERT_GE(fd, 0)
+ TH_LOG("open failed: %s", strerror(errno));
+ }
- if (ftruncate(fd, sizeof(f_private)))
- ksft_exit_fail_msg("ftruncate\n");
+ if (ftruncate(fd, sizeof(f_private))) {
+ free(args);
+ close(fd);
+ ASSERT_TRUE(0)
+ TH_LOG("ftruncate failed: %s", strerror(errno));
+ }
shm = mmap(NULL, sizeof(f_private), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
- if (shm == MAP_FAILED)
- ksft_exit_fail_msg("mmap\n");
+ if (shm == MAP_FAILED) {
+ free(args);
+ close(fd);
+ ASSERT_NE(shm, MAP_FAILED)
+ TH_LOG("mmap failed: %s", strerror(errno));
+ }
memcpy(shm, &f_private, sizeof(f_private));
futex = shm;
- ksft_print_dbg_msg("Calling shared (file backed) futex_wait on futex: %p\n", futex);
- if (pthread_create(&waiter, NULL, waiterfn, NULL))
- ksft_exit_fail_msg("pthread_create failed\n");
+ TH_LOG("Calling shared (file backed) futex_wait on futex: %p", futex);
+ ASSERT_EQ(pthread_create(&waiter, NULL, waiterfn, args), 0)
+ TH_LOG("pthread_create failed");
usleep(WAKE_WAIT_US);
- ksft_print_dbg_msg("Calling shared (file backed) futex_wake on futex: %p\n", futex);
+ TH_LOG("Calling shared (file backed) futex_wake on futex: %p", futex);
res = futex_wake(shm, 1, 0);
- if (res != 1) {
- ksft_test_result_fail("futex_wake shared (file backed) returned: %d %s\n",
- errno, strerror(errno));
- } else {
- ksft_test_result_pass("futex_wake shared (file backed) succeeds\n");
+ EXPECT_EQ(res, 1) {
+ TH_LOG("futex_wake shared (file backed) returned: %d %s",
+ res, res < 0 ? strerror(errno) : "");
}
+ pthread_join(waiter, NULL);
munmap(shm, sizeof(f_private));
remove(SHM_PATH);
close(fd);
diff --git a/tools/testing/selftests/futex/functional/futex_wait_private_mapped_file.c b/tools/testing/selftests/futex/functional/futex_wait_private_mapped_file.c
index 2a749f9b14eb..a9f7a02e3a0b 100644
--- a/tools/testing/selftests/futex/functional/futex_wait_private_mapped_file.c
+++ b/tools/testing/selftests/futex/functional/futex_wait_private_mapped_file.c
@@ -26,6 +26,7 @@
#include <pthread.h>
#include <libgen.h>
#include <signal.h>
+#include <string.h>
#include "futextest.h"
#include "kselftest_harness.h"
@@ -41,17 +42,22 @@ struct timespec wait_timeout = { .tv_sec = 5, .tv_nsec = 0};
void *thr_futex_wait(void *arg)
{
+ struct __test_metadata *_metadata = (struct __test_metadata *)arg;
int ret;
- ksft_print_dbg_msg("futex wait\n");
+ TH_LOG("futex wait");
ret = futex_wait(&val, 1, &wait_timeout, 0);
- if (ret && errno != EWOULDBLOCK && errno != ETIMEDOUT)
- ksft_exit_fail_msg("futex error.\n");
+ if (ret && errno != EWOULDBLOCK && errno != ETIMEDOUT) {
+ ASSERT_TRUE(0)
+ TH_LOG("futex error: %s", strerror(errno));
+ }
- if (ret && errno == ETIMEDOUT)
- ksft_exit_fail_msg("waiter timedout\n");
+ if (ret && errno == ETIMEDOUT) {
+ ASSERT_TRUE(0)
+ TH_LOG("waiter timedout");
+ }
- ksft_print_dbg_msg("futex_wait: ret = %d, errno = %d\n", ret, errno);
+ TH_LOG("futex_wait: ret = %d, errno = %d", ret, errno);
return NULL;
}
@@ -61,22 +67,20 @@ TEST(wait_private_mapped_file)
pthread_t thr;
int res;
- res = pthread_create(&thr, NULL, thr_futex_wait, NULL);
- if (res < 0)
- ksft_exit_fail_msg("pthread_create error\n");
+ res = pthread_create(&thr, NULL, thr_futex_wait, _metadata);
+ ASSERT_EQ(res, 0)
+ TH_LOG("pthread_create error");
- ksft_print_dbg_msg("wait a while\n");
+ TH_LOG("wait a while");
usleep(WAKE_WAIT_US);
val = 2;
res = futex_wake(&val, 1, 0);
- ksft_print_dbg_msg("futex_wake %d\n", res);
- if (res != 1)
- ksft_exit_fail_msg("FUTEX_WAKE didn't find the waiting thread.\n");
+ TH_LOG("futex_wake %d", res);
+ EXPECT_EQ(res, 1)
+ TH_LOG("FUTEX_WAKE didn't find the waiting thread");
- ksft_print_dbg_msg("join\n");
+ TH_LOG("join");
pthread_join(thr, NULL);
-
- ksft_test_result_pass("wait_private_mapped_file");
}
TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/futex/functional/futex_wait_timeout.c b/tools/testing/selftests/futex/functional/futex_wait_timeout.c
index 674dd13af421..6e6e770fe96a 100644
--- a/tools/testing/selftests/futex/functional/futex_wait_timeout.c
+++ b/tools/testing/selftests/futex/functional/futex_wait_timeout.c
@@ -31,53 +31,47 @@ static pthread_barrier_t barrier;
*/
void *get_pi_lock(void *arg)
{
+ struct __test_metadata *_metadata = (struct __test_metadata *)arg;
int ret;
volatile futex_t lock = 0;
ret = futex_lock_pi(&futex_pi, NULL, 0, 0);
- if (ret != 0)
- ksft_exit_fail_msg("futex_lock_pi failed\n");
+ ASSERT_EQ(ret, 0)
+ TH_LOG("futex_lock_pi failed");
pthread_barrier_wait(&barrier);
/* Blocks forever */
ret = futex_wait(&lock, 0, NULL, 0);
- ksft_exit_fail_msg("futex_wait failed\n");
+ ASSERT_TRUE(0)
+ TH_LOG("futex_wait returned unexpectedly: %d", ret);
return NULL;
}
-/*
- * Check if the function returned the expected error
- */
-static void test_timeout(int res, char *test_name, int err)
-{
- if (!res || errno != err) {
- ksft_test_result_fail("%s returned %d\n", test_name,
- res < 0 ? errno : res);
- } else {
- ksft_test_result_pass("%s succeeds\n", test_name);
- }
-}
-
-/*
- * Calculate absolute timeout and correct overflow
- */
-static int futex_get_abs_timeout(clockid_t clockid, struct timespec *to,
- long timeout_ns)
-{
- if (clock_gettime(clockid, to))
- ksft_exit_fail_msg("clock_gettime failed\n");
-
- to->tv_nsec += timeout_ns;
-
- if (to->tv_nsec >= 1000000000) {
- to->tv_sec++;
- to->tv_nsec -= 1000000000;
- }
-
- return 0;
-}
+#define TEST_TIMEOUT(_res, _test_name, _err) do { \
+ if ((_res) < 0 && errno == ENOSYS && (_err) != ENOSYS) { \
+ SKIP(return, "%s is not supported (ENOSYS)", _test_name); \
+ } \
+ EXPECT_EQ((_res), -1) \
+ TH_LOG("%s returned unexpected result: %d", _test_name, (_res));\
+ if ((_res) == -1) { \
+ EXPECT_EQ(errno, (_err)) { \
+ TH_LOG("%s returned unexpected errno: %d (expected %d)",\
+ _test_name, errno, (_err)); \
+ } \
+ } \
+} while (0)
+
+#define GET_ABS_TIMEOUT(_clockid, _to, _timeout_ns) do { \
+ ASSERT_EQ(clock_gettime((_clockid), (_to)), 0) \
+ TH_LOG("clock_gettime failed"); \
+ (_to)->tv_nsec += (_timeout_ns); \
+ if ((_to)->tv_nsec >= 1000000000) { \
+ (_to)->tv_sec++; \
+ (_to)->tv_nsec -= 1000000000; \
+ } \
+} while (0)
TEST(wait_bitset)
{
@@ -90,19 +84,17 @@ TEST(wait_bitset)
to.tv_nsec = timeout_ns;
res = futex_wait(&f1, f1, &to, 0);
- test_timeout(res, "futex_wait relative", ETIMEDOUT);
+ TEST_TIMEOUT(res, "futex_wait relative", ETIMEDOUT);
/* FUTEX_WAIT_BITSET with CLOCK_REALTIME */
- if (futex_get_abs_timeout(CLOCK_REALTIME, &to, timeout_ns))
- ksft_test_result_error("get_time error");
+ GET_ABS_TIMEOUT(CLOCK_REALTIME, &to, timeout_ns);
res = futex_wait_bitset(&f1, f1, &to, 1, FUTEX_CLOCK_REALTIME);
- test_timeout(res, "futex_wait_bitset realtime", ETIMEDOUT);
+ TEST_TIMEOUT(res, "futex_wait_bitset realtime", ETIMEDOUT);
/* FUTEX_WAIT_BITSET with CLOCK_MONOTONIC */
- if (futex_get_abs_timeout(CLOCK_MONOTONIC, &to, timeout_ns))
- ksft_test_result_error("get_time error");
+ GET_ABS_TIMEOUT(CLOCK_MONOTONIC, &to, timeout_ns);
res = futex_wait_bitset(&f1, f1, &to, 1, 0);
- test_timeout(res, "futex_wait_bitset monotonic", ETIMEDOUT);
+ TEST_TIMEOUT(res, "futex_wait_bitset monotonic", ETIMEDOUT);
}
TEST(requeue_pi)
@@ -112,17 +104,14 @@ TEST(requeue_pi)
int res;
/* FUTEX_WAIT_REQUEUE_PI with CLOCK_REALTIME */
- if (futex_get_abs_timeout(CLOCK_REALTIME, &to, timeout_ns))
- ksft_test_result_error("get_time error");
+ GET_ABS_TIMEOUT(CLOCK_REALTIME, &to, timeout_ns);
res = futex_wait_requeue_pi(&f1, f1, &futex_pi, &to, FUTEX_CLOCK_REALTIME);
- test_timeout(res, "futex_wait_requeue_pi realtime", ETIMEDOUT);
+ TEST_TIMEOUT(res, "futex_wait_requeue_pi realtime", ETIMEDOUT);
/* FUTEX_WAIT_REQUEUE_PI with CLOCK_MONOTONIC */
- if (futex_get_abs_timeout(CLOCK_MONOTONIC, &to, timeout_ns))
- ksft_test_result_error("get_time error");
+ GET_ABS_TIMEOUT(CLOCK_MONOTONIC, &to, timeout_ns);
res = futex_wait_requeue_pi(&f1, f1, &futex_pi, &to, 0);
- test_timeout(res, "futex_wait_requeue_pi monotonic", ETIMEDOUT);
-
+ TEST_TIMEOUT(res, "futex_wait_requeue_pi monotonic", ETIMEDOUT);
}
TEST(lock_pi)
@@ -133,7 +122,8 @@ TEST(lock_pi)
/* Create a thread that will lock forever so any waiter will timeout */
pthread_barrier_init(&barrier, NULL, 2);
- pthread_create(&thread, NULL, get_pi_lock, NULL);
+ ASSERT_EQ(pthread_create(&thread, NULL, get_pi_lock, _metadata), 0)
+ TH_LOG("pthread_create failed");
/* Wait until the other thread calls futex_lock_pi() */
pthread_barrier_wait(&barrier);
@@ -149,14 +139,13 @@ TEST(lock_pi)
* time or your time machine) the monotonic clock value is always
* smaller than realtime and the syscall will timeout immediately.
*/
- if (futex_get_abs_timeout(CLOCK_REALTIME, &to, timeout_ns))
- ksft_test_result_error("get_time error");
+ GET_ABS_TIMEOUT(CLOCK_REALTIME, &to, timeout_ns);
res = futex_lock_pi(&futex_pi, &to, 0, 0);
- test_timeout(res, "futex_lock_pi realtime", ETIMEDOUT);
+ TEST_TIMEOUT(res, "futex_lock_pi realtime", ETIMEDOUT);
/* Test operations that don't support FUTEX_CLOCK_REALTIME */
res = futex_lock_pi(&futex_pi, NULL, 0, FUTEX_CLOCK_REALTIME);
- test_timeout(res, "futex_lock_pi invalid timeout flag", ENOSYS);
+ TEST_TIMEOUT(res, "futex_lock_pi invalid timeout flag", ENOSYS);
}
TEST(waitv)
@@ -171,17 +160,18 @@ TEST(waitv)
struct timespec to;
int res;
+ if (!is_futex_waitv_supported())
+ SKIP(return, "futex_waitv syscall not supported");
+
/* futex_waitv with CLOCK_MONOTONIC */
- if (futex_get_abs_timeout(CLOCK_MONOTONIC, &to, timeout_ns))
- ksft_test_result_error("get_time error");
+ GET_ABS_TIMEOUT(CLOCK_MONOTONIC, &to, timeout_ns);
res = futex_waitv(&waitv, 1, 0, &to, CLOCK_MONOTONIC);
- test_timeout(res, "futex_waitv monotonic", ETIMEDOUT);
+ TEST_TIMEOUT(res, "futex_waitv monotonic", ETIMEDOUT);
/* futex_waitv with CLOCK_REALTIME */
- if (futex_get_abs_timeout(CLOCK_REALTIME, &to, timeout_ns))
- ksft_test_result_error("get_time error");
+ GET_ABS_TIMEOUT(CLOCK_REALTIME, &to, timeout_ns);
res = futex_waitv(&waitv, 1, 0, &to, CLOCK_REALTIME);
- test_timeout(res, "futex_waitv realtime", ETIMEDOUT);
+ TEST_TIMEOUT(res, "futex_waitv realtime", ETIMEDOUT);
}
TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/futex/functional/futex_wait_uninitialized_heap.c b/tools/testing/selftests/futex/functional/futex_wait_uninitialized_heap.c
index b07d68a67f31..bbffc23e0006 100644
--- a/tools/testing/selftests/futex/functional/futex_wait_uninitialized_heap.c
+++ b/tools/testing/selftests/futex/functional/futex_wait_uninitialized_heap.c
@@ -17,17 +17,18 @@
*
*****************************************************************************/
+#include <errno.h>
+#include <libgen.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <linux/futex.h>
#include <sys/mman.h>
#include <syscall.h>
#include <sys/types.h>
#include <sys/stat.h>
-#include <unistd.h>
-#include <errno.h>
-#include <linux/futex.h>
-#include <libgen.h>
#include "futextest.h"
#include "kselftest_harness.h"
@@ -40,6 +41,7 @@ void *buf;
void *wait_thread(void *arg)
{
+ struct __test_metadata *_metadata = (struct __test_metadata *)arg;
int res;
child_ret = true;
@@ -47,7 +49,8 @@ void *wait_thread(void *arg)
child_blocked = 0;
if (res != 0 && errno != EWOULDBLOCK) {
- ksft_exit_fail_msg("futex failure\n");
+ EXPECT_EQ(res, 0)
+ TH_LOG("futex failure: %s", strerror(errno));
child_ret = false;
}
pthread_exit(NULL);
@@ -63,21 +66,23 @@ TEST(futex_wait_uninitialized_heap)
buf = mmap(NULL, page_size, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);
- if (buf == (void *)-1)
- ksft_exit_fail_msg("mmap\n");
+ ASSERT_NE(buf, MAP_FAILED)
+ TH_LOG("mmap failed: %s", strerror(errno));
- ret = pthread_create(&thr, NULL, wait_thread, NULL);
- if (ret)
- ksft_exit_fail_msg("pthread_create\n");
+ ret = pthread_create(&thr, NULL, wait_thread, _metadata);
+ ASSERT_EQ(ret, 0)
+ TH_LOG("pthread_create failed");
- ksft_print_dbg_msg("waiting %dus for child to return\n", WAIT_US);
+ TH_LOG("waiting %dus for child to return", WAIT_US);
usleep(WAIT_US);
- if (child_blocked)
- ksft_test_result_fail("child blocked in kernel\n");
+ EXPECT_EQ(child_blocked, 0)
+ TH_LOG("child blocked in kernel");
+ EXPECT_TRUE(child_ret)
+ TH_LOG("child error");
- if (!child_ret)
- ksft_test_result_fail("child error\n");
+ pthread_join(thr, NULL);
+ munmap(buf, page_size);
}
TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/futex/functional/futex_wait_wouldblock.c b/tools/testing/selftests/futex/functional/futex_wait_wouldblock.c
index 9ff936ecf164..ab039dda3e51 100644
--- a/tools/testing/selftests/futex/functional/futex_wait_wouldblock.c
+++ b/tools/testing/selftests/futex/functional/futex_wait_wouldblock.c
@@ -28,20 +28,20 @@
#define timeout_ns 100000
+
TEST(futex_wait_wouldblock)
{
struct timespec to = {.tv_sec = 0, .tv_nsec = timeout_ns};
futex_t f1 = FUTEX_INITIALIZER;
int res;
- ksft_print_dbg_msg("Calling futex_wait on f1: %u @ %p with val=%u\n", f1, &f1, f1+1);
+ TH_LOG("Calling futex_wait on f1: %u @ %p with val=%u", f1, &f1, f1+1);
res = futex_wait(&f1, f1+1, &to, FUTEX_PRIVATE_FLAG);
- if (!res || errno != EWOULDBLOCK) {
- ksft_test_result_fail("futex_wait returned: %d %s\n",
- res ? errno : res,
- res ? strerror(errno) : "");
- } else {
- ksft_test_result_pass("futex_wait\n");
+ EXPECT_EQ(res, -1)
+ TH_LOG("futex_wait returned unexpected result: %d", res);
+ if (res == -1) {
+ EXPECT_EQ(errno, EWOULDBLOCK)
+ TH_LOG("futex_wait returned unexpected errno: %d", errno);
}
}
@@ -57,8 +57,11 @@ TEST(futex_waitv_wouldblock)
};
int res;
- if (clock_gettime(CLOCK_MONOTONIC, &to))
- ksft_exit_fail_msg("clock_gettime failed %d\n", errno);
+ if (!is_futex_waitv_supported())
+ SKIP(return, "futex_waitv syscall not supported");
+
+ ASSERT_EQ(clock_gettime(CLOCK_MONOTONIC, &to), 0)
+ TH_LOG("clock_gettime failed");
to.tv_nsec += timeout_ns;
@@ -67,14 +70,13 @@ TEST(futex_waitv_wouldblock)
to.tv_nsec -= 1000000000;
}
- ksft_print_dbg_msg("Calling futex_waitv on f1: %u @ %p with val=%u\n", f1, &f1, f1+1);
+ TH_LOG("Calling futex_waitv on f1: %u @ %p with val=%u", f1, &f1, f1+1);
res = futex_waitv(&waitv, 1, 0, &to, CLOCK_MONOTONIC);
- if (!res || errno != EWOULDBLOCK) {
- ksft_test_result_fail("futex_waitv returned: %d %s\n",
- res ? errno : res,
- res ? strerror(errno) : "");
- } else {
- ksft_test_result_pass("futex_waitv\n");
+ EXPECT_EQ(res, -1)
+ TH_LOG("futex_waitv returned unexpected result: %d", res);
+ if (res == -1) {
+ EXPECT_EQ(errno, EWOULDBLOCK)
+ TH_LOG("futex_waitv returned unexpected errno: %d", errno);
}
}
diff --git a/tools/testing/selftests/futex/functional/futex_waitv.c b/tools/testing/selftests/futex/functional/futex_waitv.c
index b5ada9fdb26f..4858d5faeecf 100644
--- a/tools/testing/selftests/futex/functional/futex_waitv.c
+++ b/tools/testing/selftests/futex/functional/futex_waitv.c
@@ -25,24 +25,26 @@
static struct futex_waitv waitv[NR_FUTEXES];
u_int32_t futexes[NR_FUTEXES] = {0};
+
void *waiterfn(void *arg)
{
+ struct __test_metadata *_metadata = (struct __test_metadata *)arg;
struct timespec to;
int res;
/* setting absolute timeout for futex2 */
- if (clock_gettime(CLOCK_MONOTONIC, &to))
- ksft_exit_fail_msg("gettime64 failed\n");
+ ASSERT_EQ(clock_gettime(CLOCK_MONOTONIC, &to), 0)
+ TH_LOG("gettime64 failed");
to.tv_sec++;
res = futex_waitv(waitv, NR_FUTEXES, 0, &to, CLOCK_MONOTONIC);
if (res < 0) {
- ksft_test_result_fail("futex_waitv returned: %d %s\n",
- errno, strerror(errno));
- } else if (res != NR_FUTEXES - 1) {
- ksft_test_result_fail("futex_waitv returned: %d, expecting %d\n",
- res, NR_FUTEXES - 1);
+ EXPECT_EQ(res, NR_FUTEXES - 1)
+ TH_LOG("futex_waitv failed: %s", strerror(errno));
+ } else {
+ EXPECT_EQ(res, NR_FUTEXES - 1)
+ TH_LOG("futex_waitv returned %d, expected %d", res, NR_FUTEXES - 1);
}
return NULL;
@@ -53,6 +55,9 @@ TEST(private_waitv)
pthread_t waiter;
int res, i;
+ if (!is_futex_waitv_supported())
+ SKIP(return, "futex_waitv syscall not supported");
+
for (i = 0; i < NR_FUTEXES; i++) {
waitv[i].uaddr = (uintptr_t)&futexes[i];
waitv[i].flags = FUTEX_32 | FUTEX_PRIVATE_FLAG;
@@ -61,19 +66,14 @@ TEST(private_waitv)
}
/* Private waitv */
- if (pthread_create(&waiter, NULL, waiterfn, NULL))
- ksft_exit_fail_msg("pthread_create failed\n");
+ ASSERT_EQ(pthread_create(&waiter, NULL, waiterfn, _metadata), 0)
+ TH_LOG("pthread_create failed");
usleep(WAKE_WAIT_US);
res = futex_wake(u64_to_ptr(waitv[NR_FUTEXES - 1].uaddr), 1, FUTEX_PRIVATE_FLAG);
- if (res != 1) {
- ksft_test_result_fail("futex_wake private returned: %d %s\n",
- res ? errno : res,
- res ? strerror(errno) : "");
- } else {
- ksft_test_result_pass("futex_waitv private\n");
- }
+ EXPECT_EQ(res, 1)
+ TH_LOG("futex_wake private returned: %d %s", res, res < 0 ? strerror(errno) : "");
}
TEST(shared_waitv)
@@ -81,15 +81,18 @@ TEST(shared_waitv)
pthread_t waiter;
int res, i;
+ if (!is_futex_waitv_supported())
+ SKIP(return, "futex_waitv syscall not supported");
+
/* Shared waitv */
for (i = 0; i < NR_FUTEXES; i++) {
int shm_id = shmget(IPC_PRIVATE, 4096, IPC_CREAT | 0666);
if (shm_id < 0) {
if (errno == ENOSYS)
- ksft_exit_skip("shmget syscall not supported\n");
- perror("shmget");
- exit(1);
+ SKIP(return, "shmget syscall not supported");
+ ASSERT_GE(shm_id, 0)
+ TH_LOG("shmget failed");
}
unsigned int *shared_data = shmat(shm_id, NULL, 0);
@@ -101,19 +104,14 @@ TEST(shared_waitv)
waitv[i].__reserved = 0;
}
- if (pthread_create(&waiter, NULL, waiterfn, NULL))
- ksft_exit_fail_msg("pthread_create failed\n");
+ ASSERT_EQ(pthread_create(&waiter, NULL, waiterfn, _metadata), 0)
+ TH_LOG("pthread_create failed");
usleep(WAKE_WAIT_US);
res = futex_wake(u64_to_ptr(waitv[NR_FUTEXES - 1].uaddr), 1, 0);
- if (res != 1) {
- ksft_test_result_fail("futex_wake shared returned: %d %s\n",
- res ? errno : res,
- res ? strerror(errno) : "");
- } else {
- ksft_test_result_pass("futex_waitv shared\n");
- }
+ EXPECT_EQ(res, 1)
+ TH_LOG("futex_wake shared returned: %d %s", res, res < 0 ? strerror(errno) : "");
for (i = 0; i < NR_FUTEXES; i++)
shmdt(u64_to_ptr(waitv[i].uaddr));
@@ -124,21 +122,24 @@ TEST(invalid_flag)
struct timespec to;
int res;
+ if (!is_futex_waitv_supported())
+ SKIP(return, "futex_waitv syscall not supported");
+
/* Testing a waiter without FUTEX_32 flag */
waitv[0].flags = FUTEX_PRIVATE_FLAG;
- if (clock_gettime(CLOCK_MONOTONIC, &to))
- ksft_exit_fail_msg("gettime64 failed\n");
+ ASSERT_EQ(clock_gettime(CLOCK_MONOTONIC, &to), 0)
+ TH_LOG("gettime64 failed");
to.tv_sec++;
res = futex_waitv(waitv, NR_FUTEXES, 0, &to, CLOCK_MONOTONIC);
- if (res == EINVAL) {
- ksft_test_result_fail("futex_waitv private returned: %d %s\n",
- res ? errno : res,
- res ? strerror(errno) : "");
- } else {
- ksft_test_result_pass("futex_waitv without FUTEX_32\n");
+
+ EXPECT_EQ(res, -1)
+ TH_LOG("futex_waitv returned unexpected result: %d", res);
+ if (res == -1) {
+ EXPECT_EQ(errno, EINVAL)
+ TH_LOG("futex_waitv returned unexpected errno: %d", errno);
}
}
@@ -147,22 +148,25 @@ TEST(unaligned_address)
struct timespec to;
int res;
+ if (!is_futex_waitv_supported())
+ SKIP(return, "futex_waitv syscall not supported");
+
/* Testing a waiter with an unaligned address */
waitv[0].flags = FUTEX_PRIVATE_FLAG | FUTEX_32;
waitv[0].uaddr = 1;
- if (clock_gettime(CLOCK_MONOTONIC, &to))
- ksft_exit_fail_msg("gettime64 failed\n");
+ ASSERT_EQ(clock_gettime(CLOCK_MONOTONIC, &to), 0)
+ TH_LOG("gettime64 failed");
to.tv_sec++;
res = futex_waitv(waitv, NR_FUTEXES, 0, &to, CLOCK_MONOTONIC);
- if (res == EINVAL) {
- ksft_test_result_fail("futex_wake private returned: %d %s\n",
- res ? errno : res,
- res ? strerror(errno) : "");
- } else {
- ksft_test_result_pass("futex_waitv with an unaligned address\n");
+
+ EXPECT_EQ(res, -1)
+ TH_LOG("futex_waitv returned unexpected result: %d", res);
+ if (res == -1) {
+ EXPECT_EQ(errno, EINVAL)
+ TH_LOG("futex_waitv returned unexpected errno: %d", errno);
}
}
@@ -171,36 +175,39 @@ TEST(null_address)
struct timespec to;
int res;
+ if (!is_futex_waitv_supported())
+ SKIP(return, "futex_waitv syscall not supported");
+
/* Testing a NULL address for waiters.uaddr */
waitv[0].uaddr = 0x00000000;
- if (clock_gettime(CLOCK_MONOTONIC, &to))
- ksft_exit_fail_msg("gettime64 failed\n");
+ ASSERT_EQ(clock_gettime(CLOCK_MONOTONIC, &to), 0)
+ TH_LOG("gettime64 failed");
to.tv_sec++;
res = futex_waitv(waitv, NR_FUTEXES, 0, &to, CLOCK_MONOTONIC);
- if (res == EINVAL) {
- ksft_test_result_fail("futex_waitv private returned: %d %s\n",
- res ? errno : res,
- res ? strerror(errno) : "");
- } else {
- ksft_test_result_pass("futex_waitv NULL address in waitv.uaddr\n");
+
+ EXPECT_EQ(res, -1)
+ TH_LOG("futex_waitv returned unexpected result: %d", res);
+ if (res == -1) {
+ EXPECT_EQ(errno, EINVAL)
+ TH_LOG("futex_waitv returned unexpected errno: %d", errno);
}
/* Testing a NULL address for *waiters */
- if (clock_gettime(CLOCK_MONOTONIC, &to))
- ksft_exit_fail_msg("gettime64 failed\n");
+ ASSERT_EQ(clock_gettime(CLOCK_MONOTONIC, &to), 0)
+ TH_LOG("gettime64 failed");
to.tv_sec++;
res = futex_waitv(NULL, NR_FUTEXES, 0, &to, CLOCK_MONOTONIC);
- if (res == EINVAL) {
- ksft_test_result_fail("futex_waitv private returned: %d %s\n",
- res ? errno : res,
- res ? strerror(errno) : "");
- } else {
- ksft_test_result_pass("futex_waitv NULL address in *waiters\n");
+
+ EXPECT_EQ(res, -1)
+ TH_LOG("futex_waitv returned unexpected result: %d", res);
+ if (res == -1) {
+ EXPECT_EQ(errno, EINVAL)
+ TH_LOG("futex_waitv returned unexpected errno: %d", errno);
}
}
@@ -209,19 +216,22 @@ TEST(invalid_clockid)
struct timespec to;
int res;
+ if (!is_futex_waitv_supported())
+ SKIP(return, "futex_waitv syscall not supported");
+
/* Testing an invalid clockid */
- if (clock_gettime(CLOCK_MONOTONIC, &to))
- ksft_exit_fail_msg("gettime64 failed\n");
+ ASSERT_EQ(clock_gettime(CLOCK_MONOTONIC, &to), 0)
+ TH_LOG("gettime64 failed");
to.tv_sec++;
res = futex_waitv(NULL, NR_FUTEXES, 0, &to, CLOCK_TAI);
- if (res == EINVAL) {
- ksft_test_result_fail("futex_waitv private returned: %d %s\n",
- res ? errno : res,
- res ? strerror(errno) : "");
- } else {
- ksft_test_result_pass("futex_waitv invalid clockid\n");
+
+ EXPECT_EQ(res, -1)
+ TH_LOG("futex_waitv returned unexpected result: %d", res);
+ if (res == -1) {
+ EXPECT_EQ(errno, EINVAL)
+ TH_LOG("futex_waitv returned unexpected errno: %d", errno);
}
}
diff --git a/tools/testing/selftests/futex/functional/robust_list.c b/tools/testing/selftests/futex/functional/robust_list.c
index b3fab60181d5..87217c549361 100644
--- a/tools/testing/selftests/futex/functional/robust_list.c
+++ b/tools/testing/selftests/futex/functional/robust_list.c
@@ -25,24 +25,24 @@
#define _GNU_SOURCE
#include "futextest.h"
-#include "../../kselftest_harness.h"
+#include "kselftest_harness.h"
#include <dlfcn.h>
#include <errno.h>
#include <pthread.h>
#include <signal.h>
-#include <stdint.h>
#include <stdatomic.h>
#include <stdbool.h>
#include <stddef.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
#include <sys/auxv.h>
#include <sys/mman.h>
#include <sys/wait.h>
#define STACK_SIZE (1024 * 1024)
-
#define FUTEX_TIMEOUT 3
-
#define SLEEP_US 100
#if __SIZEOF_LONG__ == 8
@@ -71,30 +71,46 @@ static int sys_futex_robust_unlock(_Atomic(uint32_t) *uaddr, unsigned int op, in
* Basic lock struct, contains just the futex word and the robust list element
* Real implementations have also a *prev to easily walk in the list
*/
+typedef _Atomic(unsigned int) atomic_futex_t;
+
struct lock_struct {
- _Atomic(unsigned int) futex;
+ atomic_futex_t futex;
struct robust_list list;
};
+struct child_args {
+ struct __test_metadata *_metadata;
+ void *arg;
+};
+
/*
* Helper function to spawn a child thread. Returns -1 on error, pid on success
*/
-static int create_child(int (*fn)(void *arg), void *arg)
+static int create_child(struct __test_metadata *_metadata, int (*fn)(void *arg), void *arg)
{
+ struct child_args *cargs = malloc(sizeof(*cargs));
char *stack;
pid_t pid;
+ if (!cargs)
+ return -1;
+ cargs->_metadata = _metadata;
+ cargs->arg = arg;
+
stack = mmap(NULL, STACK_SIZE, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_STACK, -1, 0);
- if (stack == MAP_FAILED)
+ if (stack == MAP_FAILED) {
+ free(cargs);
return -1;
+ }
stack += STACK_SIZE;
- pid = clone(fn, stack, CLONE_VM | SIGCHLD, arg);
-
- if (pid == -1)
+ pid = clone(fn, stack, CLONE_VM | SIGCHLD, cargs);
+ if (pid == -1) {
+ free(cargs);
return -1;
+ }
return pid;
}
@@ -123,7 +139,7 @@ static int set_list(struct robust_list_head *head)
*/
static int mutex_lock(struct lock_struct *lock, struct robust_list_head *head, bool error_inject)
{
- _Atomic(unsigned int) *futex = &lock->futex;
+ atomic_futex_t *futex = &lock->futex;
unsigned int zero = 0;
pid_t tid = gettid();
int ret = -1;
@@ -183,21 +199,21 @@ static int mutex_lock(struct lock_struct *lock, struct robust_list_head *head, b
*/
static int child_fn_lock(void *arg)
{
- struct lock_struct *lock = arg;
+ struct child_args *cargs = arg;
+ struct __test_metadata *_metadata = cargs->_metadata;
+ struct lock_struct *lock = cargs->arg;
struct robust_list_head head;
int ret;
+ free(cargs);
+
ret = set_list(&head);
- if (ret) {
- ksft_test_result_fail("set_robust_list error\n");
- return ret;
- }
+ ASSERT_EQ(ret, 0)
+ TH_LOG("set_robust_list error");
ret = mutex_lock(lock, &head, false);
- if (ret) {
- ksft_test_result_fail("mutex_lock error\n");
- return ret;
- }
+ ASSERT_EQ(ret, 0)
+ TH_LOG("mutex_lock error");
pthread_barrier_wait(&barrier);
@@ -220,7 +236,7 @@ static int child_fn_lock(void *arg)
TEST(test_robustness)
{
struct lock_struct lock = { .futex = 0 };
- _Atomic(unsigned int) *futex = &lock.futex;
+ atomic_futex_t *futex = &lock.futex;
struct robust_list_head head;
int ret, pid, wstatus;
@@ -234,7 +250,7 @@ TEST(test_robustness)
ret = pthread_barrier_init(&barrier, NULL, 2);
ASSERT_EQ(ret, 0);
- pid = create_child(&child_fn_lock, &lock);
+ pid = create_child(_metadata, &child_fn_lock, &lock);
ASSERT_NE(pid, -1);
pthread_barrier_wait(&barrier);
@@ -251,9 +267,8 @@ TEST(test_robustness)
wait(&wstatus);
pthread_barrier_destroy(&barrier);
- /* Pass only if the child hasn't return error */
- if (!WEXITSTATUS(wstatus))
- ksft_test_result_pass("%s\n", __func__);
+ EXPECT_EQ(WEXITSTATUS(wstatus), 0)
+ TH_LOG("child failed");
}
/*
@@ -279,8 +294,6 @@ TEST(test_set_robust_list_invalid_size)
ret = set_robust_list(&head, 0);
ASSERT_EQ(ret, -1);
ASSERT_EQ(errno, EINVAL);
-
- ksft_test_result_pass("%s\n", __func__);
}
/*
@@ -307,20 +320,20 @@ TEST(test_get_robust_list_self)
ASSERT_EQ(ret, 0);
ASSERT_EQ(get_head, &head2);
ASSERT_EQ(head_size, len_ptr);
-
- ksft_test_result_pass("%s\n", __func__);
}
static int child_list(void *arg)
{
- struct robust_list_head *head = arg;
+ struct child_args *cargs = arg;
+ struct __test_metadata *_metadata = cargs->_metadata;
+ struct robust_list_head *head = cargs->arg;
int ret;
+ free(cargs);
+
ret = set_robust_list(head, sizeof(*head));
- if (ret) {
- ksft_test_result_fail("set_robust_list error\n");
- return -1;
- }
+ ASSERT_EQ(ret, 0)
+ TH_LOG("set_robust_list error");
/*
* After setting the list head, wait until the main thread can call
@@ -350,7 +363,7 @@ TEST(test_get_robust_list_child)
ret = pthread_barrier_init(&barrier2, NULL, 2);
ASSERT_EQ(ret, 0);
- tid = create_child(&child_list, &head);
+ tid = create_child(_metadata, &child_list, &head);
ASSERT_NE(tid, -1);
pthread_barrier_wait(&barrier);
@@ -365,28 +378,27 @@ TEST(test_get_robust_list_child)
pthread_barrier_destroy(&barrier);
pthread_barrier_destroy(&barrier2);
- /* Pass only if the child hasn't return error */
- if (!WEXITSTATUS(wstatus))
- ksft_test_result_pass("%s\n", __func__);
+ EXPECT_EQ(WEXITSTATUS(wstatus), 0)
+ TH_LOG("child failed");
}
static int child_fn_lock_with_error(void *arg)
{
- struct lock_struct *lock = arg;
+ struct child_args *cargs = arg;
+ struct __test_metadata *_metadata = cargs->_metadata;
+ struct lock_struct *lock = cargs->arg;
struct robust_list_head head;
int ret;
+ free(cargs);
+
ret = set_list(&head);
- if (ret) {
- ksft_test_result_fail("set_robust_list error\n");
- return -1;
- }
+ ASSERT_EQ(ret, 0)
+ TH_LOG("set_robust_list error");
ret = mutex_lock(lock, &head, true);
- if (ret) {
- ksft_test_result_fail("mutex_lock error\n");
- return -1;
- }
+ ASSERT_EQ(ret, 0)
+ TH_LOG("mutex_lock error");
pthread_barrier_wait(&barrier);
@@ -404,7 +416,7 @@ static int child_fn_lock_with_error(void *arg)
TEST(test_set_list_op_pending)
{
struct lock_struct lock = { .futex = 0 };
- _Atomic(unsigned int) *futex = &lock.futex;
+ atomic_futex_t *futex = &lock.futex;
struct robust_list_head head;
int ret, wstatus;
@@ -414,7 +426,7 @@ TEST(test_set_list_op_pending)
ret = pthread_barrier_init(&barrier, NULL, 2);
ASSERT_EQ(ret, 0);
- ret = create_child(&child_fn_lock_with_error, &lock);
+ ret = create_child(_metadata, &child_fn_lock_with_error, &lock);
ASSERT_NE(ret, -1);
pthread_barrier_wait(&barrier);
@@ -427,21 +439,21 @@ TEST(test_set_list_op_pending)
wait(&wstatus);
pthread_barrier_destroy(&barrier);
- /* Pass only if the child hasn't return error */
- if (!WEXITSTATUS(wstatus))
- ksft_test_result_pass("%s\n", __func__);
- else
- ksft_test_result_fail("%s\n", __func__);
+ EXPECT_EQ(WEXITSTATUS(wstatus), 0)
+ TH_LOG("child failed");
}
#define CHILD_NR 10
static int child_lock_holder(void *arg)
{
- struct lock_struct *locks = arg;
+ struct child_args *cargs = arg;
+ struct lock_struct *locks = cargs->arg;
struct robust_list_head head;
int i;
+ free(cargs);
+
set_list(&head);
for (i = 0; i < CHILD_NR; i++) {
@@ -460,22 +472,21 @@ static int child_lock_holder(void *arg)
static int child_wait_lock(void *arg)
{
- struct lock_struct *lock = arg;
+ struct child_args *cargs = arg;
+ struct __test_metadata *_metadata = cargs->_metadata;
+ struct lock_struct *lock = cargs->arg;
struct robust_list_head head;
int ret;
+ free(cargs);
+
pthread_barrier_wait(&barrier2);
ret = mutex_lock(lock, &head, false);
+ ASSERT_EQ(ret, 0)
+ TH_LOG("mutex_lock error");
- if (ret) {
- ksft_test_result_fail("mutex_lock error\n");
- return -1;
- }
-
- if (!(lock->futex & FUTEX_OWNER_DIED)) {
- ksft_test_result_fail("futex not marked with FUTEX_OWNER_DIED\n");
- return -1;
- }
+ ASSERT_TRUE(lock->futex & FUTEX_OWNER_DIED)
+ TH_LOG("futex not marked with FUTEX_OWNER_DIED");
return 0;
}
@@ -495,18 +506,20 @@ TEST(test_robust_list_multiple_elements)
ret = pthread_barrier_init(&barrier2, NULL, CHILD_NR + 1);
ASSERT_EQ(ret, 0);
- pids[0] = create_child(&child_lock_holder, &locks);
+ pids[0] = create_child(_metadata, &child_lock_holder, &locks);
+ ASSERT_NE(pids[0], -1);
/* Wait until the locker thread takes the look */
pthread_barrier_wait(&barrier);
- for (i = 0; i < CHILD_NR; i++)
- pids[i+1] = create_child(&child_wait_lock, &locks[i]);
+ for (i = 0; i < CHILD_NR; i++) {
+ pids[i+1] = create_child(_metadata, &child_wait_lock, &locks[i]);
+ ASSERT_NE(pids[i+1], -1);
+ }
- /* Wait for all children to return */
+ /* Wait for all children to return (holder + all waiters) */
ret = 0;
-
- for (i = 0; i < CHILD_NR; i++) {
+ for (i = 0; i < CHILD_NR + 1; i++) {
waitpid(pids[i], &wstatus, 0);
if (WEXITSTATUS(wstatus))
ret = -1;
@@ -515,22 +528,23 @@ TEST(test_robust_list_multiple_elements)
pthread_barrier_destroy(&barrier);
pthread_barrier_destroy(&barrier2);
- /* Pass only if the child hasn't return error */
- if (!ret)
- ksft_test_result_pass("%s\n", __func__);
+ EXPECT_EQ(ret, 0)
+ TH_LOG("One or more children failed");
}
static int child_circular_list(void *arg)
{
- static struct robust_list_head head;
- struct lock_struct a, b, c;
+ struct child_args *cargs = arg;
+ struct __test_metadata *_metadata = cargs->_metadata;
+ static struct lock_struct a, b, c;
+ struct robust_list_head head;
int ret;
+ free(cargs);
+
ret = set_list(&head);
- if (ret) {
- ksft_test_result_fail("set_list error\n");
- return -1;
- }
+ ASSERT_EQ(ret, 0)
+ TH_LOG("set_list error");
head.list.next = &a.list;
@@ -552,14 +566,15 @@ static int child_circular_list(void *arg)
TEST(test_circular_list)
{
int wstatus;
+ pid_t pid;
- create_child(child_circular_list, NULL);
+ pid = create_child(_metadata, child_circular_list, NULL);
+ ASSERT_NE(pid, -1);
wait(&wstatus);
- /* Pass only if the child hasn't return error */
- if (!WEXITSTATUS(wstatus))
- ksft_test_result_pass("%s\n", __func__);
+ EXPECT_EQ(WEXITSTATUS(wstatus), 0)
+ TH_LOG("child failed");
}
/*
diff --git a/tools/testing/selftests/futex/functional/run.sh b/tools/testing/selftests/futex/functional/run.sh
index e88545c06d57..d1a681b798bd 100755
--- a/tools/testing/selftests/futex/functional/run.sh
+++ b/tools/testing/selftests/futex/functional/run.sh
@@ -51,3 +51,6 @@ echo
echo
./futex_numa_mpol
+
+echo
+./futex_lock_pi_exiting
diff --git a/tools/testing/selftests/futex/include/futex2test.h b/tools/testing/selftests/futex/include/futex2test.h
index 1f625b39948a..53e88b60ac6d 100644
--- a/tools/testing/selftests/futex/include/futex2test.h
+++ b/tools/testing/selftests/futex/include/futex2test.h
@@ -5,7 +5,9 @@
* Copyright 2021 Collabora Ltd.
*/
#include <linux/time_types.h>
+#include <errno.h>
#include <stdint.h>
+#include <stdbool.h>
#define u64_to_ptr(x) ((void *)(uintptr_t)(x))
@@ -96,3 +98,11 @@ static inline int futex2_wake(void *uaddr, int nr, unsigned int flags)
{
return syscall(__NR_futex_wake, uaddr, ~0U, nr, flags);
}
+
+static inline bool is_futex_waitv_supported(void)
+{
+ struct timespec ts = {0, 0};
+ int res = futex_waitv(NULL, 0, 0, &ts, CLOCK_MONOTONIC);
+
+ return !(res < 0 && errno == ENOSYS);
+}
diff --git a/tools/testing/selftests/futex/include/futex_thread.h b/tools/testing/selftests/futex/include/futex_thread.h
new file mode 100644
index 000000000000..a90882960264
--- /dev/null
+++ b/tools/testing/selftests/futex/include/futex_thread.h
@@ -0,0 +1,117 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef _FUTEX_THREAD_H
+#define _FUTEX_THREAD_H
+#include <errno.h>
+#include <pthread.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+
+#include "kselftest_harness.h"
+
+#define USEC_PER_SEC 1000000L
+#define WAIT_FOR_THREAD_SECS 1
+#define WAIT_FOR_THREAD_USECS (WAIT_FOR_THREAD_SECS * USEC_PER_SEC)
+#define WAIT_THREAD_RETRIES 100
+
+struct futex_thread {
+ pthread_t thread;
+ pthread_barrier_t barrier;
+ pid_t tid;
+ int (*threadfn)(void *arg);
+ void *arg;
+ int retval;
+};
+
+static inline int __wait_for_thread(FILE *fp, struct __test_metadata *_metadata)
+{
+ unsigned int sleep_time_us = WAIT_FOR_THREAD_USECS / WAIT_THREAD_RETRIES;
+ char buf[80] = "";
+
+ for (int i = 0; i < WAIT_THREAD_RETRIES; i++) {
+ if (!fgets(buf, sizeof(buf), fp))
+ return EIO;
+ if (!strncmp(buf, "futex", 5))
+ return 0;
+ usleep(sleep_time_us);
+ rewind(fp);
+ }
+
+ TH_LOG("/proc/$PID/wchan contains \"%s\". Trying to continue.", buf);
+ return 0;
+}
+
+static void *__futex_thread_fn(void *arg)
+{
+ struct futex_thread *t = arg;
+
+ t->tid = gettid();
+ pthread_barrier_wait(&t->barrier);
+ t->retval = t->threadfn(t->arg);
+ return NULL;
+}
+
+/**
+ * futex_wait_for_thread - Wait for the child thread to sleep in the futex context
+ * @t: Thread handle.
+ * @_metadata: Test metadata for TH_LOG() context
+ */
+static inline int futex_wait_for_thread(struct futex_thread *t, struct __test_metadata *_metadata)
+{
+ char fname[80];
+ FILE *fp;
+ int res;
+
+ snprintf(fname, sizeof(fname), "/proc/%d/wchan", t->tid);
+ fp = fopen(fname, "r");
+ if (!fp) {
+ /* If /proc/... is not available, sleep */
+ if (errno != ENOENT)
+ return errno;
+ TH_LOG("/proc/$PID/wchan not accessible, continue with sleep()");
+ sleep(WAIT_FOR_THREAD_SECS);
+ return 0;
+ }
+
+ res = __wait_for_thread(fp, _metadata);
+ fclose(fp);
+ return res;
+}
+
+/**
+ * futex_thread_create - Create a new thread for testing.
+ * @t: The handle of the newly created thread.
+ * @threadfn: The new thread starts execution by invoking threadfn
+ * @arg: The parameters passed to threadfn.
+ */
+static inline int futex_thread_create(struct futex_thread *t, int (*threadfn)(void *), void *arg)
+{
+ pthread_barrier_init(&t->barrier, NULL, 2);
+
+ t->tid = 0;
+ t->threadfn = threadfn;
+ t->arg = arg;
+
+ if (pthread_create(&t->thread, NULL, __futex_thread_fn, t) < 0) {
+ int ret = errno;
+ pthread_barrier_destroy(&t->barrier);
+ return ret;
+ }
+
+ pthread_barrier_wait(&t->barrier);
+ return 0;
+}
+
+/**
+ * futex_thread_destroy - Wait for and reclaim the resources of the thread.
+ * @t: Thread handle.
+ */
+static inline int futex_thread_destroy(struct futex_thread *t)
+{
+ pthread_join(t->thread, NULL);
+ pthread_barrier_destroy(&t->barrier);
+ return t->retval;
+}
+
+#endif
diff --git a/tools/testing/selftests/rseq/rseq-x86-thread-pointer.h b/tools/testing/selftests/rseq/rseq-x86-thread-pointer.h
index d3133587d996..5a29d6bec51f 100644
--- a/tools/testing/selftests/rseq/rseq-x86-thread-pointer.h
+++ b/tools/testing/selftests/rseq/rseq-x86-thread-pointer.h
@@ -8,13 +8,11 @@
#ifndef _RSEQ_X86_THREAD_POINTER
#define _RSEQ_X86_THREAD_POINTER
-#include <features.h>
-
#ifdef __cplusplus
extern "C" {
#endif
-#if __GNUC_PREREQ (11, 1)
+#if __GNUC__ > 11 || (__GNUC__ == 11 && __GNUC_MINOR__ >= 1)
static inline void *rseq_thread_pointer(void)
{
return __builtin_thread_pointer();
^ permalink raw reply related [flat|nested] 26+ messages in thread* [GIT pull] smp/core for v7.3-rc1
2026-08-17 11:23 [GIT pull] core/entry for v7.3-rc1 Thomas Gleixner
` (3 preceding siblings ...)
2026-08-17 11:23 ` [GIT pull] locking/futex " Thomas Gleixner
@ 2026-08-17 11:23 ` Thomas Gleixner
2026-08-19 1:10 ` pr-tracker-bot
2026-08-17 11:23 ` [GIT pull] timers/cleanups " Thomas Gleixner
` (4 subsequent siblings)
9 siblings, 1 reply; 26+ messages in thread
From: Thomas Gleixner @ 2026-08-17 11:23 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest smp/core branch from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git smp-core-2026-08-17
up to: 99b49e02f948: scftorture: Remove preempt_disable() in scftorture_invoke_one()
SMP core updates:
- Reduce the preemption disabled sections in smp_call_function*().
The various smp call functions keep preemption disabled accross the
full operation which includes the wait for completion. Especially the
latter can take some time when one of the target CPUs is not
immediately responding to the IPI, which can result in large latency
spikes.
To improve this provide a per task CPU mask to track the CPUs to wait
for. That makes the information required for the wait task local and
therefore allows to reenable preemption before the wait. While this
comes with moderate extra memory cost this reduces SMP function call
induced latency measured in a fleet for high priority tasks from ~17ms
to ~1.5ms (~90%).
- Reduce the overhead of the CSD debug code by replacing the heavy memory
barriers with smp_store_release()/acquire().
- Remove obsolute unused hotplug states
Thanks,
tglx
------------------>
Chuyi Zhou (9):
smp: Disable preemption explicitly in __csd_lock_wait()
smp: Enable preemption early in smp_call_function_single()
smp: Refactor remote CPU selection in smp_call_function_any()
smp: Use task-local IPI cpumask in smp_call_function_many_cond()
smp: Alloc percpu csd data in smpcfd_prepare_cpu() only once
smp: Enable preemption early in smp_call_function_many_cond()
smp: Remove preempt_disable() from smp_call_function()
smp: Remove preempt_disable() from on_each_cpu_cond_mask()
scftorture: Remove preempt_disable() in scftorture_invoke_one()
Usama Arif (1):
smp: Use release stores for csd_lock_record() state
Zenghui Yu (1):
cpu/hotplug: Remove CPUHP_AP_ARM_CORESIGHT_CTI_STARTING
include/linux/cpuhotplug.h | 1 -
include/linux/sched.h | 12 +++
include/linux/smp.h | 15 ++-
kernel/fork.c | 9 +-
kernel/scftorture.c | 13 +--
kernel/smp.c | 223 +++++++++++++++++++++++++++++++--------------
kernel/up.c | 3 +-
7 files changed, 195 insertions(+), 81 deletions(-)
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 0fb3a2a62eb0..feb32949aeea 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -180,7 +180,6 @@ enum cpuhp_state {
CPUHP_AP_DUMMY_TIMER_STARTING,
CPUHP_AP_ARM_XEN_STARTING,
CPUHP_AP_ARM_XEN_RUNSTATE_STARTING,
- CPUHP_AP_ARM_CORESIGHT_CTI_STARTING,
CPUHP_AP_ARM64_ISNDEP_STARTING,
CPUHP_AP_SMPCFD_DYING,
CPUHP_AP_HRTIMERS_DYING,
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 373bcc0598d1..5738c54eb85a 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -823,6 +823,17 @@ struct kmap_ctrl {
#endif
};
+#if defined(CONFIG_SMP) && defined(CONFIG_PREEMPTION)
+struct task_ipi_mask {
+ union {
+ cpumask_t *ipi_mask_ptr;
+ unsigned long ipi_mask_val;
+ };
+};
+#else
+struct task_ipi_mask { };
+#endif
+
struct task_struct {
#ifdef CONFIG_THREAD_INFO_IN_TASK
/*
@@ -1359,6 +1370,7 @@ struct task_struct {
struct list_head perf_event_list;
struct perf_ctx_data __rcu *perf_ctx_data;
#endif
+ struct task_ipi_mask __private ipi_mask;
#ifdef CONFIG_DEBUG_PREEMPT
unsigned long preempt_disable_ip;
#endif
diff --git a/include/linux/smp.h b/include/linux/smp.h
index 6925d15ccaa7..2dfa7390717a 100644
--- a/include/linux/smp.h
+++ b/include/linux/smp.h
@@ -47,8 +47,7 @@ extern void __smp_call_single_queue(int cpu, struct llist_node *node);
/* total number of cpus in this system (may exceed NR_CPUS) */
extern unsigned int total_cpus;
-int smp_call_function_single(int cpuid, smp_call_func_t func, void *info,
- int wait);
+int smp_call_function_single(int cpuid, smp_call_func_t func, void *info, bool wait);
void on_each_cpu_cond_mask(smp_cond_func_t cond_func, smp_call_func_t func,
void *info, bool wait, const struct cpumask *mask);
@@ -239,6 +238,18 @@ static inline int get_boot_cpu_id(void)
#endif /* !SMP */
+#if defined(CONFIG_PREEMPTION) && defined(CONFIG_SMP)
+int smp_task_ipi_mask_alloc(struct task_struct *task);
+void smp_task_ipi_mask_free(struct task_struct *task);
+#else
+static inline int smp_task_ipi_mask_alloc(struct task_struct *task)
+{
+ return 0;
+}
+
+static inline void smp_task_ipi_mask_free(struct task_struct *task) { }
+#endif
+
/*
* raw_smp_processor_id() - get the current (unstable) CPU id
*
diff --git a/kernel/fork.c b/kernel/fork.c
index 13e38e89a1f3..ac3fc49d90ab 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -537,6 +537,7 @@ void free_task(struct task_struct *tsk)
#endif
release_user_cpus_ptr(tsk);
scs_release(tsk);
+ smp_task_ipi_mask_free(tsk);
#ifndef CONFIG_THREAD_INFO_IN_TASK
/*
@@ -935,10 +936,14 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node)
#endif
account_kernel_stack(tsk, 1);
- err = scs_prepare(tsk, node);
+ err = smp_task_ipi_mask_alloc(tsk);
if (err)
goto free_stack;
+ err = scs_prepare(tsk, node);
+ if (err)
+ goto free_ipi_mask;
+
#ifdef CONFIG_SECCOMP
/*
* We must handle setting up seccomp filters once we're under
@@ -1011,6 +1016,8 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node)
#endif
return tsk;
+free_ipi_mask:
+ smp_task_ipi_mask_free(tsk);
free_stack:
exit_task_stack_account(tsk);
free_thread_stack(tsk);
diff --git a/kernel/scftorture.c b/kernel/scftorture.c
index 327c315f411c..2082f9b44370 100644
--- a/kernel/scftorture.c
+++ b/kernel/scftorture.c
@@ -348,6 +348,8 @@ static void scftorture_invoke_one(struct scf_statistics *scfp, struct torture_ra
int ret = 0;
struct scf_check *scfcp = NULL;
struct scf_selector *scfsp = scf_sel_rand(trsp);
+ bool is_single = (scfsp->scfs_prim == SCF_PRIM_SINGLE ||
+ scfsp->scfs_prim == SCF_PRIM_SINGLE_RPC);
if (scfsp->scfs_prim == SCF_PRIM_SINGLE || scfsp->scfs_wait) {
scfcp = kmalloc_obj(*scfcp, GFP_ATOMIC);
@@ -364,8 +366,6 @@ static void scftorture_invoke_one(struct scf_statistics *scfp, struct torture_ra
}
if (use_cpus_read_lock)
cpus_read_lock();
- else
- preempt_disable();
switch (scfsp->scfs_prim) {
case SCF_PRIM_RESCHED:
if (IS_BUILTIN(CONFIG_SCF_TORTURE_TEST)) {
@@ -411,13 +411,10 @@ static void scftorture_invoke_one(struct scf_statistics *scfp, struct torture_ra
if (!ret) {
if (use_cpus_read_lock)
cpus_read_unlock();
- else
- preempt_enable();
+
wait_for_completion(&scfcp->scfc_completion);
if (use_cpus_read_lock)
cpus_read_lock();
- else
- preempt_disable();
} else {
scfp->n_single_rpc_ofl++;
scf_add_to_free_list(scfcp);
@@ -452,7 +449,7 @@ static void scftorture_invoke_one(struct scf_statistics *scfp, struct torture_ra
scfcp->scfc_out = true;
}
if (scfcp && scfsp->scfs_wait) {
- if (WARN_ON_ONCE((num_online_cpus() > 1 || scfsp->scfs_prim == SCF_PRIM_SINGLE) &&
+ if (WARN_ON_ONCE(((use_cpus_read_lock && num_online_cpus() > 1) || is_single) &&
!scfcp->scfc_out)) {
pr_warn("%s: Memory-ordering failure, scfs_prim: %d.\n", __func__, scfsp->scfs_prim);
atomic_inc(&n_mb_out_errs); // Leak rather than trash!
@@ -463,8 +460,6 @@ static void scftorture_invoke_one(struct scf_statistics *scfp, struct torture_ra
}
if (use_cpus_read_lock)
cpus_read_unlock();
- else
- preempt_enable();
if (allocfail)
schedule_timeout_idle((1 + longwait) * HZ); // Let no-wait handlers complete.
else if (!(torture_random(trsp) & 0xfff))
diff --git a/kernel/smp.c b/kernel/smp.c
index a0bb56bd8dda..6f8f4c9d8fd8 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -16,6 +16,7 @@
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/gfp.h>
+#include <linux/slab.h>
#include <linux/smp.h>
#include <linux/cpu.h>
#include <linux/sched.h>
@@ -63,7 +64,14 @@ int smpcfd_prepare_cpu(unsigned int cpu)
free_cpumask_var(cfd->cpumask);
return -ENOMEM;
}
- cfd->csd = alloc_percpu(call_single_data_t);
+
+ /*
+ * Allocate the per-CPU CSD the first time a CPU comes up. It is
+ * not freed when the CPU is offlined, so csd_lock_wait() can access
+ * it even when the CPU was offlined after preemption was re-enabled.
+ */
+ if (!cfd->csd)
+ cfd->csd = alloc_percpu(call_single_data_t);
if (!cfd->csd) {
free_cpumask_var(cfd->cpumask);
free_cpumask_var(cfd->cpumask_ipi);
@@ -79,7 +87,6 @@ int smpcfd_dead_cpu(unsigned int cpu)
free_cpumask_var(cfd->cpumask);
free_cpumask_var(cfd->cpumask_ipi);
- free_percpu(cfd->csd);
return 0;
}
@@ -182,16 +189,22 @@ static atomic_t csd_bug_count = ATOMIC_INIT(0);
static void __csd_lock_record(call_single_data_t *csd)
{
if (!csd) {
- smp_mb(); /* NULL cur_csd after unlock. */
- __this_cpu_write(cur_csd, NULL);
+ /*
+ * Pairs with smp_load_acquire() of cur_csd in
+ * csd_lock_wait_toolong(): orders any preceding CSD
+ * callback/unlock before a remote reader observes NULL.
+ */
+ smp_store_release(this_cpu_ptr(&cur_csd), NULL);
return;
}
__this_cpu_write(cur_csd_func, csd->func);
__this_cpu_write(cur_csd_info, csd->info);
- smp_wmb(); /* func and info before csd. */
- __this_cpu_write(cur_csd, csd);
- smp_mb(); /* Update cur_csd before function call. */
- /* Or before unlock, as the case may be. */
+ /*
+ * Pairs with smp_load_acquire() of cur_csd in
+ * csd_lock_wait_toolong(): publishes cur_csd_func and
+ * cur_csd_info before the non-NULL pointer becomes visible.
+ */
+ smp_store_release(this_cpu_ptr(&cur_csd), csd);
}
static __always_inline void csd_lock_record(call_single_data_t *csd)
@@ -272,7 +285,13 @@ static bool csd_lock_wait_toolong(call_single_data_t *csd, u64 ts0, u64 *ts1, in
cpux = 0;
else
cpux = cpu;
- cpu_cur_csd = smp_load_acquire(&per_cpu(cur_csd, cpux)); /* Before func and info. */
+ /*
+ * Pairs with smp_store_release() of cur_csd in __csd_lock_record():
+ * a non-NULL cur_csd here implies cur_csd_func and cur_csd_info
+ * are the matching publication; a NULL value is ordered after any
+ * preceding CSD callback/unlock on the remote CPU.
+ */
+ cpu_cur_csd = smp_load_acquire(&per_cpu(cur_csd, cpux));
/* How long since this CSD lock was stuck. */
ts_delta = ts2 - ts0;
pr_alert("csd: %s non-responsive CSD lock (#%d) on CPU#%d, waiting %lld ns for CPU#%02d %pS(%ps).\n",
@@ -323,6 +342,8 @@ static void __csd_lock_wait(call_single_data_t *csd)
int bug_id = 0;
u64 ts0, ts1;
+ guard(preempt)();
+
ts1 = ts0 = ktime_get_mono_fast_ns();
for (;;) {
if (csd_lock_wait_toolong(csd, ts0, &ts1, &bug_id, &nmessages))
@@ -639,17 +660,9 @@ void flush_smp_call_function_queue(void)
local_irq_restore(flags);
}
-/**
- * smp_call_function_single - Run a function on a specific CPU
- * @cpu: Specific target CPU for this function.
- * @func: The function to run. This must be fast and non-blocking.
- * @info: An arbitrary pointer to pass to the function.
- * @wait: If true, wait until function has completed on other CPUs.
- *
- * Returns: %0 on success, else a negative status code.
- */
-int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
- int wait)
+static int __smp_call_function_single(int cpu, smp_call_func_t func,
+ void *info, const struct cpumask *mask,
+ bool wait)
{
call_single_data_t *csd;
call_single_data_t csd_stack = {
@@ -666,6 +679,14 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
*/
this_cpu = get_cpu();
+ if (mask) {
+ /* Try for same CPU (cheapest) */
+ if (!cpumask_test_cpu(this_cpu, mask))
+ cpu = sched_numa_find_nth_cpu(mask, 0, cpu_to_node(this_cpu));
+ else
+ cpu = this_cpu;
+ }
+
/*
* Can deadlock when called with interrupts disabled.
* We allow cpu's that are not yet online though, as no one else can
@@ -698,13 +719,32 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
err = generic_exec_single(cpu, csd);
+ /*
+ * @csd is stack-allocated when @wait is true. No concurrent access
+ * except from the IPI completion path, so we can re-enable preemption
+ * early to reduce latency.
+ */
+ put_cpu();
+
if (wait)
csd_lock_wait(csd);
- put_cpu();
-
return err;
}
+
+/**
+ * smp_call_function_single - Run a function on a specific CPU
+ * @cpu: Specific target CPU for this function.
+ * @func: The function to run. This must be fast and non-blocking.
+ * @info: An arbitrary pointer to pass to the function.
+ * @wait: If true, wait until function has completed on other CPUs.
+ *
+ * Returns: %0 on success, else a negative status code.
+ */
+int smp_call_function_single(int cpu, smp_call_func_t func, void *info, bool wait)
+{
+ return __smp_call_function_single(cpu, func, info, NULL, wait);
+}
EXPORT_SYMBOL(smp_call_function_single);
/**
@@ -755,10 +795,10 @@ EXPORT_SYMBOL_GPL(smp_call_function_single_async);
/**
* smp_call_function_any - Run a function on any of the given cpus
- * @mask: The mask of cpus it can run on.
- * @func: The function to run. This must be fast and non-blocking.
- * @info: An arbitrary pointer to pass to the function.
- * @wait: If true, wait until function has completed.
+ * @mask: The mask of cpus it can run on.
+ * @func: The function to run. This must be fast and non-blocking.
+ * @info: An arbitrary pointer to pass to the function.
+ * @wait: If true, wait until function has completed.
*
* Selection preference:
* 1) current cpu if in @mask
@@ -769,19 +809,53 @@ EXPORT_SYMBOL_GPL(smp_call_function_single_async);
int smp_call_function_any(const struct cpumask *mask,
smp_call_func_t func, void *info, int wait)
{
- unsigned int cpu;
- int ret;
+ return __smp_call_function_single(-1, func, info, mask, wait);
+}
+EXPORT_SYMBOL_GPL(smp_call_function_any);
- /* Try for same CPU (cheapest) */
- cpu = get_cpu();
- if (!cpumask_test_cpu(cpu, mask))
- cpu = sched_numa_find_nth_cpu(mask, 0, cpu_to_node(cpu));
+static DEFINE_STATIC_KEY_FALSE(ipi_mask_inlined);
- ret = smp_call_function_single(cpu, func, info, wait);
- put_cpu();
- return ret;
+#ifdef CONFIG_PREEMPTION
+
+int smp_task_ipi_mask_alloc(struct task_struct *task)
+{
+ if (static_branch_unlikely(&ipi_mask_inlined))
+ return 0;
+
+ ACCESS_PRIVATE(task, ipi_mask).ipi_mask_ptr =
+ kmalloc(cpumask_size(), GFP_KERNEL);
+ if (!ACCESS_PRIVATE(task, ipi_mask).ipi_mask_ptr)
+ return -ENOMEM;
+
+ return 0;
}
-EXPORT_SYMBOL_GPL(smp_call_function_any);
+
+void smp_task_ipi_mask_free(struct task_struct *task)
+{
+ if (static_branch_unlikely(&ipi_mask_inlined))
+ return;
+
+ kfree(ACCESS_PRIVATE(task, ipi_mask).ipi_mask_ptr);
+}
+
+static cpumask_t *smp_task_ipi_mask(struct task_struct *cur)
+{
+ /*
+ * If cpumask_size() is smaller than or equal to the pointer
+ * size, it stashes the cpumask in the pointer itself to
+ * avoid extra memory allocations.
+ */
+ if (static_branch_unlikely(&ipi_mask_inlined))
+ return (cpumask_t *)&ACCESS_PRIVATE(cur, ipi_mask).ipi_mask_val;
+
+ return ACCESS_PRIVATE(cur, ipi_mask).ipi_mask_ptr;
+}
+#else
+static cpumask_t *smp_task_ipi_mask(struct task_struct *cur)
+{
+ return NULL;
+}
+#endif
/*
* Flags to be used as scf_flags argument of smp_call_function_many_cond().
@@ -797,13 +871,20 @@ static void smp_call_function_many_cond(const struct cpumask *mask,
unsigned int scf_flags,
smp_cond_func_t cond_func)
{
- int cpu, last_cpu, this_cpu = smp_processor_id();
- struct call_function_data *cfd;
+ struct cpumask *cpumask, *task_mask;
bool wait = scf_flags & SCF_WAIT;
- int nr_cpus = 0;
+ struct call_function_data *cfd;
+ int cpu, last_cpu, this_cpu;
bool run_remote = false;
+ int nr_cpus = 0;
- lockdep_assert_preemption_disabled();
+ this_cpu = get_cpu();
+ cfd = this_cpu_ptr(&cfd_data);
+ task_mask = smp_task_ipi_mask(current);
+ if (task_mask)
+ cpumask = task_mask;
+ else
+ cpumask = cfd->cpumask;
/*
* Can deadlock when called with interrupts disabled.
@@ -825,16 +906,15 @@ static void smp_call_function_many_cond(const struct cpumask *mask,
/* Check if we need remote execution, i.e., any CPU excluding this one. */
if (cpumask_any_and_but(mask, cpu_online_mask, this_cpu) < nr_cpu_ids) {
- cfd = this_cpu_ptr(&cfd_data);
- cpumask_and(cfd->cpumask, mask, cpu_online_mask);
- __cpumask_clear_cpu(this_cpu, cfd->cpumask);
+ cpumask_and(cpumask, mask, cpu_online_mask);
+ __cpumask_clear_cpu(this_cpu, cpumask);
cpumask_clear(cfd->cpumask_ipi);
- for_each_cpu(cpu, cfd->cpumask) {
+ for_each_cpu(cpu, cpumask) {
call_single_data_t *csd = per_cpu_ptr(cfd->csd, cpu);
if (cond_func && !cond_func(cpu, info)) {
- __cpumask_clear_cpu(cpu, cfd->cpumask);
+ __cpumask_clear_cpu(cpu, cpumask);
continue;
}
@@ -884,8 +964,18 @@ static void smp_call_function_many_cond(const struct cpumask *mask,
local_irq_restore(flags);
}
+ /*
+ * The IPI work has been queued and dispatched. On PREEMPT kernels,
+ * tasks created through dup_task_struct() have task-local wait masks.
+ * The boot init_task can fall back to cfd->cpumask when the mask is
+ * not inlined, but other tasks still use task-local masks and cannot
+ * overwrite it. On !PREEMPT kernels, preempt_enable() cannot schedule
+ * another task, so the per-CPU mask remains protected.
+ */
+ put_cpu();
+
if (run_remote && wait) {
- for_each_cpu(cpu, cfd->cpumask) {
+ for_each_cpu(cpu, cpumask) {
call_single_data_t *csd;
csd = per_cpu_ptr(cfd->csd, cpu);
@@ -896,15 +986,14 @@ static void smp_call_function_many_cond(const struct cpumask *mask,
/**
* smp_call_function_many() - Run a function on a set of CPUs.
- * @mask: The set of cpus to run on (only runs on online subset).
- * @func: The function to run. This must be fast and non-blocking.
- * @info: An arbitrary pointer to pass to the function.
- * @wait: If true, wait (atomically) until function has completed
- * on other CPUs.
+ * @mask: The set of cpus to run on (only runs on online subset).
+ * @func: The function to run. This must be fast and non-blocking.
+ * @info: An arbitrary pointer to pass to the function.
+ * @wait: If true, wait (atomically) until function has completed
+ * on other CPUs.
*
* You must not call this function with disabled interrupts or from a
- * hardware interrupt handler or from a bottom half handler. Preemption
- * must be disabled when calling this function.
+ * hardware interrupt handler or from a bottom half handler.
*
* @func is not called on the local CPU even if @mask contains it. Consider
* using on_each_cpu_cond_mask() instead if this is not desirable.
@@ -918,10 +1007,10 @@ EXPORT_SYMBOL(smp_call_function_many);
/**
* smp_call_function() - Run a function on all other CPUs.
- * @func: The function to run. This must be fast and non-blocking.
- * @info: An arbitrary pointer to pass to the function.
- * @wait: If true, wait (atomically) until function has completed
- * on other CPUs.
+ * @func: The function to run. This must be fast and non-blocking.
+ * @info: An arbitrary pointer to pass to the function.
+ * @wait: If true, wait (atomically) until function has completed
+ * on other CPUs.
*
* If @wait is true, then returns once @func has returned; otherwise
* it returns just before the target cpu calls @func.
@@ -931,9 +1020,8 @@ EXPORT_SYMBOL(smp_call_function_many);
*/
void smp_call_function(smp_call_func_t func, void *info, int wait)
{
- preempt_disable();
- smp_call_function_many(cpu_online_mask, func, info, wait);
- preempt_enable();
+ smp_call_function_many_cond(cpu_online_mask, func, info,
+ wait ? SCF_WAIT : 0, NULL);
}
EXPORT_SYMBOL(smp_call_function);
@@ -999,6 +1087,9 @@ EXPORT_SYMBOL(nr_cpu_ids);
void __init setup_nr_cpu_ids(void)
{
set_nr_cpu_ids(find_last_bit(cpumask_bits(cpu_possible_mask), NR_CPUS) + 1);
+
+ if (IS_ENABLED(CONFIG_PREEMPTION) && cpumask_size() <= sizeof(unsigned long))
+ static_branch_enable(&ipi_mask_inlined);
}
/* Called by boot processor to activate the rest. */
@@ -1035,12 +1126,14 @@ void __init smp_init(void)
* @func: The function to run on all applicable CPUs.
* This must be fast and non-blocking.
* @info: An arbitrary pointer to pass to both functions.
- * @wait: If true, wait (atomically) until function has
- * completed on other CPUs.
+ * @wait: If true, wait until function has completed on other CPUs.
* @mask: The set of cpus to run on (only runs on online subset).
*
- * Preemption is disabled to protect against CPUs going offline but not online.
- * CPUs going online during the call will not be seen or sent an IPI.
+ * Target CPU selection and work queueing are done with preemption
+ * disabled. This protects against CPUs going offline, but not against
+ * CPUs coming online concurrently; newly online CPUs are not guaranteed
+ * to be seen or sent an IPI. If @wait is true, the final wait for remote
+ * completion happens after that preemption-disabled section.
*
* You must not call this function with disabled interrupts or
* from a hardware interrupt handler or from a bottom half handler.
@@ -1053,9 +1146,7 @@ void on_each_cpu_cond_mask(smp_cond_func_t cond_func, smp_call_func_t func,
if (wait)
scf_flags |= SCF_WAIT;
- preempt_disable();
smp_call_function_many_cond(mask, func, info, scf_flags, cond_func);
- preempt_enable();
}
EXPORT_SYMBOL(on_each_cpu_cond_mask);
diff --git a/kernel/up.c b/kernel/up.c
index df50828cc2f0..6d4ac9502e8b 100644
--- a/kernel/up.c
+++ b/kernel/up.c
@@ -9,8 +9,7 @@
#include <linux/smp.h>
#include <linux/hypervisor.h>
-int smp_call_function_single(int cpu, void (*func) (void *info), void *info,
- int wait)
+int smp_call_function_single(int cpu, void (*func)(void *info), void *info, bool wait)
{
unsigned long flags;
^ permalink raw reply related [flat|nested] 26+ messages in thread* [GIT pull] timers/cleanups for v7.3-rc1
2026-08-17 11:23 [GIT pull] core/entry for v7.3-rc1 Thomas Gleixner
` (4 preceding siblings ...)
2026-08-17 11:23 ` [GIT pull] smp/core " Thomas Gleixner
@ 2026-08-17 11:23 ` Thomas Gleixner
2026-08-19 1:10 ` pr-tracker-bot
2026-08-19 18:48 ` Nathan Chancellor
2026-08-17 11:23 ` [GIT pull] timers/clocksource " Thomas Gleixner
` (3 subsequent siblings)
9 siblings, 2 replies; 26+ messages in thread
From: Thomas Gleixner @ 2026-08-17 11:23 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest timers/cleanups branch from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers-cleanups-2026-08-17
up to: dfc256dac54c: calibrate: Rework delay timer calibration
Treewide timer related cleanups:
- Remove the leftover CLOCK_TICK_RATE which has been scheduled for
removal more than a decade ago along with some now empty asm/timex.h
files.
- Consolidate delay timer calibration
The construct of having a define in a header requires that
architectures provided asm/timex.h for no reason. Also the function
name for reading the delay timer is confusing at best.
Use a config switch to enable that functionality and rename the
function to delay_read_timer() to make the purpose clear.
This removes some more now empty asm/timex.h files as well.
Thanks,
tglx
------------------>
Thomas Gleixner (3):
x86: Use PIT_TICK_RATE instead of CLOCK_TICK_RATE
treewide: Remove CLOCK_TICK_RATE
calibrate: Rework delay timer calibration
arch/Kconfig | 3 +++
arch/alpha/include/asm/timex.h | 4 ----
arch/arc/include/asm/timex.h | 15 ---------------
arch/arm/Kconfig | 1 +
arch/arm/include/asm/delay.h | 1 -
arch/arm/include/asm/timex.h | 5 ++++-
arch/arm/lib/delay.c | 10 ++++------
arch/arm/mach-omap1/Kconfig | 2 +-
arch/hexagon/Kconfig | 1 +
arch/hexagon/include/asm/timex.h | 23 -----------------------
arch/hexagon/kernel/time.c | 8 +++++++-
arch/m68k/include/asm/timex.h | 15 ---------------
arch/microblaze/include/asm/timex.h | 13 -------------
arch/mips/include/asm/timex.h | 8 --------
arch/openrisc/Kconfig | 1 +
arch/openrisc/include/asm/timex.h | 5 -----
arch/openrisc/lib/delay.c | 9 ++++-----
arch/parisc/include/asm/timex.h | 2 --
arch/powerpc/include/asm/timex.h | 2 --
arch/riscv/Kconfig | 1 +
arch/riscv/include/asm/timex.h | 8 --------
arch/riscv/lib/delay.c | 7 ++++++-
arch/s390/include/asm/timex.h | 2 --
arch/sh/include/asm/timex.h | 24 ------------------------
arch/sparc/Kconfig | 1 +
arch/sparc/include/asm/timex.h | 2 +-
arch/sparc/include/asm/timex_32.h | 14 --------------
arch/sparc/include/asm/timex_64.h | 4 ----
arch/sparc/kernel/time_64.c | 4 ++--
arch/um/include/asm/timex.h | 9 ---------
arch/x86/Kconfig | 1 +
arch/x86/include/asm/timex.h | 5 -----
arch/x86/kernel/setup.c | 2 +-
arch/x86/lib/delay.c | 8 +++-----
include/asm-generic/timex.h | 7 -------
include/linux/delay.h | 2 ++
include/linux/timex.h | 2 --
init/calibrate.c | 19 +++++++++----------
38 files changed, 53 insertions(+), 197 deletions(-)
delete mode 100644 arch/arc/include/asm/timex.h
delete mode 100644 arch/hexagon/include/asm/timex.h
delete mode 100644 arch/microblaze/include/asm/timex.h
delete mode 100644 arch/sh/include/asm/timex.h
delete mode 100644 arch/sparc/include/asm/timex_32.h
delete mode 100644 arch/um/include/asm/timex.h
diff --git a/arch/Kconfig b/arch/Kconfig
index fa7507ac8e13..3d8550b12312 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -363,6 +363,9 @@ config ARCH_HAS_DMA_CLEAR_UNCACHED
config ARCH_HAS_CPU_FINALIZE_INIT
bool
+config ARCH_HAS_DELAY_TIMER
+ bool
+
# The architecture has a per-task state that includes the mm's PASID
config ARCH_HAS_CPU_PASID
bool
diff --git a/arch/alpha/include/asm/timex.h b/arch/alpha/include/asm/timex.h
index f89798da8a14..5a71ba6136b9 100644
--- a/arch/alpha/include/asm/timex.h
+++ b/arch/alpha/include/asm/timex.h
@@ -7,10 +7,6 @@
#ifndef _ASMALPHA_TIMEX_H
#define _ASMALPHA_TIMEX_H
-/* With only one or two oddballs, we use the RTC as the ticker, selecting
- the 32.768kHz reference clock, which nicely divides down to our HZ. */
-#define CLOCK_TICK_RATE 32768
-
/*
* Standard way to access the cycle counter.
* Currently only used on SMP for scheduling.
diff --git a/arch/arc/include/asm/timex.h b/arch/arc/include/asm/timex.h
deleted file mode 100644
index 48b3482bc97f..000000000000
--- a/arch/arc/include/asm/timex.h
+++ /dev/null
@@ -1,15 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-/*
- * Copyright (C) 2004, 2007-2010, 2011-2012 Synopsys, Inc. (www.synopsys.com)
- */
-
-#ifndef _ASM_ARC_TIMEX_H
-#define _ASM_ARC_TIMEX_H
-
-#define CLOCK_TICK_RATE 80000000 /* slated to be removed */
-
-#include <asm-generic/timex.h>
-
-/* XXX: get_cycles() to be implemented with RTSC insn */
-
-#endif /* _ASM_ARC_TIMEX_H */
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 9187240a02db..7788949992f4 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -11,6 +11,7 @@ config ARM
select ARCH_HAS_CPU_FINALIZE_INIT if MMU
select ARCH_HAS_CURRENT_STACK_POINTER
select ARCH_HAS_DEBUG_VIRTUAL if MMU
+ select ARCH_HAS_DELAY_TIMER
select ARCH_HAS_DMA_ALLOC if MMU
select ARCH_HAS_DMA_OPS
select ARCH_HAS_DMA_WRITE_COMBINE if !ARM_DMA_MEM_BUFFERABLE
diff --git a/arch/arm/include/asm/delay.h b/arch/arm/include/asm/delay.h
index 1d069e558d8d..d2231deb90fa 100644
--- a/arch/arm/include/asm/delay.h
+++ b/arch/arm/include/asm/delay.h
@@ -91,7 +91,6 @@ extern void __loop_udelay(unsigned long usecs);
extern void __loop_const_udelay(unsigned long);
/* Delay-loop timer registration. */
-#define ARCH_HAS_READ_CURRENT_TIMER
extern void register_current_timer_delay(const struct delay_timer *timer);
#endif /* __ASSEMBLY__ */
diff --git a/arch/arm/include/asm/timex.h b/arch/arm/include/asm/timex.h
index 6d1337c169cd..94e40c19cfc5 100644
--- a/arch/arm/include/asm/timex.h
+++ b/arch/arm/include/asm/timex.h
@@ -10,7 +10,10 @@
#define _ASMARM_TIMEX_H
typedef unsigned long cycles_t;
-#define get_cycles() ({ cycles_t c; read_current_timer(&c) ? 0 : c; })
+// Temporary workaround until timex.h is cleaned up
+bool delay_read_timer(unsigned long *t);
+
+#define get_cycles() ({ cycles_t c; delay_read_timer(&c) ? 0 : c; })
#define random_get_entropy() (((unsigned long)get_cycles()) ?: random_get_entropy_fallback())
#endif
diff --git a/arch/arm/lib/delay.c b/arch/arm/lib/delay.c
index b7fe84f68bf1..7e9304012a68 100644
--- a/arch/arm/lib/delay.c
+++ b/arch/arm/lib/delay.c
@@ -12,7 +12,6 @@
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
-#include <linux/timex.h>
/*
* Default to the loop-based delay implementation.
@@ -27,15 +26,14 @@ static const struct delay_timer *delay_timer;
static bool delay_calibrated;
static u64 delay_res;
-int read_current_timer(unsigned long *timer_val)
+bool delay_read_timer(unsigned long *timer_val)
{
if (!delay_timer)
- return -ENXIO;
-
+ return false;
*timer_val = delay_timer->read_current_timer();
- return 0;
+ return true;
}
-EXPORT_SYMBOL_GPL(read_current_timer);
+EXPORT_SYMBOL_GPL(delay_read_timer);
static inline u64 cyc_to_ns(u64 cyc, u32 mult, u32 shift)
{
diff --git a/arch/arm/mach-omap1/Kconfig b/arch/arm/mach-omap1/Kconfig
index b114f7ca2173..2a724aa6674c 100644
--- a/arch/arm/mach-omap1/Kconfig
+++ b/arch/arm/mach-omap1/Kconfig
@@ -74,7 +74,7 @@ config OMAP_32K_TIMER
currently only available for OMAP16XX, 24XX, 34XX, OMAP4/5 and DRA7XX.
On OMAP2PLUS this value is only used for CONFIG_HZ and
- CLOCK_TICK_RATE compile time calculation.
+ timer frequency compile time calculation.
The actual timer selection is done in the board file
through the (DT_)MACHINE_START structure.
diff --git a/arch/hexagon/Kconfig b/arch/hexagon/Kconfig
index d987ba368ed6..b48491140013 100644
--- a/arch/hexagon/Kconfig
+++ b/arch/hexagon/Kconfig
@@ -5,6 +5,7 @@ comment "Linux Kernel Configuration for Hexagon"
config HEXAGON
def_bool y
select ARCH_32BIT_OFF_T
+ select ARCH_HAS_DELAY_TIMER
select ARCH_HAS_SYNC_DMA_FOR_DEVICE
select ARCH_NO_PREEMPT
select ARCH_WANT_FRAME_POINTERS
diff --git a/arch/hexagon/include/asm/timex.h b/arch/hexagon/include/asm/timex.h
deleted file mode 100644
index dfe69e118b2b..000000000000
--- a/arch/hexagon/include/asm/timex.h
+++ /dev/null
@@ -1,23 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-/*
- * Copyright (c) 2010-2011, The Linux Foundation. All rights reserved.
- */
-
-#ifndef _ASM_TIMEX_H
-#define _ASM_TIMEX_H
-
-#include <asm-generic/timex.h>
-#include <asm/hexagon_vm.h>
-
-/* Using TCX0 as our clock. CLOCK_TICK_RATE scheduled to be removed. */
-#define CLOCK_TICK_RATE 19200
-
-#define ARCH_HAS_READ_CURRENT_TIMER
-
-static inline int read_current_timer(unsigned long *timer_val)
-{
- *timer_val = __vmgettime();
- return 0;
-}
-
-#endif
diff --git a/arch/hexagon/kernel/time.c b/arch/hexagon/kernel/time.c
index 6f851e1cd4ee..05c5d6c96d96 100644
--- a/arch/hexagon/kernel/time.c
+++ b/arch/hexagon/kernel/time.c
@@ -6,6 +6,7 @@
*/
#include <linux/init.h>
+#include <linux/delay.h>
#include <linux/clockchips.h>
#include <linux/clocksource.h>
#include <linux/interrupt.h>
@@ -17,7 +18,6 @@
#include <linux/of_irq.h>
#include <linux/module.h>
-#include <asm/delay.h>
#include <asm/hexagon_vm.h>
#include <asm/time.h>
@@ -231,3 +231,9 @@ void __udelay(unsigned long usecs)
cpu_relax(); /* not sure how this improves readability */
}
EXPORT_SYMBOL(__udelay);
+
+bool delay_read_timer(unsigned long *timer_val)
+{
+ *timer_val = __vmgettime();
+ return true;
+}
diff --git a/arch/m68k/include/asm/timex.h b/arch/m68k/include/asm/timex.h
index f4a7a340f4ca..b58115f3a282 100644
--- a/arch/m68k/include/asm/timex.h
+++ b/arch/m68k/include/asm/timex.h
@@ -7,21 +7,6 @@
#ifndef _ASMm68K_TIMEX_H
#define _ASMm68K_TIMEX_H
-#ifdef CONFIG_COLDFIRE
-/*
- * CLOCK_TICK_RATE should give the underlying frequency of the tick timer
- * to make ntp work best. For Coldfires, that's the main clock.
- */
-#include <asm/coldfire.h>
-#define CLOCK_TICK_RATE MCF_CLK
-#else
-/*
- * This default CLOCK_TICK_RATE is probably wrong for many 68k boards
- * Users of those boards will need to check and modify accordingly
- */
-#define CLOCK_TICK_RATE 1193180 /* Underlying HZ */
-#endif
-
typedef unsigned long cycles_t;
static inline cycles_t get_cycles(void)
diff --git a/arch/microblaze/include/asm/timex.h b/arch/microblaze/include/asm/timex.h
deleted file mode 100644
index e99cc29cbe57..000000000000
--- a/arch/microblaze/include/asm/timex.h
+++ /dev/null
@@ -1,13 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-/*
- * Copyright (C) 2006 Atmark Techno, Inc.
- */
-
-#ifndef _ASM_MICROBLAZE_TIMEX_H
-#define _ASM_MICROBLAZE_TIMEX_H
-
-#include <asm-generic/timex.h>
-
-#define CLOCK_TICK_RATE 1000 /* Timer input freq. */
-
-#endif /* _ASM_TIMEX_H */
diff --git a/arch/mips/include/asm/timex.h b/arch/mips/include/asm/timex.h
index 7ef06dcdc46e..daedc62f0fb8 100644
--- a/arch/mips/include/asm/timex.h
+++ b/arch/mips/include/asm/timex.h
@@ -18,14 +18,6 @@
#include <asm/mipsregs.h>
#include <asm/cpu-type.h>
-/*
- * This is the clock rate of the i8253 PIT. A MIPS system may not have
- * a PIT by the symbol is used all over the kernel including some APIs.
- * So keeping it defined to the number for the PIT is the only sane thing
- * for now.
- */
-#define CLOCK_TICK_RATE 1193182
-
/*
* Standard way to access the cycle counter.
* Currently only used on SMP for scheduling.
diff --git a/arch/openrisc/Kconfig b/arch/openrisc/Kconfig
index 9156635dd264..5eb995c13074 100644
--- a/arch/openrisc/Kconfig
+++ b/arch/openrisc/Kconfig
@@ -7,6 +7,7 @@
config OPENRISC
def_bool y
select ARCH_32BIT_OFF_T
+ select ARCH_HAS_DELAY_TIMER
select ARCH_HAS_DMA_SET_UNCACHED
select ARCH_HAS_DMA_CLEAR_UNCACHED
select ARCH_HAS_SYNC_DMA_FOR_DEVICE
diff --git a/arch/openrisc/include/asm/timex.h b/arch/openrisc/include/asm/timex.h
index 5487fa93dd9b..574e80e6d1f3 100644
--- a/arch/openrisc/include/asm/timex.h
+++ b/arch/openrisc/include/asm/timex.h
@@ -25,9 +25,4 @@ static inline cycles_t get_cycles(void)
}
#define get_cycles get_cycles
-/* This isn't really used any more */
-#define CLOCK_TICK_RATE 1000
-
-#define ARCH_HAS_READ_CURRENT_TIMER
-
#endif
diff --git a/arch/openrisc/lib/delay.c b/arch/openrisc/lib/delay.c
index 5e89e4131304..661f9f8f6292 100644
--- a/arch/openrisc/lib/delay.c
+++ b/arch/openrisc/lib/delay.c
@@ -13,18 +13,17 @@
*/
#include <linux/kernel.h>
+#include <linux/delay.h>
#include <linux/export.h>
#include <linux/init.h>
-#include <linux/timex.h>
+
#include <asm/param.h>
-#include <asm/delay.h>
-#include <asm/timex.h>
#include <asm/processor.h>
-int read_current_timer(unsigned long *timer_value)
+bool delay_read_timer(unsigned long *timer_value)
{
*timer_value = get_cycles();
- return 0;
+ return true;
}
void __delay(unsigned long cycles)
diff --git a/arch/parisc/include/asm/timex.h b/arch/parisc/include/asm/timex.h
index b4622cb06a75..453468ba4e3a 100644
--- a/arch/parisc/include/asm/timex.h
+++ b/arch/parisc/include/asm/timex.h
@@ -9,8 +9,6 @@
#include <asm/special_insns.h>
-#define CLOCK_TICK_RATE 1193180 /* Underlying HZ */
-
typedef unsigned long cycles_t;
static inline cycles_t get_cycles(void)
diff --git a/arch/powerpc/include/asm/timex.h b/arch/powerpc/include/asm/timex.h
index 14b4489de52c..b48b78e47958 100644
--- a/arch/powerpc/include/asm/timex.h
+++ b/arch/powerpc/include/asm/timex.h
@@ -11,8 +11,6 @@
#include <asm/cputable.h>
#include <asm/vdso/timebase.h>
-#define CLOCK_TICK_RATE 1024000 /* Underlying HZ */
-
typedef unsigned long cycles_t;
static inline cycles_t get_cycles(void)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index c0a6992933e4..a0635743cf8f 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -29,6 +29,7 @@ config RISCV
select ARCH_HAS_DEBUG_VIRTUAL if MMU
select ARCH_HAS_DEBUG_VM_PGTABLE
select ARCH_HAS_DEBUG_WX
+ select ARCH_HAS_DELAY_TIMER
select ARCH_HAS_ELF_CORE_EFLAGS if BINFMT_ELF && ELF_CORE
select ARCH_HAS_FAST_MULTIPLIER
select ARCH_HAS_FORTIFY_SOURCE
diff --git a/arch/riscv/include/asm/timex.h b/arch/riscv/include/asm/timex.h
index a06697846e69..76c9417fdba3 100644
--- a/arch/riscv/include/asm/timex.h
+++ b/arch/riscv/include/asm/timex.h
@@ -80,12 +80,4 @@ static inline u64 get_cycles64(void)
return ((u64)hi << 32) | lo;
}
#endif /* CONFIG_64BIT */
-
-#define ARCH_HAS_READ_CURRENT_TIMER
-static inline int read_current_timer(unsigned long *timer_val)
-{
- *timer_val = get_cycles();
- return 0;
-}
-
#endif /* _ASM_RISCV_TIMEX_H */
diff --git a/arch/riscv/lib/delay.c b/arch/riscv/lib/delay.c
index 49d510ba75fd..bd9b3825656f 100644
--- a/arch/riscv/lib/delay.c
+++ b/arch/riscv/lib/delay.c
@@ -6,7 +6,6 @@
#include <linux/delay.h>
#include <linux/math.h>
#include <linux/param.h>
-#include <linux/timex.h>
#include <linux/types.h>
#include <linux/export.h>
@@ -109,3 +108,9 @@ void ndelay(unsigned long nsecs)
__delay(ncycles >> NDELAY_SHIFT);
}
EXPORT_SYMBOL(ndelay);
+
+bool delay_read_timer(unsigned long *timer_val)
+{
+ *timer_val = get_cycles();
+ return true;
+}
diff --git a/arch/s390/include/asm/timex.h b/arch/s390/include/asm/timex.h
index ac3ab6c29912..23421b6eb225 100644
--- a/arch/s390/include/asm/timex.h
+++ b/arch/s390/include/asm/timex.h
@@ -159,8 +159,6 @@ static inline void local_tick_enable(unsigned long comp)
set_clock_comparator(get_lowcore()->clock_comparator);
}
-#define CLOCK_TICK_RATE 1193180 /* Underlying HZ */
-
typedef unsigned long cycles_t;
static __always_inline unsigned long get_tod_clock(void)
diff --git a/arch/sh/include/asm/timex.h b/arch/sh/include/asm/timex.h
deleted file mode 100644
index f53f95bf4288..000000000000
--- a/arch/sh/include/asm/timex.h
+++ /dev/null
@@ -1,24 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-/*
- * linux/include/asm-sh/timex.h
- *
- * sh architecture timex specifications
- */
-#ifndef __ASM_SH_TIMEX_H
-#define __ASM_SH_TIMEX_H
-
-/*
- * Only parts using the legacy CPG code for their clock framework
- * implementation need to define their own Pclk value. If provided, this
- * can be used for accurately setting CLOCK_TICK_RATE, otherwise we
- * simply fall back on the i8253 PIT value.
- */
-#ifdef CONFIG_SH_PCLK_FREQ
-#define CLOCK_TICK_RATE (CONFIG_SH_PCLK_FREQ / 4) /* Underlying HZ */
-#else
-#define CLOCK_TICK_RATE 1193180
-#endif
-
-#include <asm-generic/timex.h>
-
-#endif /* __ASM_SH_TIMEX_H */
diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index 0e9c906c4b5d..a979a4be79a9 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -70,6 +70,7 @@ config SPARC32
config SPARC64
def_bool 64BIT
select ALTERNATE_USER_ADDRESS_SPACE
+ select ARCH_HAS_DELAY_TIMER
select HAVE_FUNCTION_TRACER
select HAVE_FUNCTION_GRAPH_TRACER
select HAVE_KRETPROBES
diff --git a/arch/sparc/include/asm/timex.h b/arch/sparc/include/asm/timex.h
index 9aac26b1f45f..f0f980581590 100644
--- a/arch/sparc/include/asm/timex.h
+++ b/arch/sparc/include/asm/timex.h
@@ -4,6 +4,6 @@
#if defined(__sparc__) && defined(__arch64__)
#include <asm/timex_64.h>
#else
-#include <asm/timex_32.h>
+#include <asm-generic/timex.h>
#endif
#endif
diff --git a/arch/sparc/include/asm/timex_32.h b/arch/sparc/include/asm/timex_32.h
deleted file mode 100644
index f86326a6f89e..000000000000
--- a/arch/sparc/include/asm/timex_32.h
+++ /dev/null
@@ -1,14 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-/*
- * linux/include/asm/timex.h
- *
- * sparc architecture timex specifications
- */
-#ifndef _ASMsparc_TIMEX_H
-#define _ASMsparc_TIMEX_H
-
-#define CLOCK_TICK_RATE 1193180 /* Underlying HZ */
-
-#include <asm-generic/timex.h>
-
-#endif
diff --git a/arch/sparc/include/asm/timex_64.h b/arch/sparc/include/asm/timex_64.h
index 076c44f6845d..fa004ef3bb9a 100644
--- a/arch/sparc/include/asm/timex_64.h
+++ b/arch/sparc/include/asm/timex_64.h
@@ -9,12 +9,8 @@
#include <asm/timer.h>
-#define CLOCK_TICK_RATE 1193180 /* Underlying HZ */
-
/* Getting on the cycle counter on sparc64. */
typedef unsigned long cycles_t;
#define get_cycles() tick_ops->get_tick()
-#define ARCH_HAS_READ_CURRENT_TIMER
-
#endif
diff --git a/arch/sparc/kernel/time_64.c b/arch/sparc/kernel/time_64.c
index 87b267043ccd..b1c54ab4b15b 100644
--- a/arch/sparc/kernel/time_64.c
+++ b/arch/sparc/kernel/time_64.c
@@ -894,8 +894,8 @@ unsigned long long sched_clock(void)
return ((get_tick() * quotient) >> SPARC64_NSEC_PER_CYC_SHIFT) - offset;
}
-int read_current_timer(unsigned long *timer_val)
+bool delay_read_timer(unsigned long *timer_val)
{
*timer_val = get_tick();
- return 0;
+ return true;
}
diff --git a/arch/um/include/asm/timex.h b/arch/um/include/asm/timex.h
deleted file mode 100644
index 9f27176adb26..000000000000
--- a/arch/um/include/asm/timex.h
+++ /dev/null
@@ -1,9 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef __UM_TIMEX_H
-#define __UM_TIMEX_H
-
-#define CLOCK_TICK_RATE (HZ)
-
-#include <asm-generic/timex.h>
-
-#endif
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index bdad90f210e4..ec9e1d4c85b1 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -80,6 +80,7 @@ config X86
select ARCH_HAS_CURRENT_STACK_POINTER
select ARCH_HAS_DEBUG_VIRTUAL
select ARCH_HAS_DEBUG_VM_PGTABLE if !X86_PAE
+ select ARCH_HAS_DELAY_TIMER
select ARCH_HAS_DEVMEM_IS_ALLOWED
select ARCH_HAS_DMA_OPS if GART_IOMMU || XEN
select ARCH_HAS_EARLY_DEBUG if KGDB
diff --git a/arch/x86/include/asm/timex.h b/arch/x86/include/asm/timex.h
index 6e57e3c0fdd2..f397072cf294 100644
--- a/arch/x86/include/asm/timex.h
+++ b/arch/x86/include/asm/timex.h
@@ -13,9 +13,4 @@ static inline unsigned long random_get_entropy(void)
}
#define random_get_entropy random_get_entropy
-/* Assume we use the PIT time source for the clock tick */
-#define CLOCK_TICK_RATE PIT_TICK_RATE
-
-#define ARCH_HAS_READ_CURRENT_TIMER
-
#endif /* _ASM_X86_TIMEX_H */
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 46882ce79c3a..0e75eb07990b 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -1268,7 +1268,7 @@ void __init setup_arch(char **cmdline_p)
mcheck_init();
- register_refined_jiffies(CLOCK_TICK_RATE);
+ register_refined_jiffies(PIT_TICK_RATE);
#ifdef CONFIG_EFI
if (efi_enabled(EFI_BOOT))
diff --git a/arch/x86/lib/delay.c b/arch/x86/lib/delay.c
index eb2d2e1cbddd..70ad000e91e7 100644
--- a/arch/x86/lib/delay.c
+++ b/arch/x86/lib/delay.c
@@ -14,12 +14,10 @@
#include <linux/export.h>
#include <linux/sched.h>
-#include <linux/timex.h>
#include <linux/preempt.h>
#include <linux/delay.h>
#include <asm/processor.h>
-#include <asm/delay.h>
#include <asm/timer.h>
#include <asm/mwait.h>
@@ -189,13 +187,13 @@ void use_mwaitx_delay(void)
delay_fn = delay_halt;
}
-int read_current_timer(unsigned long *timer_val)
+bool delay_read_timer(unsigned long *timer_val)
{
if (delay_fn == delay_tsc) {
*timer_val = rdtsc();
- return 0;
+ return true;
}
- return -1;
+ return false;
}
void __delay(unsigned long loops)
diff --git a/include/asm-generic/timex.h b/include/asm-generic/timex.h
index 50ba9b5ce983..2032fda7052c 100644
--- a/include/asm-generic/timex.h
+++ b/include/asm-generic/timex.h
@@ -13,11 +13,4 @@ static inline cycles_t get_cycles(void)
}
#endif
-/*
- * Architectures are encouraged to implement read_current_timer
- * and define this in order to avoid the expensive delay loop
- * calibration during boot.
- */
-#undef ARCH_HAS_READ_CURRENT_TIMER
-
#endif /* __ASM_GENERIC_TIMEX_H */
diff --git a/include/linux/delay.h b/include/linux/delay.h
index 68b2a69dd24d..82409e55b6ae 100644
--- a/include/linux/delay.h
+++ b/include/linux/delay.h
@@ -17,6 +17,8 @@ extern unsigned long loops_per_jiffy;
#include <asm/delay.h>
+bool delay_read_timer(unsigned long *t);
+
/*
* Using udelay() for intervals greater than a few milliseconds can
* risk overflow for high loops_per_jiffy (high bogomips) machines. The
diff --git a/include/linux/timex.h b/include/linux/timex.h
index 4ee32eff3f22..7014ccc638fc 100644
--- a/include/linux/timex.h
+++ b/include/linux/timex.h
@@ -156,8 +156,6 @@ extern int do_clock_adjtime(const clockid_t which_clock, struct __kernel_timex *
extern void hardpps(const struct timespec64 *, const struct timespec64 *);
-int read_current_timer(unsigned long *timer_val);
-
/* The clock frequency of the i8253/i8254 PIT */
#define PIT_TICK_RATE 1193182ul
diff --git a/init/calibrate.c b/init/calibrate.c
index 63be4c65bc52..2bfe97034e42 100644
--- a/init/calibrate.c
+++ b/init/calibrate.c
@@ -13,7 +13,6 @@
#include <linux/printk.h>
#include <linux/smp.h>
#include <linux/stddef.h>
-#include <linux/timex.h>
unsigned long lpj_fine;
unsigned long preset_lpj;
@@ -25,9 +24,9 @@ static int __init lpj_setup(char *str)
__setup("lpj=", lpj_setup);
-#ifdef ARCH_HAS_READ_CURRENT_TIMER
+#ifdef CONFIG_ARCH_HAS_DELAY_TIMER
-/* This routine uses the read_current_timer() routine and gets the
+/* This routine uses the delay_read_timer() routine and gets the
* loops per jiffy directly, instead of guessing it using delay().
* Also, this code tries to handle non-maskable asynchronous events
* (like SMIs)
@@ -48,13 +47,13 @@ static unsigned long calibrate_delay_direct(void)
int min = -1;
int i;
- if (read_current_timer(&pre_start) < 0 )
+ if (!delay_read_timer(&pre_start))
return 0;
/*
* A simple loop like
* while ( jiffies < start_jiffies+1)
- * start = read_current_timer();
+ * start = delay_read_timer();
* will not do. As we don't really know whether jiffy switch
* happened first or timer_value was read first. And some asynchronous
* event can happen between these two events introducing errors in lpj.
@@ -72,22 +71,22 @@ static unsigned long calibrate_delay_direct(void)
for (i = 0; i < MAX_DIRECT_CALIBRATION_RETRIES; i++) {
pre_start = 0;
- read_current_timer(&start);
+ delay_read_timer(&start);
start_jiffies = jiffies;
while (time_before_eq(jiffies, start_jiffies + 1)) {
pre_start = start;
- read_current_timer(&start);
+ delay_read_timer(&start);
}
- read_current_timer(&post_start);
+ delay_read_timer(&post_start);
pre_end = 0;
end = post_start;
while (time_before_eq(jiffies, start_jiffies + 1 +
DELAY_CALIBRATION_TICKS)) {
pre_end = end;
- read_current_timer(&end);
+ delay_read_timer(&end);
}
- read_current_timer(&post_end);
+ delay_read_timer(&post_end);
timer_rate_max = (post_end - pre_start) /
DELAY_CALIBRATION_TICKS;
^ permalink raw reply related [flat|nested] 26+ messages in thread* Re: [GIT pull] timers/cleanups for v7.3-rc1
2026-08-17 11:23 ` [GIT pull] timers/cleanups " Thomas Gleixner
@ 2026-08-19 1:10 ` pr-tracker-bot
2026-08-19 18:48 ` Nathan Chancellor
1 sibling, 0 replies; 26+ messages in thread
From: pr-tracker-bot @ 2026-08-19 1:10 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, linux-kernel, x86
The pull request you sent on Mon, 17 Aug 2026 13:23:52 +0200:
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers-cleanups-2026-08-17
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/030c9f813b8e48d2b066983c94cf6294968f9496
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [GIT pull] timers/cleanups for v7.3-rc1
2026-08-17 11:23 ` [GIT pull] timers/cleanups " Thomas Gleixner
2026-08-19 1:10 ` pr-tracker-bot
@ 2026-08-19 18:48 ` Nathan Chancellor
2026-08-19 20:59 ` Thomas Gleixner
1 sibling, 1 reply; 26+ messages in thread
From: Nathan Chancellor @ 2026-08-19 18:48 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, linux-kernel, x86
Hi Thomas,
On Mon, Aug 17, 2026 at 01:23:52PM +0200, Thomas Gleixner wrote:
> calibrate: Rework delay timer calibration
...
> diff --git a/arch/arm/include/asm/timex.h b/arch/arm/include/asm/timex.h
> index 6d1337c169cd..94e40c19cfc5 100644
> --- a/arch/arm/include/asm/timex.h
> +++ b/arch/arm/include/asm/timex.h
> @@ -10,7 +10,10 @@
> #define _ASMARM_TIMEX_H
>
> typedef unsigned long cycles_t;
> -#define get_cycles() ({ cycles_t c; read_current_timer(&c) ? 0 : c; })
> +// Temporary workaround until timex.h is cleaned up
> +bool delay_read_timer(unsigned long *t);
> +
> +#define get_cycles() ({ cycles_t c; delay_read_timer(&c) ? 0 : c; })
> #define random_get_entropy() (((unsigned long)get_cycles()) ?: random_get_entropy_fallback())
>
> #endif
> diff --git a/arch/arm/lib/delay.c b/arch/arm/lib/delay.c
> index b7fe84f68bf1..7e9304012a68 100644
> --- a/arch/arm/lib/delay.c
> +++ b/arch/arm/lib/delay.c
> @@ -12,7 +12,6 @@
> #include <linux/init.h>
> #include <linux/kernel.h>
> #include <linux/module.h>
> -#include <linux/timex.h>
>
> /*
> * Default to the loop-based delay implementation.
> @@ -27,15 +26,14 @@ static const struct delay_timer *delay_timer;
> static bool delay_calibrated;
> static u64 delay_res;
>
> -int read_current_timer(unsigned long *timer_val)
> +bool delay_read_timer(unsigned long *timer_val)
> {
> if (!delay_timer)
> - return -ENXIO;
> -
> + return false;
> *timer_val = delay_timer->read_current_timer();
> - return 0;
> + return true;
> }
> -EXPORT_SYMBOL_GPL(read_current_timer);
> +EXPORT_SYMBOL_GPL(delay_read_timer);
>
> static inline u64 cyc_to_ns(u64 cyc, u32 mult, u32 shift)
> {
I bisected a hang that I see after getting to userspace when virtually
testing some ARM configurations to commit dfc256dac54c ("calibrate:
Rework delay timer calibration"). It looks like the branches in
get_cycles() were not updated for the changed meaning of the return of
delay_read_timer()? This appears to resolve it for me but I am not sure
if I am missing something.
diff --git a/arch/arm/include/asm/timex.h b/arch/arm/include/asm/timex.h
index 94e40c19cfc5..4d31eab9dba2 100644
--- a/arch/arm/include/asm/timex.h
+++ b/arch/arm/include/asm/timex.h
@@ -13,7 +13,7 @@ typedef unsigned long cycles_t;
// Temporary workaround until timex.h is cleaned up
bool delay_read_timer(unsigned long *t);
-#define get_cycles() ({ cycles_t c; delay_read_timer(&c) ? 0 : c; })
+#define get_cycles() ({ cycles_t c; delay_read_timer(&c) ? c : 0; })
#define random_get_entropy() (((unsigned long)get_cycles()) ?: random_get_entropy_fallback())
#endif
--
# bad: [f4cdf7ca9a1fdcca413157df19753f388a5a224e] Merge tag 'media/v7.3-1' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media
# good: [8d3ae59288f1e7d58d76558a6ee96d533bc5019f] Linux 7.2
git bisect start 'f4cdf7ca9a1fdcca413157df19753f388a5a224e' 'v7.2'
# bad: [cbe8aadf1551f3f4a853f24b3b96d9f0eea49c53] Merge tag 'soc-dt-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
git bisect bad cbe8aadf1551f3f4a853f24b3b96d9f0eea49c53
# good: [762fffa407d8d24288513538cf5d17c2c5425258] Merge tag 'irq-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 762fffa407d8d24288513538cf5d17c2c5425258
# good: [0b02b8f941911895775b56c428e01b117ce9d2f3] Merge tag 'spacemit-dt-for-7.3-1' of https://git.kernel.org/pub/scm/linux/kernel/git/spacemit/linux into soc/dt
git bisect good 0b02b8f941911895775b56c428e01b117ce9d2f3
# good: [f809cedd3f614e4596aae23756ed44085fee1691] Merge tag 'qcom-arm32-for-7.3' of https://git.kernel.org/pub/scm/linux/kernel/git/qcom/linux into soc/dt
git bisect good f809cedd3f614e4596aae23756ed44085fee1691
# bad: [3b4128b9f374b4219eb716f4ad8a307bc7eb3d84] Merge tag 'timers-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad 3b4128b9f374b4219eb716f4ad8a307bc7eb3d84
# bad: [030c9f813b8e48d2b066983c94cf6294968f9496] Merge tag 'timers-cleanups-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad 030c9f813b8e48d2b066983c94cf6294968f9496
# good: [8b5d31123f41957fe265deeb03ee87fe62f155a8] Merge tag 'irq-drivers-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 8b5d31123f41957fe265deeb03ee87fe62f155a8
# good: [6c4a1b972643d72bd26f674677df8b5a6a3fe94f] selftests/futex: Provide thread creation and synchronization helpers
git bisect good 6c4a1b972643d72bd26f674677df8b5a6a3fe94f
# good: [66344732b058e75f7ef25dcd0fbb483f9c000633] smp: Remove preempt_disable() from smp_call_function()
git bisect good 66344732b058e75f7ef25dcd0fbb483f9c000633
# good: [0dd1a54f44348d9cf6bae57a2b5cb0b53826a2c7] Merge tag 'smp-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 0dd1a54f44348d9cf6bae57a2b5cb0b53826a2c7
# good: [3ed403bbc967a3138b8e37566510e9b062751372] treewide: Remove CLOCK_TICK_RATE
git bisect good 3ed403bbc967a3138b8e37566510e9b062751372
# bad: [dfc256dac54c8b692110bf905c64cb130e15963d] calibrate: Rework delay timer calibration
git bisect bad dfc256dac54c8b692110bf905c64cb130e15963d
# first 'bad' commit: [dfc256dac54c8b692110bf905c64cb130e15963d] calibrate: Rework delay timer calibration
--
Cheers,
Nathan
^ permalink raw reply related [flat|nested] 26+ messages in thread* Re: [GIT pull] timers/cleanups for v7.3-rc1
2026-08-19 18:48 ` Nathan Chancellor
@ 2026-08-19 20:59 ` Thomas Gleixner
2026-08-19 21:44 ` Nathan Chancellor
0 siblings, 1 reply; 26+ messages in thread
From: Thomas Gleixner @ 2026-08-19 20:59 UTC (permalink / raw)
To: Nathan Chancellor; +Cc: Linus Torvalds, linux-kernel, x86
On Wed, Aug 19 2026 at 11:48, Nathan Chancellor wrote:
>> -int read_current_timer(unsigned long *timer_val)
>> +bool delay_read_timer(unsigned long *timer_val)
>> {
>> if (!delay_timer)
>> - return -ENXIO;
>> -
>> + return false;
>> *timer_val = delay_timer->read_current_timer();
>> - return 0;
>> + return true;
>> }
>> -EXPORT_SYMBOL_GPL(read_current_timer);
>> +EXPORT_SYMBOL_GPL(delay_read_timer);
>>
>> static inline u64 cyc_to_ns(u64 cyc, u32 mult, u32 shift)
>> {
>
> I bisected a hang that I see after getting to userspace when virtually
> testing some ARM configurations to commit dfc256dac54c ("calibrate:
> Rework delay timer calibration"). It looks like the branches in
> get_cycles() were not updated for the changed meaning of the return of
> delay_read_timer()? This appears to resolve it for me but I am not sure
> if I am missing something.
>
> diff --git a/arch/arm/include/asm/timex.h b/arch/arm/include/asm/timex.h
> index 94e40c19cfc5..4d31eab9dba2 100644
> --- a/arch/arm/include/asm/timex.h
> +++ b/arch/arm/include/asm/timex.h
> @@ -13,7 +13,7 @@ typedef unsigned long cycles_t;
> // Temporary workaround until timex.h is cleaned up
> bool delay_read_timer(unsigned long *t);
>
> -#define get_cycles() ({ cycles_t c; delay_read_timer(&c) ? 0 : c; })
> +#define get_cycles() ({ cycles_t c; delay_read_timer(&c) ? c : 0; })
Duh, yes. Stupid me.
Can you please send a patch wuth a proper change log and a Fixes tag?
Thanks,
tglx
^ permalink raw reply [flat|nested] 26+ messages in thread
* [GIT pull] timers/clocksource for v7.3-rc1
2026-08-17 11:23 [GIT pull] core/entry for v7.3-rc1 Thomas Gleixner
` (5 preceding siblings ...)
2026-08-17 11:23 ` [GIT pull] timers/cleanups " Thomas Gleixner
@ 2026-08-17 11:23 ` Thomas Gleixner
2026-08-19 1:10 ` pr-tracker-bot
2026-08-19 1:10 ` pr-tracker-bot
2026-08-17 11:24 ` [GIT pull] timers/core " Thomas Gleixner
` (2 subsequent siblings)
9 siblings, 2 replies; 26+ messages in thread
From: Thomas Gleixner @ 2026-08-17 11:23 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest timers/clocksource branch from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers-clocksource-2026-08-17
up to: 0eaed89c18ae: Merge tag 'timers-v7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/daniel.lezcano/linux into timers/clocksource
Clocksource/event driver updates:
- Remove redundant dev_err() and dev_err_probe() invocations in various
drivers.
- Handle SWAP_IO_SPACE correctly in the rtl-otto driver
- Make the Samsumg PWM timer driver PREEMPT_RT compatible
- Ensure that the SUN4I timer is programmed with a delta larger than zero
as a zero delta causes the the timer to be disabled.
- The usual fixes and improvements all over the place
Thanks,
tglx
------------------>
Felix Yan (1):
clocksource/drivers/timer-sun4i: Advertise a real minimum delta
Guangshuo Li (1):
clocksource/drivers/clps711x: Do not unmap clocksource MMIO
Marek Szyprowski (1):
clocksource/drivers/samsung_pwm: Switch to raw_spinlock_t type
Pan Chuang (1):
clocksource: Remove redundant dev_err()/dev_err_probe()
Rustam Adilov (1):
clocksource/drivers/rtl-otto: Change driver to use __raw reads and writes
Uwe Kleine-König (The Capable Hub) (2):
clocksource/drivers/sh_mtu2: Drop unused assignment of platform_device_id
clocksource/drivers/sh_cmt: Use named initializers for platform_device_id arrays
WenTao Liang (1):
clocksource/drivers/nxp-pit: Fix IRQ leak on cpuhp_setup_state error path
Yuho Choi (1):
clocksource/drivers/armada: Unwind timer clock on init failure
drivers/clocksource/arm_arch_timer_mmio.c | 4 +---
drivers/clocksource/clps711x-timer.c | 2 +-
drivers/clocksource/em_sti.c | 4 +---
drivers/clocksource/samsung_pwm_timer.c | 22 +++++++++++-----------
drivers/clocksource/sh_cmt.c | 4 ++--
drivers/clocksource/sh_mtu2.c | 4 ++--
drivers/clocksource/sh_tmu.c | 4 ++--
drivers/clocksource/timer-armada-370-xp.c | 18 +++++++++++++++---
drivers/clocksource/timer-nxp-pit.c | 4 +++-
drivers/clocksource/timer-nxp-stm.c | 2 +-
drivers/clocksource/timer-rtl-otto.c | 14 +++++++-------
drivers/clocksource/timer-sun4i.c | 2 +-
drivers/clocksource/timer-sun5i.c | 4 +---
drivers/clocksource/timer-tegra186.c | 4 +---
drivers/clocksource/timer-ti-dm.c | 4 +---
drivers/pwm/pwm-samsung.c | 22 +++++++++++-----------
include/clocksource/samsung_pwm.h | 2 +-
17 files changed, 62 insertions(+), 58 deletions(-)
diff --git a/drivers/clocksource/arm_arch_timer_mmio.c b/drivers/clocksource/arm_arch_timer_mmio.c
index d10362692fdd..d678f764d3bb 100644
--- a/drivers/clocksource/arm_arch_timer_mmio.c
+++ b/drivers/clocksource/arm_arch_timer_mmio.c
@@ -313,10 +313,8 @@ static int arch_timer_mmio_frame_register(struct platform_device *pdev,
ret = devm_request_irq(&pdev->dev, irq, arch_timer_mmio_handler,
IRQF_TIMER | IRQF_NO_AUTOEN, "arch_mem_timer",
&at->evt);
- if (ret) {
- dev_err(&pdev->dev, "Failed to request mem timer irq\n");
+ if (ret)
return ret;
- }
/* Afer this point, we're not allowed to fail anymore */
arch_timer_mmio_setup(at, irq);
diff --git a/drivers/clocksource/clps711x-timer.c b/drivers/clocksource/clps711x-timer.c
index bb0a44adaf28..63ae3a691b14 100644
--- a/drivers/clocksource/clps711x-timer.c
+++ b/drivers/clocksource/clps711x-timer.c
@@ -94,7 +94,7 @@ static int __init clps711x_timer_init(struct device_node *np)
switch (of_alias_get_id(np, "timer")) {
case CLPS711X_CLKSRC_CLOCKSOURCE:
clps711x_clksrc_init(clock, base);
- break;
+ return 0;
case CLPS711X_CLKSRC_CLOCKEVENT:
ret = _clps711x_clkevt_init(clock, base, irq);
break;
diff --git a/drivers/clocksource/em_sti.c b/drivers/clocksource/em_sti.c
index ca8d29ab70da..73a3357d173d 100644
--- a/drivers/clocksource/em_sti.c
+++ b/drivers/clocksource/em_sti.c
@@ -300,10 +300,8 @@ static int em_sti_probe(struct platform_device *pdev)
ret = devm_request_irq(&pdev->dev, irq, em_sti_interrupt,
IRQF_TIMER | IRQF_IRQPOLL | IRQF_NOBALANCING,
dev_name(&pdev->dev), p);
- if (ret) {
- dev_err(&pdev->dev, "failed to request low IRQ\n");
+ if (ret)
return ret;
- }
/* get hold of clock */
p->clk = devm_clk_get(&pdev->dev, "sclk");
diff --git a/drivers/clocksource/samsung_pwm_timer.c b/drivers/clocksource/samsung_pwm_timer.c
index b9561e3f196c..0544124cf5ce 100644
--- a/drivers/clocksource/samsung_pwm_timer.c
+++ b/drivers/clocksource/samsung_pwm_timer.c
@@ -56,7 +56,7 @@
#define TCON_AUTORELOAD(chan) \
((chan < 5) ? _TCON_AUTORELOAD(chan) : _TCON_AUTORELOAD4(chan))
-DEFINE_SPINLOCK(samsung_pwm_lock);
+DEFINE_RAW_SPINLOCK(samsung_pwm_lock);
EXPORT_SYMBOL(samsung_pwm_lock);
struct samsung_pwm_clocksource {
@@ -87,14 +87,14 @@ static void samsung_timer_set_prescale(unsigned int channel, u16 prescale)
if (channel >= 2)
shift = TCFG0_PRESCALER1_SHIFT;
- spin_lock_irqsave(&samsung_pwm_lock, flags);
+ raw_spin_lock_irqsave(&samsung_pwm_lock, flags);
reg = readl(pwm.base + REG_TCFG0);
reg &= ~(TCFG0_PRESCALER_MASK << shift);
reg |= (prescale - 1) << shift;
writel(reg, pwm.base + REG_TCFG0);
- spin_unlock_irqrestore(&samsung_pwm_lock, flags);
+ raw_spin_unlock_irqrestore(&samsung_pwm_lock, flags);
}
static void samsung_timer_set_divisor(unsigned int channel, u8 divisor)
@@ -106,14 +106,14 @@ static void samsung_timer_set_divisor(unsigned int channel, u8 divisor)
bits = (fls(divisor) - 1) - pwm.variant.div_base;
- spin_lock_irqsave(&samsung_pwm_lock, flags);
+ raw_spin_lock_irqsave(&samsung_pwm_lock, flags);
reg = readl(pwm.base + REG_TCFG1);
reg &= ~(TCFG1_MUX_MASK << shift);
reg |= bits << shift;
writel(reg, pwm.base + REG_TCFG1);
- spin_unlock_irqrestore(&samsung_pwm_lock, flags);
+ raw_spin_unlock_irqrestore(&samsung_pwm_lock, flags);
}
static void samsung_time_stop(unsigned int channel)
@@ -124,13 +124,13 @@ static void samsung_time_stop(unsigned int channel)
if (channel > 0)
++channel;
- spin_lock_irqsave(&samsung_pwm_lock, flags);
+ raw_spin_lock_irqsave(&samsung_pwm_lock, flags);
tcon = readl_relaxed(pwm.base + REG_TCON);
tcon &= ~TCON_START(channel);
writel_relaxed(tcon, pwm.base + REG_TCON);
- spin_unlock_irqrestore(&samsung_pwm_lock, flags);
+ raw_spin_unlock_irqrestore(&samsung_pwm_lock, flags);
}
static void samsung_time_setup(unsigned int channel, unsigned long tcnt)
@@ -142,7 +142,7 @@ static void samsung_time_setup(unsigned int channel, unsigned long tcnt)
if (tcon_chan > 0)
++tcon_chan;
- spin_lock_irqsave(&samsung_pwm_lock, flags);
+ raw_spin_lock_irqsave(&samsung_pwm_lock, flags);
tcon = readl_relaxed(pwm.base + REG_TCON);
@@ -153,7 +153,7 @@ static void samsung_time_setup(unsigned int channel, unsigned long tcnt)
writel_relaxed(tcnt, pwm.base + REG_TCMPB(channel));
writel_relaxed(tcon, pwm.base + REG_TCON);
- spin_unlock_irqrestore(&samsung_pwm_lock, flags);
+ raw_spin_unlock_irqrestore(&samsung_pwm_lock, flags);
}
static void samsung_time_start(unsigned int channel, bool periodic)
@@ -164,7 +164,7 @@ static void samsung_time_start(unsigned int channel, bool periodic)
if (channel > 0)
++channel;
- spin_lock_irqsave(&samsung_pwm_lock, flags);
+ raw_spin_lock_irqsave(&samsung_pwm_lock, flags);
tcon = readl_relaxed(pwm.base + REG_TCON);
@@ -178,7 +178,7 @@ static void samsung_time_start(unsigned int channel, bool periodic)
writel_relaxed(tcon, pwm.base + REG_TCON);
- spin_unlock_irqrestore(&samsung_pwm_lock, flags);
+ raw_spin_unlock_irqrestore(&samsung_pwm_lock, flags);
}
static int samsung_set_next_event(unsigned long cycles,
diff --git a/drivers/clocksource/sh_cmt.c b/drivers/clocksource/sh_cmt.c
index cf057f531a58..7977507f6ce3 100644
--- a/drivers/clocksource/sh_cmt.c
+++ b/drivers/clocksource/sh_cmt.c
@@ -974,8 +974,8 @@ static int sh_cmt_map_memory(struct sh_cmt_device *cmt)
}
static const struct platform_device_id sh_cmt_id_table[] = {
- { "sh-cmt-16", (kernel_ulong_t)&sh_cmt_info[SH_CMT_16BIT] },
- { "sh-cmt-32", (kernel_ulong_t)&sh_cmt_info[SH_CMT_32BIT] },
+ { .name = "sh-cmt-16", .driver_data = (kernel_ulong_t)&sh_cmt_info[SH_CMT_16BIT] },
+ { .name = "sh-cmt-32", .driver_data = (kernel_ulong_t)&sh_cmt_info[SH_CMT_32BIT] },
{ }
};
MODULE_DEVICE_TABLE(platform, sh_cmt_id_table);
diff --git a/drivers/clocksource/sh_mtu2.c b/drivers/clocksource/sh_mtu2.c
index 1997639b113e..3aca86d6a2d4 100644
--- a/drivers/clocksource/sh_mtu2.c
+++ b/drivers/clocksource/sh_mtu2.c
@@ -484,8 +484,8 @@ static int sh_mtu2_probe(struct platform_device *pdev)
}
static const struct platform_device_id sh_mtu2_id_table[] = {
- { "sh-mtu2", 0 },
- { },
+ { .name = "sh-mtu2" },
+ { }
};
MODULE_DEVICE_TABLE(platform, sh_mtu2_id_table);
diff --git a/drivers/clocksource/sh_tmu.c b/drivers/clocksource/sh_tmu.c
index 8d6a9e279f73..8b3cfa6727bd 100644
--- a/drivers/clocksource/sh_tmu.c
+++ b/drivers/clocksource/sh_tmu.c
@@ -614,8 +614,8 @@ static int sh_tmu_probe(struct platform_device *pdev)
}
static const struct platform_device_id sh_tmu_id_table[] = {
- { "sh-tmu", SH_TMU },
- { "sh-tmu-sh3", SH_TMU_SH3 },
+ { .name = "sh-tmu", .driver_data = SH_TMU },
+ { .name = "sh-tmu-sh3", .driver_data = SH_TMU_SH3 },
{ }
};
MODULE_DEVICE_TABLE(platform, sh_tmu_id_table);
diff --git a/drivers/clocksource/timer-armada-370-xp.c b/drivers/clocksource/timer-armada-370-xp.c
index a405a084cf72..b5a984aa1cbb 100644
--- a/drivers/clocksource/timer-armada-370-xp.c
+++ b/drivers/clocksource/timer-armada-370-xp.c
@@ -349,7 +349,11 @@ static int __init armada_xp_timer_init(struct device_node *np)
timer_clk = clk_get_rate(clk);
- return armada_370_xp_timer_common_init(np);
+ ret = armada_370_xp_timer_common_init(np);
+ if (ret)
+ clk_disable_unprepare(clk);
+
+ return ret;
}
TIMER_OF_DECLARE(armada_xp, "marvell,armada-xp-timer",
armada_xp_timer_init);
@@ -387,7 +391,11 @@ static int __init armada_375_timer_init(struct device_node *np)
timer25Mhz = false;
}
- return armada_370_xp_timer_common_init(np);
+ ret = armada_370_xp_timer_common_init(np);
+ if (ret)
+ clk_disable_unprepare(clk);
+
+ return ret;
}
TIMER_OF_DECLARE(armada_375, "marvell,armada-375-timer",
armada_375_timer_init);
@@ -410,7 +418,11 @@ static int __init armada_370_timer_init(struct device_node *np)
timer_clk = clk_get_rate(clk) / TIMER_DIVIDER;
timer25Mhz = false;
- return armada_370_xp_timer_common_init(np);
+ ret = armada_370_xp_timer_common_init(np);
+ if (ret)
+ clk_disable_unprepare(clk);
+
+ return ret;
}
TIMER_OF_DECLARE(armada_370, "marvell,armada-370-timer",
armada_370_timer_init);
diff --git a/drivers/clocksource/timer-nxp-pit.c b/drivers/clocksource/timer-nxp-pit.c
index bc5157e2ba57..2f70d1d5e21b 100644
--- a/drivers/clocksource/timer-nxp-pit.c
+++ b/drivers/clocksource/timer-nxp-pit.c
@@ -328,8 +328,10 @@ static int pit_timer_init(struct device_node *np)
if (pit_instances == max_pit_instances) {
ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "PIT timer:starting",
pit_clockevent_starting_cpu, NULL);
- if (ret < 0)
+ if (ret < 0) {
+ pit_clockevent_per_cpu_exit(pit, pit_instances);
goto out_pit_clocksource_unregister;
+ }
}
return 0;
diff --git a/drivers/clocksource/timer-nxp-stm.c b/drivers/clocksource/timer-nxp-stm.c
index 1ab907233f48..6fe098a4a33f 100644
--- a/drivers/clocksource/timer-nxp-stm.c
+++ b/drivers/clocksource/timer-nxp-stm.c
@@ -441,7 +441,7 @@ static int nxp_stm_timer_probe(struct platform_device *pdev)
ret = devm_request_irq(dev, irq, nxp_stm_module_interrupt,
IRQF_TIMER | IRQF_NOBALANCING, name, stm_timer);
if (ret)
- return dev_err_probe(dev, ret, "Unable to allocate interrupt line\n");
+ return ret;
ret = nxp_stm_clocksource_init(dev, stm_timer, name, base, clk);
if (ret)
diff --git a/drivers/clocksource/timer-rtl-otto.c b/drivers/clocksource/timer-rtl-otto.c
index dd236a7babee..0d1b9a01a94c 100644
--- a/drivers/clocksource/timer-rtl-otto.c
+++ b/drivers/clocksource/timer-rtl-otto.c
@@ -56,37 +56,37 @@ struct rttm_cs {
/* Simple internal register functions */
static inline unsigned int rttm_get_counter(void __iomem *base)
{
- return ioread32(base + RTTM_CNT);
+ return __raw_readl(base + RTTM_CNT);
}
static inline void rttm_set_period(void __iomem *base, unsigned int period)
{
- iowrite32(period, base + RTTM_DATA);
+ __raw_writel(period, base + RTTM_DATA);
}
static inline void rttm_disable_timer(void __iomem *base)
{
- iowrite32(0, base + RTTM_CTRL);
+ __raw_writel(0, base + RTTM_CTRL);
}
static inline void rttm_enable_timer(void __iomem *base, u32 mode, u32 divisor)
{
- iowrite32(RTTM_CTRL_ENABLE | mode | divisor, base + RTTM_CTRL);
+ __raw_writel(RTTM_CTRL_ENABLE | mode | divisor, base + RTTM_CTRL);
}
static inline void rttm_ack_irq(void __iomem *base)
{
- iowrite32(ioread32(base + RTTM_INT) | RTTM_INT_PENDING, base + RTTM_INT);
+ __raw_writel(__raw_readl(base + RTTM_INT) | RTTM_INT_PENDING, base + RTTM_INT);
}
static inline void rttm_enable_irq(void __iomem *base)
{
- iowrite32(RTTM_INT_ENABLE, base + RTTM_INT);
+ __raw_writel(RTTM_INT_ENABLE, base + RTTM_INT);
}
static inline void rttm_disable_irq(void __iomem *base)
{
- iowrite32(0, base + RTTM_INT);
+ __raw_writel(0, base + RTTM_INT);
}
/* Aggregated control functions for kernel clock framework */
diff --git a/drivers/clocksource/timer-sun4i.c b/drivers/clocksource/timer-sun4i.c
index 7bdcc60ad43c..c2d04ab7cf2d 100644
--- a/drivers/clocksource/timer-sun4i.c
+++ b/drivers/clocksource/timer-sun4i.c
@@ -208,7 +208,7 @@ static int __init sun4i_timer_init(struct device_node *node)
sun4i_timer_clear_interrupt(timer_of_base(&to));
clockevents_config_and_register(&to.clkevt, timer_of_rate(&to),
- TIMER_SYNC_TICKS, 0xffffffff);
+ TIMER_SYNC_TICKS + 1, 0xffffffff);
/* Enable timer0 interrupt */
val = readl(timer_of_base(&to) + TIMER_IRQ_EN_REG);
diff --git a/drivers/clocksource/timer-sun5i.c b/drivers/clocksource/timer-sun5i.c
index 6ab300d22621..bcf155fb9cac 100644
--- a/drivers/clocksource/timer-sun5i.c
+++ b/drivers/clocksource/timer-sun5i.c
@@ -247,10 +247,8 @@ static int sun5i_setup_clockevent(struct platform_device *pdev,
ret = devm_request_irq(dev, irq, sun5i_timer_interrupt,
IRQF_TIMER | IRQF_IRQPOLL,
"sun5i_timer0", ce);
- if (ret) {
- dev_err(dev, "Unable to register interrupt\n");
+ if (ret)
return ret;
- }
return 0;
}
diff --git a/drivers/clocksource/timer-tegra186.c b/drivers/clocksource/timer-tegra186.c
index 78600ddeb1c6..0f626ecf61b0 100644
--- a/drivers/clocksource/timer-tegra186.c
+++ b/drivers/clocksource/timer-tegra186.c
@@ -532,10 +532,8 @@ static int tegra186_timer_probe(struct platform_device *pdev)
if (kernel_wdt) {
err = devm_request_irq(dev, irq, tegra186_wdt_irq, 0,
dev_name(dev), kernel_wdt);
- if (err < 0) {
- dev_err(dev, "failed to request kernel WDT IRQ: %d\n", err);
+ if (err < 0)
goto unregister_usec;
- }
tegra186_wdt_set_timeout(&kernel_wdt->base, TEGRA186_KERNEL_WDT_TIMEOUT);
tegra186_wdt_enable(kernel_wdt);
diff --git a/drivers/clocksource/timer-ti-dm.c b/drivers/clocksource/timer-ti-dm.c
index bd06afb7d522..6787acac9a43 100644
--- a/drivers/clocksource/timer-ti-dm.c
+++ b/drivers/clocksource/timer-ti-dm.c
@@ -1375,10 +1375,8 @@ static int omap_dm_timer_setup_clockevent(struct dmtimer *timer)
ret = devm_request_irq(dev, timer->irq, omap_dm_timer_evt_interrupt,
IRQF_TIMER, "omap_dm_timer_clockevent", clkevt);
- if (ret) {
- dev_err(dev, "Failed to request interrupt: %d\n", ret);
+ if (ret)
return ret;
- }
__omap_dm_timer_int_enable(timer, OMAP_TIMER_INT_OVERFLOW);
diff --git a/drivers/pwm/pwm-samsung.c b/drivers/pwm/pwm-samsung.c
index 951b38ff5f8e..14fb460a4565 100644
--- a/drivers/pwm/pwm-samsung.c
+++ b/drivers/pwm/pwm-samsung.c
@@ -102,7 +102,7 @@ struct samsung_pwm_chip {
* IP. Should this change, both drivers will need to be modified to
* properly synchronize accesses to particular instances.
*/
-static DEFINE_SPINLOCK(samsung_pwm_lock);
+static DEFINE_RAW_SPINLOCK(samsung_pwm_lock);
#endif
static inline
@@ -141,14 +141,14 @@ static void pwm_samsung_set_divisor(struct samsung_pwm_chip *our_chip,
bits = (fls(divisor) - 1) - our_chip->variant.div_base;
- spin_lock_irqsave(&samsung_pwm_lock, flags);
+ raw_spin_lock_irqsave(&samsung_pwm_lock, flags);
reg = readl(our_chip->base + REG_TCFG1);
reg &= ~(TCFG1_MUX_MASK << shift);
reg |= bits << shift;
writel(reg, our_chip->base + REG_TCFG1);
- spin_unlock_irqrestore(&samsung_pwm_lock, flags);
+ raw_spin_unlock_irqrestore(&samsung_pwm_lock, flags);
}
static int pwm_samsung_is_tdiv(struct samsung_pwm_chip *our_chip, unsigned int chan)
@@ -249,7 +249,7 @@ static int pwm_samsung_enable(struct pwm_chip *chip, struct pwm_device *pwm)
unsigned long flags;
u32 tcon;
- spin_lock_irqsave(&samsung_pwm_lock, flags);
+ raw_spin_lock_irqsave(&samsung_pwm_lock, flags);
tcon = readl(our_chip->base + REG_TCON);
@@ -263,7 +263,7 @@ static int pwm_samsung_enable(struct pwm_chip *chip, struct pwm_device *pwm)
our_chip->disabled_mask &= ~BIT(pwm->hwpwm);
- spin_unlock_irqrestore(&samsung_pwm_lock, flags);
+ raw_spin_unlock_irqrestore(&samsung_pwm_lock, flags);
return 0;
}
@@ -275,7 +275,7 @@ static void pwm_samsung_disable(struct pwm_chip *chip, struct pwm_device *pwm)
unsigned long flags;
u32 tcon;
- spin_lock_irqsave(&samsung_pwm_lock, flags);
+ raw_spin_lock_irqsave(&samsung_pwm_lock, flags);
tcon = readl(our_chip->base + REG_TCON);
tcon &= ~TCON_AUTORELOAD(tcon_chan);
@@ -290,7 +290,7 @@ static void pwm_samsung_disable(struct pwm_chip *chip, struct pwm_device *pwm)
our_chip->disabled_mask |= BIT(pwm->hwpwm);
- spin_unlock_irqrestore(&samsung_pwm_lock, flags);
+ raw_spin_unlock_irqrestore(&samsung_pwm_lock, flags);
}
static void pwm_samsung_manual_update(struct samsung_pwm_chip *our_chip,
@@ -298,11 +298,11 @@ static void pwm_samsung_manual_update(struct samsung_pwm_chip *our_chip,
{
unsigned long flags;
- spin_lock_irqsave(&samsung_pwm_lock, flags);
+ raw_spin_lock_irqsave(&samsung_pwm_lock, flags);
__pwm_samsung_manual_update(our_chip, pwm);
- spin_unlock_irqrestore(&samsung_pwm_lock, flags);
+ raw_spin_unlock_irqrestore(&samsung_pwm_lock, flags);
}
static int __pwm_samsung_config(struct pwm_chip *chip, struct pwm_device *pwm,
@@ -390,7 +390,7 @@ static void pwm_samsung_set_invert(struct samsung_pwm_chip *our_chip,
unsigned long flags;
u32 tcon;
- spin_lock_irqsave(&samsung_pwm_lock, flags);
+ raw_spin_lock_irqsave(&samsung_pwm_lock, flags);
tcon = readl(our_chip->base + REG_TCON);
@@ -404,7 +404,7 @@ static void pwm_samsung_set_invert(struct samsung_pwm_chip *our_chip,
writel(tcon, our_chip->base + REG_TCON);
- spin_unlock_irqrestore(&samsung_pwm_lock, flags);
+ raw_spin_unlock_irqrestore(&samsung_pwm_lock, flags);
}
static int pwm_samsung_set_polarity(struct pwm_chip *chip,
diff --git a/include/clocksource/samsung_pwm.h b/include/clocksource/samsung_pwm.h
index 9b435caa95fe..36f6f246e559 100644
--- a/include/clocksource/samsung_pwm.h
+++ b/include/clocksource/samsung_pwm.h
@@ -15,7 +15,7 @@
* spinlock is not shared between both drivers.
*/
#ifdef CONFIG_CLKSRC_SAMSUNG_PWM
-extern spinlock_t samsung_pwm_lock;
+extern raw_spinlock_t samsung_pwm_lock;
#endif
struct samsung_pwm_variant {
^ permalink raw reply related [flat|nested] 26+ messages in thread* Re: [GIT pull] timers/clocksource for v7.3-rc1
2026-08-17 11:23 ` [GIT pull] timers/clocksource " Thomas Gleixner
@ 2026-08-19 1:10 ` pr-tracker-bot
2026-08-19 1:10 ` pr-tracker-bot
1 sibling, 0 replies; 26+ messages in thread
From: pr-tracker-bot @ 2026-08-19 1:10 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, linux-kernel, x86
The pull request you sent on Mon, 17 Aug 2026 13:23:57 +0200:
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers-clocksource-2026-08-17
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/b0239dd672306ad242545f793132938847f17e53
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [GIT pull] timers/clocksource for v7.3-rc1
2026-08-17 11:23 ` [GIT pull] timers/clocksource " Thomas Gleixner
2026-08-19 1:10 ` pr-tracker-bot
@ 2026-08-19 1:10 ` pr-tracker-bot
1 sibling, 0 replies; 26+ messages in thread
From: pr-tracker-bot @ 2026-08-19 1:10 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, linux-kernel, x86
The pull request you sent on Mon, 17 Aug 2026 13:23:57 +0200:
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers-clocksource-2026-08-17
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/b0239dd672306ad242545f793132938847f17e53
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html
^ permalink raw reply [flat|nested] 26+ messages in thread
* [GIT pull] timers/core for v7.3-rc1
2026-08-17 11:23 [GIT pull] core/entry for v7.3-rc1 Thomas Gleixner
` (6 preceding siblings ...)
2026-08-17 11:23 ` [GIT pull] timers/clocksource " Thomas Gleixner
@ 2026-08-17 11:24 ` Thomas Gleixner
2026-08-19 1:10 ` pr-tracker-bot
2026-08-19 1:10 ` pr-tracker-bot
2026-08-17 11:24 ` [GIT pull] timers/vdso " Thomas Gleixner
2026-08-19 1:10 ` [GIT pull] core/entry " pr-tracker-bot
9 siblings, 2 replies; 26+ messages in thread
From: Thomas Gleixner @ 2026-08-17 11:24 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest timers/core branch from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers-core-2026-08-17
up to: 4fa377c19e11: selftests: timers: nsleep-lat: Check all calls to clock_nanosleep() and clock_gettime()
Timers and timekeeping core updates:
- Fix a subtly inconsistency in the timekeeping code, which fails to
account for the monotonicity adjustment in ntp_error. For small changes
of the clocksource multiplicator (+/-1) which are typically used by the
NTP PLL this is hardly to observe. But for larger adjustments,
e.g. caused by a direct frequency setting through adjtimex() the
one-time uncompensated offset is significant.
Cure this by adjusting ntp_error with the resulting offset so that the
discrepancy is smoothed away over time
- Make tick length calculations correct in NTP. The timekeeping core
takes the quantisation of the clocksource into account when calculating
the tick length to compensate for the deviation of the nominal
NTP_INTERVAL_LENGTH. While timekeeping gets this right, NTP is not
aware of that, which means it operates on the nominal value and not on
the actual value which is determined by the clock source frequency. The
rounding of a coarse clocksource like the ACPI PM timer results in a
+127 PPM deviation.
Cure this by exposing the deviation to the NTP code so that it can
operate on the same data as the timekeeping core. This is purely kernel
internal. User space still sees the nominal tick lenght via adjtimex().
- The accuracy of the NTP adjustments is fairly approximate as the code
assumes that the invocations are precisely in NTP interval frequency
ticks and the final adjustment can over and under-run.
Cure this by adjusting ntp_error by the intended skew on each tick to
achieve the desired rate.
- Handle the two competing skews of time offset and time adjustment
correctly by calculating the conflict portion between the skews and
adjusting both accordingly.
- A set of updates and improvements for the selftests
- The usual small fixes and improvements all over the place
Thanks,
tglx
------------------>
Babanpreet Singh (1):
timers/migration: Fix bad line kernel-doc warning in struct tmigr_cpu
David Woodhouse (7):
MAINTAINERS: Add Miroslav as timekeeping reviewer
timekeeping: Account for monotonicity adjustment in ntp_error
timekeeping: Account for clocksource tick quantisation via NTP
timekeeping: Drive time_offset skew via per-tick ntp_error transfer
timekeeping: Drive time_adjust skew via per-tick ntp_error transfer
timekeeping: Settle competing time_offset and time_adjust skew
ntp: Remove tick_length_base, use tick_length directly
Dennis Moshegov (1):
timekeeping: Use READ_ONCE/WRITE_ONCE() for xtime_sec to prevent tearing
Jiangshan Yi (1):
selftests: timers: leap-a-day: Fix -w option and update usage comment
Julian Braha (1):
nohz: Replace dead select with choice default
Jérémy Jean (1):
timers/itimer: Zero-init old itimerval before copy to userspace
Liang Hao (1):
hrtimer: Account nr_retries on recovered interrupt retries
Malaya Kumar Rout (2):
timers/migration: Fix memory leak in tmigr_setup_groups() error path
time/namespace: Validate nanosecond field in proc_timens_set_offset()
Oleg Nesterov (1):
posix-cpu-timers: Don't abuse lock_task_sighand() in handle_posix_cpu_timers()
Randy Dunlap (2):
posix-timers: Clean up kernel-doc warnings
posix-cpu-timers: Avoid kernel-doc warnings
Thomas Gleixner (1):
timekeeping: Document monotonic raw timestamps in snapshots correctly
Thomas Weißschuh (3):
timekeeping: Fold vdso_time_update_aux() declarations into the generic ifdeffery
timekeeping: Move the vDSO update declarations into a private header
y2038: uapi: Use 64-bit __kernel_old_timespec::tv_nsec on x32
Thomas Weißschuh (Schneider Electric) (28):
hrtimer: Rename hrtimer_defs.h to hrtimer_bases.h
hrtimer: Move hrtimer_callback_running() to hrtimer_bases.h
hrtimer: Move hrtimer_update_function() to hrtimer.c
tick: Explicitly include linux/hrtimer_bases.h
hrtimer: Explicitly include linux/hrtimer_bases.h
hrtimer: Explicitly include some necessary headers in hrtimer_rearm.h
x86/speculation: Explicitly include linux/types.h
hrtimer: Remove inclusion of hrtimer_bases.h remove from hrtimer.h
timekeeping: Check the return value of tk_get_aux_ts64 in __do_adjtimex()
timekeeping: Remove the unused ktime_get_clock_ts64()
timekeeping: Annotate auxiliary clock accessors with __must_check
timekeeping: Rename clockid_aux_valid() to clockid_is_aux_clock()
timekeeping: Use u32 for clock_was_set_seq
hrtimer: Add a lockdep assertion to hrtimer_update_base()
timer_list: Use standard 'long long' format placeholders
timer_list: Use ktime_t over nanoseconds
selftests: Add clock-helpers.h
selftests: timers: Use clock_name() and constants from clock-helpers.h
selftests: timers: nanosleep: Drop output alignment
selftests: timers: nanosleep: Explicitly list the tested clocks
selftests: timers: nanosleep: Reuse kselftest error numbers
selftests: timers: nanosleep: Move all single clock tests out of the loop in main()
selftests: timers: nanosleep: Explicitly handle timer_delete() failure
selftests: timers: nanosleep: Report each test separately
selftests: timers: nsleep-lat: Use NSEC_PER_MSEC define for unreasonable latency
selftests: timers: nsleep-lat: Explicitly list the tested clocks
selftests: timers: nsleep-lat: Reuse kselftest error numbers
selftests: timers: nsleep-lat: Check all calls to clock_nanosleep() and clock_gettime()
Usama Arif (1):
hrtimer: Don't take cpu_base::lock in hrtimer_get_next_event() when hres_active
Wake Liu (1):
selftests: timers: Partially revert "Remove local NSEC_PER_SEC and USEC_PER_SEC defines"
Yu Peng (1):
timer_list: Annotate print_cpu() diagnostic reads
Yuho Choi (2):
clocksource: Unregister subsystem on device registration failure
timekeeping: Unwind aux clock sysfs children on failure
Zhan Xusheng (3):
hrtimer: Remove unused clock_base_next_timer_safe()
hrtimer: Remove unused next_timer argument from __hrtimer_reprogram()
clocksource: Remove unused WATCHDOG_INTERVAL_NS macro
MAINTAINERS | 1 +
arch/x86/include/asm/nospec-branch.h | 1 +
include/linux/hrtimer.h | 34 +--
include/linux/{hrtimer_defs.h => hrtimer_bases.h} | 16 +-
include/linux/hrtimer_rearm.h | 5 +
include/linux/posix-timers.h | 5 +-
include/linux/posix-timers_types.h | 6 +-
include/linux/timekeeper_internal.h | 32 +--
include/linux/timekeeping.h | 24 +-
include/uapi/linux/time_types.h | 2 +-
init/Kconfig | 1 +
kernel/sched/fair.c | 2 +
kernel/time/Kconfig | 1 -
kernel/time/clocksource.c | 9 +-
kernel/time/hrtimer.c | 64 +++--
kernel/time/itimer.c | 2 +-
kernel/time/namespace.c | 6 +-
kernel/time/ntp.c | 310 ++++++++++++++++++---
kernel/time/ntp_internal.h | 4 +-
kernel/time/posix-cpu-timers.c | 9 +-
kernel/time/tick-internal.h | 1 +
kernel/time/time.c | 1 +
kernel/time/timekeeping.c | 204 +++++++++-----
kernel/time/timekeeping.h | 2 +-
kernel/time/timekeeping_internal.h | 21 ++
kernel/time/timer_list.c | 78 +++---
kernel/time/timer_migration.c | 4 +-
kernel/time/timer_migration.h | 2 +-
sound/drivers/dummy.c | 1 +
tools/testing/selftests/clock-helpers.h | 76 +++++
tools/testing/selftests/timers/Makefile | 2 +-
tools/testing/selftests/timers/adjtick.c | 2 +-
.../testing/selftests/timers/alarmtimer-suspend.c | 37 +--
.../testing/selftests/timers/inconsistency-check.c | 37 +--
tools/testing/selftests/timers/leap-a-day.c | 21 +-
tools/testing/selftests/timers/mqueue-lat.c | 2 +-
tools/testing/selftests/timers/nanosleep.c | 144 ++++------
tools/testing/selftests/timers/nsleep-lat.c | 101 +++----
tools/testing/selftests/timers/posix_timers.c | 34 ++-
tools/testing/selftests/timers/raw_skew.c | 2 +-
tools/testing/selftests/timers/set-2038.c | 2 +-
tools/testing/selftests/timers/set-timer-lat.c | 40 +--
tools/testing/selftests/timers/valid-adjtimex.c | 2 +-
43 files changed, 819 insertions(+), 531 deletions(-)
rename include/linux/{hrtimer_defs.h => hrtimer_bases.h} (92%)
create mode 100644 tools/testing/selftests/clock-helpers.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 15011f5752a9..ff01c39e7f23 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -27199,6 +27199,7 @@ TIMEKEEPING, CLOCKSOURCE CORE, NTP, ALARMTIMER
M: John Stultz <jstultz@google.com>
M: Thomas Gleixner <tglx@kernel.org>
R: Stephen Boyd <sboyd@kernel.org>
+R: Miroslav Lichvar <mlichvar@redhat.com>
L: linux-kernel@vger.kernel.org
S: Supported
T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers/core
diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index 4f4b5e8a1574..6734203595fe 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -6,6 +6,7 @@
#include <linux/static_key.h>
#include <linux/objtool.h>
#include <linux/linkage.h>
+#include <linux/types.h>
#include <asm/alternative.h>
#include <asm/cpufeatures.h>
diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
index 6862dea0acc5..29072d89e5cb 100644
--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -12,7 +12,6 @@
#ifndef _LINUX_HRTIMER_H
#define _LINUX_HRTIMER_H
-#include <linux/hrtimer_defs.h>
#include <linux/hrtimer_rearm.h>
#include <linux/hrtimer_types.h>
#include <linux/init.h>
@@ -287,37 +286,8 @@ static inline bool hrtimer_is_queued(struct hrtimer *timer)
return READ_ONCE(timer->is_queued);
}
-/*
- * Helper function to check, whether the timer is running the callback
- * function
- */
-static inline int hrtimer_callback_running(struct hrtimer *timer)
-{
- return timer->base->running == timer;
-}
-
-/**
- * hrtimer_update_function - Update the timer's callback function
- * @timer: Timer to update
- * @function: New callback function
- *
- * Only safe to call if the timer is not enqueued. Can be called in the callback function if the
- * timer is not enqueued at the same time (see the comments above HRTIMER_STATE_ENQUEUED).
- */
-static inline void hrtimer_update_function(struct hrtimer *timer,
- enum hrtimer_restart (*function)(struct hrtimer *))
-{
-#ifdef CONFIG_PROVE_LOCKING
- guard(raw_spinlock_irqsave)(&timer->base->cpu_base->lock);
-
- if (WARN_ON_ONCE(hrtimer_is_queued(timer)))
- return;
-
- if (WARN_ON_ONCE(!function))
- return;
-#endif
- ACCESS_PRIVATE(timer, function) = function;
-}
+void hrtimer_update_function(struct hrtimer *timer,
+ enum hrtimer_restart (*function)(struct hrtimer *));
/* Forward a hrtimer so it expires after now: */
extern u64
diff --git a/include/linux/hrtimer_defs.h b/include/linux/hrtimer_bases.h
similarity index 92%
rename from include/linux/hrtimer_defs.h
rename to include/linux/hrtimer_bases.h
index 52ed9e46ff13..d4c83ec5c0f8 100644
--- a/include/linux/hrtimer_defs.h
+++ b/include/linux/hrtimer_bases.h
@@ -1,7 +1,8 @@
/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _LINUX_HRTIMER_DEFS_H
-#define _LINUX_HRTIMER_DEFS_H
+#ifndef _LINUX_HRTIMER_BASES_H
+#define _LINUX_HRTIMER_BASES_H
+#include <linux/hrtimer.h>
#include <linux/ktime.h>
#include <linux/timerqueue.h>
#include <linux/seqlock.h>
@@ -83,7 +84,7 @@ struct hrtimer_cpu_base {
raw_spinlock_t lock;
unsigned int cpu;
unsigned int active_bases;
- unsigned int clock_was_set_seq;
+ u32 clock_was_set_seq;
bool hres_active;
bool deferred_rearm;
bool deferred_needs_update;
@@ -110,4 +111,13 @@ struct hrtimer_cpu_base {
} ____cacheline_aligned;
+/*
+ * Helper function to check, whether the timer is running the callback
+ * function
+ */
+static inline int hrtimer_callback_running(struct hrtimer *timer)
+{
+ return timer->base->running == timer;
+}
+
#endif
diff --git a/include/linux/hrtimer_rearm.h b/include/linux/hrtimer_rearm.h
index a6f2e5d5e1c7..17a81826bd9a 100644
--- a/include/linux/hrtimer_rearm.h
+++ b/include/linux/hrtimer_rearm.h
@@ -2,7 +2,12 @@
#ifndef _LINUX_HRTIMER_REARM_H
#define _LINUX_HRTIMER_REARM_H
+#include <linux/types.h>
+
#ifdef CONFIG_HRTIMER_REARM_DEFERRED
+#include <linux/irqflags.h>
+#include <linux/lockdep.h>
+#include <linux/preempt.h>
#include <linux/thread_info.h>
void __hrtimer_rearm_deferred(void);
diff --git a/include/linux/posix-timers.h b/include/linux/posix-timers.h
index 4d3dbcef379e..9a1a0c61361c 100644
--- a/include/linux/posix-timers.h
+++ b/include/linux/posix-timers.h
@@ -37,7 +37,7 @@ static inline int clockid_to_fd(const clockid_t clk)
return ~(clk >> 3);
}
-static inline bool clockid_aux_valid(clockid_t id)
+static inline bool clockid_is_aux_clock(clockid_t id)
{
return IS_ENABLED(CONFIG_POSIX_AUX_CLOCKS) && id >= CLOCK_AUX && id <= CLOCK_AUX_LAST;
}
@@ -47,7 +47,7 @@ static inline bool clockid_aux_valid(clockid_t id)
#include <linux/signal_types.h>
/**
- * cpu_timer - Posix CPU timer representation for k_itimer
+ * struct cpu_timer - Posix CPU timer representation for k_itimer
* @node: timerqueue node to queue in the task/sig
* @head: timerqueue head on which this timer is queued
* @pid: Pointer to target task PID
@@ -174,6 +174,7 @@ static inline void posix_cputimers_init_work(void) { }
* @it_sigqueue_seq: The sequence count at the point where the signal was queued
* @it_sigev_notify: The notify word of sigevent struct for signal delivery
* @it_interval: The interval for periodic timers
+ * @it_pid_type: The type of the PID
* @it_signal: Pointer to the creators signal struct
* @it_pid: The pid of the process/task targeted by the signal
* @it_process: The task to wakeup on clock_nanosleep (CPU timers)
diff --git a/include/linux/posix-timers_types.h b/include/linux/posix-timers_types.h
index a4712c1008c9..b40cf352e01d 100644
--- a/include/linux/posix-timers_types.h
+++ b/include/linux/posix-timers_types.h
@@ -34,7 +34,7 @@
#ifdef CONFIG_POSIX_TIMERS
/**
- * posix_cputimer_base - Container per posix CPU clock
+ * struct posix_cputimer_base - Container per posix CPU clock
* @nextevt: Earliest-expiration cache
* @tqhead: timerqueue head for cpu_timers
*/
@@ -44,7 +44,7 @@ struct posix_cputimer_base {
};
/**
- * posix_cputimers - Container for posix CPU timer related data
+ * struct posix_cputimers - Container for posix CPU timer related data
* @bases: Base container for posix CPU clocks
* @timers_active: Timers are queued.
* @expiry_active: Timer expiry is active. Used for
@@ -60,7 +60,7 @@ struct posix_cputimers {
};
/**
- * posix_cputimers_work - Container for task work based posix CPU timer expiry
+ * struct posix_cputimers_work - Container for task work based posix CPU timer expiry
* @work: The task work to be scheduled
* @mutex: Mutex held around expiry in context of this task work
* @scheduled: @work has been scheduled already, no further processing
diff --git a/include/linux/timekeeper_internal.h b/include/linux/timekeeper_internal.h
index 4486dfd5d0de..fe077d97b5f8 100644
--- a/include/linux/timekeeper_internal.h
+++ b/include/linux/timekeeper_internal.h
@@ -84,8 +84,6 @@ struct tk_read_base {
* @cycle_interval: Number of clock cycles in one NTP interval
* @xtime_interval: Number of clock shifted nano seconds in one NTP
* interval.
- * @xtime_remainder: Shifted nano seconds left over when rounding
- * @cycle_interval
* @raw_interval: Shifted raw nano seconds accumulated per NTP interval.
* @next_leap_ktime: CLOCK_MONOTONIC time value of a pending leap-second
* @ntp_tick: The ntp_tick_length() value currently being
@@ -99,6 +97,10 @@ struct tk_read_base {
* @ntp_error_shift: Shift conversion between clock shifted nano seconds and
* ntp shifted nano seconds.
* @ntp_err_mult: Multiplication factor for scaled math conversion
+ * @cs_tick_adj: Per-second adjustment handed to NTP via ntp_clear()
+ * accounting for the difference between the nominal
+ * NTP interval and the real time taken by the
+ * clocksource's integer @cycle_interval (upscaled).
* @skip_second_overflow: Flag used to avoid updating NTP twice with same second
* @tai_offset: The current UTC to TAI offset in seconds
*
@@ -167,7 +169,7 @@ struct timekeeper {
u32 cs_ns_to_cyc_mult;
u32 cs_ns_to_cyc_shift;
u64 cs_ns_to_cyc_maxns;
- unsigned int clock_was_set_seq;
+ u32 clock_was_set_seq;
u8 cs_was_changed_seq;
u8 clock_valid;
@@ -178,7 +180,6 @@ struct timekeeper {
u64 cycle_interval;
u64 xtime_interval;
- s64 xtime_remainder;
u64 raw_interval;
ktime_t next_leap_ktime;
@@ -186,29 +187,10 @@ struct timekeeper {
s64 ntp_error;
u32 ntp_error_shift;
u32 ntp_err_mult;
+ s64 cs_tick_adj;
u32 skip_second_overflow;
+ s64 skew_delta;
s32 tai_offset;
};
-#ifdef CONFIG_GENERIC_GETTIMEOFDAY
-
-extern void update_vsyscall(struct timekeeper *tk);
-extern void update_vsyscall_tz(void);
-
-#else
-
-static inline void update_vsyscall(struct timekeeper *tk)
-{
-}
-static inline void update_vsyscall_tz(void)
-{
-}
-#endif
-
-#if defined(CONFIG_GENERIC_GETTIMEOFDAY) && defined(CONFIG_POSIX_AUX_CLOCKS)
-extern void vdso_time_update_aux(struct timekeeper *tk);
-#else
-static inline void vdso_time_update_aux(struct timekeeper *tk) { }
-#endif
-
#endif /* _LINUX_TIMEKEEPER_INTERNAL_H */
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index 984a866d293b..efa9442aaeef 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -44,7 +44,6 @@ extern void ktime_get_ts64(struct timespec64 *ts);
extern void ktime_get_real_ts64(struct timespec64 *tv);
extern void ktime_get_coarse_ts64(struct timespec64 *ts);
extern void ktime_get_coarse_real_ts64(struct timespec64 *ts);
-extern void ktime_get_clock_ts64(clockid_t id, struct timespec64 *ts);
/* Multigrain timestamp interfaces */
extern void ktime_get_coarse_real_ts64_mg(struct timespec64 *ts);
@@ -268,15 +267,18 @@ extern void timekeeping_inject_sleeptime64(const struct timespec64 *delta);
* Auxiliary clock interfaces
*/
#ifdef CONFIG_POSIX_AUX_CLOCKS
-extern bool ktime_get_aux(clockid_t id, ktime_t *kt);
-extern bool ktime_get_aux_ts64(clockid_t id, struct timespec64 *kt);
+extern bool __must_check ktime_get_aux(clockid_t id, ktime_t *kt);
+extern bool __must_check ktime_get_aux_ts64(clockid_t id, struct timespec64 *kt);
#else
-static inline bool ktime_get_aux(clockid_t id, ktime_t *kt) { return false; }
-static inline bool ktime_get_aux_ts64(clockid_t id, struct timespec64 *kt) { return false; }
+static inline bool __must_check ktime_get_aux(clockid_t id, ktime_t *kt) { return false; }
+static inline bool __must_check ktime_get_aux_ts64(clockid_t id, struct timespec64 *kt)
+{
+ return false;
+}
#endif
/**
- * struct system_time_snapshot - Simultaneous time capture of CLOCK_MONOTONIC_RAW,
+ * struct system_time_snapshot - Simultaneous time capture of monotonic raw time,
* a selected CLOCK_* and the clocksource counter value
* @cycles: Clocksource counter value to produce the system times
* @hw_cycles: For derived clocksources, the hardware counter value from
@@ -289,6 +291,10 @@ static inline bool ktime_get_aux_ts64(clockid_t id, struct timespec64 *kt) { ret
* @clock_was_set_seq: The sequence number of clock-was-set events
* @cs_was_changed_seq: The sequence number of clocksource change events
* @valid: True if the snapshot is valid
+ *
+ * @monoraw is CLOCK_MONOTONIC_RAW for system time CLOCK ids. For CLOCK_AUX$N
+ * clock ids it's the monotonic raw time related to the AUX clock, which is
+ * CLOCK_MONOTONIC_RAW plus a AUX clock specific offset.
*/
struct system_time_snapshot {
u64 cycles;
@@ -297,7 +303,7 @@ struct system_time_snapshot {
ktime_t monoraw;
enum clocksource_ids cs_id;
enum clocksource_ids hw_csid;
- unsigned int clock_was_set_seq;
+ u32 clock_was_set_seq;
u8 cs_was_changed_seq;
u8 valid;
};
@@ -326,6 +332,10 @@ struct system_counterval_t {
* @sys_counter: Clocksource counter value simultaneous with device time
* @sys_systime: System time for @clock_id
* @sys_monoraw: Monotonic raw simultaneous with device time
+ *
+ * @sys_monoraw is CLOCK_MONOTONIC_RAW for system time CLOCK ids. For
+ * CLOCK_AUX$N clock ids it's the monotonic raw time related to the AUX clock,
+ * which is CLOCK_MONOTONIC_RAW plus a AUX clock specific offset.
*/
struct system_device_crosststamp {
clockid_t clock_id;
diff --git a/include/uapi/linux/time_types.h b/include/uapi/linux/time_types.h
index bcc0002115d3..03a0d8aaadca 100644
--- a/include/uapi/linux/time_types.h
+++ b/include/uapi/linux/time_types.h
@@ -30,7 +30,7 @@ struct __kernel_old_timeval {
struct __kernel_old_timespec {
__kernel_old_time_t tv_sec; /* seconds */
- long tv_nsec; /* nanoseconds */
+ __kernel_long_t tv_nsec; /* nanoseconds */
};
struct __kernel_old_itimerval {
diff --git a/init/Kconfig b/init/Kconfig
index 5230d4879b1c..523e41d48539 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -562,6 +562,7 @@ config VIRT_CPU_ACCOUNTING
choice
prompt "Cputime accounting"
+ default VIRT_CPU_ACCOUNTING_GEN if NO_HZ_FULL
default TICK_CPU_ACCOUNTING
# Kind of a stub config for the pure tick based cputime accounting
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index d78467ec6ee1..09197f8e4b76 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -26,6 +26,8 @@
#include <linux/jiffies.h>
#include <linux/mm_api.h>
#include <linux/highmem.h>
+#include <linux/hrtimer.h>
+#include <linux/hrtimer_bases.h>
#include <linux/spinlock_api.h>
#include <linux/cpumask_api.h>
#include <linux/lockdep_api.h>
diff --git a/kernel/time/Kconfig b/kernel/time/Kconfig
index d098ac39bde4..ddfb6bee0745 100644
--- a/kernel/time/Kconfig
+++ b/kernel/time/Kconfig
@@ -133,7 +133,6 @@ config NO_HZ_FULL
depends on HAVE_VIRT_CPU_ACCOUNTING_GEN
select NO_HZ_COMMON
select RCU_NOCB_CPU
- select VIRT_CPU_ACCOUNTING_GEN
select IRQ_WORK
select CPU_ISOLATION
help
diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c
index e48c4d379a7c..f1253f5795c6 100644
--- a/kernel/time/clocksource.c
+++ b/kernel/time/clocksource.c
@@ -123,7 +123,6 @@ static atomic_t watchdog_reset_pending;
/* Watchdog interval: 0.5sec. */
#define WATCHDOG_INTERVAL (HZ >> 1)
-#define WATCHDOG_INTERVAL_NS (WATCHDOG_INTERVAL * (NSEC_PER_SEC / HZ))
/* Maximum time between two reference watchdog readouts */
#define WATCHDOG_READOUT_MAX_NS (50U * NSEC_PER_USEC)
@@ -1566,8 +1565,12 @@ static int __init init_clocksource_sysfs(void)
{
int error = subsys_system_register(&clocksource_subsys, NULL);
- if (!error)
- error = device_register(&device_clocksource);
+ if (error)
+ return error;
+
+ error = device_register(&device_clocksource);
+ if (error)
+ bus_unregister(&clocksource_subsys);
return error;
}
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 313dcea127fe..530d61257b9a 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -26,6 +26,7 @@
#include <linux/export.h>
#include <linux/percpu.h>
#include <linux/hrtimer.h>
+#include <linux/hrtimer_bases.h>
#include <linux/notifier.h>
#include <linux/syscalls.h>
#include <linux/interrupt.h>
@@ -676,6 +677,8 @@ static ktime_t hrtimer_update_next_event(struct hrtimer_cpu_base *cpu_base)
static inline ktime_t hrtimer_update_base(struct hrtimer_cpu_base *base)
{
+ lockdep_assert_held(&base->lock);
+
ktime_t *offs_real = &base->clock_base[HRTIMER_BASE_REALTIME].offset;
ktime_t *offs_boot = &base->clock_base[HRTIMER_BASE_BOOTTIME].offset;
ktime_t *offs_tai = &base->clock_base[HRTIMER_BASE_TAI].offset;
@@ -707,7 +710,7 @@ static inline void hrtimer_rearm_event(ktime_t expires_next, bool deferred)
tick_program_event(expires_next, 1);
}
-static void __hrtimer_reprogram(struct hrtimer_cpu_base *cpu_base, struct hrtimer *next_timer,
+static void __hrtimer_reprogram(struct hrtimer_cpu_base *cpu_base,
ktime_t expires_next)
{
cpu_base->expires_next = expires_next;
@@ -743,7 +746,7 @@ static void hrtimer_force_reprogram(struct hrtimer_cpu_base *cpu_base, bool skip
if (skip_equal && expires_next == cpu_base->expires_next)
return;
- __hrtimer_reprogram(cpu_base, cpu_base->next_timer, expires_next);
+ __hrtimer_reprogram(cpu_base, expires_next);
}
/* High resolution timer related functions */
@@ -896,14 +899,14 @@ static void hrtimer_reprogram(struct hrtimer *timer, bool reprogram)
cpu_base->next_timer = timer;
- __hrtimer_reprogram(cpu_base, timer, expires);
+ __hrtimer_reprogram(cpu_base, expires);
}
static bool update_needs_ipi(struct hrtimer_cpu_base *cpu_base, unsigned int active)
{
struct hrtimer_clock_base *base;
- unsigned int seq;
ktime_t expires;
+ u32 seq;
/*
* Update the base offsets unconditionally so the following
@@ -1039,6 +1042,30 @@ static inline void unlock_hrtimer_base(const struct hrtimer *timer, unsigned lon
raw_spin_unlock_irqrestore(&timer->base->cpu_base->lock, *flags);
}
+/**
+ * hrtimer_update_function - Update the timer's callback function
+ * @timer: Timer to update
+ * @function: New callback function
+ *
+ * Only safe to call if the timer is not enqueued. Can be called in the callback function if the
+ * timer is not enqueued at the same time (see the comments above HRTIMER_STATE_ENQUEUED).
+ */
+void hrtimer_update_function(struct hrtimer *timer,
+ enum hrtimer_restart (*function)(struct hrtimer *))
+{
+#ifdef CONFIG_PROVE_LOCKING
+ guard(raw_spinlock_irqsave)(&timer->base->cpu_base->lock);
+
+ if (WARN_ON_ONCE(hrtimer_is_queued(timer)))
+ return;
+
+ if (WARN_ON_ONCE(!function))
+ return;
+#endif
+ ACCESS_PRIVATE(timer, function) = function;
+}
+EXPORT_SYMBOL_GPL(hrtimer_update_function);
+
/**
* hrtimer_forward() - forward the timer expiry
* @timer: hrtimer to forward
@@ -1786,13 +1813,21 @@ EXPORT_SYMBOL_GPL(__hrtimer_get_remaining);
ktime_t hrtimer_get_next_event(void)
{
struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
- ktime_t expires = KTIME_MAX;
- guard(raw_spinlock_irqsave)(&cpu_base->lock);
- if (!hrtimer_hres_active(cpu_base))
- expires = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_ALL);
+ /*
+ * When HRES is active cmp_next_hrtimer_event() expects KTIME_MAX.
+ *
+ * cpu_base->hres_active is written only by the local CPU in
+ * hrtimer_switch_to_hres() from hard interrupt context and in
+ * hrtimers_cpu_starting() during CPU bring-up, and all callers reach
+ * this with interrupts disabled on the same CPU, so an unlocked read is
+ * stable without holding the lock.
+ */
+ if (hrtimer_hres_active(cpu_base))
+ return KTIME_MAX;
- return expires;
+ guard(raw_spinlock_irqsave)(&cpu_base->lock);
+ return __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_ALL);
}
/**
@@ -2060,13 +2095,6 @@ static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base, struct hrtimer_cloc
base->running = NULL;
}
-static __always_inline struct hrtimer *clock_base_next_timer_safe(struct hrtimer_clock_base *base)
-{
- struct timerqueue_linked_node *next = timerqueue_linked_first(&base->active);
-
- return next ? hrtimer_from_timerqueue_node(next) : NULL;
-}
-
static void __hrtimer_run_queues(struct hrtimer_cpu_base *cpu_base, ktime_t now,
unsigned long flags, unsigned int active_mask)
{
@@ -2228,8 +2256,10 @@ void hrtimer_interrupt(struct clock_event_device *dev)
expires_next = hrtimer_update_next_event(cpu_base);
cpu_base->hang_detected = false;
if (expires_next < now) {
- if (++retries < 3)
+ if (++retries < 3) {
+ cpu_base->nr_retries++;
goto retry;
+ }
delta = ktime_sub(now, entry_time);
cpu_base->max_hang_time = max_t(unsigned int, cpu_base->max_hang_time, delta);
diff --git a/kernel/time/itimer.c b/kernel/time/itimer.c
index 7c6110e964e7..03a32dffc56c 100644
--- a/kernel/time/itimer.c
+++ b/kernel/time/itimer.c
@@ -100,7 +100,7 @@ static int do_getitimer(int which, struct itimerspec64 *value)
static int put_itimerval(struct __kernel_old_itimerval __user *o,
const struct itimerspec64 *i)
{
- struct __kernel_old_itimerval v;
+ struct __kernel_old_itimerval v = {};
v.it_interval.tv_sec = i->it_interval.tv_sec;
v.it_interval.tv_usec = i->it_interval.tv_nsec / NSEC_PER_USEC;
diff --git a/kernel/time/namespace.c b/kernel/time/namespace.c
index 5fa0af66cf3f..3aff27bb0a15 100644
--- a/kernel/time/namespace.c
+++ b/kernel/time/namespace.c
@@ -293,10 +293,12 @@ int proc_timens_set_offset(struct file *file, struct task_struct *p,
return -EINVAL;
}
- if (off->val.tv_sec > KTIME_SEC_MAX ||
- off->val.tv_sec < -KTIME_SEC_MAX)
+ if (off->val.tv_sec > KTIME_SEC_MAX || off->val.tv_sec < -KTIME_SEC_MAX)
return -ERANGE;
+ if (off->val.tv_nsec < 0 || off->val.tv_nsec >= NSEC_PER_SEC)
+ return -EINVAL;
+
tp = timespec64_add(tp, off->val);
/*
* KTIME_SEC_MAX is divided by 2 to be sure that KTIME_MAX is
diff --git a/kernel/time/ntp.c b/kernel/time/ntp.c
index 97fa99b96dd0..d22b532ec536 100644
--- a/kernel/time/ntp.c
+++ b/kernel/time/ntp.c
@@ -26,11 +26,13 @@
/**
* struct ntp_data - Structure holding all NTP related state
* @tick_usec: USER_HZ period in microseconds
- * @tick_length: Adjusted tick length
- * @tick_length_base: Base value for @tick_length
+ * @tick_length: Tick length in ns << NTP_SCALE_SHIFT
* @time_state: State of the clock synchronization
* @time_status: Clock status bits
* @time_offset: Time adjustment in nanoseconds
+ * @skew_delta: Per-tick phase slew rate for the coming second, in
+ * @time_offset units (shifted-ns / HZ). Set by
+ * second_overflow().
* @time_constant: PLL time constant
* @time_maxerror: Maximum error in microseconds holding the NTP sync distance
* (NTP dispersion + delay / 2)
@@ -38,7 +40,13 @@
* @time_freq: Frequency offset scaled nsecs/secs
* @time_reftime: Time at last adjustment in seconds
* @time_adjust: Adjustment value
+ * @time_adjust_frac: Sub-microsecond remainder of @time_adjust being
+ * delivered, in ns << NTP_SCALE_SHIFT (not divided by HZ).
* @ntp_tick_adj: Constant boot-param configurable NTP tick adjustment (upscaled)
+ * @cs_tick_adj: Fixed per-second adjustment compensating for the difference
+ * between the nominal NTP interval and the real time taken
+ * by the clocksource's integer @cycle_interval (upscaled).
+ * Set by the timekeeping core via ntp_clear().
* @ntp_next_leap_sec: Second value of the next pending leapsecond, or TIME64_MAX if no leap
*
* @pps_valid: PPS signal watchdog counter
@@ -59,17 +67,19 @@
struct ntp_data {
unsigned long tick_usec;
u64 tick_length;
- u64 tick_length_base;
int time_state;
int time_status;
s64 time_offset;
+ s64 skew_delta;
long time_constant;
long time_maxerror;
long time_esterror;
s64 time_freq;
time64_t time_reftime;
long time_adjust;
+ s64 time_adjust_frac;
s64 ntp_tick_adj;
+ s64 cs_tick_adj;
time64_t ntp_next_leap_sec;
#ifdef CONFIG_NTP_PPS
int pps_valid;
@@ -101,6 +111,9 @@ static struct ntp_data tk_ntp_data[TIMEKEEPERS_MAX] = {
#define SECS_PER_DAY 86400
#define MAX_TICKADJ 500LL /* usecs */
+/* One microsecond of phase, in plain shifted-ns (ns << NTP_SCALE_SHIFT) */
+#define ONE_US_NS ((s64)NSEC_PER_USEC << NTP_SCALE_SHIFT)
+/* Per-tick MAX_TICKADJ slew, in plain shifted-ns */
#define MAX_TICKADJ_SCALED \
(((MAX_TICKADJ * NSEC_PER_USEC) << NTP_SCALE_SHIFT) / NTP_INTERVAL_FREQ)
#define MAX_TAI_OFFSET 100000
@@ -245,8 +258,7 @@ static inline void pps_fill_timex(struct ntp_data *ntpdata, struct __kernel_time
#endif /* CONFIG_NTP_PPS */
/*
- * Update tick_length and tick_length_base, based on tick_usec, ntp_tick_adj and
- * time_freq:
+ * Update tick_length based on tick_usec, ntp_tick_adj and time_freq:
*/
static void ntp_update_frequency(struct ntp_data *ntpdata)
{
@@ -255,6 +267,7 @@ static void ntp_update_frequency(struct ntp_data *ntpdata)
second_length = (u64)(tick_usec * NSEC_PER_USEC * USER_HZ) << NTP_SCALE_SHIFT;
second_length += ntpdata->ntp_tick_adj;
+ second_length += ntpdata->cs_tick_adj;
second_length += ntpdata->time_freq;
new_base = div_u64(second_length, NTP_INTERVAL_FREQ);
@@ -263,8 +276,7 @@ static void ntp_update_frequency(struct ntp_data *ntpdata)
* Don't wait for the next second_overflow, apply the change to the
* tick length immediately:
*/
- ntpdata->tick_length += new_base - ntpdata->tick_length_base;
- ntpdata->tick_length_base = new_base;
+ ntpdata->tick_length = new_base;
}
static inline s64 ntp_update_offset_fll(struct ntp_data *ntpdata, s64 offset64, long secs)
@@ -335,14 +347,15 @@ static void __ntp_clear(struct ntp_data *ntpdata)
{
/* Stop active adjtime() */
ntpdata->time_adjust = 0;
+ ntpdata->time_adjust_frac = 0;
ntpdata->time_status |= STA_UNSYNC;
ntpdata->time_maxerror = NTP_PHASE_LIMIT;
ntpdata->time_esterror = NTP_PHASE_LIMIT;
ntp_update_frequency(ntpdata);
- ntpdata->tick_length = ntpdata->tick_length_base;
ntpdata->time_offset = 0;
+ ntpdata->skew_delta = 0;
ntpdata->ntp_next_leap_sec = TIME64_MAX;
/* Clear PPS state variables */
@@ -350,11 +363,26 @@ static void __ntp_clear(struct ntp_data *ntpdata)
}
/**
- * ntp_clear - Clears the NTP state variables
- * @tkid: Timekeeper ID to be able to select proper ntp data array member
+ * ntp_clear - Clear NTP state and set the clocksource quantisation adjustment
+ * @tkid: Timekeeper ID
+ * @cs_tick_adj: Per-second adjustment in ns << NTP_SCALE_SHIFT
+ *
+ * The timekeeping core uses an integer number of cycles (@cycle_interval)
+ * per NTP interval, so the real time that interval represents differs from
+ * the nominal NTP_INTERVAL_LENGTH by up to half a counter period. Folding
+ * this fixed offset into @cs_tick_adj makes it an explicit part of the NTP
+ * tick_length computation in ntp.c, instead of being applied during
+ * timekeeping accumulation where the NTP code never saw it. Like
+ * @ntp_tick_adj it stays internal to the kernel; userspace still sees the
+ * nominal tick via adjtimex. NTP retains its full symmetric ±MAXFREQ range
+ * around the corrected base rate.
+ *
+ * Called whenever the clocksource is (re)configured, which is also when the
+ * rest of the NTP state must be cleared, so the two are done together.
*/
-void ntp_clear(unsigned int tkid)
+void ntp_clear(unsigned int tkid, s64 cs_tick_adj)
{
+ tk_ntp_data[tkid].cs_tick_adj = cs_tick_adj;
__ntp_clear(&tk_ntp_data[tkid]);
}
@@ -364,6 +392,186 @@ u64 ntp_tick_length(unsigned int tkid)
return tk_ntp_data[tkid].tick_length;
}
+s64 ntp_get_skew_delta(unsigned int tkid)
+{
+ return tk_ntp_data[tkid].skew_delta;
+}
+
+/* Sign of @x as +1 or -1 (zero counts as positive; callers pass nonzero). */
+static inline int signof(s64 x)
+{
+ return x < 0 ? -1 : 1;
+}
+
+static s64 ntp_drain_time_offset(unsigned int tkid, s64 amount)
+{
+ struct ntp_data *ntpdata = &tk_ntp_data[tkid];
+
+ /* Only drain if amount and time_offset have the same sign */
+ if (!amount || signof(amount) != signof(ntpdata->time_offset))
+ return amount;
+
+ /* Clamp: don't overshoot zero */
+ if (abs(amount) > abs(ntpdata->time_offset)) {
+ s64 undrained = amount - ntpdata->time_offset;
+
+ ntpdata->time_offset = 0;
+ return undrained;
+ }
+
+ ntpdata->time_offset -= amount;
+ return 0;
+}
+
+/*
+ * Drain the legacy adjtime() correction (time_adjust) as it is delivered.
+ *
+ * @amount is the total intentional per-tick skew for this accumulation
+ * (skew_delta << shift), in time_offset units (shifted_ns / HZ); it covers
+ * both the exponential time_offset slew and the linear adjtime slew. This
+ * function claims only the adjtime share — capped at the MAX_TICKADJ rate —
+ * and returns the remainder for ntp_drain_time_offset().
+ *
+ * time_adjust is in whole µs. The sub-µs remainder being delivered lives in
+ * time_adjust_frac (plain shifted-ns, i.e. ns << NTP_SCALE_SHIFT -- unlike
+ * time_offset these are NOT pre-divided by HZ); we top it up by borrowing
+ * whole microseconds from time_adjust as the drain consumes it.
+ */
+static s64 ntp_drain_time_adjust(unsigned int tkid, s64 amount, unsigned int shift)
+{
+ struct ntp_data *ntpdata = &tk_ntp_data[tkid];
+ /* Sign reference: time_adjust if any whole us remain, else the drawer */
+ s64 ref = ntpdata->time_adjust ? (s64)ntpdata->time_adjust
+ : ntpdata->time_adjust_frac;
+ s64 deliver, deficit, claimed;
+
+ if (!amount || !ref || signof(amount) != signof(ref))
+ return amount;
+
+ /*
+ * Phase to deliver this accumulation, in plain shifted-ns. The drain
+ * @amount is in ÷HZ units, so multiply by HZ first, then clamp to the
+ * MAX_TICKADJ rate (MAX_TICKADJ_SCALED is the per-tick slew in
+ * shifted-ns). Multiply-then-clamp avoids an s64 divide for the cap.
+ */
+ deliver = min(abs(amount) * NTP_INTERVAL_FREQ,
+ (s64)MAX_TICKADJ_SCALED << shift);
+
+ /* Top up the sub-µs drawer from whole-µs time_adjust as needed */
+ deficit = deliver - abs(ntpdata->time_adjust_frac);
+ if (deficit > 0 && ntpdata->time_adjust) {
+ long borrow = div64_u64(deficit + ONE_US_NS - 1, ONE_US_NS);
+
+ if (ntpdata->time_adjust > 0) {
+ borrow = min(borrow, ntpdata->time_adjust);
+ ntpdata->time_adjust -= borrow;
+ ntpdata->time_adjust_frac += (s64)borrow * ONE_US_NS;
+ } else {
+ /* Clamp without negating time_adjust (UB for LONG_MIN) */
+ if (ntpdata->time_adjust > -borrow)
+ borrow = -ntpdata->time_adjust;
+ ntpdata->time_adjust += borrow;
+ ntpdata->time_adjust_frac -= (s64)borrow * ONE_US_NS;
+ }
+ }
+
+ /* Never deliver more than the drawer holds */
+ deliver = min(deliver, abs(ntpdata->time_adjust_frac));
+ if (ntpdata->time_adjust_frac > 0)
+ ntpdata->time_adjust_frac -= deliver;
+ else
+ ntpdata->time_adjust_frac += deliver;
+
+ /* Return the unclaimed remainder in ÷HZ drain units for time_offset */
+ claimed = div_s64(deliver, NTP_INTERVAL_FREQ);
+ return amount - signof(amount) * claimed;
+}
+
+/*
+ * Drain one accumulation's worth of intentional skew as it is delivered.
+ *
+ * @amount is the total intentional per-tick skew for this accumulation
+ * (skew_delta << shift), in time_offset units (shifted_ns / HZ). The
+ * adjtime() linear share is taken from time_adjust first (capped at the
+ * MAX_TICKADJ rate, hence @shift), then the exponential remainder from
+ * time_offset. Returns the amount actually claimed (same ÷HZ units).
+ */
+s64 ntp_drain_skew(unsigned int tkid, s64 amount, unsigned int shift)
+{
+ s64 unclaimed = ntp_drain_time_adjust(tkid, amount, shift);
+
+ unclaimed = ntp_drain_time_offset(tkid, unclaimed);
+
+ /*
+ * Return the amount actually drained from the intentional
+ * phase offset in time_offset and/or time_adjust.
+ */
+ return amount - unclaimed;
+}
+
+/*
+ * time_offset (drained exponentially) and time_adjust (drained linearly at the
+ * MAX_TICKADJ rate) can be asked to slew the clock in opposite directions.
+ * second_overflow() only folds their *net* into skew_delta, so the cancelling
+ * part would never be drained from either tracker via the per-tick code -- and
+ * if they cancel exactly, skew_delta is zero and neither converges at all.
+ *
+ * Settle that cancelling phase directly between the two here. No clock motion
+ * results (the opposing slews annihilate), but both move toward zero so neither
+ * stalls. @amount is the phase to take off time_offset, in its (÷HZ) units and
+ * with its sign; the same real magnitude comes off time_adjust in the opposite
+ * direction. Clamped so neither tracker is driven past zero.
+ */
+static void ntp_transfer_offset_adjust(struct ntp_data *ntpdata, s64 amount)
+{
+ s64 frac_delta, carry;
+
+ /*
+ * Don't drain time_offset past zero. @amount shares its sign and is
+ * normally bounded below it by ntp_offset_chunk(), but the ±1 skew_delta
+ * floor for a tiny time_offset can exceed it, so clamp.
+ */
+ if (abs(amount) > abs(ntpdata->time_offset))
+ amount = ntpdata->time_offset;
+ if (!amount)
+ return;
+
+ /*
+ * Remove the matching phase from time_adjust, in plain shifted-ns. No
+ * clamp against time_adjust's zero is needed: @amount is bounded by the
+ * adjtime chunk, which second_overflow() never lets exceed time_adjust's
+ * own pending phase, so this cannot overshoot.
+ */
+ frac_delta = amount * NTP_INTERVAL_FREQ;
+
+ ntpdata->time_offset -= amount;
+
+ /* Add the matching phase to time_adjust, carrying whole µs (O(1)). */
+ ntpdata->time_adjust_frac += frac_delta;
+ if (ntpdata->time_adjust_frac >= ONE_US_NS ||
+ ntpdata->time_adjust_frac <= -ONE_US_NS) {
+ carry = div64_s64(ntpdata->time_adjust_frac, ONE_US_NS);
+ ntpdata->time_adjust += carry;
+ ntpdata->time_adjust_frac -= carry * ONE_US_NS;
+ }
+
+ /*
+ * Keep time_adjust and its sub-µs remainder the same sign. The
+ * truncating carry above can leave them opposed (e.g. +4 µs paired
+ * with -250 ns), and ntp_drain_time_adjust() treats abs(time_adjust_frac)
+ * as same-direction drawer capacity -- an opposing remainder there makes
+ * it over-deliver phase that was never removed from the pile. Borrow or
+ * repay a single whole µs to realign; the total phase is unchanged.
+ */
+ if (ntpdata->time_adjust > 0 && ntpdata->time_adjust_frac < 0) {
+ ntpdata->time_adjust--;
+ ntpdata->time_adjust_frac += ONE_US_NS;
+ } else if (ntpdata->time_adjust < 0 && ntpdata->time_adjust_frac > 0) {
+ ntpdata->time_adjust++;
+ ntpdata->time_adjust_frac -= ONE_US_NS;
+ }
+}
+
/**
* ntp_get_next_leap - Returns the next leapsecond in CLOCK_REALTIME ktime_t
* @tkid: Timekeeper ID
@@ -398,7 +606,6 @@ ktime_t ntp_get_next_leap(unsigned int tkid)
int second_overflow(unsigned int tkid, time64_t secs)
{
struct ntp_data *ntpdata = &tk_ntp_data[tkid];
- s64 delta;
int leap = 0;
s32 rem;
@@ -458,35 +665,70 @@ int second_overflow(unsigned int tkid, time64_t secs)
}
/* Compute the phase adjustment for the next second */
- ntpdata->tick_length = ntpdata->tick_length_base;
-
- delta = ntp_offset_chunk(ntpdata, ntpdata->time_offset);
- ntpdata->time_offset -= delta;
- ntpdata->tick_length += delta;
/* Check PPS signal */
pps_dec_valid(ntpdata);
- if (!ntpdata->time_adjust)
- goto out;
+ /*
+ * Set the per-tick skew rate for the next second. This is in
+ * the same units as time_offset: (ns << NTP_SCALE_SHIFT) / HZ.
+ * If the result is so low that the skew imparted would round
+ * to zero, pass the bare minimum ±1 to ensure that it *does*
+ * actually drain completely to zero. It won't overshoot because
+ * logarithmic_accumulation() only drains what it can from
+ * time_offset or time_adjust, and the rest ends up in ntp_error
+ * which drives the selection of 'mult' immediately each tick.
+ */
+ if (ntpdata->time_offset || ntpdata->time_adjust ||
+ ntpdata->time_adjust_frac) {
+ s64 off_chunk = ntp_offset_chunk(ntpdata, ntpdata->time_offset);
+ s64 adj_chunk = 0, net;
- if (ntpdata->time_adjust > MAX_TICKADJ) {
- ntpdata->time_adjust -= MAX_TICKADJ;
- ntpdata->tick_length += MAX_TICKADJ_SCALED;
- goto out;
- }
+ /*
+ * Once the exponential chunk rounds to zero, deliver the last
+ * remaining offset this second so it converges to zero instead
+ * of stalling just above it.
+ */
+ if (!off_chunk)
+ off_chunk = ntpdata->time_offset;
+
+ if (ntpdata->time_adjust || ntpdata->time_adjust_frac) {
+ s64 adj;
+
+ if (ntpdata->time_adjust >= MAX_TICKADJ)
+ adj = MAX_TICKADJ * ONE_US_NS;
+ else if (ntpdata->time_adjust <= -MAX_TICKADJ)
+ adj = -MAX_TICKADJ * ONE_US_NS;
+ else
+ adj = ntpdata->time_adjust * ONE_US_NS +
+ ntpdata->time_adjust_frac;
+
+ adj_chunk = div_s64(adj, NTP_INTERVAL_FREQ);
+ if (!adj_chunk)
+ adj_chunk = signof(ntpdata->time_adjust_frac);
+ }
- if (ntpdata->time_adjust < -MAX_TICKADJ) {
- ntpdata->time_adjust += MAX_TICKADJ;
- ntpdata->tick_length -= MAX_TICKADJ_SCALED;
- goto out;
- }
+ /*
+ * If the two slews oppose, only their net would drive the
+ * per-tick drain, so the cancelling part would never drain from
+ * either tracker and an exact cancellation would stall both.
+ * Settle that overlap directly between them (no clock motion).
+ */
+ if (off_chunk && adj_chunk && signof(off_chunk) != signof(adj_chunk)) {
+ s64 conflict = min(abs(off_chunk), abs(adj_chunk));
- ntpdata->tick_length += (s64)(ntpdata->time_adjust * NSEC_PER_USEC / NTP_INTERVAL_FREQ)
- << NTP_SCALE_SHIFT;
- ntpdata->time_adjust = 0;
+ ntp_transfer_offset_adjust(ntpdata, signof(off_chunk) * conflict);
+ }
+
+ /* Net is what the clock delivers; reduce to per-tick, then floor. */
+ net = off_chunk + adj_chunk;
+ ntpdata->skew_delta = div_s64(net, NTP_INTERVAL_FREQ);
+ if (!ntpdata->skew_delta && net)
+ ntpdata->skew_delta = signof(net);
+ } else {
+ ntpdata->skew_delta = 0;
+ }
-out:
return leap;
}
@@ -779,6 +1021,7 @@ int ntp_adjtimex(unsigned int tkid, struct __kernel_timex *txc, const struct tim
if (!(txc->modes & ADJ_OFFSET_READONLY)) {
/* adjtime() is independent from ntp_adjtime() */
ntpdata->time_adjust = txc->offset;
+ ntpdata->time_adjust_frac = 0;
ntp_update_frequency(ntpdata);
audit_ntp_set_old(ad, AUDIT_NTP_ADJUST, save_adjust);
@@ -1020,6 +1263,7 @@ static void hardpps_update_phase(struct ntp_data *ntpdata, long error)
NTP_INTERVAL_FREQ);
/* Cancel running adjtime() */
ntpdata->time_adjust = 0;
+ ntpdata->time_adjust_frac = 0;
}
/* Update jitter */
ntpdata->pps_jitter += (jitter - ntpdata->pps_jitter) >> PPS_INTMIN;
diff --git a/kernel/time/ntp_internal.h b/kernel/time/ntp_internal.h
index 7084d839c207..0474a761bafc 100644
--- a/kernel/time/ntp_internal.h
+++ b/kernel/time/ntp_internal.h
@@ -3,9 +3,11 @@
#define _LINUX_NTP_INTERNAL_H
extern void ntp_init(void);
-extern void ntp_clear(unsigned int tkid);
+extern void ntp_clear(unsigned int tkid, s64 cs_tick_adj);
/* Returns how long ticks are at present, in ns / 2^NTP_SCALE_SHIFT. */
extern u64 ntp_tick_length(unsigned int tkid);
+extern s64 ntp_get_skew_delta(unsigned int tkid);
+extern s64 ntp_drain_skew(unsigned int tkid, s64 amount, unsigned int shift);
extern ktime_t ntp_get_next_leap(unsigned int tkid);
extern int second_overflow(unsigned int tkid, time64_t secs);
extern int ntp_adjtimex(unsigned int tkid, struct __kernel_timex *txc, const struct timespec64 *ts,
diff --git a/kernel/time/posix-cpu-timers.c b/kernel/time/posix-cpu-timers.c
index 5e633d8750d1..c8a9b52cc937 100644
--- a/kernel/time/posix-cpu-timers.c
+++ b/kernel/time/posix-cpu-timers.c
@@ -1300,8 +1300,11 @@ static void handle_posix_cpu_timers(struct task_struct *tsk)
unsigned long flags, start;
LIST_HEAD(firing);
- if (!lock_task_sighand(tsk, &flags))
- return;
+ /*
+ * tsk is current and ->sighand is stable, see the
+ * tsk->exit_state check in run_posix_cpu_timers()
+ */
+ spin_lock_irqsave(&tsk->sighand->siglock, flags);
do {
/*
@@ -1361,7 +1364,7 @@ static void handle_posix_cpu_timers(struct task_struct *tsk)
* that gets the timer lock before we do will give it up and
* spin until we've taken care of that timer below.
*/
- unlock_task_sighand(tsk, &flags);
+ spin_unlock_irqrestore(&tsk->sighand->siglock, flags);
/*
* Now that all the timers on our list have the firing flag,
diff --git a/kernel/time/tick-internal.h b/kernel/time/tick-internal.h
index 597d816d22e8..182974c4f21b 100644
--- a/kernel/time/tick-internal.h
+++ b/kernel/time/tick-internal.h
@@ -3,6 +3,7 @@
* tick internal variable and functions used by low/high res code
*/
#include <linux/hrtimer.h>
+#include <linux/hrtimer_bases.h>
#include <linux/tick.h>
#include "timekeeping.h"
diff --git a/kernel/time/time.c b/kernel/time/time.c
index 0dd63a91e7c5..d1a7efd80bf5 100644
--- a/kernel/time/time.c
+++ b/kernel/time/time.c
@@ -42,6 +42,7 @@
#include <generated/timeconst.h>
#include "timekeeping.h"
+#include "timekeeping_internal.h"
/*
* The timezone where the local system is located. Used as a default by some
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index b1b5ec43c0f2..ea2e6e55f37b 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -339,7 +339,6 @@ static inline void clocksource_enable_inline_read(void) { }
static void tk_setup_internals(struct timekeeper *tk, struct clocksource *clock)
{
u64 interval;
- u64 tmp, ntpinterval;
struct clocksource *old_clock;
++tk->cs_was_changed_seq;
@@ -353,20 +352,16 @@ static void tk_setup_internals(struct timekeeper *tk, struct clocksource *clock)
tk->tkr_raw.cycle_last = tk->tkr_mono.cycle_last;
/* Do the ns -> cycle conversion first, using original mult */
- tmp = NTP_INTERVAL_LENGTH;
- tmp <<= clock->shift;
- ntpinterval = tmp;
- tmp += clock->mult/2;
- do_div(tmp, clock->mult);
- if (tmp == 0)
- tmp = 1;
-
- interval = (u64) tmp;
+ interval = (u64)NTP_INTERVAL_LENGTH << clock->shift;
+ interval += clock->mult / 2;
+ do_div(interval, clock->mult);
+ if (interval == 0)
+ interval = 1;
+
tk->cycle_interval = interval;
/* Go back from cycles -> shifted ns */
tk->xtime_interval = interval * clock->mult;
- tk->xtime_remainder = ntpinterval - tk->xtime_interval;
tk->raw_interval = interval * clock->mult;
/* if changing clocks, convert xtime_nsec shift units */
@@ -386,7 +381,38 @@ static void tk_setup_internals(struct timekeeper *tk, struct clocksource *clock)
tk->ntp_error = 0;
tk->ntp_error_shift = NTP_SCALE_SHIFT - clock->shift;
- tk->ntp_tick = ntpinterval << tk->ntp_error_shift;
+
+ /*
+ * ntp_tick is the tick length that NTP disciplines (its ±500 PPM
+ * scales only this part), in NTP-shifted ns: the real interval of
+ * a whole number of counter cycles. Because cycle_interval is
+ * rounded to an integer number of cycles, this ntp_tick differs
+ * from the true intended 1/HZ tick length by up to half a cycle
+ * period.
+ */
+ tk->ntp_tick = (u64)tk->xtime_interval << tk->ntp_error_shift;
+
+ /*
+ * cs_tick_adj is the constant difference between the disciplined
+ * ntp_tick above and the true 1/HZ tick, expressed per-second to
+ * match the ntp_update_frequency() addends and handed to NTP via
+ * ntp_clear() to be explicitly included in its tick_length.
+ *
+ * Worked example: HZ=1000, ACPI PM timer at 3.579545 MHz, which
+ * has 3579.545 cycles in 1ms, rounded to cycle_interval = 3580.
+ *
+ * So ntp_tick is actually 1.000127ms, as that is the amount of
+ * time that 3580 cycles will take at the nominal frequency. This
+ * is the part that NTP disciplines, causing each 3580 counts to
+ * advance the clock by up to NTP's ±500PPM of that amount.
+ *
+ * The "extra" 127ns/tick is what's stored in cs_tick_adj and
+ * applied as a constant correction by ntp_update_frequency() so
+ * that NTP *believes* it's disciplining a 1ms tick.
+ */
+ tk->cs_tick_adj = (s64)tk->ntp_tick -
+ ((s64)NTP_INTERVAL_LENGTH << NTP_SCALE_SHIFT);
+ tk->cs_tick_adj *= NTP_INTERVAL_FREQ;
/*
* The timekeeper keeps its own mult values for the currently
@@ -397,6 +423,7 @@ static void tk_setup_internals(struct timekeeper *tk, struct clocksource *clock)
tk->tkr_raw.mult = clock->mult;
tk->ntp_err_mult = 0;
tk->skip_second_overflow = 0;
+ tk->skew_delta = 0;
tk->cs_id = clock->id;
@@ -803,7 +830,7 @@ static void timekeeping_update_from_shadow(struct tk_data *tkd, unsigned int act
if (action & TK_CLEAR_NTP) {
tk->ntp_error = 0;
- ntp_clear(tk->id);
+ ntp_clear(tk->id, tk->cs_tick_adj);
}
tk_update_leap_state(tk);
@@ -831,7 +858,11 @@ static void timekeeping_update_from_shadow(struct tk_data *tkd, unsigned int act
* the downside that the reader side does not longer benefit from
* the cacheline optimized data layout of the timekeeper and requires
* another indirection.
+ *
+ * Write xtime_sec first so that even if the memcpy() tears the store
+ * data integrity is provided for ktime_get_real_seconds().
*/
+ WRITE_ONCE(tkd->timekeeper.xtime_sec, tk->xtime_sec);
memcpy(&tkd->timekeeper, tk, sizeof(*tk));
write_seqcount_end(&tkd->seq);
}
@@ -1159,11 +1190,11 @@ time64_t ktime_get_real_seconds(void)
unsigned int seq;
if (IS_ENABLED(CONFIG_64BIT))
- return tk->xtime_sec;
+ return READ_ONCE(tk->xtime_sec);
do {
seq = read_seqcount_begin(&tk_core.seq);
- seconds = tk->xtime_sec;
+ seconds = READ_ONCE(tk->xtime_sec);
} while (read_seqcount_retry(&tk_core.seq, seq));
@@ -1185,7 +1216,7 @@ noinstr time64_t __ktime_get_real_seconds(void)
{
struct timekeeper *tk = &tk_core.timekeeper;
- return tk->xtime_sec;
+ return READ_ONCE(tk->xtime_sec);
}
static inline u64 tk_clock_read_snapshot(const struct tk_read_base *tkr,
@@ -1202,10 +1233,21 @@ static inline u64 tk_clock_read_snapshot(const struct tk_read_base *tkr,
/**
* ktime_get_snapshot_id - Simultaneously snapshot a given clock ID with
- * CLOCK_MONOTONIC_RAW and the underlying
+ * the corresponding monotonic raw and the underlying
* clocksource counter value.
* @clock_id: The clock ID to snapshot
* @systime_snapshot: Pointer to struct receiving the system time snapshot
+ *
+ * For the system time keeping clocks (REALTIME, MONOTONIC and BOOTTIME) the
+ * monotonic raw clock is CLOCK_MONOTONIC_RAW. For AUX clocks this is the
+ * monotonic raw clock related to the AUX clock. These AUX clock related
+ * monotonic raw clocks have a strict linear offset to the system time
+ * CLOCK_MONOTONIC_RAW:
+ *
+ * MONOTONIC_RAW(AUX$N) = CLOCK_MONOTONIC_RAW(system) + offset(AUX$N)
+ *
+ * The offset is established when a AUX clock is initialized, but it is
+ * currently not accessible.
*/
void ktime_get_snapshot_id(clockid_t clock_id, struct system_time_snapshot *systime_snapshot)
{
@@ -1512,6 +1554,9 @@ EXPORT_SYMBOL_GPL(ktime_real_to_base_clock);
* @xtstamp: Receives simultaneously captured system and device time
*
* Reads a timestamp from a device and correlates it to system time
+ *
+ * See documentation for ktime_get_snapshot_id() for information about the raw
+ * monotonic time stamp which is used here.
*/
int get_device_system_crosststamp(int (*get_time_fn)
(ktime_t *device_time,
@@ -1522,10 +1567,11 @@ int get_device_system_crosststamp(int (*get_time_fn)
struct system_device_crosststamp *xtstamp)
{
u64 syscnt_cycles, cycles, now, interval_start;
- unsigned int seq, clock_was_set_seq = 0;
ktime_t base_sys, base_raw, *offs;
+ u32 clock_was_set_seq = 0;
u64 nsec_sys, nsec_raw;
u8 cs_was_changed_seq;
+ unsigned int seq;
bool do_interp;
struct timekeeper *tk;
struct tk_data *tkd;
@@ -1896,40 +1942,6 @@ void ktime_get_raw_ts64(struct timespec64 *ts)
}
EXPORT_SYMBOL(ktime_get_raw_ts64);
-/**
- * ktime_get_clock_ts64 - Returns time of a clock in a timespec
- * @id: POSIX clock ID of the clock to read
- * @ts: Pointer to the timespec64 to be set
- *
- * The timestamp is invalidated (@ts->sec is set to -1) if the
- * clock @id is not available.
- */
-void ktime_get_clock_ts64(clockid_t id, struct timespec64 *ts)
-{
- /* Invalidate time stamp */
- ts->tv_sec = -1;
- ts->tv_nsec = 0;
-
- switch (id) {
- case CLOCK_REALTIME:
- ktime_get_real_ts64(ts);
- return;
- case CLOCK_MONOTONIC:
- ktime_get_ts64(ts);
- return;
- case CLOCK_MONOTONIC_RAW:
- ktime_get_raw_ts64(ts);
- return;
- case CLOCK_AUX ... CLOCK_AUX_LAST:
- if (IS_ENABLED(CONFIG_POSIX_AUX_CLOCKS))
- ktime_get_aux_ts64(id, ts);
- return;
- default:
- WARN_ON_ONCE(1);
- }
-}
-EXPORT_SYMBOL_GPL(ktime_get_clock_ts64);
-
/**
* timekeeping_valid_for_hres - Check if timekeeping is suitable for hres
*/
@@ -2076,7 +2088,12 @@ void __init timekeeping_init(void)
tk_set_wall_to_mono(tks, wall_to_mono);
- timekeeping_update_from_shadow(&tk_core, TK_CLOCK_WAS_SET);
+ /*
+ * Use TK_UPDATE_ALL so the NTP layer picks up the clocksource's
+ * cs_tick_adj via ntp_clear(). Clearing NTP here is otherwise
+ * redundant as ntp_init() already initialised it above.
+ */
+ timekeeping_update_from_shadow(&tk_core, TK_UPDATE_ALL);
}
/* time in seconds when suspend began for persistent clock */
@@ -2390,6 +2407,11 @@ static __always_inline void timekeeping_apply_adjustment(struct timekeeper *tk,
* xtime_nsec_2 = xtime_nsec_1 - offset
* Which simplifies to:
* xtime_nsec -= offset
+ *
+ * When subtracting offset from xtime_nsec, the same amount
+ * (in appropriate units) has to be added to ntp_error, in
+ * order to correctly track the delta between the time
+ * reported in xtime_nsec, and the intended time.
*/
if ((mult_adj > 0) && (tk->tkr_mono.mult + mult_adj < mult_adj)) {
/* NTP adjustment caused clocksource mult overflow */
@@ -2400,6 +2422,7 @@ static __always_inline void timekeeping_apply_adjustment(struct timekeeper *tk,
tk->tkr_mono.mult += mult_adj;
tk->xtime_interval += interval;
tk->tkr_mono.xtime_nsec -= offset;
+ tk->ntp_error += offset << tk->ntp_error_shift;
}
/*
@@ -2409,18 +2432,27 @@ static __always_inline void timekeeping_apply_adjustment(struct timekeeper *tk,
static void timekeeping_adjust(struct timekeeper *tk, s64 offset)
{
u64 ntp_tl = ntp_tick_length(tk->id);
+ s64 skew = ntp_get_skew_delta(tk->id);
u32 mult;
/*
- * Determine the multiplier from the current NTP tick length.
- * Avoid expensive division when the tick length doesn't change.
+ * Determine the multiplier from the current NTP tick length plus
+ * skew_delta. The skew biases mult so that ±1 dithering can deliver
+ * the time_offset slew rate. Recompute when either changes.
*/
- if (likely(tk->ntp_tick == ntp_tl)) {
+ if (likely(tk->ntp_tick == ntp_tl && tk->skew_delta == skew)) {
+ /* Revert to the base mult rate. */
mult = tk->tkr_mono.mult - tk->ntp_err_mult;
} else {
tk->ntp_tick = ntp_tl;
- mult = div64_u64((tk->ntp_tick >> tk->ntp_error_shift) -
- tk->xtime_remainder, tk->cycle_interval);
+ tk->skew_delta = skew;
+ /*
+ * skew_delta is stored pre-divided by HZ (matching time_offset);
+ * scale it back up to the full per-tick rate for the mult bias.
+ */
+ skew *= NTP_INTERVAL_FREQ;
+ mult = div64_u64((tk->ntp_tick + skew) >> tk->ntp_error_shift,
+ tk->cycle_interval);
}
/*
@@ -2545,8 +2577,25 @@ static u64 logarithmic_accumulation(struct timekeeper *tk, u64 offset,
/* Accumulate error between NTP and clock interval */
tk->ntp_error += tk->ntp_tick << shift;
- tk->ntp_error -= (tk->xtime_interval + tk->xtime_remainder) <<
- (tk->ntp_error_shift + shift);
+ tk->ntp_error -= tk->xtime_interval << (tk->ntp_error_shift + shift);
+
+ /*
+ * When skewing, do so by adjusting ntp_error to impart an extra
+ * target delta into ntp_error per tick, limited to what can be
+ * drained from time_offset / time_adjust to avoid overshoot.
+ *
+ * The base 'mult' value was calculated with the skew taken into
+ * account, such that the per-tick choice of 'mult' vs. 'mult+1'
+ * allows for the desired effective rate and ntp_error does not
+ * grow unbounded.
+ *
+ * Once the full desired phase offset is delivered, any remaining
+ * skew imparted by the adjusted 'mult', accounted above, remains
+ * in ntp_error and will be compensated by the dithering over time.
+ */
+ if (tk->skew_delta)
+ tk->ntp_error += ntp_drain_skew(tk->id, tk->skew_delta << shift,
+ shift) * NTP_INTERVAL_FREQ;
return offset;
}
@@ -2795,7 +2844,7 @@ void do_timer(unsigned long ticks)
*
* Called from hrtimer_interrupt() or retrigger_next_event()
*/
-ktime_t ktime_get_update_offsets_now(unsigned int *cwsseq, ktime_t *offs_real,
+ktime_t ktime_get_update_offsets_now(u32 *cwsseq, ktime_t *offs_real,
ktime_t *offs_boot, ktime_t *offs_tai)
{
struct timekeeper *tk = &tk_core.timekeeper;
@@ -2943,10 +2992,12 @@ static int __do_adjtimex(struct tk_data *tkd, struct __kernel_timex *txc,
return ret;
add_device_randomness(txc, sizeof(*txc));
- if (!aux_clock)
+ if (!aux_clock) {
ktime_get_real_ts64(&ts);
- else
- tk_get_aux_ts64(tkd->timekeeper.id, &ts);
+ } else {
+ if (!tk_get_aux_ts64(tkd->timekeeper.id, &ts))
+ return -ENODEV;
+ }
add_device_randomness(&ts, sizeof(ts));
@@ -3051,7 +3102,7 @@ static inline unsigned int clockid_to_tkid(unsigned int id)
static inline struct tk_data *aux_get_tk_data(clockid_t id)
{
- if (!clockid_aux_valid(id))
+ if (!clockid_is_aux_clock(id))
return NULL;
return &timekeeper_data[clockid_to_tkid(id)];
}
@@ -3146,7 +3197,7 @@ EXPORT_SYMBOL_GPL(ktime_get_aux_ts64);
static int aux_get_res(clockid_t id, struct timespec64 *tp)
{
- if (!clockid_aux_valid(id))
+ if (!clockid_is_aux_clock(id))
return -ENODEV;
tp->tv_sec = aux_clock_resolution_ns() / NSEC_PER_SEC;
@@ -3313,7 +3364,9 @@ static const struct attribute_group aux_clock_enable_attr_group = {
static int __init tk_aux_sysfs_init(void)
{
struct kobject *auxo, *tko = kobject_create_and_add("time", kernel_kobj);
+ struct kobject *clks[MAX_AUX_CLOCKS];
int ret = -ENOMEM;
+ int i;
if (!tko)
return ret;
@@ -3322,21 +3375,28 @@ static int __init tk_aux_sysfs_init(void)
if (!auxo)
goto err_clean;
- for (int i = 0; i < MAX_AUX_CLOCKS; i++) {
+ for (i = 0; i < MAX_AUX_CLOCKS; i++) {
char id[2] = { [0] = '0' + i, };
- struct kobject *clk = kobject_create_and_add(id, auxo);
+ clks[i] = kobject_create_and_add(id, auxo);
- if (!clk) {
+ if (!clks[i]) {
ret = -ENOMEM;
- goto err_clean;
+ goto err_clks;
}
- ret = sysfs_create_group(clk, &aux_clock_enable_attr_group);
+ ret = sysfs_create_group(clks[i], &aux_clock_enable_attr_group);
if (ret)
- goto err_clean;
+ goto err_clk;
}
return 0;
+err_clk:
+ kobject_put(clks[i]);
+err_clks:
+ while (--i >= 0) {
+ sysfs_remove_group(clks[i], &aux_clock_enable_attr_group);
+ kobject_put(clks[i]);
+ }
err_clean:
kobject_put(auxo);
kobject_put(tko);
diff --git a/kernel/time/timekeeping.h b/kernel/time/timekeeping.h
index 198d0608db74..4201f9e90813 100644
--- a/kernel/time/timekeeping.h
+++ b/kernel/time/timekeeping.h
@@ -4,7 +4,7 @@
/*
* Internal interfaces for kernel/time/
*/
-extern ktime_t ktime_get_update_offsets_now(unsigned int *cwsseq,
+extern ktime_t ktime_get_update_offsets_now(u32 *cwsseq,
ktime_t *offs_real,
ktime_t *offs_boot,
ktime_t *offs_tai);
diff --git a/kernel/time/timekeeping_internal.h b/kernel/time/timekeeping_internal.h
index 973ede670a36..6d719b8e5ea2 100644
--- a/kernel/time/timekeeping_internal.h
+++ b/kernel/time/timekeeping_internal.h
@@ -6,6 +6,8 @@
#include <linux/spinlock.h>
#include <linux/time.h>
+struct timekeeper;
+
/*
* timekeeping debug functions
*/
@@ -48,4 +50,23 @@ void timekeeper_unlock_irqrestore(unsigned long flags);
/* NTP specific interface to access the current seconds value */
long ktime_get_ntp_seconds(unsigned int id);
+#ifdef CONFIG_GENERIC_GETTIMEOFDAY
+
+extern void update_vsyscall(struct timekeeper *tk);
+extern void update_vsyscall_tz(void);
+extern void vdso_time_update_aux(struct timekeeper *tk);
+
+#else
+
+static inline void update_vsyscall(struct timekeeper *tk)
+{
+}
+static inline void update_vsyscall_tz(void)
+{
+}
+static inline void vdso_time_update_aux(struct timekeeper *tk)
+{
+}
+#endif
+
#endif /* _TIMEKEEPING_INTERNAL_H */
diff --git a/kernel/time/timer_list.c b/kernel/time/timer_list.c
index 514802def1e0..0406bf4488e9 100644
--- a/kernel/time/timer_list.c
+++ b/kernel/time/timer_list.c
@@ -20,7 +20,7 @@
struct timer_list_iter {
int cpu;
bool second_pass;
- u64 now;
+ ktime_t now;
};
/*
@@ -44,19 +44,19 @@ static void SEQ_printf(struct seq_file *m, const char *fmt, ...)
static void
print_timer(struct seq_file *m, struct hrtimer *taddr, struct hrtimer *timer,
- int idx, u64 now)
+ int idx, ktime_t now)
{
SEQ_printf(m, " #%d: <%p>, %ps", idx, taddr, ACCESS_PRIVATE(timer, function));
SEQ_printf(m, ", S:%02x", timer->is_queued);
SEQ_printf(m, "\n");
- SEQ_printf(m, " # expires at %Lu-%Lu nsecs [in %Ld to %Ld nsecs]\n",
- (unsigned long long)ktime_to_ns(hrtimer_get_softexpires(timer)),
- (unsigned long long)ktime_to_ns(hrtimer_get_expires(timer)),
- (long long)(ktime_to_ns(hrtimer_get_softexpires(timer)) - now),
- (long long)(ktime_to_ns(hrtimer_get_expires(timer)) - now));
+ SEQ_printf(m, " # expires at %lld-%lld nsecs [in %lld to %lld nsecs]\n",
+ (long long)hrtimer_get_softexpires(timer),
+ (long long)hrtimer_get_expires(timer),
+ (long long)ktime_sub(hrtimer_get_softexpires(timer), now),
+ (long long)ktime_sub(hrtimer_get_expires(timer), now));
}
-static void print_active_timers(struct seq_file *m, struct hrtimer_clock_base *base, u64 now)
+static void print_active_timers(struct seq_file *m, struct hrtimer_clock_base *base, ktime_t now)
{
struct timerqueue_linked_node *curr;
struct hrtimer *timer, tmp;
@@ -94,21 +94,21 @@ static void print_active_timers(struct seq_file *m, struct hrtimer_clock_base *b
}
static void
-print_base(struct seq_file *m, struct hrtimer_clock_base *base, u64 now)
+print_base(struct seq_file *m, struct hrtimer_clock_base *base, ktime_t now)
{
SEQ_printf(m, " .base: %p\n", base);
SEQ_printf(m, " .index: %d\n", base->index);
SEQ_printf(m, " .resolution: %u nsecs\n", hrtimer_resolution);
#ifdef CONFIG_HIGH_RES_TIMERS
- SEQ_printf(m, " .offset: %Ld nsecs\n",
+ SEQ_printf(m, " .offset: %lld nsecs\n",
(long long) base->offset);
#endif
SEQ_printf(m, "active timers:\n");
- print_active_timers(m, base, now + ktime_to_ns(base->offset));
+ print_active_timers(m, base, ktime_add(now, base->offset));
}
-static void print_cpu(struct seq_file *m, int cpu, u64 now)
+static void print_cpu(struct seq_file *m, int cpu, ktime_t now)
{
struct hrtimer_cpu_base *cpu_base = &per_cpu(hrtimer_bases, cpu);
int i;
@@ -118,15 +118,17 @@ static void print_cpu(struct seq_file *m, int cpu, u64 now)
SEQ_printf(m, " clock %d:\n", i);
print_base(m, cpu_base->clock_base + i, now);
}
-#define P(x) \
- SEQ_printf(m, " .%-15s: %Lu\n", #x, \
- (unsigned long long)(cpu_base->x))
-#define P_ns(x) \
- SEQ_printf(m, " .%-15s: %Lu nsecs\n", #x, \
- (unsigned long long)(ktime_to_ns(cpu_base->x)))
+
+#define DIAG_READ(x) data_race(READ_ONCE(x))
+
+#define P(x) \
+ SEQ_printf(m, " .%-15s: %llu\n", #x, \
+ (unsigned long long)DIAG_READ(cpu_base->x))
+#define P_ktime(x) \
+ SEQ_printf(m, " .%-15s: %lld nsecs\n", #x, (long long)DIAG_READ(cpu_base->x))
#ifdef CONFIG_HIGH_RES_TIMERS
- P_ns(expires_next);
+ P_ktime(expires_next);
P(hres_active);
P(nr_events);
P(nr_retries);
@@ -134,38 +136,39 @@ static void print_cpu(struct seq_file *m, int cpu, u64 now)
P(max_hang_time);
#endif
#undef P
-#undef P_ns
+#undef P_ktime
#ifdef CONFIG_TICK_ONESHOT
# define P(x) \
- SEQ_printf(m, " .%-15s: %Lu\n", #x, \
- (unsigned long long)(ts->x))
-# define P_ns(x) \
- SEQ_printf(m, " .%-15s: %Lu nsecs\n", #x, \
- (unsigned long long)(ktime_to_ns(ts->x)))
+ SEQ_printf(m, " .%-15s: %llu\n", #x, \
+ (unsigned long long)DIAG_READ(ts->x))
+# define P_ktime(x) \
+ SEQ_printf(m, " .%-15s: %lld nsecs\n", #x, (long long)DIAG_READ(ts->x))
# define P_flag(x, f) \
- SEQ_printf(m, " .%-15s: %d\n", #x, !!(ts->flags & (f)))
+ SEQ_printf(m, " .%-15s: %d\n", #x, !!(DIAG_READ(ts->flags) & (f)))
{
struct tick_sched *ts = tick_get_tick_sched(cpu);
P_flag(nohz, TS_FLAG_NOHZ);
P_flag(highres, TS_FLAG_HIGHRES);
- P_ns(last_tick);
+ P_ktime(last_tick);
P_flag(tick_stopped, TS_FLAG_STOPPED);
P(idle_calls);
P(idle_sleeps);
- P_ns(idle_entrytime);
- P_ns(idle_waketime);
+ P_ktime(idle_entrytime);
+ P_ktime(idle_waketime);
P(last_jiffies);
P(next_timer);
- P_ns(idle_expires);
- SEQ_printf(m, "jiffies: %Lu\n",
+ P_ktime(idle_expires);
+ SEQ_printf(m, "jiffies: %llu\n",
(unsigned long long)jiffies);
}
#endif
#undef P
-#undef P_ns
+#undef P_ktime
+#undef P_flag
+#undef DIAG_READ
SEQ_printf(m, "\n");
}
@@ -196,8 +199,7 @@ print_tickdevice(struct seq_file *m, struct tick_device *td, int cpu)
SEQ_printf(m, " mult: %u\n", dev->mult);
SEQ_printf(m, " shift: %u\n", dev->shift);
SEQ_printf(m, " mode: %d\n", clockevent_get_state(dev));
- SEQ_printf(m, " next_event: %Ld nsecs\n",
- (unsigned long long) ktime_to_ns(dev->next_event));
+ SEQ_printf(m, " next_event: %lld nsecs\n", (long long)dev->next_event);
SEQ_printf(m, " set_next_event: %ps\n", dev->set_next_event);
@@ -250,17 +252,17 @@ static void timer_list_show_tickdevices_header(struct seq_file *m)
}
#endif
-static inline void timer_list_header(struct seq_file *m, u64 now)
+static inline void timer_list_header(struct seq_file *m, ktime_t now)
{
SEQ_printf(m, "Timer List Version: v0.11\n");
SEQ_printf(m, "HRTIMER_MAX_CLOCK_BASES: %d\n", HRTIMER_MAX_CLOCK_BASES);
- SEQ_printf(m, "now at %Ld nsecs\n", (unsigned long long)now);
+ SEQ_printf(m, "now at %lld nsecs\n", (long long)now);
SEQ_printf(m, "\n");
}
void sysrq_timer_list_show(void)
{
- u64 now = ktime_to_ns(ktime_get());
+ ktime_t now = ktime_get();
int cpu;
timer_list_header(NULL, now);
@@ -318,7 +320,7 @@ static void *timer_list_start(struct seq_file *file, loff_t *offset)
struct timer_list_iter *iter = file->private;
if (!*offset)
- iter->now = ktime_to_ns(ktime_get());
+ iter->now = ktime_get();
iter->cpu = -1;
iter->second_pass = false;
return move_iter(iter, *offset);
diff --git a/kernel/time/timer_migration.c b/kernel/time/timer_migration.c
index 806c23cf71fc..059d43355e65 100644
--- a/kernel/time/timer_migration.c
+++ b/kernel/time/timer_migration.c
@@ -1847,8 +1847,10 @@ static int tmigr_setup_groups(struct tmigr_hierarchy *hier, unsigned int cpu,
}
/* Assert single root without parent */
- if (WARN_ON_ONCE(i >= tmigr_hierarchy_levels))
+ if (WARN_ON_ONCE(i >= tmigr_hierarchy_levels)) {
+ kfree(stack);
return -EINVAL;
+ }
for (; i >= start_lvl; i--) {
group = stack[i];
diff --git a/kernel/time/timer_migration.h b/kernel/time/timer_migration.h
index 31735dd52327..c9c1c29f011d 100644
--- a/kernel/time/timer_migration.h
+++ b/kernel/time/timer_migration.h
@@ -103,7 +103,7 @@ struct tmigr_group {
* before the timer migration hierarchy hotplug callback is
* reached. During this phase, the CPU has to handle the
* global timers on its own and must not act as a migrator.
-
+ *
* @idle: Indicates whether the CPU is idle in the timer migration
* hierarchy
* @remote: Is set when timers of the CPU are expired remotely
diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
index 7283f0f18813..ce7ab986dee6 100644
--- a/sound/drivers/dummy.c
+++ b/sound/drivers/dummy.c
@@ -13,6 +13,7 @@
#include <linux/time.h>
#include <linux/wait.h>
#include <linux/hrtimer.h>
+#include <linux/hrtimer_bases.h>
#include <linux/math64.h>
#include <linux/module.h>
#include <sound/core.h>
diff --git a/tools/testing/selftests/clock-helpers.h b/tools/testing/selftests/clock-helpers.h
new file mode 100644
index 000000000000..01451f538e71
--- /dev/null
+++ b/tools/testing/selftests/clock-helpers.h
@@ -0,0 +1,76 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __CLOCK_HELPERS_H
+#define __CLOCK_HELPERS_H
+
+#include <sys/types.h>
+#include <time.h>
+
+#define MSEC_PER_SEC 1000LL
+#define USEC_PER_MSEC 1000LL
+#define NSEC_PER_USEC 1000LL
+#define NSEC_PER_MSEC 1000000LL
+#define USEC_PER_SEC 1000000LL
+#define NSEC_PER_SEC 1000000000LL
+#define PSEC_PER_SEC 1000000000000LL
+#define FSEC_PER_SEC 1000000000000000LL
+
+#ifndef CLOCK_AUX
+#define CLOCK_AUX 16
+#endif
+
+#ifndef MAX_AUX_CLOCKS
+#define MAX_AUX_CLOCKS 8
+#endif
+
+#ifndef CLOCK_AUX_LAST
+#define CLOCK_AUX_LAST (CLOCK_AUX + MAX_AUX_CLOCKS - 1)
+#endif
+
+__attribute__((unused))
+static inline const char *clock_name(clockid_t clockid)
+{
+ switch (clockid) {
+ case CLOCK_REALTIME:
+ return "CLOCK_REALTIME";
+ case CLOCK_MONOTONIC:
+ return "CLOCK_MONOTONIC";
+ case CLOCK_PROCESS_CPUTIME_ID:
+ return "CLOCK_PROCESS_CPUTIME_ID";
+ case CLOCK_THREAD_CPUTIME_ID:
+ return "CLOCK_THREAD_CPUTIME_ID";
+ case CLOCK_MONOTONIC_RAW:
+ return "CLOCK_MONOTONIC_RAW";
+ case CLOCK_REALTIME_COARSE:
+ return "CLOCK_REALTIME_COARSE";
+ case CLOCK_MONOTONIC_COARSE:
+ return "CLOCK_MONOTONIC_COARSE";
+ case CLOCK_BOOTTIME:
+ return "CLOCK_BOOTTIME";
+ case CLOCK_REALTIME_ALARM:
+ return "CLOCK_REALTIME_ALARM";
+ case CLOCK_BOOTTIME_ALARM:
+ return "CLOCK_BOOTTIME_ALARM";
+ case CLOCK_TAI:
+ return "CLOCK_TAI";
+ case CLOCK_AUX + 0:
+ return "CLOCK_AUX0";
+ case CLOCK_AUX + 1:
+ return "CLOCK_AUX1";
+ case CLOCK_AUX + 2:
+ return "CLOCK_AUX2";
+ case CLOCK_AUX + 3:
+ return "CLOCK_AUX3";
+ case CLOCK_AUX + 4:
+ return "CLOCK_AUX4";
+ case CLOCK_AUX + 5:
+ return "CLOCK_AUX5";
+ case CLOCK_AUX + 6:
+ return "CLOCK_AUX6";
+ case CLOCK_AUX + 7:
+ return "CLOCK_AUX7";
+ };
+ return "UNKNOWN_CLOCKID";
+}
+
+#endif /* __CLOCK_HELPERS_H */
diff --git a/tools/testing/selftests/timers/Makefile b/tools/testing/selftests/timers/Makefile
index 32203593c62e..0e73a16874c4 100644
--- a/tools/testing/selftests/timers/Makefile
+++ b/tools/testing/selftests/timers/Makefile
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
-CFLAGS += -O3 -Wl,-no-as-needed -Wall -I $(top_srcdir)
+CFLAGS += -O3 -Wl,-no-as-needed -Wall
LDLIBS += -lrt -lpthread -lm
# these are all "safe" tests that don't modify
diff --git a/tools/testing/selftests/timers/adjtick.c b/tools/testing/selftests/timers/adjtick.c
index 5b3ef708d6e9..68009a6d6de8 100644
--- a/tools/testing/selftests/timers/adjtick.c
+++ b/tools/testing/selftests/timers/adjtick.c
@@ -22,8 +22,8 @@
#include <sys/time.h>
#include <sys/timex.h>
#include <time.h>
-#include <include/vdso/time64.h>
+#include "clock-helpers.h"
#include "kselftest.h"
#define MILLION 1000000
diff --git a/tools/testing/selftests/timers/alarmtimer-suspend.c b/tools/testing/selftests/timers/alarmtimer-suspend.c
index aa66c805f6a4..120b3ce8b39e 100644
--- a/tools/testing/selftests/timers/alarmtimer-suspend.c
+++ b/tools/testing/selftests/timers/alarmtimer-suspend.c
@@ -28,8 +28,8 @@
#include <signal.h>
#include <stdlib.h>
#include <pthread.h>
-#include <include/vdso/time64.h>
#include <errno.h>
+#include "clock-helpers.h"
#include "kselftest.h"
#define UNREASONABLE_LAT (NSEC_PER_SEC * 5) /* hopefully we resume in 5 secs */
@@ -39,37 +39,6 @@ int alarmcount;
int alarm_clock_id;
struct timespec start_time;
-
-char *clockstring(int clockid)
-{
- switch (clockid) {
- case CLOCK_REALTIME:
- return "CLOCK_REALTIME";
- case CLOCK_MONOTONIC:
- return "CLOCK_MONOTONIC";
- case CLOCK_PROCESS_CPUTIME_ID:
- return "CLOCK_PROCESS_CPUTIME_ID";
- case CLOCK_THREAD_CPUTIME_ID:
- return "CLOCK_THREAD_CPUTIME_ID";
- case CLOCK_MONOTONIC_RAW:
- return "CLOCK_MONOTONIC_RAW";
- case CLOCK_REALTIME_COARSE:
- return "CLOCK_REALTIME_COARSE";
- case CLOCK_MONOTONIC_COARSE:
- return "CLOCK_MONOTONIC_COARSE";
- case CLOCK_BOOTTIME:
- return "CLOCK_BOOTTIME";
- case CLOCK_REALTIME_ALARM:
- return "CLOCK_REALTIME_ALARM";
- case CLOCK_BOOTTIME_ALARM:
- return "CLOCK_BOOTTIME_ALARM";
- case CLOCK_TAI:
- return "CLOCK_TAI";
- }
- return "UNKNOWN_CLOCKID";
-}
-
-
long long timespec_sub(struct timespec a, struct timespec b)
{
long long ret = NSEC_PER_SEC * b.tv_sec + b.tv_nsec;
@@ -129,12 +98,12 @@ int main(void)
alarmcount = 0;
if (timer_create(alarm_clock_id, &se, &tm1) == -1) {
printf("timer_create failed, %s unsupported?: %s\n",
- clockstring(alarm_clock_id), strerror(errno));
+ clock_name(alarm_clock_id), strerror(errno));
break;
}
clock_gettime(alarm_clock_id, &start_time);
- printf("Start time (%s): %ld:%ld\n", clockstring(alarm_clock_id),
+ printf("Start time (%s): %ld:%ld\n", clock_name(alarm_clock_id),
start_time.tv_sec, start_time.tv_nsec);
printf("Setting alarm for every %i seconds\n", SUSPEND_SECS);
its1.it_value = start_time;
diff --git a/tools/testing/selftests/timers/inconsistency-check.c b/tools/testing/selftests/timers/inconsistency-check.c
index e53e63e18683..d7982ac4bd18 100644
--- a/tools/testing/selftests/timers/inconsistency-check.c
+++ b/tools/testing/selftests/timers/inconsistency-check.c
@@ -28,7 +28,7 @@
#include <sys/timex.h>
#include <string.h>
#include <signal.h>
-#include <include/vdso/time64.h>
+#include "clock-helpers.h"
#include "kselftest.h"
/* CLOCK_HWSPECIFIC == CLOCK_SGI_CYCLE (Deprecated) */
@@ -36,35 +36,6 @@
#define CALLS_PER_LOOP 64
-char *clockstring(int clockid)
-{
- switch (clockid) {
- case CLOCK_REALTIME:
- return "CLOCK_REALTIME";
- case CLOCK_MONOTONIC:
- return "CLOCK_MONOTONIC";
- case CLOCK_PROCESS_CPUTIME_ID:
- return "CLOCK_PROCESS_CPUTIME_ID";
- case CLOCK_THREAD_CPUTIME_ID:
- return "CLOCK_THREAD_CPUTIME_ID";
- case CLOCK_MONOTONIC_RAW:
- return "CLOCK_MONOTONIC_RAW";
- case CLOCK_REALTIME_COARSE:
- return "CLOCK_REALTIME_COARSE";
- case CLOCK_MONOTONIC_COARSE:
- return "CLOCK_MONOTONIC_COARSE";
- case CLOCK_BOOTTIME:
- return "CLOCK_BOOTTIME";
- case CLOCK_REALTIME_ALARM:
- return "CLOCK_REALTIME_ALARM";
- case CLOCK_BOOTTIME_ALARM:
- return "CLOCK_BOOTTIME_ALARM";
- case CLOCK_TAI:
- return "CLOCK_TAI";
- }
- return "UNKNOWN_CLOCKID";
-}
-
/* returns 1 if a <= b, 0 otherwise */
static inline int in_order(struct timespec a, struct timespec b)
{
@@ -171,15 +142,15 @@ int main(int argc, char *argv[])
for (clockid = userclock; clockid < maxclocks; clockid++) {
if (clockid == CLOCK_HWSPECIFIC || clock_gettime(clockid, &ts)) {
- ksft_test_result_skip("%-31s\n", clockstring(clockid));
+ ksft_test_result_skip("%-31s\n", clock_name(clockid));
continue;
}
if (consistency_test(clockid, runtime)) {
- ksft_test_result_fail("%-31s\n", clockstring(clockid));
+ ksft_test_result_fail("%-31s\n", clock_name(clockid));
ksft_exit_fail();
} else {
- ksft_test_result_pass("%-31s\n", clockstring(clockid));
+ ksft_test_result_pass("%-31s\n", clock_name(clockid));
}
}
ksft_exit_pass();
diff --git a/tools/testing/selftests/timers/leap-a-day.c b/tools/testing/selftests/timers/leap-a-day.c
index 3568cfb3e815..b93cb5714b37 100644
--- a/tools/testing/selftests/timers/leap-a-day.c
+++ b/tools/testing/selftests/timers/leap-a-day.c
@@ -9,16 +9,19 @@
* kernel's leap-second behavior, as well as how well applications
* handle the leap-second discontinuity.
*
- * Usage: leap-a-day [-s] [-i <num>]
+ * Usage: leap-a-day [-w] [-i <num>] [-t]
*
* Options:
- * -s: Each iteration, set the date to 10 seconds before midnight GMT.
- * This speeds up the number of leapsecond transitions tested,
- * but because it calls settimeofday frequently, advancing the
- * time by 24 hours every ~16 seconds, it may cause application
- * disruption.
+ * -w: Only set the leap-second flag and wait for the leap second
+ * each iteration, instead of advancing the time. By default the
+ * date is set to 10 seconds before midnight GMT, which speeds up
+ * the number of leapsecond transitions tested, but because it
+ * calls settimeofday frequently, advancing the time by 24 hours
+ * every ~16 seconds, it may cause application disruption.
*
- * -i: Number of iterations to run (default: infinite)
+ * -i: Number of iterations to run (-1 = infinite, default: 10)
+ *
+ * -t: Print TAI time.
*
* Other notes: Disabling NTP prior to running this is advised, as the two
* may conflict in their commands to the kernel.
@@ -48,7 +51,7 @@
#include <string.h>
#include <signal.h>
#include <unistd.h>
-#include <include/vdso/time64.h>
+#include "clock-helpers.h"
#include "kselftest.h"
#define CLOCK_TAI 11
@@ -186,7 +189,7 @@ int main(int argc, char **argv)
int opt;
/* Process arguments */
- while ((opt = getopt(argc, argv, "sti:")) != -1) {
+ while ((opt = getopt(argc, argv, "wti:")) != -1) {
switch (opt) {
case 'w':
printf("Only setting leap-flag, not changing time. It could take up to a day for leap to trigger.\n");
diff --git a/tools/testing/selftests/timers/mqueue-lat.c b/tools/testing/selftests/timers/mqueue-lat.c
index c0d9368e4fca..fa4c3e3f58fe 100644
--- a/tools/testing/selftests/timers/mqueue-lat.c
+++ b/tools/testing/selftests/timers/mqueue-lat.c
@@ -29,7 +29,7 @@
#include <signal.h>
#include <errno.h>
#include <mqueue.h>
-#include <include/vdso/time64.h>
+#include "clock-helpers.h"
#include "kselftest.h"
diff --git a/tools/testing/selftests/timers/nanosleep.c b/tools/testing/selftests/timers/nanosleep.c
index a054680b3372..b45e4c855259 100644
--- a/tools/testing/selftests/timers/nanosleep.c
+++ b/tools/testing/selftests/timers/nanosleep.c
@@ -27,43 +27,9 @@
#include <sys/timex.h>
#include <string.h>
#include <signal.h>
-#include <include/vdso/time64.h>
+#include "clock-helpers.h"
#include "kselftest.h"
-/* CLOCK_HWSPECIFIC == CLOCK_SGI_CYCLE (Deprecated) */
-#define CLOCK_HWSPECIFIC 10
-
-#define UNSUPPORTED 0xf00f
-
-char *clockstring(int clockid)
-{
- switch (clockid) {
- case CLOCK_REALTIME:
- return "CLOCK_REALTIME";
- case CLOCK_MONOTONIC:
- return "CLOCK_MONOTONIC";
- case CLOCK_PROCESS_CPUTIME_ID:
- return "CLOCK_PROCESS_CPUTIME_ID";
- case CLOCK_THREAD_CPUTIME_ID:
- return "CLOCK_THREAD_CPUTIME_ID";
- case CLOCK_MONOTONIC_RAW:
- return "CLOCK_MONOTONIC_RAW";
- case CLOCK_REALTIME_COARSE:
- return "CLOCK_REALTIME_COARSE";
- case CLOCK_MONOTONIC_COARSE:
- return "CLOCK_MONOTONIC_COARSE";
- case CLOCK_BOOTTIME:
- return "CLOCK_BOOTTIME";
- case CLOCK_REALTIME_ALARM:
- return "CLOCK_REALTIME_ALARM";
- case CLOCK_BOOTTIME_ALARM:
- return "CLOCK_BOOTTIME_ALARM";
- case CLOCK_TAI:
- return "CLOCK_TAI";
- };
- return "UNKNOWN_CLOCKID";
-}
-
/* returns 1 if a <= b, 0 otherwise */
static inline int in_order(struct timespec a, struct timespec b)
{
@@ -92,15 +58,15 @@ int nanosleep_test(int clockid, long long ns)
/* First check abs time */
if (clock_gettime(clockid, &now))
- return UNSUPPORTED;
+ return KSFT_SKIP;
target = timespec_add(now, ns);
if (clock_nanosleep(clockid, TIMER_ABSTIME, &target, NULL))
- return UNSUPPORTED;
+ return KSFT_SKIP;
clock_gettime(clockid, &now);
if (!in_order(target, now))
- return -1;
+ return KSFT_FAIL;
/* Second check reltime */
clock_gettime(clockid, &now);
@@ -112,8 +78,8 @@ int nanosleep_test(int clockid, long long ns)
clock_gettime(clockid, &now);
if (!in_order(target, now))
- return -1;
- return 0;
+ return KSFT_FAIL;
+ return KSFT_PASS;
}
static void dummy_event_handler(int val)
@@ -132,82 +98,86 @@ static int nanosleep_test_remaining(int clockid)
sa.sa_handler = dummy_event_handler;
ret = sigaction(SIGALRM, &sa, NULL);
if (ret)
- return -1;
+ return KSFT_FAIL;
ret = timer_create(clockid, NULL, &timer);
if (ret)
- return -1;
+ return KSFT_FAIL;
itimer.it_value.tv_nsec = NSEC_PER_SEC / 4;
ret = timer_settime(timer, 0, &itimer, NULL);
if (ret)
- return -1;
+ return KSFT_FAIL;
rqtp.tv_nsec = NSEC_PER_SEC / 2;
ret = clock_nanosleep(clockid, 0, &rqtp, &rmtp);
- if (ret != EINTR)
- return -1;
- ret = timer_delete(timer);
- if (ret)
- return -1;
+ if (timer_delete(timer)) {
+ ksft_exit_fail_msg("Unable to delete the timeout timer for %s. "
+ "This might interfere with following testcases.\n",
+ clock_name(clockid));
+ }
+
+ if (ret != EINTR)
+ return KSFT_FAIL;
sa.sa_handler = SIG_DFL;
ret = sigaction(SIGALRM, &sa, NULL);
if (ret)
- return -1;
+ return KSFT_FAIL;
if (!in_order((struct timespec) {}, rmtp))
- return -1;
+ return KSFT_FAIL;
if (!in_order(rmtp, rqtp))
- return -1;
+ return KSFT_FAIL;
- return 0;
+ return KSFT_PASS;
+}
+
+static void nanosleep_test_clock(clockid_t clockid)
+{
+ long long length = 10;
+ int ret;
+
+ while (length <= (NSEC_PER_SEC * 10)) {
+ ret = nanosleep_test(clockid, length);
+ if (ret != KSFT_PASS) {
+ ksft_test_result_report(ret, "%s\n", clock_name(clockid));
+ ksft_test_result_skip("%s (remaining)\n", clock_name(clockid));
+ return;
+ }
+
+ length *= 100;
+ }
+ ksft_test_result_pass("%s\n", clock_name(clockid));
+
+ ret = nanosleep_test_remaining(clockid);
+ ksft_test_result_report(ret, "%s (remaining)\n", clock_name(clockid));
}
int main(int argc, char **argv)
{
- long long length;
- int clockid, ret;
- int max_clocks = CLOCK_TAI + 1;
+ int clockid;
+
+ static const clockid_t tested_clocks[] = {
+ CLOCK_REALTIME,
+ CLOCK_MONOTONIC,
+ CLOCK_BOOTTIME,
+ CLOCK_BOOTTIME_ALARM,
+ CLOCK_REALTIME_ALARM,
+ CLOCK_TAI,
+ };
ksft_print_header();
- ksft_set_plan(max_clocks);
-
- for (clockid = CLOCK_REALTIME; clockid < max_clocks; clockid++) {
+ ksft_set_plan(ARRAY_SIZE(tested_clocks) * 2);
- /* Skip cputime clockids since nanosleep won't increment cputime */
- if (clockid == CLOCK_PROCESS_CPUTIME_ID ||
- clockid == CLOCK_THREAD_CPUTIME_ID ||
- clockid == CLOCK_HWSPECIFIC) {
- ksft_test_result_skip("%-31s\n", clockstring(clockid));
- continue;
- }
+ for (size_t clock_index = 0; clock_index < ARRAY_SIZE(tested_clocks); clock_index++) {
+ clockid = tested_clocks[clock_index];
fflush(stdout);
- length = 10;
- while (length <= (NSEC_PER_SEC * 10)) {
- ret = nanosleep_test(clockid, length);
- if (ret == UNSUPPORTED) {
- ksft_test_result_skip("%-31s\n", clockstring(clockid));
- goto next;
- }
- if (ret < 0) {
- ksft_test_result_fail("%-31s\n", clockstring(clockid));
- ksft_exit_fail();
- }
- length *= 100;
- }
- ret = nanosleep_test_remaining(clockid);
- if (ret < 0) {
- ksft_test_result_fail("%-31s\n", clockstring(clockid));
- ksft_exit_fail();
- }
- ksft_test_result_pass("%-31s\n", clockstring(clockid));
-next:
- ret = 0;
+ nanosleep_test_clock(clockid);
}
- ksft_exit_pass();
+ ksft_finished();
}
diff --git a/tools/testing/selftests/timers/nsleep-lat.c b/tools/testing/selftests/timers/nsleep-lat.c
index a7ba1eb1e21b..5de0051ac8e3 100644
--- a/tools/testing/selftests/timers/nsleep-lat.c
+++ b/tools/testing/selftests/timers/nsleep-lat.c
@@ -24,44 +24,10 @@
#include <sys/timex.h>
#include <string.h>
#include <signal.h>
-#include <include/vdso/time64.h>
+#include "clock-helpers.h"
#include "kselftest.h"
-#define UNRESONABLE_LATENCY 40000000 /* 40ms in nanosecs */
-
-/* CLOCK_HWSPECIFIC == CLOCK_SGI_CYCLE (Deprecated) */
-#define CLOCK_HWSPECIFIC 10
-
-#define UNSUPPORTED 0xf00f
-
-char *clockstring(int clockid)
-{
- switch (clockid) {
- case CLOCK_REALTIME:
- return "CLOCK_REALTIME";
- case CLOCK_MONOTONIC:
- return "CLOCK_MONOTONIC";
- case CLOCK_PROCESS_CPUTIME_ID:
- return "CLOCK_PROCESS_CPUTIME_ID";
- case CLOCK_THREAD_CPUTIME_ID:
- return "CLOCK_THREAD_CPUTIME_ID";
- case CLOCK_MONOTONIC_RAW:
- return "CLOCK_MONOTONIC_RAW";
- case CLOCK_REALTIME_COARSE:
- return "CLOCK_REALTIME_COARSE";
- case CLOCK_MONOTONIC_COARSE:
- return "CLOCK_MONOTONIC_COARSE";
- case CLOCK_BOOTTIME:
- return "CLOCK_BOOTTIME";
- case CLOCK_REALTIME_ALARM:
- return "CLOCK_REALTIME_ALARM";
- case CLOCK_BOOTTIME_ALARM:
- return "CLOCK_BOOTTIME_ALARM";
- case CLOCK_TAI:
- return "CLOCK_TAI";
- };
- return "UNKNOWN_CLOCKID";
-}
+#define UNRESONABLE_LATENCY (40 * NSEC_PER_MSEC)
struct timespec timespec_add(struct timespec ts, unsigned long long ns)
{
@@ -92,58 +58,68 @@ int nanosleep_lat_test(int clockid, long long ns)
target.tv_nsec = ns%NSEC_PER_SEC;
if (clock_gettime(clockid, &start))
- return UNSUPPORTED;
+ return KSFT_SKIP;
if (clock_nanosleep(clockid, 0, &target, NULL))
- return UNSUPPORTED;
+ return KSFT_SKIP;
count = 10;
/* First check relative latency */
- clock_gettime(clockid, &start);
- for (i = 0; i < count; i++)
- clock_nanosleep(clockid, 0, &target, NULL);
- clock_gettime(clockid, &end);
+ if (clock_gettime(clockid, &start))
+ return KSFT_FAIL;
+
+ for (i = 0; i < count; i++) {
+ if (clock_nanosleep(clockid, 0, &target, NULL))
+ return KSFT_FAIL;
+ }
+
+ if (clock_gettime(clockid, &end))
+ return KSFT_FAIL;
if (((timespec_sub(start, end)/count)-ns) > UNRESONABLE_LATENCY) {
ksft_print_msg("Large rel latency: %lld ns :", (timespec_sub(start, end)/count)-ns);
- return -1;
+ return KSFT_FAIL;
}
/* Next check absolute latency */
for (i = 0; i < count; i++) {
- clock_gettime(clockid, &start);
+ if (clock_gettime(clockid, &start))
+ return KSFT_FAIL;
target = timespec_add(start, ns);
- clock_nanosleep(clockid, TIMER_ABSTIME, &target, NULL);
- clock_gettime(clockid, &end);
+ if (clock_nanosleep(clockid, TIMER_ABSTIME, &target, NULL))
+ return KSFT_FAIL;
+ if (clock_gettime(clockid, &end))
+ return KSFT_FAIL;
latency += timespec_sub(target, end);
}
if (latency/count > UNRESONABLE_LATENCY) {
ksft_print_msg("Large abs latency: %lld ns :", latency/count);
- return -1;
+ return KSFT_FAIL;
}
- return 0;
+ return KSFT_PASS;
}
-#define SKIPPED_CLOCK_COUNT 3
-
int main(int argc, char **argv)
{
long long length;
int clockid, ret;
- int max_clocks = CLOCK_TAI + 1;
- ksft_print_header();
- ksft_set_plan(max_clocks - CLOCK_REALTIME - SKIPPED_CLOCK_COUNT);
+ static const clockid_t tested_clocks[] = {
+ CLOCK_REALTIME,
+ CLOCK_MONOTONIC,
+ CLOCK_BOOTTIME,
+ CLOCK_BOOTTIME_ALARM,
+ CLOCK_REALTIME_ALARM,
+ CLOCK_TAI,
+ };
- for (clockid = CLOCK_REALTIME; clockid < max_clocks; clockid++) {
+ ksft_print_header();
+ ksft_set_plan(ARRAY_SIZE(tested_clocks));
- /* Skip cputime clockids since nanosleep won't increment cputime */
- if (clockid == CLOCK_PROCESS_CPUTIME_ID ||
- clockid == CLOCK_THREAD_CPUTIME_ID ||
- clockid == CLOCK_HWSPECIFIC)
- continue;
+ for (size_t clock_index = 0; clock_index < ARRAY_SIZE(tested_clocks); clock_index++) {
+ clockid = tested_clocks[clock_index];
length = 10;
while (length <= (NSEC_PER_SEC * 10)) {
@@ -154,12 +130,7 @@ int main(int argc, char **argv)
}
- if (ret == UNSUPPORTED) {
- ksft_test_result_skip("%s\n", clockstring(clockid));
- } else {
- ksft_test_result(ret >= 0, "%s\n",
- clockstring(clockid));
- }
+ ksft_test_result_report(ret, "%s\n", clock_name(clockid));
}
ksft_finished();
diff --git a/tools/testing/selftests/timers/posix_timers.c b/tools/testing/selftests/timers/posix_timers.c
index 2f3bac9fc6e8..a92d4b957747 100644
--- a/tools/testing/selftests/timers/posix_timers.c
+++ b/tools/testing/selftests/timers/posix_timers.c
@@ -16,10 +16,10 @@
#include <string.h>
#include <unistd.h>
#include <time.h>
-#include <include/vdso/time64.h>
#include <pthread.h>
#include <stdbool.h>
+#include "clock-helpers.h"
#include "kselftest.h"
#define DELAY 2
@@ -141,8 +141,9 @@ static void check_itimer(int which, const char *name)
ksft_test_result(check_diff(start, end) == 0, "%s\n", name);
}
-static void check_timer_create(int which, const char *name)
+static void check_timer_create(int which)
{
+ const char *name = clock_name(which);
struct timespec start, end;
struct itimerspec val = {
.it_value.tv_sec = DELAY,
@@ -455,8 +456,9 @@ static void check_delete(void)
ksft_test_result(!tsig.signals, "check_delete\n");
}
-static void check_sigev_none(int which, const char *name)
+static void check_sigev_none(int which)
{
+ const char *name = clock_name(which);
struct timespec start, now;
struct itimerspec its;
struct sigevent sev;
@@ -493,8 +495,9 @@ static void check_sigev_none(int which, const char *name)
"check_sigev_none %s\n", name);
}
-static void check_gettime(int which, const char *name)
+static void check_gettime(int which)
{
+ const char *name = clock_name(which);
struct itimerspec its, prev;
struct timespec start, now;
struct sigevent sev;
@@ -546,8 +549,9 @@ static void check_gettime(int which, const char *name)
ksft_test_result(wraps > 1, "check_gettime %s\n", name);
}
-static void check_overrun(int which, const char *name)
+static void check_overrun(int which)
{
+ const char *name = clock_name(which);
struct timespec start, now;
struct tmrsig tsig = { };
struct itimerspec its;
@@ -689,7 +693,7 @@ int main(int argc, char **argv)
check_itimer(ITIMER_VIRTUAL, "ITIMER_VIRTUAL");
check_itimer(ITIMER_PROF, "ITIMER_PROF");
check_itimer(ITIMER_REAL, "ITIMER_REAL");
- check_timer_create(CLOCK_THREAD_CPUTIME_ID, "CLOCK_THREAD_CPUTIME_ID");
+ check_timer_create(CLOCK_THREAD_CPUTIME_ID);
/*
* It's unfortunately hard to reliably test a timer expiration
@@ -700,7 +704,7 @@ int main(int argc, char **argv)
* to ensure true parallelism. So test only one thread until we
* find a better solution.
*/
- check_timer_create(CLOCK_PROCESS_CPUTIME_ID, "CLOCK_PROCESS_CPUTIME_ID");
+ check_timer_create(CLOCK_PROCESS_CPUTIME_ID);
check_timer_distribution();
if (run_sig_ign_tests) {
@@ -708,18 +712,18 @@ int main(int argc, char **argv)
check_sig_ign(1);
check_rearm();
check_delete();
- check_sigev_none(CLOCK_MONOTONIC, "CLOCK_MONOTONIC");
- check_sigev_none(CLOCK_PROCESS_CPUTIME_ID, "CLOCK_PROCESS_CPUTIME_ID");
- check_gettime(CLOCK_MONOTONIC, "CLOCK_MONOTONIC");
- check_gettime(CLOCK_PROCESS_CPUTIME_ID, "CLOCK_PROCESS_CPUTIME_ID");
- check_gettime(CLOCK_THREAD_CPUTIME_ID, "CLOCK_THREAD_CPUTIME_ID");
+ check_sigev_none(CLOCK_MONOTONIC);
+ check_sigev_none(CLOCK_PROCESS_CPUTIME_ID);
+ check_gettime(CLOCK_MONOTONIC);
+ check_gettime(CLOCK_PROCESS_CPUTIME_ID);
+ check_gettime(CLOCK_THREAD_CPUTIME_ID);
} else {
ksft_print_msg("Skipping SIG_IGN tests on kernel < 6.13\n");
}
- check_overrun(CLOCK_MONOTONIC, "CLOCK_MONOTONIC");
- check_overrun(CLOCK_PROCESS_CPUTIME_ID, "CLOCK_PROCESS_CPUTIME_ID");
- check_overrun(CLOCK_THREAD_CPUTIME_ID, "CLOCK_THREAD_CPUTIME_ID");
+ check_overrun(CLOCK_MONOTONIC);
+ check_overrun(CLOCK_PROCESS_CPUTIME_ID);
+ check_overrun(CLOCK_THREAD_CPUTIME_ID);
ksft_finished();
}
diff --git a/tools/testing/selftests/timers/raw_skew.c b/tools/testing/selftests/timers/raw_skew.c
index a7bae7d80916..0c87a8fb0d7f 100644
--- a/tools/testing/selftests/timers/raw_skew.c
+++ b/tools/testing/selftests/timers/raw_skew.c
@@ -25,7 +25,7 @@
#include <sys/time.h>
#include <sys/timex.h>
#include <time.h>
-#include <include/vdso/time64.h>
+#include "clock-helpers.h"
#include "kselftest.h"
#define shift_right(x, s) ({ \
diff --git a/tools/testing/selftests/timers/set-2038.c b/tools/testing/selftests/timers/set-2038.c
index ecc171de4728..f522a3035ec6 100644
--- a/tools/testing/selftests/timers/set-2038.c
+++ b/tools/testing/selftests/timers/set-2038.c
@@ -27,7 +27,7 @@
#include <unistd.h>
#include <time.h>
#include <sys/time.h>
-#include <include/vdso/time64.h>
+#include "clock-helpers.h"
#include "kselftest.h"
#define KTIME_MAX ((long long)~((unsigned long long)1 << 63))
diff --git a/tools/testing/selftests/timers/set-timer-lat.c b/tools/testing/selftests/timers/set-timer-lat.c
index 44d2e3614fa5..79ddba25d314 100644
--- a/tools/testing/selftests/timers/set-timer-lat.c
+++ b/tools/testing/selftests/timers/set-timer-lat.c
@@ -28,7 +28,7 @@
#include <signal.h>
#include <stdlib.h>
#include <pthread.h>
-#include <include/vdso/time64.h>
+#include "clock-helpers.h"
#include "kselftest.h"
/* CLOCK_HWSPECIFIC == CLOCK_SGI_CYCLE (Deprecated) */
@@ -43,36 +43,6 @@ struct timespec start_time;
long long max_latency_ns;
int timer_fired_early;
-char *clockstring(int clockid)
-{
- switch (clockid) {
- case CLOCK_REALTIME:
- return "CLOCK_REALTIME";
- case CLOCK_MONOTONIC:
- return "CLOCK_MONOTONIC";
- case CLOCK_PROCESS_CPUTIME_ID:
- return "CLOCK_PROCESS_CPUTIME_ID";
- case CLOCK_THREAD_CPUTIME_ID:
- return "CLOCK_THREAD_CPUTIME_ID";
- case CLOCK_MONOTONIC_RAW:
- return "CLOCK_MONOTONIC_RAW";
- case CLOCK_REALTIME_COARSE:
- return "CLOCK_REALTIME_COARSE";
- case CLOCK_MONOTONIC_COARSE:
- return "CLOCK_MONOTONIC_COARSE";
- case CLOCK_BOOTTIME:
- return "CLOCK_BOOTTIME";
- case CLOCK_REALTIME_ALARM:
- return "CLOCK_REALTIME_ALARM";
- case CLOCK_BOOTTIME_ALARM:
- return "CLOCK_BOOTTIME_ALARM";
- case CLOCK_TAI:
- return "CLOCK_TAI";
- }
- return "UNKNOWN_CLOCKID";
-}
-
-
long long timespec_sub(struct timespec a, struct timespec b)
{
long long ret = NSEC_PER_SEC * b.tv_sec + b.tv_nsec;
@@ -103,7 +73,7 @@ void sigalarm(int signo)
void describe_timer(int flags, int interval)
{
printf("%-22s %s %s ",
- clockstring(clock_id),
+ clock_name(clock_id),
flags ? "ABSTIME":"RELTIME",
interval ? "PERIODIC":"ONE-SHOT");
}
@@ -129,12 +99,12 @@ int setup_timer(int clock_id, int flags, int interval, timer_t *tm1)
if ((clock_id == CLOCK_REALTIME_ALARM) ||
(clock_id == CLOCK_BOOTTIME_ALARM)) {
printf("%-22s %s missing CAP_WAKE_ALARM? : [UNSUPPORTED]\n",
- clockstring(clock_id),
+ clock_name(clock_id),
flags ? "ABSTIME":"RELTIME");
/* Indicate timer isn't set, so caller doesn't wait */
return 1;
}
- printf("%s - timer_create() failed\n", clockstring(clock_id));
+ printf("%s - timer_create() failed\n", clock_name(clock_id));
return -1;
}
@@ -151,7 +121,7 @@ int setup_timer(int clock_id, int flags, int interval, timer_t *tm1)
err = timer_settime(*tm1, flags, &its1, &its2);
if (err) {
- printf("%s - timer_settime() failed\n", clockstring(clock_id));
+ printf("%s - timer_settime() failed\n", clock_name(clock_id));
return -1;
}
diff --git a/tools/testing/selftests/timers/valid-adjtimex.c b/tools/testing/selftests/timers/valid-adjtimex.c
index e1e56d3097d6..f641d5fb0902 100644
--- a/tools/testing/selftests/timers/valid-adjtimex.c
+++ b/tools/testing/selftests/timers/valid-adjtimex.c
@@ -29,7 +29,7 @@
#include <string.h>
#include <signal.h>
#include <unistd.h>
-#include <include/vdso/time64.h>
+#include "clock-helpers.h"
#include "kselftest.h"
#define ADJ_SETOFFSET 0x0100
^ permalink raw reply related [flat|nested] 26+ messages in thread* Re: [GIT pull] timers/core for v7.3-rc1
2026-08-17 11:24 ` [GIT pull] timers/core " Thomas Gleixner
@ 2026-08-19 1:10 ` pr-tracker-bot
2026-08-19 1:10 ` pr-tracker-bot
1 sibling, 0 replies; 26+ messages in thread
From: pr-tracker-bot @ 2026-08-19 1:10 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, linux-kernel, x86
The pull request you sent on Mon, 17 Aug 2026 13:24:01 +0200:
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers-core-2026-08-17
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/3b4128b9f374b4219eb716f4ad8a307bc7eb3d84
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [GIT pull] timers/core for v7.3-rc1
2026-08-17 11:24 ` [GIT pull] timers/core " Thomas Gleixner
2026-08-19 1:10 ` pr-tracker-bot
@ 2026-08-19 1:10 ` pr-tracker-bot
1 sibling, 0 replies; 26+ messages in thread
From: pr-tracker-bot @ 2026-08-19 1:10 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, linux-kernel, x86
The pull request you sent on Mon, 17 Aug 2026 13:24:01 +0200:
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers-core-2026-08-17
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/3b4128b9f374b4219eb716f4ad8a307bc7eb3d84
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html
^ permalink raw reply [flat|nested] 26+ messages in thread
* [GIT pull] timers/vdso for v7.3-rc1
2026-08-17 11:23 [GIT pull] core/entry for v7.3-rc1 Thomas Gleixner
` (7 preceding siblings ...)
2026-08-17 11:24 ` [GIT pull] timers/core " Thomas Gleixner
@ 2026-08-17 11:24 ` Thomas Gleixner
2026-08-19 1:10 ` pr-tracker-bot
2026-08-19 1:10 ` pr-tracker-bot
2026-08-19 1:10 ` [GIT pull] core/entry " pr-tracker-bot
9 siblings, 2 replies; 26+ messages in thread
From: Thomas Gleixner @ 2026-08-17 11:24 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest timers/vdso branch from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers-vdso-2026-08-17
up to: 92e785d887a8: random: vDSO: Drop custom PAGE_SIZE definitions
VDSO updates:
- Consolidate the VDSO datastore further and provide support for
mlock_all() and prefaulting.
- Provide 32-bit legacy time related functionality only if
CONFIG_COMPAT_32BIT_TIME is enabled. The config switch exists, but
architecture code still exposes the legacy functionality even disabled.
Clean this up by adding the missing guards and validating at build time
that the VDSO is legacy free if disabled.
- Consolidate the VDSO related config options in core and drivers, which
removes some non-sensical dependencies and quite an amount of #ifdeffery.
- Clean up the PAGE_SIZE definition maze
Thanks,
tglx
------------------>
Thomas Huth (1):
vdso: Replace __ASSEMBLY__ with __ASSEMBLER__ in header files
Thomas Weißschuh (29):
vdso/datastore: Rename data pages variable
vdso/datastore: Map pages in terms of the faults pgoff
vdso/datastore: Map zeroed pages for unavailable data
vdso/datastore: Explicitly prevent remote access to timens vvar page
vdso/datastore: Allow prefaulting by mlockall()
vdso/datastore: Simplify the mapping logic for VDSO_TIME_PAGE_OFFSET
time: Respect COMPAT_32BIT_TIME for old time type functions
vdso/gettimeofday: Validate system call existence for time() and gettimeofday()
x86/vdso: Respect COMPAT_32BIT_TIME
arm64: vdso32: Respect COMPAT_32BIT_TIME
ARM: VDSO: Respect COMPAT_32BIT_TIME
powerpc/vdso: Respect COMPAT_32BIT_TIME
MIPS: VDSO: Respect COMPAT_32BIT_TIME
sparc: vdso: Respect COMPAT_32BIT_TIME
vdso/gettimeofday: Verify COMPAT_32BIT_TIME interactions
futex: Remove dependency on HAVE_GENERIC_VDSO from FUTEX_ROBUST_UNLOCK
vdso: Remove the dependency on HAVE_GENERIC_VDSO from ARCH_HAS_VDSO_ARCH_DATA
MIPS: vdso: Stop using CONFIG_HAVE_GENERIC_VDSO
vdso: Automatically select HAVE_GENERIC_VDSO if necessary
vdso: Drop HAVE_GENERIC_VDSO from architecture kconfig files
vdso: Rename HAVE_GENERIC_VDSO to VDSO_DATASTORE
kbuild: Support generated asm-headers in subdirectories
vDSO: Make clockmode constants available without CONFIG_GENERIC_GETTIMEOFDAY
MIPS: csrc-r4k: Remove CONFIG_GENERIC_GETTIMEOFDAY ifdeffery
clocksource/drivers/mips-gic-timer: Remove CONFIG_GENERIC_GETTIMEOFDAY ifdeffery
clocksource/drivers/arm_arch_timer: Remove CONFIG_GENERIC_GETTIMEOFDAY ifdeffery
clocksource/drivers/timer-riscv: Remove CONFIG_GENERIC_GETTIMEOFDAY ifdeffery
LoongArch: Remove CONFIG_GENERIC_GETTIMEOFDAY ifdeffery
random: vDSO: Drop custom PAGE_SIZE definitions
arch/Kconfig | 1 -
arch/arm/mm/Kconfig | 1 -
arch/arm/vdso/vdso.lds.S | 2 +
arch/arm/vdso/vgettimeofday.c | 14 ++++---
arch/arm64/Kconfig | 1 -
arch/arm64/kernel/vdso32/vdso.lds.S | 2 +
arch/arm64/kernel/vdso32/vgettimeofday.c | 14 ++++---
arch/loongarch/Kconfig | 2 +-
arch/loongarch/kernel/time.c | 2 -
arch/mips/Kconfig | 1 -
arch/mips/kernel/csrc-r4k.c | 2 -
arch/mips/kernel/vdso.c | 4 +-
arch/mips/vdso/vdso.lds.S | 2 +
arch/mips/vdso/vgettimeofday.c | 3 ++
arch/powerpc/Kconfig | 2 +-
arch/powerpc/kernel/vdso/gettimeofday.S | 8 ++++
arch/powerpc/kernel/vdso/vdso32.lds.S | 8 ++--
arch/powerpc/kernel/vdso/vgettimeofday.c | 16 ++++---
arch/riscv/Kconfig | 8 ++--
arch/s390/Kconfig | 1 -
arch/sparc/Kconfig | 1 -
arch/sparc/vdso/vclock_gettime.c | 4 ++
arch/sparc/vdso/vdso32/vdso32.lds.S | 6 ++-
arch/x86/Kconfig | 1 -
arch/x86/entry/vdso/common/vclock_gettime.c | 20 +++++----
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 2 +
drivers/clocksource/arm_arch_timer.c | 4 --
drivers/clocksource/mips-gic-timer.c | 2 -
drivers/clocksource/timer-riscv.c | 4 --
include/asm-generic/Kbuild | 1 +
include/asm-generic/vdso/clocksource.h | 0
include/asm-generic/vdso/vsyscall.h | 4 +-
include/linux/vdso_datastore.h | 6 +--
include/vdso/clocksource.h | 4 +-
include/vdso/datapage.h | 6 +--
include/vdso/helpers.h | 4 +-
include/vdso/processor.h | 4 +-
include/vdso/vsyscall.h | 4 +-
init/Kconfig | 2 +-
kernel/sys_ni.c | 4 ++
kernel/time/namespace_vdso.c | 7 +---
kernel/time/time.c | 24 +++++++++--
lib/vdso/Kconfig | 8 ++--
lib/vdso/Makefile | 2 +-
lib/vdso/datastore.c | 65 +++++++++++++----------------
lib/vdso/getrandom.c | 6 +--
lib/vdso/gettimeofday.c | 20 +++++++++
| 11 +++--
48 files changed, 181 insertions(+), 139 deletions(-)
create mode 100644 include/asm-generic/vdso/clocksource.h
diff --git a/arch/Kconfig b/arch/Kconfig
index fa7507ac8e13..8a8d1ab39757 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -1682,7 +1682,6 @@ config HAVE_SPARSE_SYSCALL_NR
related optimizations for a given architecture.
config ARCH_HAS_VDSO_ARCH_DATA
- depends on HAVE_GENERIC_VDSO
bool
config ARCH_HAS_VDSO_TIME_DATA
diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig
index 871bd58d2ccc..f7bea397a201 100644
--- a/arch/arm/mm/Kconfig
+++ b/arch/arm/mm/Kconfig
@@ -924,7 +924,6 @@ config VDSO
bool "Enable VDSO for acceleration of some system calls"
depends on AEABI && MMU && CPU_V7
default y if ARM_ARCH_TIMER
- select HAVE_GENERIC_VDSO
select GENERIC_GETTIMEOFDAY
help
Place in the process address space an ELF shared object
diff --git a/arch/arm/vdso/vdso.lds.S b/arch/arm/vdso/vdso.lds.S
index 74d8d8bc8a40..e61038c0195a 100644
--- a/arch/arm/vdso/vdso.lds.S
+++ b/arch/arm/vdso/vdso.lds.S
@@ -70,9 +70,11 @@ VERSION
{
LINUX_2.6 {
global:
+#ifdef CONFIG_COMPAT_32BIT_TIME
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_clock_getres;
+#endif /* CONFIG_COMPAT_32BIT_TIME */
__vdso_clock_gettime64;
__vdso_clock_getres_time64;
local: *;
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index f7a2f5dc2fdc..3eebeddbfd18 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -10,16 +10,17 @@
#include <asm/unwind.h>
#include <vdso/gettime.h>
+#ifdef CONFIG_COMPAT_32BIT_TIME
int __vdso_clock_gettime(clockid_t clock,
struct old_timespec32 *ts)
{
return __cvdso_clock_gettime32(clock, ts);
}
-int __vdso_clock_gettime64(clockid_t clock,
- struct __kernel_timespec *ts)
+int __vdso_clock_getres(clockid_t clock_id,
+ struct old_timespec32 *res)
{
- return __cvdso_clock_gettime(clock, ts);
+ return __cvdso_clock_getres_time32(clock_id, res);
}
int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
@@ -27,11 +28,12 @@ int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
{
return __cvdso_gettimeofday(tv, tz);
}
+#endif /* CONFIG_COMPAT_32BIT_TIME */
-int __vdso_clock_getres(clockid_t clock_id,
- struct old_timespec32 *res)
+int __vdso_clock_gettime64(clockid_t clock,
+ struct __kernel_timespec *ts)
{
- return __cvdso_clock_getres_time32(clock_id, res);
+ return __cvdso_clock_gettime(clock, ts);
}
int __vdso_clock_getres_time64(clockid_t clock_id, struct __kernel_timespec *res)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index b3afe0688919..11e733b6a3cf 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -229,7 +229,6 @@ config ARM64
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_KPROBES
select HAVE_KRETPROBES
- select HAVE_GENERIC_VDSO
select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select HOTPLUG_SMT if HOTPLUG_CPU
select IRQ_DOMAIN
diff --git a/arch/arm64/kernel/vdso32/vdso.lds.S b/arch/arm64/kernel/vdso32/vdso.lds.S
index c374fb0146f3..12bfc39e8aab 100644
--- a/arch/arm64/kernel/vdso32/vdso.lds.S
+++ b/arch/arm64/kernel/vdso32/vdso.lds.S
@@ -82,9 +82,11 @@ VERSION
{
LINUX_2.6 {
global:
+#ifdef CONFIG_COMPAT_32BIT_TIME
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_clock_getres;
+#endif /* CONFIG_COMPAT_32BIT_TIME */
__vdso_clock_gettime64;
__vdso_clock_getres_time64;
local: *;
diff --git a/arch/arm64/kernel/vdso32/vgettimeofday.c b/arch/arm64/kernel/vdso32/vgettimeofday.c
index 0c6998ebe491..12d0255cc2cf 100644
--- a/arch/arm64/kernel/vdso32/vgettimeofday.c
+++ b/arch/arm64/kernel/vdso32/vgettimeofday.c
@@ -8,16 +8,17 @@
#define BUILD_VDSO32_64
#include <vdso/gettime.h>
+#ifdef CONFIG_COMPAT_32BIT_TIME
int __vdso_clock_gettime(clockid_t clock,
struct old_timespec32 *ts)
{
return __cvdso_clock_gettime32(clock, ts);
}
-int __vdso_clock_gettime64(clockid_t clock,
- struct __kernel_timespec *ts)
+int __vdso_clock_getres(clockid_t clock_id,
+ struct old_timespec32 *res)
{
- return __cvdso_clock_gettime(clock, ts);
+ return __cvdso_clock_getres_time32(clock_id, res);
}
int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
@@ -25,11 +26,12 @@ int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
{
return __cvdso_gettimeofday(tv, tz);
}
+#endif /* CONFIG_COMPAT_32BIT_TIME */
-int __vdso_clock_getres(clockid_t clock_id,
- struct old_timespec32 *res)
+int __vdso_clock_gettime64(clockid_t clock,
+ struct __kernel_timespec *ts)
{
- return __cvdso_clock_getres_time32(clock_id, res);
+ return __cvdso_clock_gettime(clock, ts);
}
int __vdso_clock_getres_time64(clockid_t clock_id, struct __kernel_timespec *res)
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index d8d252325017..41ed648fcec9 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -153,7 +153,6 @@ config LOONGARCH
select HAVE_FUNCTION_GRAPH_TRACER
select HAVE_FUNCTION_TRACER
select HAVE_GCC_PLUGINS
- select HAVE_GENERIC_VDSO
select HAVE_HW_BREAKPOINT if PERF_EVENTS
select HAVE_IOREMAP_PROT
select HAVE_IRQ_EXIT_ON_IRQ_STACK
@@ -213,6 +212,7 @@ config LOONGARCH
select TRACE_IRQFLAGS_SUPPORT
select USE_PERCPU_NUMA_NODE_ID
select USER_STACKTRACE_SUPPORT
+ select VDSO_DATASTORE
select VDSO_GETRANDOM
select ZONE_DMA32 if 64BIT
diff --git a/arch/loongarch/kernel/time.c b/arch/loongarch/kernel/time.c
index dbaaabcaf6f0..5892f6da07a5 100644
--- a/arch/loongarch/kernel/time.c
+++ b/arch/loongarch/kernel/time.c
@@ -212,9 +212,7 @@ static struct clocksource clocksource_const = {
.read = read_const_counter,
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
-#ifdef CONFIG_GENERIC_GETTIMEOFDAY
.vdso_clock_mode = VDSO_CLOCKMODE_CPU,
-#endif
};
int __init constant_clocksource_init(void)
diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 8555bbf47c63..a06f24317306 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -3163,7 +3163,6 @@ config MIPS_EXTERNAL_TIMER
config MIPS_GENERIC_GETTIMEOFDAY
def_bool y
select GENERIC_GETTIMEOFDAY
- select HAVE_GENERIC_VDSO
depends on CSRC_R4K || CLKSRC_MIPS_GIC
# GCC (at least up to version 9.2) appears to emit function calls that make use
# of the GOT when targeting microMIPS, which we can't use in the VDSO due to
diff --git a/arch/mips/kernel/csrc-r4k.c b/arch/mips/kernel/csrc-r4k.c
index 241a934543a8..59eca397f297 100644
--- a/arch/mips/kernel/csrc-r4k.c
+++ b/arch/mips/kernel/csrc-r4k.c
@@ -126,14 +126,12 @@ int __init init_r4k_clocksource(void)
clocksource_mips.rating = 200;
clocksource_mips.rating += clamp(mips_hpt_frequency / 10000000, 0, 99);
-#ifdef CONFIG_GENERIC_GETTIMEOFDAY
/*
* R2 onwards makes the count accessible to user mode so it can be used
* by the VDSO (HWREna is configured by configure_hwrena()).
*/
if (cpu_has_mips_r2_r6 && rdhwr_count_usable())
clocksource_mips.vdso_clock_mode = VDSO_CLOCKMODE_R4K;
-#endif
clocksource_register_hz(&clocksource_mips, mips_hpt_frequency);
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index bd1fc17d3975..29a10045f2b6 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -129,7 +129,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
* This ensures that when the kernel updates the VDSO data userland
* will observe it without requiring cache invalidations.
*/
- if (cpu_has_dc_aliases && IS_ENABLED(CONFIG_HAVE_GENERIC_VDSO)) {
+ if (cpu_has_dc_aliases && IS_ENABLED(CONFIG_MIPS_GENERIC_GETTIMEOFDAY)) {
base = __ALIGN_MASK(base, shm_align_mask);
base += ((unsigned long)vdso_k_time_data - gic_size) & shm_align_mask;
}
@@ -137,7 +137,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
data_addr = base + gic_size;
vdso_addr = data_addr + VDSO_NR_PAGES * PAGE_SIZE;
- if (IS_ENABLED(CONFIG_HAVE_GENERIC_VDSO)) {
+ if (IS_ENABLED(CONFIG_MIPS_GENERIC_GETTIMEOFDAY)) {
vma = vdso_install_vvar_mapping(mm, data_addr);
if (IS_ERR(vma)) {
ret = PTR_ERR(vma);
diff --git a/arch/mips/vdso/vdso.lds.S b/arch/mips/vdso/vdso.lds.S
index 278ab6444e98..b11ee493c67f 100644
--- a/arch/mips/vdso/vdso.lds.S
+++ b/arch/mips/vdso/vdso.lds.S
@@ -97,9 +97,11 @@ VERSION
LINUX_2.6 {
#ifdef CONFIG_GENERIC_GETTIMEOFDAY
global:
+#if _MIPS_SIM == _MIPS_SIM_ABI64 || defined(CONFIG_COMPAT_32BIT_TIME)
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_clock_getres;
+#endif
#if _MIPS_SIM != _MIPS_SIM_ABI64
__vdso_clock_gettime64;
__vdso_clock_getres_time64;
diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
index 00f9fcfc327e..a1fb06b8973e 100644
--- a/arch/mips/vdso/vgettimeofday.c
+++ b/arch/mips/vdso/vgettimeofday.c
@@ -12,6 +12,8 @@
#include <vdso/gettime.h>
#if _MIPS_SIM != _MIPS_SIM_ABI64
+
+#ifdef CONFIG_COMPAT_32BIT_TIME
int __vdso_clock_gettime(clockid_t clock,
struct old_timespec32 *ts)
{
@@ -29,6 +31,7 @@ int __vdso_clock_getres(clockid_t clock_id,
{
return __cvdso_clock_getres_time32(clock_id, res);
}
+#endif /* CONFIG_COMPAT_32BIT_TIME */
int __vdso_clock_gettime64(clockid_t clock,
struct __kernel_timespec *ts)
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index f7ce5fff81f0..c6bc2cd6cc83 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -256,7 +256,6 @@ config PPC
select HAVE_FUNCTION_GRAPH_TRACER
select HAVE_FUNCTION_TRACER if !COMPILE_TEST && (PPC64 || (PPC32 && CC_IS_GCC))
select HAVE_GCC_PLUGINS
- select HAVE_GENERIC_VDSO
select HAVE_HARDLOCKUP_DETECTOR_ARCH if PPC_BOOK3S_64 && SMP
select HAVE_HARDLOCKUP_DETECTOR_PERF if PERF_EVENTS && HAVE_PERF_EVENTS_NMI
select HAVE_HW_BREAKPOINT if PERF_EVENTS && (PPC_BOOK3S || PPC_8xx)
@@ -328,6 +327,7 @@ config PPC
select SYSCTL_EXCEPTION_TRACE
select THREAD_INFO_IN_TASK
select TRACE_IRQFLAGS_SUPPORT
+ select VDSO_DATASTORE
select VDSO_GETRANDOM
#
# Please keep this list sorted alphabetically.
diff --git a/arch/powerpc/kernel/vdso/gettimeofday.S b/arch/powerpc/kernel/vdso/gettimeofday.S
index 1c8e51691bf8..c635cd1e77be 100644
--- a/arch/powerpc/kernel/vdso/gettimeofday.S
+++ b/arch/powerpc/kernel/vdso/gettimeofday.S
@@ -67,9 +67,11 @@
* int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz);
*
*/
+#if defined(__powerpc64__) || defined(CONFIG_COMPAT_32BIT_TIME)
V_FUNCTION_BEGIN(__kernel_gettimeofday)
cvdso_call __c_kernel_gettimeofday
V_FUNCTION_END(__kernel_gettimeofday)
+#endif
/*
* Exact prototype of clock_gettime()
@@ -77,9 +79,11 @@ V_FUNCTION_END(__kernel_gettimeofday)
* int __kernel_clock_gettime(clockid_t clock_id, struct timespec *tp);
*
*/
+#if defined(__powerpc64__) || defined(CONFIG_COMPAT_32BIT_TIME)
V_FUNCTION_BEGIN(__kernel_clock_gettime)
cvdso_call __c_kernel_clock_gettime
V_FUNCTION_END(__kernel_clock_gettime)
+#endif
/*
* Exact prototype of clock_gettime64()
@@ -99,9 +103,11 @@ V_FUNCTION_END(__kernel_clock_gettime64)
* int __kernel_clock_getres(clockid_t clock_id, struct timespec *res);
*
*/
+#if defined(__powerpc64__) || defined(CONFIG_COMPAT_32BIT_TIME)
V_FUNCTION_BEGIN(__kernel_clock_getres)
cvdso_call __c_kernel_clock_getres
V_FUNCTION_END(__kernel_clock_getres)
+#endif
/*
* Exact prototype of clock_getres_time64()
@@ -122,6 +128,8 @@ V_FUNCTION_END(__kernel_clock_getres_time64)
* time_t time(time *t);
*
*/
+#if defined(__powerpc64__) || defined(CONFIG_COMPAT_32BIT_TIME)
V_FUNCTION_BEGIN(__kernel_time)
cvdso_call __c_kernel_time call_time=1
V_FUNCTION_END(__kernel_time)
+#endif
diff --git a/arch/powerpc/kernel/vdso/vdso32.lds.S b/arch/powerpc/kernel/vdso/vdso32.lds.S
index 3f384a2526ae..5e87c18fd282 100644
--- a/arch/powerpc/kernel/vdso/vdso32.lds.S
+++ b/arch/powerpc/kernel/vdso/vdso32.lds.S
@@ -119,13 +119,15 @@ VERSION
{
VDSO_VERSION_STRING {
global:
- __kernel_get_syscall_map;
+#ifdef CONFIG_COMPAT_32BIT_TIME
__kernel_gettimeofday;
__kernel_clock_gettime;
- __kernel_clock_gettime64;
__kernel_clock_getres;
- __kernel_clock_getres_time64;
__kernel_time;
+#endif /* CONFIG_COMPAT_32BIT_TIME */
+ __kernel_get_syscall_map;
+ __kernel_clock_gettime64;
+ __kernel_clock_getres_time64;
__kernel_get_tbfreq;
__kernel_sync_dicache;
__kernel_sigtramp32;
diff --git a/arch/powerpc/kernel/vdso/vgettimeofday.c b/arch/powerpc/kernel/vdso/vgettimeofday.c
index 3c194e1ab562..4b712fb01a3f 100644
--- a/arch/powerpc/kernel/vdso/vgettimeofday.c
+++ b/arch/powerpc/kernel/vdso/vgettimeofday.c
@@ -18,23 +18,25 @@ int __c_kernel_clock_getres(clockid_t clock_id, struct __kernel_timespec *res,
return __cvdso_clock_getres_data(vd, clock_id, res);
}
#else
+#ifdef CONFIG_COMPAT_32BIT_TIME
int __c_kernel_clock_gettime(clockid_t clock, struct old_timespec32 *ts,
const struct vdso_time_data *vd)
{
return __cvdso_clock_gettime32_data(vd, clock, ts);
}
-int __c_kernel_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts,
- const struct vdso_time_data *vd)
-{
- return __cvdso_clock_gettime_data(vd, clock, ts);
-}
-
int __c_kernel_clock_getres(clockid_t clock_id, struct old_timespec32 *res,
const struct vdso_time_data *vd)
{
return __cvdso_clock_getres_time32_data(vd, clock_id, res);
}
+#endif /* CONFIG_COMPAT_32BIT_TIME */
+
+int __c_kernel_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts,
+ const struct vdso_time_data *vd)
+{
+ return __cvdso_clock_gettime_data(vd, clock, ts);
+}
int __c_kernel_clock_getres_time64(clockid_t clock_id, struct __kernel_timespec *res,
const struct vdso_time_data *vd)
@@ -43,6 +45,7 @@ int __c_kernel_clock_getres_time64(clockid_t clock_id, struct __kernel_timespec
}
#endif
+#if defined(__powerpc64__) || defined(CONFIG_COMPAT_32BIT_TIME)
int __c_kernel_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz,
const struct vdso_time_data *vd)
{
@@ -53,3 +56,4 @@ __kernel_old_time_t __c_kernel_time(__kernel_old_time_t *time, const struct vdso
{
return __cvdso_time_data(vd, time);
}
+#endif
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 3f0a647218e4..019fb4799943 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -54,7 +54,7 @@ config RISCV
select ARCH_HAS_SYSCALL_WRAPPER
select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST
select ARCH_HAS_UBSAN
- select ARCH_HAS_VDSO_ARCH_DATA if HAVE_GENERIC_VDSO
+ select ARCH_HAS_VDSO_ARCH_DATA
select ARCH_HAVE_NMI_SAFE_CMPXCHG
select ARCH_KEEP_MEMBLOCK if ACPI || KEXEC
select ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE if 64BIT && MMU
@@ -110,7 +110,7 @@ config RISCV
select GENERIC_CPU_VULNERABILITIES
select GENERIC_EARLY_IOREMAP
select GENERIC_ENTRY
- select GENERIC_GETTIMEOFDAY if HAVE_GENERIC_VDSO && 64BIT
+ select GENERIC_GETTIMEOFDAY if MMU && 64BIT
select GENERIC_IDLE_POLL_SETUP
select GENERIC_IOREMAP if MMU
select HAVE_IOREMAP_PROT if MMU
@@ -171,7 +171,6 @@ config RISCV
select HAVE_FUNCTION_ARG_ACCESS_API
select HAVE_FUNCTION_ERROR_INJECTION
select HAVE_GCC_PLUGINS
- select HAVE_GENERIC_VDSO if MMU
select HAVE_IRQ_TIME_ACCOUNTING
select HAVE_KERNEL_BZIP2 if !EFI_ZBOOT
select HAVE_KERNEL_GZIP if !EFI_ZBOOT
@@ -227,7 +226,8 @@ config RISCV
select THREAD_INFO_IN_TASK
select TRACE_IRQFLAGS_SUPPORT
select UACCESS_MEMCPY if !MMU
- select VDSO_GETRANDOM if HAVE_GENERIC_VDSO && 64BIT
+ select VDSO_DATASTORE if MMU
+ select VDSO_GETRANDOM if MMU && 64BIT
select USER_STACKTRACE_SUPPORT
select ZONE_DMA32 if 64BIT
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index 84404e6778d5..c9c55e3ddf1d 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -215,7 +215,6 @@ config S390
select HAVE_FUNCTION_GRAPH_TRACER
select HAVE_FUNCTION_TRACER
select HAVE_GCC_PLUGINS
- select HAVE_GENERIC_VDSO
select HAVE_IOREMAP_PROT if PCI
select HAVE_KERNEL_BZIP2
select HAVE_KERNEL_GZIP
diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index 0e9c906c4b5d..4dd79311133f 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -113,7 +113,6 @@ config SPARC64
select ARCH_SUPPORTS_SCHED_SMT if SMP
select ARCH_SUPPORTS_SCHED_MC if SMP
select ARCH_HAS_LAZY_MMU_MODE
- select HAVE_GENERIC_VDSO
select GENERIC_GETTIMEOFDAY
config ARCH_PROC_KCORE_TEXT
diff --git a/arch/sparc/vdso/vclock_gettime.c b/arch/sparc/vdso/vclock_gettime.c
index 1d9859392e4c..221bd4ed19f5 100644
--- a/arch/sparc/vdso/vclock_gettime.c
+++ b/arch/sparc/vdso/vclock_gettime.c
@@ -21,6 +21,7 @@
#include "../../../../lib/vdso/gettimeofday.c"
+#if defined(CONFIG_SPARC64) || defined(CONFIG_COMPAT_32BIT_TIME)
int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
@@ -28,6 +29,7 @@ int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz)
int gettimeofday(struct __kernel_old_timeval *, struct timezone *)
__weak __alias(__vdso_gettimeofday);
+#endif
#if defined(CONFIG_SPARC64)
int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
@@ -40,6 +42,7 @@ int clock_gettime(clockid_t, struct __kernel_timespec *)
#else
+#if defined(CONFIG_COMPAT_32BIT_TIME)
int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
{
return __cvdso_clock_gettime32(clock, ts);
@@ -47,6 +50,7 @@ int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
int clock_gettime(clockid_t, struct old_timespec32 *)
__weak __alias(__vdso_clock_gettime);
+#endif
int __vdso_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts)
{
diff --git a/arch/sparc/vdso/vdso32/vdso32.lds.S b/arch/sparc/vdso/vdso32/vdso32.lds.S
index a14e4f77e6f2..28052168b875 100644
--- a/arch/sparc/vdso/vdso32/vdso32.lds.S
+++ b/arch/sparc/vdso/vdso32/vdso32.lds.S
@@ -15,12 +15,14 @@
VERSION {
LINUX_2.6 {
global:
+#ifdef CONFIG_COMPAT_32BIT_TIME
clock_gettime;
__vdso_clock_gettime;
- clock_gettime64;
- __vdso_clock_gettime64;
gettimeofday;
__vdso_gettimeofday;
+#endif
+ clock_gettime64;
+ __vdso_clock_gettime64;
local: *;
};
}
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index bdad90f210e4..2f4a78024a96 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -302,7 +302,6 @@ config X86
select HAVE_UNSTABLE_SCHED_CLOCK
select HAVE_UNWIND_USER_FP if X86_64
select HAVE_USER_RETURN_NOTIFIER
- select HAVE_GENERIC_VDSO
select VDSO_GETRANDOM if X86_64
select HOTPLUG_PARALLEL if SMP && X86_64
select HOTPLUG_SMT if SMP
diff --git a/arch/x86/entry/vdso/common/vclock_gettime.c b/arch/x86/entry/vdso/common/vclock_gettime.c
index 57066f346b3f..304dbd1f9db4 100644
--- a/arch/x86/entry/vdso/common/vclock_gettime.c
+++ b/arch/x86/entry/vdso/common/vclock_gettime.c
@@ -15,6 +15,7 @@
#include "lib/vdso/gettimeofday.c"
+#if defined(__x86_64__) || defined(CONFIG_COMPAT_32BIT_TIME)
int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
@@ -29,6 +30,7 @@ __kernel_old_time_t __vdso_time(__kernel_old_time_t *t)
}
__kernel_old_time_t time(__kernel_old_time_t *t) __attribute__((weak, alias("__vdso_time")));
+#endif /* CONFIG_COMPAT_32BIT_TIME */
#if defined(CONFIG_X86_64) && !defined(BUILD_VDSO32_64)
@@ -51,6 +53,7 @@ int clock_getres(clockid_t, struct __kernel_timespec *)
#else
/* i386 only */
+#ifdef CONFIG_COMPAT_32BIT_TIME
int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
{
return __cvdso_clock_gettime32(clock, ts);
@@ -59,14 +62,6 @@ int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
int clock_gettime(clockid_t, struct old_timespec32 *)
__attribute__((weak, alias("__vdso_clock_gettime")));
-int __vdso_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts)
-{
- return __cvdso_clock_gettime(clock, ts);
-}
-
-int clock_gettime64(clockid_t, struct __kernel_timespec *)
- __attribute__((weak, alias("__vdso_clock_gettime64")));
-
int __vdso_clock_getres(clockid_t clock, struct old_timespec32 *res)
{
return __cvdso_clock_getres_time32(clock, res);
@@ -74,6 +69,15 @@ int __vdso_clock_getres(clockid_t clock, struct old_timespec32 *res)
int clock_getres(clockid_t, struct old_timespec32 *)
__attribute__((weak, alias("__vdso_clock_getres")));
+#endif /* CONFIG_COMPAT_32BIT_TIME */
+
+int __vdso_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+int clock_gettime64(clockid_t, struct __kernel_timespec *)
+ __attribute__((weak, alias("__vdso_clock_gettime64")));
int __vdso_clock_getres_time64(clockid_t clock, struct __kernel_timespec *ts)
{
diff --git a/arch/x86/entry/vdso/vdso32/vdso32.lds.S b/arch/x86/entry/vdso/vdso32/vdso32.lds.S
index cee8f7f9fe80..00629192db56 100644
--- a/arch/x86/entry/vdso/vdso32/vdso32.lds.S
+++ b/arch/x86/entry/vdso/vdso32/vdso32.lds.S
@@ -23,10 +23,12 @@ VERSION
{
LINUX_2.6 {
global:
+#ifdef CONFIG_COMPAT_32BIT_TIME
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_time;
__vdso_clock_getres;
+#endif /* CONFIG_COMPAT_32BIT_TIME */
__vdso_clock_gettime64;
__vdso_clock_getres_time64;
__vdso_getcpu;
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 4adf756423de..a5157d8a8a13 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -56,11 +56,7 @@ static struct clock_event_device __percpu *arch_timer_evt;
static enum arch_timer_ppi_nr arch_timer_uses_ppi __ro_after_init = ARCH_TIMER_VIRT_PPI;
static bool arch_timer_c3stop __ro_after_init;
static bool arch_counter_suspend_stop __ro_after_init;
-#ifdef CONFIG_GENERIC_GETTIMEOFDAY
static enum vdso_clock_mode vdso_default = VDSO_CLOCKMODE_ARCHTIMER;
-#else
-static enum vdso_clock_mode vdso_default = VDSO_CLOCKMODE_NONE;
-#endif /* CONFIG_GENERIC_GETTIMEOFDAY */
static cpumask_t evtstrm_available = CPU_MASK_NONE;
static bool evtstrm_enable __ro_after_init = IS_ENABLED(CONFIG_ARM_ARCH_TIMER_EVTSTREAM);
diff --git a/drivers/clocksource/mips-gic-timer.c b/drivers/clocksource/mips-gic-timer.c
index a1669266c94d..1501c7db9a8e 100644
--- a/drivers/clocksource/mips-gic-timer.c
+++ b/drivers/clocksource/mips-gic-timer.c
@@ -198,9 +198,7 @@ static struct clocksource gic_clocksource = {
.name = "GIC",
.read = gic_hpt_read,
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
-#ifdef CONFIG_GENERIC_GETTIMEOFDAY
.vdso_clock_mode = VDSO_CLOCKMODE_GIC,
-#endif
};
static void gic_clocksource_unstable(char *reason)
diff --git a/drivers/clocksource/timer-riscv.c b/drivers/clocksource/timer-riscv.c
index cfc4d83c42c0..b310e6721fac 100644
--- a/drivers/clocksource/timer-riscv.c
+++ b/drivers/clocksource/timer-riscv.c
@@ -98,11 +98,7 @@ static struct clocksource riscv_clocksource = {
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
.read = riscv_clocksource_rdtime,
-#if IS_ENABLED(CONFIG_GENERIC_GETTIMEOFDAY)
.vdso_clock_mode = VDSO_CLOCKMODE_ARCHTIMER,
-#else
- .vdso_clock_mode = VDSO_CLOCKMODE_NONE,
-#endif
};
static int riscv_timer_starting_cpu(unsigned int cpu)
diff --git a/include/asm-generic/Kbuild b/include/asm-generic/Kbuild
index 15df9dcb42a5..2bc00c67dc54 100644
--- a/include/asm-generic/Kbuild
+++ b/include/asm-generic/Kbuild
@@ -62,6 +62,7 @@ mandatory-y += topology.h
mandatory-y += trace_clock.h
mandatory-y += uaccess.h
mandatory-y += unwind_user.h
+mandatory-y += vdso/clocksource.h
mandatory-y += vermagic.h
mandatory-y += vga.h
mandatory-y += video.h
diff --git a/include/asm-generic/vdso/clocksource.h b/include/asm-generic/vdso/clocksource.h
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/include/asm-generic/vdso/vsyscall.h b/include/asm-generic/vdso/vsyscall.h
index 5c6d9799f4e7..a6b03cfba0e2 100644
--- a/include/asm-generic/vdso/vsyscall.h
+++ b/include/asm-generic/vdso/vsyscall.h
@@ -2,7 +2,7 @@
#ifndef __ASM_GENERIC_VSYSCALL_H
#define __ASM_GENERIC_VSYSCALL_H
-#ifndef __ASSEMBLY__
+#ifndef __ASSEMBLER__
#ifndef __arch_get_vdso_u_time_data
static __always_inline const struct vdso_time_data *__arch_get_vdso_u_time_data(void)
@@ -30,6 +30,6 @@ static __always_inline void __arch_sync_vdso_time_data(struct vdso_time_data *vd
}
#endif /* __arch_sync_vdso_time_data */
-#endif /* !__ASSEMBLY__ */
+#endif /* !__ASSEMBLER__ */
#endif /* __ASM_GENERIC_VSYSCALL_H */
diff --git a/include/linux/vdso_datastore.h b/include/linux/vdso_datastore.h
index 3dfba9502d78..13b01baf3497 100644
--- a/include/linux/vdso_datastore.h
+++ b/include/linux/vdso_datastore.h
@@ -7,10 +7,10 @@
extern const struct vm_special_mapping vdso_vvar_mapping;
struct vm_area_struct *vdso_install_vvar_mapping(struct mm_struct *mm, unsigned long addr);
-#ifdef CONFIG_HAVE_GENERIC_VDSO
+#ifdef CONFIG_VDSO_DATASTORE
void __init vdso_setup_data_pages(void);
-#else /* !CONFIG_HAVE_GENERIC_VDSO */
+#else /* !CONFIG_VDSO_DATASTORE */
static inline void vdso_setup_data_pages(void) { }
-#endif /* CONFIG_HAVE_GENERIC_VDSO */
+#endif /* CONFIG_VDSO_DATASTORE */
#endif /* _LINUX_VDSO_DATASTORE_H */
diff --git a/include/vdso/clocksource.h b/include/vdso/clocksource.h
index c682e7c60273..bce81c02a7c6 100644
--- a/include/vdso/clocksource.h
+++ b/include/vdso/clocksource.h
@@ -4,13 +4,11 @@
#include <vdso/limits.h>
-#ifdef CONFIG_GENERIC_GETTIMEOFDAY
#include <asm/vdso/clocksource.h>
-#endif /* CONFIG_GENERIC_GETTIMEOFDAY */
enum vdso_clock_mode {
VDSO_CLOCKMODE_NONE,
-#ifdef CONFIG_GENERIC_GETTIMEOFDAY
+#ifdef VDSO_ARCH_CLOCKMODES
VDSO_ARCH_CLOCKMODES,
#endif
VDSO_CLOCKMODE_MAX,
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index 5977723fb3b5..09897f76ae07 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -2,7 +2,7 @@
#ifndef __VDSO_DATAPAGE_H
#define __VDSO_DATAPAGE_H
-#ifndef __ASSEMBLY__
+#ifndef __ASSEMBLER__
#include <linux/types.h>
@@ -176,7 +176,7 @@ enum vdso_pages {
VDSO_NR_PAGES
};
-#else /* !__ASSEMBLY__ */
+#else /* !__ASSEMBLER__ */
#ifdef CONFIG_VDSO_GETRANDOM
#define __vdso_u_rng_data PROVIDE(vdso_u_rng_data = vdso_u_data + 2 * PAGE_SIZE);
@@ -197,6 +197,6 @@ enum vdso_pages {
__vdso_u_arch_data \
-#endif /* !__ASSEMBLY__ */
+#endif /* !__ASSEMBLER__ */
#endif /* __VDSO_DATAPAGE_H */
diff --git a/include/vdso/helpers.h b/include/vdso/helpers.h
index a3bf4f1c0d37..65151b681c4f 100644
--- a/include/vdso/helpers.h
+++ b/include/vdso/helpers.h
@@ -2,7 +2,7 @@
#ifndef __VDSO_HELPERS_H
#define __VDSO_HELPERS_H
-#ifndef __ASSEMBLY__
+#ifndef __ASSEMBLER__
#include <asm/barrier.h>
#include <vdso/datapage.h>
@@ -111,6 +111,6 @@ static __always_inline void vdso_write_end(struct vdso_time_data *vd)
vdso_write_seq_end(&vc[CS_RAW]);
}
-#endif /* !__ASSEMBLY__ */
+#endif /* !__ASSEMBLER__ */
#endif /* __VDSO_HELPERS_H */
diff --git a/include/vdso/processor.h b/include/vdso/processor.h
index fbe8265ea3c4..cc781912a696 100644
--- a/include/vdso/processor.h
+++ b/include/vdso/processor.h
@@ -5,10 +5,10 @@
#ifndef __VDSO_PROCESSOR_H
#define __VDSO_PROCESSOR_H
-#ifndef __ASSEMBLY__
+#ifndef __ASSEMBLER__
#include <asm/vdso/processor.h>
-#endif /* __ASSEMBLY__ */
+#endif /* __ASSEMBLER__ */
#endif /* __VDSO_PROCESSOR_H */
diff --git a/include/vdso/vsyscall.h b/include/vdso/vsyscall.h
index b0fdc9c6bf43..c5c2a2c07857 100644
--- a/include/vdso/vsyscall.h
+++ b/include/vdso/vsyscall.h
@@ -2,13 +2,13 @@
#ifndef __VDSO_VSYSCALL_H
#define __VDSO_VSYSCALL_H
-#ifndef __ASSEMBLY__
+#ifndef __ASSEMBLER__
#include <asm/vdso/vsyscall.h>
unsigned long vdso_update_begin(void);
void vdso_update_end(unsigned long flags);
-#endif /* !__ASSEMBLY__ */
+#endif /* !__ASSEMBLER__ */
#endif /* __VDSO_VSYSCALL_H */
diff --git a/init/Kconfig b/init/Kconfig
index 5230d4879b1c..53178ea4bc93 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1857,7 +1857,7 @@ config HAVE_FUTEX_ROBUST_UNLOCK
bool
config FUTEX_ROBUST_UNLOCK
- def_bool FUTEX && HAVE_GENERIC_VDSO && GENERIC_IRQ_ENTRY && RSEQ && HAVE_FUTEX_ROBUST_UNLOCK
+ def_bool FUTEX && GENERIC_IRQ_ENTRY && RSEQ && HAVE_FUTEX_ROBUST_UNLOCK
config EPOLL
bool "Enable eventpoll support" if EXPERT
diff --git a/kernel/sys_ni.c b/kernel/sys_ni.c
index add3032da16f..c8be0abaa407 100644
--- a/kernel/sys_ni.c
+++ b/kernel/sys_ni.c
@@ -351,6 +351,10 @@ COND_SYSCALL(ppoll_time32);
COND_SYSCALL_COMPAT(ppoll_time32);
COND_SYSCALL(utimensat_time32);
COND_SYSCALL(clock_adjtime32);
+COND_SYSCALL(gettimeofday);
+COND_SYSCALL_COMPAT(gettimeofday);
+COND_SYSCALL(time);
+COND_SYSCALL(stime);
/*
* The syscalls below are not found in include/uapi/asm-generic/unistd.h
diff --git a/kernel/time/namespace_vdso.c b/kernel/time/namespace_vdso.c
index 0d74d160eec9..5ac7b6a6d3a8 100644
--- a/kernel/time/namespace_vdso.c
+++ b/kernel/time/namespace_vdso.c
@@ -74,11 +74,8 @@ struct page *find_timens_vvar_page(struct vm_area_struct *vma)
return current->nsproxy->time_ns->vvar_page;
/*
- * VM_PFNMAP | VM_IO protect .fault() handler from being called
- * through interfaces like /proc/$pid/mem or
- * process_vm_{readv,writev}() as long as there's no .access()
- * in special_mapping_vmops().
- * For more details check_vma_flags() and __access_remote_vm()
+ * vvar_fault() protects this from being called through remote interfaces like
+ * /proc/$pid/mem or process_vm_{readv,writev}().
*/
WARN(1, "vvar_page accessed remotely");
diff --git a/kernel/time/time.c b/kernel/time/time.c
index 0dd63a91e7c5..f836a31751f4 100644
--- a/kernel/time/time.c
+++ b/kernel/time/time.c
@@ -43,6 +43,12 @@
#include <generated/timeconst.h>
#include "timekeeping.h"
+#if defined(CONFIG_64BIT) || defined(CONFIG_COMPAT_32BIT_TIME)
+#define __WANT_OLD_TIME_TYPE_SYSCALL 1
+#endif
+
+static_assert(sizeof(__kernel_old_time_t) == 8 ? IS_ENABLED(__WANT_OLD_TIME_TYPE_SYSCALL) : true);
+
/*
* The timezone where the local system is located. Used as a default by some
* programs who obtain this value by using gettimeofday.
@@ -51,7 +57,7 @@ struct timezone sys_tz;
EXPORT_SYMBOL(sys_tz);
-#ifdef __ARCH_WANT_SYS_TIME
+#if defined(__ARCH_WANT_SYS_TIME) && defined(__WANT_OLD_TIME_TYPE_SYSCALL)
/*
* sys_time() can be implemented in user-level using
@@ -96,7 +102,7 @@ SYSCALL_DEFINE1(stime, __kernel_old_time_t __user *, tptr)
return 0;
}
-#endif /* __ARCH_WANT_SYS_TIME */
+#endif /* __ARCH_WANT_SYS_TIME && __WANT_OLD_TIME_TYPE_SYSCALL */
#ifdef CONFIG_COMPAT_32BIT_TIME
#ifdef __ARCH_WANT_SYS_TIME32
@@ -137,6 +143,7 @@ SYSCALL_DEFINE1(stime32, old_time32_t __user *, tptr)
#endif /* __ARCH_WANT_SYS_TIME32 */
#endif
+#ifdef __WANT_OLD_TIME_TYPE_SYSCALL
SYSCALL_DEFINE2(gettimeofday, struct __kernel_old_timeval __user *, tv,
struct timezone __user *, tz)
{
@@ -154,6 +161,7 @@ SYSCALL_DEFINE2(gettimeofday, struct __kernel_old_timeval __user *, tv,
}
return 0;
}
+#endif /* __WANT_OLD_TIME_TYPE_SYSCALL */
/*
* In case for some reason the CMOS clock has not already been running
@@ -203,6 +211,9 @@ SYSCALL_DEFINE2(settimeofday, struct __kernel_old_timeval __user *, tv,
struct timezone new_tz;
if (tv) {
+ if (!IS_ENABLED(__WANT_OLD_TIME_TYPE_SYSCALL))
+ return -EINVAL;
+
if (get_user(new_ts.tv_sec, &tv->tv_sec) ||
get_user(new_ts.tv_nsec, &tv->tv_usec))
return -EFAULT;
@@ -220,7 +231,7 @@ SYSCALL_DEFINE2(settimeofday, struct __kernel_old_timeval __user *, tv,
return do_sys_settimeofday64(tv ? &new_ts : NULL, tz ? &new_tz : NULL);
}
-#ifdef CONFIG_COMPAT
+#ifdef CONFIG_COMPAT_32BIT_TIME
COMPAT_SYSCALL_DEFINE2(gettimeofday, struct old_timeval32 __user *, tv,
struct timezone __user *, tz)
{
@@ -239,7 +250,9 @@ COMPAT_SYSCALL_DEFINE2(gettimeofday, struct old_timeval32 __user *, tv,
return 0;
}
+#endif /* CONFIG_COMPAT_32BIT_TIME */
+#ifdef CONFIG_COMPAT
COMPAT_SYSCALL_DEFINE2(settimeofday, struct old_timeval32 __user *, tv,
struct timezone __user *, tz)
{
@@ -247,6 +260,9 @@ COMPAT_SYSCALL_DEFINE2(settimeofday, struct old_timeval32 __user *, tv,
struct timezone new_tz;
if (tv) {
+ if (!IS_ENABLED(CONFIG_COMPAT_32BIT_TIME))
+ return -EINVAL;
+
if (get_user(new_ts.tv_sec, &tv->tv_sec) ||
get_user(new_ts.tv_nsec, &tv->tv_usec))
return -EFAULT;
@@ -263,7 +279,7 @@ COMPAT_SYSCALL_DEFINE2(settimeofday, struct old_timeval32 __user *, tv,
return do_sys_settimeofday64(tv ? &new_ts : NULL, tz ? &new_tz : NULL);
}
-#endif
+#endif /* CONFIG_COMPAT */
#ifdef CONFIG_64BIT
SYSCALL_DEFINE1(adjtimex, struct __kernel_timex __user *, txc_p)
diff --git a/lib/vdso/Kconfig b/lib/vdso/Kconfig
index db87ba34ef19..597f5f0f9681 100644
--- a/lib/vdso/Kconfig
+++ b/lib/vdso/Kconfig
@@ -1,12 +1,11 @@
# SPDX-License-Identifier: GPL-2.0
-config HAVE_GENERIC_VDSO
+config VDSO_DATASTORE
bool
-if HAVE_GENERIC_VDSO
-
config GENERIC_GETTIMEOFDAY
bool
+ select VDSO_DATASTORE
help
This is a generic implementation of gettimeofday vdso.
Each architecture that enables this feature has to
@@ -21,7 +20,6 @@ config GENERIC_VDSO_OVERFLOW_PROTECT
config VDSO_GETRANDOM
bool
+ select VDSO_DATASTORE
help
Selected by architectures that support vDSO getrandom().
-
-endif
diff --git a/lib/vdso/Makefile b/lib/vdso/Makefile
index 405f743253d7..ac304def42d6 100644
--- a/lib/vdso/Makefile
+++ b/lib/vdso/Makefile
@@ -1,3 +1,3 @@
# SPDX-License-Identifier: GPL-2.0-only
-obj-$(CONFIG_HAVE_GENERIC_VDSO) += datastore.o
+obj-$(CONFIG_VDSO_DATASTORE) += datastore.o
diff --git a/lib/vdso/datastore.c b/lib/vdso/datastore.c
index 17d37b82ebc6..1426bf4e0c12 100644
--- a/lib/vdso/datastore.c
+++ b/lib/vdso/datastore.c
@@ -29,10 +29,11 @@ struct vdso_arch_data *vdso_k_arch_data __ro_after_init =
(void *)&vdso_initdata[VDSO_ARCH_PAGES_START * PAGE_SIZE];
#endif /* CONFIG_ARCH_HAS_VDSO_ARCH_DATA */
+static struct page *vdso_data_pages __ro_after_init;
+
void __init vdso_setup_data_pages(void)
{
unsigned int order = get_order(VDSO_NR_PAGES * PAGE_SIZE);
- struct page *pages;
/*
* Allocate the data pages dynamically. SPARC does not support mapping
@@ -42,24 +43,24 @@ void __init vdso_setup_data_pages(void)
* Do not use folios. In time namespaces the pages are mapped in a different order
* to userspace, which is not handled by the folio optimizations in finish_fault().
*/
- pages = alloc_pages(GFP_KERNEL, order);
- if (!pages)
+ vdso_data_pages = alloc_pages(GFP_KERNEL, order);
+ if (!vdso_data_pages)
panic("Unable to allocate VDSO storage pages");
/* The pages are mapped one-by-one into userspace and each one needs to be refcounted. */
- split_page(pages, order);
+ split_page(vdso_data_pages, order);
/* Move the data already written by other subsystems to the new pages */
- memcpy(page_address(pages), vdso_initdata, VDSO_NR_PAGES * PAGE_SIZE);
+ memcpy(page_address(vdso_data_pages), vdso_initdata, VDSO_NR_PAGES * PAGE_SIZE);
if (IS_ENABLED(CONFIG_GENERIC_GETTIMEOFDAY))
- vdso_k_time_data = page_address(pages + VDSO_TIME_PAGE_OFFSET);
+ vdso_k_time_data = page_address(vdso_data_pages + VDSO_TIME_PAGE_OFFSET);
if (IS_ENABLED(CONFIG_VDSO_GETRANDOM))
- vdso_k_rng_data = page_address(pages + VDSO_RNG_PAGE_OFFSET);
+ vdso_k_rng_data = page_address(vdso_data_pages + VDSO_RNG_PAGE_OFFSET);
if (IS_ENABLED(CONFIG_ARCH_HAS_VDSO_ARCH_DATA))
- vdso_k_arch_data = page_address(pages + VDSO_ARCH_PAGES_START);
+ vdso_k_arch_data = page_address(vdso_data_pages + VDSO_ARCH_PAGES_START);
}
static vm_fault_t vvar_fault(const struct vm_special_mapping *sm,
@@ -67,27 +68,28 @@ static vm_fault_t vvar_fault(const struct vm_special_mapping *sm,
{
struct page *page, *timens_page;
+ if (unlikely(vmf->flags & FAULT_FLAG_REMOTE))
+ return VM_FAULT_SIGBUS;
+
+ page = vdso_data_pages + vmf->pgoff;
timens_page = find_timens_vvar_page(vma);
switch (vmf->pgoff) {
case VDSO_TIME_PAGE_OFFSET:
- if (!IS_ENABLED(CONFIG_GENERIC_GETTIMEOFDAY))
- return VM_FAULT_SIGBUS;
- page = virt_to_page(vdso_k_time_data);
- if (timens_page) {
- /*
- * Fault in VVAR page too, since it will be accessed
- * to get clock data anyway.
- */
- unsigned long addr;
- vm_fault_t err;
-
- addr = vmf->address + VDSO_TIMENS_PAGE_OFFSET * PAGE_SIZE;
- err = vmf_insert_page(vma, addr, page);
- if (unlikely(err & VM_FAULT_ERROR))
- return err;
- page = timens_page;
- }
+ if (!IS_ENABLED(CONFIG_GENERIC_GETTIMEOFDAY) || !timens_page)
+ break;
+ /*
+ * Fault in VVAR page too, since it will be accessed
+ * to get clock data anyway.
+ */
+ unsigned long addr;
+ vm_fault_t err;
+
+ addr = vmf->address + VDSO_TIMENS_PAGE_OFFSET * PAGE_SIZE;
+ err = vmf_insert_page(vma, addr, page);
+ if (unlikely(err & VM_FAULT_ERROR))
+ return err;
+ page = timens_page;
break;
case VDSO_TIMENS_PAGE_OFFSET:
/*
@@ -98,18 +100,11 @@ static vm_fault_t vvar_fault(const struct vm_special_mapping *sm,
* See also the comment near timens_setup_vdso_data().
*/
if (!IS_ENABLED(CONFIG_TIME_NS) || !timens_page)
- return VM_FAULT_SIGBUS;
- page = virt_to_page(vdso_k_time_data);
+ break;
+ page = vdso_data_pages + VDSO_TIME_PAGE_OFFSET;
break;
case VDSO_RNG_PAGE_OFFSET:
- if (!IS_ENABLED(CONFIG_VDSO_GETRANDOM))
- return VM_FAULT_SIGBUS;
- page = virt_to_page(vdso_k_rng_data);
- break;
case VDSO_ARCH_PAGES_START ... VDSO_ARCH_PAGES_END:
- if (!IS_ENABLED(CONFIG_ARCH_HAS_VDSO_ARCH_DATA))
- return VM_FAULT_SIGBUS;
- page = virt_to_page(vdso_k_arch_data) + vmf->pgoff - VDSO_ARCH_PAGES_START;
break;
default:
return VM_FAULT_SIGBUS;
@@ -128,7 +123,7 @@ const struct vm_special_mapping vdso_vvar_mapping = {
struct vm_area_struct *vdso_install_vvar_mapping(struct mm_struct *mm, unsigned long addr)
{
return _install_special_mapping(mm, addr, VDSO_NR_PAGES * PAGE_SIZE,
- VM_READ | VM_MAYREAD | VM_IO | VM_DONTDUMP |
+ VM_READ | VM_MAYREAD | VM_DONTDUMP |
VM_MIXEDMAP | VM_SEALED_SYSMAP,
&vdso_vvar_mapping);
}
diff --git a/lib/vdso/getrandom.c b/lib/vdso/getrandom.c
index 7e29005aa208..2851afa9154f 100644
--- a/lib/vdso/getrandom.c
+++ b/lib/vdso/getrandom.c
@@ -8,6 +8,7 @@
#include <vdso/datapage.h>
#include <vdso/getrandom.h>
#include <vdso/limits.h>
+#include <vdso/page.h>
#include <vdso/unaligned.h>
#include <asm/barrier.h>
#include <asm/vdso/getrandom.h>
@@ -18,11 +19,6 @@
/* Bring in default accessors */
#include <vdso/vsyscall.h>
-#undef PAGE_SIZE
-#undef PAGE_MASK
-#define PAGE_SIZE (1UL << CONFIG_PAGE_SHIFT)
-#define PAGE_MASK (~(PAGE_SIZE - 1))
-
#define MEMCPY_AND_ZERO_SRC(type, dst, src, len) do { \
while (len >= sizeof(type)) { \
__put_unaligned_t(type, __get_unaligned_t(type, src), dst); \
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index e0f289d3d110..f7a591aba59f 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -12,6 +12,8 @@
#include <vdso/time32.h>
#include <vdso/time64.h>
+#include <uapi/linux/unistd.h>
+
/*
* The generic vDSO implementation requires that gettimeofday.h
* provides:
@@ -23,6 +25,8 @@
*/
#include <asm/vdso/gettimeofday.h>
+#include <linux/build_bug.h>
+
/* Bring in default accessors */
#include <vdso/vsyscall.h>
@@ -323,6 +327,8 @@ __cvdso_clock_gettime32_data(const struct vdso_time_data *vd, clockid_t clock,
struct __kernel_timespec ts;
bool ok;
+ BUILD_BUG_ON(!IS_ENABLED(CONFIG_COMPAT_32BIT_TIME));
+
ok = __cvdso_clock_gettime_common(vd, clock, &ts);
if (unlikely(!ok))
@@ -348,6 +354,12 @@ __cvdso_gettimeofday_data(const struct vdso_time_data *vd,
{
const struct vdso_clock *vc = vd->clock_data;
+#ifndef __NR_gettimeofday
+ BUILD_BUG();
+#endif
+
+ BUILD_BUG_ON(sizeof(tv->tv_sec) != 8 && !IS_ENABLED(CONFIG_COMPAT_32BIT_TIME));
+
if (likely(tv != NULL)) {
struct __kernel_timespec ts;
@@ -382,6 +394,12 @@ __cvdso_time_data(const struct vdso_time_data *vd, __kernel_old_time_t *time)
const struct vdso_clock *vc = vd->clock_data;
__kernel_old_time_t t;
+#ifndef __NR_time
+ BUILD_BUG();
+#endif
+
+ BUILD_BUG_ON(sizeof(*time) != 8 && !IS_ENABLED(CONFIG_COMPAT_32BIT_TIME));
+
if (vdso_is_timens_clock(vc)) {
vd = vdso_timens_data(vd);
vc = vd->clock_data;
@@ -471,6 +489,8 @@ __cvdso_clock_getres_time32_data(const struct vdso_time_data *vd, clockid_t cloc
struct __kernel_timespec ts;
bool ok;
+ BUILD_BUG_ON(!IS_ENABLED(CONFIG_COMPAT_32BIT_TIME));
+
ok = __cvdso_clock_getres_common(vd, clock, &ts);
if (unlikely(!ok))
--git a/scripts/Makefile.asm-headers b/scripts/Makefile.asm-headers
index 8a4856e74180..b38931314ad7 100644
--- a/scripts/Makefile.asm-headers
+++ b/scripts/Makefile.asm-headers
@@ -48,14 +48,13 @@ syscall-y := $(addprefix $(obj)/, $(syscall-y))
generated-y := $(addprefix $(obj)/, $(generated-y))
# Remove stale wrappers when the corresponding files are removed from generic-y
-old-headers := $(wildcard $(obj)/*.h)
+old-headers := $(shell test -d $(obj) && find $(obj) -name *.h)
unwanted := $(filter-out $(generic-y) $(generated-y) $(syscall-y),$(old-headers))
-quiet_cmd_wrap = WRAP $@
- cmd_wrap = echo "\#include <asm-generic/$*.h>" > $@
+filechk_wrap = echo "\#include <asm-generic/$*.h>"
quiet_cmd_remove = REMOVE $(unwanted)
- cmd_remove = rm -f $(unwanted)
+ cmd_remove = rm -f $(unwanted); find $(obj) -type d -empty -delete
quiet_cmd_syshdr = SYSHDR $@
cmd_syshdr = $(CONFIG_SHELL) $(syshdr) \
@@ -74,8 +73,8 @@ all: $(generic-y) $(syscall-y)
$(if $(unwanted),$(call cmd,remove))
@:
-$(obj)/%.h: $(srctree)/$(generic)/%.h
- $(call cmd,wrap)
+$(obj)/%.h: $(srctree)/$(generic)/%.h FORCE
+ $(call filechk,wrap)
$(obj)/unistd_%.h: $(syscalltbl) $(syshdr) FORCE
$(call if_changed,syshdr)
^ permalink raw reply related [flat|nested] 26+ messages in thread* Re: [GIT pull] timers/vdso for v7.3-rc1
2026-08-17 11:24 ` [GIT pull] timers/vdso " Thomas Gleixner
@ 2026-08-19 1:10 ` pr-tracker-bot
2026-08-19 1:10 ` pr-tracker-bot
1 sibling, 0 replies; 26+ messages in thread
From: pr-tracker-bot @ 2026-08-19 1:10 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, linux-kernel, x86
The pull request you sent on Mon, 17 Aug 2026 13:24:06 +0200:
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers-vdso-2026-08-17
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/104a813376837da3b9651457d6fdc99596257fba
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [GIT pull] timers/vdso for v7.3-rc1
2026-08-17 11:24 ` [GIT pull] timers/vdso " Thomas Gleixner
2026-08-19 1:10 ` pr-tracker-bot
@ 2026-08-19 1:10 ` pr-tracker-bot
1 sibling, 0 replies; 26+ messages in thread
From: pr-tracker-bot @ 2026-08-19 1:10 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, linux-kernel, x86
The pull request you sent on Mon, 17 Aug 2026 13:24:06 +0200:
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers-vdso-2026-08-17
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/104a813376837da3b9651457d6fdc99596257fba
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [GIT pull] core/entry for v7.3-rc1
2026-08-17 11:23 [GIT pull] core/entry for v7.3-rc1 Thomas Gleixner
` (8 preceding siblings ...)
2026-08-17 11:24 ` [GIT pull] timers/vdso " Thomas Gleixner
@ 2026-08-19 1:10 ` pr-tracker-bot
9 siblings, 0 replies; 26+ messages in thread
From: pr-tracker-bot @ 2026-08-19 1:10 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, linux-kernel, x86
The pull request you sent on Mon, 17 Aug 2026 13:23:21 +0200:
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git core-entry-2026-08-17
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/3424d8c18a7da1010d03391a22e728b857d0d5c5
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html
^ permalink raw reply [flat|nested] 26+ messages in thread