* [patch 01/55] tile: Convert VDSO timekeeping to the precise mechanism
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
@ 2014-07-11 13:43 ` Thomas Gleixner
2014-07-18 2:32 ` Chris Metcalf
2014-07-11 13:44 ` [patch 02/55] timekeeping: Simplify arch_gettimeoffset() Thomas Gleixner
` (53 subsequent siblings)
54 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:43 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: tile-fix-broken-vdso.patch --]
[-- Type: text/plain, Size: 2131 bytes --]
The code was only halfarsed converted to the new VSDO update mechanism
and still uses the inaccurate base value which lacks the fractional
part of xtime_nsec. Fix it up.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/tile/kernel/time.c | 9 ++++-----
arch/tile/kernel/vdso/vgettimeofday.c | 7 ++++---
2 files changed, 8 insertions(+), 8 deletions(-)
Index: tip/arch/tile/kernel/time.c
===================================================================
--- tip.orig/arch/tile/kernel/time.c
+++ tip/arch/tile/kernel/time.c
@@ -260,7 +260,6 @@ void update_vsyscall_tz(void)
void update_vsyscall(struct timekeeper *tk)
{
- struct timespec wall_time = tk_xtime(tk);
struct timespec *wtm = &tk->wall_to_monotonic;
struct clocksource *clock = tk->clock;
@@ -271,12 +270,12 @@ void update_vsyscall(struct timekeeper *
++vdso_data->tb_update_count;
smp_wmb();
vdso_data->xtime_tod_stamp = clock->cycle_last;
- vdso_data->xtime_clock_sec = wall_time.tv_sec;
- vdso_data->xtime_clock_nsec = wall_time.tv_nsec;
+ vdso_data->xtime_clock_sec = tk->xtime_sec;
+ vdso_data->xtime_clock_nsec = tk->xtime_nsec;
vdso_data->wtom_clock_sec = wtm->tv_sec;
vdso_data->wtom_clock_nsec = wtm->tv_nsec;
- vdso_data->mult = clock->mult;
- vdso_data->shift = clock->shift;
+ vdso_data->mult = tk->mult;
+ vdso_data->shift = tk->shift;
smp_wmb();
++vdso_data->tb_update_count;
}
Index: tip/arch/tile/kernel/vdso/vgettimeofday.c
===================================================================
--- tip.orig/arch/tile/kernel/vdso/vgettimeofday.c
+++ tip/arch/tile/kernel/vdso/vgettimeofday.c
@@ -83,10 +83,11 @@ int __vdso_gettimeofday(struct timeval *
if (count & 1)
continue;
- cycles = (get_cycles() - vdso_data->xtime_tod_stamp);
- ns = (cycles * vdso_data->mult) >> vdso_data->shift;
sec = vdso_data->xtime_clock_sec;
- ns += vdso_data->xtime_clock_nsec;
+ cycles = get_cycles() - vdso_data->xtime_tod_stamp;
+ ns = (cycles * vdso_data->mult) + vdso_data->xtime_clock_nsec;
+ ns >>= vdso_data->shift;
+
if (ns >= NSEC_PER_SEC) {
ns -= NSEC_PER_SEC;
sec += 1;
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 02/55] timekeeping: Simplify arch_gettimeoffset()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
2014-07-11 13:43 ` [patch 01/55] tile: Convert VDSO timekeeping to the precise mechanism Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 14:10 ` Geert Uytterhoeven
2014-07-11 13:44 ` [patch 03/55] hrtimer: Cleanup hrtimer accessors to the timekepeing state Thomas Gleixner
` (52 subsequent siblings)
54 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-simplify-arch-offset.patch --]
[-- Type: text/plain, Size: 1986 bytes --]
Provide a default stub function instead of having the extra
conditional. Cuts binary size on a m68k build by ~100 bytes.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/time/timekeeping.c | 18 ++++++------------
1 file changed, 6 insertions(+), 12 deletions(-)
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -153,16 +153,10 @@ static void tk_setup_internals(struct ti
/* Timekeeper helper functions. */
#ifdef CONFIG_ARCH_USES_GETTIMEOFFSET
-u32 (*arch_gettimeoffset)(void);
-
-u32 get_arch_timeoffset(void)
-{
- if (likely(arch_gettimeoffset))
- return arch_gettimeoffset();
- return 0;
-}
+static u32 default_arch_gettimeoffset(void) { return 0; }
+u32 (*arch_gettimeoffset)(void) = default_arch_gettimeoffset;
#else
-static inline u32 get_arch_timeoffset(void) { return 0; }
+static inline u32 arch_gettimeoffset(void) { return 0; }
#endif
static inline s64 timekeeping_get_ns(struct timekeeper *tk)
@@ -182,7 +176,7 @@ static inline s64 timekeeping_get_ns(str
nsec >>= tk->shift;
/* If arch requires, add in get_arch_timeoffset() */
- return nsec + get_arch_timeoffset();
+ return nsec + arch_gettimeoffset();
}
static inline s64 timekeeping_get_ns_raw(struct timekeeper *tk)
@@ -202,7 +196,7 @@ static inline s64 timekeeping_get_ns_raw
nsec = clocksource_cyc2ns(cycle_delta, clock->mult, clock->shift);
/* If arch requires, add in get_arch_timeoffset() */
- return nsec + get_arch_timeoffset();
+ return nsec + arch_gettimeoffset();
}
static RAW_NOTIFIER_HEAD(pvclock_gtod_chain);
@@ -282,7 +276,7 @@ static void timekeeping_forward_now(stru
tk->xtime_nsec += cycle_delta * tk->mult;
/* If arch requires, add in get_arch_timeoffset() */
- tk->xtime_nsec += (u64)get_arch_timeoffset() << tk->shift;
+ tk->xtime_nsec += (u64)arch_gettimeoffset() << tk->shift;
tk_normalize_xtime(tk);
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 02/55] timekeeping: Simplify arch_gettimeoffset()
2014-07-11 13:44 ` [patch 02/55] timekeeping: Simplify arch_gettimeoffset() Thomas Gleixner
@ 2014-07-11 14:10 ` Geert Uytterhoeven
0 siblings, 0 replies; 91+ messages in thread
From: Geert Uytterhoeven @ 2014-07-11 14:10 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Peter Zijlstra
Hi Thomas,
On Fri, Jul 11, 2014 at 3:44 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
> Provide a default stub function instead of having the extra
> conditional. Cuts binary size on a m68k build by ~100 bytes.
Thanks about caring about binary size.
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply [flat|nested] 91+ messages in thread
* [patch 03/55] hrtimer: Cleanup hrtimer accessors to the timekepeing state
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
2014-07-11 13:43 ` [patch 01/55] tile: Convert VDSO timekeeping to the precise mechanism Thomas Gleixner
2014-07-11 13:44 ` [patch 02/55] timekeeping: Simplify arch_gettimeoffset() Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 04/55] ktime: Kill non-scalar ktime_t implementation for 2038 Thomas Gleixner
` (51 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: jstultz-tk-ed502d8.patch --]
[-- Type: text/plain, Size: 6077 bytes --]
From: John Stultz <john.stultz@linaro.org>
Rather then having two similar but totally different implementations
that provide timekeeping state to the hrtimer code, try to unify the
two implementations to be more simliar.
Thus this clarifies ktime_get_update_offsets to
ktime_get_update_offsets_now and changes get_xtime... to
ktime_get_update_offsets_tick.
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/hrtimer.h | 9 ++++++---
include/linux/time.h | 2 --
kernel/time/hrtimer.c | 23 ++++++++++-------------
kernel/time/timekeeping.c | 36 +++++++++++++++++++++++-------------
4 files changed, 39 insertions(+), 31 deletions(-)
Index: tip/include/linux/hrtimer.h
===================================================================
--- tip.orig/include/linux/hrtimer.h
+++ tip/include/linux/hrtimer.h
@@ -331,9 +331,12 @@ extern ktime_t ktime_get_real(void);
extern ktime_t ktime_get_boottime(void);
extern ktime_t ktime_get_monotonic_offset(void);
extern ktime_t ktime_get_clocktai(void);
-extern ktime_t ktime_get_update_offsets(ktime_t *offs_real, ktime_t *offs_boot,
- ktime_t *offs_tai);
-
+extern ktime_t ktime_get_update_offsets_tick(ktime_t *offs_real,
+ ktime_t *offs_boot,
+ ktime_t *offs_tai);
+extern ktime_t ktime_get_update_offsets_now(ktime_t *offs_real,
+ ktime_t *offs_boot,
+ ktime_t *offs_tai);
DECLARE_PER_CPU(struct tick_device, tick_cpu_device);
Index: tip/include/linux/time.h
===================================================================
--- tip.orig/include/linux/time.h
+++ tip/include/linux/time.h
@@ -133,8 +133,6 @@ unsigned long get_seconds(void);
struct timespec current_kernel_time(void);
struct timespec __current_kernel_time(void); /* does not take xtime_lock */
struct timespec get_monotonic_coarse(void);
-void get_xtime_and_monotonic_and_sleep_offset(struct timespec *xtim,
- struct timespec *wtom, struct timespec *sleep);
void timekeeping_inject_sleeptime(struct timespec *delta);
#define CURRENT_TIME (current_kernel_time())
Index: tip/kernel/time/hrtimer.c
===================================================================
--- tip.orig/kernel/time/hrtimer.c
+++ tip/kernel/time/hrtimer.c
@@ -114,21 +114,18 @@ static inline int hrtimer_clockid_to_bas
*/
static void hrtimer_get_softirq_time(struct hrtimer_cpu_base *base)
{
- ktime_t xtim, mono, boot;
- struct timespec xts, tom, slp;
- s32 tai_offset;
-
- get_xtime_and_monotonic_and_sleep_offset(&xts, &tom, &slp);
- tai_offset = timekeeping_get_tai_offset();
-
- xtim = timespec_to_ktime(xts);
- mono = ktime_add(xtim, timespec_to_ktime(tom));
- boot = ktime_add(mono, timespec_to_ktime(slp));
+ ktime_t xtim, mono, boot, tai;
+ ktime_t off_real, off_boot, off_tai;
+
+ mono = ktime_get_update_offsets_tick(&off_real, &off_boot, &off_tai);
+ boot = ktime_add(mono, off_boot);
+ xtim = ktime_add(mono, off_real);
+ tai = ktime_add(xtim, off_tai);
+
base->clock_base[HRTIMER_BASE_REALTIME].softirq_time = xtim;
base->clock_base[HRTIMER_BASE_MONOTONIC].softirq_time = mono;
base->clock_base[HRTIMER_BASE_BOOTTIME].softirq_time = boot;
- base->clock_base[HRTIMER_BASE_TAI].softirq_time =
- ktime_add(xtim, ktime_set(tai_offset, 0));
+ base->clock_base[HRTIMER_BASE_TAI].softirq_time = tai;
}
/*
@@ -673,7 +670,7 @@ static inline ktime_t hrtimer_update_bas
ktime_t *offs_boot = &base->clock_base[HRTIMER_BASE_BOOTTIME].offset;
ktime_t *offs_tai = &base->clock_base[HRTIMER_BASE_TAI].offset;
- return ktime_get_update_offsets(offs_real, offs_boot, offs_tai);
+ return ktime_get_update_offsets_now(offs_real, offs_boot, offs_tai);
}
/*
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -1581,29 +1581,39 @@ void do_timer(unsigned long ticks)
}
/**
- * get_xtime_and_monotonic_and_sleep_offset() - get xtime, wall_to_monotonic,
- * and sleep offsets.
- * @xtim: pointer to timespec to be set with xtime
- * @wtom: pointer to timespec to be set with wall_to_monotonic
- * @sleep: pointer to timespec to be set with time in suspend
+ * ktime_get_update_offsets_tick - hrtimer helper
+ * @offs_real: pointer to storage for monotonic -> realtime offset
+ * @offs_boot: pointer to storage for monotonic -> boottime offset
+ * @offs_tai: pointer to storage for monotonic -> clock tai offset
+ *
+ * Returns monotonic time at last tick and various offsets
*/
-void get_xtime_and_monotonic_and_sleep_offset(struct timespec *xtim,
- struct timespec *wtom, struct timespec *sleep)
+ktime_t ktime_get_update_offsets_tick(ktime_t *offs_real, ktime_t *offs_boot,
+ ktime_t *offs_tai)
{
struct timekeeper *tk = &timekeeper;
- unsigned long seq;
+ struct timespec ts;
+ ktime_t now;
+ unsigned int seq;
do {
seq = read_seqcount_begin(&timekeeper_seq);
- *xtim = tk_xtime(tk);
- *wtom = tk->wall_to_monotonic;
- *sleep = tk->total_sleep_time;
+
+ ts = tk_xtime(tk);
+
+ *offs_real = tk->offs_real;
+ *offs_boot = tk->offs_boot;
+ *offs_tai = tk->offs_tai;
} while (read_seqcount_retry(&timekeeper_seq, seq));
+
+ now = ktime_set(ts.tv_sec, ts.tv_nsec);
+ now = ktime_sub(now, *offs_real);
+ return now;
}
#ifdef CONFIG_HIGH_RES_TIMERS
/**
- * ktime_get_update_offsets - hrtimer helper
+ * ktime_get_update_offsets_now - hrtimer helper
* @offs_real: pointer to storage for monotonic -> realtime offset
* @offs_boot: pointer to storage for monotonic -> boottime offset
* @offs_tai: pointer to storage for monotonic -> clock tai offset
@@ -1611,7 +1621,7 @@ void get_xtime_and_monotonic_and_sleep_o
* Returns current monotonic time and updates the offsets
* Called from hrtimer_interrupt() or retrigger_next_event()
*/
-ktime_t ktime_get_update_offsets(ktime_t *offs_real, ktime_t *offs_boot,
+ktime_t ktime_get_update_offsets_now(ktime_t *offs_real, ktime_t *offs_boot,
ktime_t *offs_tai)
{
struct timekeeper *tk = &timekeeper;
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 04/55] ktime: Kill non-scalar ktime_t implementation for 2038
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (2 preceding siblings ...)
2014-07-11 13:44 ` [patch 03/55] hrtimer: Cleanup hrtimer accessors to the timekepeing state Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 05/55] ktime: Sanitize ktime_to_us/ms conversion Thomas Gleixner
` (50 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: jstultz-tk-d63a235.patch --]
[-- Type: text/plain, Size: 12482 bytes --]
From: John Stultz <john.stultz@linaro.org>
The non-scalar ktime_t implementation is basically a timespec
which has to be changed to support dates past 2038 on 32bit
systems.
This patch removes the non-scalar ktime_t implementation, forcing
the scalar s64 nanosecond version on all architectures.
This may have additional performance overhead on some 32bit
systems when converting between ktime_t and timespec structures,
however the majority of 32bit systems (arm and i386) were already
using scalar ktime_t, so no performance regressions will be seen
on those platforms.
On affected platforms, I'm open to finding optimizations, including
avoiding converting to timespecs where possible.
[ tglx: We can now cleanup the ktime_t.tv64 mess, but thats a
different issue and we can throw a coccinelle script at it ]
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/arm/Kconfig | 1
arch/hexagon/Kconfig | 1
arch/s390/Kconfig | 1
arch/x86/Kconfig | 1
include/linux/ktime.h | 173 ----------------------------------------------
include/linux/time.h | 11 +-
kernel/time/Kconfig | 4 -
kernel/time/hrtimer.c | 54 --------------
kernel/time/timekeeping.c | 7 -
9 files changed, 7 insertions(+), 246 deletions(-)
Index: tip/arch/arm/Kconfig
===================================================================
--- tip.orig/arch/arm/Kconfig
+++ tip/arch/arm/Kconfig
@@ -64,7 +64,6 @@ config ARM
select HAVE_UID16
select HAVE_VIRT_CPU_ACCOUNTING_GEN
select IRQ_FORCED_THREADING
- select KTIME_SCALAR
select MODULES_USE_ELF_REL
select NO_BOOTMEM
select OLD_SIGACTION
Index: tip/arch/hexagon/Kconfig
===================================================================
--- tip.orig/arch/hexagon/Kconfig
+++ tip/arch/hexagon/Kconfig
@@ -23,7 +23,6 @@ config HEXAGON
select GENERIC_IOMAP
select GENERIC_SMP_IDLE_THREAD
select STACKTRACE_SUPPORT
- select KTIME_SCALAR
select GENERIC_CLOCKEVENTS
select GENERIC_CLOCKEVENTS_BROADCAST
select MODULES_USE_ELF_RELA
Index: tip/arch/s390/Kconfig
===================================================================
--- tip.orig/arch/s390/Kconfig
+++ tip/arch/s390/Kconfig
@@ -137,7 +137,6 @@ config S390
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UID16 if 32BIT
select HAVE_VIRT_CPU_ACCOUNTING
- select KTIME_SCALAR if 32BIT
select MODULES_USE_ELF_RELA
select NO_BOOTMEM
select OLD_SIGACTION
Index: tip/arch/x86/Kconfig
===================================================================
--- tip.orig/arch/x86/Kconfig
+++ tip/arch/x86/Kconfig
@@ -111,7 +111,6 @@ config X86
select ARCH_CLOCKSOURCE_DATA
select GENERIC_CLOCKEVENTS_BROADCAST if X86_64 || (X86_32 && X86_LOCAL_APIC)
select GENERIC_TIME_VSYSCALL
- select KTIME_SCALAR if X86_32
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select HAVE_CONTEXT_TRACKING if X86_64
Index: tip/include/linux/ktime.h
===================================================================
--- tip.orig/include/linux/ktime.h
+++ tip/include/linux/ktime.h
@@ -27,43 +27,19 @@
/*
* ktime_t:
*
- * On 64-bit CPUs a single 64-bit variable is used to store the hrtimers
+ * A single 64-bit variable is used to store the hrtimers
* internal representation of time values in scalar nanoseconds. The
* design plays out best on 64-bit CPUs, where most conversions are
* NOPs and most arithmetic ktime_t operations are plain arithmetic
* operations.
*
- * On 32-bit CPUs an optimized representation of the timespec structure
- * is used to avoid expensive conversions from and to timespecs. The
- * endian-aware order of the tv struct members is chosen to allow
- * mathematical operations on the tv64 member of the union too, which
- * for certain operations produces better code.
- *
- * For architectures with efficient support for 64/32-bit conversions the
- * plain scalar nanosecond based representation can be selected by the
- * config switch CONFIG_KTIME_SCALAR.
*/
union ktime {
s64 tv64;
-#if BITS_PER_LONG != 64 && !defined(CONFIG_KTIME_SCALAR)
- struct {
-# ifdef __BIG_ENDIAN
- s32 sec, nsec;
-# else
- s32 nsec, sec;
-# endif
- } tv;
-#endif
};
typedef union ktime ktime_t; /* Kill this */
-/*
- * ktime_t definitions when using the 64-bit scalar representation:
- */
-
-#if (BITS_PER_LONG == 64) || defined(CONFIG_KTIME_SCALAR)
-
/**
* ktime_set - Set a ktime_t variable from a seconds/nanoseconds value
* @secs: seconds to set
@@ -123,153 +99,6 @@ static inline ktime_t timeval_to_ktime(s
/* Convert ktime_t to nanoseconds - NOP in the scalar storage format: */
#define ktime_to_ns(kt) ((kt).tv64)
-#else /* !((BITS_PER_LONG == 64) || defined(CONFIG_KTIME_SCALAR)) */
-
-/*
- * Helper macros/inlines to get the ktime_t math right in the timespec
- * representation. The macros are sometimes ugly - their actual use is
- * pretty okay-ish, given the circumstances. We do all this for
- * performance reasons. The pure scalar nsec_t based code was nice and
- * simple, but created too many 64-bit / 32-bit conversions and divisions.
- *
- * Be especially aware that negative values are represented in a way
- * that the tv.sec field is negative and the tv.nsec field is greater
- * or equal to zero but less than nanoseconds per second. This is the
- * same representation which is used by timespecs.
- *
- * tv.sec < 0 and 0 >= tv.nsec < NSEC_PER_SEC
- */
-
-/* Set a ktime_t variable to a value in sec/nsec representation: */
-static inline ktime_t ktime_set(const long secs, const unsigned long nsecs)
-{
- return (ktime_t) { .tv = { .sec = secs, .nsec = nsecs } };
-}
-
-/**
- * ktime_sub - subtract two ktime_t variables
- * @lhs: minuend
- * @rhs: subtrahend
- *
- * Return: The remainder of the subtraction.
- */
-static inline ktime_t ktime_sub(const ktime_t lhs, const ktime_t rhs)
-{
- ktime_t res;
-
- res.tv64 = lhs.tv64 - rhs.tv64;
- if (res.tv.nsec < 0)
- res.tv.nsec += NSEC_PER_SEC;
-
- return res;
-}
-
-/**
- * ktime_add - add two ktime_t variables
- * @add1: addend1
- * @add2: addend2
- *
- * Return: The sum of @add1 and @add2.
- */
-static inline ktime_t ktime_add(const ktime_t add1, const ktime_t add2)
-{
- ktime_t res;
-
- res.tv64 = add1.tv64 + add2.tv64;
- /*
- * performance trick: the (u32) -NSEC gives 0x00000000Fxxxxxxx
- * so we subtract NSEC_PER_SEC and add 1 to the upper 32 bit.
- *
- * it's equivalent to:
- * tv.nsec -= NSEC_PER_SEC
- * tv.sec ++;
- */
- if (res.tv.nsec >= NSEC_PER_SEC)
- res.tv64 += (u32)-NSEC_PER_SEC;
-
- return res;
-}
-
-/**
- * ktime_add_ns - Add a scalar nanoseconds value to a ktime_t variable
- * @kt: addend
- * @nsec: the scalar nsec value to add
- *
- * Return: The sum of @kt and @nsec in ktime_t format.
- */
-extern ktime_t ktime_add_ns(const ktime_t kt, u64 nsec);
-
-/**
- * ktime_sub_ns - Subtract a scalar nanoseconds value from a ktime_t variable
- * @kt: minuend
- * @nsec: the scalar nsec value to subtract
- *
- * Return: The subtraction of @nsec from @kt in ktime_t format.
- */
-extern ktime_t ktime_sub_ns(const ktime_t kt, u64 nsec);
-
-/**
- * timespec_to_ktime - convert a timespec to ktime_t format
- * @ts: the timespec variable to convert
- *
- * Return: A ktime_t variable with the converted timespec value.
- */
-static inline ktime_t timespec_to_ktime(const struct timespec ts)
-{
- return (ktime_t) { .tv = { .sec = (s32)ts.tv_sec,
- .nsec = (s32)ts.tv_nsec } };
-}
-
-/**
- * timeval_to_ktime - convert a timeval to ktime_t format
- * @tv: the timeval variable to convert
- *
- * Return: A ktime_t variable with the converted timeval value.
- */
-static inline ktime_t timeval_to_ktime(const struct timeval tv)
-{
- return (ktime_t) { .tv = { .sec = (s32)tv.tv_sec,
- .nsec = (s32)(tv.tv_usec *
- NSEC_PER_USEC) } };
-}
-
-/**
- * ktime_to_timespec - convert a ktime_t variable to timespec format
- * @kt: the ktime_t variable to convert
- *
- * Return: The timespec representation of the ktime value.
- */
-static inline struct timespec ktime_to_timespec(const ktime_t kt)
-{
- return (struct timespec) { .tv_sec = (time_t) kt.tv.sec,
- .tv_nsec = (long) kt.tv.nsec };
-}
-
-/**
- * ktime_to_timeval - convert a ktime_t variable to timeval format
- * @kt: the ktime_t variable to convert
- *
- * Return: The timeval representation of the ktime value.
- */
-static inline struct timeval ktime_to_timeval(const ktime_t kt)
-{
- return (struct timeval) {
- .tv_sec = (time_t) kt.tv.sec,
- .tv_usec = (suseconds_t) (kt.tv.nsec / NSEC_PER_USEC) };
-}
-
-/**
- * ktime_to_ns - convert a ktime_t variable to scalar nanoseconds
- * @kt: the ktime_t variable to convert
- *
- * Return: The scalar nanoseconds representation of @kt.
- */
-static inline s64 ktime_to_ns(const ktime_t kt)
-{
- return (s64) kt.tv.sec * NSEC_PER_SEC + kt.tv.nsec;
-}
-
-#endif /* !((BITS_PER_LONG == 64) || defined(CONFIG_KTIME_SCALAR)) */
/**
* ktime_equal - Compares two ktime_t variables to see if they are equal
Index: tip/include/linux/time.h
===================================================================
--- tip.orig/include/linux/time.h
+++ tip/include/linux/time.h
@@ -19,6 +19,10 @@ extern struct timezone sys_tz;
#define TIME_T_MAX (time_t)((1UL << ((sizeof(time_t) << 3) - 1)) - 1)
+/* Located here for timespec_valid_strict */
+#define KTIME_MAX ((s64)~((u64)1 << 63))
+#define KTIME_SEC_MAX (KTIME_MAX / NSEC_PER_SEC)
+
static inline int timespec_equal(const struct timespec *a,
const struct timespec *b)
{
@@ -84,13 +88,6 @@ static inline struct timespec timespec_s
return ts_delta;
}
-#define KTIME_MAX ((s64)~((u64)1 << 63))
-#if (BITS_PER_LONG == 64)
-# define KTIME_SEC_MAX (KTIME_MAX / NSEC_PER_SEC)
-#else
-# define KTIME_SEC_MAX LONG_MAX
-#endif
-
/*
* Returns true if the timespec is norm, false if denorm:
*/
Index: tip/kernel/time/Kconfig
===================================================================
--- tip.orig/kernel/time/Kconfig
+++ tip/kernel/time/Kconfig
@@ -20,10 +20,6 @@ config GENERIC_TIME_VSYSCALL
config GENERIC_TIME_VSYSCALL_OLD
bool
-# ktime_t scalar 64bit nsec representation
-config KTIME_SCALAR
- bool
-
# Old style timekeeping
config ARCH_USES_GETTIMEOFFSET
bool
Index: tip/kernel/time/hrtimer.c
===================================================================
--- tip.orig/kernel/time/hrtimer.c
+++ tip/kernel/time/hrtimer.c
@@ -261,60 +261,6 @@ lock_hrtimer_base(const struct hrtimer *
* too large for inlining:
*/
#if BITS_PER_LONG < 64
-# ifndef CONFIG_KTIME_SCALAR
-/**
- * ktime_add_ns - Add a scalar nanoseconds value to a ktime_t variable
- * @kt: addend
- * @nsec: the scalar nsec value to add
- *
- * Returns the sum of kt and nsec in ktime_t format
- */
-ktime_t ktime_add_ns(const ktime_t kt, u64 nsec)
-{
- ktime_t tmp;
-
- if (likely(nsec < NSEC_PER_SEC)) {
- tmp.tv64 = nsec;
- } else {
- unsigned long rem = do_div(nsec, NSEC_PER_SEC);
-
- /* Make sure nsec fits into long */
- if (unlikely(nsec > KTIME_SEC_MAX))
- return (ktime_t){ .tv64 = KTIME_MAX };
-
- tmp = ktime_set((long)nsec, rem);
- }
-
- return ktime_add(kt, tmp);
-}
-
-EXPORT_SYMBOL_GPL(ktime_add_ns);
-
-/**
- * ktime_sub_ns - Subtract a scalar nanoseconds value from a ktime_t variable
- * @kt: minuend
- * @nsec: the scalar nsec value to subtract
- *
- * Returns the subtraction of @nsec from @kt in ktime_t format
- */
-ktime_t ktime_sub_ns(const ktime_t kt, u64 nsec)
-{
- ktime_t tmp;
-
- if (likely(nsec < NSEC_PER_SEC)) {
- tmp.tv64 = nsec;
- } else {
- unsigned long rem = do_div(nsec, NSEC_PER_SEC);
-
- tmp = ktime_set((long)nsec, rem);
- }
-
- return ktime_sub(kt, tmp);
-}
-
-EXPORT_SYMBOL_GPL(ktime_sub_ns);
-# endif /* !CONFIG_KTIME_SCALAR */
-
/*
* Divide a ktime value by a nanosecond value
*/
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -344,11 +344,8 @@ ktime_t ktime_get(void)
nsecs = timekeeping_get_ns(tk) + tk->wall_to_monotonic.tv_nsec;
} while (read_seqcount_retry(&timekeeper_seq, seq));
- /*
- * Use ktime_set/ktime_add_ns to create a proper ktime on
- * 32-bit architectures without CONFIG_KTIME_SCALAR.
- */
- return ktime_add_ns(ktime_set(secs, 0), nsecs);
+
+ return ktime_set(secs, nsecs);
}
EXPORT_SYMBOL_GPL(ktime_get);
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 05/55] ktime: Sanitize ktime_to_us/ms conversion
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (3 preceding siblings ...)
2014-07-11 13:44 ` [patch 04/55] ktime: Kill non-scalar ktime_t implementation for 2038 Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 06/55] ktime: Change ktime_set() to take 64bit seconds value Thomas Gleixner
` (49 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: ktime-sanitize-conversion.patch --]
[-- Type: text/plain, Size: 1878 bytes --]
With the plain nanoseconds based ktime_t we can simply use
ktime_divns() instead of going through loops and hoops of
timespec/timeval conversion.
Reported-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/hrtimer.h | 6 ------
include/linux/ktime.h | 12 ++++++++----
2 files changed, 8 insertions(+), 10 deletions(-)
Index: tip/include/linux/hrtimer.h
===================================================================
--- tip.orig/include/linux/hrtimer.h
+++ tip/include/linux/hrtimer.h
@@ -457,12 +457,6 @@ extern void hrtimer_run_pending(void);
/* Bootup initialization: */
extern void __init hrtimers_init(void);
-#if BITS_PER_LONG < 64
-extern u64 ktime_divns(const ktime_t kt, s64 div);
-#else /* BITS_PER_LONG < 64 */
-# define ktime_divns(kt, div) (u64)((kt).tv64 / (div))
-#endif
-
/* Show pending timers: */
extern void sysrq_timer_list_show(void);
Index: tip/include/linux/ktime.h
===================================================================
--- tip.orig/include/linux/ktime.h
+++ tip/include/linux/ktime.h
@@ -157,16 +157,20 @@ static inline bool ktime_before(const kt
return ktime_compare(cmp1, cmp2) < 0;
}
+#if BITS_PER_LONG < 64
+extern u64 ktime_divns(const ktime_t kt, s64 div);
+#else /* BITS_PER_LONG < 64 */
+# define ktime_divns(kt, div) (u64)((kt).tv64 / (div))
+#endif
+
static inline s64 ktime_to_us(const ktime_t kt)
{
- struct timeval tv = ktime_to_timeval(kt);
- return (s64) tv.tv_sec * USEC_PER_SEC + tv.tv_usec;
+ return ktime_divns(kt, NSEC_PER_USEC);
}
static inline s64 ktime_to_ms(const ktime_t kt)
{
- struct timeval tv = ktime_to_timeval(kt);
- return (s64) tv.tv_sec * MSEC_PER_SEC + tv.tv_usec / USEC_PER_MSEC;
+ return ktime_divns(kt, NSEC_PER_MSEC);
}
static inline s64 ktime_us_delta(const ktime_t later, const ktime_t earlier)
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 06/55] ktime: Change ktime_set() to take 64bit seconds value
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (4 preceding siblings ...)
2014-07-11 13:44 ` [patch 05/55] ktime: Sanitize ktime_to_us/ms conversion Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 07/55] time64: Add time64.h header and define struct timespec64 Thomas Gleixner
` (48 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: jstultz-tk-696511b.patch --]
[-- Type: text/plain, Size: 1148 bytes --]
From: John Stultz <john.stultz@linaro.org>
In order to support dates past 2038 on 32bit systems, ktime_set()
needs to handle 64bit second values.
[ tglx: Removed the BITS_PER_LONG check ]
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/ktime.h | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
Index: tip/include/linux/ktime.h
===================================================================
--- tip.orig/include/linux/ktime.h
+++ tip/include/linux/ktime.h
@@ -47,13 +47,12 @@ typedef union ktime ktime_t; /* Kill th
*
* Return: The ktime_t representation of the value.
*/
-static inline ktime_t ktime_set(const long secs, const unsigned long nsecs)
+static inline ktime_t ktime_set(const s64 secs, const unsigned long nsecs)
{
-#if (BITS_PER_LONG == 64)
if (unlikely(secs >= KTIME_SEC_MAX))
return (ktime_t){ .tv64 = KTIME_MAX };
-#endif
- return (ktime_t) { .tv64 = (s64)secs * NSEC_PER_SEC + (s64)nsecs };
+
+ return (ktime_t) { .tv64 = secs * NSEC_PER_SEC + (s64)nsecs };
}
/* Subtract two ktime_t variables. rem = lhs -rhs: */
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 07/55] time64: Add time64.h header and define struct timespec64
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (5 preceding siblings ...)
2014-07-11 13:44 ` [patch 06/55] ktime: Change ktime_set() to take 64bit seconds value Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 18:41 ` Richard Cochran
2014-07-11 13:44 ` [patch 08/55] time: More core infrastructure for timespec64 Thomas Gleixner
` (47 subsequent siblings)
54 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: jstultz-tk-d8e0c56.patch --]
[-- Type: text/plain, Size: 6131 bytes --]
From: John Stultz <john.stultz@linaro.org>
Define the timespec64 structure and standard helper functions.
[ tglx: Make it 32bit only. 64bit really can map timespec to timespec64 ]
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/time.h | 15 ----
include/linux/time64.h | 163 +++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 164 insertions(+), 14 deletions(-)
Index: tip/include/linux/time.h
===================================================================
--- tip.orig/include/linux/time.h
+++ tip/include/linux/time.h
@@ -4,25 +4,12 @@
# include <linux/cache.h>
# include <linux/seqlock.h>
# include <linux/math64.h>
-#include <uapi/linux/time.h>
+# include <linux/time64.h>
extern struct timezone sys_tz;
-/* Parameters used to convert the timespec values: */
-#define MSEC_PER_SEC 1000L
-#define USEC_PER_MSEC 1000L
-#define NSEC_PER_USEC 1000L
-#define NSEC_PER_MSEC 1000000L
-#define USEC_PER_SEC 1000000L
-#define NSEC_PER_SEC 1000000000L
-#define FSEC_PER_SEC 1000000000000000LL
-
#define TIME_T_MAX (time_t)((1UL << ((sizeof(time_t) << 3) - 1)) - 1)
-/* Located here for timespec_valid_strict */
-#define KTIME_MAX ((s64)~((u64)1 << 63))
-#define KTIME_SEC_MAX (KTIME_MAX / NSEC_PER_SEC)
-
static inline int timespec_equal(const struct timespec *a,
const struct timespec *b)
{
Index: tip/include/linux/time64.h
===================================================================
--- /dev/null
+++ tip/include/linux/time64.h
@@ -0,0 +1,163 @@
+#ifndef _LINUX_TIME64_H
+#define _LINUX_TIME64_H
+
+#include <uapi/linux/time.h>
+
+typedef __s64 time64_t;
+
+/*
+ * This wants to go into uapi/linux/time.h once we agreed about the
+ * userspace interfaces.
+ */
+#if __BITS_PER_LONG == 64
+# define timespec64 timespec
+#else
+struct timespec64 {
+ time64_t tv_sec; /* seconds */
+ long tv_nsec; /* nanoseconds */
+};
+#endif
+
+/* Parameters used to convert the timespec values: */
+#define MSEC_PER_SEC 1000L
+#define USEC_PER_MSEC 1000L
+#define NSEC_PER_USEC 1000L
+#define NSEC_PER_MSEC 1000000L
+#define USEC_PER_SEC 1000000L
+#define NSEC_PER_SEC 1000000000L
+#define FSEC_PER_SEC 1000000000000000LL
+
+/* Located here for timespec[64]_valid_strict */
+#define KTIME_MAX ((s64)~((u64)1 << 63))
+#define KTIME_SEC_MAX (KTIME_MAX / NSEC_PER_SEC)
+
+#if __BITS_PER_LONG == 64
+
+# define timespec64_equal timespec_equal
+# define timespec64_compare timespec_compare
+# define set_normalized_timespec64 set_normalized_timespec
+# define timespec64_add_safe timespec_add_safe
+# define timespec64_add timespec_add
+# define timespec64_sub timespec_sub
+# define timespec64_valid timespec_valid
+# define timespec64_valid_strict timespec_valid_strict
+# define timespec64_to_ns timespec_to_ns
+# define ns_to_timespec64 ns_to_timespec
+# define timespec64_add_ns timespec_add_ns
+
+#else
+
+static inline int timespec64_equal(const struct timespec64 *a,
+ const struct timespec64 *b)
+{
+ return (a->tv_sec == b->tv_sec) && (a->tv_nsec == b->tv_nsec);
+}
+
+/*
+ * lhs < rhs: return <0
+ * lhs == rhs: return 0
+ * lhs > rhs: return >0
+ */
+static inline int timespec64_compare(const struct timespec64 *lhs, const struct timespec64 *rhs)
+{
+ if (lhs->tv_sec < rhs->tv_sec)
+ return -1;
+ if (lhs->tv_sec > rhs->tv_sec)
+ return 1;
+ return lhs->tv_nsec - rhs->tv_nsec;
+}
+
+extern void set_normalized_timespec64(struct timespec64 *ts, time64_t sec, s64 nsec);
+
+/*
+ * timespec64_add_safe assumes both values are positive and checks
+ * for overflow. It will return TIME_T_MAX if the reutrn would be
+ * smaller then either of the arguments.
+ */
+extern struct timespec64 timespec64_add_safe(const struct timespec64 lhs,
+ const struct timespec64 rhs);
+
+
+static inline struct timespec64 timespec64_add(struct timespec64 lhs,
+ struct timespec64 rhs)
+{
+ struct timespec64 ts_delta;
+ set_normalized_timespec64(&ts_delta, lhs.tv_sec + rhs.tv_sec,
+ lhs.tv_nsec + rhs.tv_nsec);
+ return ts_delta;
+}
+
+/*
+ * sub = lhs - rhs, in normalized form
+ */
+static inline struct timespec64 timespec64_sub(struct timespec64 lhs,
+ struct timespec64 rhs)
+{
+ struct timespec64 ts_delta;
+ set_normalized_timespec64(&ts_delta, lhs.tv_sec - rhs.tv_sec,
+ lhs.tv_nsec - rhs.tv_nsec);
+ return ts_delta;
+}
+
+/*
+ * Returns true if the timespec64 is norm, false if denorm:
+ */
+static inline bool timespec64_valid(const struct timespec64 *ts)
+{
+ /* Dates before 1970 are bogus */
+ if (ts->tv_sec < 0)
+ return false;
+ /* Can't have more nanoseconds then a second */
+ if ((unsigned long)ts->tv_nsec >= NSEC_PER_SEC)
+ return false;
+ return true;
+}
+
+static inline bool timespec64_valid_strict(const struct timespec64 *ts)
+{
+ if (!timespec64_valid(ts))
+ return false;
+ /* Disallow values that could overflow ktime_t */
+ if ((unsigned long long)ts->tv_sec >= KTIME_SEC_MAX)
+ return false;
+ return true;
+}
+
+/**
+ * timespec64_to_ns - Convert timespec64 to nanoseconds
+ * @ts: pointer to the timespec64 variable to be converted
+ *
+ * Returns the scalar nanosecond representation of the timespec64
+ * parameter.
+ */
+static inline s64 timespec64_to_ns(const struct timespec64 *ts)
+{
+ return ((s64) ts->tv_sec * NSEC_PER_SEC) + ts->tv_nsec;
+}
+
+/**
+ * ns_to_timespec64 - Convert nanoseconds to timespec64
+ * @nsec: the nanoseconds value to be converted
+ *
+ * Returns the timespec64 representation of the nsec parameter.
+ */
+extern struct timespec64 ns_to_timespec64(const s64 nsec);
+
+/**
+ * timespec64_add_ns - Adds nanoseconds to a timespec64
+ * @a: pointer to timespec64 to be incremented
+ * @ns: unsigned nanoseconds value to be added
+ *
+ * This must always be inlined because its used from the x86-64 vdso,
+ * which cannot call other kernel functions.
+ */
+static __always_inline void timespec64_add_ns(struct timespec64 *a, u64 ns)
+{
+ a->tv_sec += __iter_div_u64_rem(a->tv_nsec + ns, NSEC_PER_SEC, &ns);
+ a->tv_nsec = ns;
+}
+
+#endif
+
+#endif /* _LINUX_TIME64_H */
+
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 08/55] time: More core infrastructure for timespec64
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (6 preceding siblings ...)
2014-07-11 13:44 ` [patch 07/55] time64: Add time64.h header and define struct timespec64 Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 09/55] timekeeping: Convert timekeeping core to use timespec64s Thomas Gleixner
` (46 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: jstultz-tk-99c585d.patch --]
[-- Type: text/plain, Size: 5114 bytes --]
From: John Stultz <john.stultz@linaro.org>
Helper and conversion functions for timespec64.
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/ktime.h | 28 ++++++++++++++++++++++
include/linux/time64.h | 28 ++++++++++++++++++++++
kernel/time/time.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 118 insertions(+)
Index: tip/include/linux/ktime.h
===================================================================
--- tip.orig/include/linux/ktime.h
+++ tip/include/linux/ktime.h
@@ -83,6 +83,12 @@ static inline ktime_t timespec_to_ktime(
return ktime_set(ts.tv_sec, ts.tv_nsec);
}
+/* convert a timespec64 to ktime_t format: */
+static inline ktime_t timespec64_to_ktime(struct timespec64 ts)
+{
+ return ktime_set(ts.tv_sec, ts.tv_nsec);
+}
+
/* convert a timeval to ktime_t format: */
static inline ktime_t timeval_to_ktime(struct timeval tv)
{
@@ -92,6 +98,9 @@ static inline ktime_t timeval_to_ktime(s
/* Map the ktime_t to timespec conversion to ns_to_timespec function */
#define ktime_to_timespec(kt) ns_to_timespec((kt).tv64)
+/* Map the ktime_t to timespec conversion to ns_to_timespec function */
+#define ktime_to_timespec64(kt) ns_to_timespec64((kt).tv64)
+
/* Map the ktime_t to timeval conversion to ns_to_timeval function */
#define ktime_to_timeval(kt) ns_to_timeval((kt).tv64)
@@ -186,6 +195,25 @@ static inline __must_check bool ktime_to
return true;
} else {
return false;
+ }
+}
+
+/**
+ * ktime_to_timespec64_cond - convert a ktime_t variable to timespec64
+ * format only if the variable contains data
+ * @kt: the ktime_t variable to convert
+ * @ts: the timespec variable to store the result in
+ *
+ * Return: %true if there was a successful conversion, %false if kt was 0.
+ */
+static inline __must_check bool ktime_to_timespec64_cond(const ktime_t kt,
+ struct timespec64 *ts)
+{
+ if (kt.tv64) {
+ *ts = ktime_to_timespec64(kt);
+ return true;
+ } else {
+ return false;
}
}
Index: tip/include/linux/time64.h
===================================================================
--- tip.orig/include/linux/time64.h
+++ tip/include/linux/time64.h
@@ -33,6 +33,16 @@ struct timespec64 {
#if __BITS_PER_LONG == 64
+static inline struct timespec timespec64_to_timespec(const struct timespec64 ts64)
+{
+ return ts64;
+}
+
+static inline struct timespec64 timespec_to_timespec64(const struct timespec ts)
+{
+ return ts;
+}
+
# define timespec64_equal timespec_equal
# define timespec64_compare timespec_compare
# define set_normalized_timespec64 set_normalized_timespec
@@ -47,6 +57,24 @@ struct timespec64 {
#else
+static inline struct timespec timespec64_to_timespec(const struct timespec64 ts64)
+{
+ struct timespec ret;
+
+ ret.tv_sec = (time_t)ts64.tv_sec;
+ ret.tv_nsec = ts64.tv_nsec;
+ return ret;
+}
+
+static inline struct timespec64 timespec_to_timespec64(const struct timespec ts)
+{
+ struct timespec64 ret;
+
+ ret.tv_sec = ts.tv_sec;
+ ret.tv_nsec = ts.tv_nsec;
+ return ret;
+}
+
static inline int timespec64_equal(const struct timespec64 *a,
const struct timespec64 *b)
{
Index: tip/kernel/time/time.c
===================================================================
--- tip.orig/kernel/time/time.c
+++ tip/kernel/time/time.c
@@ -420,6 +420,68 @@ struct timeval ns_to_timeval(const s64 n
}
EXPORT_SYMBOL(ns_to_timeval);
+#if BITS_PER_LONG == 32
+/**
+ * set_normalized_timespec - set timespec sec and nsec parts and normalize
+ *
+ * @ts: pointer to timespec variable to be set
+ * @sec: seconds to set
+ * @nsec: nanoseconds to set
+ *
+ * Set seconds and nanoseconds field of a timespec variable and
+ * normalize to the timespec storage format
+ *
+ * Note: The tv_nsec part is always in the range of
+ * 0 <= tv_nsec < NSEC_PER_SEC
+ * For negative values only the tv_sec field is negative !
+ */
+void set_normalized_timespec64(struct timespec64 *ts, time64_t sec, s64 nsec)
+{
+ while (nsec >= NSEC_PER_SEC) {
+ /*
+ * The following asm() prevents the compiler from
+ * optimising this loop into a modulo operation. See
+ * also __iter_div_u64_rem() in include/linux/time.h
+ */
+ asm("" : "+rm"(nsec));
+ nsec -= NSEC_PER_SEC;
+ ++sec;
+ }
+ while (nsec < 0) {
+ asm("" : "+rm"(nsec));
+ nsec += NSEC_PER_SEC;
+ --sec;
+ }
+ ts->tv_sec = sec;
+ ts->tv_nsec = nsec;
+}
+EXPORT_SYMBOL(set_normalized_timespec64);
+
+/**
+ * ns_to_timespec64 - Convert nanoseconds to timespec64
+ * @nsec: the nanoseconds value to be converted
+ *
+ * Returns the timespec64 representation of the nsec parameter.
+ */
+struct timespec64 ns_to_timespec64(const s64 nsec)
+{
+ struct timespec64 ts;
+ s32 rem;
+
+ if (!nsec)
+ return (struct timespec64) {0, 0};
+
+ ts.tv_sec = div_s64_rem(nsec, NSEC_PER_SEC, &rem);
+ if (unlikely(rem < 0)) {
+ ts.tv_sec--;
+ rem += NSEC_PER_SEC;
+ }
+ ts.tv_nsec = rem;
+
+ return ts;
+}
+EXPORT_SYMBOL(ns_to_timespec64);
+#endif
/*
* When we convert to jiffies then we interpret incoming values
* the following way:
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 09/55] timekeeping: Convert timekeeping core to use timespec64s
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (7 preceding siblings ...)
2014-07-11 13:44 ` [patch 08/55] time: More core infrastructure for timespec64 Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 10/55] time: Consolidate the time accessor prototypes Thomas Gleixner
` (45 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: jstultz-tk-480e2d6.patch --]
[-- Type: text/plain, Size: 21065 bytes --]
From: John Stultz <john.stultz@linaro.org>
Convert the core timekeeping logic to use timespec64s. This moves the
2038 issues out of the core logic and into all of the accessor
functions.
Future changes will need to push the timespec64s out to all
timekeeping users, but that can be done interface by interface.
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeper_internal.h | 10 +-
kernel/time/ntp.c | 8 -
kernel/time/ntp_internal.h | 2
kernel/time/timekeeping.c | 172 ++++++++++++++++++++----------------
kernel/time/timekeeping_debug.c | 2
kernel/time/timekeeping_internal.h | 2
6 files changed, 109 insertions(+), 87 deletions(-)
Index: tip/include/linux/timekeeper_internal.h
===================================================================
--- tip.orig/include/linux/timekeeper_internal.h
+++ tip/include/linux/timekeeper_internal.h
@@ -55,15 +55,15 @@ struct timekeeper {
* - wall_to_monotonic is no longer the boot time, getboottime must be
* used instead.
*/
- struct timespec wall_to_monotonic;
+ struct timespec64 wall_to_monotonic;
/* Offset clock monotonic -> clock realtime */
ktime_t offs_real;
/* time spent in suspend */
- struct timespec total_sleep_time;
+ struct timespec64 total_sleep_time;
/* Offset clock monotonic -> clock boottime */
ktime_t offs_boot;
/* The raw monotonic time for the CLOCK_MONOTONIC_RAW posix clock. */
- struct timespec raw_time;
+ struct timespec64 raw_time;
/* The current UTC to TAI offset in seconds */
s32 tai_offset;
/* Offset clock monotonic -> clock tai */
@@ -71,9 +71,9 @@ struct timekeeper {
};
-static inline struct timespec tk_xtime(struct timekeeper *tk)
+static inline struct timespec64 tk_xtime(struct timekeeper *tk)
{
- struct timespec ts;
+ struct timespec64 ts;
ts.tv_sec = tk->xtime_sec;
ts.tv_nsec = (long)(tk->xtime_nsec >> tk->shift);
Index: tip/kernel/time/ntp.c
===================================================================
--- tip.orig/kernel/time/ntp.c
+++ tip/kernel/time/ntp.c
@@ -531,7 +531,7 @@ void ntp_notify_cmos_timer(void) { }
/*
* Propagate a new txc->status value into the NTP state:
*/
-static inline void process_adj_status(struct timex *txc, struct timespec *ts)
+static inline void process_adj_status(struct timex *txc, struct timespec64 *ts)
{
if ((time_status & STA_PLL) && !(txc->status & STA_PLL)) {
time_state = TIME_OK;
@@ -554,7 +554,7 @@ static inline void process_adj_status(st
static inline void process_adjtimex_modes(struct timex *txc,
- struct timespec *ts,
+ struct timespec64 *ts,
s32 *time_tai)
{
if (txc->modes & ADJ_STATUS)
@@ -640,7 +640,7 @@ int ntp_validate_timex(struct timex *txc
* adjtimex mainly allows reading (and writing, if superuser) of
* kernel time-keeping variables. used by xntpd.
*/
-int __do_adjtimex(struct timex *txc, struct timespec *ts, s32 *time_tai)
+int __do_adjtimex(struct timex *txc, struct timespec64 *ts, s32 *time_tai)
{
int result;
@@ -684,7 +684,7 @@ int __do_adjtimex(struct timex *txc, str
/* fill PPS status fields */
pps_fill_timex(txc);
- txc->time.tv_sec = ts->tv_sec;
+ txc->time.tv_sec = (time_t)ts->tv_sec;
txc->time.tv_usec = ts->tv_nsec;
if (!(time_status & STA_NANO))
txc->time.tv_usec /= NSEC_PER_USEC;
Index: tip/kernel/time/ntp_internal.h
===================================================================
--- tip.orig/kernel/time/ntp_internal.h
+++ tip/kernel/time/ntp_internal.h
@@ -7,6 +7,6 @@ extern void ntp_clear(void);
extern u64 ntp_tick_length(void);
extern int second_overflow(unsigned long secs);
extern int ntp_validate_timex(struct timex *);
-extern int __do_adjtimex(struct timex *, struct timespec *, s32 *);
+extern int __do_adjtimex(struct timex *, struct timespec64 *, s32 *);
extern void __hardpps(const struct timespec *, const struct timespec *);
#endif /* _LINUX_NTP_INTERNAL_H */
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -51,43 +51,43 @@ static inline void tk_normalize_xtime(st
}
}
-static void tk_set_xtime(struct timekeeper *tk, const struct timespec *ts)
+static void tk_set_xtime(struct timekeeper *tk, const struct timespec64 *ts)
{
tk->xtime_sec = ts->tv_sec;
tk->xtime_nsec = (u64)ts->tv_nsec << tk->shift;
}
-static void tk_xtime_add(struct timekeeper *tk, const struct timespec *ts)
+static void tk_xtime_add(struct timekeeper *tk, const struct timespec64 *ts)
{
tk->xtime_sec += ts->tv_sec;
tk->xtime_nsec += (u64)ts->tv_nsec << tk->shift;
tk_normalize_xtime(tk);
}
-static void tk_set_wall_to_mono(struct timekeeper *tk, struct timespec wtm)
+static void tk_set_wall_to_mono(struct timekeeper *tk, struct timespec64 wtm)
{
- struct timespec tmp;
+ struct timespec64 tmp;
/*
* Verify consistency of: offset_real = -wall_to_monotonic
* before modifying anything
*/
- set_normalized_timespec(&tmp, -tk->wall_to_monotonic.tv_sec,
+ set_normalized_timespec64(&tmp, -tk->wall_to_monotonic.tv_sec,
-tk->wall_to_monotonic.tv_nsec);
- WARN_ON_ONCE(tk->offs_real.tv64 != timespec_to_ktime(tmp).tv64);
+ WARN_ON_ONCE(tk->offs_real.tv64 != timespec64_to_ktime(tmp).tv64);
tk->wall_to_monotonic = wtm;
- set_normalized_timespec(&tmp, -wtm.tv_sec, -wtm.tv_nsec);
- tk->offs_real = timespec_to_ktime(tmp);
+ set_normalized_timespec64(&tmp, -wtm.tv_sec, -wtm.tv_nsec);
+ tk->offs_real = timespec64_to_ktime(tmp);
tk->offs_tai = ktime_add(tk->offs_real, ktime_set(tk->tai_offset, 0));
}
-static void tk_set_sleep_time(struct timekeeper *tk, struct timespec t)
+static void tk_set_sleep_time(struct timekeeper *tk, struct timespec64 t)
{
/* Verify consistency before modifying */
- WARN_ON_ONCE(tk->offs_boot.tv64 != timespec_to_ktime(tk->total_sleep_time).tv64);
+ WARN_ON_ONCE(tk->offs_boot.tv64 != timespec64_to_ktime(tk->total_sleep_time).tv64);
tk->total_sleep_time = t;
- tk->offs_boot = timespec_to_ktime(t);
+ tk->offs_boot = timespec64_to_ktime(t);
}
/**
@@ -281,7 +281,7 @@ static void timekeeping_forward_now(stru
tk_normalize_xtime(tk);
nsec = clocksource_cyc2ns(cycle_delta, clock->mult, clock->shift);
- timespec_add_ns(&tk->raw_time, nsec);
+ timespec64_add_ns(&tk->raw_time, nsec);
}
/**
@@ -360,7 +360,7 @@ EXPORT_SYMBOL_GPL(ktime_get);
void ktime_get_ts(struct timespec *ts)
{
struct timekeeper *tk = &timekeeper;
- struct timespec tomono;
+ struct timespec64 ts64, tomono;
s64 nsec;
unsigned int seq;
@@ -368,15 +368,16 @@ void ktime_get_ts(struct timespec *ts)
do {
seq = read_seqcount_begin(&timekeeper_seq);
- ts->tv_sec = tk->xtime_sec;
+ ts64.tv_sec = tk->xtime_sec;
nsec = timekeeping_get_ns(tk);
tomono = tk->wall_to_monotonic;
} while (read_seqcount_retry(&timekeeper_seq, seq));
- ts->tv_sec += tomono.tv_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsec + tomono.tv_nsec);
+ ts64.tv_sec += tomono.tv_sec;
+ ts64.tv_nsec = 0;
+ timespec64_add_ns(&ts64, nsec + tomono.tv_nsec);
+ *ts = timespec64_to_timespec(ts64);
}
EXPORT_SYMBOL_GPL(ktime_get_ts);
@@ -390,6 +391,7 @@ EXPORT_SYMBOL_GPL(ktime_get_ts);
void timekeeping_clocktai(struct timespec *ts)
{
struct timekeeper *tk = &timekeeper;
+ struct timespec64 ts64;
unsigned long seq;
u64 nsecs;
@@ -398,13 +400,14 @@ void timekeeping_clocktai(struct timespe
do {
seq = read_seqcount_begin(&timekeeper_seq);
- ts->tv_sec = tk->xtime_sec + tk->tai_offset;
+ ts64.tv_sec = tk->xtime_sec + tk->tai_offset;
nsecs = timekeeping_get_ns(tk);
} while (read_seqcount_retry(&timekeeper_seq, seq));
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs);
+ ts64.tv_nsec = 0;
+ timespec64_add_ns(&ts64, nsecs);
+ *ts = timespec64_to_timespec(ts64);
}
EXPORT_SYMBOL(timekeeping_clocktai);
@@ -446,7 +449,7 @@ void getnstime_raw_and_real(struct times
do {
seq = read_seqcount_begin(&timekeeper_seq);
- *ts_raw = tk->raw_time;
+ *ts_raw = timespec64_to_timespec(tk->raw_time);
ts_real->tv_sec = tk->xtime_sec;
ts_real->tv_nsec = 0;
@@ -487,7 +490,7 @@ EXPORT_SYMBOL(do_gettimeofday);
int do_settimeofday(const struct timespec *tv)
{
struct timekeeper *tk = &timekeeper;
- struct timespec ts_delta, xt;
+ struct timespec64 ts_delta, xt, tmp;
unsigned long flags;
if (!timespec_valid_strict(tv))
@@ -502,9 +505,10 @@ int do_settimeofday(const struct timespe
ts_delta.tv_sec = tv->tv_sec - xt.tv_sec;
ts_delta.tv_nsec = tv->tv_nsec - xt.tv_nsec;
- tk_set_wall_to_mono(tk, timespec_sub(tk->wall_to_monotonic, ts_delta));
+ tk_set_wall_to_mono(tk, timespec64_sub(tk->wall_to_monotonic, ts_delta));
- tk_set_xtime(tk, tv);
+ tmp = timespec_to_timespec64(*tv);
+ tk_set_xtime(tk, &tmp);
timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET);
@@ -528,26 +532,28 @@ int timekeeping_inject_offset(struct tim
{
struct timekeeper *tk = &timekeeper;
unsigned long flags;
- struct timespec tmp;
+ struct timespec64 ts64, tmp;
int ret = 0;
if ((unsigned long)ts->tv_nsec >= NSEC_PER_SEC)
return -EINVAL;
+ ts64 = timespec_to_timespec64(*ts);
+
raw_spin_lock_irqsave(&timekeeper_lock, flags);
write_seqcount_begin(&timekeeper_seq);
timekeeping_forward_now(tk);
/* Make sure the proposed value is valid */
- tmp = timespec_add(tk_xtime(tk), *ts);
- if (!timespec_valid_strict(&tmp)) {
+ tmp = timespec64_add(tk_xtime(tk), ts64);
+ if (!timespec64_valid_strict(&tmp)) {
ret = -EINVAL;
goto error;
}
- tk_xtime_add(tk, ts);
- tk_set_wall_to_mono(tk, timespec_sub(tk->wall_to_monotonic, *ts));
+ tk_xtime_add(tk, &ts64);
+ tk_set_wall_to_mono(tk, timespec64_sub(tk->wall_to_monotonic, ts64));
error: /* even if we error out, we forwarded the time, so call update */
timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET);
@@ -691,17 +697,19 @@ EXPORT_SYMBOL_GPL(ktime_get_real);
void getrawmonotonic(struct timespec *ts)
{
struct timekeeper *tk = &timekeeper;
+ struct timespec64 ts64;
unsigned long seq;
s64 nsecs;
do {
seq = read_seqcount_begin(&timekeeper_seq);
nsecs = timekeeping_get_ns_raw(tk);
- *ts = tk->raw_time;
+ ts64 = tk->raw_time;
} while (read_seqcount_retry(&timekeeper_seq, seq));
- timespec_add_ns(ts, nsecs);
+ timespec64_add_ns(&ts64, nsecs);
+ *ts = timespec64_to_timespec(ts64);
}
EXPORT_SYMBOL(getrawmonotonic);
@@ -781,11 +789,12 @@ void __init timekeeping_init(void)
struct timekeeper *tk = &timekeeper;
struct clocksource *clock;
unsigned long flags;
- struct timespec now, boot, tmp;
-
- read_persistent_clock(&now);
+ struct timespec64 now, boot, tmp;
+ struct timespec ts;
- if (!timespec_valid_strict(&now)) {
+ read_persistent_clock(&ts);
+ now = timespec_to_timespec64(ts);
+ if (!timespec64_valid_strict(&now)) {
pr_warn("WARNING: Persistent clock returned invalid value!\n"
" Check your CMOS/BIOS settings.\n");
now.tv_sec = 0;
@@ -793,8 +802,9 @@ void __init timekeeping_init(void)
} else if (now.tv_sec || now.tv_nsec)
persistent_clock_exist = true;
- read_boot_clock(&boot);
- if (!timespec_valid_strict(&boot)) {
+ read_boot_clock(&ts);
+ boot = timespec_to_timespec64(ts);
+ if (!timespec64_valid_strict(&boot)) {
pr_warn("WARNING: Boot clock returned invalid value!\n"
" Check your CMOS/BIOS settings.\n");
boot.tv_sec = 0;
@@ -816,7 +826,7 @@ void __init timekeeping_init(void)
if (boot.tv_sec == 0 && boot.tv_nsec == 0)
boot = tk_xtime(tk);
- set_normalized_timespec(&tmp, -boot.tv_sec, -boot.tv_nsec);
+ set_normalized_timespec64(&tmp, -boot.tv_sec, -boot.tv_nsec);
tk_set_wall_to_mono(tk, tmp);
tmp.tv_sec = 0;
@@ -830,7 +840,7 @@ void __init timekeeping_init(void)
}
/* time in seconds when suspend began */
-static struct timespec timekeeping_suspend_time;
+static struct timespec64 timekeeping_suspend_time;
/**
* __timekeeping_inject_sleeptime - Internal function to add sleep interval
@@ -840,17 +850,17 @@ static struct timespec timekeeping_suspe
* adds the sleep offset to the timekeeping variables.
*/
static void __timekeeping_inject_sleeptime(struct timekeeper *tk,
- struct timespec *delta)
+ struct timespec64 *delta)
{
- if (!timespec_valid_strict(delta)) {
+ if (!timespec64_valid_strict(delta)) {
printk_deferred(KERN_WARNING
"__timekeeping_inject_sleeptime: Invalid "
"sleep delta value!\n");
return;
}
tk_xtime_add(tk, delta);
- tk_set_wall_to_mono(tk, timespec_sub(tk->wall_to_monotonic, *delta));
- tk_set_sleep_time(tk, timespec_add(tk->total_sleep_time, *delta));
+ tk_set_wall_to_mono(tk, timespec64_sub(tk->wall_to_monotonic, *delta));
+ tk_set_sleep_time(tk, timespec64_add(tk->total_sleep_time, *delta));
tk_debug_account_sleep_time(delta);
}
@@ -867,6 +877,7 @@ static void __timekeeping_inject_sleepti
void timekeeping_inject_sleeptime(struct timespec *delta)
{
struct timekeeper *tk = &timekeeper;
+ struct timespec64 tmp;
unsigned long flags;
/*
@@ -881,7 +892,8 @@ void timekeeping_inject_sleeptime(struct
timekeeping_forward_now(tk);
- __timekeeping_inject_sleeptime(tk, delta);
+ tmp = timespec_to_timespec64(*delta);
+ __timekeeping_inject_sleeptime(tk, &tmp);
timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET);
@@ -904,11 +916,13 @@ static void timekeeping_resume(void)
struct timekeeper *tk = &timekeeper;
struct clocksource *clock = tk->clock;
unsigned long flags;
- struct timespec ts_new, ts_delta;
+ struct timespec64 ts_new, ts_delta;
+ struct timespec tmp;
cycle_t cycle_now, cycle_delta;
bool suspendtime_found = false;
- read_persistent_clock(&ts_new);
+ read_persistent_clock(&tmp);
+ ts_new = timespec_to_timespec64(tmp);
clockevents_resume();
clocksource_resume();
@@ -951,10 +965,10 @@ static void timekeeping_resume(void)
}
nsec += ((u64) cycle_delta * mult) >> shift;
- ts_delta = ns_to_timespec(nsec);
+ ts_delta = ns_to_timespec64(nsec);
suspendtime_found = true;
- } else if (timespec_compare(&ts_new, &timekeeping_suspend_time) > 0) {
- ts_delta = timespec_sub(ts_new, timekeeping_suspend_time);
+ } else if (timespec64_compare(&ts_new, &timekeeping_suspend_time) > 0) {
+ ts_delta = timespec64_sub(ts_new, timekeeping_suspend_time);
suspendtime_found = true;
}
@@ -981,10 +995,12 @@ static int timekeeping_suspend(void)
{
struct timekeeper *tk = &timekeeper;
unsigned long flags;
- struct timespec delta, delta_delta;
- static struct timespec old_delta;
+ struct timespec64 delta, delta_delta;
+ static struct timespec64 old_delta;
+ struct timespec tmp;
- read_persistent_clock(&timekeeping_suspend_time);
+ read_persistent_clock(&tmp);
+ timekeeping_suspend_time = timespec_to_timespec64(tmp);
/*
* On some systems the persistent_clock can not be detected at
@@ -1005,8 +1021,8 @@ static int timekeeping_suspend(void)
* try to compensate so the difference in system time
* and persistent_clock time stays close to constant.
*/
- delta = timespec_sub(tk_xtime(tk), timekeeping_suspend_time);
- delta_delta = timespec_sub(delta, old_delta);
+ delta = timespec64_sub(tk_xtime(tk), timekeeping_suspend_time);
+ delta_delta = timespec64_sub(delta, old_delta);
if (abs(delta_delta.tv_sec) >= 2) {
/*
* if delta_delta is too large, assume time correction
@@ -1016,7 +1032,7 @@ static int timekeeping_suspend(void)
} else {
/* Otherwise try to adjust old_system to compensate */
timekeeping_suspend_time =
- timespec_add(timekeeping_suspend_time, delta_delta);
+ timespec64_add(timekeeping_suspend_time, delta_delta);
}
timekeeping_update(tk, TK_MIRROR);
@@ -1253,14 +1269,14 @@ static inline unsigned int accumulate_ns
/* Figure out if its a leap sec and apply if needed */
leap = second_overflow(tk->xtime_sec);
if (unlikely(leap)) {
- struct timespec ts;
+ struct timespec64 ts;
tk->xtime_sec += leap;
ts.tv_sec = leap;
ts.tv_nsec = 0;
tk_set_wall_to_mono(tk,
- timespec_sub(tk->wall_to_monotonic, ts));
+ timespec64_sub(tk->wall_to_monotonic, ts));
__timekeeping_set_tai_offset(tk, tk->tai_offset - leap);
@@ -1469,7 +1485,7 @@ EXPORT_SYMBOL_GPL(getboottime);
void get_monotonic_boottime(struct timespec *ts)
{
struct timekeeper *tk = &timekeeper;
- struct timespec tomono, sleep;
+ struct timespec64 tomono, sleep, ret;
s64 nsec;
unsigned int seq;
@@ -1477,16 +1493,17 @@ void get_monotonic_boottime(struct times
do {
seq = read_seqcount_begin(&timekeeper_seq);
- ts->tv_sec = tk->xtime_sec;
+ ret.tv_sec = tk->xtime_sec;
nsec = timekeeping_get_ns(tk);
tomono = tk->wall_to_monotonic;
sleep = tk->total_sleep_time;
} while (read_seqcount_retry(&timekeeper_seq, seq));
- ts->tv_sec += tomono.tv_sec + sleep.tv_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsec + tomono.tv_nsec + sleep.tv_nsec);
+ ret.tv_sec += tomono.tv_sec + sleep.tv_sec;
+ ret.tv_nsec = 0;
+ timespec64_add_ns(&ret, nsec + tomono.tv_nsec + sleep.tv_nsec);
+ *ts = timespec64_to_timespec(ret);
}
EXPORT_SYMBOL_GPL(get_monotonic_boottime);
@@ -1514,8 +1531,11 @@ EXPORT_SYMBOL_GPL(ktime_get_boottime);
void monotonic_to_bootbased(struct timespec *ts)
{
struct timekeeper *tk = &timekeeper;
+ struct timespec64 ts64;
- *ts = timespec_add(*ts, tk->total_sleep_time);
+ ts64 = timespec_to_timespec64(*ts);
+ ts64 = timespec64_add(ts64, tk->total_sleep_time);
+ *ts = timespec64_to_timespec(ts64);
}
EXPORT_SYMBOL_GPL(monotonic_to_bootbased);
@@ -1531,13 +1551,13 @@ struct timespec __current_kernel_time(vo
{
struct timekeeper *tk = &timekeeper;
- return tk_xtime(tk);
+ return timespec64_to_timespec(tk_xtime(tk));
}
struct timespec current_kernel_time(void)
{
struct timekeeper *tk = &timekeeper;
- struct timespec now;
+ struct timespec64 now;
unsigned long seq;
do {
@@ -1546,14 +1566,14 @@ struct timespec current_kernel_time(void
now = tk_xtime(tk);
} while (read_seqcount_retry(&timekeeper_seq, seq));
- return now;
+ return timespec64_to_timespec(now);
}
EXPORT_SYMBOL(current_kernel_time);
struct timespec get_monotonic_coarse(void)
{
struct timekeeper *tk = &timekeeper;
- struct timespec now, mono;
+ struct timespec64 now, mono;
unsigned long seq;
do {
@@ -1563,9 +1583,10 @@ struct timespec get_monotonic_coarse(voi
mono = tk->wall_to_monotonic;
} while (read_seqcount_retry(&timekeeper_seq, seq));
- set_normalized_timespec(&now, now.tv_sec + mono.tv_sec,
+ set_normalized_timespec64(&now, now.tv_sec + mono.tv_sec,
now.tv_nsec + mono.tv_nsec);
- return now;
+
+ return timespec64_to_timespec(now);
}
/*
@@ -1589,7 +1610,7 @@ ktime_t ktime_get_update_offsets_tick(kt
ktime_t *offs_tai)
{
struct timekeeper *tk = &timekeeper;
- struct timespec ts;
+ struct timespec64 ts;
ktime_t now;
unsigned int seq;
@@ -1597,7 +1618,6 @@ ktime_t ktime_get_update_offsets_tick(kt
seq = read_seqcount_begin(&timekeeper_seq);
ts = tk_xtime(tk);
-
*offs_real = tk->offs_real;
*offs_boot = tk->offs_boot;
*offs_tai = tk->offs_tai;
@@ -1650,14 +1670,14 @@ ktime_t ktime_get_monotonic_offset(void)
{
struct timekeeper *tk = &timekeeper;
unsigned long seq;
- struct timespec wtom;
+ struct timespec64 wtom;
do {
seq = read_seqcount_begin(&timekeeper_seq);
wtom = tk->wall_to_monotonic;
} while (read_seqcount_retry(&timekeeper_seq, seq));
- return timespec_to_ktime(wtom);
+ return timespec64_to_ktime(wtom);
}
EXPORT_SYMBOL_GPL(ktime_get_monotonic_offset);
@@ -1668,7 +1688,8 @@ int do_adjtimex(struct timex *txc)
{
struct timekeeper *tk = &timekeeper;
unsigned long flags;
- struct timespec ts;
+ struct timespec64 ts;
+ struct timespec tmp;
s32 orig_tai, tai;
int ret;
@@ -1688,7 +1709,8 @@ int do_adjtimex(struct timex *txc)
return ret;
}
- getnstimeofday(&ts);
+ getnstimeofday(&tmp);
+ ts = timespec_to_timespec64(tmp);
raw_spin_lock_irqsave(&timekeeper_lock, flags);
write_seqcount_begin(&timekeeper_seq);
Index: tip/kernel/time/timekeeping_debug.c
===================================================================
--- tip.orig/kernel/time/timekeeping_debug.c
+++ tip/kernel/time/timekeeping_debug.c
@@ -67,7 +67,7 @@ static int __init tk_debug_sleep_time_in
}
late_initcall(tk_debug_sleep_time_init);
-void tk_debug_account_sleep_time(struct timespec *t)
+void tk_debug_account_sleep_time(struct timespec64 *t)
{
sleep_time_bin[fls(t->tv_sec)]++;
}
Index: tip/kernel/time/timekeeping_internal.h
===================================================================
--- tip.orig/kernel/time/timekeeping_internal.h
+++ tip/kernel/time/timekeeping_internal.h
@@ -6,7 +6,7 @@
#include <linux/time.h>
#ifdef CONFIG_DEBUG_FS
-extern void tk_debug_account_sleep_time(struct timespec *t);
+extern void tk_debug_account_sleep_time(struct timespec64 *t);
#else
#define tk_debug_account_sleep_time(x)
#endif
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 10/55] time: Consolidate the time accessor prototypes
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (8 preceding siblings ...)
2014-07-11 13:44 ` [patch 09/55] timekeeping: Convert timekeeping core to use timespec64s Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 11/55] timekeeping: Provide timespec64 based interfaces Thomas Gleixner
` (44 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: time-move-all-accessor-prototypes-into-separate-header.patch --]
[-- Type: text/plain, Size: 9480 bytes --]
Right now we have time related prototypes in 3 different header
files. Move it to a single timekeeping header file and move the core
internal stuff into a core private header.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/hrtimer.h | 11 ------
include/linux/ktime.h | 8 +---
include/linux/time.h | 45 ++-----------------------
include/linux/timekeeping.h | 78 ++++++++++++++++++++++++++++++++++++++++++++
kernel/time/hrtimer.c | 2 +
kernel/time/posix-timers.c | 2 +
kernel/time/tick-internal.h | 2 +
kernel/time/time.c | 1
kernel/time/timekeeping.h | 20 +++++++++++
9 files changed, 111 insertions(+), 58 deletions(-)
Index: tip/include/linux/hrtimer.h
===================================================================
--- tip.orig/include/linux/hrtimer.h
+++ tip/include/linux/hrtimer.h
@@ -326,17 +326,6 @@ static inline void timerfd_clock_was_set
#endif
extern void hrtimers_resume(void);
-extern ktime_t ktime_get(void);
-extern ktime_t ktime_get_real(void);
-extern ktime_t ktime_get_boottime(void);
-extern ktime_t ktime_get_monotonic_offset(void);
-extern ktime_t ktime_get_clocktai(void);
-extern ktime_t ktime_get_update_offsets_tick(ktime_t *offs_real,
- ktime_t *offs_boot,
- ktime_t *offs_tai);
-extern ktime_t ktime_get_update_offsets_now(ktime_t *offs_real,
- ktime_t *offs_boot,
- ktime_t *offs_tai);
DECLARE_PER_CPU(struct tick_device, tick_cpu_device);
Index: tip/include/linux/ktime.h
===================================================================
--- tip.orig/include/linux/ktime.h
+++ tip/include/linux/ktime.h
@@ -226,12 +226,6 @@ static inline __must_check bool ktime_to
#define LOW_RES_NSEC TICK_NSEC
#define KTIME_LOW_RES (ktime_t){ .tv64 = LOW_RES_NSEC }
-/* Get the monotonic time in timespec format: */
-extern void ktime_get_ts(struct timespec *ts);
-
-/* Get the real (wall-) time in timespec format: */
-#define ktime_get_real_ts(ts) getnstimeofday(ts)
-
static inline ktime_t ns_to_ktime(u64 ns)
{
static const ktime_t ktime_zero = { .tv64 = 0 };
@@ -246,4 +240,6 @@ static inline ktime_t ms_to_ktime(u64 ms
return ktime_add_ms(ktime_zero, ms);
}
+# include <linux/timekeeping.h>
+
#endif
Index: tip/include/linux/time.h
===================================================================
--- tip.orig/include/linux/time.h
+++ tip/include/linux/time.h
@@ -99,25 +99,7 @@ static inline bool timespec_valid_strict
return true;
}
-extern bool persistent_clock_exist;
-
-static inline bool has_persistent_clock(void)
-{
- return persistent_clock_exist;
-}
-
-extern void read_persistent_clock(struct timespec *ts);
-extern void read_boot_clock(struct timespec *ts);
-extern int persistent_clock_is_local;
-extern int update_persistent_clock(struct timespec now);
-void timekeeping_init(void);
-extern int timekeeping_suspended;
-
-unsigned long get_seconds(void);
-struct timespec current_kernel_time(void);
-struct timespec __current_kernel_time(void); /* does not take xtime_lock */
-struct timespec get_monotonic_coarse(void);
-void timekeeping_inject_sleeptime(struct timespec *delta);
+extern struct timespec timespec_trunc(struct timespec t, unsigned gran);
#define CURRENT_TIME (current_kernel_time())
#define CURRENT_TIME_SEC ((struct timespec) { get_seconds(), 0 })
@@ -135,33 +117,14 @@ void timekeeping_inject_sleeptime(struct
extern u32 (*arch_gettimeoffset)(void);
#endif
-extern void do_gettimeofday(struct timeval *tv);
-extern int do_settimeofday(const struct timespec *tv);
-extern int do_sys_settimeofday(const struct timespec *tv,
- const struct timezone *tz);
-#define do_posix_clock_monotonic_gettime(ts) ktime_get_ts(ts)
-extern long do_utimes(int dfd, const char __user *filename, struct timespec *times, int flags);
struct itimerval;
extern int do_setitimer(int which, struct itimerval *value,
struct itimerval *ovalue);
-extern unsigned int alarm_setitimer(unsigned int seconds);
extern int do_getitimer(int which, struct itimerval *value);
-extern int __getnstimeofday(struct timespec *tv);
-extern void getnstimeofday(struct timespec *tv);
-extern void getrawmonotonic(struct timespec *ts);
-extern void getnstime_raw_and_real(struct timespec *ts_raw,
- struct timespec *ts_real);
-extern void getboottime(struct timespec *ts);
-extern void monotonic_to_bootbased(struct timespec *ts);
-extern void get_monotonic_boottime(struct timespec *ts);
-extern struct timespec timespec_trunc(struct timespec t, unsigned gran);
-extern int timekeeping_valid_for_hres(void);
-extern u64 timekeeping_max_deferment(void);
-extern int timekeeping_inject_offset(struct timespec *ts);
-extern s32 timekeeping_get_tai_offset(void);
-extern void timekeeping_set_tai_offset(s32 tai_offset);
-extern void timekeeping_clocktai(struct timespec *ts);
+extern unsigned int alarm_setitimer(unsigned int seconds);
+
+extern long do_utimes(int dfd, const char __user *filename, struct timespec *times, int flags);
struct tms;
extern void do_sys_times(struct tms *);
Index: tip/include/linux/timekeeping.h
===================================================================
--- /dev/null
+++ tip/include/linux/timekeeping.h
@@ -0,0 +1,78 @@
+#ifndef _LINUX_TIMEKEEPING_H
+#define _LINUX_TIMEKEEPING_H
+
+/* Included from linux/ktime.h */
+
+void timekeeping_init(void);
+extern int timekeeping_suspended;
+
+/*
+ * Get and set timeofday
+ */
+extern void do_gettimeofday(struct timeval *tv);
+extern int do_settimeofday(const struct timespec *tv);
+extern int do_sys_settimeofday(const struct timespec *tv,
+ const struct timezone *tz);
+
+/*
+ * Kernel time accessors
+ */
+unsigned long get_seconds(void);
+struct timespec current_kernel_time(void);
+/* does not take xtime_lock */
+struct timespec __current_kernel_time(void);
+
+/*
+ * timespec based interfaces
+ */
+struct timespec get_monotonic_coarse(void);
+extern void getrawmonotonic(struct timespec *ts);
+extern void monotonic_to_bootbased(struct timespec *ts);
+extern void get_monotonic_boottime(struct timespec *ts);
+extern void ktime_get_ts(struct timespec *ts);
+
+extern int __getnstimeofday(struct timespec *tv);
+extern void getnstimeofday(struct timespec *tv);
+extern void getboottime(struct timespec *ts);
+
+#define do_posix_clock_monotonic_gettime(ts) ktime_get_ts(ts)
+#define ktime_get_real_ts(ts) getnstimeofday(ts)
+
+
+/*
+ * ktime_t based interfaces
+ */
+extern ktime_t ktime_get(void);
+extern ktime_t ktime_get_real(void);
+extern ktime_t ktime_get_boottime(void);
+extern ktime_t ktime_get_monotonic_offset(void);
+extern ktime_t ktime_get_clocktai(void);
+
+/*
+ * RTC specific
+ */
+extern void timekeeping_inject_sleeptime(struct timespec *delta);
+
+/*
+ * PPS accessor
+ */
+extern void getnstime_raw_and_real(struct timespec *ts_raw,
+ struct timespec *ts_real);
+
+/*
+ * Persistent clock related interfaces
+ */
+extern bool persistent_clock_exist;
+extern int persistent_clock_is_local;
+
+static inline bool has_persistent_clock(void)
+{
+ return persistent_clock_exist;
+}
+
+extern void read_persistent_clock(struct timespec *ts);
+extern void read_boot_clock(struct timespec *ts);
+extern int update_persistent_clock(struct timespec now);
+
+
+#endif
Index: tip/kernel/time/hrtimer.c
===================================================================
--- tip.orig/kernel/time/hrtimer.c
+++ tip/kernel/time/hrtimer.c
@@ -54,6 +54,8 @@
#include <trace/events/timer.h>
+#include "timekeeping.h"
+
/*
* The timer bases:
*
Index: tip/kernel/time/posix-timers.c
===================================================================
--- tip.orig/kernel/time/posix-timers.c
+++ tip/kernel/time/posix-timers.c
@@ -49,6 +49,8 @@
#include <linux/export.h>
#include <linux/hashtable.h>
+#include "timekeeping.h"
+
/*
* Management arrays for POSIX timers. Timers are now kept in static hash table
* with 512 entries.
Index: tip/kernel/time/tick-internal.h
===================================================================
--- tip.orig/kernel/time/tick-internal.h
+++ tip/kernel/time/tick-internal.h
@@ -4,6 +4,8 @@
#include <linux/hrtimer.h>
#include <linux/tick.h>
+#include "timekeeping.h"
+
extern seqlock_t jiffies_lock;
#define CS_NAME_LEN 32
Index: tip/kernel/time/time.c
===================================================================
--- tip.orig/kernel/time/time.c
+++ tip/kernel/time/time.c
@@ -42,6 +42,7 @@
#include <asm/unistd.h>
#include "timeconst.h"
+#include "timekeeping.h"
/*
* The timezone where the local system is located. Used as a default by some
Index: tip/kernel/time/timekeeping.h
===================================================================
--- /dev/null
+++ tip/kernel/time/timekeeping.h
@@ -0,0 +1,20 @@
+#ifndef _KERNEL_TIME_TIMEKEEPING_H
+#define _KERNEL_TIME_TIMEKEEPING_H
+/*
+ * Internal interfaces for kernel/time/
+ */
+extern ktime_t ktime_get_update_offsets_tick(ktime_t *offs_real,
+ ktime_t *offs_boot,
+ ktime_t *offs_tai);
+extern ktime_t ktime_get_update_offsets_now(ktime_t *offs_real,
+ ktime_t *offs_boot,
+ ktime_t *offs_tai);
+
+extern int timekeeping_valid_for_hres(void);
+extern u64 timekeeping_max_deferment(void);
+extern int timekeeping_inject_offset(struct timespec *ts);
+extern s32 timekeeping_get_tai_offset(void);
+extern void timekeeping_set_tai_offset(s32 tai_offset);
+extern void timekeeping_clocktai(struct timespec *ts);
+
+#endif
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 11/55] timekeeping: Provide timespec64 based interfaces
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (9 preceding siblings ...)
2014-07-11 13:44 ` [patch 10/55] time: Consolidate the time accessor prototypes Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 12/55] timekeeper: Move tk_xtime to core code Thomas Gleixner
` (43 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-provide-ts64-interfaces.patch --]
[-- Type: text/plain, Size: 7147 bytes --]
To convert callers of the core code to timespec64 we need to provide
the proper interfaces.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 66 ++++++++++++++++++++++++++++++++++++++++----
kernel/time/ntp.c | 7 ++--
kernel/time/timekeeping.c | 47 ++++++++++++++-----------------
3 files changed, 87 insertions(+), 33 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -29,15 +29,71 @@ struct timespec get_monotonic_coarse(voi
extern void getrawmonotonic(struct timespec *ts);
extern void monotonic_to_bootbased(struct timespec *ts);
extern void get_monotonic_boottime(struct timespec *ts);
-extern void ktime_get_ts(struct timespec *ts);
+extern void ktime_get_ts64(struct timespec64 *ts);
+
+extern int __getnstimeofday64(struct timespec64 *tv);
+extern void getnstimeofday64(struct timespec64 *tv);
+
+#if BITS_PER_LONG == 64
+static inline int __getnstimeofday(struct timespec *ts)
+{
+ return __getnstimeofday64(ts);
+}
+
+static inline void getnstimeofday(struct timespec *ts)
+{
+ getnstimeofday64(ts);
+}
+
+static inline void ktime_get_ts(struct timespec *ts)
+{
+ ktime_get_ts64(ts);
+}
+
+static inline void ktime_get_real_ts(struct timespec *ts)
+{
+ getnstimeofday64(ts);
+}
+
+#else
+static inline int __getnstimeofday(struct timespec *ts)
+{
+ struct timespec64 ts64;
+ int ret = __getnstimeofday64(&ts64);
+
+ *ts = timespec64_to_timespec(ts64);
+ return ret;
+}
+
+static inline void getnstimeofday(struct timespec *ts)
+{
+ struct timespec64 ts64;
+
+ getnstimeofday64(&ts64);
+ *ts = timespec64_to_timespec(ts64);
+}
+
+static inline void ktime_get_ts(struct timespec *ts)
+{
+ struct timespec64 ts64;
+
+ ktime_get_ts64(&ts64);
+ *ts = timespec64_to_timespec(ts64);
+}
+
+static inline void ktime_get_real_ts(struct timespec *ts)
+{
+ struct timespec64 ts64;
+
+ getnstimeofday64(&ts64);
+ *ts = timespec64_to_timespec(ts64);
+}
+#endif
-extern int __getnstimeofday(struct timespec *tv);
-extern void getnstimeofday(struct timespec *tv);
extern void getboottime(struct timespec *ts);
#define do_posix_clock_monotonic_gettime(ts) ktime_get_ts(ts)
-#define ktime_get_real_ts(ts) getnstimeofday(ts)
-
+#define ktime_get_real_ts64(ts) getnstimeofday64(ts)
/*
* ktime_t based interfaces
Index: tip/kernel/time/ntp.c
===================================================================
--- tip.orig/kernel/time/ntp.c
+++ tip/kernel/time/ntp.c
@@ -466,7 +466,8 @@ static DECLARE_DELAYED_WORK(sync_cmos_wo
static void sync_cmos_clock(struct work_struct *work)
{
- struct timespec now, next;
+ struct timespec64 now;
+ struct timespec next;
int fail = 1;
/*
@@ -485,9 +486,9 @@ static void sync_cmos_clock(struct work_
return;
}
- getnstimeofday(&now);
+ getnstimeofday64(&now);
if (abs(now.tv_nsec - (NSEC_PER_SEC / 2)) <= tick_nsec * 5) {
- struct timespec adjust = now;
+ struct timespec adjust = timespec64_to_timespec(now);
fail = -ENODEV;
if (persistent_clock_is_local)
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -285,13 +285,13 @@ static void timekeeping_forward_now(stru
}
/**
- * __getnstimeofday - Returns the time of day in a timespec.
+ * __getnstimeofday64 - Returns the time of day in a timespec64.
* @ts: pointer to the timespec to be set
*
* Updates the time of day in the timespec.
* Returns 0 on success, or -ve when suspended (timespec will be undefined).
*/
-int __getnstimeofday(struct timespec *ts)
+int __getnstimeofday64(struct timespec64 *ts)
{
struct timekeeper *tk = &timekeeper;
unsigned long seq;
@@ -306,7 +306,7 @@ int __getnstimeofday(struct timespec *ts
} while (read_seqcount_retry(&timekeeper_seq, seq));
ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs);
+ timespec64_add_ns(ts, nsecs);
/*
* Do not bail out early, in case there were callers still using
@@ -316,19 +316,19 @@ int __getnstimeofday(struct timespec *ts
return -EAGAIN;
return 0;
}
-EXPORT_SYMBOL(__getnstimeofday);
+EXPORT_SYMBOL(__getnstimeofday64);
/**
- * getnstimeofday - Returns the time of day in a timespec.
+ * getnstimeofday64 - Returns the time of day in a timespec64.
* @ts: pointer to the timespec to be set
*
* Returns the time of day in a timespec (WARN if suspended).
*/
-void getnstimeofday(struct timespec *ts)
+void getnstimeofday64(struct timespec64 *ts)
{
- WARN_ON(__getnstimeofday(ts));
+ WARN_ON(__getnstimeofday64(ts));
}
-EXPORT_SYMBOL(getnstimeofday);
+EXPORT_SYMBOL(getnstimeofday64);
ktime_t ktime_get(void)
{
@@ -350,17 +350,17 @@ ktime_t ktime_get(void)
EXPORT_SYMBOL_GPL(ktime_get);
/**
- * ktime_get_ts - get the monotonic clock in timespec format
+ * ktime_get_ts64 - get the monotonic clock in timespec64 format
* @ts: pointer to timespec variable
*
* The function calculates the monotonic clock from the realtime
* clock and the wall_to_monotonic offset and stores the result
* in normalized timespec format in the variable pointed to by @ts.
*/
-void ktime_get_ts(struct timespec *ts)
+void ktime_get_ts64(struct timespec64 *ts)
{
struct timekeeper *tk = &timekeeper;
- struct timespec64 ts64, tomono;
+ struct timespec64 tomono;
s64 nsec;
unsigned int seq;
@@ -368,18 +368,17 @@ void ktime_get_ts(struct timespec *ts)
do {
seq = read_seqcount_begin(&timekeeper_seq);
- ts64.tv_sec = tk->xtime_sec;
+ ts->tv_sec = tk->xtime_sec;
nsec = timekeeping_get_ns(tk);
tomono = tk->wall_to_monotonic;
} while (read_seqcount_retry(&timekeeper_seq, seq));
- ts64.tv_sec += tomono.tv_sec;
- ts64.tv_nsec = 0;
- timespec64_add_ns(&ts64, nsec + tomono.tv_nsec);
- *ts = timespec64_to_timespec(ts64);
+ ts->tv_sec += tomono.tv_sec;
+ ts->tv_nsec = 0;
+ timespec64_add_ns(ts, nsec + tomono.tv_nsec);
}
-EXPORT_SYMBOL_GPL(ktime_get_ts);
+EXPORT_SYMBOL_GPL(ktime_get_ts64);
/**
@@ -473,9 +472,9 @@ EXPORT_SYMBOL(getnstime_raw_and_real);
*/
void do_gettimeofday(struct timeval *tv)
{
- struct timespec now;
+ struct timespec64 now;
- getnstimeofday(&now);
+ getnstimeofday64(&now);
tv->tv_sec = now.tv_sec;
tv->tv_usec = now.tv_nsec/1000;
}
@@ -680,11 +679,11 @@ int timekeeping_notify(struct clocksourc
*/
ktime_t ktime_get_real(void)
{
- struct timespec now;
+ struct timespec64 now;
- getnstimeofday(&now);
+ getnstimeofday64(&now);
- return timespec_to_ktime(now);
+ return timespec64_to_ktime(now);
}
EXPORT_SYMBOL_GPL(ktime_get_real);
@@ -1689,7 +1688,6 @@ int do_adjtimex(struct timex *txc)
struct timekeeper *tk = &timekeeper;
unsigned long flags;
struct timespec64 ts;
- struct timespec tmp;
s32 orig_tai, tai;
int ret;
@@ -1709,8 +1707,7 @@ int do_adjtimex(struct timex *txc)
return ret;
}
- getnstimeofday(&tmp);
- ts = timespec_to_timespec64(tmp);
+ getnstimeofday64(&ts);
raw_spin_lock_irqsave(&timekeeper_lock, flags);
write_seqcount_begin(&timekeeper_seq);
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 12/55] timekeeper: Move tk_xtime to core code
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (10 preceding siblings ...)
2014-07-11 13:44 ` [patch 11/55] timekeeping: Provide timespec64 based interfaces Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 13/55] timekeeping: Cache optimize struct timekeeper Thomas Gleixner
` (42 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeper-move-tk-xtime-to-core.patch --]
[-- Type: text/plain, Size: 4190 bytes --]
No users outside of the core.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeper_internal.h | 18 ---------
kernel/time/timekeeping.c | 70 ++++++++++++++++++++++--------------
2 files changed, 43 insertions(+), 45 deletions(-)
Index: tip/include/linux/timekeeper_internal.h
===================================================================
--- tip.orig/include/linux/timekeeper_internal.h
+++ tip/include/linux/timekeeper_internal.h
@@ -71,16 +71,6 @@ struct timekeeper {
};
-static inline struct timespec64 tk_xtime(struct timekeeper *tk)
-{
- struct timespec64 ts;
-
- ts.tv_sec = tk->xtime_sec;
- ts.tv_nsec = (long)(tk->xtime_nsec >> tk->shift);
- return ts;
-}
-
-
#ifdef CONFIG_GENERIC_TIME_VSYSCALL
extern void update_vsyscall(struct timekeeper *tk);
@@ -92,14 +82,6 @@ extern void update_vsyscall_old(struct t
struct clocksource *c, u32 mult);
extern void update_vsyscall_tz(void);
-static inline void update_vsyscall(struct timekeeper *tk)
-{
- struct timespec xt;
-
- xt = tk_xtime(tk);
- update_vsyscall_old(&xt, &tk->wall_to_monotonic, tk->clock, tk->mult);
-}
-
#else
static inline void update_vsyscall(struct timekeeper *tk)
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -51,6 +51,15 @@ static inline void tk_normalize_xtime(st
}
}
+static inline struct timespec64 tk_xtime(struct timekeeper *tk)
+{
+ struct timespec64 ts;
+
+ ts.tv_sec = tk->xtime_sec;
+ ts.tv_nsec = (long)(tk->xtime_nsec >> tk->shift);
+ return ts;
+}
+
static void tk_set_xtime(struct timekeeper *tk, const struct timespec64 *ts)
{
tk->xtime_sec = ts->tv_sec;
@@ -199,6 +208,40 @@ static inline s64 timekeeping_get_ns_raw
return nsec + arch_gettimeoffset();
}
+#ifdef CONFIG_GENERIC_TIME_VSYSCALL_OLD
+
+static inline void update_vsyscall(struct timekeeper *tk)
+{
+ struct timespec xt;
+
+ xt = tk_xtime(tk);
+ update_vsyscall_old(&xt, &tk->wall_to_monotonic, tk->clock, tk->mult);
+}
+
+static inline void old_vsyscall_fixup(struct timekeeper *tk)
+{
+ s64 remainder;
+
+ /*
+ * Store only full nanoseconds into xtime_nsec after rounding
+ * it up and add the remainder to the error difference.
+ * XXX - This is necessary to avoid small 1ns inconsistnecies caused
+ * by truncating the remainder in vsyscalls. However, it causes
+ * additional work to be done in timekeeping_adjust(). Once
+ * the vsyscall implementations are converted to use xtime_nsec
+ * (shifted nanoseconds), and CONFIG_GENERIC_TIME_VSYSCALL_OLD
+ * users are removed, this can be killed.
+ */
+ remainder = tk->xtime_nsec & ((1ULL << tk->shift) - 1);
+ tk->xtime_nsec -= remainder;
+ tk->xtime_nsec += 1ULL << tk->shift;
+ tk->ntp_error += remainder << tk->ntp_error_shift;
+ tk->ntp_error -= (1ULL << tk->shift) << tk->ntp_error_shift;
+}
+#else
+#define old_vsyscall_fixup(tk)
+#endif
+
static RAW_NOTIFIER_HEAD(pvclock_gtod_chain);
static void update_pvclock_gtod(struct timekeeper *tk, bool was_set)
@@ -1330,33 +1373,6 @@ static cycle_t logarithmic_accumulation(
return offset;
}
-#ifdef CONFIG_GENERIC_TIME_VSYSCALL_OLD
-static inline void old_vsyscall_fixup(struct timekeeper *tk)
-{
- s64 remainder;
-
- /*
- * Store only full nanoseconds into xtime_nsec after rounding
- * it up and add the remainder to the error difference.
- * XXX - This is necessary to avoid small 1ns inconsistnecies caused
- * by truncating the remainder in vsyscalls. However, it causes
- * additional work to be done in timekeeping_adjust(). Once
- * the vsyscall implementations are converted to use xtime_nsec
- * (shifted nanoseconds), and CONFIG_GENERIC_TIME_VSYSCALL_OLD
- * users are removed, this can be killed.
- */
- remainder = tk->xtime_nsec & ((1ULL << tk->shift) - 1);
- tk->xtime_nsec -= remainder;
- tk->xtime_nsec += 1ULL << tk->shift;
- tk->ntp_error += remainder << tk->ntp_error_shift;
- tk->ntp_error -= (1ULL << tk->shift) << tk->ntp_error_shift;
-}
-#else
-#define old_vsyscall_fixup(tk)
-#endif
-
-
-
/**
* update_wall_time - Uses the current clocksource to increment the wall time
*
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 13/55] timekeeping: Cache optimize struct timekeeper
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (11 preceding siblings ...)
2014-07-11 13:44 ` [patch 12/55] timekeeper: Move tk_xtime to core code Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 14/55] timekeeping: Provide internal ktime_t based data Thomas Gleixner
` (41 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-optimize-cache-layout.patch --]
[-- Type: text/plain, Size: 25341 bytes --]
struct timekeeper is quite badly sorted for the hot readout path. Most
time access functions need to load two cache lines.
Rearrange it so ktime_get() and getnstimeofday() are happy with a
single cache line.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeper_internal.h | 84 ++++++++--------
kernel/time/timekeeping.c | 185 ++++++++++++++++++------------------
2 files changed, 143 insertions(+), 126 deletions(-)
Index: tip/include/linux/timekeeper_internal.h
===================================================================
--- tip.orig/include/linux/timekeeper_internal.h
+++ tip/include/linux/timekeeper_internal.h
@@ -10,7 +10,22 @@
#include <linux/jiffies.h>
#include <linux/time.h>
-/* Structure holding internal timekeeping values. */
+/*
+ * Structure holding internal timekeeping values.
+ *
+ * Note: wall_to_monotonic is what we need to add to xtime (or xtime
+ * corrected for sub jiffie times) to get to monotonic time.
+ * Monotonic is pegged at zero at system boot time, so
+ * wall_to_monotonic will be negative, however, we will ALWAYS keep
+ * the tv_nsec part positive so we can use the usual normalization.
+ *
+ * wall_to_monotonic is moved after resume from suspend for the
+ * monotonic time not to jump. We need to add total_sleep_time to
+ * wall_to_monotonic to get the real boot based time offset.
+ *
+ * - wall_to_monotonic is no longer the boot time, getboottime must be
+ * used instead.
+ */
struct timekeeper {
/* Current clocksource used for timekeeping. */
struct clocksource *clock;
@@ -18,6 +33,29 @@ struct timekeeper {
u32 mult;
/* The shift value of the current clocksource. */
u32 shift;
+ /* Clock shifted nano seconds */
+ u64 xtime_nsec;
+
+ /* Current CLOCK_REALTIME time in seconds */
+ u64 xtime_sec;
+ /* CLOCK_REALTIME to CLOCK_MONOTONIC offset */
+ struct timespec64 wall_to_monotonic;
+
+ /* Offset clock monotonic -> clock realtime */
+ ktime_t offs_real;
+ /* Offset clock monotonic -> clock boottime */
+ ktime_t offs_boot;
+ /* Offset clock monotonic -> clock tai */
+ ktime_t offs_tai;
+
+ /* time spent in suspend */
+ struct timespec64 total_sleep_time;
+ /* The current UTC to TAI offset in seconds */
+ s32 tai_offset;
+
+ /* The raw monotonic time for the CLOCK_MONOTONIC_RAW posix clock. */
+ struct timespec64 raw_time;
+
/* Number of clock cycles in one NTP interval. */
cycle_t cycle_interval;
/* Last cycle value (also stored in clock->cycle_last) */
@@ -29,46 +67,16 @@ struct timekeeper {
/* Raw nano seconds accumulated per NTP interval. */
u32 raw_interval;
- /* Current CLOCK_REALTIME time in seconds */
- u64 xtime_sec;
- /* Clock shifted nano seconds */
- u64 xtime_nsec;
-
- /* Difference between accumulated time and NTP time in ntp
- * shifted nano seconds. */
+ /*
+ * Difference between accumulated time and NTP time in ntp
+ * shifted nano seconds.
+ */
s64 ntp_error;
- /* Shift conversion between clock shifted nano seconds and
- * ntp shifted nano seconds. */
- u32 ntp_error_shift;
-
/*
- * wall_to_monotonic is what we need to add to xtime (or xtime corrected
- * for sub jiffie times) to get to monotonic time. Monotonic is pegged
- * at zero at system boot time, so wall_to_monotonic will be negative,
- * however, we will ALWAYS keep the tv_nsec part positive so we can use
- * the usual normalization.
- *
- * wall_to_monotonic is moved after resume from suspend for the
- * monotonic time not to jump. We need to add total_sleep_time to
- * wall_to_monotonic to get the real boot based time offset.
- *
- * - wall_to_monotonic is no longer the boot time, getboottime must be
- * used instead.
+ * Shift conversion between clock shifted nano seconds and
+ * ntp shifted nano seconds.
*/
- struct timespec64 wall_to_monotonic;
- /* Offset clock monotonic -> clock realtime */
- ktime_t offs_real;
- /* time spent in suspend */
- struct timespec64 total_sleep_time;
- /* Offset clock monotonic -> clock boottime */
- ktime_t offs_boot;
- /* The raw monotonic time for the CLOCK_MONOTONIC_RAW posix clock. */
- struct timespec64 raw_time;
- /* The current UTC to TAI offset in seconds */
- s32 tai_offset;
- /* Offset clock monotonic -> clock tai */
- ktime_t offs_tai;
-
+ u32 ntp_error_shift;
};
#ifdef CONFIG_GENERIC_TIME_VSYSCALL
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -32,9 +32,16 @@
#define TK_MIRROR (1 << 1)
#define TK_CLOCK_WAS_SET (1 << 2)
-static struct timekeeper timekeeper;
+/*
+ * The most important data for readout fits into a single 64 byte
+ * cache line.
+ */
+static struct {
+ seqcount_t seq;
+ struct timekeeper timekeeper;
+} tk_core ____cacheline_aligned;
+
static DEFINE_RAW_SPINLOCK(timekeeper_lock);
-static seqcount_t timekeeper_seq;
static struct timekeeper shadow_timekeeper;
/* flag for if timekeeping is suspended */
@@ -254,7 +261,7 @@ static void update_pvclock_gtod(struct t
*/
int pvclock_gtod_register_notifier(struct notifier_block *nb)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long flags;
int ret;
@@ -295,7 +302,8 @@ static void timekeeping_update(struct ti
update_pvclock_gtod(tk, action & TK_CLOCK_WAS_SET);
if (action & TK_MIRROR)
- memcpy(&shadow_timekeeper, &timekeeper, sizeof(timekeeper));
+ memcpy(&shadow_timekeeper, &tk_core.timekeeper,
+ sizeof(tk_core.timekeeper));
}
/**
@@ -336,17 +344,17 @@ static void timekeeping_forward_now(stru
*/
int __getnstimeofday64(struct timespec64 *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long seq;
s64 nsecs = 0;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ts->tv_sec = tk->xtime_sec;
nsecs = timekeeping_get_ns(tk);
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
ts->tv_nsec = 0;
timespec64_add_ns(ts, nsecs);
@@ -375,18 +383,18 @@ EXPORT_SYMBOL(getnstimeofday64);
ktime_t ktime_get(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned int seq;
s64 secs, nsecs;
WARN_ON(timekeeping_suspended);
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
secs = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
nsecs = timekeeping_get_ns(tk) + tk->wall_to_monotonic.tv_nsec;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
return ktime_set(secs, nsecs);
}
@@ -402,7 +410,7 @@ EXPORT_SYMBOL_GPL(ktime_get);
*/
void ktime_get_ts64(struct timespec64 *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 tomono;
s64 nsec;
unsigned int seq;
@@ -410,12 +418,12 @@ void ktime_get_ts64(struct timespec64 *t
WARN_ON(timekeeping_suspended);
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ts->tv_sec = tk->xtime_sec;
nsec = timekeeping_get_ns(tk);
tomono = tk->wall_to_monotonic;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
ts->tv_sec += tomono.tv_sec;
ts->tv_nsec = 0;
@@ -432,7 +440,7 @@ EXPORT_SYMBOL_GPL(ktime_get_ts64);
*/
void timekeeping_clocktai(struct timespec *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 ts64;
unsigned long seq;
u64 nsecs;
@@ -440,12 +448,12 @@ void timekeeping_clocktai(struct timespe
WARN_ON(timekeeping_suspended);
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ts64.tv_sec = tk->xtime_sec + tk->tai_offset;
nsecs = timekeeping_get_ns(tk);
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
ts64.tv_nsec = 0;
timespec64_add_ns(&ts64, nsecs);
@@ -482,14 +490,14 @@ EXPORT_SYMBOL(ktime_get_clocktai);
*/
void getnstime_raw_and_real(struct timespec *ts_raw, struct timespec *ts_real)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long seq;
s64 nsecs_raw, nsecs_real;
WARN_ON_ONCE(timekeeping_suspended);
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
*ts_raw = timespec64_to_timespec(tk->raw_time);
ts_real->tv_sec = tk->xtime_sec;
@@ -498,7 +506,7 @@ void getnstime_raw_and_real(struct times
nsecs_raw = timekeeping_get_ns_raw(tk);
nsecs_real = timekeeping_get_ns(tk);
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
timespec_add_ns(ts_raw, nsecs_raw);
timespec_add_ns(ts_real, nsecs_real);
@@ -531,7 +539,7 @@ EXPORT_SYMBOL(do_gettimeofday);
*/
int do_settimeofday(const struct timespec *tv)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 ts_delta, xt, tmp;
unsigned long flags;
@@ -539,7 +547,7 @@ int do_settimeofday(const struct timespe
return -EINVAL;
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
timekeeping_forward_now(tk);
@@ -554,7 +562,7 @@ int do_settimeofday(const struct timespe
timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
/* signal hrtimers about time change */
@@ -572,7 +580,7 @@ EXPORT_SYMBOL(do_settimeofday);
*/
int timekeeping_inject_offset(struct timespec *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long flags;
struct timespec64 ts64, tmp;
int ret = 0;
@@ -583,7 +591,7 @@ int timekeeping_inject_offset(struct tim
ts64 = timespec_to_timespec64(*ts);
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
timekeeping_forward_now(tk);
@@ -600,7 +608,7 @@ int timekeeping_inject_offset(struct tim
error: /* even if we error out, we forwarded the time, so call update */
timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
/* signal hrtimers about time change */
@@ -617,14 +625,14 @@ EXPORT_SYMBOL(timekeeping_inject_offset)
*/
s32 timekeeping_get_tai_offset(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned int seq;
s32 ret;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ret = tk->tai_offset;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
return ret;
}
@@ -645,14 +653,14 @@ static void __timekeeping_set_tai_offset
*/
void timekeeping_set_tai_offset(s32 tai_offset)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long flags;
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
__timekeeping_set_tai_offset(tk, tai_offset);
timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
clock_was_set();
}
@@ -664,14 +672,14 @@ void timekeeping_set_tai_offset(s32 tai_
*/
static int change_clocksource(void *data)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct clocksource *new, *old;
unsigned long flags;
new = (struct clocksource *) data;
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
timekeeping_forward_now(tk);
/*
@@ -691,7 +699,7 @@ static int change_clocksource(void *data
}
timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
return 0;
@@ -706,7 +714,7 @@ static int change_clocksource(void *data
*/
int timekeeping_notify(struct clocksource *clock)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
if (tk->clock == clock)
return 0;
@@ -738,17 +746,17 @@ EXPORT_SYMBOL_GPL(ktime_get_real);
*/
void getrawmonotonic(struct timespec *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 ts64;
unsigned long seq;
s64 nsecs;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
nsecs = timekeeping_get_ns_raw(tk);
ts64 = tk->raw_time;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
timespec64_add_ns(&ts64, nsecs);
*ts = timespec64_to_timespec(ts64);
@@ -760,16 +768,16 @@ EXPORT_SYMBOL(getrawmonotonic);
*/
int timekeeping_valid_for_hres(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long seq;
int ret;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ret = tk->clock->flags & CLOCK_SOURCE_VALID_FOR_HRES;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
return ret;
}
@@ -779,16 +787,16 @@ int timekeeping_valid_for_hres(void)
*/
u64 timekeeping_max_deferment(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long seq;
u64 ret;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ret = tk->clock->max_idle_ns;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
return ret;
}
@@ -828,7 +836,7 @@ void __weak read_boot_clock(struct times
*/
void __init timekeeping_init(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct clocksource *clock;
unsigned long flags;
struct timespec64 now, boot, tmp;
@@ -854,7 +862,7 @@ void __init timekeeping_init(void)
}
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
ntp_init();
clock = clocksource_default_clock();
@@ -875,9 +883,10 @@ void __init timekeeping_init(void)
tmp.tv_nsec = 0;
tk_set_sleep_time(tk, tmp);
- memcpy(&shadow_timekeeper, &timekeeper, sizeof(timekeeper));
+ memcpy(&shadow_timekeeper, &tk_core.timekeeper,
+ sizeof(tk_core.timekeeper));
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
}
@@ -918,7 +927,7 @@ static void __timekeeping_inject_sleepti
*/
void timekeeping_inject_sleeptime(struct timespec *delta)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 tmp;
unsigned long flags;
@@ -930,7 +939,7 @@ void timekeeping_inject_sleeptime(struct
return;
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
timekeeping_forward_now(tk);
@@ -939,7 +948,7 @@ void timekeeping_inject_sleeptime(struct
timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
/* signal hrtimers about time change */
@@ -955,7 +964,7 @@ void timekeeping_inject_sleeptime(struct
*/
static void timekeeping_resume(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct clocksource *clock = tk->clock;
unsigned long flags;
struct timespec64 ts_new, ts_delta;
@@ -970,7 +979,7 @@ static void timekeeping_resume(void)
clocksource_resume();
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
/*
* After system resumes, we need to calculate the suspended time and
@@ -1022,7 +1031,7 @@ static void timekeeping_resume(void)
tk->ntp_error = 0;
timekeeping_suspended = 0;
timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
touch_softlockup_watchdog();
@@ -1035,7 +1044,7 @@ static void timekeeping_resume(void)
static int timekeeping_suspend(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long flags;
struct timespec64 delta, delta_delta;
static struct timespec64 old_delta;
@@ -1053,7 +1062,7 @@ static int timekeeping_suspend(void)
persistent_clock_exist = true;
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
timekeeping_forward_now(tk);
timekeeping_suspended = 1;
@@ -1078,7 +1087,7 @@ static int timekeeping_suspend(void)
}
timekeeping_update(tk, TK_MIRROR);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
clockevents_notify(CLOCK_EVT_NOTIFY_SUSPEND, NULL);
@@ -1380,7 +1389,7 @@ static cycle_t logarithmic_accumulation(
void update_wall_time(void)
{
struct clocksource *clock;
- struct timekeeper *real_tk = &timekeeper;
+ struct timekeeper *real_tk = &tk_core.timekeeper;
struct timekeeper *tk = &shadow_timekeeper;
cycle_t offset;
int shift = 0, maxshift;
@@ -1440,7 +1449,7 @@ void update_wall_time(void)
*/
clock_set |= accumulate_nsecs_to_secs(tk);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
/* Update clock->cycle_last with the new value */
clock->cycle_last = tk->cycle_last;
/*
@@ -1450,12 +1459,12 @@ void update_wall_time(void)
* requires changes to all other timekeeper usage sites as
* well, i.e. move the timekeeper pointer getter into the
* spinlocked/seqcount protected sections. And we trade this
- * memcpy under the timekeeper_seq against one before we start
+ * memcpy under the tk_core.seq against one before we start
* updating.
*/
memcpy(real_tk, tk, sizeof(*tk));
timekeeping_update(real_tk, clock_set);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
out:
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
if (clock_set)
@@ -1476,7 +1485,7 @@ out:
*/
void getboottime(struct timespec *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec boottime = {
.tv_sec = tk->wall_to_monotonic.tv_sec +
tk->total_sleep_time.tv_sec,
@@ -1499,7 +1508,7 @@ EXPORT_SYMBOL_GPL(getboottime);
*/
void get_monotonic_boottime(struct timespec *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 tomono, sleep, ret;
s64 nsec;
unsigned int seq;
@@ -1507,13 +1516,13 @@ void get_monotonic_boottime(struct times
WARN_ON(timekeeping_suspended);
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ret.tv_sec = tk->xtime_sec;
nsec = timekeeping_get_ns(tk);
tomono = tk->wall_to_monotonic;
sleep = tk->total_sleep_time;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
ret.tv_sec += tomono.tv_sec + sleep.tv_sec;
ret.tv_nsec = 0;
@@ -1545,7 +1554,7 @@ EXPORT_SYMBOL_GPL(ktime_get_boottime);
*/
void monotonic_to_bootbased(struct timespec *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 ts64;
ts64 = timespec_to_timespec64(*ts);
@@ -1556,7 +1565,7 @@ EXPORT_SYMBOL_GPL(monotonic_to_bootbased
unsigned long get_seconds(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
return tk->xtime_sec;
}
@@ -1564,22 +1573,22 @@ EXPORT_SYMBOL(get_seconds);
struct timespec __current_kernel_time(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
return timespec64_to_timespec(tk_xtime(tk));
}
struct timespec current_kernel_time(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 now;
unsigned long seq;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
now = tk_xtime(tk);
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
return timespec64_to_timespec(now);
}
@@ -1587,16 +1596,16 @@ EXPORT_SYMBOL(current_kernel_time);
struct timespec get_monotonic_coarse(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 now, mono;
unsigned long seq;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
now = tk_xtime(tk);
mono = tk->wall_to_monotonic;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
set_normalized_timespec64(&now, now.tv_sec + mono.tv_sec,
now.tv_nsec + mono.tv_nsec);
@@ -1624,19 +1633,19 @@ void do_timer(unsigned long ticks)
ktime_t ktime_get_update_offsets_tick(ktime_t *offs_real, ktime_t *offs_boot,
ktime_t *offs_tai)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 ts;
ktime_t now;
unsigned int seq;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ts = tk_xtime(tk);
*offs_real = tk->offs_real;
*offs_boot = tk->offs_boot;
*offs_tai = tk->offs_tai;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
now = ktime_set(ts.tv_sec, ts.tv_nsec);
now = ktime_sub(now, *offs_real);
@@ -1656,13 +1665,13 @@ ktime_t ktime_get_update_offsets_tick(kt
ktime_t ktime_get_update_offsets_now(ktime_t *offs_real, ktime_t *offs_boot,
ktime_t *offs_tai)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
ktime_t now;
unsigned int seq;
u64 secs, nsecs;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
secs = tk->xtime_sec;
nsecs = timekeeping_get_ns(tk);
@@ -1670,7 +1679,7 @@ ktime_t ktime_get_update_offsets_now(kti
*offs_real = tk->offs_real;
*offs_boot = tk->offs_boot;
*offs_tai = tk->offs_tai;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
now = ktime_add_ns(ktime_set(secs, 0), nsecs);
now = ktime_sub(now, *offs_real);
@@ -1683,14 +1692,14 @@ ktime_t ktime_get_update_offsets_now(kti
*/
ktime_t ktime_get_monotonic_offset(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long seq;
struct timespec64 wtom;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
wtom = tk->wall_to_monotonic;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
return timespec64_to_ktime(wtom);
}
@@ -1701,7 +1710,7 @@ EXPORT_SYMBOL_GPL(ktime_get_monotonic_of
*/
int do_adjtimex(struct timex *txc)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long flags;
struct timespec64 ts;
s32 orig_tai, tai;
@@ -1726,7 +1735,7 @@ int do_adjtimex(struct timex *txc)
getnstimeofday64(&ts);
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
orig_tai = tai = tk->tai_offset;
ret = __do_adjtimex(txc, &ts, &tai);
@@ -1735,7 +1744,7 @@ int do_adjtimex(struct timex *txc)
__timekeeping_set_tai_offset(tk, tai);
timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET);
}
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
if (tai != orig_tai)
@@ -1755,11 +1764,11 @@ void hardpps(const struct timespec *phas
unsigned long flags;
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
__hardpps(phase_ts, raw_ts);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
}
EXPORT_SYMBOL(hardpps);
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 14/55] timekeeping: Provide internal ktime_t based data
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (12 preceding siblings ...)
2014-07-11 13:44 ` [patch 13/55] timekeeping: Cache optimize struct timekeeper Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-16 3:29 ` John Stultz
2014-07-11 13:44 ` [patch 15/55] timekeeping: Use ktime_t based data for ktime_get() Thomas Gleixner
` (40 subsequent siblings)
54 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-provide-internal-ktime-based-data.patch --]
[-- Type: text/plain, Size: 3262 bytes --]
The ktime_t based interfaces are used a lot in performance critical
code pathes. Add ktime_t based data so the interfaces don't have to
convert from the xtime/timespec based data.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeper_internal.h | 3 +++
kernel/time/timekeeping.c | 25 +++++++++++++++++++++++++
2 files changed, 28 insertions(+)
Index: tip/include/linux/timekeeper_internal.h
===================================================================
--- tip.orig/include/linux/timekeeper_internal.h
+++ tip/include/linux/timekeeper_internal.h
@@ -36,6 +36,9 @@ struct timekeeper {
/* Clock shifted nano seconds */
u64 xtime_nsec;
+ /* Monotonic base time */
+ ktime_t base_mono;
+
/* Current CLOCK_REALTIME time in seconds */
u64 xtime_sec;
/* CLOCK_REALTIME to CLOCK_MONOTONIC offset */
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -50,12 +50,30 @@ int __read_mostly timekeeping_suspended;
/* Flag for if there is a persistent clock on this platform */
bool __read_mostly persistent_clock_exist = false;
+/*
+ * The xtime based monotonic readout is:
+ * nsec = (xtime_sec + wtm_sec) * 1e9 + wtm_nsec + now();
+ * The ktime based monotonic readout is:
+ * nsec = base_mono + now();
+ * ==> base_mono = (xtime_sec + wtm_sec) * 1e9 + wtm_nsec
+ */
+static inline void tk_update_ktime_base(struct timekeeper *tk)
+{
+ s64 nsec;
+
+ nsec = (s64)(tk->xtime_sec + tk->wall_to_monotonic.tv_sec);
+ nsec *= NSEC_PER_SEC;
+ nsec += tk->wall_to_monotonic.tv_nsec;
+ tk->base_mono = ns_to_ktime(nsec);
+}
+
static inline void tk_normalize_xtime(struct timekeeper *tk)
{
while (tk->xtime_nsec >= ((u64)NSEC_PER_SEC << tk->shift)) {
tk->xtime_nsec -= (u64)NSEC_PER_SEC << tk->shift;
tk->xtime_sec++;
}
+ tk_update_ktime_base(tk);
}
static inline struct timespec64 tk_xtime(struct timekeeper *tk)
@@ -71,6 +89,7 @@ static void tk_set_xtime(struct timekeep
{
tk->xtime_sec = ts->tv_sec;
tk->xtime_nsec = (u64)ts->tv_nsec << tk->shift;
+ tk_update_ktime_base(tk);
}
static void tk_xtime_add(struct timekeeper *tk, const struct timespec64 *ts)
@@ -93,6 +112,7 @@ static void tk_set_wall_to_mono(struct t
WARN_ON_ONCE(tk->offs_real.tv64 != timespec64_to_ktime(tmp).tv64);
tk->wall_to_monotonic = wtm;
set_normalized_timespec64(&tmp, -wtm.tv_sec, -wtm.tv_nsec);
+ tk_update_ktime_base(tk);
tk->offs_real = timespec64_to_ktime(tmp);
tk->offs_tai = ktime_add(tk->offs_real, ktime_set(tk->tai_offset, 0));
}
@@ -329,6 +349,7 @@ static void timekeeping_forward_now(stru
/* If arch requires, add in get_arch_timeoffset() */
tk->xtime_nsec += (u64)arch_gettimeoffset() << tk->shift;
+ /* Updates tk->base_mono as well */
tk_normalize_xtime(tk);
nsec = clocksource_cyc2ns(cycle_delta, clock->mult, clock->shift);
@@ -1448,6 +1469,10 @@ void update_wall_time(void)
* xtime_nsec isn't larger than NSEC_PER_SEC
*/
clock_set |= accumulate_nsecs_to_secs(tk);
+ /*
+ * Update the monotonic base
+ */
+ tk_update_ktime_base(tk);
write_seqcount_begin(&tk_core.seq);
/* Update clock->cycle_last with the new value */
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 14/55] timekeeping: Provide internal ktime_t based data
2014-07-11 13:44 ` [patch 14/55] timekeeping: Provide internal ktime_t based data Thomas Gleixner
@ 2014-07-16 3:29 ` John Stultz
2014-07-16 7:00 ` Thomas Gleixner
0 siblings, 1 reply; 91+ messages in thread
From: John Stultz @ 2014-07-16 3:29 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, Peter Zijlstra, Arnd Bergmann
On Fri, Jul 11, 2014 at 6:44 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> The ktime_t based interfaces are used a lot in performance critical
> code pathes. Add ktime_t based data so the interfaces don't have to
> convert from the xtime/timespec based data.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
> include/linux/timekeeper_internal.h | 3 +++
> kernel/time/timekeeping.c | 25 +++++++++++++++++++++++++
> 2 files changed, 28 insertions(+)
>
> Index: tip/include/linux/timekeeper_internal.h
> ===================================================================
> --- tip.orig/include/linux/timekeeper_internal.h
> +++ tip/include/linux/timekeeper_internal.h
> @@ -36,6 +36,9 @@ struct timekeeper {
> /* Clock shifted nano seconds */
> u64 xtime_nsec;
>
> + /* Monotonic base time */
> + ktime_t base_mono;
> +
> /* Current CLOCK_REALTIME time in seconds */
> u64 xtime_sec;
> /* CLOCK_REALTIME to CLOCK_MONOTONIC offset */
> Index: tip/kernel/time/timekeeping.c
> ===================================================================
> --- tip.orig/kernel/time/timekeeping.c
> +++ tip/kernel/time/timekeeping.c
> @@ -50,12 +50,30 @@ int __read_mostly timekeeping_suspended;
> /* Flag for if there is a persistent clock on this platform */
> bool __read_mostly persistent_clock_exist = false;
>
> +/*
> + * The xtime based monotonic readout is:
> + * nsec = (xtime_sec + wtm_sec) * 1e9 + wtm_nsec + now();
> + * The ktime based monotonic readout is:
> + * nsec = base_mono + now();
> + * ==> base_mono = (xtime_sec + wtm_sec) * 1e9 + wtm_nsec
> + */
> +static inline void tk_update_ktime_base(struct timekeeper *tk)
> +{
> + s64 nsec;
> +
> + nsec = (s64)(tk->xtime_sec + tk->wall_to_monotonic.tv_sec);
> + nsec *= NSEC_PER_SEC;
> + nsec += tk->wall_to_monotonic.tv_nsec;
> + tk->base_mono = ns_to_ktime(nsec);
> +}
Hrmm.. So I do understand why this is useful performance wise.
However, I'm really starting to feel that keeping all this duplicate
data is a real maintenance burden, as remembering to keep the values
in sync always is prone to error.
So I may have to just put up with it, but I'd like to start thinking
about how to reduce the duplicated data in the future. Arnd had an
interesting idea for something like storing fixed point seconds, which
could be cheaply converted to either ktime_t or timespec values.
However, I suspect that would be even more complex for folks to
understand, which I'd rather not do.
Overall, it might be best if we just kill the timespec
wall_to_monotonic/total_sleep_time/tai_offset values and keep the
timekeeper values almost all in timespecs. Then we can leave the
conversion process to basically cache the timespec values to the
vsyscall_update logic?
Or if we do want to maintain performance for non-vdso implementations,
maybe we can split the timespec functions out of the timekeeping core
and do a similar single shot update each tick to set the update the
tiemspec cached values. But that can be layered over the timekeeping
core and we can avoid the crazy updating 2 values any time we change
one logic we currently have.
thanks
-joh
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 14/55] timekeeping: Provide internal ktime_t based data
2014-07-16 3:29 ` John Stultz
@ 2014-07-16 7:00 ` Thomas Gleixner
2014-07-16 7:12 ` Thomas Gleixner
0 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-16 7:00 UTC (permalink / raw)
To: John Stultz; +Cc: LKML, Peter Zijlstra, Arnd Bergmann
On Tue, 15 Jul 2014, John Stultz wrote:
> Hrmm.. So I do understand why this is useful performance wise.
> However, I'm really starting to feel that keeping all this duplicate
> data is a real maintenance burden, as remembering to keep the values
> in sync always is prone to error.
>
> So I may have to just put up with it, but I'd like to start thinking
> about how to reduce the duplicated data in the future. Arnd had an
> interesting idea for something like storing fixed point seconds, which
> could be cheaply converted to either ktime_t or timespec values.
> However, I suspect that would be even more complex for folks to
> understand, which I'd rather not do.
>
> Overall, it might be best if we just kill the timespec
> wall_to_monotonic/total_sleep_time/tai_offset values and keep the
> timekeeper values almost all in timespecs. Then we can leave the
So we kill the time specs and store everything in timespecs :)
> conversion process to basically cache the timespec values to the
> vsyscall_update logic?
> Or if we do want to maintain performance for non-vdso implementations,
> maybe we can split the timespec functions out of the timekeeping core
> and do a similar single shot update each tick to set the update the
> tiemspec cached values. But that can be layered over the timekeeping
> core and we can avoid the crazy updating 2 values any time we change
> one logic we currently have.
Makes sense. Lemme have a stab on it.
Thanks,
tglx
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [patch 14/55] timekeeping: Provide internal ktime_t based data
2014-07-16 7:00 ` Thomas Gleixner
@ 2014-07-16 7:12 ` Thomas Gleixner
2014-07-16 7:20 ` Peter Zijlstra
0 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-16 7:12 UTC (permalink / raw)
To: John Stultz; +Cc: LKML, Peter Zijlstra, Arnd Bergmann
On Wed, 16 Jul 2014, Thomas Gleixner wrote:
> On Tue, 15 Jul 2014, John Stultz wrote:
> > Hrmm.. So I do understand why this is useful performance wise.
> > However, I'm really starting to feel that keeping all this duplicate
> > data is a real maintenance burden, as remembering to keep the values
> > in sync always is prone to error.
> >
> > So I may have to just put up with it, but I'd like to start thinking
> > about how to reduce the duplicated data in the future. Arnd had an
> > interesting idea for something like storing fixed point seconds, which
> > could be cheaply converted to either ktime_t or timespec values.
> > However, I suspect that would be even more complex for folks to
> > understand, which I'd rather not do.
> >
> > Overall, it might be best if we just kill the timespec
> > wall_to_monotonic/total_sleep_time/tai_offset values and keep the
> > timekeeper values almost all in timespecs. Then we can leave the
>
> So we kill the time specs and store everything in timespecs :)
>
> > conversion process to basically cache the timespec values to the
> > vsyscall_update logic?
Looking into it I think for now it's the least risky approach to keep
the core logic based on the timespec stuff unmodified and update the
ktime_t members in timekeeping_update(). Converting the whole thing to
a pure nsec based mechanism and update the timespec stuff in
timekeeping_update() needs a lot more thought and we should do that
later on. It wont change any of the interfaces.
Thanks,
tglx
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [patch 14/55] timekeeping: Provide internal ktime_t based data
2014-07-16 7:12 ` Thomas Gleixner
@ 2014-07-16 7:20 ` Peter Zijlstra
0 siblings, 0 replies; 91+ messages in thread
From: Peter Zijlstra @ 2014-07-16 7:20 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: John Stultz, LKML, Arnd Bergmann
[-- Attachment #1: Type: text/plain, Size: 783 bytes --]
On Wed, Jul 16, 2014 at 09:12:52AM +0200, Thomas Gleixner wrote:
> Looking into it I think for now it's the least risky approach to keep
> the core logic based on the timespec stuff unmodified and update the
> ktime_t members in timekeeping_update(). Converting the whole thing to
> a pure nsec based mechanism and update the timespec stuff in
> timekeeping_update() needs a lot more thought and we should do that
> later on. It wont change any of the interfaces.
So I don't think you can only do nsec, seeing how the conversion from
nsec to timespec is expensive. So if we want to also avoid the timespec
-> nsec conversion we need to keep both, no two ways around it.
That said the timespec -> nsec conversion is heaps cheaper, although
still not what you call really sheep ;-)
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 91+ messages in thread
* [patch 15/55] timekeeping: Use ktime_t based data for ktime_get()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (13 preceding siblings ...)
2014-07-11 13:44 ` [patch 14/55] timekeeping: Provide internal ktime_t based data Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 16/55] timekeeping: Provide ktime_get_with_offset() Thomas Gleixner
` (39 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-use-ktime-based-for-ktime-get.patch --]
[-- Type: text/plain, Size: 1018 bytes --]
Speed up ktime_get() by using ktime_t based data. Text size shrinks by
64 bytes on x8664.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/time/timekeeping.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -406,18 +406,19 @@ ktime_t ktime_get(void)
{
struct timekeeper *tk = &tk_core.timekeeper;
unsigned int seq;
- s64 secs, nsecs;
+ ktime_t base;
+ s64 nsecs;
WARN_ON(timekeeping_suspended);
do {
seq = read_seqcount_begin(&tk_core.seq);
- secs = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
- nsecs = timekeeping_get_ns(tk) + tk->wall_to_monotonic.tv_nsec;
+ base = tk->base_mono;
+ nsecs = timekeeping_get_ns(tk);
} while (read_seqcount_retry(&tk_core.seq, seq));
- return ktime_set(secs, nsecs);
+ return ktime_add_ns(base, nsecs);
}
EXPORT_SYMBOL_GPL(ktime_get);
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 16/55] timekeeping: Provide ktime_get_with_offset()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (14 preceding siblings ...)
2014-07-11 13:44 ` [patch 15/55] timekeeping: Use ktime_t based data for ktime_get() Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 17/55] timekeeping: Use ktime_t based data for ktime_get_real() Thomas Gleixner
` (38 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-provide-__ktime_get.patch --]
[-- Type: text/plain, Size: 1958 bytes --]
Provide a helper function which lets us implement ktime_t based
interfaces for real, boot and tai clocks.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 9 +++++++++
kernel/time/timekeeping.c | 27 +++++++++++++++++++++++++++
2 files changed, 36 insertions(+)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -98,7 +98,16 @@ extern void getboottime(struct timespec
/*
* ktime_t based interfaces
*/
+
+enum tk_offsets {
+ TK_OFFS_REAL,
+ TK_OFFS_BOOT,
+ TK_OFFS_TAI,
+ TK_OFFS_MAX,
+};
+
extern ktime_t ktime_get(void);
+extern ktime_t ktime_get_with_offset(enum tk_offsets offs);
extern ktime_t ktime_get_real(void);
extern ktime_t ktime_get_boottime(void);
extern ktime_t ktime_get_monotonic_offset(void);
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -422,6 +422,33 @@ ktime_t ktime_get(void)
}
EXPORT_SYMBOL_GPL(ktime_get);
+static ktime_t *offsets[TK_OFFS_MAX] = {
+ [TK_OFFS_REAL] = &tk_core.timekeeper.offs_real,
+ [TK_OFFS_BOOT] = &tk_core.timekeeper.offs_boot,
+ [TK_OFFS_TAI] = &tk_core.timekeeper.offs_tai,
+};
+
+ktime_t ktime_get_with_offset(enum tk_offsets offs)
+{
+ struct timekeeper *tk = &tk_core.timekeeper;
+ unsigned int seq;
+ ktime_t base, *offset = offsets[offs];
+ s64 nsecs;
+
+ WARN_ON(timekeeping_suspended);
+
+ do {
+ seq = read_seqcount_begin(&tk_core.seq);
+ base = ktime_add(tk->base_mono, *offset);
+ nsecs = timekeeping_get_ns(tk);
+
+ } while (read_seqcount_retry(&tk_core.seq, seq));
+
+ return ktime_add_ns(base, nsecs);
+
+}
+EXPORT_SYMBOL_GPL(ktime_get_with_offset);
+
/**
* ktime_get_ts64 - get the monotonic clock in timespec64 format
* @ts: pointer to timespec variable
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 17/55] timekeeping: Use ktime_t based data for ktime_get_real()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (15 preceding siblings ...)
2014-07-11 13:44 ` [patch 16/55] timekeeping: Provide ktime_get_with_offset() Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 18/55] timekeeping; Use ktime_t based data for ktime_get_boottime() Thomas Gleixner
` (37 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-use-ktime-based-for-ktime-get-real.patch --]
[-- Type: text/plain, Size: 1571 bytes --]
Speed up the readout.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 9 ++++++++-
kernel/time/timekeeping.c | 15 ---------------
2 files changed, 8 insertions(+), 16 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -108,11 +108,18 @@ enum tk_offsets {
extern ktime_t ktime_get(void);
extern ktime_t ktime_get_with_offset(enum tk_offsets offs);
-extern ktime_t ktime_get_real(void);
extern ktime_t ktime_get_boottime(void);
extern ktime_t ktime_get_monotonic_offset(void);
extern ktime_t ktime_get_clocktai(void);
+/**
+ * ktime_get_real - get the real (wall-) time in ktime_t format
+ */
+static inline ktime_t ktime_get_real(void)
+{
+ return ktime_get_with_offset(TK_OFFS_REAL);
+}
+
/*
* RTC specific
*/
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -773,21 +773,6 @@ int timekeeping_notify(struct clocksourc
}
/**
- * ktime_get_real - get the real (wall-) time in ktime_t format
- *
- * returns the time in ktime_t format
- */
-ktime_t ktime_get_real(void)
-{
- struct timespec64 now;
-
- getnstimeofday64(&now);
-
- return timespec64_to_ktime(now);
-}
-EXPORT_SYMBOL_GPL(ktime_get_real);
-
-/**
* getrawmonotonic - Returns the raw monotonic time in a timespec
* @ts: pointer to the timespec to be set
*
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 18/55] timekeeping; Use ktime_t based data for ktime_get_boottime()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (16 preceding siblings ...)
2014-07-11 13:44 ` [patch 17/55] timekeeping: Use ktime_t based data for ktime_get_real() Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 19/55] timekeeping: Use ktime_t based data for ktime_get_clocktai() Thomas Gleixner
` (36 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-use-ktime-based-for-ktime-get-boottime.patch --]
[-- Type: text/plain, Size: 1925 bytes --]
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 12 +++++++++++-
kernel/time/timekeeping.c | 17 -----------------
2 files changed, 11 insertions(+), 18 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -108,7 +108,6 @@ enum tk_offsets {
extern ktime_t ktime_get(void);
extern ktime_t ktime_get_with_offset(enum tk_offsets offs);
-extern ktime_t ktime_get_boottime(void);
extern ktime_t ktime_get_monotonic_offset(void);
extern ktime_t ktime_get_clocktai(void);
@@ -120,6 +119,17 @@ static inline ktime_t ktime_get_real(voi
return ktime_get_with_offset(TK_OFFS_REAL);
}
+/**
+ * ktime_get_boottime - Returns monotonic time since boot in ktime_t format
+ *
+ * This is similar to CLOCK_MONTONIC/ktime_get, but also includes the
+ * time spent in suspend.
+ */
+static inline ktime_t ktime_get_boottime(void)
+{
+ return ktime_get_with_offset(TK_OFFS_BOOT);
+}
+
/*
* RTC specific
*/
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -1570,23 +1570,6 @@ void get_monotonic_boottime(struct times
EXPORT_SYMBOL_GPL(get_monotonic_boottime);
/**
- * ktime_get_boottime - Returns monotonic time since boot in a ktime
- *
- * Returns the monotonic time since boot in a ktime
- *
- * This is similar to CLOCK_MONTONIC/ktime_get, but also
- * includes the time spent in suspend.
- */
-ktime_t ktime_get_boottime(void)
-{
- struct timespec ts;
-
- get_monotonic_boottime(&ts);
- return timespec_to_ktime(ts);
-}
-EXPORT_SYMBOL_GPL(ktime_get_boottime);
-
-/**
* monotonic_to_bootbased - Convert the monotonic time to boot based.
* @ts: pointer to the timespec to be converted
*/
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 19/55] timekeeping: Use ktime_t based data for ktime_get_clocktai()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (17 preceding siblings ...)
2014-07-11 13:44 ` [patch 18/55] timekeeping; Use ktime_t based data for ktime_get_boottime() Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 20/55] timekeeping: Use ktime_t data for ktime_get_update_offsets_now() Thomas Gleixner
` (35 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-use-ktime-based-for-ktime-clocktai.patch --]
[-- Type: text/plain, Size: 1605 bytes --]
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 9 ++++++++-
kernel/time/timekeeping.c | 15 ---------------
2 files changed, 8 insertions(+), 16 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -109,7 +109,6 @@ enum tk_offsets {
extern ktime_t ktime_get(void);
extern ktime_t ktime_get_with_offset(enum tk_offsets offs);
extern ktime_t ktime_get_monotonic_offset(void);
-extern ktime_t ktime_get_clocktai(void);
/**
* ktime_get_real - get the real (wall-) time in ktime_t format
@@ -130,6 +129,14 @@ static inline ktime_t ktime_get_boottime
return ktime_get_with_offset(TK_OFFS_BOOT);
}
+/**
+ * ktime_get_clocktai - Returns the TAI time of day in ktime_t format
+ */
+static inline ktime_t ktime_get_clocktai(void)
+{
+ return ktime_get_with_offset(TK_OFFS_TAI);
+}
+
/*
* RTC specific
*/
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -511,21 +511,6 @@ void timekeeping_clocktai(struct timespe
}
EXPORT_SYMBOL(timekeeping_clocktai);
-
-/**
- * ktime_get_clocktai - Returns the TAI time of day in a ktime
- *
- * Returns the time of day in a ktime.
- */
-ktime_t ktime_get_clocktai(void)
-{
- struct timespec ts;
-
- timekeeping_clocktai(&ts);
- return timespec_to_ktime(ts);
-}
-EXPORT_SYMBOL(ktime_get_clocktai);
-
#ifdef CONFIG_NTP_PPS
/**
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 20/55] timekeeping: Use ktime_t data for ktime_get_update_offsets_now()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (18 preceding siblings ...)
2014-07-11 13:44 ` [patch 19/55] timekeeping: Use ktime_t based data for ktime_get_clocktai() Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 21/55] timekeeping; Use ktime based data for ktime_get_update_offsets_tick() Thomas Gleixner
` (34 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-use-ktime-for-hrtimer-if.patch --]
[-- Type: text/plain, Size: 1058 bytes --]
No need to juggle with timespecs.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/time/timekeeping.c | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -1672,14 +1672,14 @@ ktime_t ktime_get_update_offsets_now(kti
ktime_t *offs_tai)
{
struct timekeeper *tk = &tk_core.timekeeper;
- ktime_t now;
unsigned int seq;
- u64 secs, nsecs;
+ ktime_t base;
+ u64 nsecs;
do {
seq = read_seqcount_begin(&tk_core.seq);
- secs = tk->xtime_sec;
+ base = tk->base_mono;
nsecs = timekeeping_get_ns(tk);
*offs_real = tk->offs_real;
@@ -1687,9 +1687,7 @@ ktime_t ktime_get_update_offsets_now(kti
*offs_tai = tk->offs_tai;
} while (read_seqcount_retry(&tk_core.seq, seq));
- now = ktime_add_ns(ktime_set(secs, 0), nsecs);
- now = ktime_sub(now, *offs_real);
- return now;
+ return ktime_add_ns(base, nsecs);
}
#endif
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 21/55] timekeeping; Use ktime based data for ktime_get_update_offsets_tick()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (19 preceding siblings ...)
2014-07-11 13:44 ` [patch 20/55] timekeeping: Use ktime_t data for ktime_get_update_offsets_now() Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 22/55] timekeeping: Provide ktime_mono_to_any() Thomas Gleixner
` (33 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-use-ktime-for-hrtimer-if-2.patch --]
[-- Type: text/plain, Size: 1051 bytes --]
No need to juggle with timespecs.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/time/timekeeping.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -1640,22 +1640,22 @@ ktime_t ktime_get_update_offsets_tick(kt
ktime_t *offs_tai)
{
struct timekeeper *tk = &tk_core.timekeeper;
- struct timespec64 ts;
- ktime_t now;
unsigned int seq;
+ ktime_t base;
+ u64 nsecs;
do {
seq = read_seqcount_begin(&tk_core.seq);
- ts = tk_xtime(tk);
+ base = tk->base_mono;
+ nsecs = tk->xtime_nsec >> tk->shift;
+
*offs_real = tk->offs_real;
*offs_boot = tk->offs_boot;
*offs_tai = tk->offs_tai;
} while (read_seqcount_retry(&tk_core.seq, seq));
- now = ktime_set(ts.tv_sec, ts.tv_nsec);
- now = ktime_sub(now, *offs_real);
- return now;
+ return ktime_add_ns(base, nsecs);
}
#ifdef CONFIG_HIGH_RES_TIMERS
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 22/55] timekeeping: Provide ktime_mono_to_any()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (20 preceding siblings ...)
2014-07-11 13:44 ` [patch 21/55] timekeeping; Use ktime based data for ktime_get_update_offsets_tick() Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 23/55] timerfd: Use ktime_mono_to_real() Thomas Gleixner
` (32 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-provide-ktime-mono_to-wall.patch --]
[-- Type: text/plain, Size: 1999 bytes --]
ktime based conversion function to map a monotonic time stamp to a
different CLOCK.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 9 +++++++++
kernel/time/timekeeping.c | 20 ++++++++++++++++++++
2 files changed, 29 insertions(+)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -109,6 +109,7 @@ enum tk_offsets {
extern ktime_t ktime_get(void);
extern ktime_t ktime_get_with_offset(enum tk_offsets offs);
extern ktime_t ktime_get_monotonic_offset(void);
+extern ktime_t ktime_mono_to_any(ktime_t tmono, enum tk_offsets offs);
/**
* ktime_get_real - get the real (wall-) time in ktime_t format
@@ -137,6 +138,14 @@ static inline ktime_t ktime_get_clocktai
return ktime_get_with_offset(TK_OFFS_TAI);
}
+/**
+ * ktime_mono_to_real - Convert monotonic time to clock realtime
+ */
+static inline ktime_t ktime_mono_to_real(ktime_t mono)
+{
+ return ktime_mono_to_any(mono, TK_OFFS_REAL);
+}
+
/*
* RTC specific
*/
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -450,6 +450,26 @@ ktime_t ktime_get_with_offset(enum tk_of
EXPORT_SYMBOL_GPL(ktime_get_with_offset);
/**
+ * ktime_mono_to_any() - convert mononotic time to any other time
+ * @tmono: time to convert.
+ * @offs: which offset to use
+ */
+ktime_t ktime_mono_to_any(ktime_t tmono, enum tk_offsets offs)
+{
+ ktime_t *offset = offsets[offs];
+ unsigned long seq;
+ ktime_t tconv;
+
+ do {
+ seq = read_seqcount_begin(&tk_core.seq);
+ tconv = ktime_add(tmono, *offset);
+ } while (read_seqcount_retry(&tk_core.seq, seq));
+
+ return tconv;
+}
+EXPORT_SYMBOL_GPL(ktime_mono_to_any);
+
+/**
* ktime_get_ts64 - get the monotonic clock in timespec64 format
* @ts: pointer to timespec variable
*
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 23/55] timerfd: Use ktime_mono_to_real()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (21 preceding siblings ...)
2014-07-11 13:44 ` [patch 22/55] timekeeping: Provide ktime_mono_to_any() Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 24/55] input: evdev: " Thomas Gleixner
` (31 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: fs-timerfd-use-ktime-mono-to-real.patch --]
[-- Type: text/plain, Size: 1430 bytes --]
We have a few other use cases of ktime_get_monotonic_offset() which
can be optimized with ktime_mono_to_real(). The timerfd code uses the
offset only for comparison, so we can use ktime_mono_to_real(0) for
this as well.
Funny enough text size shrinks with that on ARM and x8664 !?
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
fs/timerfd.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
Index: tip/fs/timerfd.c
===================================================================
--- tip.orig/fs/timerfd.c
+++ tip/fs/timerfd.c
@@ -92,7 +92,7 @@ static enum alarmtimer_restart timerfd_a
*/
void timerfd_clock_was_set(void)
{
- ktime_t moffs = ktime_get_monotonic_offset();
+ ktime_t moffs = ktime_mono_to_real((ktime_t){ .tv64 = 0 });
struct timerfd_ctx *ctx;
unsigned long flags;
@@ -125,7 +125,7 @@ static bool timerfd_canceled(struct time
{
if (!ctx->might_cancel || ctx->moffs.tv64 != KTIME_MAX)
return false;
- ctx->moffs = ktime_get_monotonic_offset();
+ ctx->moffs = ktime_mono_to_real((ktime_t){ .tv64 = 0 });
return true;
}
@@ -336,7 +336,7 @@ SYSCALL_DEFINE2(timerfd_create, int, clo
else
hrtimer_init(&ctx->t.tmr, clockid, HRTIMER_MODE_ABS);
- ctx->moffs = ktime_get_monotonic_offset();
+ ctx->moffs = ktime_mono_to_real((ktime_t){ .tv64 = 0 });
ufd = anon_inode_getfd("[timerfd]", &timerfd_fops, ctx,
O_RDWR | (flags & TFD_SHARED_FCNTL_FLAGS));
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 24/55] input: evdev: Use ktime_mono_to_real()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (22 preceding siblings ...)
2014-07-11 13:44 ` [patch 23/55] timerfd: Use ktime_mono_to_real() Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 25/55] drm: " Thomas Gleixner
` (30 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: input-evdev-use-ktime-mono-to-real.patch --]
[-- Type: text/plain, Size: 1162 bytes --]
Convert the monotonic timestamp with ktime_mono_to_real() in
evdev_events().
In evdev_queue_syn_dropped() we can call either ktime_get() or
ktime_get_real() depending on the clkid. No point in having two calls
for CLOCK_REALTIME.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
drivers/input/evdev.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
Index: tip/drivers/input/evdev.c
===================================================================
--- tip.orig/drivers/input/evdev.c
+++ tip/drivers/input/evdev.c
@@ -108,9 +108,8 @@ static void evdev_queue_syn_dropped(stru
struct input_event ev;
ktime_t time;
- time = ktime_get();
- if (client->clkid != CLOCK_MONOTONIC)
- time = ktime_sub(time, ktime_get_monotonic_offset());
+ time = (client->clkid == CLOCK_MONOTONIC) ?
+ ktime_get() : ktime_get_real();
ev.time = ktime_to_timeval(time);
ev.type = EV_SYN;
@@ -202,7 +201,7 @@ static void evdev_events(struct input_ha
ktime_t time_mono, time_real;
time_mono = ktime_get();
- time_real = ktime_sub(time_mono, ktime_get_monotonic_offset());
+ time_real = ktime_mono_to_real(time_mono);
rcu_read_lock();
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 25/55] drm: Use ktime_mono_to_real()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (23 preceding siblings ...)
2014-07-11 13:44 ` [patch 24/55] input: evdev: " Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 26/55] timekeeping: Remove ktime_get_monotonic_offset() Thomas Gleixner
` (29 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: drm-use-ktime-mono-to-real.patch --]
[-- Type: text/plain, Size: 2123 bytes --]
Convert the monotonic timestamp with ktime_mono_to_real() in
drm_calc_vbltimestamp_from_scanoutpos().
In get_drm_timestamp we can call either ktime_get() or
ktime_get_real() depending on drm_timestamp_monotonic. No point in
having two calls into the core for CLOCK_REALTIME.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
drivers/gpu/drm/drm_irq.c | 16 +++-------------
1 file changed, 3 insertions(+), 13 deletions(-)
Index: tip/drivers/gpu/drm/drm_irq.c
===================================================================
--- tip.orig/drivers/gpu/drm/drm_irq.c
+++ tip/drivers/gpu/drm/drm_irq.c
@@ -542,8 +542,8 @@ int drm_calc_vbltimestamp_from_scanoutpo
const struct drm_crtc *refcrtc,
const struct drm_display_mode *mode)
{
- ktime_t stime, etime, mono_time_offset;
struct timeval tv_etime;
+ ktime_t stime, etime;
int vbl_status;
int vpos, hpos, i;
int framedur_ns, linedur_ns, pixeldur_ns, delta_ns, duration_ns;
@@ -588,13 +588,6 @@ int drm_calc_vbltimestamp_from_scanoutpo
vbl_status = dev->driver->get_scanout_position(dev, crtc, flags, &vpos,
&hpos, &stime, &etime);
- /*
- * Get correction for CLOCK_MONOTONIC -> CLOCK_REALTIME if
- * CLOCK_REALTIME is requested.
- */
- if (!drm_timestamp_monotonic)
- mono_time_offset = ktime_get_monotonic_offset();
-
/* Return as no-op if scanout query unsupported or failed. */
if (!(vbl_status & DRM_SCANOUTPOS_VALID)) {
DRM_DEBUG("crtc %d : scanoutpos query failed [%d].\n",
@@ -633,7 +626,7 @@ int drm_calc_vbltimestamp_from_scanoutpo
delta_ns = vpos * linedur_ns + hpos * pixeldur_ns;
if (!drm_timestamp_monotonic)
- etime = ktime_sub(etime, mono_time_offset);
+ etime = ktime_mono_to_real(etime);
/* save this only for debugging purposes */
tv_etime = ktime_to_timeval(etime);
@@ -664,10 +657,7 @@ static struct timeval get_drm_timestamp(
{
ktime_t now;
- now = ktime_get();
- if (!drm_timestamp_monotonic)
- now = ktime_sub(now, ktime_get_monotonic_offset());
-
+ now = drm_timestamp_monotonic ? ktime_get() : ktime_get_real();
return ktime_to_timeval(now);
}
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 26/55] timekeeping: Remove ktime_get_monotonic_offset()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (24 preceding siblings ...)
2014-07-11 13:44 ` [patch 25/55] drm: " Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 27/55] timekeeping: Provide ktime_get[*]_ns() helpers Thomas Gleixner
` (28 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-remove-ktime_get_monotonic_offset.patch --]
[-- Type: text/plain, Size: 1468 bytes --]
No more users.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 1 -
kernel/time/timekeeping.c | 18 ------------------
2 files changed, 19 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -108,7 +108,6 @@ enum tk_offsets {
extern ktime_t ktime_get(void);
extern ktime_t ktime_get_with_offset(enum tk_offsets offs);
-extern ktime_t ktime_get_monotonic_offset(void);
extern ktime_t ktime_mono_to_any(ktime_t tmono, enum tk_offsets offs);
/**
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -1712,24 +1712,6 @@ ktime_t ktime_get_update_offsets_now(kti
#endif
/**
- * ktime_get_monotonic_offset() - get wall_to_monotonic in ktime_t format
- */
-ktime_t ktime_get_monotonic_offset(void)
-{
- struct timekeeper *tk = &tk_core.timekeeper;
- unsigned long seq;
- struct timespec64 wtom;
-
- do {
- seq = read_seqcount_begin(&tk_core.seq);
- wtom = tk->wall_to_monotonic;
- } while (read_seqcount_retry(&tk_core.seq, seq));
-
- return timespec64_to_ktime(wtom);
-}
-EXPORT_SYMBOL_GPL(ktime_get_monotonic_offset);
-
-/**
* do_adjtimex() - Accessor function to NTP __do_adjtimex function
*/
int do_adjtimex(struct timex *txc)
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 27/55] timekeeping: Provide ktime_get[*]_ns() helpers
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (25 preceding siblings ...)
2014-07-11 13:44 ` [patch 26/55] timekeeping: Remove ktime_get_monotonic_offset() Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 28/55] time-export-nsecs-to-jiffies.patch Thomas Gleixner
` (27 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-provide-ktime-get-ns.patch --]
[-- Type: text/plain, Size: 831 bytes --]
A lot of code converts either timespecs or ktime_t to
nanoseconds. Provide helper functions.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 15 +++++++++++++++
1 file changed, 15 insertions(+)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -145,6 +145,21 @@ static inline ktime_t ktime_mono_to_real
return ktime_mono_to_any(mono, TK_OFFS_REAL);
}
+static inline u64 ktime_get_ns(void)
+{
+ return ktime_to_ns(ktime_get());
+}
+
+static inline u64 ktime_get_real_ns(void)
+{
+ return ktime_to_ns(ktime_get_real());
+}
+
+static inline u64 ktime_get_boot_ns(void)
+{
+ return ktime_to_ns(ktime_get_boottime());
+}
+
/*
* RTC specific
*/
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 28/55] time-export-nsecs-to-jiffies.patch
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (26 preceding siblings ...)
2014-07-11 13:44 ` [patch 27/55] timekeeping: Provide ktime_get[*]_ns() helpers Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 29/55] sched: Make task->real_start_time nanoseconds based Thomas Gleixner
` (26 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: time-export-nsecs-to-jiffies.patch --]
[-- Type: text/plain, Size: 496 bytes --]
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/time/time.c | 1 +
1 file changed, 1 insertion(+)
Index: tip/kernel/time/time.c
===================================================================
--- tip.orig/kernel/time/time.c
+++ tip/kernel/time/time.c
@@ -757,6 +757,7 @@ unsigned long nsecs_to_jiffies(u64 n)
{
return (unsigned long)nsecs_to_jiffies64(n);
}
+EXPORT_SYMBOL_GPL(nsecs_to_jiffies);
/*
* Add two timespec values and do a safety check for overflow.
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 29/55] sched: Make task->real_start_time nanoseconds based
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (27 preceding siblings ...)
2014-07-11 13:44 ` [patch 28/55] time-export-nsecs-to-jiffies.patch Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 30/55] sched: Make task->start_time " Thomas Gleixner
` (25 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: sched-make-real-start-time-ktime-t.patch --]
[-- Type: text/plain, Size: 2040 bytes --]
Simplify the only user of this data by removing the timespec
conversion.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
fs/proc/array.c | 7 +------
include/linux/sched.h | 2 +-
kernel/fork.c | 3 +--
3 files changed, 3 insertions(+), 9 deletions(-)
Index: tip/fs/proc/array.c
===================================================================
--- tip.orig/fs/proc/array.c
+++ tip/fs/proc/array.c
@@ -473,13 +473,8 @@ static int do_task_stat(struct seq_file
priority = task_prio(task);
nice = task_nice(task);
- /* Temporary variable needed for gcc-2.96 */
- /* convert timespec -> nsec*/
- start_time =
- (unsigned long long)task->real_start_time.tv_sec * NSEC_PER_SEC
- + task->real_start_time.tv_nsec;
/* convert nsec -> ticks */
- start_time = nsec_to_clock_t(start_time);
+ start_time = nsec_to_clock_t(task->real_start_time);
seq_printf(m, "%d (%s) %c", pid_nr_ns(pid, ns), tcomm, state);
seq_put_decimal_ll(m, ' ', ppid);
Index: tip/include/linux/sched.h
===================================================================
--- tip.orig/include/linux/sched.h
+++ tip/include/linux/sched.h
@@ -1368,7 +1368,7 @@ struct task_struct {
#endif
unsigned long nvcsw, nivcsw; /* context switch counts */
struct timespec start_time; /* monotonic time */
- struct timespec real_start_time; /* boot based time */
+ u64 real_start_time; /* boot based time in nsec */
/* mm fault and swap info: this can arguably be seen as either mm-specific or thread-specific */
unsigned long min_flt, maj_flt;
Index: tip/kernel/fork.c
===================================================================
--- tip.orig/kernel/fork.c
+++ tip/kernel/fork.c
@@ -1263,8 +1263,7 @@ static struct task_struct *copy_process(
posix_cpu_timers_init(p);
ktime_get_ts(&p->start_time);
- p->real_start_time = p->start_time;
- monotonic_to_bootbased(&p->real_start_time);
+ p->real_start_time = ktime_get_boot_ns();
p->io_context = NULL;
p->audit_context = NULL;
if (clone_flags & CLONE_THREAD)
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 30/55] sched: Make task->start_time nanoseconds based
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (28 preceding siblings ...)
2014-07-11 13:44 ` [patch 29/55] sched: Make task->real_start_time nanoseconds based Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 31/55] delayacct: Make accounting nanosecond based Thomas Gleixner
` (24 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: sched-make-start-time-nsec.patch --]
[-- Type: text/plain, Size: 3419 bytes --]
Simplify the timespec to nsec/usec conversions.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/sched.h | 2 +-
kernel/acct.c | 10 +++-------
kernel/fork.c | 2 +-
kernel/tsacct.c | 19 +++++++++----------
4 files changed, 14 insertions(+), 19 deletions(-)
Index: tip/include/linux/sched.h
===================================================================
--- tip.orig/include/linux/sched.h
+++ tip/include/linux/sched.h
@@ -1367,7 +1367,7 @@ struct task_struct {
} vtime_snap_whence;
#endif
unsigned long nvcsw, nivcsw; /* context switch counts */
- struct timespec start_time; /* monotonic time */
+ u64 start_time; /* monotonic time in nsec */
u64 real_start_time; /* boot based time in nsec */
/* mm fault and swap info: this can arguably be seen as either mm-specific or thread-specific */
unsigned long min_flt, maj_flt;
Index: tip/kernel/acct.c
===================================================================
--- tip.orig/kernel/acct.c
+++ tip/kernel/acct.c
@@ -458,9 +458,7 @@ static void do_acct_process(struct bsd_a
acct_t ac;
mm_segment_t fs;
unsigned long flim;
- u64 elapsed;
- u64 run_time;
- struct timespec uptime;
+ u64 elapsed, run_time;
struct tty_struct *tty;
const struct cred *orig_cred;
@@ -484,10 +482,8 @@ static void do_acct_process(struct bsd_a
strlcpy(ac.ac_comm, current->comm, sizeof(ac.ac_comm));
/* calculate run_time in nsec*/
- ktime_get_ts(&uptime);
- run_time = (u64)uptime.tv_sec*NSEC_PER_SEC + uptime.tv_nsec;
- run_time -= (u64)current->group_leader->start_time.tv_sec * NSEC_PER_SEC
- + current->group_leader->start_time.tv_nsec;
+ run_time = ktime_get_ns();
+ run_time -= current->group_leader->start_time;
/* convert nsec -> AHZ */
elapsed = nsec_to_AHZ(run_time);
#if ACCT_VERSION==3
Index: tip/kernel/fork.c
===================================================================
--- tip.orig/kernel/fork.c
+++ tip/kernel/fork.c
@@ -1262,7 +1262,7 @@ static struct task_struct *copy_process(
posix_cpu_timers_init(p);
- ktime_get_ts(&p->start_time);
+ p->start_time = ktime_get_ns();
p->real_start_time = ktime_get_boot_ns();
p->io_context = NULL;
p->audit_context = NULL;
Index: tip/kernel/tsacct.c
===================================================================
--- tip.orig/kernel/tsacct.c
+++ tip/kernel/tsacct.c
@@ -31,20 +31,19 @@ void bacct_add_tsk(struct user_namespace
struct taskstats *stats, struct task_struct *tsk)
{
const struct cred *tcred;
- struct timespec uptime, ts;
cputime_t utime, stime, utimescaled, stimescaled;
- u64 ac_etime;
+ u64 delta;
BUILD_BUG_ON(TS_COMM_LEN < TASK_COMM_LEN);
- /* calculate task elapsed time in timespec */
- ktime_get_ts(&uptime);
- ts = timespec_sub(uptime, tsk->start_time);
- /* rebase elapsed time to usec (should never be negative) */
- ac_etime = timespec_to_ns(&ts);
- do_div(ac_etime, NSEC_PER_USEC);
- stats->ac_etime = ac_etime;
- stats->ac_btime = get_seconds() - ts.tv_sec;
+ /* calculate task elapsed time in nsec */
+ delta = ktime_get_ns() - tsk->start_time;
+ /* Convert to micro seconds */
+ do_div(delta, NSEC_PER_USEC);
+ stats->ac_etime = delta;
+ /* Convert to seconds for btime */
+ do_div(delta, USEC_PER_SEC);
+ stats->ac_btime = get_seconds() - delta;
if (thread_group_leader(tsk)) {
stats->ac_exitcode = tsk->exit_code;
if (tsk->flags & PF_FORKNOEXEC)
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 31/55] delayacct: Make accounting nanosecond based
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (29 preceding siblings ...)
2014-07-11 13:44 ` [patch 30/55] sched: Make task->start_time " Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 32/55] delayacct: Remove braindamaged type conversions Thomas Gleixner
` (23 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: delayacct-use-ktimet.patch --]
[-- Type: text/plain, Size: 3477 bytes --]
Kill the timespec juggling and calculate with plain nanoseconds.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/sched.h | 4 ++--
kernel/delayacct.c | 34 ++++++++++++----------------------
2 files changed, 14 insertions(+), 24 deletions(-)
Index: tip/include/linux/sched.h
===================================================================
--- tip.orig/include/linux/sched.h
+++ tip/include/linux/sched.h
@@ -813,7 +813,7 @@ struct task_delay_info {
* associated with the operation is added to XXX_delay.
* XXX_delay contains the accumulated delay time in nanoseconds.
*/
- struct timespec blkio_start, blkio_end; /* Shared by blkio, swapin */
+ u64 blkio_start; /* Shared by blkio, swapin */
u64 blkio_delay; /* wait for sync block io completion */
u64 swapin_delay; /* wait for swapin block io completion */
u32 blkio_count; /* total count of the number of sync block */
@@ -821,7 +821,7 @@ struct task_delay_info {
u32 swapin_count; /* total count of the number of swapin block */
/* io operations performed */
- struct timespec freepages_start, freepages_end;
+ u64 freepages_start;
u64 freepages_delay; /* wait for memory reclaim */
u32 freepages_count; /* total count of memory reclaim */
};
Index: tip/kernel/delayacct.c
===================================================================
--- tip.orig/kernel/delayacct.c
+++ tip/kernel/delayacct.c
@@ -46,32 +46,25 @@ void __delayacct_tsk_init(struct task_st
}
/*
- * Finish delay accounting for a statistic using
- * its timestamps (@start, @end), accumalator (@total) and @count
+ * Finish delay accounting for a statistic using its timestamps (@start),
+ * accumalator (@total) and @count
*/
-
-static void delayacct_end(struct timespec *start, struct timespec *end,
- u64 *total, u32 *count)
+static void delayacct_end(u64 *start, u64 *total, u32 *count)
{
- struct timespec ts;
- s64 ns;
+ s64 ns = ktime_get_ns() - *start;
unsigned long flags;
- ktime_get_ts(end);
- ts = timespec_sub(*end, *start);
- ns = timespec_to_ns(&ts);
- if (ns < 0)
- return;
-
- spin_lock_irqsave(¤t->delays->lock, flags);
- *total += ns;
- (*count)++;
- spin_unlock_irqrestore(¤t->delays->lock, flags);
+ if (ns > 0) {
+ spin_lock_irqsave(¤t->delays->lock, flags);
+ *total += ns;
+ (*count)++;
+ spin_unlock_irqrestore(¤t->delays->lock, flags);
+ }
}
void __delayacct_blkio_start(void)
{
- ktime_get_ts(¤t->delays->blkio_start);
+ current->delays->blkio_start = ktime_get_ns();
}
void __delayacct_blkio_end(void)
@@ -79,12 +72,10 @@ void __delayacct_blkio_end(void)
if (current->delays->flags & DELAYACCT_PF_SWAPIN)
/* Swapin block I/O */
delayacct_end(¤t->delays->blkio_start,
- ¤t->delays->blkio_end,
¤t->delays->swapin_delay,
¤t->delays->swapin_count);
else /* Other block I/O */
delayacct_end(¤t->delays->blkio_start,
- ¤t->delays->blkio_end,
¤t->delays->blkio_delay,
¤t->delays->blkio_count);
}
@@ -159,13 +150,12 @@ __u64 __delayacct_blkio_ticks(struct tas
void __delayacct_freepages_start(void)
{
- ktime_get_ts(¤t->delays->freepages_start);
+ current->delays->freepages_start = ktime_get_ns();
}
void __delayacct_freepages_end(void)
{
delayacct_end(¤t->delays->freepages_start,
- ¤t->delays->freepages_end,
¤t->delays->freepages_delay,
¤t->delays->freepages_count);
}
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 32/55] delayacct: Remove braindamaged type conversions
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (30 preceding siblings ...)
2014-07-11 13:44 ` [patch 31/55] delayacct: Make accounting nanosecond based Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 33/55] powerpc: cell: Use ktime_get_ns() Thomas Gleixner
` (22 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: delayacct-simplify.patch --]
[-- Type: text/plain, Size: 1454 bytes --]
Converting cputime to timespec and timespec to nanoseconds makes no
sense. Use cputime_to_ns() and be done with it.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/delayacct.c | 18 +++++++-----------
1 file changed, 7 insertions(+), 11 deletions(-)
Index: tip/kernel/delayacct.c
===================================================================
--- tip.orig/kernel/delayacct.c
+++ tip/kernel/delayacct.c
@@ -82,23 +82,19 @@ void __delayacct_blkio_end(void)
int __delayacct_add_tsk(struct taskstats *d, struct task_struct *tsk)
{
- s64 tmp;
- unsigned long t1;
- unsigned long long t2, t3;
- unsigned long flags;
- struct timespec ts;
cputime_t utime, stime, stimescaled, utimescaled;
+ unsigned long long t2, t3;
+ unsigned long flags, t1;
+ s64 tmp;
- tmp = (s64)d->cpu_run_real_total;
task_cputime(tsk, &utime, &stime);
- cputime_to_timespec(utime + stime, &ts);
- tmp += timespec_to_ns(&ts);
+ tmp = (s64)d->cpu_run_real_total;
+ tmp += cputime_to_nsecs(utime + stime);
d->cpu_run_real_total = (tmp < (s64)d->cpu_run_real_total) ? 0 : tmp;
- tmp = (s64)d->cpu_scaled_run_real_total;
task_cputime_scaled(tsk, &utimescaled, &stimescaled);
- cputime_to_timespec(utimescaled + stimescaled, &ts);
- tmp += timespec_to_ns(&ts);
+ tmp = (s64)d->cpu_scaled_run_real_total;
+ tmp += cputime_to_nsecs(utimescaled + stimescaled);
d->cpu_scaled_run_real_total =
(tmp < (s64)d->cpu_scaled_run_real_total) ? 0 : tmp;
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 33/55] powerpc: cell: Use ktime_get_ns()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (31 preceding siblings ...)
2014-07-11 13:44 ` [patch 32/55] delayacct: Remove braindamaged type conversions Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 17:02 ` Arnd Bergmann
2014-07-11 13:44 ` [patch 34/55] connector: " Thomas Gleixner
` (21 subsequent siblings)
54 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Arnd Bergmann
[-- Attachment #1: powerpc-cell-remove-silly-timespec-dance.patch --]
[-- Type: text/plain, Size: 4010 bytes --]
Replace the ever recurring:
ts = ktime_get_ts();
ns = timespec_to_ns(&ts);
with
ns = ktime_get_ns();
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Arnd Bergmann <arnd@arndb.de>
---
arch/powerpc/platforms/cell/spu_base.c | 11 +++--------
arch/powerpc/platforms/cell/spufs/context.c | 4 +---
arch/powerpc/platforms/cell/spufs/file.c | 4 +---
arch/powerpc/platforms/cell/spufs/sched.c | 4 +---
4 files changed, 6 insertions(+), 17 deletions(-)
Index: tip/arch/powerpc/platforms/cell/spu_base.c
===================================================================
--- tip.orig/arch/powerpc/platforms/cell/spu_base.c
+++ tip/arch/powerpc/platforms/cell/spu_base.c
@@ -611,7 +611,6 @@ static int __init create_spu(void *data)
int ret;
static int number;
unsigned long flags;
- struct timespec ts;
ret = -ENOMEM;
spu = kzalloc(sizeof (*spu), GFP_KERNEL);
@@ -652,8 +651,7 @@ static int __init create_spu(void *data)
mutex_unlock(&spu_full_list_mutex);
spu->stats.util_state = SPU_UTIL_IDLE_LOADED;
- ktime_get_ts(&ts);
- spu->stats.tstamp = timespec_to_ns(&ts);
+ spu->stats.tstamp = ktime_get_ns();
INIT_LIST_HEAD(&spu->aff_list);
@@ -676,7 +674,6 @@ static const char *spu_state_names[] = {
static unsigned long long spu_acct_time(struct spu *spu,
enum spu_utilization_state state)
{
- struct timespec ts;
unsigned long long time = spu->stats.times[state];
/*
@@ -684,10 +681,8 @@ static unsigned long long spu_acct_time(
* statistics are not updated. Apply the time delta from the
* last recorded state of the spu.
*/
- if (spu->stats.util_state == state) {
- ktime_get_ts(&ts);
- time += timespec_to_ns(&ts) - spu->stats.tstamp;
- }
+ if (spu->stats.util_state == state)
+ time += ktime_get_ns() - spu->stats.tstamp;
return time / NSEC_PER_MSEC;
}
Index: tip/arch/powerpc/platforms/cell/spufs/context.c
===================================================================
--- tip.orig/arch/powerpc/platforms/cell/spufs/context.c
+++ tip/arch/powerpc/platforms/cell/spufs/context.c
@@ -36,7 +36,6 @@ atomic_t nr_spu_contexts = ATOMIC_INIT(0
struct spu_context *alloc_spu_context(struct spu_gang *gang)
{
struct spu_context *ctx;
- struct timespec ts;
ctx = kzalloc(sizeof *ctx, GFP_KERNEL);
if (!ctx)
@@ -67,8 +66,7 @@ struct spu_context *alloc_spu_context(st
__spu_update_sched_info(ctx);
spu_set_timeslice(ctx);
ctx->stats.util_state = SPU_UTIL_IDLE_LOADED;
- ktime_get_ts(&ts);
- ctx->stats.tstamp = timespec_to_ns(&ts);
+ ctx->stats.tstamp = ktime_get_ns();
atomic_inc(&nr_spu_contexts);
goto out;
Index: tip/arch/powerpc/platforms/cell/spufs/file.c
===================================================================
--- tip.orig/arch/powerpc/platforms/cell/spufs/file.c
+++ tip/arch/powerpc/platforms/cell/spufs/file.c
@@ -2338,7 +2338,6 @@ static const char *ctx_state_names[] = {
static unsigned long long spufs_acct_time(struct spu_context *ctx,
enum spu_utilization_state state)
{
- struct timespec ts;
unsigned long long time = ctx->stats.times[state];
/*
@@ -2351,8 +2350,7 @@ static unsigned long long spufs_acct_tim
* of the spu context.
*/
if (ctx->spu && ctx->stats.util_state == state) {
- ktime_get_ts(&ts);
- time += timespec_to_ns(&ts) - ctx->stats.tstamp;
+ time += ktime_get_ns() - ctx->stats.tstamp;
}
return time / NSEC_PER_MSEC;
Index: tip/arch/powerpc/platforms/cell/spufs/sched.c
===================================================================
--- tip.orig/arch/powerpc/platforms/cell/spufs/sched.c
+++ tip/arch/powerpc/platforms/cell/spufs/sched.c
@@ -1039,13 +1039,11 @@ void spuctx_switch_state(struct spu_cont
{
unsigned long long curtime;
signed long long delta;
- struct timespec ts;
struct spu *spu;
enum spu_utilization_state old_state;
int node;
- ktime_get_ts(&ts);
- curtime = timespec_to_ns(&ts);
+ curtime = ktime_get_ns();
delta = curtime - ctx->stats.tstamp;
WARN_ON(!mutex_is_locked(&ctx->state_mutex));
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 34/55] connector: Use ktime_get_ns()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (32 preceding siblings ...)
2014-07-11 13:44 ` [patch 33/55] powerpc: cell: Use ktime_get_ns() Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 35/55] mfd: cros_ec_spi: " Thomas Gleixner
` (20 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Evgeniy Polyakov
[-- Attachment #1: connector-remove-silly-timestamp-dance.patch --]
[-- Type: text/plain, Size: 6565 bytes --]
Replace the ever recurring:
ts = ktime_get_ts();
ns = timespec_to_ns(&ts);
with
ns = ktime_get_ns();
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Evgeniy Polyakov <zbr@ioremap.net>
---
drivers/connector/cn_proc.c | 36 +++++++++---------------------------
1 file changed, 9 insertions(+), 27 deletions(-)
Index: tip/drivers/connector/cn_proc.c
===================================================================
--- tip.orig/drivers/connector/cn_proc.c
+++ tip/drivers/connector/cn_proc.c
@@ -69,7 +69,6 @@ void proc_fork_connector(struct task_str
struct cn_msg *msg;
struct proc_event *ev;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
- struct timespec ts;
struct task_struct *parent;
if (atomic_read(&proc_event_num_listeners) < 1)
@@ -79,8 +78,7 @@ void proc_fork_connector(struct task_str
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->what = PROC_EVENT_FORK;
rcu_read_lock();
parent = rcu_dereference(task->real_parent);
@@ -102,7 +100,6 @@ void proc_exec_connector(struct task_str
{
struct cn_msg *msg;
struct proc_event *ev;
- struct timespec ts;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
if (atomic_read(&proc_event_num_listeners) < 1)
@@ -112,8 +109,7 @@ void proc_exec_connector(struct task_str
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->what = PROC_EVENT_EXEC;
ev->event_data.exec.process_pid = task->pid;
ev->event_data.exec.process_tgid = task->tgid;
@@ -130,7 +126,6 @@ void proc_id_connector(struct task_struc
struct cn_msg *msg;
struct proc_event *ev;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
- struct timespec ts;
const struct cred *cred;
if (atomic_read(&proc_event_num_listeners) < 1)
@@ -156,8 +151,7 @@ void proc_id_connector(struct task_struc
}
rcu_read_unlock();
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
memcpy(&msg->id, &cn_proc_event_id, sizeof(msg->id));
msg->ack = 0; /* not used */
@@ -170,7 +164,6 @@ void proc_sid_connector(struct task_stru
{
struct cn_msg *msg;
struct proc_event *ev;
- struct timespec ts;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
if (atomic_read(&proc_event_num_listeners) < 1)
@@ -180,8 +173,7 @@ void proc_sid_connector(struct task_stru
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->what = PROC_EVENT_SID;
ev->event_data.sid.process_pid = task->pid;
ev->event_data.sid.process_tgid = task->tgid;
@@ -197,7 +189,6 @@ void proc_ptrace_connector(struct task_s
{
struct cn_msg *msg;
struct proc_event *ev;
- struct timespec ts;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
if (atomic_read(&proc_event_num_listeners) < 1)
@@ -207,8 +198,7 @@ void proc_ptrace_connector(struct task_s
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->what = PROC_EVENT_PTRACE;
ev->event_data.ptrace.process_pid = task->pid;
ev->event_data.ptrace.process_tgid = task->tgid;
@@ -232,7 +222,6 @@ void proc_comm_connector(struct task_str
{
struct cn_msg *msg;
struct proc_event *ev;
- struct timespec ts;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
if (atomic_read(&proc_event_num_listeners) < 1)
@@ -242,8 +231,7 @@ void proc_comm_connector(struct task_str
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->what = PROC_EVENT_COMM;
ev->event_data.comm.process_pid = task->pid;
ev->event_data.comm.process_tgid = task->tgid;
@@ -261,7 +249,6 @@ void proc_coredump_connector(struct task
struct cn_msg *msg;
struct proc_event *ev;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
- struct timespec ts;
if (atomic_read(&proc_event_num_listeners) < 1)
return;
@@ -270,8 +257,7 @@ void proc_coredump_connector(struct task
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->what = PROC_EVENT_COREDUMP;
ev->event_data.coredump.process_pid = task->pid;
ev->event_data.coredump.process_tgid = task->tgid;
@@ -288,7 +274,6 @@ void proc_exit_connector(struct task_str
struct cn_msg *msg;
struct proc_event *ev;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
- struct timespec ts;
if (atomic_read(&proc_event_num_listeners) < 1)
return;
@@ -297,8 +282,7 @@ void proc_exit_connector(struct task_str
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->what = PROC_EVENT_EXIT;
ev->event_data.exit.process_pid = task->pid;
ev->event_data.exit.process_tgid = task->tgid;
@@ -325,7 +309,6 @@ static void cn_proc_ack(int err, int rcv
struct cn_msg *msg;
struct proc_event *ev;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
- struct timespec ts;
if (atomic_read(&proc_event_num_listeners) < 1)
return;
@@ -334,8 +317,7 @@ static void cn_proc_ack(int err, int rcv
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
msg->seq = rcvd_seq;
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->cpu = -1;
ev->what = PROC_EVENT_NONE;
ev->event_data.ack.err = err;
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 35/55] mfd: cros_ec_spi: Use ktime_get_ns()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (33 preceding siblings ...)
2014-07-11 13:44 ` [patch 34/55] connector: " Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 14:52 ` Lee Jones
2014-07-11 13:44 ` [patch 36/55] misc: ioc4: " Thomas Gleixner
` (19 subsequent siblings)
54 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Samuel Ortiz
[-- Attachment #1: mfd-cros-ec-spi-remove-silly-timestamp-dance.patch --]
[-- Type: text/plain, Size: 1329 bytes --]
Replace the ever recurring:
ts = ktime_get_ts();
ns = timespec_to_ns(&ts);
with
ns = ktime_get_ns();
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Samuel Ortiz <sameo@linux.intel.com>
---
drivers/mfd/cros_ec_spi.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
Index: tip/drivers/mfd/cros_ec_spi.c
===================================================================
--- tip.orig/drivers/mfd/cros_ec_spi.c
+++ tip/drivers/mfd/cros_ec_spi.c
@@ -225,7 +225,6 @@ static int cros_ec_command_spi_xfer(stru
u8 *ptr;
int sum;
int ret = 0, final_ret;
- struct timespec ts;
/*
* We have the shared ec_dev buffer plus we do lots of separate spi_sync
@@ -242,8 +241,7 @@ static int cros_ec_command_spi_xfer(stru
struct timespec ts;
unsigned long delay; /* The delay completed so far */
- ktime_get_ts(&ts);
- delay = timespec_to_ns(&ts) - ec_spi->last_transfer_ns;
+ delay = ktime_get_ns() - ec_spi->last_transfer_ns;
if (delay < EC_SPI_RECOVERY_TIME_NS)
ndelay(EC_SPI_RECOVERY_TIME_NS - delay);
}
@@ -280,8 +278,7 @@ static int cros_ec_command_spi_xfer(stru
}
final_ret = spi_sync(ec_spi->spi, &msg);
- ktime_get_ts(&ts);
- ec_spi->last_transfer_ns = timespec_to_ns(&ts);
+ ec_spi->last_transfer_ns = ktime_get_ns();
if (!ret)
ret = final_ret;
if (ret < 0) {
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 35/55] mfd: cros_ec_spi: Use ktime_get_ns()
2014-07-11 13:44 ` [patch 35/55] mfd: cros_ec_spi: " Thomas Gleixner
@ 2014-07-11 14:52 ` Lee Jones
2014-07-12 19:40 ` Thomas Gleixner
0 siblings, 1 reply; 91+ messages in thread
From: Lee Jones @ 2014-07-11 14:52 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Peter Zijlstra, Samuel Ortiz
> Replace the ever recurring:
> ts = ktime_get_ts();
> ns = timespec_to_ns(&ts);
> with
> ns = ktime_get_ns();
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Samuel Ortiz <sameo@linux.intel.com>
Only Sam? How old is your version of MAINTAINERS?
Sam hasn't actively maintained MFD for over a year now.
Patch applied though, thanks.
--
Lee Jones
Linaro STMicroelectronics Landing Team Lead
Linaro.org │ Open source software for ARM SoCs
Follow Linaro: Facebook | Twitter | Blog
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [patch 35/55] mfd: cros_ec_spi: Use ktime_get_ns()
2014-07-11 14:52 ` Lee Jones
@ 2014-07-12 19:40 ` Thomas Gleixner
2014-07-14 7:07 ` Lee Jones
0 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-12 19:40 UTC (permalink / raw)
To: Lee Jones; +Cc: LKML, John Stultz, Peter Zijlstra, Samuel Ortiz
On Fri, 11 Jul 2014, Lee Jones wrote:
> > Replace the ever recurring:
> > ts = ktime_get_ts();
> > ns = timespec_to_ns(&ts);
> > with
> > ns = ktime_get_ns();
> >
> > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> > Cc: Samuel Ortiz <sameo@linux.intel.com>
>
> Only Sam? How old is your version of MAINTAINERS?
Hmm. I use get_maintainers.pl and manually edit it afterwards. Seems
that I killed one extra line :)
> Sam hasn't actively maintained MFD for over a year now.
>
> Patch applied though, thanks.
ktime_get_ns() is not upstream yet, it's part of the series I posted :)
If you ack it, i'll carry it and also remove the now unused variable
ts before applying it.
Thanks,
tglx
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [patch 35/55] mfd: cros_ec_spi: Use ktime_get_ns()
2014-07-12 19:40 ` Thomas Gleixner
@ 2014-07-14 7:07 ` Lee Jones
0 siblings, 0 replies; 91+ messages in thread
From: Lee Jones @ 2014-07-14 7:07 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Peter Zijlstra, Samuel Ortiz
On Sat, 12 Jul 2014, Thomas Gleixner wrote:
> On Fri, 11 Jul 2014, Lee Jones wrote:
>
> > > Replace the ever recurring:
> > > ts = ktime_get_ts();
> > > ns = timespec_to_ns(&ts);
> > > with
> > > ns = ktime_get_ns();
> > >
> > > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> > > Cc: Samuel Ortiz <sameo@linux.intel.com>
> >
> > Only Sam? How old is your version of MAINTAINERS?
>
> Hmm. I use get_maintainers.pl and manually edit it afterwards. Seems
> that I killed one extra line :)
>
> > Sam hasn't actively maintained MFD for over a year now.
> >
> > Patch applied though, thanks.
>
> ktime_get_ns() is not upstream yet, it's part of the series I posted :)
>
> If you ack it, i'll carry it and also remove the now unused variable
> ts before applying it.
Very well. Patch looks good to me:
Acked-by: Lee Jones <lee.jones@linaro.org>
--
Lee Jones
Linaro STMicroelectronics Landing Team Lead
Linaro.org │ Open source software for ARM SoCs
Follow Linaro: Facebook | Twitter | Blog
^ permalink raw reply [flat|nested] 91+ messages in thread
* [patch 36/55] misc: ioc4: Use ktime_get_ns()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (34 preceding siblings ...)
2014-07-11 13:44 ` [patch 35/55] mfd: cros_ec_spi: " Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 17:03 ` Arnd Bergmann
2014-07-12 0:26 ` Greg Kroah-Hartman
2014-07-11 13:44 ` [patch 37/55] net: mlx5: " Thomas Gleixner
` (18 subsequent siblings)
54 siblings, 2 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Arnd Bergmann, Greg Kroah-Hartman
[-- Attachment #1: misc-ioc4-remove-silly-time-stamp-dance.patch --]
[-- Type: text/plain, Size: 1505 bytes --]
Replace the ever recurring:
ts = ktime_get_ts();
ns = timespec_to_ns(&ts);
with
ns = ktime_get_ns();
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/misc/ioc4.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
Index: tip/drivers/misc/ioc4.c
===================================================================
--- tip.orig/drivers/misc/ioc4.c
+++ tip/drivers/misc/ioc4.c
@@ -145,7 +145,6 @@ ioc4_clock_calibrate(struct ioc4_driver_
union ioc4_int_out int_out;
union ioc4_gpcr gpcr;
unsigned int state, last_state = 1;
- struct timespec start_ts, end_ts;
uint64_t start, end, period;
unsigned int count = 0;
@@ -174,10 +173,10 @@ ioc4_clock_calibrate(struct ioc4_driver_
if (!last_state && state) {
count++;
if (count == IOC4_CALIBRATE_END) {
- ktime_get_ts(&end_ts);
+ end = ktime_get_ns();
break;
} else if (count == IOC4_CALIBRATE_DISCARD)
- ktime_get_ts(&start_ts);
+ start = ktime_get_ns();
}
last_state = state;
} while (1);
@@ -192,8 +191,6 @@ ioc4_clock_calibrate(struct ioc4_driver_
* by which the IOC4 generates the square wave, to get the
* period of an IOC4 INT_OUT count.
*/
- end = end_ts.tv_sec * NSEC_PER_SEC + end_ts.tv_nsec;
- start = start_ts.tv_sec * NSEC_PER_SEC + start_ts.tv_nsec;
period = (end - start) /
(IOC4_CALIBRATE_CYCLES * 2 * (IOC4_CALIBRATE_COUNT + 1));
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 36/55] misc: ioc4: Use ktime_get_ns()
2014-07-11 13:44 ` [patch 36/55] misc: ioc4: " Thomas Gleixner
@ 2014-07-11 17:03 ` Arnd Bergmann
2014-07-12 0:26 ` Greg Kroah-Hartman
1 sibling, 0 replies; 91+ messages in thread
From: Arnd Bergmann @ 2014-07-11 17:03 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Peter Zijlstra, Greg Kroah-Hartman
On Friday 11 July 2014, Thomas Gleixner wrote:
> Replace the ever recurring:
> ts = ktime_get_ts();
> ns = timespec_to_ns(&ts);
> with
> ns = ktime_get_ns();
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Arnd Bergmann <arnd@arndb.de>
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [patch 36/55] misc: ioc4: Use ktime_get_ns()
2014-07-11 13:44 ` [patch 36/55] misc: ioc4: " Thomas Gleixner
2014-07-11 17:03 ` Arnd Bergmann
@ 2014-07-12 0:26 ` Greg Kroah-Hartman
1 sibling, 0 replies; 91+ messages in thread
From: Greg Kroah-Hartman @ 2014-07-12 0:26 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Peter Zijlstra, Arnd Bergmann
On Fri, Jul 11, 2014 at 01:44:52PM -0000, Thomas Gleixner wrote:
> Replace the ever recurring:
> ts = ktime_get_ts();
> ns = timespec_to_ns(&ts);
> with
> ns = ktime_get_ns();
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
^ permalink raw reply [flat|nested] 91+ messages in thread
* [patch 37/55] net: mlx5: Use ktime_get_ns()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (35 preceding siblings ...)
2014-07-11 13:44 ` [patch 36/55] misc: ioc4: " Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 38/55] fs: lockd: " Thomas Gleixner
` (17 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Eli Cohen
[-- Attachment #1: net-mlx5-remove-silly-timespec-dance.patch --]
[-- Type: text/plain, Size: 2977 bytes --]
This code is beyond silly:
struct timespec ts = ktime_get_ts();
ktime_t ktime = timespec_to_ktime(ts);
Further down the code builds the delta of two ktime_t values and
converts the result to nanoseconds.
Use ktime_get_ns() and replace all the nonsense.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Eli Cohen <eli@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/cmd.c | 16 ++++------------
include/linux/mlx5/driver.h | 4 ++--
2 files changed, 6 insertions(+), 14 deletions(-)
Index: tip/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
===================================================================
--- tip.orig/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
+++ tip/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
@@ -548,7 +548,7 @@ static void cmd_work_handler(struct work
lay->status_own = CMD_OWNER_HW;
set_signature(ent, !cmd->checksum_disabled);
dump_command(dev, ent, 1);
- ktime_get_ts(&ent->ts1);
+ ent->ts1 = ktime_get_ns();
/* ring doorbell after the descriptor is valid */
wmb();
@@ -637,7 +637,6 @@ static int mlx5_cmd_invoke(struct mlx5_c
{
struct mlx5_cmd *cmd = &dev->cmd;
struct mlx5_cmd_work_ent *ent;
- ktime_t t1, t2, delta;
struct mlx5_cmd_stats *stats;
int err = 0;
s64 ds;
@@ -668,10 +667,7 @@ static int mlx5_cmd_invoke(struct mlx5_c
if (err == -ETIMEDOUT)
goto out;
- t1 = timespec_to_ktime(ent->ts1);
- t2 = timespec_to_ktime(ent->ts2);
- delta = ktime_sub(t2, t1);
- ds = ktime_to_ns(delta);
+ ds = ent->ts2 - ent->ts1;
op = be16_to_cpu(((struct mlx5_inbox_hdr *)in->first.data)->opcode);
if (op < ARRAY_SIZE(cmd->stats)) {
stats = &cmd->stats[op];
@@ -1135,7 +1131,6 @@ void mlx5_cmd_comp_handler(struct mlx5_c
void *context;
int err;
int i;
- ktime_t t1, t2, delta;
s64 ds;
struct mlx5_cmd_stats *stats;
unsigned long flags;
@@ -1149,7 +1144,7 @@ void mlx5_cmd_comp_handler(struct mlx5_c
sem = &cmd->pages_sem;
else
sem = &cmd->sem;
- ktime_get_ts(&ent->ts2);
+ ent->ts2 = ktime_get_ns();
memcpy(ent->out->first.data, ent->lay->out, sizeof(ent->lay->out));
dump_command(dev, ent, 0);
if (!ent->ret) {
@@ -1163,10 +1158,7 @@ void mlx5_cmd_comp_handler(struct mlx5_c
}
free_ent(cmd, ent->idx);
if (ent->callback) {
- t1 = timespec_to_ktime(ent->ts1);
- t2 = timespec_to_ktime(ent->ts2);
- delta = ktime_sub(t2, t1);
- ds = ktime_to_ns(delta);
+ ds = ent->ts2 - ent->ts1;
if (ent->op < ARRAY_SIZE(cmd->stats)) {
stats = &cmd->stats[ent->op];
spin_lock_irqsave(&stats->lock, flags);
Index: tip/include/linux/mlx5/driver.h
===================================================================
--- tip.orig/include/linux/mlx5/driver.h
+++ tip/include/linux/mlx5/driver.h
@@ -605,8 +605,8 @@ struct mlx5_cmd_work_ent {
int page_queue;
u8 status;
u8 token;
- struct timespec ts1;
- struct timespec ts2;
+ u64 ts1;
+ u64 ts2;
u16 op;
};
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 38/55] fs: lockd: Use ktime_get_ns()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (36 preceding siblings ...)
2014-07-11 13:44 ` [patch 37/55] net: mlx5: " Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 14:04 ` Trond Myklebust
2014-07-11 13:44 ` [patch 39/55] hwmon: ibmaem: " Thomas Gleixner
` (16 subsequent siblings)
54 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, J. Bruce Fields, Trond Myklebust
[-- Attachment #1: fs-lockd-remove-silly-timestamp-dance.patch --]
[-- Type: text/plain, Size: 839 bytes --]
Replace the ever recurring:
ts = ktime_get_ts();
ns = timespec_to_ns(&ts);
with
ns = ktime_get_ns();
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Cc: Trond Myklebust <trond.myklebust@primarydata.com>
---
fs/lockd/mon.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
Index: tip/fs/lockd/mon.c
===================================================================
--- tip.orig/fs/lockd/mon.c
+++ tip/fs/lockd/mon.c
@@ -306,11 +306,9 @@ static struct nsm_handle *nsm_lookup_pri
static void nsm_init_private(struct nsm_handle *nsm)
{
u64 *p = (u64 *)&nsm->sm_priv.data;
- struct timespec ts;
s64 ns;
- ktime_get_ts(&ts);
- ns = timespec_to_ns(&ts);
+ ns = ktime_get_ns();
put_unaligned(ns, p);
put_unaligned((unsigned long)nsm, p + 1);
}
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 38/55] fs: lockd: Use ktime_get_ns()
2014-07-11 13:44 ` [patch 38/55] fs: lockd: " Thomas Gleixner
@ 2014-07-11 14:04 ` Trond Myklebust
0 siblings, 0 replies; 91+ messages in thread
From: Trond Myklebust @ 2014-07-11 14:04 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Peter Zijlstra, J. Bruce Fields
On Fri, Jul 11, 2014 at 9:44 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> Replace the ever recurring:
> ts = ktime_get_ts();
> ns = timespec_to_ns(&ts);
> with
> ns = ktime_get_ns();
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: "J. Bruce Fields" <bfields@fieldses.org>
> Cc: Trond Myklebust <trond.myklebust@primarydata.com>
> ---
> fs/lockd/mon.c | 4 +---
> 1 file changed, 1 insertion(+), 3 deletions(-)
>
> Index: tip/fs/lockd/mon.c
> ===================================================================
> --- tip.orig/fs/lockd/mon.c
> +++ tip/fs/lockd/mon.c
> @@ -306,11 +306,9 @@ static struct nsm_handle *nsm_lookup_pri
> static void nsm_init_private(struct nsm_handle *nsm)
> {
> u64 *p = (u64 *)&nsm->sm_priv.data;
> - struct timespec ts;
> s64 ns;
>
> - ktime_get_ts(&ts);
> - ns = timespec_to_ns(&ts);
> + ns = ktime_get_ns();
> put_unaligned(ns, p);
> put_unaligned((unsigned long)nsm, p + 1);
> }
>
>
Looks fine to me.
Acked-by: Trond Myklebust <trond.myklebust@primarydata.com>
--
Trond Myklebust
Linux NFS client maintainer, PrimaryData
trond.myklebust@primarydata.com
^ permalink raw reply [flat|nested] 91+ messages in thread
* [patch 39/55] hwmon: ibmaem: Use ktime_get_ns()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (37 preceding siblings ...)
2014-07-11 13:44 ` [patch 38/55] fs: lockd: " Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:44 ` [patch 40/55] iio: Use ktime_get_real_ns() Thomas Gleixner
` (15 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Jean Delvare
[-- Attachment #1: hwmon-ibmaem-use-ktime.patch --]
[-- Type: text/plain, Size: 1395 bytes --]
Using the wall clock time for delta time calculations is wrong to
begin with because wall clock time can be set from userspace and NTP.
Such data wants to be based on clock monotonic.
The calcuations also are done on a nanosecond basis. Use the
nanoseconds based interface right away.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Jean Delvare <jdelvare@suse.de>
---
drivers/hwmon/ibmaem.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
Index: tip/drivers/hwmon/ibmaem.c
===================================================================
--- tip.orig/drivers/hwmon/ibmaem.c
+++ tip/drivers/hwmon/ibmaem.c
@@ -842,11 +842,10 @@ static ssize_t aem_show_power(struct dev
struct aem_data *data = dev_get_drvdata(dev);
u64 before, after, delta, time;
signed long leftover;
- struct timespec b, a;
mutex_lock(&data->lock);
update_aem_energy_one(data, attr->index);
- getnstimeofday(&b);
+ time = ktime_get_ns();
before = data->energy[attr->index];
leftover = schedule_timeout_interruptible(
@@ -858,11 +857,10 @@ static ssize_t aem_show_power(struct dev
}
update_aem_energy_one(data, attr->index);
- getnstimeofday(&a);
+ time = ktime_get_ns() - time;
after = data->energy[attr->index];
mutex_unlock(&data->lock);
- time = timespec_to_ns(&a) - timespec_to_ns(&b);
delta = (after - before) * UJ_PER_MJ;
return sprintf(buf, "%llu\n",
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 40/55] iio: Use ktime_get_real_ns()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (38 preceding siblings ...)
2014-07-11 13:44 ` [patch 39/55] hwmon: ibmaem: " Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-13 9:21 ` Jonathan Cameron
2014-07-11 13:44 ` [patch 41/55] arm: bL_switcher:k " Thomas Gleixner
` (14 subsequent siblings)
54 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Jonathan Cameron
[-- Attachment #1: iio-use-ktime-get-ns.patch --]
[-- Type: text/plain, Size: 846 bytes --]
No idea why iio needs wall clock based time stamps, but we can avoid
the timespec conversion dance by using the new interfaces.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Jonathan Cameron <jic23@kernel.org>
---
include/linux/iio/iio.h | 9 +--------
1 file changed, 1 insertion(+), 8 deletions(-)
Index: tip/include/linux/iio/iio.h
===================================================================
--- tip.orig/include/linux/iio/iio.h
+++ tip/include/linux/iio/iio.h
@@ -277,14 +277,7 @@ static inline bool iio_channel_has_info(
**/
static inline s64 iio_get_time_ns(void)
{
- struct timespec ts;
- /*
- * calls getnstimeofday.
- * If hrtimers then up to ns accurate, if not microsecond.
- */
- ktime_get_real_ts(&ts);
-
- return timespec_to_ns(&ts);
+ return ktime_get_real_ns();
}
/* Device operating modes */
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 41/55] arm: bL_switcher:k Use ktime_get_real_ns()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (39 preceding siblings ...)
2014-07-11 13:44 ` [patch 40/55] iio: Use ktime_get_real_ns() Thomas Gleixner
@ 2014-07-11 13:44 ` Thomas Gleixner
2014-07-11 13:45 ` [patch 42/55] x86: kvm: Use ktime_get_boot_ns() Thomas Gleixner
` (13 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:44 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Russell King, linux-arm-kernel
[-- Attachment #1: arm-bL_switcher-use-ktime_get_ns.patch --]
[-- Type: text/plain, Size: 1646 bytes --]
Use the nanoseconds based interface instead of converting from a
timespec.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: linux-arm-kernel@lists.infradead.org
---
arch/arm/common/bL_switcher.c | 16 +++-------------
1 file changed, 3 insertions(+), 13 deletions(-)
Index: tip/arch/arm/common/bL_switcher.c
===================================================================
--- tip.orig/arch/arm/common/bL_switcher.c
+++ tip/arch/arm/common/bL_switcher.c
@@ -58,16 +58,6 @@ static int read_mpidr(void)
}
/*
- * Get a global nanosecond time stamp for tracing.
- */
-static s64 get_ns(void)
-{
- struct timespec ts;
- getnstimeofday(&ts);
- return timespec_to_ns(&ts);
-}
-
-/*
* bL switcher core code.
*/
@@ -224,7 +214,7 @@ static int bL_switch_to(unsigned int new
*/
local_irq_disable();
local_fiq_disable();
- trace_cpu_migrate_begin(get_ns(), ob_mpidr);
+ trace_cpu_migrate_begin(ktime_get_real_ns(), ob_mpidr);
/* redirect GIC's SGIs to our counterpart */
gic_migrate_target(bL_gic_id[ib_cpu][ib_cluster]);
@@ -267,7 +257,7 @@ static int bL_switch_to(unsigned int new
tdev->evtdev->next_event, 1);
}
- trace_cpu_migrate_finish(get_ns(), ib_mpidr);
+ trace_cpu_migrate_finish(ktime_get_real_ns(), ib_mpidr);
local_fiq_enable();
local_irq_enable();
@@ -558,7 +548,7 @@ int bL_switcher_get_logical_index(u32 mp
static void bL_switcher_trace_trigger_cpu(void *__always_unused info)
{
- trace_cpu_migrate_current(get_ns(), read_mpidr());
+ trace_cpu_migrate_current(ktime_get_real_ns(), read_mpidr());
}
int bL_switcher_trace_trigger(void)
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 42/55] x86: kvm: Use ktime_get_boot_ns()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (40 preceding siblings ...)
2014-07-11 13:44 ` [patch 41/55] arm: bL_switcher:k " Thomas Gleixner
@ 2014-07-11 13:45 ` Thomas Gleixner
2014-07-11 13:45 ` [patch 43/55] x86: kvm: Make kvm_get_time_and_clockread() nanoseconds based Thomas Gleixner
` (12 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:45 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Gleb Natapov, kvm
[-- Attachment #1: x86-kvm-use-ktime.patch --]
[-- Type: text/plain, Size: 714 bytes --]
Use the new nanoseconds based interface and get rid of the timespec
conversion dance.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Gleb Natapov <gleb@kernel.org>
Cc: kvm@vger.kernel.org
---
arch/x86/kvm/x86.c | 6 +-----
1 file changed, 1 insertion(+), 5 deletions(-)
Index: tip/arch/x86/kvm/x86.c
===================================================================
--- tip.orig/arch/x86/kvm/x86.c
+++ tip/arch/x86/kvm/x86.c
@@ -1109,11 +1109,7 @@ static void kvm_get_time_scale(uint32_t
static inline u64 get_kernel_ns(void)
{
- struct timespec ts;
-
- ktime_get_ts(&ts);
- monotonic_to_bootbased(&ts);
- return timespec_to_ns(&ts);
+ return ktime_get_boot_ns();
}
#ifdef CONFIG_X86_64
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 43/55] x86: kvm: Make kvm_get_time_and_clockread() nanoseconds based
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (41 preceding siblings ...)
2014-07-11 13:45 ` [patch 42/55] x86: kvm: Use ktime_get_boot_ns() Thomas Gleixner
@ 2014-07-11 13:45 ` Thomas Gleixner
2014-07-11 13:45 ` [patch 44/55] timekeeping: Remove monotonic_to_bootbased Thomas Gleixner
` (11 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:45 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Gleb Natapov, kvm
[-- Attachment #1: x86-kvm-use-ktime-based.patch --]
[-- Type: text/plain, Size: 3093 bytes --]
Convert the relevant base data right away to nanoseconds instead of
doing the conversion on every readout. Reduces text size by 160 bytes.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Gleb Natapov <gleb@kernel.org>
Cc: kvm@vger.kernel.org
---
arch/x86/kvm/x86.c | 44 ++++++++++++++------------------------------
1 file changed, 14 insertions(+), 30 deletions(-)
Index: tip/arch/x86/kvm/x86.c
===================================================================
--- tip.orig/arch/x86/kvm/x86.c
+++ tip/arch/x86/kvm/x86.c
@@ -984,9 +984,8 @@ struct pvclock_gtod_data {
u32 shift;
} clock;
- /* open coded 'struct timespec' */
- u64 monotonic_time_snsec;
- time_t monotonic_time_sec;
+ u64 boot_ns;
+ u64 nsec_base;
};
static struct pvclock_gtod_data pvclock_gtod_data;
@@ -994,6 +993,9 @@ static struct pvclock_gtod_data pvclock_
static void update_pvclock_gtod(struct timekeeper *tk)
{
struct pvclock_gtod_data *vdata = &pvclock_gtod_data;
+ u64 boot_ns;
+
+ boot_ns = ktime_to_ns(ktime_add(tk->base_mono, tk->offs_boot));
write_seqcount_begin(&vdata->seq);
@@ -1004,17 +1006,8 @@ static void update_pvclock_gtod(struct t
vdata->clock.mult = tk->mult;
vdata->clock.shift = tk->shift;
- vdata->monotonic_time_sec = tk->xtime_sec
- + tk->wall_to_monotonic.tv_sec;
- vdata->monotonic_time_snsec = tk->xtime_nsec
- + (tk->wall_to_monotonic.tv_nsec
- << tk->shift);
- while (vdata->monotonic_time_snsec >=
- (((u64)NSEC_PER_SEC) << tk->shift)) {
- vdata->monotonic_time_snsec -=
- ((u64)NSEC_PER_SEC) << tk->shift;
- vdata->monotonic_time_sec++;
- }
+ vdata->boot_ns = boot_ns;
+ vdata->nsec_base = tk->xtime_nsec;
write_seqcount_end(&vdata->seq);
}
@@ -1371,23 +1364,22 @@ static inline u64 vgettsc(cycle_t *cycle
return v * gtod->clock.mult;
}
-static int do_monotonic(struct timespec *ts, cycle_t *cycle_now)
+static int do_monotonic_boot(s64 *t, cycle_t *cycle_now)
{
+ struct pvclock_gtod_data *gtod = &pvclock_gtod_data;
unsigned long seq;
- u64 ns;
int mode;
- struct pvclock_gtod_data *gtod = &pvclock_gtod_data;
+ u64 ns;
- ts->tv_nsec = 0;
do {
seq = read_seqcount_begin(>od->seq);
mode = gtod->clock.vclock_mode;
- ts->tv_sec = gtod->monotonic_time_sec;
- ns = gtod->monotonic_time_snsec;
+ ns = gtod->nsec_base;
ns += vgettsc(cycle_now);
ns >>= gtod->clock.shift;
+ ns += gtod->boot_ns;
} while (unlikely(read_seqcount_retry(>od->seq, seq)));
- timespec_add_ns(ts, ns);
+ *t = ns;
return mode;
}
@@ -1395,19 +1387,11 @@ static int do_monotonic(struct timespec
/* returns true if host is using tsc clocksource */
static bool kvm_get_time_and_clockread(s64 *kernel_ns, cycle_t *cycle_now)
{
- struct timespec ts;
-
/* checked again under seqlock below */
if (pvclock_gtod_data.clock.vclock_mode != VCLOCK_TSC)
return false;
- if (do_monotonic(&ts, cycle_now) != VCLOCK_TSC)
- return false;
-
- monotonic_to_bootbased(&ts);
- *kernel_ns = timespec_to_ns(&ts);
-
- return true;
+ return do_monotonic_boot(kernel_ns, cycle_now) == VCLOCK_TSC;
}
#endif
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 44/55] timekeeping: Remove monotonic_to_bootbased
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (42 preceding siblings ...)
2014-07-11 13:45 ` [patch 43/55] x86: kvm: Make kvm_get_time_and_clockread() nanoseconds based Thomas Gleixner
@ 2014-07-11 13:45 ` Thomas Gleixner
2014-07-11 13:45 ` [patch 45/55] timekeeping: Use ktime_get_boottime() for get_monotonic_boottime() Thomas Gleixner
` (10 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:45 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-remove-monotonic-to-bootbased.patch --]
[-- Type: text/plain, Size: 1524 bytes --]
No more users.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 1 -
kernel/time/timekeeping.c | 15 ---------------
2 files changed, 16 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -27,7 +27,6 @@ struct timespec __current_kernel_time(vo
*/
struct timespec get_monotonic_coarse(void);
extern void getrawmonotonic(struct timespec *ts);
-extern void monotonic_to_bootbased(struct timespec *ts);
extern void get_monotonic_boottime(struct timespec *ts);
extern void ktime_get_ts64(struct timespec64 *ts);
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -1574,21 +1574,6 @@ void get_monotonic_boottime(struct times
}
EXPORT_SYMBOL_GPL(get_monotonic_boottime);
-/**
- * monotonic_to_bootbased - Convert the monotonic time to boot based.
- * @ts: pointer to the timespec to be converted
- */
-void monotonic_to_bootbased(struct timespec *ts)
-{
- struct timekeeper *tk = &tk_core.timekeeper;
- struct timespec64 ts64;
-
- ts64 = timespec_to_timespec64(*ts);
- ts64 = timespec64_add(ts64, tk->total_sleep_time);
- *ts = timespec64_to_timespec(ts64);
-}
-EXPORT_SYMBOL_GPL(monotonic_to_bootbased);
-
unsigned long get_seconds(void)
{
struct timekeeper *tk = &tk_core.timekeeper;
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 45/55] timekeeping: Use ktime_get_boottime() for get_monotonic_boottime()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (43 preceding siblings ...)
2014-07-11 13:45 ` [patch 44/55] timekeeping: Remove monotonic_to_bootbased Thomas Gleixner
@ 2014-07-11 13:45 ` Thomas Gleixner
2014-07-11 13:45 ` [patch 46/55] timekeeping: Provide ktime_get_raw() Thomas Gleixner
` (9 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:45 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-simplify-get-monotonic-boottime.patch --]
[-- Type: text/plain, Size: 2500 bytes --]
get_monotonic_boottime() is not used in fast pathes, so the extra
timespec conversion is not problematic.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 9 ++++++++-
kernel/time/timekeeping.c | 34 ----------------------------------
2 files changed, 8 insertions(+), 35 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -27,7 +27,6 @@ struct timespec __current_kernel_time(vo
*/
struct timespec get_monotonic_coarse(void);
extern void getrawmonotonic(struct timespec *ts);
-extern void get_monotonic_boottime(struct timespec *ts);
extern void ktime_get_ts64(struct timespec64 *ts);
extern int __getnstimeofday64(struct timespec64 *tv);
@@ -160,6 +159,14 @@ static inline u64 ktime_get_boot_ns(void
}
/*
+ * Timespec interfaces utilizing the ktime based ones
+ */
+static inline void get_monotonic_boottime(struct timespec *ts)
+{
+ *ts = ktime_to_timespec(ktime_get_boottime());
+}
+
+/*
* RTC specific
*/
extern void timekeeping_inject_sleeptime(struct timespec *delta);
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -1540,40 +1540,6 @@ void getboottime(struct timespec *ts)
}
EXPORT_SYMBOL_GPL(getboottime);
-/**
- * get_monotonic_boottime - Returns monotonic time since boot
- * @ts: pointer to the timespec to be set
- *
- * Returns the monotonic time since boot in a timespec.
- *
- * This is similar to CLOCK_MONTONIC/ktime_get_ts, but also
- * includes the time spent in suspend.
- */
-void get_monotonic_boottime(struct timespec *ts)
-{
- struct timekeeper *tk = &tk_core.timekeeper;
- struct timespec64 tomono, sleep, ret;
- s64 nsec;
- unsigned int seq;
-
- WARN_ON(timekeeping_suspended);
-
- do {
- seq = read_seqcount_begin(&tk_core.seq);
- ret.tv_sec = tk->xtime_sec;
- nsec = timekeeping_get_ns(tk);
- tomono = tk->wall_to_monotonic;
- sleep = tk->total_sleep_time;
-
- } while (read_seqcount_retry(&tk_core.seq, seq));
-
- ret.tv_sec += tomono.tv_sec + sleep.tv_sec;
- ret.tv_nsec = 0;
- timespec64_add_ns(&ret, nsec + tomono.tv_nsec + sleep.tv_nsec);
- *ts = timespec64_to_timespec(ret);
-}
-EXPORT_SYMBOL_GPL(get_monotonic_boottime);
-
unsigned long get_seconds(void)
{
struct timekeeper *tk = &tk_core.timekeeper;
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 46/55] timekeeping: Provide ktime_get_raw()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (44 preceding siblings ...)
2014-07-11 13:45 ` [patch 45/55] timekeeping: Use ktime_get_boottime() for get_monotonic_boottime() Thomas Gleixner
@ 2014-07-11 13:45 ` Thomas Gleixner
2014-07-11 13:45 ` [patch 47/55] hangcheck-timer: Use ktime_get_raw_ns() Thomas Gleixner
` (8 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:45 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-provide-ktime-get-raw.patch --]
[-- Type: text/plain, Size: 3235 bytes --]
Provide a ktime_t based interface for raw monotonic time.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeper_internal.h | 3 +++
include/linux/timekeeping.h | 6 ++++++
kernel/time/timekeeping.c | 26 ++++++++++++++++++++++++++
3 files changed, 35 insertions(+)
Index: tip/include/linux/timekeeper_internal.h
===================================================================
--- tip.orig/include/linux/timekeeper_internal.h
+++ tip/include/linux/timekeeper_internal.h
@@ -56,6 +56,9 @@ struct timekeeper {
/* The current UTC to TAI offset in seconds */
s32 tai_offset;
+ /* Monotonic raw base time */
+ ktime_t base_raw;
+
/* The raw monotonic time for the CLOCK_MONOTONIC_RAW posix clock. */
struct timespec64 raw_time;
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -107,6 +107,7 @@ enum tk_offsets {
extern ktime_t ktime_get(void);
extern ktime_t ktime_get_with_offset(enum tk_offsets offs);
extern ktime_t ktime_mono_to_any(ktime_t tmono, enum tk_offsets offs);
+extern ktime_t ktime_get_raw(void);
/**
* ktime_get_real - get the real (wall-) time in ktime_t format
@@ -158,6 +159,11 @@ static inline u64 ktime_get_boot_ns(void
return ktime_to_ns(ktime_get_boottime());
}
+static inline u64 ktime_get_raw_ns(void)
+{
+ return ktime_to_ns(ktime_get_raw());
+}
+
/*
* Timespec interfaces utilizing the ktime based ones
*/
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -354,6 +354,7 @@ static void timekeeping_forward_now(stru
nsec = clocksource_cyc2ns(cycle_delta, clock->mult, clock->shift);
timespec64_add_ns(&tk->raw_time, nsec);
+ tk->base_raw = ktime_add_ns(tk->base_raw, nsec);
}
/**
@@ -470,6 +471,27 @@ ktime_t ktime_mono_to_any(ktime_t tmono,
EXPORT_SYMBOL_GPL(ktime_mono_to_any);
/**
+ * ktime_get_raw - Returns the raw monotonic time in ktime_t format
+ */
+ktime_t ktime_get_raw(void)
+{
+ struct timekeeper *tk = &tk_core.timekeeper;
+ unsigned int seq;
+ ktime_t base;
+ s64 nsecs;
+
+ do {
+ seq = read_seqcount_begin(&tk_core.seq);
+ base = tk->base_raw;
+ nsecs = timekeeping_get_ns_raw(tk);
+
+ } while (read_seqcount_retry(&tk_core.seq, seq));
+
+ return ktime_add_ns(base, nsecs);
+}
+EXPORT_SYMBOL_GPL(ktime_get_raw);
+
+/**
* ktime_get_ts64 - get the monotonic clock in timespec64 format
* @ts: pointer to timespec variable
*
@@ -912,6 +934,7 @@ void __init timekeeping_init(void)
tk_set_xtime(tk, &now);
tk->raw_time.tv_sec = 0;
tk->raw_time.tv_nsec = 0;
+ tk->base_raw.tv64 = 0;
if (boot.tv_sec == 0 && boot.tv_nsec == 0)
boot = tk_xtime(tk);
@@ -1405,6 +1428,9 @@ static cycle_t logarithmic_accumulation(
/* Accumulate raw time */
raw_nsecs = (u64)tk->raw_interval << shift;
+ /* Update ktime_t base */
+ tk->base_raw = ktime_add_ns(tk->base_raw, raw_nsecs);
+ /* Update the timespec base */
raw_nsecs += tk->raw_time.tv_nsec;
if (raw_nsecs >= NSEC_PER_SEC) {
u64 raw_secs = raw_nsecs;
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 47/55] hangcheck-timer: Use ktime_get_raw_ns()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (45 preceding siblings ...)
2014-07-11 13:45 ` [patch 46/55] timekeeping: Provide ktime_get_raw() Thomas Gleixner
@ 2014-07-11 13:45 ` Thomas Gleixner
2014-07-11 17:03 ` Arnd Bergmann
2014-07-12 0:27 ` Greg Kroah-Hartman
2014-07-11 13:45 ` [patch 48/55] drm: i915: Use nsec based interfaces Thomas Gleixner
` (7 subsequent siblings)
54 siblings, 2 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:45 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Arnd Bergmann, Greg Kroah-Hartman
[-- Attachment #1: hangcheck-timer-get-rid-of-cruft.patch --]
[-- Type: text/plain, Size: 1524 bytes --]
Use the nanoseconds based interface instead of converting timespecs.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/char/hangcheck-timer.c | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
Index: tip/drivers/char/hangcheck-timer.c
===================================================================
--- tip.orig/drivers/char/hangcheck-timer.c
+++ tip/drivers/char/hangcheck-timer.c
@@ -49,7 +49,7 @@
#include <asm/uaccess.h>
#include <linux/sysrq.h>
#include <linux/timer.h>
-#include <linux/time.h>
+#include <linux/hrtimer.h>
#define VERSION_STR "0.9.1"
@@ -129,9 +129,7 @@ extern unsigned long long monotonic_cloc
#else
static inline unsigned long long monotonic_clock(void)
{
- struct timespec ts;
- getrawmonotonic(&ts);
- return timespec_to_ns(&ts);
+ return ktime_get_raw_ns();
}
#endif /* HAVE_MONOTONIC */
@@ -186,9 +184,9 @@ static int __init hangcheck_init(void)
printk("Hangcheck: starting hangcheck timer %s (tick is %d seconds, margin is %d seconds).\n",
VERSION_STR, hangcheck_tick, hangcheck_margin);
#if defined (HAVE_MONOTONIC)
- printk("Hangcheck: Using monotonic_clock().\n");
+ printk("Hangcheck: Using monotonic clock\n");
#else
- printk("Hangcheck: Using getrawmonotonic().\n");
+ printk("Hangcheck: Using raw monotonic clock.\n");
#endif /* HAVE_MONOTONIC */
hangcheck_tsc_margin =
(unsigned long long)(hangcheck_margin + hangcheck_tick);
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 47/55] hangcheck-timer: Use ktime_get_raw_ns()
2014-07-11 13:45 ` [patch 47/55] hangcheck-timer: Use ktime_get_raw_ns() Thomas Gleixner
@ 2014-07-11 17:03 ` Arnd Bergmann
2014-07-12 0:27 ` Greg Kroah-Hartman
1 sibling, 0 replies; 91+ messages in thread
From: Arnd Bergmann @ 2014-07-11 17:03 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Peter Zijlstra, Greg Kroah-Hartman
On Friday 11 July 2014, Thomas Gleixner wrote:
> Use the nanoseconds based interface instead of converting timespecs.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Arnd Bergmann <arnd@arndb.de>
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [patch 47/55] hangcheck-timer: Use ktime_get_raw_ns()
2014-07-11 13:45 ` [patch 47/55] hangcheck-timer: Use ktime_get_raw_ns() Thomas Gleixner
2014-07-11 17:03 ` Arnd Bergmann
@ 2014-07-12 0:27 ` Greg Kroah-Hartman
1 sibling, 0 replies; 91+ messages in thread
From: Greg Kroah-Hartman @ 2014-07-12 0:27 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Peter Zijlstra, Arnd Bergmann
On Fri, Jul 11, 2014 at 01:45:08PM -0000, Thomas Gleixner wrote:
> Use the nanoseconds based interface instead of converting timespecs.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
^ permalink raw reply [flat|nested] 91+ messages in thread
* [patch 48/55] drm: i915: Use nsec based interfaces
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (46 preceding siblings ...)
2014-07-11 13:45 ` [patch 47/55] hangcheck-timer: Use ktime_get_raw_ns() Thomas Gleixner
@ 2014-07-11 13:45 ` Thomas Gleixner
2014-07-14 9:21 ` Daniel Vetter
2014-07-11 13:45 ` [patch 49/55] drm: vmwgfx: " Thomas Gleixner
` (6 subsequent siblings)
54 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:45 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Daniel Vetter
[-- Attachment #1: drm-i915-use-nsec-interfaces.patch --]
[-- Type: text/plain, Size: 5419 bytes --]
No point in converting timespecs back and forth.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
---
drivers/gpu/drm/i915/i915_drv.h | 2 +-
drivers/gpu/drm/i915/i915_gem.c | 33 ++++++++++++---------------------
drivers/gpu/drm/i915/intel_pm.c | 12 +++++-------
3 files changed, 18 insertions(+), 29 deletions(-)
Index: tip/drivers/gpu/drm/i915/i915_drv.h
===================================================================
--- tip.orig/drivers/gpu/drm/i915/i915_drv.h
+++ tip/drivers/gpu/drm/i915/i915_drv.h
@@ -931,7 +931,7 @@ struct intel_ilk_power_mgmt {
unsigned long last_time1;
unsigned long chipset_power;
u64 last_count2;
- struct timespec last_time2;
+ u64 last_time2;
unsigned long gfx_power;
u8 corr;
Index: tip/drivers/gpu/drm/i915/i915_gem.c
===================================================================
--- tip.orig/drivers/gpu/drm/i915/i915_gem.c
+++ tip/drivers/gpu/drm/i915/i915_gem.c
@@ -1149,16 +1149,16 @@ static bool can_wait_boost(struct drm_i9
static int __wait_seqno(struct intel_engine_cs *ring, u32 seqno,
unsigned reset_counter,
bool interruptible,
- struct timespec *timeout,
+ s64 *timeout,
struct drm_i915_file_private *file_priv)
{
struct drm_device *dev = ring->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
const bool irq_test_in_progress =
ACCESS_ONCE(dev_priv->gpu_error.test_irq_rings) & intel_ring_flag(ring);
- struct timespec before, now;
DEFINE_WAIT(wait);
unsigned long timeout_expire;
+ s64 before, now;
int ret;
WARN(dev_priv->pm.irqs_disabled, "IRQs disabled\n");
@@ -1166,7 +1166,7 @@ static int __wait_seqno(struct intel_eng
if (i915_seqno_passed(ring->get_seqno(ring, true), seqno))
return 0;
- timeout_expire = timeout ? jiffies + timespec_to_jiffies_timeout(timeout) : 0;
+ timeout_expire = timeout ? jiffies + nsecs_to_jiffies((u64)timeout) : 0;
if (INTEL_INFO(dev)->gen >= 6 && can_wait_boost(file_priv)) {
gen6_rps_boost(dev_priv);
@@ -1181,7 +1181,7 @@ static int __wait_seqno(struct intel_eng
/* Record current time in case interrupted by signal, or wedged */
trace_i915_gem_request_wait_begin(ring, seqno);
- getrawmonotonic(&before);
+ before = ktime_get_raw_ns();
for (;;) {
struct timer_list timer;
@@ -1230,7 +1230,7 @@ static int __wait_seqno(struct intel_eng
destroy_timer_on_stack(&timer);
}
}
- getrawmonotonic(&now);
+ now = ktime_get_raw_ns();
trace_i915_gem_request_wait_end(ring, seqno);
if (!irq_test_in_progress)
@@ -1239,10 +1239,9 @@ static int __wait_seqno(struct intel_eng
finish_wait(&ring->irq_queue, &wait);
if (timeout) {
- struct timespec sleep_time = timespec_sub(now, before);
- *timeout = timespec_sub(*timeout, sleep_time);
- if (!timespec_valid(timeout)) /* i.e. negative time remains */
- set_normalized_timespec(timeout, 0, 0);
+ s64 tres = *timeout - (now - before);
+
+ *timeout = tres < 0 ? 0 : tres;
}
return ret;
@@ -2753,16 +2752,10 @@ i915_gem_wait_ioctl(struct drm_device *d
struct drm_i915_gem_wait *args = data;
struct drm_i915_gem_object *obj;
struct intel_engine_cs *ring = NULL;
- struct timespec timeout_stack, *timeout = NULL;
unsigned reset_counter;
u32 seqno = 0;
int ret = 0;
- if (args->timeout_ns >= 0) {
- timeout_stack = ns_to_timespec(args->timeout_ns);
- timeout = &timeout_stack;
- }
-
ret = i915_mutex_lock_interruptible(dev);
if (ret)
return ret;
@@ -2787,9 +2780,9 @@ i915_gem_wait_ioctl(struct drm_device *d
goto out;
/* Do this after OLR check to make sure we make forward progress polling
- * on this IOCTL with a 0 timeout (like busy ioctl)
+ * on this IOCTL with a timeout <=0 (like busy ioctl)
*/
- if (!args->timeout_ns) {
+ if (args->timeout_ns <= 0) {
ret = -ETIME;
goto out;
}
@@ -2798,10 +2791,8 @@ i915_gem_wait_ioctl(struct drm_device *d
reset_counter = atomic_read(&dev_priv->gpu_error.reset_counter);
mutex_unlock(&dev->struct_mutex);
- ret = __wait_seqno(ring, seqno, reset_counter, true, timeout, file->driver_priv);
- if (timeout)
- args->timeout_ns = timespec_to_ns(timeout);
- return ret;
+ return __wait_seqno(ring, seqno, reset_counter, true, &args->timeout_ns,
+ file->driver_priv);
out:
drm_gem_object_unreference(&obj->base);
Index: tip/drivers/gpu/drm/i915/intel_pm.c
===================================================================
--- tip.orig/drivers/gpu/drm/i915/intel_pm.c
+++ tip/drivers/gpu/drm/i915/intel_pm.c
@@ -2993,7 +2993,7 @@ static void ironlake_enable_drps(struct
I915_READ(0x112e0);
dev_priv->ips.last_time1 = jiffies_to_msecs(jiffies);
dev_priv->ips.last_count2 = I915_READ(0x112f4);
- getrawmonotonic(&dev_priv->ips.last_time2);
+ dev_priv->ips.last_time2 = ktime_get_raw_ns();
spin_unlock_irq(&mchdev_lock);
}
@@ -4314,18 +4314,16 @@ static u16 pvid_to_extvid(struct drm_i91
static void __i915_update_gfx_val(struct drm_i915_private *dev_priv)
{
- struct timespec now, diff1;
- u64 diff;
- unsigned long diffms;
+ u64 now, diff, diffms;
u32 count;
assert_spin_locked(&mchdev_lock);
- getrawmonotonic(&now);
- diff1 = timespec_sub(now, dev_priv->ips.last_time2);
+ now = ktime_get_raw_ns();
+ diffms = now - dev_priv->ips.last_time2;
+ do_div(diffms, NSEC_PER_MSEC);
/* Don't divide by 0 */
- diffms = diff1.tv_sec * 1000 + diff1.tv_nsec / 1000000;
if (!diffms)
return;
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 48/55] drm: i915: Use nsec based interfaces
2014-07-11 13:45 ` [patch 48/55] drm: i915: Use nsec based interfaces Thomas Gleixner
@ 2014-07-14 9:21 ` Daniel Vetter
0 siblings, 0 replies; 91+ messages in thread
From: Daniel Vetter @ 2014-07-14 9:21 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Peter Zijlstra, intel-gfx
On Fri, Jul 11, 2014 at 3:45 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
> No point in converting timespecs back and forth.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
I guess simplest to merge together with the new ktime_get_raw_ns.
Acked-by: me for that.
-Daniel
> ---
> drivers/gpu/drm/i915/i915_drv.h | 2 +-
> drivers/gpu/drm/i915/i915_gem.c | 33 ++++++++++++---------------------
> drivers/gpu/drm/i915/intel_pm.c | 12 +++++-------
> 3 files changed, 18 insertions(+), 29 deletions(-)
>
> Index: tip/drivers/gpu/drm/i915/i915_drv.h
> ===================================================================
> --- tip.orig/drivers/gpu/drm/i915/i915_drv.h
> +++ tip/drivers/gpu/drm/i915/i915_drv.h
> @@ -931,7 +931,7 @@ struct intel_ilk_power_mgmt {
> unsigned long last_time1;
> unsigned long chipset_power;
> u64 last_count2;
> - struct timespec last_time2;
> + u64 last_time2;
> unsigned long gfx_power;
> u8 corr;
>
> Index: tip/drivers/gpu/drm/i915/i915_gem.c
> ===================================================================
> --- tip.orig/drivers/gpu/drm/i915/i915_gem.c
> +++ tip/drivers/gpu/drm/i915/i915_gem.c
> @@ -1149,16 +1149,16 @@ static bool can_wait_boost(struct drm_i9
> static int __wait_seqno(struct intel_engine_cs *ring, u32 seqno,
> unsigned reset_counter,
> bool interruptible,
> - struct timespec *timeout,
> + s64 *timeout,
> struct drm_i915_file_private *file_priv)
> {
> struct drm_device *dev = ring->dev;
> struct drm_i915_private *dev_priv = dev->dev_private;
> const bool irq_test_in_progress =
> ACCESS_ONCE(dev_priv->gpu_error.test_irq_rings) & intel_ring_flag(ring);
> - struct timespec before, now;
> DEFINE_WAIT(wait);
> unsigned long timeout_expire;
> + s64 before, now;
> int ret;
>
> WARN(dev_priv->pm.irqs_disabled, "IRQs disabled\n");
> @@ -1166,7 +1166,7 @@ static int __wait_seqno(struct intel_eng
> if (i915_seqno_passed(ring->get_seqno(ring, true), seqno))
> return 0;
>
> - timeout_expire = timeout ? jiffies + timespec_to_jiffies_timeout(timeout) : 0;
> + timeout_expire = timeout ? jiffies + nsecs_to_jiffies((u64)timeout) : 0;
>
> if (INTEL_INFO(dev)->gen >= 6 && can_wait_boost(file_priv)) {
> gen6_rps_boost(dev_priv);
> @@ -1181,7 +1181,7 @@ static int __wait_seqno(struct intel_eng
>
> /* Record current time in case interrupted by signal, or wedged */
> trace_i915_gem_request_wait_begin(ring, seqno);
> - getrawmonotonic(&before);
> + before = ktime_get_raw_ns();
> for (;;) {
> struct timer_list timer;
>
> @@ -1230,7 +1230,7 @@ static int __wait_seqno(struct intel_eng
> destroy_timer_on_stack(&timer);
> }
> }
> - getrawmonotonic(&now);
> + now = ktime_get_raw_ns();
> trace_i915_gem_request_wait_end(ring, seqno);
>
> if (!irq_test_in_progress)
> @@ -1239,10 +1239,9 @@ static int __wait_seqno(struct intel_eng
> finish_wait(&ring->irq_queue, &wait);
>
> if (timeout) {
> - struct timespec sleep_time = timespec_sub(now, before);
> - *timeout = timespec_sub(*timeout, sleep_time);
> - if (!timespec_valid(timeout)) /* i.e. negative time remains */
> - set_normalized_timespec(timeout, 0, 0);
> + s64 tres = *timeout - (now - before);
> +
> + *timeout = tres < 0 ? 0 : tres;
> }
>
> return ret;
> @@ -2753,16 +2752,10 @@ i915_gem_wait_ioctl(struct drm_device *d
> struct drm_i915_gem_wait *args = data;
> struct drm_i915_gem_object *obj;
> struct intel_engine_cs *ring = NULL;
> - struct timespec timeout_stack, *timeout = NULL;
> unsigned reset_counter;
> u32 seqno = 0;
> int ret = 0;
>
> - if (args->timeout_ns >= 0) {
> - timeout_stack = ns_to_timespec(args->timeout_ns);
> - timeout = &timeout_stack;
> - }
> -
> ret = i915_mutex_lock_interruptible(dev);
> if (ret)
> return ret;
> @@ -2787,9 +2780,9 @@ i915_gem_wait_ioctl(struct drm_device *d
> goto out;
>
> /* Do this after OLR check to make sure we make forward progress polling
> - * on this IOCTL with a 0 timeout (like busy ioctl)
> + * on this IOCTL with a timeout <=0 (like busy ioctl)
> */
> - if (!args->timeout_ns) {
> + if (args->timeout_ns <= 0) {
> ret = -ETIME;
> goto out;
> }
> @@ -2798,10 +2791,8 @@ i915_gem_wait_ioctl(struct drm_device *d
> reset_counter = atomic_read(&dev_priv->gpu_error.reset_counter);
> mutex_unlock(&dev->struct_mutex);
>
> - ret = __wait_seqno(ring, seqno, reset_counter, true, timeout, file->driver_priv);
> - if (timeout)
> - args->timeout_ns = timespec_to_ns(timeout);
> - return ret;
> + return __wait_seqno(ring, seqno, reset_counter, true, &args->timeout_ns,
> + file->driver_priv);
>
> out:
> drm_gem_object_unreference(&obj->base);
> Index: tip/drivers/gpu/drm/i915/intel_pm.c
> ===================================================================
> --- tip.orig/drivers/gpu/drm/i915/intel_pm.c
> +++ tip/drivers/gpu/drm/i915/intel_pm.c
> @@ -2993,7 +2993,7 @@ static void ironlake_enable_drps(struct
> I915_READ(0x112e0);
> dev_priv->ips.last_time1 = jiffies_to_msecs(jiffies);
> dev_priv->ips.last_count2 = I915_READ(0x112f4);
> - getrawmonotonic(&dev_priv->ips.last_time2);
> + dev_priv->ips.last_time2 = ktime_get_raw_ns();
>
> spin_unlock_irq(&mchdev_lock);
> }
> @@ -4314,18 +4314,16 @@ static u16 pvid_to_extvid(struct drm_i91
>
> static void __i915_update_gfx_val(struct drm_i915_private *dev_priv)
> {
> - struct timespec now, diff1;
> - u64 diff;
> - unsigned long diffms;
> + u64 now, diff, diffms;
> u32 count;
>
> assert_spin_locked(&mchdev_lock);
>
> - getrawmonotonic(&now);
> - diff1 = timespec_sub(now, dev_priv->ips.last_time2);
> + now = ktime_get_raw_ns();
> + diffms = now - dev_priv->ips.last_time2;
> + do_div(diffms, NSEC_PER_MSEC);
>
> /* Don't divide by 0 */
> - diffms = diff1.tv_sec * 1000 + diff1.tv_nsec / 1000000;
> if (!diffms)
> return;
>
>
>
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 91+ messages in thread
* [patch 49/55] drm: vmwgfx: Use nsec based interfaces
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (47 preceding siblings ...)
2014-07-11 13:45 ` [patch 48/55] drm: i915: Use nsec based interfaces Thomas Gleixner
@ 2014-07-11 13:45 ` Thomas Gleixner
2014-07-11 13:45 ` [patch 50/55] wireless: ath9k: Get rid of timespec conversions Thomas Gleixner
` (5 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:45 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Thomas Hellstrom
[-- Attachment #1: drm-vmwgfx-use-nsec-raw.patch --]
[-- Type: text/plain, Size: 3563 bytes --]
No point in converting timespecs back and forth.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
---
drivers/gpu/drm/vmwgfx/vmwgfx_drv.h | 4 +--
drivers/gpu/drm/vmwgfx/vmwgfx_marker.c | 44 ++++++++++-----------------------
2 files changed, 16 insertions(+), 32 deletions(-)
Index: tip/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
===================================================================
--- tip.orig/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
+++ tip/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
@@ -159,8 +159,8 @@ struct vmw_surface {
struct vmw_marker_queue {
struct list_head head;
- struct timespec lag;
- struct timespec lag_time;
+ u64 lag;
+ u64 lag_time;
spinlock_t lock;
};
Index: tip/drivers/gpu/drm/vmwgfx/vmwgfx_marker.c
===================================================================
--- tip.orig/drivers/gpu/drm/vmwgfx/vmwgfx_marker.c
+++ tip/drivers/gpu/drm/vmwgfx/vmwgfx_marker.c
@@ -31,14 +31,14 @@
struct vmw_marker {
struct list_head head;
uint32_t seqno;
- struct timespec submitted;
+ u64 submitted;
};
void vmw_marker_queue_init(struct vmw_marker_queue *queue)
{
INIT_LIST_HEAD(&queue->head);
- queue->lag = ns_to_timespec(0);
- getrawmonotonic(&queue->lag_time);
+ queue->lag = 0;
+ queue->lag_time = ktime_get_raw_ns();
spin_lock_init(&queue->lock);
}
@@ -62,7 +62,7 @@ int vmw_marker_push(struct vmw_marker_qu
return -ENOMEM;
marker->seqno = seqno;
- getrawmonotonic(&marker->submitted);
+ marker->submitted = ktime_get_raw_ns();
spin_lock(&queue->lock);
list_add_tail(&marker->head, &queue->head);
spin_unlock(&queue->lock);
@@ -74,14 +74,14 @@ int vmw_marker_pull(struct vmw_marker_qu
uint32_t signaled_seqno)
{
struct vmw_marker *marker, *next;
- struct timespec now;
bool updated = false;
+ u64 now;
spin_lock(&queue->lock);
- getrawmonotonic(&now);
+ now = ktime_get_raw_ns();
if (list_empty(&queue->head)) {
- queue->lag = ns_to_timespec(0);
+ queue->lag = 0;
queue->lag_time = now;
updated = true;
goto out_unlock;
@@ -91,7 +91,7 @@ int vmw_marker_pull(struct vmw_marker_qu
if (signaled_seqno - marker->seqno > (1 << 30))
continue;
- queue->lag = timespec_sub(now, marker->submitted);
+ queue->lag = now - marker->submitted;
queue->lag_time = now;
updated = true;
list_del(&marker->head);
@@ -104,27 +104,13 @@ out_unlock:
return (updated) ? 0 : -EBUSY;
}
-static struct timespec vmw_timespec_add(struct timespec t1,
- struct timespec t2)
+static u64 vmw_fifo_lag(struct vmw_marker_queue *queue)
{
- t1.tv_sec += t2.tv_sec;
- t1.tv_nsec += t2.tv_nsec;
- if (t1.tv_nsec >= 1000000000L) {
- t1.tv_sec += 1;
- t1.tv_nsec -= 1000000000L;
- }
-
- return t1;
-}
-
-static struct timespec vmw_fifo_lag(struct vmw_marker_queue *queue)
-{
- struct timespec now;
+ u64 now;
spin_lock(&queue->lock);
- getrawmonotonic(&now);
- queue->lag = vmw_timespec_add(queue->lag,
- timespec_sub(now, queue->lag_time));
+ now = ktime_get_raw_ns();
+ queue->lag += now - queue->lag_time;
queue->lag_time = now;
spin_unlock(&queue->lock);
return queue->lag;
@@ -134,11 +120,9 @@ static struct timespec vmw_fifo_lag(stru
static bool vmw_lag_lt(struct vmw_marker_queue *queue,
uint32_t us)
{
- struct timespec lag, cond;
+ u64 cond = (u64) us * NSEC_PER_USEC;
- cond = ns_to_timespec((s64) us * 1000);
- lag = vmw_fifo_lag(queue);
- return (timespec_compare(&lag, &cond) < 1);
+ return vmw_fifo_lag(queue) <= cond;
}
int vmw_wait_lag(struct vmw_private *dev_priv,
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 50/55] wireless: ath9k: Get rid of timespec conversions
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (48 preceding siblings ...)
2014-07-11 13:45 ` [patch 49/55] drm: vmwgfx: " Thomas Gleixner
@ 2014-07-11 13:45 ` Thomas Gleixner
2014-07-11 13:45 ` [patch 51/55] clocksource: Make delta calculation a function Thomas Gleixner
` (4 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:45 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, QCA ath9k Development,
John W. Linville
[-- Attachment #1: wireless-ath9-use-proper-interfaces.patch --]
[-- Type: text/plain, Size: 1474 bytes --]
We have interfaces. Remove the open coded cruft. Reduces text size
along with the code.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: QCA ath9k Development <ath9k-devel@qca.qualcomm.com>
Cc: "John W. Linville" <linville@tuxdriver.com>
---
drivers/net/wireless/ath/ath9k/hw.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
Index: tip/drivers/net/wireless/ath/ath9k/hw.c
===================================================================
--- tip.orig/drivers/net/wireless/ath/ath9k/hw.c
+++ tip/drivers/net/wireless/ath/ath9k/hw.c
@@ -1734,7 +1734,6 @@ int ath9k_hw_reset(struct ath_hw *ah, st
struct ath9k_hw_cal_data *caldata, bool fastcc)
{
struct ath_common *common = ath9k_hw_common(ah);
- struct timespec ts;
u32 saveLedState;
u32 saveDefAntenna;
u32 macStaId1;
@@ -1784,8 +1783,7 @@ int ath9k_hw_reset(struct ath_hw *ah, st
/* Save TSF before chip reset, a cold reset clears it */
tsf = ath9k_hw_gettsf64(ah);
- getrawmonotonic(&ts);
- usec = ts.tv_sec * 1000000ULL + ts.tv_nsec / 1000;
+ usec = ktime_to_us(ktime_get_raw());
saveLedState = REG_READ(ah, AR_CFG_LED) &
(AR_CFG_LED_ASSOC_CTL | AR_CFG_LED_MODE_SEL |
@@ -1818,8 +1816,7 @@ int ath9k_hw_reset(struct ath_hw *ah, st
}
/* Restore TSF */
- getrawmonotonic(&ts);
- usec = ts.tv_sec * 1000000ULL + ts.tv_nsec / 1000 - usec;
+ usec = ktime_to_us(ktime_get_raw()) - usec;
ath9k_hw_settsf64(ah, tsf + usec);
if (AR_SREV_9280_20_OR_LATER(ah))
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 51/55] clocksource: Make delta calculation a function
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (49 preceding siblings ...)
2014-07-11 13:45 ` [patch 50/55] wireless: ath9k: Get rid of timespec conversions Thomas Gleixner
@ 2014-07-11 13:45 ` Thomas Gleixner
2014-07-14 8:31 ` Peter Zijlstra
2014-07-16 3:50 ` John Stultz
2014-07-11 13:45 ` [patch 52/55] clocksource: Move cycle_last validation to core code Thomas Gleixner
` (3 subsequent siblings)
54 siblings, 2 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:45 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: clocksource-make-delta-calculation-a-function.patch --]
[-- Type: text/plain, Size: 5797 bytes --]
We want to move the TSC sanity check into core code to make NMI safe
accessors to clock monotonic[_raw] possible. For this we need to
sanity check the delta calculation. Create a helper function and
convert all sites to use it.
[ Build fix from jstultz ]
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/time/clocksource.c | 12 +++++++-----
kernel/time/timekeeping.c | 27 ++++++++++++++-------------
kernel/time/timekeeping_internal.h | 6 ++++++
3 files changed, 27 insertions(+), 18 deletions(-)
Index: tip/kernel/time/clocksource.c
===================================================================
--- tip.orig/kernel/time/clocksource.c
+++ tip/kernel/time/clocksource.c
@@ -32,6 +32,7 @@
#include <linux/kthread.h>
#include "tick-internal.h"
+#include "timekeeping_internal.h"
void timecounter_init(struct timecounter *tc,
const struct cyclecounter *cc,
@@ -249,7 +250,7 @@ void clocksource_mark_unstable(struct cl
static void clocksource_watchdog(unsigned long data)
{
struct clocksource *cs;
- cycle_t csnow, wdnow;
+ cycle_t csnow, wdnow, delta;
int64_t wd_nsec, cs_nsec;
int next_cpu, reset_pending;
@@ -282,11 +283,12 @@ static void clocksource_watchdog(unsigne
continue;
}
- wd_nsec = clocksource_cyc2ns((wdnow - cs->wd_last) & watchdog->mask,
- watchdog->mult, watchdog->shift);
+ delta = clocksource_delta(wdnow, cs->wd_last, watchdog->mask);
+ wd_nsec = clocksource_cyc2ns(delta, watchdog->mult,
+ watchdog->shift);
- cs_nsec = clocksource_cyc2ns((csnow - cs->cs_last) &
- cs->mask, cs->mult, cs->shift);
+ delta = clocksource_delta(csnow, cs->cs_last, cs->mask);
+ cs_nsec = clocksource_cyc2ns(delta, cs->mult, cs->shift);
cs->cs_last = csnow;
cs->wd_last = wdnow;
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -197,7 +197,7 @@ static inline u32 arch_gettimeoffset(voi
static inline s64 timekeeping_get_ns(struct timekeeper *tk)
{
- cycle_t cycle_now, cycle_delta;
+ cycle_t cycle_now, delta;
struct clocksource *clock;
s64 nsec;
@@ -206,9 +206,9 @@ static inline s64 timekeeping_get_ns(str
cycle_now = clock->read(clock);
/* calculate the delta since the last update_wall_time: */
- cycle_delta = (cycle_now - clock->cycle_last) & clock->mask;
+ delta = clocksource_delta(cycle_now, clock->cycle_last, clock->mask);
- nsec = cycle_delta * tk->mult + tk->xtime_nsec;
+ nsec = delta * tk->mult + tk->xtime_nsec;
nsec >>= tk->shift;
/* If arch requires, add in get_arch_timeoffset() */
@@ -217,7 +217,7 @@ static inline s64 timekeeping_get_ns(str
static inline s64 timekeeping_get_ns_raw(struct timekeeper *tk)
{
- cycle_t cycle_now, cycle_delta;
+ cycle_t cycle_now, delta;
struct clocksource *clock;
s64 nsec;
@@ -226,10 +226,10 @@ static inline s64 timekeeping_get_ns_raw
cycle_now = clock->read(clock);
/* calculate the delta since the last update_wall_time: */
- cycle_delta = (cycle_now - clock->cycle_last) & clock->mask;
+ delta = clocksource_delta(cycle_now, clock->cycle_last, clock->mask);
/* convert delta to nanoseconds. */
- nsec = clocksource_cyc2ns(cycle_delta, clock->mult, clock->shift);
+ nsec = clocksource_cyc2ns(delta, clock->mult, clock->shift);
/* If arch requires, add in get_arch_timeoffset() */
return nsec + arch_gettimeoffset();
@@ -335,16 +335,15 @@ static void timekeeping_update(struct ti
*/
static void timekeeping_forward_now(struct timekeeper *tk)
{
- cycle_t cycle_now, cycle_delta;
+ cycle_t cycle_now, delta;
struct clocksource *clock;
s64 nsec;
clock = tk->clock;
cycle_now = clock->read(clock);
- cycle_delta = (cycle_now - clock->cycle_last) & clock->mask;
- tk->cycle_last = clock->cycle_last = cycle_now;
+ delta = clocksource_delta(cycle_now, clock->cycle_last, clock->mask);
- tk->xtime_nsec += cycle_delta * tk->mult;
+ tk->xtime_nsec += delta * tk->mult;
/* If arch requires, add in get_arch_timeoffset() */
tk->xtime_nsec += (u64)arch_gettimeoffset() << tk->shift;
@@ -352,7 +351,7 @@ static void timekeeping_forward_now(stru
/* Updates tk->base_mono as well */
tk_normalize_xtime(tk);
- nsec = clocksource_cyc2ns(cycle_delta, clock->mult, clock->shift);
+ nsec = clocksource_cyc2ns(delta, clock->mult, clock->shift);
timespec64_add_ns(&tk->raw_time, nsec);
tk->base_raw = ktime_add_ns(tk->base_raw, nsec);
}
@@ -1063,7 +1062,8 @@ static void timekeeping_resume(void)
u32 shift = clock->shift;
s64 nsec = 0;
- cycle_delta = (cycle_now - clock->cycle_last) & clock->mask;
+ cycle_delta = clocksource_delta(cycle_now, clock->cycle_last,
+ clock->mask);
/*
* "cycle_delta * mutl" may cause 64 bits overflow, if the
@@ -1472,7 +1472,8 @@ void update_wall_time(void)
#ifdef CONFIG_ARCH_USES_GETTIMEOFFSET
offset = real_tk->cycle_interval;
#else
- offset = (clock->read(clock) - clock->cycle_last) & clock->mask;
+ offset = clocksource_delta(clock->read(clock), clock->cycle_last,
+ clock->mask);
#endif
/* Check if there's really nothing to do */
Index: tip/kernel/time/timekeeping_internal.h
===================================================================
--- tip.orig/kernel/time/timekeeping_internal.h
+++ tip/kernel/time/timekeeping_internal.h
@@ -3,6 +3,7 @@
/*
* timekeeping debug functions
*/
+#include <linux/clocksource.h>
#include <linux/time.h>
#ifdef CONFIG_DEBUG_FS
@@ -11,4 +12,9 @@ extern void tk_debug_account_sleep_time(
#define tk_debug_account_sleep_time(x)
#endif
+static inline cycle_t clocksource_delta(cycle_t now, cycle_t last, cycle_t mask)
+{
+ return (now -last) & mask;
+}
+
#endif /* _TIMEKEEPING_INTERNAL_H */
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 51/55] clocksource: Make delta calculation a function
2014-07-11 13:45 ` [patch 51/55] clocksource: Make delta calculation a function Thomas Gleixner
@ 2014-07-14 8:31 ` Peter Zijlstra
2014-07-16 3:50 ` John Stultz
1 sibling, 0 replies; 91+ messages in thread
From: Peter Zijlstra @ 2014-07-14 8:31 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz
[-- Attachment #1: Type: text/plain, Size: 222 bytes --]
On Fri, Jul 11, 2014 at 01:45:15PM -0000, Thomas Gleixner wrote:
> +static inline cycle_t clocksource_delta(cycle_t now, cycle_t last, cycle_t mask)
> +{
> + return (now -last) & mask;
> +}
Inconsistent whitespace there.
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 51/55] clocksource: Make delta calculation a function
2014-07-11 13:45 ` [patch 51/55] clocksource: Make delta calculation a function Thomas Gleixner
2014-07-14 8:31 ` Peter Zijlstra
@ 2014-07-16 3:50 ` John Stultz
2014-07-16 6:58 ` Thomas Gleixner
1 sibling, 1 reply; 91+ messages in thread
From: John Stultz @ 2014-07-16 3:50 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, Peter Zijlstra
On Fri, Jul 11, 2014 at 6:45 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> @@ -335,16 +335,15 @@ static void timekeeping_update(struct ti
> */
> static void timekeeping_forward_now(struct timekeeper *tk)
> {
> - cycle_t cycle_now, cycle_delta;
> + cycle_t cycle_now, delta;
> struct clocksource *clock;
> s64 nsec;
>
> clock = tk->clock;
> cycle_now = clock->read(clock);
> - cycle_delta = (cycle_now - clock->cycle_last) & clock->mask;
> - tk->cycle_last = clock->cycle_last = cycle_now;
> + delta = clocksource_delta(cycle_now, clock->cycle_last, clock->mask);
Errr.. Was dropping the update to tk->cycle_last and clock->cycle_last
intended? That seems broken.
thanks
-john
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 51/55] clocksource: Make delta calculation a function
2014-07-16 3:50 ` John Stultz
@ 2014-07-16 6:58 ` Thomas Gleixner
0 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-16 6:58 UTC (permalink / raw)
To: John Stultz; +Cc: LKML, Peter Zijlstra
On Tue, 15 Jul 2014, John Stultz wrote:
> On Fri, Jul 11, 2014 at 6:45 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> > @@ -335,16 +335,15 @@ static void timekeeping_update(struct ti
> > */
> > static void timekeeping_forward_now(struct timekeeper *tk)
> > {
> > - cycle_t cycle_now, cycle_delta;
> > + cycle_t cycle_now, delta;
> > struct clocksource *clock;
> > s64 nsec;
> >
> > clock = tk->clock;
> > cycle_now = clock->read(clock);
> > - cycle_delta = (cycle_now - clock->cycle_last) & clock->mask;
> > - tk->cycle_last = clock->cycle_last = cycle_now;
> > + delta = clocksource_delta(cycle_now, clock->cycle_last, clock->mask);
>
> Errr.. Was dropping the update to tk->cycle_last and clock->cycle_last
> intended? That seems broken.
Grr, was definitely not intended.
^ permalink raw reply [flat|nested] 91+ messages in thread
* [patch 52/55] clocksource: Move cycle_last validation to core code
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (50 preceding siblings ...)
2014-07-11 13:45 ` [patch 51/55] clocksource: Make delta calculation a function Thomas Gleixner
@ 2014-07-11 13:45 ` Thomas Gleixner
2014-07-11 13:45 ` [patch 53/55] seqcount: Provide raw_read_seqcount() Thomas Gleixner
` (2 subsequent siblings)
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:45 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: clocksource-move-cycle_last-validation-to-core.patch --]
[-- Type: text/plain, Size: 4102 bytes --]
The only user of the cycle_last validation is the x86 TSC. In order to
provide NMI safe accessor functions for clock monotonic and
monotonic_raw we need to do that in the core.
We can't do the TSC specific
if (now < cycle_last)
now = cycle_last;
for the other wrapping around clocksources, but TSC has
CLOCKSOURCE_MASK(64) which actually does not mask out anything so if
now is less than cycle_last the subtraction will give a negative
result. So we can check for that in clocksource_delta() and return 0
for that case.
Implement and enable it for x86
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/Kconfig | 1 +
arch/x86/kernel/tsc.c | 14 +++++++++-----
kernel/time/Kconfig | 5 +++++
kernel/time/timekeeping_internal.h | 9 +++++++++
4 files changed, 24 insertions(+), 5 deletions(-)
Index: tip/arch/x86/Kconfig
===================================================================
--- tip.orig/arch/x86/Kconfig
+++ tip/arch/x86/Kconfig
@@ -109,6 +109,7 @@ config X86
select CLOCKSOURCE_WATCHDOG
select GENERIC_CLOCKEVENTS
select ARCH_CLOCKSOURCE_DATA
+ select CLOCKSOURCE_VALIDATE_LAST_CYCLE
select GENERIC_CLOCKEVENTS_BROADCAST if X86_64 || (X86_32 && X86_LOCAL_APIC)
select GENERIC_TIME_VSYSCALL
select GENERIC_STRNCPY_FROM_USER
Index: tip/arch/x86/kernel/tsc.c
===================================================================
--- tip.orig/arch/x86/kernel/tsc.c
+++ tip/arch/x86/kernel/tsc.c
@@ -951,7 +951,7 @@ core_initcall(cpufreq_tsc);
static struct clocksource clocksource_tsc;
/*
- * We compare the TSC to the cycle_last value in the clocksource
+ * We used to compare the TSC to the cycle_last value in the clocksource
* structure to avoid a nasty time-warp. This can be observed in a
* very small window right after one CPU updated cycle_last under
* xtime/vsyscall_gtod lock and the other CPU reads a TSC value which
@@ -961,13 +961,14 @@ static struct clocksource clocksource_ts
* due to the unsigned delta calculation of the time keeping core
* code, which is necessary to support wrapping clocksources like pm
* timer.
+ *
+ * This sanity check is now done in the core timekeeping code.
+ * checking the result of read_tsc() - cycle_last for being negative.
+ * That works because CLOCKSOURCE_MASK(64) does not mask out any bit.
*/
static cycle_t read_tsc(struct clocksource *cs)
{
- cycle_t ret = (cycle_t)get_cycles();
-
- return ret >= clocksource_tsc.cycle_last ?
- ret : clocksource_tsc.cycle_last;
+ return (cycle_t)get_cycles();
}
static void resume_tsc(struct clocksource *cs)
@@ -976,6 +977,9 @@ static void resume_tsc(struct clocksourc
clocksource_tsc.cycle_last = 0;
}
+/*
+ * .mask MUST be CLOCKSOURCE_MASK(64). See comment above read_tsc()
+ */
static struct clocksource clocksource_tsc = {
.name = "tsc",
.rating = 300,
Index: tip/kernel/time/Kconfig
===================================================================
--- tip.orig/kernel/time/Kconfig
+++ tip/kernel/time/Kconfig
@@ -12,6 +12,11 @@ config CLOCKSOURCE_WATCHDOG
config ARCH_CLOCKSOURCE_DATA
bool
+# Clocksources require validation of the clocksource against the last
+# cycle update - x86/TSC misfeature
+config CLOCKSOURCE_VALIDATE_LAST_CYCLE
+ bool
+
# Timekeeping vsyscall support
config GENERIC_TIME_VSYSCALL
bool
Index: tip/kernel/time/timekeeping_internal.h
===================================================================
--- tip.orig/kernel/time/timekeeping_internal.h
+++ tip/kernel/time/timekeeping_internal.h
@@ -12,9 +12,18 @@ extern void tk_debug_account_sleep_time(
#define tk_debug_account_sleep_time(x)
#endif
+#ifdef CONFIG_CLOCKSOURCE_VALIDATE_LAST_CYCLE
+static inline cycle_t clocksource_delta(cycle_t now, cycle_t last, cycle_t mask)
+{
+ cycle_t ret = (now -last) & mask;
+
+ return (s64) ret > 0 ? ret : 0;
+}
+#else
static inline cycle_t clocksource_delta(cycle_t now, cycle_t last, cycle_t mask)
{
return (now -last) & mask;
}
+#endif
#endif /* _TIMEKEEPING_INTERNAL_H */
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 53/55] seqcount: Provide raw_read_seqcount()
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (51 preceding siblings ...)
2014-07-11 13:45 ` [patch 52/55] clocksource: Move cycle_last validation to core code Thomas Gleixner
@ 2014-07-11 13:45 ` Thomas Gleixner
2014-07-11 13:45 ` [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW] Thomas Gleixner
2014-07-11 13:45 ` [patch 55/55] ftrace: Provide trace clocks mono and mono_raw Thomas Gleixner
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:45 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Ingo Molnar
[-- Attachment #1: seqcount-add-raw-read-seqcount.patch --]
[-- Type: text/plain, Size: 1384 bytes --]
raw_read_seqcount opens a read critical section of the given seqcount
without any lockdep checking and without checking or masking the
LSB. Calling code is responsible for handling that.
Preparatory patch to provide NMI safe clock monotonic[_raw] accessor
functions.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: John Stultz <john.stultz@linaro.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
---
include/linux/seqlock.h | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
Index: tip/include/linux/seqlock.h
===================================================================
--- tip.orig/include/linux/seqlock.h
+++ tip/include/linux/seqlock.h
@@ -117,6 +117,22 @@ repeat:
}
/**
+ * raw_read_seqcount - Read the raw seqcount
+ * @s: pointer to seqcount_t
+ * Returns: count to be passed to read_seqcount_retry
+ *
+ * raw_read_seqcount opens a read critical section of the given
+ * seqcount without any lockdep checking and without checking or
+ * masking the LSB. Calling code is responsible for handling that.
+ */
+static inline unsigned raw_read_seqcount(const seqcount_t *s)
+{
+ unsigned ret = ACCESS_ONCE(s->sequence);
+ smp_rmb();
+ return ret;
+}
+
+/**
* raw_read_seqcount_begin - start seq-read critical section w/o lockdep
* @s: pointer to seqcount_t
* Returns: count to be passed to read_seqcount_retry
^ permalink raw reply [flat|nested] 91+ messages in thread* [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (52 preceding siblings ...)
2014-07-11 13:45 ` [patch 53/55] seqcount: Provide raw_read_seqcount() Thomas Gleixner
@ 2014-07-11 13:45 ` Thomas Gleixner
2014-07-11 20:04 ` Mathieu Desnoyers
` (3 more replies)
2014-07-11 13:45 ` [patch 55/55] ftrace: Provide trace clocks mono and mono_raw Thomas Gleixner
54 siblings, 4 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:45 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Steven Rostedt, Mathieu Desnoyers
[-- Attachment #1: timekeeping-nmi-safe-access-to-mono-raw.patch --]
[-- Type: text/plain, Size: 10133 bytes --]
Tracers want a correlated time between the kernel instrumentation and
user space. We really do not want to export sched_clock() to user
space, so we need to provide something sensible for this.
Using separate data structures with an non blocking sequence count
based update mechanism allows us to do that. The data structure
required for the readout has a sequence counter and two copies of the
timekeeping data.
On the update side:
tkf->seq++;
smp_wmb();
update(tkf->base[0], tk;
tkf->seq++;
smp_wmb();
update(tkf->base[1], tk;
On the reader side:
do {
seq = tkf->seq;
smp_rmb();
idx = seq & 0x01;
now = now(tkf->base[idx]);
smp_rmb();
} while (seq != tkf->seq)
So if NMI hits the update of base[0] it will use base[1] which is
still consistent. In case of CLOCK_MONOTONIC this can result in
slightly wrong timestamps (a few nanoseconds) accross an update. Not a
big issue for the intended use case.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
include/linux/timekeeping.h | 2
kernel/time/timekeeping.c | 208 ++++++++++++++++++++++++++++++++++++++------
2 files changed, 183 insertions(+), 27 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -164,6 +164,8 @@ static inline u64 ktime_get_raw_ns(void)
return ktime_to_ns(ktime_get_raw());
}
+extern u64 ktime_get_mono_fast_ns(void);
+
/*
* Timespec interfaces utilizing the ktime based ones
*/
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -50,6 +50,42 @@ int __read_mostly timekeeping_suspended;
/* Flag for if there is a persistent clock on this platform */
bool __read_mostly persistent_clock_exist = false;
+/**
+ * struct tk_fast_base - timekeeper data for NMI safe fast access
+ * @clock: Pointer to the clocksource
+ * @cycle_last: The reference cycles for delta calculation
+ * @base: The base value for the readout
+ * @shift: Shift factor for scaled math
+ * @mult: Mult factor for scaled math
+ *
+ * Note: We store cycle_last independent from clock->cycle_last so the
+ * update of the real timekeeper does not disturb the fast ones.
+ */
+struct tk_fast_base {
+ struct clocksource *clock;
+ cycle_t cycle_last;
+ u64 base;
+ u32 shift;
+ u32 mult;
+};
+
+/**
+ * struct tk_fast - NMI safe timekeeper
+ * @seq: Sequence counter for protecting updates. The lowest bit
+ * is the index for the tk_fast_base array
+ * @base: tk_fast_base array. Access is indexed by the lowest bit of
+ * @seq.
+ *
+ * See @update_fast_timekeeper() below.
+ */
+struct tk_fast {
+ seqcount_t seq;
+ struct tk_fast_base base[2];
+};
+
+static struct tk_fast tk_fast_raw ____cacheline_aligned;
+static struct tk_fast tk_fast_mono ____cacheline_aligned;
+
/*
* The xtime based monotonic readout is:
* nsec = (xtime_sec + wtm_sec) * 1e9 + wtm_nsec + now();
@@ -215,7 +251,7 @@ static inline s64 timekeeping_get_ns(str
return nsec + arch_gettimeoffset();
}
-static inline s64 timekeeping_get_ns_raw(struct timekeeper *tk)
+static inline s64 notrace timekeeping_get_ns_raw(struct tk_fast_base *tk)
{
cycle_t cycle_now, delta;
struct clocksource *clock;
@@ -226,7 +262,7 @@ static inline s64 timekeeping_get_ns_raw
cycle_now = clock->read(clock);
/* calculate the delta since the last update_wall_time: */
- delta = clocksource_delta(cycle_now, clock->cycle_last, clock->mask);
+ delta = clocksource_delta(cycle_now, tk->cycle_last, clock->mask);
/* convert delta to nanoseconds. */
nsec = clocksource_cyc2ns(delta, clock->mult, clock->shift);
@@ -235,6 +271,136 @@ static inline s64 timekeeping_get_ns_raw
return nsec + arch_gettimeoffset();
}
+/**
+ * update_fast_timekeeper - Update the fast and NMI safe monotonic timekeeper.
+ * @tk: The timekeeper from which we take the update
+ * @tkf: The fast timekeeper to update
+ * @tbase: The time base for the fast timekeeper (mono/raw)
+ *
+ * We want to use this from any context including NMI and tracing /
+ * instrumenting the timekeeping code itself.
+ *
+ * So we handle this differently than the other timekeeping accessor
+ * functions which retry when the sequence count has changed. The
+ * update side does:
+ *
+ * tkf->seq++;
+ * smp_wmb();
+ * update(tkf->base[0], tk;
+ * tkf->seq++;
+ * smp_wmb();
+ * update(tkf->base[1], tk;
+ *
+ * The reader side does:
+ *
+ * do {
+ * seq = tkf->seq;
+ * smp_rmb();
+ * idx = seq & 0x01;
+ * now = now(tkf->base[idx]);
+ * smp_rmb();
+ * } while (seq != tkf->seq)
+ *
+ * As long as we update base[0] readers are forced off to
+ * base[1]. Once base[0] is updated readers are redirected to base[0]
+ * and the base[1] update takes place.
+ *
+ * Soif NMI hits the update of base[0] then it will use base[1] which
+ * is still consistent. In the worst case this can result is a
+ * slightly wrong timestamp (a few nanoseconds) for CLOCK_MONOTONIC
+ * only. Tracing and instrumentation is blury anyway, so this is not
+ * really an issue.
+ */
+static void update_fast_timekeeper(struct clocksource *clk, struct tk_fast *tkf,
+ s64 tbase, u32 mult, u32 shift)
+{
+ struct tk_fast_base *base = tkf->base;
+
+ /* Force readers off to base[1] */
+ raw_write_seqcount_begin(&tkf->seq);
+
+ /* Update base[0] */
+ base->clock = clk;
+ base->cycle_last = clk->cycle_last;
+ base->base = tbase;
+ base->shift = shift;
+ base->mult = mult;
+
+ /* Force readers back to base[0] */
+ raw_write_seqcount_end(&tkf->seq);
+
+ /* Update base[1] */
+ base++;
+ base->clock = clk;
+ base->cycle_last = clk->cycle_last;
+ base->base = tbase;
+ base->shift = shift;
+ base->mult = mult;
+}
+
+static void update_fast_timekeepers(struct timekeeper *tk)
+{
+ struct clocksource *clk = tk->clock;
+ s64 base;
+
+ /*
+ * Calulate the monotonic base in nano seconds. That's less
+ * accurate than the real monotonic time as we drop the
+ * fractial nsecs of xtime_nsec with the shift. But good
+ * enough for the fast stuff we want.
+ */
+ base = ktime_to_ns(tk->base_mono) + (tk->xtime_nsec >> tk->shift);
+ update_fast_timekeeper(clk, &tk_fast_mono, base, tk->mult, tk->shift);
+ /* Update the raw timekeeper */
+ base = ktime_to_ns(tk->base_raw);
+ update_fast_timekeeper(clk, &tk_fast_raw, base, clk->mult, clk->shift);
+}
+
+/*
+ * The reader function for the fast NMI safe timekeepers.
+ */
+static u64 notrace ktime_get_fast_ns(struct tk_fast *tkf)
+{
+ struct tk_fast_base *b;
+ unsigned int seq;
+ u64 now;
+
+ do {
+ seq = raw_read_seqcount(&tkf->seq);
+ b = tkf->base + (seq & 0x01);
+ now = b->base + timekeeping_get_ns_raw(b);
+
+ } while (read_seqcount_retry(&tkf->seq, seq));
+ return now;
+}
+
+/**
+ * ktime_get_raw - Returns the raw monotonic time in ktime_t format
+ *
+ * Can be called from any context including NMI
+ */
+ktime_t notrace ktime_get_raw(void)
+{
+ return ns_to_ktime(ktime_get_fast_ns(&tk_fast_raw));
+}
+EXPORT_SYMBOL_GPL(ktime_get_raw);
+
+/**
+ * ktime_get_mono_fast_ns - Fast NMI safe access to clock monotonic
+ *
+ * This timestamp is not guaranteed to be monotonic because the
+ * nanoseconds reminder of the base time is not accounted. So accross
+ * an update time can go slighty backwards in the single digit
+ * nanoseconds range, if the mult/shift factors are adjusted by the
+ * update. So don't use this for code which might be sensitive about
+ * that. For the intended use case of tracing and instrumentation its
+ * a non issue.
+ */
+u64 notrace ktime_get_mono_fast_ns(void)
+{
+ return ktime_get_fast_ns(&tk_fast_mono);
+}
+
#ifdef CONFIG_GENERIC_TIME_VSYSCALL_OLD
static inline void update_vsyscall(struct timekeeper *tk)
@@ -324,6 +490,8 @@ static void timekeeping_update(struct ti
if (action & TK_MIRROR)
memcpy(&shadow_timekeeper, &tk_core.timekeeper,
sizeof(tk_core.timekeeper));
+
+ update_fast_timekeepers(tk);
}
/**
@@ -470,27 +638,6 @@ ktime_t ktime_mono_to_any(ktime_t tmono,
EXPORT_SYMBOL_GPL(ktime_mono_to_any);
/**
- * ktime_get_raw - Returns the raw monotonic time in ktime_t format
- */
-ktime_t ktime_get_raw(void)
-{
- struct timekeeper *tk = &tk_core.timekeeper;
- unsigned int seq;
- ktime_t base;
- s64 nsecs;
-
- do {
- seq = read_seqcount_begin(&tk_core.seq);
- base = tk->base_raw;
- nsecs = timekeeping_get_ns_raw(tk);
-
- } while (read_seqcount_retry(&tk_core.seq, seq));
-
- return ktime_add_ns(base, nsecs);
-}
-EXPORT_SYMBOL_GPL(ktime_get_raw);
-
-/**
* ktime_get_ts64 - get the monotonic clock in timespec64 format
* @ts: pointer to timespec variable
*
@@ -574,13 +721,19 @@ void getnstime_raw_and_real(struct times
do {
seq = read_seqcount_begin(&tk_core.seq);
- *ts_raw = timespec64_to_timespec(tk->raw_time);
ts_real->tv_sec = tk->xtime_sec;
ts_real->tv_nsec = 0;
-
- nsecs_raw = timekeeping_get_ns_raw(tk);
nsecs_real = timekeeping_get_ns(tk);
+ /*
+ * base[0] of tk_fast_raw is valid here as we are
+ * protected by the tk_core.seq counter. The raw_base
+ * has it's own sequence counter, but that is updated
+ * under tk_core.seq.
+ */
+ *ts_raw = timespec64_to_timespec(tk->raw_time);
+ nsecs_raw = timekeeping_get_ns_raw(tk_fast_raw.base);
+
} while (read_seqcount_retry(&tk_core.seq, seq));
timespec_add_ns(ts_raw, nsecs_raw);
@@ -813,7 +966,7 @@ void getrawmonotonic(struct timespec *ts
do {
seq = read_seqcount_begin(&tk_core.seq);
- nsecs = timekeeping_get_ns_raw(tk);
+ nsecs = timekeeping_get_ns_raw(tk_fast_raw.base);
ts64 = tk->raw_time;
} while (read_seqcount_retry(&tk_core.seq, seq));
@@ -946,6 +1099,7 @@ void __init timekeeping_init(void)
memcpy(&shadow_timekeeper, &tk_core.timekeeper,
sizeof(tk_core.timekeeper));
+ update_fast_timekeepers(tk);
write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
2014-07-11 13:45 ` [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW] Thomas Gleixner
@ 2014-07-11 20:04 ` Mathieu Desnoyers
2014-07-12 8:11 ` Thomas Gleixner
2014-07-12 14:53 ` Mathieu Desnoyers
` (2 subsequent siblings)
3 siblings, 1 reply; 91+ messages in thread
From: Mathieu Desnoyers @ 2014-07-11 20:04 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Peter Zijlstra, Steven Rostedt
----- Original Message -----
> From: "Thomas Gleixner" <tglx@linutronix.de>
> To: "LKML" <linux-kernel@vger.kernel.org>
> Cc: "John Stultz" <john.stultz@linaro.org>, "Peter Zijlstra" <peterz@infradead.org>, "Steven Rostedt"
> <rostedt@goodmis.org>, "Mathieu Desnoyers" <mathieu.desnoyers@efficios.com>
> Sent: Friday, July 11, 2014 9:45:19 AM
> Subject: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
>
Hi Thomas,
Thanks for submitting this patch. It will be very useful for tracing!
A few comments,
> Tracers want a correlated time between the kernel instrumentation and
> user space. We really do not want to export sched_clock() to user
> space, so we need to provide something sensible for this.
> Using separate data structures with an non blocking sequence count
"an non blocking" -> "a non-blocking"
> based update mechanism allows us to do that. The data structure
> required for the readout has a sequence counter and two copies of the
> timekeeping data.
>
> On the update side:
>
> tkf->seq++;
> smp_wmb();
> update(tkf->base[0], tk;
missing ")"
> tkf->seq++;
> smp_wmb();
> update(tkf->base[1], tk;
missing ")"
Any reason why the updater wouldn't do:
tkf->seq++;
smp_wmb();
update(tkf->base[1 - (tkf->seq & 0x01)], tk);
instead of updating both array entries each time ?
>
> On the reader side:
>
> do {
> seq = tkf->seq;
> smp_rmb();
> idx = seq & 0x01;
> now = now(tkf->base[idx]);
> smp_rmb();
> } while (seq != tkf->seq)
>
> So if NMI hits the update of base[0] it will use base[1] which is
> still consistent. In case of CLOCK_MONOTONIC this can result in
> slightly wrong timestamps (a few nanoseconds) accross an update. Not a
"accross" -> "across"
> big issue for the intended use case.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> ---
> include/linux/timekeeping.h | 2
> kernel/time/timekeeping.c | 208
> ++++++++++++++++++++++++++++++++++++++------
> 2 files changed, 183 insertions(+), 27 deletions(-)
>
> Index: tip/include/linux/timekeeping.h
> ===================================================================
> --- tip.orig/include/linux/timekeeping.h
> +++ tip/include/linux/timekeeping.h
> @@ -164,6 +164,8 @@ static inline u64 ktime_get_raw_ns(void)
> return ktime_to_ns(ktime_get_raw());
> }
>
> +extern u64 ktime_get_mono_fast_ns(void);
> +
> /*
> * Timespec interfaces utilizing the ktime based ones
> */
> Index: tip/kernel/time/timekeeping.c
> ===================================================================
> --- tip.orig/kernel/time/timekeeping.c
> +++ tip/kernel/time/timekeeping.c
> @@ -50,6 +50,42 @@ int __read_mostly timekeeping_suspended;
> /* Flag for if there is a persistent clock on this platform */
> bool __read_mostly persistent_clock_exist = false;
>
> +/**
> + * struct tk_fast_base - timekeeper data for NMI safe fast access
> + * @clock: Pointer to the clocksource
> + * @cycle_last: The reference cycles for delta calculation
> + * @base: The base value for the readout
> + * @shift: Shift factor for scaled math
> + * @mult: Mult factor for scaled math
> + *
> + * Note: We store cycle_last independent from clock->cycle_last so the
> + * update of the real timekeeper does not disturb the fast ones.
> + */
> +struct tk_fast_base {
> + struct clocksource *clock;
> + cycle_t cycle_last;
> + u64 base;
> + u32 shift;
> + u32 mult;
> +};
> +
> +/**
> + * struct tk_fast - NMI safe timekeeper
> + * @seq: Sequence counter for protecting updates. The lowest bit
> + * is the index for the tk_fast_base array
> + * @base: tk_fast_base array. Access is indexed by the lowest bit of
> + * @seq.
> + *
> + * See @update_fast_timekeeper() below.
> + */
> +struct tk_fast {
> + seqcount_t seq;
> + struct tk_fast_base base[2];
> +};
> +
> +static struct tk_fast tk_fast_raw ____cacheline_aligned;
> +static struct tk_fast tk_fast_mono ____cacheline_aligned;
> +
> /*
> * The xtime based monotonic readout is:
> * nsec = (xtime_sec + wtm_sec) * 1e9 + wtm_nsec + now();
> @@ -215,7 +251,7 @@ static inline s64 timekeeping_get_ns(str
> return nsec + arch_gettimeoffset();
> }
>
> -static inline s64 timekeeping_get_ns_raw(struct timekeeper *tk)
> +static inline s64 notrace timekeeping_get_ns_raw(struct tk_fast_base *tk)
So here, am I correct in saying that CLOCK_MONOTONIC_RAW would now
use this implementation ? Why can we assume that the tk_fast_base will
ensure that time never goes even slightly backwards from the point of
view of a thread ?
> {
> cycle_t cycle_now, delta;
> struct clocksource *clock;
> @@ -226,7 +262,7 @@ static inline s64 timekeeping_get_ns_raw
> cycle_now = clock->read(clock);
>
> /* calculate the delta since the last update_wall_time: */
> - delta = clocksource_delta(cycle_now, clock->cycle_last, clock->mask);
> + delta = clocksource_delta(cycle_now, tk->cycle_last, clock->mask);
>
> /* convert delta to nanoseconds. */
> nsec = clocksource_cyc2ns(delta, clock->mult, clock->shift);
> @@ -235,6 +271,136 @@ static inline s64 timekeeping_get_ns_raw
> return nsec + arch_gettimeoffset();
> }
>
> +/**
> + * update_fast_timekeeper - Update the fast and NMI safe monotonic
> timekeeper.
> + * @tk: The timekeeper from which we take the update
> + * @tkf: The fast timekeeper to update
> + * @tbase: The time base for the fast timekeeper (mono/raw)
> + *
> + * We want to use this from any context including NMI and tracing /
> + * instrumenting the timekeeping code itself.
> + *
> + * So we handle this differently than the other timekeeping accessor
> + * functions which retry when the sequence count has changed. The
> + * update side does:
> + *
> + * tkf->seq++;
> + * smp_wmb();
> + * update(tkf->base[0], tk;
missing ")";
> + * tkf->seq++;
> + * smp_wmb();
> + * update(tkf->base[1], tk;
missing ")".
> + *
> + * The reader side does:
> + *
> + * do {
> + * seq = tkf->seq;
> + * smp_rmb();
> + * idx = seq & 0x01;
> + * now = now(tkf->base[idx]);
> + * smp_rmb();
> + * } while (seq != tkf->seq)
> + *
> + * As long as we update base[0] readers are forced off to
> + * base[1]. Once base[0] is updated readers are redirected to base[0]
> + * and the base[1] update takes place.
> + *
> + * Soif NMI hits the update of base[0] then it will use base[1] which
"Soif" -> "So if"
> + * is still consistent. In the worst case this can result is a
> + * slightly wrong timestamp (a few nanoseconds) for CLOCK_MONOTONIC
> + * only. Tracing and instrumentation is blury anyway, so this is not
> + * really an issue.
A time source can be "slightly wrong" without ever going backwards from the
POV of a thread. We might want to explicitly spell out that time can go
slightly backward from the POV of a single thread, and that the caller
should expect this.
> + */
> +static void update_fast_timekeeper(struct clocksource *clk, struct tk_fast
> *tkf,
> + s64 tbase, u32 mult, u32 shift)
> +{
> + struct tk_fast_base *base = tkf->base;
> +
> + /* Force readers off to base[1] */
> + raw_write_seqcount_begin(&tkf->seq);
> +
> + /* Update base[0] */
> + base->clock = clk;
> + base->cycle_last = clk->cycle_last;
> + base->base = tbase;
> + base->shift = shift;
> + base->mult = mult;
> +
> + /* Force readers back to base[0] */
> + raw_write_seqcount_end(&tkf->seq);
> +
> + /* Update base[1] */
> + base++;
> + base->clock = clk;
> + base->cycle_last = clk->cycle_last;
> + base->base = tbase;
> + base->shift = shift;
> + base->mult = mult;
> +}
> +
> +static void update_fast_timekeepers(struct timekeeper *tk)
> +{
> + struct clocksource *clk = tk->clock;
> + s64 base;
> +
> + /*
> + * Calulate the monotonic base in nano seconds. That's less
> + * accurate than the real monotonic time as we drop the
> + * fractial nsecs of xtime_nsec with the shift. But good
> + * enough for the fast stuff we want.
> + */
> + base = ktime_to_ns(tk->base_mono) + (tk->xtime_nsec >> tk->shift);
> + update_fast_timekeeper(clk, &tk_fast_mono, base, tk->mult, tk->shift);
> + /* Update the raw timekeeper */
> + base = ktime_to_ns(tk->base_raw);
> + update_fast_timekeeper(clk, &tk_fast_raw, base, clk->mult, clk->shift);
> +}
> +
> +/*
> + * The reader function for the fast NMI safe timekeepers.
> + */
> +static u64 notrace ktime_get_fast_ns(struct tk_fast *tkf)
> +{
> + struct tk_fast_base *b;
> + unsigned int seq;
> + u64 now;
> +
> + do {
> + seq = raw_read_seqcount(&tkf->seq);
> + b = tkf->base + (seq & 0x01);
> + now = b->base + timekeeping_get_ns_raw(b);
> +
> + } while (read_seqcount_retry(&tkf->seq, seq));
> + return now;
> +}
> +
> +/**
> + * ktime_get_raw - Returns the raw monotonic time in ktime_t format
> + *
> + * Can be called from any context including NMI
> + */
> +ktime_t notrace ktime_get_raw(void)
> +{
> + return ns_to_ktime(ktime_get_fast_ns(&tk_fast_raw));
> +}
> +EXPORT_SYMBOL_GPL(ktime_get_raw);
> +
> +/**
> + * ktime_get_mono_fast_ns - Fast NMI safe access to clock monotonic
> + *
> + * This timestamp is not guaranteed to be monotonic because the
> + * nanoseconds reminder of the base time is not accounted. So accross
"accross" -> "across"
"reminder" -> "remainder"
> + * an update time can go slighty backwards in the single digit
> + * nanoseconds range, if the mult/shift factors are adjusted by the
> + * update. So don't use this for code which might be sensitive about
> + * that. For the intended use case of tracing and instrumentation its
"its" -> "it's"
Thanks,
Mathieu
> + * a non issue.
> + */
> +u64 notrace ktime_get_mono_fast_ns(void)
> +{
> + return ktime_get_fast_ns(&tk_fast_mono);
> +}
> +
> #ifdef CONFIG_GENERIC_TIME_VSYSCALL_OLD
>
> static inline void update_vsyscall(struct timekeeper *tk)
> @@ -324,6 +490,8 @@ static void timekeeping_update(struct ti
> if (action & TK_MIRROR)
> memcpy(&shadow_timekeeper, &tk_core.timekeeper,
> sizeof(tk_core.timekeeper));
> +
> + update_fast_timekeepers(tk);
> }
>
> /**
> @@ -470,27 +638,6 @@ ktime_t ktime_mono_to_any(ktime_t tmono,
> EXPORT_SYMBOL_GPL(ktime_mono_to_any);
>
> /**
> - * ktime_get_raw - Returns the raw monotonic time in ktime_t format
> - */
> -ktime_t ktime_get_raw(void)
> -{
> - struct timekeeper *tk = &tk_core.timekeeper;
> - unsigned int seq;
> - ktime_t base;
> - s64 nsecs;
> -
> - do {
> - seq = read_seqcount_begin(&tk_core.seq);
> - base = tk->base_raw;
> - nsecs = timekeeping_get_ns_raw(tk);
> -
> - } while (read_seqcount_retry(&tk_core.seq, seq));
> -
> - return ktime_add_ns(base, nsecs);
> -}
> -EXPORT_SYMBOL_GPL(ktime_get_raw);
> -
> -/**
> * ktime_get_ts64 - get the monotonic clock in timespec64 format
> * @ts: pointer to timespec variable
> *
> @@ -574,13 +721,19 @@ void getnstime_raw_and_real(struct times
> do {
> seq = read_seqcount_begin(&tk_core.seq);
>
> - *ts_raw = timespec64_to_timespec(tk->raw_time);
> ts_real->tv_sec = tk->xtime_sec;
> ts_real->tv_nsec = 0;
> -
> - nsecs_raw = timekeeping_get_ns_raw(tk);
> nsecs_real = timekeeping_get_ns(tk);
>
> + /*
> + * base[0] of tk_fast_raw is valid here as we are
> + * protected by the tk_core.seq counter. The raw_base
> + * has it's own sequence counter, but that is updated
> + * under tk_core.seq.
> + */
> + *ts_raw = timespec64_to_timespec(tk->raw_time);
> + nsecs_raw = timekeeping_get_ns_raw(tk_fast_raw.base);
> +
> } while (read_seqcount_retry(&tk_core.seq, seq));
>
> timespec_add_ns(ts_raw, nsecs_raw);
> @@ -813,7 +966,7 @@ void getrawmonotonic(struct timespec *ts
>
> do {
> seq = read_seqcount_begin(&tk_core.seq);
> - nsecs = timekeeping_get_ns_raw(tk);
> + nsecs = timekeeping_get_ns_raw(tk_fast_raw.base);
> ts64 = tk->raw_time;
>
> } while (read_seqcount_retry(&tk_core.seq, seq));
> @@ -946,6 +1099,7 @@ void __init timekeeping_init(void)
>
> memcpy(&shadow_timekeeper, &tk_core.timekeeper,
> sizeof(tk_core.timekeeper));
> + update_fast_timekeepers(tk);
>
> write_seqcount_end(&tk_core.seq);
> raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
>
>
>
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
2014-07-11 20:04 ` Mathieu Desnoyers
@ 2014-07-12 8:11 ` Thomas Gleixner
2014-07-12 13:59 ` Mathieu Desnoyers
0 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-12 8:11 UTC (permalink / raw)
To: Mathieu Desnoyers; +Cc: LKML, John Stultz, Peter Zijlstra, Steven Rostedt
On Fri, 11 Jul 2014, Mathieu Desnoyers wrote:
> ----- Original Message -----
> > From: "Thomas Gleixner" <tglx@linutronix.de>
> > To: "LKML" <linux-kernel@vger.kernel.org>
> > Cc: "John Stultz" <john.stultz@linaro.org>, "Peter Zijlstra" <peterz@infradead.org>, "Steven Rostedt"
> > <rostedt@goodmis.org>, "Mathieu Desnoyers" <mathieu.desnoyers@efficios.com>
> > Sent: Friday, July 11, 2014 9:45:19 AM
> > Subject: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
> > On the update side:
> >
> > tkf->seq++;
> > smp_wmb();
> > update(tkf->base[0], tk;
>
> missing ")"
:)
> Any reason why the updater wouldn't do:
>
> tkf->seq++;
> smp_wmb();
> update(tkf->base[1 - (tkf->seq & 0x01)], tk);
>
> instead of updating both array entries each time ?
base[0]; <- Current active
seq++; -> Makes base[1] the active one for readers
update(base[0]);
So readers are always one update cycle behind. Probably not an issue
most of the time, but think about fast wrapping clocksources and
NOHZ....
Thanks,
tglx
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
2014-07-12 8:11 ` Thomas Gleixner
@ 2014-07-12 13:59 ` Mathieu Desnoyers
2014-07-12 19:31 ` Thomas Gleixner
0 siblings, 1 reply; 91+ messages in thread
From: Mathieu Desnoyers @ 2014-07-12 13:59 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Peter Zijlstra, Steven Rostedt
----- Original Message -----
> From: "Thomas Gleixner" <tglx@linutronix.de>
> To: "Mathieu Desnoyers" <mathieu.desnoyers@efficios.com>
> Cc: "LKML" <linux-kernel@vger.kernel.org>, "John Stultz" <john.stultz@linaro.org>, "Peter Zijlstra"
> <peterz@infradead.org>, "Steven Rostedt" <rostedt@goodmis.org>
> Sent: Saturday, July 12, 2014 4:11:17 AM
> Subject: Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
>
> On Fri, 11 Jul 2014, Mathieu Desnoyers wrote:
>
> > ----- Original Message -----
> > > From: "Thomas Gleixner" <tglx@linutronix.de>
> > > To: "LKML" <linux-kernel@vger.kernel.org>
> > > Cc: "John Stultz" <john.stultz@linaro.org>, "Peter Zijlstra"
> > > <peterz@infradead.org>, "Steven Rostedt"
> > > <rostedt@goodmis.org>, "Mathieu Desnoyers"
> > > <mathieu.desnoyers@efficios.com>
> > > Sent: Friday, July 11, 2014 9:45:19 AM
> > > Subject: [patch 54/55] timekeeping: Provide fast and NMI safe access to
> > > CLOCK_MONOTONIC[_RAW]
> > > On the update side:
> > >
> > > tkf->seq++;
> > > smp_wmb();
> > > update(tkf->base[0], tk;
> >
> > missing ")"
>
> :)
>
> > Any reason why the updater wouldn't do:
> >
> > tkf->seq++;
> > smp_wmb();
> > update(tkf->base[1 - (tkf->seq & 0x01)], tk);
> >
> > instead of updating both array entries each time ?
>
> base[0]; <- Current active
> seq++; -> Makes base[1] the active one for readers
> update(base[0]);
>
> So readers are always one update cycle behind. Probably not an issue
> most of the time, but think about fast wrapping clocksources and
> NOHZ....
Ah, yep, got it.
So instead of calling update() twice per update, could we do
the following instead ?
tkf->seq++;
smp_wmb();
update(tkf->base[0], tk);
tkf->seq++;
smp_wmb();
memcpy(tkf->base[1], tkf->base[0], sizeof(tkf->base[1]);
just in case "update" would happen to be expensive for
some clock implementation.
Thanks,
Mathieu
>
> Thanks,
>
> tglx
>
>
>
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
2014-07-12 13:59 ` Mathieu Desnoyers
@ 2014-07-12 19:31 ` Thomas Gleixner
0 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-12 19:31 UTC (permalink / raw)
To: Mathieu Desnoyers; +Cc: LKML, John Stultz, Peter Zijlstra, Steven Rostedt
On Sat, 12 Jul 2014, Mathieu Desnoyers wrote:
> ----- Original Message -----
> > From: "Thomas Gleixner" <tglx@linutronix.de>
> > On Fri, 11 Jul 2014, Mathieu Desnoyers wrote:
> > > Any reason why the updater wouldn't do:
> > >
> > > tkf->seq++;
> > > smp_wmb();
> > > update(tkf->base[1 - (tkf->seq & 0x01)], tk);
> > >
> > > instead of updating both array entries each time ?
> >
> > base[0]; <- Current active
> > seq++; -> Makes base[1] the active one for readers
> > update(base[0]);
> >
> > So readers are always one update cycle behind. Probably not an issue
> > most of the time, but think about fast wrapping clocksources and
> > NOHZ....
>
> Ah, yep, got it.
>
> So instead of calling update() twice per update, could we do
> the following instead ?
>
> tkf->seq++;
> smp_wmb();
> update(tkf->base[0], tk);
> tkf->seq++;
> smp_wmb();
> memcpy(tkf->base[1], tkf->base[0], sizeof(tkf->base[1]);
>
> just in case "update" would happen to be expensive for
> some clock implementation.
Sure, same end result ...
Thanks,
tglx
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
2014-07-11 13:45 ` [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW] Thomas Gleixner
2014-07-11 20:04 ` Mathieu Desnoyers
@ 2014-07-12 14:53 ` Mathieu Desnoyers
2014-07-12 19:28 ` Thomas Gleixner
2014-07-14 8:37 ` Peter Zijlstra
2014-07-14 9:47 ` Peter Zijlstra
3 siblings, 1 reply; 91+ messages in thread
From: Mathieu Desnoyers @ 2014-07-12 14:53 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Peter Zijlstra, Steven Rostedt
----- Original Message -----
> From: "Thomas Gleixner" <tglx@linutronix.de>
> To: "LKML" <linux-kernel@vger.kernel.org>
> Cc: "John Stultz" <john.stultz@linaro.org>, "Peter Zijlstra" <peterz@infradead.org>, "Steven Rostedt"
> <rostedt@goodmis.org>, "Mathieu Desnoyers" <mathieu.desnoyers@efficios.com>
> Sent: Friday, July 11, 2014 9:45:19 AM
> Subject: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
>
> Tracers want a correlated time between the kernel instrumentation and
> user space. We really do not want to export sched_clock() to user
> space, so we need to provide something sensible for this.
>
> Using separate data structures with an non blocking sequence count
> based update mechanism allows us to do that. The data structure
> required for the readout has a sequence counter and two copies of the
> timekeeping data.
>
> On the update side:
>
> tkf->seq++;
> smp_wmb();
> update(tkf->base[0], tk;
> tkf->seq++;
> smp_wmb();
> update(tkf->base[1], tk;
>
> On the reader side:
>
> do {
> seq = tkf->seq;
> smp_rmb();
> idx = seq & 0x01;
> now = now(tkf->base[idx]);
> smp_rmb();
> } while (seq != tkf->seq)
>
> So if NMI hits the update of base[0] it will use base[1] which is
> still consistent. In case of CLOCK_MONOTONIC this can result in
> slightly wrong timestamps (a few nanoseconds) accross an update. Not a
> big issue for the intended use case.
Hi Thomas,
I'm perhaps missing something here, but what happens with the
following scenario ?
Initial conditions:
tkf->seq = 0
tkf->base[0] and tkf->base[1] are initialized.
CPU 0 CPU 1
------------ ----------------
update:
tkf->seq++
smb_wmb()
tkf->seq++ (reordered before update)
reader:
seq = tkf->seq (reads 2)
smp_rmb()
idx = seq & 0x01
now = now(tkf->base[idx] (reads base[0])
update(tkf->base[0], tk) (racy concurrent update)
smp_rmb()
while (seq != tkf->seq) (they are equal)
So AFAIU, we end up returning a corrupted value. Adding a
smp_wmb() between update of base[0] and increment of seq,
as well as between update of base[1] and the _following_
increment of seq (next update call) would fix this.
Thoughts ?
Thanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
2014-07-12 14:53 ` Mathieu Desnoyers
@ 2014-07-12 19:28 ` Thomas Gleixner
2014-07-12 20:04 ` Thomas Gleixner
0 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-12 19:28 UTC (permalink / raw)
To: Mathieu Desnoyers; +Cc: LKML, John Stultz, Peter Zijlstra, Steven Rostedt
On Sat, 12 Jul 2014, Mathieu Desnoyers wrote:
> I'm perhaps missing something here, but what happens with the
> following scenario ?
>
> Initial conditions:
>
> tkf->seq = 0
> tkf->base[0] and tkf->base[1] are initialized.
>
> CPU 0 CPU 1
> ------------ ----------------
> update:
> tkf->seq++
> smb_wmb()
> tkf->seq++ (reordered before update)
> reader:
> seq = tkf->seq (reads 2)
> smp_rmb()
> idx = seq & 0x01
> now = now(tkf->base[idx] (reads base[0])
> update(tkf->base[0], tk) (racy concurrent update)
> smp_rmb()
> while (seq != tkf->seq) (they are equal)
>
> So AFAIU, we end up returning a corrupted value. Adding a
> smp_wmb() between update of base[0] and increment of seq,
> as well as between update of base[1] and the _following_
> increment of seq (next update call) would fix this.
>
> Thoughts ?
Well, the actual implementation does:
+ /* Force readers off to base[1] */
+ raw_write_seqcount_begin(&tkf->seq);
+
+ /* Update base[0] */
+ base->clock = clk;
+ base->cycle_last = clk->cycle_last;
+ base->base = tbase;
+ base->shift = shift;
+ base->mult = mult;
+
+ /* Force readers back to base[0] */
+ raw_write_seqcount_end(&tkf->seq);
+
+ /* Update base[1] */
+ base++;
+ base->clock = clk;
+ base->cycle_last = clk->cycle_last;
+ base->base = tbase;
+ base->shift = shift;
+ base->mult = mult;
Where raw_write_seqcount_begin/raw_write_seqcount_end provides the
required memory barriers.
Sure I should update the documentaion ....
Thanks,
tglx
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
2014-07-12 19:28 ` Thomas Gleixner
@ 2014-07-12 20:04 ` Thomas Gleixner
2014-07-12 20:33 ` Mathieu Desnoyers
0 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-12 20:04 UTC (permalink / raw)
To: Mathieu Desnoyers; +Cc: LKML, John Stultz, Peter Zijlstra, Steven Rostedt
On Sat, 12 Jul 2014, Thomas Gleixner wrote:
> On Sat, 12 Jul 2014, Mathieu Desnoyers wrote:
> > I'm perhaps missing something here, but what happens with the
> > following scenario ?
> >
> > Initial conditions:
> >
> > tkf->seq = 0
> > tkf->base[0] and tkf->base[1] are initialized.
> >
> > CPU 0 CPU 1
> > ------------ ----------------
> > update:
> > tkf->seq++
> > smb_wmb()
> > tkf->seq++ (reordered before update)
> > reader:
> > seq = tkf->seq (reads 2)
> > smp_rmb()
> > idx = seq & 0x01
> > now = now(tkf->base[idx] (reads base[0])
> > update(tkf->base[0], tk) (racy concurrent update)
> > smp_rmb()
> > while (seq != tkf->seq) (they are equal)
> >
> > So AFAIU, we end up returning a corrupted value. Adding a
> > smp_wmb() between update of base[0] and increment of seq,
> > as well as between update of base[1] and the _following_
> > increment of seq (next update call) would fix this.
> >
> > Thoughts ?
Second thoughts :)
> Well, the actual implementation does:
>
> + /* Force readers off to base[1] */
> + raw_write_seqcount_begin(&tkf->seq);
i.e:
seq++;
smp_wmb();
> +
> + /* Update base[0] */
> + base->clock = clk;
> + base->cycle_last = clk->cycle_last;
> + base->base = tbase;
> + base->shift = shift;
> + base->mult = mult;
> +
> + /* Force readers back to base[0] */
> + raw_write_seqcount_end(&tkf->seq);
i.e:
smp_wmb();
seq++;
So while this orders against the update of base0, it does not prevent
reordering against the update of base1. So you're right, we need a
smp_wmb();
before we start updating base1.
> + /* Update base[1] */
> + base++;
> + base->clock = clk;
> + base->cycle_last = clk->cycle_last;
> + base->base = tbase;
> + base->shift = shift;
> + base->mult = mult;
So as a consequence we need another one here:
smp_wmb();
to protect against the unlikely, but possible seq++ at the begin of
the update. Debatable whether this can happen without another wmb()
between the two calls, but yes for sanity reasons we should add it
until we can prove that the actual call chains prevent this.
Nice catch!
tglx
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
2014-07-12 20:04 ` Thomas Gleixner
@ 2014-07-12 20:33 ` Mathieu Desnoyers
0 siblings, 0 replies; 91+ messages in thread
From: Mathieu Desnoyers @ 2014-07-12 20:33 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Peter Zijlstra, Steven Rostedt
----- Original Message -----
> From: "Thomas Gleixner" <tglx@linutronix.de>
> To: "Mathieu Desnoyers" <mathieu.desnoyers@efficios.com>
> Cc: "LKML" <linux-kernel@vger.kernel.org>, "John Stultz" <john.stultz@linaro.org>, "Peter Zijlstra"
> <peterz@infradead.org>, "Steven Rostedt" <rostedt@goodmis.org>
> Sent: Saturday, July 12, 2014 4:04:59 PM
> Subject: Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
>
> On Sat, 12 Jul 2014, Thomas Gleixner wrote:
> > On Sat, 12 Jul 2014, Mathieu Desnoyers wrote:
> > > I'm perhaps missing something here, but what happens with the
> > > following scenario ?
> > >
> > > Initial conditions:
> > >
> > > tkf->seq = 0
> > > tkf->base[0] and tkf->base[1] are initialized.
> > >
> > > CPU 0 CPU 1
> > > ------------ ----------------
> > > update:
> > > tkf->seq++
> > > smb_wmb()
> > > tkf->seq++ (reordered before update)
> > > reader:
> > > seq = tkf->seq (reads 2)
> > > smp_rmb()
> > > idx = seq & 0x01
> > > now = now(tkf->base[idx]
> > > (reads base[0])
> > > update(tkf->base[0], tk) (racy concurrent update)
> > > smp_rmb()
> > > while (seq != tkf->seq) (they
> > > are equal)
> > >
> > > So AFAIU, we end up returning a corrupted value. Adding a
> > > smp_wmb() between update of base[0] and increment of seq,
> > > as well as between update of base[1] and the _following_
> > > increment of seq (next update call) would fix this.
> > >
> > > Thoughts ?
>
> Second thoughts :)
>
> > Well, the actual implementation does:
> >
> > + /* Force readers off to base[1] */
> > + raw_write_seqcount_begin(&tkf->seq);
>
> i.e:
> seq++;
> smp_wmb();
>
> > +
> > + /* Update base[0] */
> > + base->clock = clk;
> > + base->cycle_last = clk->cycle_last;
> > + base->base = tbase;
> > + base->shift = shift;
> > + base->mult = mult;
> > +
> > + /* Force readers back to base[0] */
> > + raw_write_seqcount_end(&tkf->seq);
>
> i.e:
> smp_wmb();
> seq++;
>
> So while this orders against the update of base0, it does not prevent
> reordering against the update of base1. So you're right, we need a
>
> smp_wmb();
>
> before we start updating base1.
>
> > + /* Update base[1] */
> > + base++;
> > + base->clock = clk;
> > + base->cycle_last = clk->cycle_last;
> > + base->base = tbase;
> > + base->shift = shift;
> > + base->mult = mult;
>
> So as a consequence we need another one here:
>
> smp_wmb();
>
> to protect against the unlikely, but possible seq++ at the begin of
> the update. Debatable whether this can happen without another wmb()
> between the two calls, but yes for sanity reasons we should add it
> until we can prove that the actual call chains prevent this.
>
> Nice catch!
Thanks! Yep, the barriers you propose are what appears
to be missing,
Mathieu
>
> tglx
>
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
2014-07-11 13:45 ` [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW] Thomas Gleixner
2014-07-11 20:04 ` Mathieu Desnoyers
2014-07-12 14:53 ` Mathieu Desnoyers
@ 2014-07-14 8:37 ` Peter Zijlstra
2014-07-14 9:04 ` Thomas Gleixner
2014-07-14 9:47 ` Peter Zijlstra
3 siblings, 1 reply; 91+ messages in thread
From: Peter Zijlstra @ 2014-07-14 8:37 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Steven Rostedt, Mathieu Desnoyers
[-- Attachment #1: Type: text/plain, Size: 1331 bytes --]
On Fri, Jul 11, 2014 at 01:45:19PM -0000, Thomas Gleixner wrote:
> Tracers want a correlated time between the kernel instrumentation and
> user space. We really do not want to export sched_clock() to user
> space, so we need to provide something sensible for this.
>
> Using separate data structures with an non blocking sequence count
> based update mechanism allows us to do that. The data structure
> required for the readout has a sequence counter and two copies of the
> timekeeping data.
>
> On the update side:
>
> tkf->seq++;
> smp_wmb();
> update(tkf->base[0], tk;
> tkf->seq++;
> smp_wmb();
> update(tkf->base[1], tk;
>
> On the reader side:
>
> do {
> seq = tkf->seq;
> smp_rmb();
> idx = seq & 0x01;
> now = now(tkf->base[idx]);
> smp_rmb();
> } while (seq != tkf->seq)
>
> So if NMI hits the update of base[0] it will use base[1] which is
> still consistent. In case of CLOCK_MONOTONIC this can result in
> slightly wrong timestamps (a few nanoseconds) accross an update. Not a
> big issue for the intended use case.
But it breaks monotonicity, right? ;-)
Also, what happens when TSC is not available as a clocksource? There's
still a metric ton of hardware (including the latest generation HSW)
that has fucked firmware/TSC.
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
2014-07-14 8:37 ` Peter Zijlstra
@ 2014-07-14 9:04 ` Thomas Gleixner
2014-07-14 9:45 ` Peter Zijlstra
0 siblings, 1 reply; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-14 9:04 UTC (permalink / raw)
To: Peter Zijlstra; +Cc: LKML, John Stultz, Steven Rostedt, Mathieu Desnoyers
On Mon, 14 Jul 2014, Peter Zijlstra wrote:
> On Fri, Jul 11, 2014 at 01:45:19PM -0000, Thomas Gleixner wrote:
> > Tracers want a correlated time between the kernel instrumentation and
> > user space. We really do not want to export sched_clock() to user
> > space, so we need to provide something sensible for this.
> >
> > Using separate data structures with an non blocking sequence count
> > based update mechanism allows us to do that. The data structure
> > required for the readout has a sequence counter and two copies of the
> > timekeeping data.
> >
> > On the update side:
> >
> > tkf->seq++;
> > smp_wmb();
> > update(tkf->base[0], tk;
> > tkf->seq++;
> > smp_wmb();
> > update(tkf->base[1], tk;
> >
> > On the reader side:
> >
> > do {
> > seq = tkf->seq;
> > smp_rmb();
> > idx = seq & 0x01;
> > now = now(tkf->base[idx]);
> > smp_rmb();
> > } while (seq != tkf->seq)
> >
> > So if NMI hits the update of base[0] it will use base[1] which is
> > still consistent. In case of CLOCK_MONOTONIC this can result in
> > slightly wrong timestamps (a few nanoseconds) accross an update. Not a
> > big issue for the intended use case.
>
> But it breaks monotonicity, right? ;-)
It can in theory, but does it really matter for tracing?
> Also, what happens when TSC is not available as a clocksource? There's
> still a metric ton of hardware (including the latest generation HSW)
> that has fucked firmware/TSC.
Well, bad luck then. You end up using hpet or worse, but it's still
your decision whether to base your instrumentation on that or not. For
sane clock sources (i.e. almost anything except TSC) it works
perfectly fine.
Thanks,
tglx
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
2014-07-14 9:04 ` Thomas Gleixner
@ 2014-07-14 9:45 ` Peter Zijlstra
0 siblings, 0 replies; 91+ messages in thread
From: Peter Zijlstra @ 2014-07-14 9:45 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Steven Rostedt, Mathieu Desnoyers
[-- Attachment #1: Type: text/plain, Size: 2463 bytes --]
On Mon, Jul 14, 2014 at 11:04:39AM +0200, Thomas Gleixner wrote:
> On Mon, 14 Jul 2014, Peter Zijlstra wrote:
> > On Fri, Jul 11, 2014 at 01:45:19PM -0000, Thomas Gleixner wrote:
> > > Tracers want a correlated time between the kernel instrumentation and
> > > user space. We really do not want to export sched_clock() to user
> > > space, so we need to provide something sensible for this.
> > >
> > > Using separate data structures with an non blocking sequence count
> > > based update mechanism allows us to do that. The data structure
> > > required for the readout has a sequence counter and two copies of the
> > > timekeeping data.
> > >
> > > On the update side:
> > >
> > > tkf->seq++;
> > > smp_wmb();
> > > update(tkf->base[0], tk;
> > > tkf->seq++;
> > > smp_wmb();
> > > update(tkf->base[1], tk;
> > >
> > > On the reader side:
> > >
> > > do {
> > > seq = tkf->seq;
> > > smp_rmb();
> > > idx = seq & 0x01;
> > > now = now(tkf->base[idx]);
> > > smp_rmb();
> > > } while (seq != tkf->seq)
> > >
> > > So if NMI hits the update of base[0] it will use base[1] which is
> > > still consistent. In case of CLOCK_MONOTONIC this can result in
> > > slightly wrong timestamps (a few nanoseconds) accross an update. Not a
> > > big issue for the intended use case.
> >
> > But it breaks monotonicity, right? ;-)
>
> It can in theory, but does it really matter for tracing?
Not so much; but I think the interfaces aren't quite as explicit as they
maybe should be about this.
> > Also, what happens when TSC is not available as a clocksource? There's
> > still a metric ton of hardware (including the latest generation HSW)
> > that has fucked firmware/TSC.
>
> Well, bad luck then. You end up using hpet or worse, but it's still
> your decision whether to base your instrumentation on that or not. For
> sane clock sources (i.e. almost anything except TSC) it works
> perfectly fine.
So I can imagine clock sources themselves not being NMI safe on a
hardware level. But also, I'm fairly sure you do not want HPET reads in
a function tracer :-)
And I absolutely detest this configurability muck. I want a single
'clock'.
But yes, we can do what you suggested, we can do something like:
if (tsc-stable)
return nmi-mono
else
return sched_clock
Seeing how with tsc-unstable the sched_clock muck is synched to clock
monotonic.
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
2014-07-11 13:45 ` [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW] Thomas Gleixner
` (2 preceding siblings ...)
2014-07-14 8:37 ` Peter Zijlstra
@ 2014-07-14 9:47 ` Peter Zijlstra
2014-07-14 12:39 ` Mathieu Desnoyers
3 siblings, 1 reply; 91+ messages in thread
From: Peter Zijlstra @ 2014-07-14 9:47 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Steven Rostedt, Mathieu Desnoyers
[-- Attachment #1: Type: text/plain, Size: 304 bytes --]
On Fri, Jul 11, 2014 at 01:45:19PM -0000, Thomas Gleixner wrote:
> +static struct tk_fast tk_fast_raw ____cacheline_aligned;
> +static struct tk_fast tk_fast_mono ____cacheline_aligned;
Question to the 'trace maniacs'; why do you lot need access to mono_raw?
I would much prefer to limit choice here.
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 91+ messages in thread* Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
2014-07-14 9:47 ` Peter Zijlstra
@ 2014-07-14 12:39 ` Mathieu Desnoyers
2014-07-14 14:15 ` Thomas Gleixner
0 siblings, 1 reply; 91+ messages in thread
From: Mathieu Desnoyers @ 2014-07-14 12:39 UTC (permalink / raw)
To: Peter Zijlstra; +Cc: Thomas Gleixner, LKML, John Stultz, Steven Rostedt
----- Original Message -----
> From: "Peter Zijlstra" <peterz@infradead.org>
> To: "Thomas Gleixner" <tglx@linutronix.de>
> Cc: "LKML" <linux-kernel@vger.kernel.org>, "John Stultz" <john.stultz@linaro.org>, "Steven Rostedt"
> <rostedt@goodmis.org>, "Mathieu Desnoyers" <mathieu.desnoyers@efficios.com>
> Sent: Monday, July 14, 2014 5:47:50 AM
> Subject: Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
>
> On Fri, Jul 11, 2014 at 01:45:19PM -0000, Thomas Gleixner wrote:
> > +static struct tk_fast tk_fast_raw ____cacheline_aligned;
> > +static struct tk_fast tk_fast_mono ____cacheline_aligned;
>
> Question to the 'trace maniacs'; why do you lot need access to mono_raw?
>
> I would much prefer to limit choice here.
For LTTng, which I guess qualifies as bring part of the aforementioned
trace maniacs group, we only need the "tk_fast_mono" clock. We don't
need mono_raw.
We can then correlate with user-space traces by using tk_fast_mono in
kernel-space and using CLOCK_MONOTONIC in userspace.
Thanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
2014-07-14 12:39 ` Mathieu Desnoyers
@ 2014-07-14 14:15 ` Thomas Gleixner
0 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-14 14:15 UTC (permalink / raw)
To: Mathieu Desnoyers; +Cc: Peter Zijlstra, LKML, John Stultz, Steven Rostedt
On Mon, 14 Jul 2014, Mathieu Desnoyers wrote:
> ----- Original Message -----
> > From: "Peter Zijlstra" <peterz@infradead.org>
> > To: "Thomas Gleixner" <tglx@linutronix.de>
> > Cc: "LKML" <linux-kernel@vger.kernel.org>, "John Stultz" <john.stultz@linaro.org>, "Steven Rostedt"
> > <rostedt@goodmis.org>, "Mathieu Desnoyers" <mathieu.desnoyers@efficios.com>
> > Sent: Monday, July 14, 2014 5:47:50 AM
> > Subject: Re: [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW]
> >
> > On Fri, Jul 11, 2014 at 01:45:19PM -0000, Thomas Gleixner wrote:
> > > +static struct tk_fast tk_fast_raw ____cacheline_aligned;
> > > +static struct tk_fast tk_fast_mono ____cacheline_aligned;
> >
> > Question to the 'trace maniacs'; why do you lot need access to mono_raw?
> >
> > I would much prefer to limit choice here.
>
> For LTTng, which I guess qualifies as bring part of the aforementioned
> trace maniacs group, we only need the "tk_fast_mono" clock. We don't
> need mono_raw.
>
> We can then correlate with user-space traces by using tk_fast_mono in
> kernel-space and using CLOCK_MONOTONIC in userspace.
I'm happy to ditch the raw one ...
^ permalink raw reply [flat|nested] 91+ messages in thread
* [patch 55/55] ftrace: Provide trace clocks mono and mono_raw
2014-07-11 13:43 [patch 00/55] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (53 preceding siblings ...)
2014-07-11 13:45 ` [patch 54/55] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC[_RAW] Thomas Gleixner
@ 2014-07-11 13:45 ` Thomas Gleixner
54 siblings, 0 replies; 91+ messages in thread
From: Thomas Gleixner @ 2014-07-11 13:45 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Steven Rostedt, Mathieu Desnoyers
[-- Attachment #1: ftrace-use-clock-mono-raw.patch --]
[-- Type: text/plain, Size: 1159 bytes --]
Expose the new NMI safe accessors to clock monotonic and monotonic raw
to the tracer.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
kernel/trace/trace.c | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
Index: tip/kernel/trace/trace.c
===================================================================
--- tip.orig/kernel/trace/trace.c
+++ tip/kernel/trace/trace.c
@@ -806,11 +806,13 @@ static struct {
const char *name;
int in_ns; /* is this clock in nanoseconds? */
} trace_clocks[] = {
- { trace_clock_local, "local", 1 },
- { trace_clock_global, "global", 1 },
- { trace_clock_counter, "counter", 0 },
- { trace_clock_jiffies, "uptime", 1 },
- { trace_clock, "perf", 1 },
+ { trace_clock_local, "local", 1 },
+ { trace_clock_global, "global", 1 },
+ { trace_clock_counter, "counter", 0 },
+ { trace_clock_jiffies, "uptime", 1 },
+ { trace_clock, "perf", 1 },
+ { ktime_get_mono_fast_ns, "mono", 1 },
+ { ktime_get_raw_ns, "mono_raw", 1 },
ARCH_TRACE_CLOCKS
};
^ permalink raw reply [flat|nested] 91+ messages in thread