diff options
| author | Adrian Hunter <adrian.hunter@intel.com> | 2024-03-25 08:40:22 +0200 |
|---|---|---|
| committer | Thomas Gleixner <tglx@linutronix.de> | 2024-04-08 15:03:08 +0200 |
| commit | 135225a363ae67bc90bde7a2cbbe1ea0f152ba22 (patch) | |
| tree | f807bda1fb68e96c9537602e6bf66fa6478a6d40 /kernel/time | |
| parent | fcf190c369149c3b04539797cedf28741eb14164 (diff) | |
| download | linux-135225a363ae67bc90bde7a2cbbe1ea0f152ba22.tar.gz linux-135225a363ae67bc90bde7a2cbbe1ea0f152ba22.tar.bz2 linux-135225a363ae67bc90bde7a2cbbe1ea0f152ba22.zip | |
timekeeping: Let timekeeping_cycles_to_ns() handle both under and overflow
For the case !CONFIG_CLOCKSOURCE_VALIDATE_LAST_CYCLE, forego overflow
protection in the range (mask << 1) < delta <= mask, and interpret it
always as an inconsistency between CPU clock values. That allows
slightly neater code, and it is on a slow path so has no effect on
performance.
Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/r/20240325064023.2997-19-adrian.hunter@intel.com
Diffstat (limited to 'kernel/time')
| -rw-r--r-- | kernel/time/timekeeping.c | 31 |
1 files changed, 13 insertions, 18 deletions
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 111dfdbd488f..4e18db1819f8 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -266,17 +266,14 @@ static inline u64 timekeeping_debug_get_ns(const struct tk_read_base *tkr) * Try to catch underflows by checking if we are seeing small * mask-relative negative values. */ - if (unlikely((~delta & mask) < (mask >> 3))) { + if (unlikely((~delta & mask) < (mask >> 3))) tk->underflow_seen = 1; - now = last; - } - /* Cap delta value to the max_cycles values to avoid mult overflows */ - if (unlikely(delta > max)) { + /* Check for multiplication overflows */ + if (unlikely(delta > max)) tk->overflow_seen = 1; - now = last + max; - } + /* timekeeping_cycles_to_ns() handles both under and overflow */ return timekeeping_cycles_to_ns(tkr, now); } #else @@ -375,19 +372,17 @@ static inline u64 timekeeping_cycles_to_ns(const struct tk_read_base *tkr, u64 c u64 mask = tkr->mask, delta = (cycles - tkr->cycle_last) & mask; /* - * This detects the case where the delta overflows the multiplication - * with tkr->mult. + * This detects both negative motion and the case where the delta + * overflows the multiplication with tkr->mult. */ if (unlikely(delta > tkr->clock->max_cycles)) { - if (IS_ENABLED(CONFIG_CLOCKSOURCE_VALIDATE_LAST_CYCLE)) { - /* - * Handle clocksource inconsistency between CPUs to prevent - * time from going backwards by checking for the MSB of the - * mask being set in the delta. - */ - if (unlikely(delta & ~(mask >> 1))) - return tkr->xtime_nsec >> tkr->shift; - } + /* + * Handle clocksource inconsistency between CPUs to prevent + * time from going backwards by checking for the MSB of the + * mask being set in the delta. + */ + if (delta & ~(mask >> 1)) + return tkr->xtime_nsec >> tkr->shift; return delta_to_ns_safe(tkr, delta); } |