diff options
author | Andy Lutomirski <luto@amacapital.net> | 2014-12-21 08:57:46 -0800 |
---|---|---|
committer | Andy Lutomirski <luto@amacapital.net> | 2014-12-23 13:05:30 -0800 |
commit | 1ddf0b1b11aa8a90cef6706e935fc31c75c406ba (patch) | |
tree | 78e79b96126a6f54c5f4afa37d4f11f07aaab6d8 /arch/x86 | |
parent | 394f56fe480140877304d342dec46d50dc823d46 (diff) | |
download | linux-1ddf0b1b11aa8a90cef6706e935fc31c75c406ba.tar.gz linux-1ddf0b1b11aa8a90cef6706e935fc31c75c406ba.tar.bz2 linux-1ddf0b1b11aa8a90cef6706e935fc31c75c406ba.zip |
x86, vdso: Use asm volatile in __getcpu
In Linux 3.18 and below, GCC hoists the lsl instructions in the
pvclock code all the way to the beginning of __vdso_clock_gettime,
slowing the non-paravirt case significantly. For unknown reasons,
presumably related to the removal of a branch, the performance issue
is gone as of
e76b027e6408 x86,vdso: Use LSL unconditionally for vgetcpu
but I don't trust GCC enough to expect the problem to stay fixed.
There should be no correctness issue, because the __getcpu calls in
__vdso_vlock_gettime were never necessary in the first place.
Note to stable maintainers: In 3.18 and below, depending on
configuration, gcc 4.9.2 generates code like this:
9c3: 44 0f 03 e8 lsl %ax,%r13d
9c7: 45 89 eb mov %r13d,%r11d
9ca: 0f 03 d8 lsl %ax,%ebx
This patch won't apply as is to any released kernel, but I'll send a
trivial backported version if needed.
Fixes: 51c19b4f5927 x86: vdso: pvclock gettime support
Cc: stable@vger.kernel.org # 3.8+
Cc: Marcelo Tosatti <mtosatti@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andy Lutomirski <luto@amacapital.net>
Diffstat (limited to 'arch/x86')
-rw-r--r-- | arch/x86/include/asm/vgtod.h | 6 |
1 files changed, 4 insertions, 2 deletions
diff --git a/arch/x86/include/asm/vgtod.h b/arch/x86/include/asm/vgtod.h index e7e9682a33e9..f556c4843aa1 100644 --- a/arch/x86/include/asm/vgtod.h +++ b/arch/x86/include/asm/vgtod.h @@ -80,9 +80,11 @@ static inline unsigned int __getcpu(void) /* * Load per CPU data from GDT. LSL is faster than RDTSCP and - * works on all CPUs. + * works on all CPUs. This is volatile so that it orders + * correctly wrt barrier() and to keep gcc from cleverly + * hoisting it out of the calling function. */ - asm("lsl %1,%0" : "=r" (p) : "r" (__PER_CPU_SEG)); + asm volatile ("lsl %1,%0" : "=r" (p) : "r" (__PER_CPU_SEG)); return p; } |