diff options
author | Paolo Bonzini <pbonzini@redhat.com> | 2017-08-01 17:24:04 +0200 |
---|---|---|
committer | Ingo Molnar <mingo@kernel.org> | 2017-08-10 12:28:56 +0200 |
commit | 1dbb6704de91b169a58d0c8221624afd6a95cfc7 (patch) | |
tree | 46273515a9b79413d2f00629b7e253189e903219 /include/linux/jump_label.h | |
parent | 83ced169d9a01f22eb39f1fcc1f89ad9d223238f (diff) | |
download | linux-1dbb6704de91b169a58d0c8221624afd6a95cfc7.tar.gz linux-1dbb6704de91b169a58d0c8221624afd6a95cfc7.tar.bz2 linux-1dbb6704de91b169a58d0c8221624afd6a95cfc7.zip |
jump_label: Fix concurrent static_key_enable/disable()
static_key_enable/disable are trying to cap the static key count to
0/1. However, their use of key->enabled is outside jump_label_lock
so they do not really ensure that.
Rewrite them to do a quick check for an already enabled (respectively,
already disabled), and then recheck under the jump label lock. Unlike
static_key_slow_inc/dec, a failed check under the jump label lock does
not modify key->enabled.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Jason Baron <jbaron@akamai.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1501601046-35683-2-git-send-email-pbonzini@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Diffstat (limited to 'include/linux/jump_label.h')
-rw-r--r-- | include/linux/jump_label.h | 22 |
1 files changed, 12 insertions, 10 deletions
diff --git a/include/linux/jump_label.h b/include/linux/jump_label.h index 2afd74b9d844..740a42ea7f7f 100644 --- a/include/linux/jump_label.h +++ b/include/linux/jump_label.h @@ -234,22 +234,24 @@ static inline int jump_label_apply_nops(struct module *mod) static inline void static_key_enable(struct static_key *key) { - int count = static_key_count(key); - - WARN_ON_ONCE(count < 0 || count > 1); + STATIC_KEY_CHECK_USE(); - if (!count) - static_key_slow_inc(key); + if (atomic_read(&key->enabled) != 0) { + WARN_ON_ONCE(atomic_read(&key->enabled) != 1); + return; + } + atomic_set(&key->enabled, 1); } static inline void static_key_disable(struct static_key *key) { - int count = static_key_count(key); - - WARN_ON_ONCE(count < 0 || count > 1); + STATIC_KEY_CHECK_USE(); - if (count) - static_key_slow_dec(key); + if (atomic_read(&key->enabled) != 1) { + WARN_ON_ONCE(atomic_read(&key->enabled) != 0); + return; + } + atomic_set(&key->enabled, 0); } #define STATIC_KEY_INIT_TRUE { .enabled = ATOMIC_INIT(1) } |