diff options
| author | Paul E. McKenney <paulmck@kernel.org> | 2024-05-01 16:12:37 -0700 |
|---|---|---|
| committer | Steven Rostedt (Google) <rostedt@goodmis.org> | 2024-05-13 19:36:40 -0400 |
| commit | 33f137143e651321f10eb67ae6404a13bfbf69f8 (patch) | |
| tree | 19bacae637427dfb39f9613428882f8aa554fe6a /kernel | |
| parent | c5963a0990d1da70d1bd399e6811887ff2231b0d (diff) | |
| download | linux-stable-33f137143e651321f10eb67ae6404a13bfbf69f8.tar.gz linux-stable-33f137143e651321f10eb67ae6404a13bfbf69f8.tar.bz2 linux-stable-33f137143e651321f10eb67ae6404a13bfbf69f8.zip | |
ftrace: Use asynchronous grace period for register_ftrace_direct()
When running heavy test workloads with KASAN enabled, RCU Tasks grace
periods can extend for many tens of seconds, significantly slowing
trace registration. Therefore, make the registration-side RCU Tasks
grace period be asynchronous via call_rcu_tasks().
Link: https://lore.kernel.org/linux-trace-kernel/ac05be77-2972-475b-9b57-56bef15aa00a@paulmck-laptop
Reported-by: Jakub Kicinski <kuba@kernel.org>
Reported-by: Alexei Starovoitov <ast@kernel.org>
Reported-by: Chris Mason <clm@fb.com>
Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Diffstat (limited to 'kernel')
| -rw-r--r-- | kernel/trace/ftrace.c | 13 |
1 files changed, 9 insertions, 4 deletions
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 3951c0d0ec63..f9223513414a 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -5366,6 +5366,13 @@ static void remove_direct_functions_hash(struct ftrace_hash *hash, unsigned long } } +static void register_ftrace_direct_cb(struct rcu_head *rhp) +{ + struct ftrace_hash *fhp = container_of(rhp, struct ftrace_hash, rcu); + + free_ftrace_hash(fhp); +} + /** * register_ftrace_direct - Call a custom trampoline directly * for multiple functions registered in @ops @@ -5464,10 +5471,8 @@ int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) out_unlock: mutex_unlock(&direct_mutex); - if (free_hash && free_hash != EMPTY_HASH) { - synchronize_rcu_tasks(); - free_ftrace_hash(free_hash); - } + if (free_hash && free_hash != EMPTY_HASH) + call_rcu_tasks(&free_hash->rcu, register_ftrace_direct_cb); if (new_hash) free_ftrace_hash(new_hash); |