summaryrefslogtreecommitdiffstats
path: root/include
diff options
context:
space:
mode:
authorPaul E. McKenney <paulmck@linux.vnet.ibm.com>2010-04-01 17:37:01 -0700
committerPaul E. McKenney <paulmck@linux.vnet.ibm.com>2010-05-10 11:08:33 -0700
commit25502a6c13745f4650cc59322bd198194f55e796 (patch)
treed76cc659d3ea797c5da4630e219ac363d17c44a6 /include
parent99652b54de1ee094236f7171485214071af4ef31 (diff)
downloadlinux-stable-25502a6c13745f4650cc59322bd198194f55e796.tar.gz
linux-stable-25502a6c13745f4650cc59322bd198194f55e796.tar.bz2
linux-stable-25502a6c13745f4650cc59322bd198194f55e796.zip
rcu: refactor RCU's context-switch handling
The addition of preemptible RCU to treercu resulted in a bit of confusion and inefficiency surrounding the handling of context switches for RCU-sched and for RCU-preempt. For RCU-sched, a context switch is a quiescent state, pure and simple, just like it always has been. For RCU-preempt, a context switch is in no way a quiescent state, but special handling is required when a task blocks in an RCU read-side critical section. However, the callout from the scheduler and the outer loop in ksoftirqd still calls something named rcu_sched_qs(), whose name is no longer accurate. Furthermore, when rcu_check_callbacks() notes an RCU-sched quiescent state, it ends up unnecessarily (though harmlessly, aside from the performance hit) enqueuing the current task if it happens to be running in an RCU-preempt read-side critical section. This not only increases the maximum latency of scheduler_tick(), it also needlessly increases the overhead of the next outermost rcu_read_unlock() invocation. This patch addresses this situation by separating the notion of RCU's context-switch handling from that of RCU-sched's quiescent states. The context-switch handling is covered by rcu_note_context_switch() in general and by rcu_preempt_note_context_switch() for preemptible RCU. This permits rcu_sched_qs() to handle quiescent states and only quiescent states. It also reduces the maximum latency of scheduler_tick(), though probably by much less than a microsecond. Finally, it means that tasks within preemptible-RCU read-side critical sections avoid incurring the overhead of queuing unless there really is a context switch. Suggested-by: Lai Jiangshan <laijs@cn.fujitsu.com> Acked-by: Lai Jiangshan <laijs@cn.fujitsu.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: Peter Zijlstra <peterz@infradead.org>
Diffstat (limited to 'include')
-rw-r--r--include/linux/rcutiny.h4
-rw-r--r--include/linux/rcutree.h1
2 files changed, 5 insertions, 0 deletions
diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h
index bbeb55b7709b..ff22b97fb979 100644
--- a/include/linux/rcutiny.h
+++ b/include/linux/rcutiny.h
@@ -29,6 +29,10 @@
void rcu_sched_qs(int cpu);
void rcu_bh_qs(int cpu);
+static inline void rcu_note_context_switch(int cpu)
+{
+ rcu_sched_qs(cpu);
+}
#define __rcu_read_lock() preempt_disable()
#define __rcu_read_unlock() preempt_enable()
diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h
index 7484fe66a3aa..b9f74606f320 100644
--- a/include/linux/rcutree.h
+++ b/include/linux/rcutree.h
@@ -34,6 +34,7 @@ struct notifier_block;
extern void rcu_sched_qs(int cpu);
extern void rcu_bh_qs(int cpu);
+extern void rcu_note_context_switch(int cpu);
extern int rcu_needs_cpu(int cpu);
extern int rcu_expedited_torture_stats(char *page);