diff options
| author | Waiman Long <longman@redhat.com> | 2023-11-30 15:43:27 -0500 |
|---|---|---|
| committer | Tejun Heo <tj@kernel.org> | 2023-12-01 07:40:20 -1000 |
| commit | 77070eeb882124614a40616f01bfe60947be5778 (patch) | |
| tree | 07903b21226638c02f407f8dc7378221b234213c /include/linux/cgroup-defs.h | |
| parent | d499fd418fa15949d86d28bb5442ab88203fc513 (diff) | |
| download | linux-77070eeb882124614a40616f01bfe60947be5778.tar.gz linux-77070eeb882124614a40616f01bfe60947be5778.tar.bz2 linux-77070eeb882124614a40616f01bfe60947be5778.zip | |
cgroup: Avoid false cacheline sharing of read mostly rstat_cpu
The rstat_cpu and also rstat_css_list of the cgroup structure are read
mostly variables. However, they may share the same cacheline as the
subsequent rstat_flush_next and *bstat variables which can be updated
frequently. That will slow down the cgroup_rstat_cpu() call which is
called pretty frequently in the rstat code. Add a CACHELINE_PADDING()
line in between them to avoid false cacheline sharing.
A parallel kernel build on a 2-socket x86-64 server is used as the
benchmarking tool for measuring the lock hold time. Below were the lock
hold time frequency distribution before and after the patch:
Run time Before patch After patch
-------- ------------ -----------
0-01 us 9,928,562 9,820,428
01-05 us 110,151 50,935
05-10 us 270 93
10-15 us 273 146
15-20 us 135 76
20-25 us 0 2
25-30 us 1 0
It can be seen that the patch further pushes the lock hold time towards
the lower end.
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Diffstat (limited to 'include/linux/cgroup-defs.h')
| -rw-r--r-- | include/linux/cgroup-defs.h | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h index 37518436cfe7..5a97ea95b564 100644 --- a/include/linux/cgroup-defs.h +++ b/include/linux/cgroup-defs.h @@ -497,6 +497,13 @@ struct cgroup { struct list_head rstat_css_list; /* + * Add padding to separate the read mostly rstat_cpu and + * rstat_css_list into a different cacheline from the following + * rstat_flush_next and *bstat fields which can have frequent updates. + */ + CACHELINE_PADDING(_pad_); + + /* * A singly-linked list of cgroup structures to be rstat flushed. * This is a scratch field to be used exclusively by * cgroup_rstat_flush_locked() and protected by cgroup_rstat_lock. |