summaryrefslogtreecommitdiffstats
path: root/block/blk-mq.h
diff options
context:
space:
mode:
authorMing Lei <ming.lei@redhat.com>2021-12-03 21:15:32 +0800
committerJens Axboe <axboe@kernel.dk>2021-12-03 14:51:29 -0700
commit704b914f15fb7daaf517e3acc4bed472b50ca19e (patch)
tree5fffc1140fef4ddceeaae58e44fbb65333c33861 /block/blk-mq.h
parent2a904d00855f94cb85751e45fa494f225d44ae0d (diff)
downloadlinux-stable-704b914f15fb7daaf517e3acc4bed472b50ca19e.tar.gz
linux-stable-704b914f15fb7daaf517e3acc4bed472b50ca19e.tar.bz2
linux-stable-704b914f15fb7daaf517e3acc4bed472b50ca19e.zip
blk-mq: move srcu from blk_mq_hw_ctx to request_queue
In case of BLK_MQ_F_BLOCKING, per-hctx srcu is used to protect dispatch critical area. However, this srcu instance stays at the end of hctx, and it often takes standalone cacheline, often cold. Inside srcu_read_lock() and srcu_read_unlock(), WRITE is always done on the indirect percpu variable which is allocated from heap instead of being embedded, srcu->srcu_idx is read only in srcu_read_lock(). It doesn't matter if srcu structure stays in hctx or request queue. So switch to per-request-queue srcu for protecting dispatch, and this way simplifies quiesce a lot, not mention quiesce is always done on the request queue wide. Signed-off-by: Ming Lei <ming.lei@redhat.com> Link: https://lore.kernel.org/r/20211203131534.3668411-3-ming.lei@redhat.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
Diffstat (limited to 'block/blk-mq.h')
-rw-r--r--block/blk-mq.h4
1 files changed, 2 insertions, 2 deletions
diff --git a/block/blk-mq.h b/block/blk-mq.h
index e4c396204928..792f0b29c6eb 100644
--- a/block/blk-mq.h
+++ b/block/blk-mq.h
@@ -385,9 +385,9 @@ do { \
int srcu_idx; \
\
might_sleep(); \
- srcu_idx = srcu_read_lock((hctx)->srcu); \
+ srcu_idx = srcu_read_lock((hctx)->queue->srcu); \
(dispatch_ops); \
- srcu_read_unlock((hctx)->srcu, srcu_idx); \
+ srcu_read_unlock((hctx)->queue->srcu, srcu_idx); \
} \
} while (0)