summaryrefslogtreecommitdiffstats
path: root/block/bio.c
diff options
context:
space:
mode:
authorPavel Begunkov <asml.silence@gmail.com>2024-02-07 14:14:28 +0000
committerJens Axboe <axboe@kernel.dk>2024-02-08 10:18:47 -0700
commitc9f5f3aa19c617fe85085b19abbf7a9a077336d0 (patch)
tree52ca30cebabae974e1792ed1c905e410fb417de8 /block/bio.c
parent79ae56fc475869d636071f66d9e4ef2a3819eee6 (diff)
downloadlinux-c9f5f3aa19c617fe85085b19abbf7a9a077336d0.tar.gz
linux-c9f5f3aa19c617fe85085b19abbf7a9a077336d0.tar.bz2
linux-c9f5f3aa19c617fe85085b19abbf7a9a077336d0.zip
block: extend bio caching to task context
bio_put_percpu_cache() puts all non-iopoll bios into the irq-safe list, which entails disabling irqs. The overhead of that is not that bad when interrupts are already off but getting worse otherwise. We can optimise it when we're in the task context by using ->free_list directly just as the IOPOLL path does. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/4774e1a0f905f96c63174b0f3e4f79f0d9b63246.1707314970.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
Diffstat (limited to 'block/bio.c')
-rw-r--r--block/bio.c3
1 files changed, 2 insertions, 1 deletions
diff --git a/block/bio.c b/block/bio.c
index b9642a41f286..8da941974f88 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -770,8 +770,9 @@ static inline void bio_put_percpu_cache(struct bio *bio)
bio_uninit(bio);
- if ((bio->bi_opf & REQ_POLLED) && !WARN_ON_ONCE(in_interrupt())) {
+ if (in_task()) {
bio->bi_next = cache->free_list;
+ /* Not necessary but helps not to iopoll already freed bios */
bio->bi_bdev = NULL;
cache->free_list = bio;
cache->nr++;