diff options
| author | Pavel Begunkov <asml.silence@gmail.com> | 2022-10-06 02:06:10 +0100 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2022-10-12 16:30:56 -0600 |
| commit | fc86f9d3bb4904117eea70347d323fde34a47c79 (patch) | |
| tree | 5c4b611b28fb3ad7f5412edcd3b057540768d1c5 /io_uring/io_uring.c | |
| parent | 3fb1bd68817288729179444caf1fd5c5c4d2d65d (diff) | |
| download | linux-fc86f9d3bb4904117eea70347d323fde34a47c79.tar.gz linux-fc86f9d3bb4904117eea70347d323fde34a47c79.tar.bz2 linux-fc86f9d3bb4904117eea70347d323fde34a47c79.zip | |
io_uring: remove redundant memory barrier in io_req_local_work_add
io_cqring_wake() needs a barrier for the waitqueue_active() check.
However, in the case of io_req_local_work_add(), we call llist_add()
first, which implies an atomic. Hence we can replace smb_mb() with
smp_mb__after_atomic().
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/43983bc8bc507172adda7a0f00cab1aff09fd238.1665018309.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Diffstat (limited to 'io_uring/io_uring.c')
| -rw-r--r-- | io_uring/io_uring.c | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index b12ec6b5a464..12870cd7cb07 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -1106,6 +1106,8 @@ static void io_req_local_work_add(struct io_kiocb *req) if (!llist_add(&req->io_task_work.node, &ctx->work_llist)) return; + /* need it for the following io_cqring_wake() */ + smp_mb__after_atomic(); if (unlikely(atomic_read(&req->task->io_uring->in_idle))) { io_move_task_work_from_local(ctx); @@ -1117,8 +1119,7 @@ static void io_req_local_work_add(struct io_kiocb *req) if (ctx->has_evfd) io_eventfd_signal(ctx); - io_cqring_wake(ctx); - + __io_cqring_wake(ctx); } static inline void __io_req_task_work_add(struct io_kiocb *req, bool allow_local) |