io_uring: fix race condition reading SQ entries

A read memory barrier is required between reading SQ tail and reading
the actual data belonging to the SQ entry.

Userspace needs a matching write barrier between writing SQ entries and
updating SQ tail (using smp_store_release to update tail will do).

Signed-off-by: Stefan Bühler <source@stbuehler.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/fs/io_uring.c b/fs/io_uring.c
index a2f39faed6a7e1992f83cfe8d3b79719e8ea75e1..41e3a6f6a096afe3b70320a1d67b6420aef78001 100644 (file)
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -1739,7 +1739,8 @@ static bool io_get_sqring(struct io_ring_ctx *ctx, struct sqe_submit *s)
        head = ctx->cached_sq_head;
        /* See comment at the top of this file */
        smp_rmb();
-       if (head == READ_ONCE(ring->r.tail))
+       /* make sure SQ entry isn't read before tail */
+       if (head == smp_load_acquire(&ring->r.tail))
                return false;
 
        head = READ_ONCE(ring->array[head & ctx->sq_mask]);