]> www.infradead.org Git - users/dwmw2/linux.git/commit
futex: improve user space accesses
authorLinus Torvalds <torvalds@linux-foundation.org>
Fri, 22 Nov 2024 19:18:25 +0000 (11:18 -0800)
committerLinus Torvalds <torvalds@linux-foundation.org>
Mon, 25 Nov 2024 20:11:55 +0000 (12:11 -0800)
commit43a43faf5376114161aa684834d24e06da596287
tree26d2cec5a7803db3a589e5ddc0b31b271c54d1e3
parent4e07155dd58cab024813e97dc384d48f34e3d16e
futex: improve user space accesses

Josh Poimboeuf reports that he got a "will-it-scale.per_process_ops 1.9%
improvement" report for his patch that changed __get_user() to use
pointer masking instead of the explicit speculation barrier.  However,
that patch doesn't actually work in the general case, because some (very
bad) architecture-specific code actually depends on __get_user() also
working on kernel addresses.

A profile showed that the offending __get_user() was the futex code,
which really should be fixed up to not use that horrid legacy case.
Rewrite futex_get_value_locked() to use the modern user acccess helpers,
and inline it so that the compiler not only avoids the function call for
a few instructions, but can do CSE on the address masking.

It also turns out the x86 futex functions have unnecessary barriers in
other places, so let's fix those up too.

Link: https://lore.kernel.org/all/20241115230653.hfvzyf3aqqntgp63@jpoimboe/
Reported-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
arch/x86/include/asm/futex.h
kernel/futex/core.c
kernel/futex/futex.h