mm/slub: Avoid waiting on empty per-cpu sheaf replacement

When the per-cpu sheaf is empty, attempt to replace it with a full one
from the barn.  If the barn lock is contended then fail quickly and
enter the recovery code that will refill the empty sheaf.

This is to avoid congestion on the barn in the event of many tasks
requesting allocations at the same time.  Discovered by brk1_processes()
regression from will-it-scale benchmarks.

Signed-off-by: Liam R. Howlett <Liam.Howlett@Oracle.com>

diff --git a/mm/slub.c b/mm/slub.c
index 1907a21c232aeaf6cf801d469c7d7f7a0cf68f5b..5d09c3a32fb6aa5b35dc97bf71b37f732aebec7c 100644 (file)
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -2855,7 +2855,8 @@ barn_replace_empty_sheaf(struct node_barn *barn, struct slab_sheaf *empty)
        struct slab_sheaf *full = NULL;
        unsigned long flags;
 
-       spin_lock_irqsave(&barn->lock, flags);
+       if (!spin_trylock_irqsave(&barn->lock, flags))
+               return NULL;
 
        if (barn->nr_full) {
                full = list_first_entry(&barn->sheaves_full, struct slab_sheaf,