From: Vlastimil Babka Date: Fri, 14 Feb 2025 16:27:46 +0000 (+0100) Subject: maple_tree: use percpu sheaves for maple_node_cache X-Git-Url: https://www.infradead.org/git/?a=commitdiff_plain;h=e1ef9835ef04d444f453a3771c69ce369aaf51a0;p=users%2Fjedix%2Flinux-maple.git maple_tree: use percpu sheaves for maple_node_cache Setup the maple_node_cache with percpu sheaves of size 32 to hopefully improve its performance. Change the single node rcu freeing in ma_free_rcu() to use kfree_rcu() instead of the custom callback, which allows the rcu_free sheaf batching to be used. Note there are other users of mt_free_rcu() where larger parts of maple tree are submitted to call_rcu() as a whole, and that cannot use the rcu_free sheaf, but it's still possible for maple nodes freed this way to be reused via the barn, even if only some cpus are allowed to process rcu callbacks. Signed-off-by: Vlastimil Babka Reviewed-by: Suren Baghdasaryan --- diff --git a/lib/maple_tree.c b/lib/maple_tree.c index ea1f0acac118..850e0aa96595 100644 --- a/lib/maple_tree.c +++ b/lib/maple_tree.c @@ -208,7 +208,7 @@ static void mt_free_rcu(struct rcu_head *head) static void ma_free_rcu(struct maple_node *node) { WARN_ON(node->parent != ma_parent_ptr(node)); - call_rcu(&node->rcu, mt_free_rcu); + kfree_rcu(node, rcu); } static void mt_set_height(struct maple_tree *mt, unsigned char height) @@ -6291,9 +6291,13 @@ bool mas_nomem(struct ma_state *mas, gfp_t gfp) void __init maple_tree_init(void) { + struct kmem_cache_args args = { + .align = sizeof(struct maple_node), + .sheaf_capacity = 32, + }; + maple_node_cache = kmem_cache_create("maple_node", - sizeof(struct maple_node), sizeof(struct maple_node), - SLAB_PANIC, NULL); + sizeof(struct maple_node), &args, SLAB_PANIC); } /**