slab: remove slub sysfs interface files early for empty memcg caches

With kmem cgroup support enabled, kmem_caches can be created and
destroyed frequently and a great number of near empty kmem_caches can
accumulate if there are a lot of transient cgroups and the system is not
under memory pressure.  When memory reclaim starts under such
conditions, it can lead to consecutive deactivation and destruction of
many kmem_caches, easily hundreds of thousands on moderately large
systems, exposing scalability issues in the current slab management
code.  This is one of the patches to address the issue.

Each cache has a number of sysfs interface files under /sys/kernel/slab.
On a system with a lot of memory and transient memcgs, the number of
interface files which have to be removed once memory reclaim kicks in
can reach millions.

Link: http://lkml.kernel.org/r/20170117235411.9408-10-tj@kernel.org
Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Jay Vana <jsvana@fb.com>
Acked-by: Vladimir Davydov <vdavydov.dev@gmail.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

diff --git a/mm/slub.c b/mm/slub.c
index 62d0b557a596616b71ddf19faa251f47b93c0a7e..af38aaad34b0244760af75edfe0200222f5b6e48 100644 (file)
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -3959,8 +3959,20 @@ int __kmem_cache_shrink(struct kmem_cache *s)
 #ifdef CONFIG_MEMCG
 static void kmemcg_cache_deact_after_rcu(struct kmem_cache *s)
 {
-       /* called with all the locks held after a sched RCU grace period */
-       __kmem_cache_shrink(s);
+       /*
+        * Called with all the locks held after a sched RCU grace period.
+        * Even if @s becomes empty after shrinking, we can't know that @s
+        * doesn't have allocations already in-flight and thus can't
+        * destroy @s until the associated memcg is released.
+        *
+        * However, let's remove the sysfs files for empty caches here.
+        * Each cache has a lot of interface files which aren't
+        * particularly useful for empty draining caches; otherwise, we can
+        * easily end up with millions of unnecessary sysfs files on
+        * systems which have a lot of memory and transient cgroups.
+        */
+       if (!__kmem_cache_shrink(s))
+               sysfs_slab_remove(s);
 }
 
 void __kmemcg_cache_deactivate(struct kmem_cache *s)
@@ -5659,6 +5671,15 @@ static void sysfs_slab_remove(struct kmem_cache *s)
                 */
                return;
 
+       if (!s->kobj.state_in_sysfs)
+               /*
+                * For a memcg cache, this may be called during
+                * deactivation and again on shutdown.  Remove only once.
+                * A cache is never shut down before deactivation is
+                * complete, so no need to worry about synchronization.
+                */
+               return;
+
 #ifdef CONFIG_MEMCG
        kset_unregister(s->memcg_kset);
 #endif