From: Saurabh Sengar Date: Fri, 5 Jul 2024 08:48:21 +0000 (-0700) Subject: mm/vmstat: defer the refresh_zone_stat_thresholds after all CPUs bringup X-Git-Url: https://www.infradead.org/git/?a=commitdiff_plain;h=7ee52f4284ab1e4a195e15d326a733ef7d4eabcd;p=users%2Fjedix%2Flinux-maple.git mm/vmstat: defer the refresh_zone_stat_thresholds after all CPUs bringup refresh_zone_stat_thresholds function has two loops which is expensive for higher number of CPUs and NUMA nodes. Below is the rough estimation of total iterations done by these loops based on number of NUMA and CPUs. Total number of iterations: nCPU * 2 * Numa * mCPU Where: nCPU = total number of CPUs Numa = total number of NUMA nodes mCPU = mean value of total CPUs (e.g., 512 for 1024 total CPUs) For the system under test with 16 NUMA nodes and 1024 CPUs, this results in a substantial increase in the number of loop iterations during boot-up when NUMA is enabled: No NUMA = 1024*2*1*512 = 1,048,576 : Here refresh_zone_stat_thresholds takes around 224 ms total for all the CPUs in the system under test. 16 NUMA = 1024*2*16*512 = 16,777,216 : Here refresh_zone_stat_thresholds takes around 4.5 seconds total for all the CPUs in the system under test. Calling this for each CPU is expensive when there are large number of CPUs along with multiple NUMAs. Fix this by deferring refresh_zone_stat_thresholds to be called later at once when all the secondary CPUs are up. Also, register the DYN hooks to keep the existing hotplug functionality intact. Without this patch, refresh_zone_stat_threshold was being called 1024 times. After applying the patch, it is called only once, which is same as the last iteration of the earlier 1024 calls. Further testing with this patch, I observed a 4.5-second improvement in the overall boot timing due to this fix, which is same as the total time taken by refresh_zone_stat_thresholds without thie patch, leading me to reasonably conclude that refresh_zone_stat_threshold now takes a negligible amount of time (likely just a few milliseconds). Link: https://lkml.kernel.org/r/1720169301-21002-1-git-send-email-ssengar@linux.microsoft.com Signed-off-by: Saurabh Sengar Cc: Wei Liu Signed-off-by: Andrew Morton --- diff --git a/mm/vmstat.c b/mm/vmstat.c index aea58e9fce60..e361c2b3a8bc 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -31,6 +31,7 @@ #include "internal.h" +static int vmstat_late_init_done; #ifdef CONFIG_NUMA int sysctl_vm_numa_stat = ENABLE_NUMA_STAT; @@ -2130,7 +2131,8 @@ static void __init init_cpu_node_state(void) static int vmstat_cpu_online(unsigned int cpu) { - refresh_zone_stat_thresholds(); + if (vmstat_late_init_done) + refresh_zone_stat_thresholds(); if (!node_state(cpu_to_node(cpu), N_CPU)) { node_set_state(cpu_to_node(cpu), N_CPU); @@ -2162,6 +2164,14 @@ static int vmstat_cpu_dead(unsigned int cpu) return 0; } +static int __init vmstat_late_init(void) +{ + refresh_zone_stat_thresholds(); + vmstat_late_init_done = 1; + + return 0; +} +late_initcall(vmstat_late_init); #endif struct workqueue_struct *mm_percpu_wq;