]> www.infradead.org Git - users/jedix/linux-maple.git/commit
mm: page_alloc: fix missed updates of PGFREE in free_unref_{page/folios}
authorYosry Ahmed <yosryahmed@google.com>
Wed, 4 Sep 2024 20:54:19 +0000 (20:54 +0000)
committerAndrew Morton <akpm@linux-foundation.org>
Mon, 9 Sep 2024 23:39:14 +0000 (16:39 -0700)
commitec867977fed07b599fd9f24c6950628f88cbc29a
tree0440f85ea2e095f3ec92ae46ece4038b556bf35b
parentdf7e1286b1dc3d6cff952cf3ef4f0c36831e2fbb
mm: page_alloc: fix missed updates of PGFREE in free_unref_{page/folios}

PGFREE is currently updated in two code paths:

- __free_pages_ok(): for pages freed to the buddy allocator.
- free_unref_page_commit(): for pages freed to the pcplists.

Before commit df1acc856923 ("mm/page_alloc: avoid conflating IRQs disabled
with zone->lock"), free_unref_page_commit() used to fallback to freeing
isolated pages directly to the buddy allocator through free_one_page().
This was done _after_ updating PGFREE, so the counter was correctly
updated.

However, that commit moved the fallback logic to its callers (now called
free_unref_page() and free_unref_folios()), so PGFREE was no longer
updated in this fallback case.

Now that the code has developed, there are more cases in free_unref_page()
and free_unref_folios() where we fallback to calling free_one_page() (e.g.
!pcp_allowed_order(), pcp_spin_trylock() fails).  These cases also miss
updating PGFREE.

To make sure PGFREE is updated in all cases where pages are freed to the
buddy allocator, move the update down the stack to free_one_page().

This was noticed through code inspection, although it should be noticeable
at runtime (at least with some workloads).

Link: https://lkml.kernel.org/r/20240904205419.821776-1-yosryahmed@google.com
Fixes: df1acc856923 ("mm/page_alloc: avoid conflating IRQs disabled with zone->lock")
Signed-off-by: Yosry Ahmed <yosryahmed@google.com>
Cc: Brendan Jackman <jackmanb@google.com>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
mm/page_alloc.c