mm/thp: fix MTE tag mismatch when replacing zero-filled subpages

When both THP and MTE are enabled, splitting a THP and replacing its
zero-filled subpages with the shared zeropage can cause MTE tag mismatch
faults in userspace.

Remapping zero-filled subpages to the shared zeropage is unsafe, as the
zeropage has a fixed tag of zero, which may not match the tag expected by
the userspace pointer.

KSM already avoids this problem by using memcmp_pages(), which on arm64
intentionally reports MTE-tagged pages as non-identical to prevent unsafe
merging.

As suggested by David[1], this patch adopts the same pattern, replacing the
memchr_inv() byte-level check with a call to pages_identical(). This
leverages existing architecture-specific logic to determine if a page is
truly identical to the shared zeropage.

Having both the THP shrinker and KSM rely on pages_identical() makes the
design more future-proof, IMO. Instead of handling quirks in generic code,
we just let the architecture decide what makes two pages identical.

[1] https://lore.kernel.org/all/ca2106a3-4bb2-4457-81af-301fd99fbef4@redhat.com

Link: https://lkml.kernel.org/r/20250922021458.68123-1-lance.yang@linux.dev
Fixes: b1f202060afe ("mm: remap unused subpages to shared zeropage when splitting isolated thp")
Signed-off-by: Lance Yang <lance.yang@linux.dev>
Reported-by: Qun-wei Lin <Qun-wei.Lin@mediatek.com>
Closes: https://lore.kernel.org/all/a7944523fcc3634607691c35311a5d59d1a3f8d4.camel@mediatek.com
Suggested-by: David Hildenbrand <david@redhat.com>
Acked-by: Zi Yan <ziy@nvidia.com>
Acked-by: David Hildenbrand <david@redhat.com>
Acked-by: Usama Arif <usamaarif642@gmail.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Wei Yang <richard.weiyang@gmail.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: andrew.yang <andrew.yang@mediatek.com>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Byungchul Park <byungchul@sk.com>
Cc: Charlie Jenkins <charlie@rivosinc.com>
Cc: Chinwen Chang <chinwen.chang@mediatek.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Domenico Cerasuolo <cerasuolodomenico@gmail.com>
Cc: Gregory Price <gourry@gourry.net>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Kairui Song <ryncsn@gmail.com>
Cc: Kalesh Singh <kaleshsingh@google.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Cc: Mariano Pache <npache@redhat.com>
Cc: Mathew Brost <matthew.brost@intel.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Palmer Dabbelt <palmer@rivosinc.com>
Cc: Rakie Kim <rakie.kim@sk.com>
Cc: Rik van Riel <riel@surriel.com>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Samuel Holland <samuel.holland@sifive.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Yu Zhao <yuzhao@google.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 5acca24bbabbec2ab6c7b881ddf30441bf14cf77..1b81680b4225f34b928753251d8b1bbf5b4c9252 100644 (file)
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -4104,32 +4104,23 @@ static unsigned long deferred_split_count(struct shrinker *shrink,
 static bool thp_underused(struct folio *folio)
 {
        int num_zero_pages = 0, num_filled_pages = 0;
-       void *kaddr;
        int i;
 
        if (khugepaged_max_ptes_none == HPAGE_PMD_NR - 1)
                return false;
 
        for (i = 0; i < folio_nr_pages(folio); i++) {
-               kaddr = kmap_local_folio(folio, i * PAGE_SIZE);
-               if (!memchr_inv(kaddr, 0, PAGE_SIZE)) {
-                       num_zero_pages++;
-                       if (num_zero_pages > khugepaged_max_ptes_none) {
-                               kunmap_local(kaddr);
+               if (pages_identical(folio_page(folio, i), ZERO_PAGE(0))) {
+                       if (++num_zero_pages > khugepaged_max_ptes_none)
                                return true;
-                       }
                } else {
                        /*
                         * Another path for early exit once the number
                         * of non-zero filled pages exceeds threshold.
                         */
-                       num_filled_pages++;
-                       if (num_filled_pages >= HPAGE_PMD_NR - khugepaged_max_ptes_none) {
-                               kunmap_local(kaddr);
+                       if (++num_filled_pages >= HPAGE_PMD_NR - khugepaged_max_ptes_none)
                                return false;
-                       }
                }
-               kunmap_local(kaddr);
        }
        return false;
 }

diff --git a/mm/migrate.c b/mm/migrate.c
index aee61a980374002f2c3c50fffd96fbc4f6f9bcc9..ce83c2c3c2870dc6a84075a3a19db12e0388b0e9 100644 (file)
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -300,9 +300,7 @@ static bool try_to_map_unused_to_zeropage(struct page_vma_mapped_walk *pvmw,
                                          unsigned long idx)
 {
        struct page *page = folio_page(folio, idx);
-       bool contains_data;
        pte_t newpte;
-       void *addr;
 
        if (PageCompound(page))
                return false;
@@ -319,11 +317,7 @@ static bool try_to_map_unused_to_zeropage(struct page_vma_mapped_walk *pvmw,
         * this subpage has been non present. If the subpage is only zero-filled
         * then map it to the shared zeropage.
         */
-       addr = kmap_local_page(page);
-       contains_data = memchr_inv(addr, 0, PAGE_SIZE);
-       kunmap_local(addr);
-
-       if (contains_data)
+       if (!pages_identical(page, ZERO_PAGE(0)))
                return false;
 
        newpte = pte_mkspecial(pfn_pte(my_zero_pfn(pvmw->address),