mm: skip folio_activate() for mlocked folios

__mlock_folio() does not move folio to unevicable LRU, when
folio_activate() removes folio from LRU.

To prevent this case also check for folio_test_mlocked() in
folio_mark_accessed().  If folio is not yet marked as unevictable, but
already marked as mlocked, then skip folio_activate() call to allow
__mlock_folio() to make all necessary updates.  It should be safe to skip
folio_activate() here, because mlocked folio should end up in unevictable
LRU eventually anyway.

The user-visible effect is that we unnecessary postpone moving pages to
unevictable LRU that lead to unexpected stats: Mlocked > Unevictable.

To observe the problem mmap() and mlock() big file and check Unevictable
and Mlocked values from /proc/meminfo.  On freshly booted system without
any other mlocked memory we expect them to match or be quite close.

See below for more detailed reproduction steps.  Source code of stat.c is
available at [1].

  $ head -c 8G < /dev/urandom > /tmp/random.bin

  $ cc -pedantic -Wall -std=c99 stat.c -O3 -o /tmp/stat
  $ /tmp/stat
  Unevictable:     8389668 kB
  Mlocked:         8389700 kB

  Need to run binary twice. Problem does not reproduce on the first run,
  but always reproduces on the second run.

  $ /tmp/stat
  Unevictable:     5374676 kB
  Mlocked:         8389332 kB

Link: https://lkml.kernel.org/r/aOPDRmk2Zd20qxfk@shell.ilvokhin.com
Link: https://gist.github.com/ilvokhin/e50c3d2ff5d9f70dcbb378c6695386dd
Co-developed-by: Kiryl Shutsemau <kas@kernel.org>
Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
Acked-by: Usama Arif <usamaarif642@gmail.com>
Acked-by: Nhat Pham <nphamcs@gmail.com>
Reviewed-by: SeongJae Park <sj@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Baoquan He <bhe@redhat.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Li <chrisl@kernel.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/mm/swap.c b/mm/swap.c
index 2260dcd2775e7514682d8f71296ac8ebc800d5bf..f682f070160b7d532a695c86074e9ceea97049f2 100644 (file)
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -469,6 +469,16 @@ void folio_mark_accessed(struct folio *folio)
                 * this list is never rotated or maintained, so marking an
                 * unevictable page accessed has no effect.
                 */
+       } else if (folio_test_mlocked(folio)) {
+               /*
+                * Pages that are mlocked, but not yet on unevictable LRU.
+                * They might be still in mlock_fbatch waiting to be processed
+                * and activating it here might interfere with
+                * mlock_folio_batch(). __mlock_folio() will fail
+                * folio_test_clear_lru() check and give up. It happens because
+                * __folio_batch_add_and_move() clears LRU flag, when adding
+                * folio to activate batch.
+                */
        } else if (!folio_test_active(folio)) {
                /*
                 * If the folio is on the LRU, queue it for activation via