mm/memory-failure: support disabling soft offline for HugeTLB pages

Some BIOS suppress ("cloak") corrected memory errors until a threshold
is reached.  Once that threshold is reached, BIOS reports a CPER with
the "error threshold exceeded" bit set via GHES and the corresponding
page is soft offlined.

BIOS does not know the page type of the corresponding page.  If the
corresponding page happens to be a HugeTLB page, it will be dissolved,
permanently reducing the HugeTLB page pool.  This can be problematic
for workloads that depend on a fixed number of HugeTLB pages.

Currently, soft offline must be disabled to prevent HugeTLB pages from
being soft offlined.

This patch provides a middle ground. Soft offline can be disabled for
HugeTLB pages while remaining enabled for non-HugeTLB pages, preserving
the benefits of soft offline without the risk of BIOS soft offlining
HugeTLB pages.

Commit 56374430c5dfc ("mm/memory-failure: userspace controls
soft-offlining pages") introduced the following sysctl interface to
control soft offline:

/proc/sys/vm/enable_soft_offline

The interface does not distinguish between page types:

    0 - Soft offline is disabled
    1 - Soft offline is enabled

Convert enable_soft_offline to a bitmask and support disabling soft
offline for HugeTLB pages:

Bits:

    0 - Enable soft offline
    1 - Disable soft offline for HugeTLB pages

Supported values:

    0 - Soft offline is disabled
    1 - Soft offline is enabled
    3 - Soft offline is enabled (disabled for HugeTLB pages)

Existing behavior is preserved.

Update documentation and HugeTLB soft offline self tests.

Tony said:

: Recap of original problem is that some BIOS keep track of error
: threshold per-rank and use this GHES mechanism to report threshold
: exceeded on the rank.
:
: Systems that stay up a long time can accumulate enough soft errors to
: trigger this threshold.  But the action of taking a page offline isn't
: going to help.  For a 4K page this is merely annoying.  For 1G page it
: can mess things up badly.
:
: My original patch for this just skipped the GHES->offline process for
: huge pages.  But I wasn't aware of the sysctl control.  That provides a
: better solution.

Link: https://lkml.kernel.org/r/aMiu_Uku6Y5ZbuhM@hpe.com
Signed-off-by: Kyle Meyer <kyle.meyer@hpe.com>
Reported-by: Shawn Fan <shawn.fan@intel.com>
Suggested-by: Tony Luck <tony.luck@intel.com>
Cc: Borislav Betkov <bp@alien8.de>
Cc: David Hildenbrand <david@redhat.com>
Cc: Jane Chu <jane.chu@oracle.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Jiaqi Yan <jiaqiyan@google.com>
Cc: Joel Granados <joel.granados@kernel.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Cc: Miaohe Lin <linmiaohe@huawei.com>
Cc: Michal Clapinski <mclapinski@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Naoya Horiguchi <nao.horiguchi@gmail.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Russ Anderson <russ.anderson@hpe.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Yafang <laoar.shao@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/Documentation/ABI/testing/sysfs-memory-page-offline b/Documentation/ABI/testing/sysfs-memory-page-offline
index 00f4e35f916f01b23379404d342f2ccc2ac56e53..d3f05ed6605ebd397fe3dfd7345d8cba080ec2ae 100644 (file)
--- a/Documentation/ABI/testing/sysfs-memory-page-offline
+++ b/Documentation/ABI/testing/sysfs-memory-page-offline
@@ -20,6 +20,9 @@ Description:
                number, or a error when the offlining failed.  Reading
                the file is not allowed.
 
+               Soft-offline can be controlled via sysctl, see:
+               Documentation/admin-guide/sysctl/vm.rst
+
 What:          /sys/devices/system/memory/hard_offline_page
 Date:          Sep 2009
 KernelVersion: 2.6.33

diff --git a/Documentation/admin-guide/sysctl/vm.rst b/Documentation/admin-guide/sysctl/vm.rst
index 4d71211fdad8d061ccdef57477ce3dca5c78741d..ace73480eb9d2cfb5b518a5a4a17d915b996ad3c 100644 (file)
--- a/Documentation/admin-guide/sysctl/vm.rst
+++ b/Documentation/admin-guide/sysctl/vm.rst
@@ -309,19 +309,39 @@ physical memory) vs performance / capacity implications in transparent and
 HugeTLB cases.
 
 For all architectures, enable_soft_offline controls whether to soft offline
-memory pages.  When set to 1, kernel attempts to soft offline the pages
-whenever it thinks needed.  When set to 0, kernel returns EOPNOTSUPP to
-the request to soft offline the pages.  Its default value is 1.
+memory pages.
+
+enable_soft_offline is a bitmask:
+
+Bits::
+
+       0 - Enable soft offline
+       1 - Disable soft offline for HugeTLB pages
+
+Supported values::
+
+       0 - Soft offline is disabled
+       1 - Soft offline is enabled
+       3 - Soft offline is enabled (disabled for HugeTLB pages)
+
+The default value is 1.
+
+If soft offline is disabled for the requested page type, EOPNOTSUPP is returned.
 
 It is worth mentioning that after setting enable_soft_offline to 0, the
 following requests to soft offline pages will not be performed:
 
+- Request to soft offline from sysfs (soft_offline_page).
+
 - Request to soft offline pages from RAS Correctable Errors Collector.
 
-- On ARM, the request to soft offline pages from GHES driver.
+- On ARM and X86, the request to soft offline pages from GHES driver.
 
 - On PARISC, the request to soft offline pages from Page Deallocation Table.
 
+Note:
+       Soft offlining a HugeTLB page reduces the HugeTLB page pool.
+
 extfrag_threshold
 =================
 

diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 3edebb0cda30b39f5f83c94e94a795beba6ae0e6..a24806bb8e821945aa1f5d1f5ff082488f385f1f 100644 (file)
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -64,11 +64,14 @@
 #include "internal.h"
 #include "ras/ras_event.h"
 
+#define SOFT_OFFLINE_ENABLED           BIT(0)
+#define SOFT_OFFLINE_SKIP_HUGETLB      BIT(1)
+
 static int sysctl_memory_failure_early_kill __read_mostly;
 
 static int sysctl_memory_failure_recovery __read_mostly = 1;
 
-static int sysctl_enable_soft_offline __read_mostly = 1;
+static int sysctl_enable_soft_offline __read_mostly = SOFT_OFFLINE_ENABLED;
 
 atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0);
 
@@ -150,7 +153,7 @@ static const struct ctl_table memory_failure_table[] = {
                .mode           = 0644,
                .proc_handler   = proc_dointvec_minmax,
                .extra1         = SYSCTL_ZERO,
-               .extra2         = SYSCTL_ONE,
+               .extra2         = SYSCTL_THREE,
        }
 };
 
@@ -2725,12 +2728,20 @@ int soft_offline_page(unsigned long pfn, int flags)
                return -EIO;
        }
 
-       if (!sysctl_enable_soft_offline) {
+       if (!(sysctl_enable_soft_offline & SOFT_OFFLINE_ENABLED)) {
                pr_info_once("disabled by /proc/sys/vm/enable_soft_offline\n");
                put_ref_page(pfn, flags);
                return -EOPNOTSUPP;
        }
 
+       if (sysctl_enable_soft_offline & SOFT_OFFLINE_SKIP_HUGETLB) {
+               if (folio_test_hugetlb(pfn_folio(pfn))) {
+                       pr_info_once("disabled for HugeTLB pages by /proc/sys/vm/enable_soft_offline\n");
+                       put_ref_page(pfn, flags);
+                       return -EOPNOTSUPP;
+               }
+       }
+
        mutex_lock(&mf_mutex);
 
        if (PageHWPoison(page)) {

diff --git a/tools/testing/selftests/mm/hugetlb-soft-offline.c b/tools/testing/selftests/mm/hugetlb-soft-offline.c
index f086f0e04756f2e28932e5719da27ac88403fdd9..b87c8778cadfbce1e6267e9130a988c52239872d 100644 (file)
--- a/tools/testing/selftests/mm/hugetlb-soft-offline.c
+++ b/tools/testing/selftests/mm/hugetlb-soft-offline.c
@@ -5,6 +5,8 @@
  *   offlining failed with EOPNOTSUPP.
  * - if enable_soft_offline = 1, a hugepage should be dissolved and
  *   nr_hugepages/free_hugepages should be reduced by 1.
+ * - if enable_soft_offline = 3, hugepages should stay intact and soft
+ *   offlining failed with EOPNOTSUPP.
  *
  * Before running, make sure more than 2 hugepages of default_hugepagesz
  * are allocated. For example, if /proc/meminfo/Hugepagesize is 2048kB:
@@ -32,6 +34,9 @@
 
 #define EPREFIX " !!! "
 
+#define SOFT_OFFLINE_ENABLED           (1 << 0)
+#define SOFT_OFFLINE_SKIP_HUGETLB      (1 << 1)
+
 static int do_soft_offline(int fd, size_t len, int expect_errno)
 {
        char *filemap = NULL;
@@ -56,6 +61,7 @@ static int do_soft_offline(int fd, size_t len, int expect_errno)
        ksft_print_msg("Allocated %#lx bytes of hugetlb pages\n", len);
 
        hwp_addr = filemap + len / 2;
+       errno = 0;
        ret = madvise(hwp_addr, pagesize, MADV_SOFT_OFFLINE);
        ksft_print_msg("MADV_SOFT_OFFLINE %p ret=%d, errno=%d\n",
                       hwp_addr, ret, errno);
@@ -83,7 +89,7 @@ static int set_enable_soft_offline(int value)
        char cmd[256] = {0};
        FILE *cmdfile = NULL;
 
-       if (value != 0 && value != 1)
+       if (value < 0 || value > 3)
                return -EINVAL;
 
        sprintf(cmd, "echo %d > /proc/sys/vm/enable_soft_offline", value);
@@ -155,13 +161,17 @@ close:
 static void test_soft_offline_common(int enable_soft_offline)
 {
        int fd;
-       int expect_errno = enable_soft_offline ? 0 : EOPNOTSUPP;
+       int expect_errno = 0;
        struct statfs file_stat;
        unsigned long hugepagesize_kb = 0;
        unsigned long nr_hugepages_before = 0;
        unsigned long nr_hugepages_after = 0;
        int ret;
 
+       if (!(enable_soft_offline & SOFT_OFFLINE_ENABLED) ||
+            (enable_soft_offline & SOFT_OFFLINE_SKIP_HUGETLB))
+               expect_errno = EOPNOTSUPP;
+
        ksft_print_msg("Test soft-offline when enabled_soft_offline=%d\n",
                       enable_soft_offline);
 
@@ -198,7 +208,7 @@ static void test_soft_offline_common(int enable_soft_offline)
        // No need for the hugetlbfs file from now on.
        close(fd);
 
-       if (enable_soft_offline) {
+       if (expect_errno == 0) {
                if (nr_hugepages_before != nr_hugepages_after + 1) {
                        ksft_test_result_fail("MADV_SOFT_OFFLINE should reduced 1 hugepage\n");
                        return;
@@ -219,8 +229,9 @@ static void test_soft_offline_common(int enable_soft_offline)
 int main(int argc, char **argv)
 {
        ksft_print_header();
-       ksft_set_plan(2);
+       ksft_set_plan(3);
 
+       test_soft_offline_common(3);
        test_soft_offline_common(1);
        test_soft_offline_common(0);