kmsan: support SLAB_POISON

Avoid false KMSAN negatives with SLUB_DEBUG by allowing kmsan_slab_free()
to poison the freed memory, and by preventing init_object() from
unpoisoning new allocations by using __memset().

There are two alternatives to this approach.  First, init_object() can be
marked with __no_sanitize_memory.  This annotation should be used with
great care, because it drops all instrumentation from the function, and
any shadow writes will be lost.  Even though this is not a concern with
the current init_object() implementation, this may change in the future.

Second, kmsan_poison_memory() calls may be added after memset() calls.
The downside is that init_object() is called from free_debug_processing(),
in which case poisoning will erase the distinction between simply
uninitialized memory and UAF.

Link: https://lkml.kernel.org/r/20240621113706.315500-14-iii@linux.ibm.com
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: <kasan-dev@googlegroups.com>
Cc: Marco Elver <elver@google.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Steven Rostedt (Google) <rostedt@goodmis.org>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c
index 267d0afa2e8ba49da770a599c7767062bee32192..26d86dfdc819751e18c7bdd3408c0a098a400059 100644 (file)
--- a/mm/kmsan/hooks.c
+++ b/mm/kmsan/hooks.c
@@ -74,7 +74,7 @@ void kmsan_slab_free(struct kmem_cache *s, void *object)
                return;
 
        /* RCU slabs could be legally used after free within the RCU period */
-       if (unlikely(s->flags & (SLAB_TYPESAFE_BY_RCU | SLAB_POISON)))
+       if (unlikely(s->flags & SLAB_TYPESAFE_BY_RCU))
                return;
        /*
         * If there's a constructor, freed memory must remain in the same state

diff --git a/mm/slub.c b/mm/slub.c
index 4927edec6a8c938ba5902c45d6da4ef0eaf55d6c..dbe8a6303937ada0561c8120dd7bb9b722c1fbda 100644 (file)
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1139,7 +1139,13 @@ static void init_object(struct kmem_cache *s, void *object, u8 val)
        unsigned int poison_size = s->object_size;
 
        if (s->flags & SLAB_RED_ZONE) {
-               memset(p - s->red_left_pad, val, s->red_left_pad);
+               /*
+                * Here and below, avoid overwriting the KMSAN shadow. Keeping
+                * the shadow makes it possible to distinguish uninit-value
+                * from use-after-free.
+                */
+               memset_no_sanitize_memory(p - s->red_left_pad, val,
+                                         s->red_left_pad);
 
                if (slub_debug_orig_size(s) && val == SLUB_RED_ACTIVE) {
                        /*
@@ -1152,12 +1158,13 @@ static void init_object(struct kmem_cache *s, void *object, u8 val)
        }
 
        if (s->flags & __OBJECT_POISON) {
-               memset(p, POISON_FREE, poison_size - 1);
-               p[poison_size - 1] = POISON_END;
+               memset_no_sanitize_memory(p, POISON_FREE, poison_size - 1);
+               memset_no_sanitize_memory(p + poison_size - 1, POISON_END, 1);
        }
 
        if (s->flags & SLAB_RED_ZONE)
-               memset(p + poison_size, val, s->inuse - poison_size);
+               memset_no_sanitize_memory(p + poison_size, val,
+                                         s->inuse - poison_size);
 }
 
 static void restore_bytes(struct kmem_cache *s, char *message, u8 data,