kasan: improve free meta storage in Generic KASAN

Currently free meta can only be stored in object if the object is not
smaller than free meta.

After the improvement, even when the object is smaller than free meta, it
is still possible to store part of the free meta in the object, reducing
the increased size of the redzone.

Example:

free meta size: 16 bytes
alloc meta size: 16 bytes
object size: 8 bytes
optimal redzone size (object_size <= 64): 16 bytes

Before improvement:
actual redzone size = alloc meta size + free meta size = 32 bytes

After improvement:
actual redzone size = alloc meta size + (free meta size - object size)
                    = 24 bytes

Link: https://lkml.kernel.org/r/VI1P193MB0752DE2CCD9046B5FED0AA8E99B5A@VI1P193MB0752.EURP193.PROD.OUTLOOK.COM
Signed-off-by: Juntong Deng <juntong.deng@outlook.com>
Suggested-by: Dmitry Vyukov <dvyukov@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Konovalov <andreyknvl@gmail.com>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c
index 50cc519e23f4f008f2a6bb38cf2179ea3a1831d1..7ca5d0b9158596caf5c058240eb469969820c4f8 100644 (file)
--- a/mm/kasan/generic.c
+++ b/mm/kasan/generic.c
@@ -362,6 +362,8 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size,
 {
        unsigned int ok_size;
        unsigned int optimal_size;
+       unsigned int rem_free_meta_size;
+       unsigned int orig_alloc_meta_offset;
 
        if (!kasan_requires_meta())
                return;
@@ -395,6 +397,9 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size,
                /* Continue, since free meta might still fit. */
        }
 
+       ok_size = *size;
+       orig_alloc_meta_offset = cache->kasan_info.alloc_meta_offset;
+
        /*
         * Add free meta into redzone when it's not possible to store
         * it in the object. This is the case when:
@@ -402,21 +407,26 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size,
         *    be touched after it was freed, or
         * 2. Object has a constructor, which means it's expected to
         *    retain its content until the next allocation, or
-        * 3. Object is too small.
         * Otherwise cache->kasan_info.free_meta_offset = 0 is implied.
+        * Even if the object is smaller than free meta, it is still
+        * possible to store part of the free meta in the object.
         */
-       if ((cache->flags & SLAB_TYPESAFE_BY_RCU) || cache->ctor ||
-           cache->object_size < sizeof(struct kasan_free_meta)) {
-               ok_size = *size;
-
+       if ((cache->flags & SLAB_TYPESAFE_BY_RCU) || cache->ctor) {
                cache->kasan_info.free_meta_offset = *size;
                *size += sizeof(struct kasan_free_meta);
+       } else if (cache->object_size < sizeof(struct kasan_free_meta)) {
+               rem_free_meta_size = sizeof(struct kasan_free_meta) -
+                                                               cache->object_size;
+               *size += rem_free_meta_size;
+               if (cache->kasan_info.alloc_meta_offset != 0)
+                       cache->kasan_info.alloc_meta_offset += rem_free_meta_size;
+       }
 
-               /* If free meta doesn't fit, don't add it. */
-               if (*size > KMALLOC_MAX_SIZE) {
-                       cache->kasan_info.free_meta_offset = KASAN_NO_FREE_META;
-                       *size = ok_size;
-               }
+       /* If free meta doesn't fit, don't add it. */
+       if (*size > KMALLOC_MAX_SIZE) {
+               cache->kasan_info.free_meta_offset = KASAN_NO_FREE_META;
+               cache->kasan_info.alloc_meta_offset = orig_alloc_meta_offset;
+               *size = ok_size;
        }
 
        /* Calculate size with optimal redzone. */