x86/fineibt: Poison ENDBR at +0

Alyssa noticed that when building the kernel with CFI_CLANG+IBT and
booting on IBT enabled hardware to obtain FineIBT, the indirect
functions look like:

  __cfi_foo:
	endbr64
	subl	$hash, %r10d
	jz	1f
	ud2
	nop
  1:
  foo:
	endbr64

This is because the compiler generates code for kCFI+IBT. In that case
the caller does the hash check and will jump to +0, so there must be
an ENDBR there. The compiler doesn't know about FineIBT at all; also
it is possible to actually use kCFI+IBT when booting with 'cfi=kcfi'
on IBT enabled hardware.

Having this second ENDBR however makes it possible to elide the CFI
check. Therefore, we should poison this second ENDBR when switching to
FineIBT mode.

Fixes: 931ab63664f0 ("x86/ibt: Implement FineIBT")
Reported-by: "Milburn, Alyssa" <alyssa.milburn@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
Link: https://lore.kernel.org/r/20230615193722.194131053@infradead.org

diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index 04b25a27523395f1f1960558cebad41e0aaa9cee..d77aaabf9fe8e5d1326149df897cc05dcd326cc9 100644 (file)
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -1068,6 +1068,17 @@ static int cfi_rewrite_preamble(s32 *start, s32 *end)
        return 0;
 }
 
+static void cfi_rewrite_endbr(s32 *start, s32 *end)
+{
+       s32 *s;
+
+       for (s = start; s < end; s++) {
+               void *addr = (void *)s + *s;
+
+               poison_endbr(addr+16, false);
+       }
+}
+
 /* .retpoline_sites */
 static int cfi_rand_callers(s32 *start, s32 *end)
 {
@@ -1162,14 +1173,19 @@ static void __apply_fineibt(s32 *start_retpoline, s32 *end_retpoline,
                return;
 
        case CFI_FINEIBT:
+               /* place the FineIBT preamble at func()-16 */
                ret = cfi_rewrite_preamble(start_cfi, end_cfi);
                if (ret)
                        goto err;
 
+               /* rewrite the callers to target func()-16 */
                ret = cfi_rewrite_callers(start_retpoline, end_retpoline);
                if (ret)
                        goto err;
 
+               /* now that nobody targets func()+0, remove ENDBR there */
+               cfi_rewrite_endbr(start_cfi, end_cfi);
+
                if (builtin)
                        pr_info("Using FineIBT CFI\n");
                return;