[PATCH] x86: Use RDTSCP for synchronous get_cycles if possible

RDTSCP is already synchronous and doesn't need an explicit CPUID.
This is a little faster and more importantly avoids VMEXITs on Hypervisors.

Original patch from Joerg Roedel, but reworked by AK
Also includes miscompilation fix by Eric Biederman

Cc: "Joerg Roedel" <joerg.roedel@amd.com>
Signed-off-by: Andi Kleen <ak@suse.de>

diff --git a/include/asm-i386/tsc.h b/include/asm-i386/tsc.h
index 0181f9df75393cf56174778f85b21e9f87d54ff0..3f3c1fa000b48ed8a0786435b557499922cac8db 100644 (file)
--- a/include/asm-i386/tsc.h
+++ b/include/asm-i386/tsc.h
@@ -37,6 +37,15 @@ static __always_inline cycles_t get_cycles_sync(void)
        unsigned long long ret;
        unsigned eax;
 
+       /*
+        * Use RDTSCP if possible; it is guaranteed to be synchronous
+        * and doesn't cause a VMEXIT on Hypervisors
+        */
+       alternative_io(ASM_NOP3, ".byte 0x0f,0x01,0xf9", X86_FEATURE_RDTSCP,
+                                "=A" (ret), "0" (0ULL) : "ecx", "memory");
+       if (ret)
+               return ret;
+
        /*
         * Don't do an additional sync on CPUs where we know
         * RDTSC is already synchronous: