From: Mridula Shastry Date: Wed, 6 Mar 2019 21:29:55 +0000 (-0800) Subject: Revert x86/apic/x2apic: set affinity of a single interrupt to one cpu X-Git-Tag: v4.1.12-124.31.3~247 X-Git-Url: https://www.infradead.org/git/?a=commitdiff_plain;h=45d6ad0e5a83c8b9378fc4e991ddb81a432495ca;p=users%2Fjedix%2Flinux-maple.git Revert x86/apic/x2apic: set affinity of a single interrupt to one cpu The commit 092aa78c11f0 ("x86/apic/x2apic: set affinity of a single interrupt to one cpu") was causing performance regression on block storage server on X5. On OCI X5 server, they were not binding irqs to CPUs 1:1, irq to cpu affinity was set to multiple cpus (/proc/$irq/smp_affinity: 00,003ffff0,0003ffff, cpu0-17 and 36-53). This is not the default behavior of bnxt_en. From bnxt_en, driver, when NIC link is up, it sets irq affinity, OCI assumed that most of the bnxt_en interrupts will go to cpu3. After the patch "x86/apic/x2apic: set affinity of a single interrupt to one cpu", if we set irq to cpu 1:1, it works fine, but if we set irq affinity to multiple cpus, it only sets irq_cfg->domain/cpumask to the first online cpu which is on the cpu affinity list. With the current setting which caused the perf issue, although /proc/$irq/smp_affinity is set to multiple cpus, irq_cfg->domain cpumask only has cpu 0, this lead all ens4f0-TxRx interrupts to route to cpu0, also iscsi target application was being run on CPU0 during the testing which led to the performace issue. The issue is no longer seen after the patch was reverted. Orabug: 29449976 Signed-off-by: Mridula Shastry Reviewed-by: Joe Jin Signed-off-by: Somasundaram Krishnasamy --- diff --git a/arch/x86/kernel/apic/x2apic_cluster.c b/arch/x86/kernel/apic/x2apic_cluster.c index fb2edecc470a..ab3219b3fbda 100644 --- a/arch/x86/kernel/apic/x2apic_cluster.c +++ b/arch/x86/kernel/apic/x2apic_cluster.c @@ -219,7 +219,19 @@ static const struct cpumask *x2apic_cluster_target_cpus(void) static void cluster_vector_allocation_domain(int cpu, struct cpumask *retmask, const struct cpumask *mask) { - cpumask_copy(retmask, cpumask_of(cpu)); + /* + * To minimize vector pressure, default case of boot, device bringup + * etc will use a single cpu for the interrupt destination. + * + * On explicit migration requests coming from irqbalance etc, + * interrupts will be routed to the x2apic cluster (cluster-id + * derived from the first cpu in the mask) members specified + * in the mask. + */ + if (mask == x2apic_cluster_target_cpus()) + cpumask_copy(retmask, cpumask_of(cpu)); + else + cpumask_and(retmask, mask, per_cpu(cpus_in_cluster, cpu)); } static struct apic apic_x2apic_cluster = {