stop_machine: Fix rcu_momentary_eqs() call in multi_cpu_stop()

The multi_cpu_stop() contains a loop that can initially be executed with
interrupts enabled (in the MULTI_STOP_NONE and MULTI_STOP_PREPARE states).
Interrupts are guaranteed to be once the MULTI_STOP_DISABLE_IRQ state
is reached.  Unfortunately, the rcu_momentary_eqs() function that is
currently invoked on each pass through this loop requires that interrupts
be disabled.

This commit therefore moves this call to rcu_momentary_eqs() to the body
of the "else if (curstate > MULTI_STOP_PREPARE)" portion of the loop, thus
guaranteeing that interrupts will be disabled on each call, as required.

Kudos to 朱恺乾 (Kaiqian) for noting that this had not made it to mainline.

[ paulmck: Update from rcu_momentary_dyntick_idle() to rcu_momentary_eqs(). ]

Link: https://lore.kernel.org/all/1712649736-27058-1-git-send-email-quic_mojha@quicinc.com/
Signed-off-by: Mukesh Ojha <quic_mojha@quicinc.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>

diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c
index da821ce258ea7f8e65e6692cc0bb68fe45279bb5..8896d844d738ffdf50bd9e7b7ad2218ae2573f05 100644 (file)
--- a/kernel/stop_machine.c
+++ b/kernel/stop_machine.c
@@ -250,8 +250,8 @@ static int multi_cpu_stop(void *data)
                         * be detected and reported on their side.
                         */
                        touch_nmi_watchdog();
+                       rcu_momentary_eqs();
                }
-               rcu_momentary_eqs();
        } while (curstate != MULTI_STOP_EXIT);
 
        local_irq_restore(flags);