sif: sqflush: Handle the race condition between sqflush and modify_qp

Orabug: 23759723

Due to a SIF HW bug where SIF might generate duplicate completions,
the QP state must be transitioned into shadowed ERR state (HW state is
in RESET).  In this case, modify_qp(ERR) will cause the QP state
transitions(HW/SW): from ERR (ERR) to RESET (ERR).

As a result, this means that SIF driver needs to generate
FLUSHED-IN-ERR completions when IB user performs post_send while the
QP is in shadowed ERR state. HW will not generate them as the HW QP
state is already in RESET state. The SIF driver generates
FLUSHED-IN-ERR if the last_set_state is in ERR state.  last_set_state
is a "best effort" tracked state because QP mutex cannot be held in a
non-sleep context (post_send).

The issue happens in a multi-threaded scenario where one thread is
constantly performing post_send whereas another thread is performing
modify_qp (ERR). During the QP state transition from ERR (ERR) to
RESET (ERR), both HW and SIF driver generate the FLUSHED-IN-ERR and
eventually causing duplicate completion. This patch adds a
test in post_wa4074 to mask out this condition.

Signed-off-by: Wei Lin Guay <wei.lin.guay@oracle.com>
Reviewed-by: Knut Omang <knut.omang@oracle.com>

diff --git a/drivers/infiniband/hw/sif/sif_r3.c b/drivers/infiniband/hw/sif/sif_r3.c
index f60e6082eda85d1446b7e61cf9324ddaf5096e22..b9a996ec12ce2c76af9b7487351da38ce7b7a98b 100644 (file)
--- a/drivers/infiniband/hw/sif/sif_r3.c
+++ b/drivers/infiniband/hw/sif/sif_r3.c
@@ -552,6 +552,12 @@ int post_process_wa4074(struct sif_dev *sdev, struct sif_qp *qp)
                return -1;
        }
 
+       if (qp->flags & SIF_QPF_HW_OWNED) {
+               sif_log(sdev, SIF_INFO, "qp %d is not in SHADOWED ERR state yet",
+                       qp->qp_idx);
+               return ret;
+       }
+
        /* if flush SQ is in progress, set FLUSH_SQ_IN_FLIGHT.
         */
        if (test_bit(FLUSH_SQ_IN_PROGRESS, &sq_sw->flags)) {