sif: fmr: invalidate keys before TLB bulk invalidates

This commit reorders and sequentializes the cleanup phase when
bulk invalidates are used. The order was to post the TLB flushing
operation to the EPSC, then invalidate keys (potentially in parallel with the
ongoing flushing) before finally waiting for the TLB flushing to complete.
This way is not considered safe in general, as an incoming access to a key
can cause an invalidated PTE or PTW to be cached again and later cause
sif to read or write to a no longer valid location.

This commit makes sure that all keys are invalidated before
the TLB flushing is triggered.

Orabug: 24438867

Signed-off-by: Knut Omang <knut.omang@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>

diff --git a/drivers/infiniband/hw/sif/sif_fmr.c b/drivers/infiniband/hw/sif/sif_fmr.c
index bb88476fa5ba40e409260e1d40ab1743b1d43f6b..0dc18dd28ea9e33ae1592187aa47284ed1149673 100644 (file)
--- a/drivers/infiniband/hw/sif/sif_fmr.c
+++ b/drivers/infiniband/hw/sif/sif_fmr.c
@@ -173,7 +173,6 @@ int sif_unmap_phys_fmr_list(struct list_head *fmr_list)
        list_for_each_entry(ib_fmr, fmr_list, list) {
                cnt++;
                if (cnt >= sif_fmr_cache_flush_threshold) {
-                       ret = sif_post_flush_tlb(sdev, true);
                        flush_all = true;
                        goto key_to_invalid;
                }
@@ -214,6 +213,9 @@ key_to_invalid:
                cnt, (spqp ? " (stencil)" : ""));
 
        if (flush_all) {
+               ret = sif_post_flush_tlb(sdev, true);
+               if (ret)
+                       goto out;
                ret = sif_complete_flush_tlb(sdev);
                if (ret)
                        goto out;