rds: Make sure updates to cp_send_gen can be observed

cp->cp_send_gen is treated as a normal variable, although it may be
used by different threads.

This is fixed by using {READ,WRITE}_ONCE when it is incremented and
READ_ONCE when it is read outside the {acquire,release}_in_xmit
protection.

Normative reference from the Linux-Kernel Memory Model:

    Loads from and stores to shared (but non-atomic) variables should
    be protected with the READ_ONCE(), WRITE_ONCE(), and
    ACCESS_ONCE().

Clause 5.1.2.4/25 in the C standard is also relevant.

Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Knut Omang <knut.omang@oracle.com>
Acked-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
(cherry-picked from upstream e623a48ee433985f6ca0fb238f0002cc2eccdf53)

Orabug: 26519030

Reviewed-by: Alan Maguire <alan.maguire@oracle.com>

diff --git a/net/rds/send.c b/net/rds/send.c
index 0977678bee58e061f60f0b58705e7cdaecfac0bc..b907486cb9e4694b76b23d546b4ccd898cdbd9b7 100644 (file)
--- a/net/rds/send.c
+++ b/net/rds/send.c
@@ -221,8 +221,8 @@ restart:
         * we don't need a lock because the counter is only incremented
         * while we have the in_xmit bit held.
         */
-       cp->cp_send_gen++;
-       send_gen = cp->cp_send_gen;
+       send_gen = READ_ONCE(cp->cp_send_gen) + 1;
+       WRITE_ONCE(cp->cp_send_gen, send_gen);
 
        /*
         * rds_conn_shutdown() sets the conn state and then tests RDS_IN_XMIT,
@@ -498,7 +498,7 @@ over_batch:
                smp_mb();
                if ((test_bit(0, &conn->c_map_queued) ||
                     !list_empty(&cp->cp_send_queue)) &&
-                   send_gen == cp->cp_send_gen) {
+                       send_gen == READ_ONCE(cp->cp_send_gen)) {
                        rds_stats_inc(s_send_lock_queue_raced);
                        if (batch_count < send_batch_count)
                                goto restart;