RDS: fix IB transport flow control

Orabug: 22306628

IB flow control is always disabled regardless of
rds_ib_sysctl_flow_control flag.
The issue is that the initial credit advertisement
annouces zero credits, because ib_recv_refill() has
not yet been called. An initial credit offering
of zero effectively disables flow control.

IB flow control is only enabled if both active and
passive connections have set the rds_ib_sysctl
flow_control flag. E.g,

Conn. A (on),  Conn. B (on)  = enable
Conn. A (off), Conn. B (on)  = disable
Conn. A (on),  Conn. B (off) = disable
Conn. A (off), Conn. B (off) = disable

Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Knut Omang <knut.omang@oracle.com>
Reviewed-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>
Acked-by: Ajaykumar Hotchandani <ajaykumar.hotchandani@oracle.com>
Signed-off-by: Wei Lin Guay <wei.lin.guay@oracle.com>

diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c
index 1fb8c441876e079c9d87a42daf64b747c78e14d9..83baf19f42758b457e2ba8524aaefa407e53a709 100644 (file)
--- a/net/rds/ib_cm.c
+++ b/net/rds/ib_cm.c
@@ -973,6 +973,12 @@ int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id,
        conn->c_connection_start = get_seconds();
 
        rds_ib_set_flow_control(conn, be32_to_cpu(dp->dp_credit));
+       /* Use ic->i_flowctl as the first post credit to enable
+        * IB transport flow control. This first post credit is
+        * deducted after advertise the credit to the remote
+        * connection.
+        */
+       atomic_set(&ic->i_credits, IB_SET_POST_CREDITS(ic->i_flowctl));
 
        /* If the peer gave us the last packet it saw, process this as if
         * we had received a regular ACK. */
@@ -1046,6 +1052,12 @@ int rds_ib_cm_initiate_connect(struct rdma_cm_id *cm_id)
 
        rds_ib_set_protocol(conn, RDS_PROTOCOL_4_1);
        ic->i_flowctl = rds_ib_sysctl_flow_control;     /* advertise flow control */
+       /* Use ic->i_flowctl as the first post credit to enable
+        * IB transport flow control. This first post credit is
+        * deducted after advertise the credit to the remote
+        * connection.
+        */
+       atomic_set(&ic->i_credits, IB_SET_POST_CREDITS(ic->i_flowctl));
 
        pr_debug("RDS/IB: Initiate conn <%pI4, %pI4,%d> with Frags <init,ic>: {%d,%d}\n",
                 &conn->c_laddr, &conn->c_faddr, conn->c_tos,