SUNRPC: Reduce thread wake-up rate when receiving large RPC messages

With large NFS WRITE requests on TCP, I measured 5-10 thread wake-
ups to receive each request. This is because the socket layer
calls ->sk_data_ready() frequently, and each call triggers a
thread wake-up. Each recvmsg() seems to pull in less than 100KB.

Have the socket layer hold ->sk_data_ready() calls until the full
incoming message has arrived to reduce the wake-up rate.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>

diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c
index 8db8227e6c83444424f837e669731c5293dea202..2864af3abdca4c177df9f75213ab14cbb3fac6bc 100644 (file)
--- a/net/sunrpc/svcsock.c
+++ b/net/sunrpc/svcsock.c
@@ -1089,6 +1089,9 @@ static void svc_tcp_fragment_received(struct svc_sock *svsk)
        /* If we have more data, signal svc_xprt_enqueue() to try again */
        svsk->sk_tcplen = 0;
        svsk->sk_marker = xdr_zero;
+
+       smp_wmb();
+       tcp_set_rcvlowat(svsk->sk_sk, 1);
 }
 
 /**
@@ -1178,10 +1181,17 @@ err_incomplete:
                goto err_delete;
        if (len == want)
                svc_tcp_fragment_received(svsk);
-       else
+       else {
+               /* Avoid more ->sk_data_ready() calls until the rest
+                * of the message has arrived. This reduces service
+                * thread wake-ups on large incoming messages. */
+               tcp_set_rcvlowat(svsk->sk_sk,
+                                svc_sock_reclen(svsk) - svsk->sk_tcplen);
+
                trace_svcsock_tcp_recv_short(&svsk->sk_xprt,
                                svc_sock_reclen(svsk),
                                svsk->sk_tcplen - sizeof(rpc_fraghdr));
+       }
        goto err_noclose;
 error:
        if (len != -EAGAIN)