ice: optimize for XDP_REDIRECT in xsk path

Optimize ice_run_xdp_zc() for the XDP program verdict being
XDP_REDIRECT in the xsk zero-copy path. This path is only used when
having AF_XDP zero-copy on and in that case most packets will be
directed to user space. This provides a little over 100k extra packets
in throughput on my server when running l2fwd in xdpsock.

Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
Tested-by: George Kuruvinakunnel <george.kuruvinakunnel@intel.com>
Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>

diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c
index 83f3c9574ed1ebde51eeb8b090584e771adc603f..727f277e9d75dfd4a72b3957f63f1dbdcd952b07 100644 (file)
--- a/drivers/net/ethernet/intel/ice/ice_xsk.c
+++ b/drivers/net/ethernet/intel/ice/ice_xsk.c
@@ -473,6 +473,14 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp)
        xdp_prog = READ_ONCE(rx_ring->xdp_prog);
 
        act = bpf_prog_run_xdp(xdp_prog, xdp);
+
+       if (likely(act == XDP_REDIRECT)) {
+               err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
+               result = !err ? ICE_XDP_REDIR : ICE_XDP_CONSUMED;
+               rcu_read_unlock();
+               return result;
+       }
+
        switch (act) {
        case XDP_PASS:
                break;
@@ -480,10 +488,6 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp)
                xdp_ring = rx_ring->vsi->xdp_rings[rx_ring->q_index];
                result = ice_xmit_xdp_buff(xdp, xdp_ring);
                break;
-       case XDP_REDIRECT:
-               err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
-               result = !err ? ICE_XDP_REDIR : ICE_XDP_CONSUMED;
-               break;
        default:
                bpf_warn_invalid_xdp_action(act);
                fallthrough;