]> www.infradead.org Git - users/jedix/linux-maple.git/commit
net/mlx5e: TX, Utilize WQ fragments edge for multi-packet WQEs
authorTariq Toukan <tariqt@nvidia.com>
Wed, 19 Mar 2025 13:42:26 +0000 (15:42 +0200)
committerJakub Kicinski <kuba@kernel.org>
Tue, 25 Mar 2025 14:27:08 +0000 (07:27 -0700)
commit9da10c2d69c31eb88cb2df8599953eddd5371617
treee6a7f607fef63465ef964af7e7ca6fd450b91f0d
parent3a17f23f7c36bac3a3584aaf97d3e3e0b2790396
net/mlx5e: TX, Utilize WQ fragments edge for multi-packet WQEs

For simplicity reasons, the driver avoids crossing work queue fragment
boundaries within the same TX WQE (Work-Queue Element). Until today, as
the number of packets in a TX MPWQE (Multi-Packet WQE) descriptor is not
known in advance, the driver pre-prepared contiguous memory for the
largest possible WQE. For this, when getting too close to the fragment
edge, having no room for the largest WQE possible, the driver was
filling the fragment remainder with NOP descriptors, aligning the next
descriptor to the beginning of the next fragment.

Generating and handling these NOPs wastes resources, like: CPU cycles,
work-queue entries fetched to the device, and PCI bandwidth.

In this patch, we replace this NOPs filling mechanism in the TX MPWQE
flow. Instead, we utilize the remaining entries of the fragment with a
TX MPWQE. If this room turns out to be too small, we simply open an
additional descriptor starting at the beginning of the next fragment.

Performance benchmark:
uperf test, single server against 3 clients.
TCP multi-stream, bidir, traffic profile "2x350B read, 1400B write".
Bottleneck is in inbound PCI bandwidth (device POV).

+---------------+------------+------------+--------+
|               | Before     | After      |        |
+---------------+------------+------------+--------+
| BW            | 117.4 Gbps | 121.1 Gbps | +3.1%  |
+---------------+------------+------------+--------+
| tx_packets    | 15 M/sec   | 15.5 M/sec | +3.3%  |
+---------------+------------+------------+--------+
| tx_nops       | 3  M/sec   | 0          | -100%  |
+---------------+------------+------------+--------+

Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
Reviewed-by: Dragos Tatulea <dtatulea@nvidia.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Link: https://patch.msgid.link/1742391746-118647-1-git-send-email-tariqt@nvidia.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
drivers/net/ethernet/mellanox/mlx5/core/en.h
drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h
drivers/net/ethernet/mellanox/mlx5/core/en_tx.c