bnxt: fill data page pool with frags if PAGE_SIZE > BNXT_RX_PAGE_SIZE

The data page pool always fills the HW rx ring with pages. On arm64 with
64K pages, this will waste _at least_ 32K of memory per entry in the rx
ring.

Fix by fragmenting the pages if PAGE_SIZE > BNXT_RX_PAGE_SIZE. This
makes the data page pool the same as the header pool.

Tested with iperf3 with a small (64 entries) rx ring to encourage buffer
circulation.

Fixes: cd1fafe7da1f ("eth: bnxt: add support rx side device memory TCP")
Reviewed-by: Michael Chan <michael.chan@broadcom.com>
Signed-off-by: David Wei <dw@davidwei.uk>
Link: https://patch.msgid.link/20250812182907.1540755-1-dw@davidwei.uk
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 76a4c5ae8000d3554e8219d6b12c8c920fd35e79..2800a90fba1f3746ec881c58d635bc5c8e500a06 100644 (file)
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -926,15 +926,21 @@ static struct page *__bnxt_alloc_rx_page(struct bnxt *bp, dma_addr_t *mapping,
 
 static netmem_ref __bnxt_alloc_rx_netmem(struct bnxt *bp, dma_addr_t *mapping,
                                         struct bnxt_rx_ring_info *rxr,
+                                        unsigned int *offset,
                                         gfp_t gfp)
 {
        netmem_ref netmem;
 
-       netmem = page_pool_alloc_netmems(rxr->page_pool, gfp);
+       if (PAGE_SIZE > BNXT_RX_PAGE_SIZE) {
+               netmem = page_pool_alloc_frag_netmem(rxr->page_pool, offset, BNXT_RX_PAGE_SIZE, gfp);
+       } else {
+               netmem = page_pool_alloc_netmems(rxr->page_pool, gfp);
+               *offset = 0;
+       }
        if (!netmem)
                return 0;
 
-       *mapping = page_pool_get_dma_addr_netmem(netmem);
+       *mapping = page_pool_get_dma_addr_netmem(netmem) + *offset;
        return netmem;
 }
 
@@ -1029,7 +1035,7 @@ static int bnxt_alloc_rx_netmem(struct bnxt *bp, struct bnxt_rx_ring_info *rxr,
        dma_addr_t mapping;
        netmem_ref netmem;
 
-       netmem = __bnxt_alloc_rx_netmem(bp, &mapping, rxr, gfp);
+       netmem = __bnxt_alloc_rx_netmem(bp, &mapping, rxr, &offset, gfp);
        if (!netmem)
                return -ENOMEM;