mlx4: change the ICM table allocations to lowest needed size

The driver currently allocates 256KB contig memort for ICM
tables which puts pressure on memory management to allocate such
large contig page in fragmented memory system. Such allocation
itself contributes to memory fragmentation and at times user
process stalls for 10's of seconds leading to slow path
dumps with mm lock contention.

This change makes the driver allocate lowest page size
needed for ICM allocation(8K), which fixes these stalls.

With 4K chunk sizes the QP table size is 4MB, which cannot be allocated
by kmalloc. A larger design change would be neccesary to break up the
table. With 8k chunks the same table is 2MB, which can be allocated by
kmalloc. This large table allocation only happens once at driver load
time.

Orabug: 27718303

Signed-off-by: Daniel Jurgens <danielj@mellanox.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>
Signed-off-by: Qing Huang <qing.huang@oracle.com>
Reviewed-by: Ajaykumar Hotchandani <ajaykumar.hotchandani@oracle.com>
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com>

diff --git a/drivers/net/ethernet/mellanox/mlx4/icm.c b/drivers/net/ethernet/mellanox/mlx4/icm.c
index d633e926cf8f3c0f7c2bff030ec18e007d296352..84c5506aed9aa1c7755f4612ae075e9721e2c287 100644 (file)
--- a/drivers/net/ethernet/mellanox/mlx4/icm.c
+++ b/drivers/net/ethernet/mellanox/mlx4/icm.c
@@ -42,13 +42,9 @@
 #include "icm.h"
 #include "fw.h"
 
-/*
- * We allocate in as big chunks as we can, up to a maximum of 256 KB
- * per chunk.
- */
 enum {
-       MLX4_ICM_ALLOC_SIZE     = 1 << 18,
-       MLX4_TABLE_CHUNK_SIZE   = 1 << 18
+       MLX4_ICM_ALLOC_SIZE     = 1 << 13,
+       MLX4_TABLE_CHUNK_SIZE   = 1 << 13
 };
 
 static void mlx4_free_icm_pages(struct mlx4_dev *dev, struct mlx4_icm_chunk *chunk)