mlx4: change the ICM table allocations to lowest needed size

mlx4: change the ICM table allocations to lowest needed size

The driver currently allocates 256KB contig memort for ICM
tables which puts pressure on memory management to allocate such
large contig page in fragmented memory system. Such allocation
itself contributes to memory fragmentation and at times user
process stalls for 10's of seconds leading to slow path
dumps with mm lock contention.

This change makes the driver allocate lowest page size
needed for ICM allocation(8K), which fixes these stalls.

With 4K chunk sizes the QP table size is 4MB, which cannot be allocated
by kmalloc. A larger design change would be neccesary to break up the
table. With 8k chunks the same table is 2MB, which can be allocated by
kmalloc. This large table allocation only happens once at driver load
time.

Orabug: 27718303

Signed-off-by: Daniel Jurgens <danielj@mellanox.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>
Signed-off-by: Qing Huang <qing.huang@oracle.com>
Reviewed-by: Ajaykumar Hotchandani <ajaykumar.hotchandani@oracle.com>
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com>