dma-buf/heaps: system_heap: avoid too much allocation

dma-buf/heaps: system_heap: avoid too much allocation

Normal free:212600kB min:7664kB low:57100kB high:106536kB
  reserved_highatomic:4096KB active_anon:276kB inactive_anon:180kB
  active_file:1200kB inactive_file:0kB unevictable:2932kB
  writepending:0kB present:4109312kB managed:3689488kB mlocked:2932kB
  pagetables:13600kB bounce:0kB free_pcp:0kB local_pcp:0kB
  free_cma:200844kB
Out of memory and no killable processes...
Kernel panic - not syncing: System is deadlocked on memory

An OoM panic was reported.  The log shows there were only native processes
which are non-killable as OOM_SCORE_ADJ_MIN.  After looking into the dump,
I've found the dma-buf system heap was trying to allocate a huge size.  It
seems to be a signed negative value.

dma_heap_ioctl_allocate(inline)
    |  heap_allocation = 0xFFFFFFC02247BD38 -> (
    |    len = 0xFFFFFFFFE7225100,

To avoid this invalid request, check if the requested size is bigger than
system total memory.  Actually the old ion system heap had similar policy
with commit c9e8440eca61 ("staging: ion: Fix overflow and list bugs in
system heap").

Even with this sanity check, there is still risk of too much allocations
from the system_heap.  Allocating multiple big size buffers may cause oom.
Add __GFP_RETRY_MAYFAIL.  With this gfp, the allocation may fail, but we
can avoid oom panic.

Link: https://lkml.kernel.org/r/20230410073228.23043-1-jaewon31.kim@samsung.com
Signed-off-by: Jaewon Kim <jaewon31.kim@samsung.com>
Acked-by: John Stultz <jstultz@google.com>
Reviewed-by: T.J. Mercier <tjmercier@google.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>