drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw image_set()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw xlut_clr()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw xlut_set()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw csc_clr()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw csc_set()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw ntfy_wait_begun()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw ntfy_clr()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw ntfy_set()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw ntfy_reset()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw sema_clr()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw sema_set()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wimm update()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wimm point()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wimm space()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/fence: convert sync() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/fence: convert emit() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gem: convert indirect pushbufs to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/chan: convert nvsw init to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau: convert nvsw init to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv04-: no need for multiple nvsw objects

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv04: convert page_flip() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv04: convert page_flip_emit() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/svm: convert migrate_clear to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/svm: convert migrate_copy to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/bo: convert move move() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/bo: convert move init() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/bo: split buffer move functions into their own source files

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/fbcon: convert copyarea() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/fbcon: convert fillrect() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/fbcon: convert imageblit() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/fbcon: convert accel_init() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau: interop with new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: implement proper push buffer control logic

We had a, what was supposed to be temporary, hack in the KMS code where we'd
completely drain an EVO/NVD channel's push buffer when wrapping to the start
again, instead of treating it as a ring buffer.

Let's fix that, finally.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/kms/nv50-: convert core crc_set_ctx() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core crc_set_src() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_or() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_procamp() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_dither() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_ovly() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_base() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_curs_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_curs_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_core_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_core_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_olut_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_olut_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_mode() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_view() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core or_ctrl() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core wndw_owner() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core update() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core caps_init() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core init() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw update() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw blend_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw scale_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw image_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw image_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw xlut_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw xlut_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw csc_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw csc_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw ntfy_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw ntfy_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw sema_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw sema_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wimm update() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wimm point() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: wrap existing command submission in nvif_push interface

This commit pulls in a bunch of new push buffer macros which are able to
support NVIDIA's class headers, and provide more useful debug output and
error checking (compile-time, where possible) than we had previously.

Will incrementally transition each function over to the unified interfaces.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: add support for object-level debug output

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every notify object a human-readable name

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every disp object a human-readable identifier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every usermode object a human-readable identifier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every vmm object a human-readable identifier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every mem object a human-readable identifier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every mmu object a human-readable identifier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every device object a human-readable identifier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every object a human-readable identifier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: rename client ctor/dtor

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/tu102: set NVC57D_HEAD_SET_HEAD_USAGE_BOUNDS_UPSCALING_ALLOWED to TRUE

Fixes issues when switching between scaling modes.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Use fallthrough pseudo-keyword

Replace the existing /* fall through */ comments and its variants with
the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary
fall-through markings when it is the case.

[1] https://www.kernel.org/doc/html/latest/process/deprecated.html?highlight=fallthrough#implicit-switch-case-fall-through

Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/vmm/gp100-: fix mapping 2MB sysmem pages

The nvif_object_ioctl() method NVIF_VMM_V0_PFNMAP wasn't correctly
setting the hardware specific GPU page table entries for 2MB sized
pages. Fix this by adding functions to set and clear PD0 GPU page
table entries.

Signed-off-by: Ralph Campbell <rcampbell@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/mmu: make nvkm_vmm_ctor() static

The function nvkm_vmm_ctor() is not called outside of the file defining
it, so make it static.

Signed-off-by: Ralph Campbell <rcampbell@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: fix reference count leak in nouveau_debugfs_strap_peek

nouveau_debugfs_strap_peek() calls pm_runtime_get_sync() that
increments the reference count. In case of failure, decrement the
ref count before returning the error.

Signed-off-by: Aditya Pakki <pakki001@umn.edu>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Fix reference count leak in nouveau_connector_detect

nouveau_connector_detect() calls pm_runtime_get_sync and in turn
increments the reference count. In case of failure, decrement the
ref count before returning the error.

Signed-off-by: Aditya Pakki <pakki001@umn.edu>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: fix reference count leak in nv50_disp_atomic_commit

nv50_disp_atomic_commit() calls calls pm_runtime_get_sync and in turn
increments the reference count. In case of failure, decrement the
ref count before returning the error.

Signed-off-by: Aditya Pakki <pakki001@umn.edu>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: fix multiple instances of reference count leaks

On calling pm_runtime_get_sync() the reference count of the device
is incremented. In case of failure, decrement the
ref count before returning the error.

Signed-off-by: Aditya Pakki <pakki001@umn.edu>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/drm/noveau: fix reference count leak in nouveau_fbcon_open

nouveau_fbcon_open() calls calls pm_runtime_get_sync() that
increments the reference count. In case of failure, decrement the
ref count before returning the error.

Signed-off-by: Aditya Pakki <pakki001@umn.edu>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/sec2/gp102: allow module to load when LSFW is missing

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gm200-: explicitly handle nofw

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/pmu/gm200-: explicitly handle nofw

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/pmu/gm20x: don't pretend we support loading with our custom FW

It technically loads, and runs, but is ultimately pointless outside of
a very narrow window (fanless systems where one wants to attempt using
the, broken for a lot of gm20x, memory reclocking code).

It's also potentially dangerous to override the VBIOS-provided "Pre-OS"
PMU, which would be responsible for fan control otherwise.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/acr: store a mask of LS falcons the controlling LSFW can bootstrap

This will prevent some pain with broken firmware trees, as under some
circumstances the HSFW can fail and leave the GPU in a state we don't
know how to recover from.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/acr: store a mask of LS falcons the HSFW can bootstrap

This will prevent reloading of HS FW where it's pointless, and bypass
hitting some timeouts.

Not a situation one should generally hit, but can occur with a messed
up firmware installation.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/acr: allow module to load when HSFW(s) are missing

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/acr: refuse to load LSFW if HSFW is missing

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/core: drop error message when no compatible FW found

This is less than useful with some subdevs having _nofw variants in their
FWIF lists - it's cleaner to handle them all in the same way.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/mmu/gp100-: enable mmu invalidate depth optimisation

This causes us to invalidate MMU only at the level we made modifications -
ie: if we've only modified PTEs, there's no need to have MMU dump the PDs
it's fetched into L2.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/nvfw: firmware structures should begin with nvfw_

Rename all structures that are used directly by firmware to have a nvfw_
prefix.

This makes it easier to identify structures that have a fixed, specific
layout.  A future patch will define several more such structures, so it's
important to be consistent now.

Signed-off-by: Timur Tabi <ttabi@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/tmr: fix nvkm_usec/nvkm_msec definitions

nvkm_timer_wait_init() takes a u64 as a duration parameter, but the
expression "(m) * 1000" will be promoted only to a 32-bit integer,
if 'm' is also an integer.  Changing the 1000 to 1000ULL ensures that
the expression will be 64 bits.

This change currently has no effect as there are no callers of
nvkm_msec() that exceed 2000ms.

Signed-off-by: Timur Tabi <ttabi@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>