Merge tag 'for-5.17/io_uring-2022-01-11' of git://git.kernel.dk/linux-block

Pull io_uring updates from Jens Axboe:

 - Support for prioritized work completions (Hao)

 - Simplification of reissue (Pavel)

 - Add support for CQE skip (Pavel)

 - Memory leak fix going to 5.15-stable (Pavel)

 - Re-write of internal poll. This both cleans up that code, and gets us
   ready to fix the POLLFREE issue (Pavel)

 - Various cleanups (GuoYong, Pavel, Hao)

* tag 'for-5.17/io_uring-2022-01-11' of git://git.kernel.dk/linux-block: (31 commits)
  io_uring: fix not released cached task refs
  io_uring: remove redundant tab space
  io_uring: remove unused function parameter
  io_uring: use completion batching for poll rem/upd
  io_uring: single shot poll removal optimisation
  io_uring: poll rework
  io_uring: kill poll linking optimisation
  io_uring: move common poll bits
  io_uring: refactor poll update
  io_uring: remove double poll on poll update
  io_uring: code clean for some ctx usage
  io_uring: batch completion in prior_task_list
  io_uring: split io_req_complete_post() and add a helper
  io_uring: add helper for task work execution code
  io_uring: add a priority tw list for irq completion work
  io-wq: add helper to merge two wq_lists
  io_uring: reuse io_req_task_complete for timeouts
  io_uring: tweak iopoll CQE_SKIP event counting
  io_uring: simplify selected buf handling
  io_uring: move up io_put_kbuf() and io_put_rw_kbuf()
  ...

diff --cc fs/io_uring.c
index fb2a0cb4aaf835d872b9276534c714d3a789745b,684d77c179a0f2e484d6f5dfeb1a159fa7051dc4..a4f20b8c74a4ffc28569e43c294f2228bfe922f3
--- 1/fs/io_uring.c
--- 2/fs/io_uring.c
+++ b/fs/io_uring.c
@@@ -5748,8 -5782,10 +5826,10 @@@ static __cold bool io_poll_remove_all(s
  
                list = &ctx->cancel_hash[i];
                hlist_for_each_entry_safe(req, tmp, list, hash_node) {
-                       if (io_match_task_safe(req, tsk, cancel_all))
-                               posted += io_poll_remove_one(req);
 -                      if (io_match_task(req, tsk, cancel_all)) {
++                      if (io_match_task_safe(req, tsk, cancel_all)) {
+                               io_poll_cancel_req(req);
+                               found = true;
+                       }
                }
        }
        spin_unlock(&ctx->completion_lock);
@@@ -9814,21 -9819,9 +9854,9 @@@ static s64 tctx_inflight(struct io_urin
        return percpu_counter_sum(&tctx->inflight);
  }
  
- static __cold void io_uring_drop_tctx_refs(struct task_struct *task)
- {
-       struct io_uring_task *tctx = task->io_uring;
-       unsigned int refs = tctx->cached_refs;
- 
-       if (refs) {
-               tctx->cached_refs = 0;
-               percpu_counter_sub(&tctx->inflight, refs);
-               put_task_struct_many(task, refs);
-       }
- }
- 
  /*
   * Find any io_uring ctx that this task has registered or done IO on, and cancel
 - * requests. @sqd should be not-null IIF it's an SQPOLL thread cancellation.
 + * requests. @sqd should be not-null IFF it's an SQPOLL thread cancellation.
   */
  static __cold void io_uring_cancel_generic(bool cancel_all,
                                           struct io_sq_data *sqd)