block: use largest_zero_folio in __blkdev_issue_zero_pages()

Use largest_zero_folio() in __blkdev_issue_zero_pages().  On systems with
CONFIG_PERSISTENT_HUGE_ZERO_FOLIO enabled, we will end up sending larger
bvecs instead of multiple small ones.

Noticed a 4% increase in performance on a commercial NVMe SSD which does
not support OP_WRITE_ZEROES.  The device's MDTS was 128K.  The performance
gains might be bigger if the device supports bigger MDTS.

Link: https://lkml.kernel.org/r/20250811084113.647267-6-kernel@pankajraghav.com
Signed-off-by: Pankaj Raghav <p.raghav@samsung.com>
Acked-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Cc: Kiryl Shutsemau <kirill@shutemov.name>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: "Darrick J. Wong" <djwong@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Luis Chamberalin <mcgrof@kernel.org>
Cc: Mariano Pache <npache@redhat.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: "Ritesh Harjani (IBM)" <ritesh.list@gmail.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Thomas Gleinxer <tglx@linutronix.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/block/blk-lib.c b/block/blk-lib.c
index 4c9f20a689f7bbea999eaa614f1e2b1c83f5d984..3030a772d3aa01c988fa901ae4ae8faea509f8a2 100644 (file)
--- a/block/blk-lib.c
+++ b/block/blk-lib.c
@@ -196,6 +196,8 @@ static void __blkdev_issue_zero_pages(struct block_device *bdev,
                sector_t sector, sector_t nr_sects, gfp_t gfp_mask,
                struct bio **biop, unsigned int flags)
 {
+       struct folio *zero_folio = largest_zero_folio();
+
        while (nr_sects) {
                unsigned int nr_vecs = __blkdev_sectors_to_bio_pages(nr_sects);
                struct bio *bio;
@@ -208,15 +210,14 @@ static void __blkdev_issue_zero_pages(struct block_device *bdev,
                        break;
 
                do {
-                       unsigned int len, added;
+                       unsigned int len;
 
-                       len = min_t(sector_t,
-                               PAGE_SIZE, nr_sects << SECTOR_SHIFT);
-                       added = bio_add_page(bio, ZERO_PAGE(0), len, 0);
-                       if (added < len)
+                       len = min_t(sector_t, folio_size(zero_folio),
+                                   nr_sects << SECTOR_SHIFT);
+                       if (!bio_add_folio(bio, zero_folio, len, 0))
                                break;
-                       nr_sects -= added >> SECTOR_SHIFT;
-                       sector += added >> SECTOR_SHIFT;
+                       nr_sects -= len >> SECTOR_SHIFT;
+                       sector += len >> SECTOR_SHIFT;
                } while (nr_sects);
 
                *biop = bio_chain_and_submit(*biop, bio);