ptdesc: Remove references to folios from __pagetable_ctor() and pagetable_dtor()
In preparation for splitting struct ptdesc from struct page and struct
folio, remove mentions of struct folio from these functions. Introduce
ptdesc_nr_pages() to avoid using lruvec_stat_add/sub_folio()
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Use the new memdesc_flags_t type to show that these are the same
bits as page/folio/slab and thesefore have the zone/node/section
information in them. Remove a use of ptdesc_folio() by converting
pagetable_is_reserved() to use test_bit() directly.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
mm: Scale mlock_count to avoid using the bottom bits
We intend to use these bottom two bits to distinguish pages which have
a memdesc. Multiplying mlock_count by four achieves this. We don't
need to worry about overflow as the number of mlocks is limited by the
number of VMAs we can allocate.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
We already use page->private for storing the order of a page while it's
in the buddy allocator system; extend that to also storing the order
while it's in the pcp_llist.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Cc: Alexei Starovoitov <ast@kernel.org>
Matthew Wilcox (Oracle) [Fri, 29 Aug 2025 03:31:12 +0000 (23:31 -0400)]
mm: Remove redundant test in validate_page_before_insert()
The page_has_type() call would have included slab since commit 46df8e73a4a3 and now we don't even get that far because slab
pages have a zero refcount since commit 9aec2fb0fd5e.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Matthew Wilcox (Oracle) [Thu, 28 Aug 2025 19:35:08 +0000 (15:35 -0400)]
kasan: Remove references to folio in __kasan_mempool_poison_object()
In preparation for splitting struct slab from struct page and struct
folio, remove mentions of struct folio from this function. We can
discard the comment as using PageLargeKmalloc() rather than
!folio_test_slab() makes it obvious.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Matthew Wilcox (Oracle) [Thu, 28 Aug 2025 19:30:48 +0000 (15:30 -0400)]
memcg: Convert mem_cgroup_from_obj_folio() to mem_cgroup_from_obj_slab()
In preparation for splitting struct slab from struct page and struct
folio, convert the pointer to a slab rather than a folio. This means
we can end up passing a NULL slab pointer to mem_cgroup_from_obj_slab()
if the pointer is not to a page allocated to slab, and we handle that
appropriately by returning NULL.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Matthew Wilcox (Oracle) [Thu, 28 Aug 2025 19:20:12 +0000 (15:20 -0400)]
usercopy: Remove folio references from check_heap_object()
Because the pointer being checked may not lie within the first PAGE_SIZE
bytes of the object, we have to mark all pages as being LargeKmalloc.
We could use virt_to_head_page() instead, but that would pessimize
slab objects. Once we move to memdescs properly, we'll tag each page
as being LargeKmalloc anyway, so this is more in keeping with how code
will be written in the future.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Matthew Wilcox (Oracle) [Fri, 29 Aug 2025 15:10:23 +0000 (11:10 -0400)]
slab: Remove folio references from kfree_rcu_sheaf()
In preparation for splitting struct slab from struct page and struct
folio, remove mentions of struct folio from this function. Since
we don't need to handle large kmalloc objects specially here, we
can just use virt_to_slab().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Matthew Wilcox (Oracle) [Thu, 28 Aug 2025 16:49:47 +0000 (12:49 -0400)]
slab: Remove folio references from build_detached_freelist()
As with memcg_slab_post_charge(), we save a call to compound_head() for
large kmallocs. This has a slight change of behaviour in that non-slab,
non-kmalloc pointers will now cause a NULL pointer dereference rather
than a warning. We could add that back if really needed.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Matthew Wilcox (Oracle) [Thu, 28 Aug 2025 16:49:47 +0000 (12:49 -0400)]
slab: Remove folio references from __do_krealloc()
As with memcg_slab_post_charge(), we save a call to compound_head()
for large kmallocs. This has a slight change of behaviour in that
non-vmalloc, non-slab, non-kmalloc pointers will now cause a NULL
pointer dereference rather than a warning. We could add that back if
really needed.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Matthew Wilcox (Oracle) [Thu, 28 Aug 2025 16:49:47 +0000 (12:49 -0400)]
slab: Remove folio references from kvfree()
As with memcg_slab_post_charge(), we save a call to compound_head()
for large kmallocs. This has a slight change of behaviour in that
non-vmalloc, non-slab, non-kmalloc pointers will now cause a NULL
pointer dereference rather than a warning. We could add that back if
really needed.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Matthew Wilcox (Oracle) [Thu, 28 Aug 2025 16:49:47 +0000 (12:49 -0400)]
slab: Remove folio references from kvfree_rcu_cb()
As with memcg_slab_post_charge(), we save a call to compound_head()
for large kmallocs. This has a slight change of behaviour in that
non-vmalloc, non-slab, non-kmalloc pointers will now cause a NULL
pointer dereference rather than a warning. We could add that back if
really needed.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Matthew Wilcox (Oracle) [Thu, 28 Aug 2025 15:46:23 +0000 (11:46 -0400)]
slab: Remove folio references in memcg_slab_post_charge()
This allows us to skip the compound_head() call for large kmalloc
objects as the virt_to_page() call will always give us the head page
for the large kmalloc case.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Matthew Wilcox (Oracle) [Mon, 25 Aug 2025 20:48:06 +0000 (16:48 -0400)]
slab: Remove folio references from __ksize()
In the future, we will separate slab, folio and page from each other
and calling virt_to_folio() on an address allocated from slab will
return NULL. Delay the conversion from struct page to struct slab
until we know we're not dealing with a large kmalloc allocation.
This deprecates calling ksize() on memory allocated by alloc_pages().
Today it becomes a warning and support will be removed entirely in
the future.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
In order to separate slabs from folios, we need to convert from any page
in a slab to the slab directly without going through a page to folio
conversion first. page_slab() is a little different from other memdesc
converters we have in that it will return NULL if the page is not part
of a slab. This will be the normal style for memdesc converters in
the future.
kfence was the only user of page_slab(), so adjust it to the new way
of working. It will need to be touched again when we separate slab
from page.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Matthew Wilcox (Oracle) [Mon, 25 Aug 2025 20:47:02 +0000 (16:47 -0400)]
mm: Constify compound_order() and page_size()
These functions do not modify their arguments. Telling the compiler
this may improve code generation, and allows us to pass const arguments
from other functions.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Matthew Wilcox (Oracle) [Mon, 18 Aug 2025 20:56:02 +0000 (16:56 -0400)]
slab: Remove dead code in free_consistency_checks()
We already know that slab is a valid slab as that's checked by the
caller. In the future, we won't be able to get to a slab pointer
from a non-slab page.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Matthew Wilcox (Oracle) [Wed, 6 Aug 2025 17:01:11 +0000 (13:01 -0400)]
slab: Remove unnecessary test from alloc_debug_processing()
In alloc_debug_processing() we know that the slab pointer passed in
really is a pointer to a slab. Both callers modify members of the slab
data structure. So we can remove this check.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
projects / users / willy / pagecache.git / log