]> www.infradead.org Git - users/jedix/linux-maple.git/commit
mm: memcontrol: make lruvec lock safe when LRU pages are reparented
authorMuchun Song <songmuchun@bytedance.com>
Tue, 21 Jun 2022 12:56:51 +0000 (20:56 +0800)
committerLiam R. Howlett <Liam.Howlett@oracle.com>
Wed, 20 Jul 2022 00:15:10 +0000 (20:15 -0400)
commita4d1f3b68e7fe6ecb1a33d38d1410fda734a7afc
tree6b70f6b78510b812952158ea1fdf675287ea80c4
parent88f261c8a9730faf5085c0d1338b3c4ee2ccce48
mm: memcontrol: make lruvec lock safe when LRU pages are reparented

The diagram below shows how to make the folio lruvec lock safe when LRU
pages are reparented.

folio_lruvec_lock(folio)
rcu_read_lock();
    retry:
lruvec = folio_lruvec(folio);

        // The folio is reparented at this time.
        spin_lock(&lruvec->lru_lock);

        if (unlikely(lruvec_memcg(lruvec) != folio_memcg(folio)))
            // Acquired the wrong lruvec lock and need to retry.
            // Because this folio is on the parent memcg lruvec list.
            spin_unlock(&lruvec->lru_lock);
    goto retry;

        // If we reach here, it means that folio_memcg(folio) is stable.

memcg_reparent_objcgs(memcg)
    // lruvec belongs to memcg and lruvec_parent belongs to parent memcg.
    spin_lock(&lruvec->lru_lock);
    spin_lock(&lruvec_parent->lru_lock);

    // Move all the pages from the lruvec list to the parent lruvec list.

    spin_unlock(&lruvec_parent->lru_lock);
    spin_unlock(&lruvec->lru_lock);

After we acquire the lruvec lock, we need to check whether the folio is
reparented.  If so, we need to reacquire the new lruvec lock.  On the
routine of the LRU pages reparenting, we will also acquire the lruvec lock
(will be implemented in the later patch).  So folio_memcg() cannot be
changed when we hold the lruvec lock.

Since lruvec_memcg(lruvec) is always equal to folio_memcg(folio) after we
hold the lruvec lock, lruvec_memcg_debug() check is pointless.  So remove
it.

This is a preparation for reparenting the LRU pages.

Link: https://lkml.kernel.org/r/20220621125658.64935-5-songmuchun@bytedance.com
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Michal Koutný <mkoutny@suse.com>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Shakeel Butt <shakeelb@google.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Xiongchun Duan <duanxiongchun@bytedance.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
include/linux/memcontrol.h
mm/compaction.c
mm/memcontrol.c
mm/swap.c