bpf: Add more WARN_ON_ONCE checks for mismatched alloc and free

There are two possible mismatched alloc and free cases in BPF memory
allocator:

1) allocate from cache X but free by cache Y with a different unit_size
2) allocate from per-cpu cache but free by kmalloc cache or vice versa

So add more WARN_ON_ONCE checks in free_bulk() and __free_by_rcu() to
spot these mismatched alloc and free early.

Signed-off-by: Hou Tao <houtao1@huawei.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20231021014959.3563841-1-houtao@huaweicloud.com

diff --git a/kernel/bpf/memalloc.c b/kernel/bpf/memalloc.c
index 5308e386380af1098a5e5bbe4fefeaef6eb0593d..63b909d277d47925c70215adbbc4b11b4e5ad558 100644 (file)
--- a/kernel/bpf/memalloc.c
+++ b/kernel/bpf/memalloc.c
@@ -340,6 +340,7 @@ static void free_bulk(struct bpf_mem_cache *c)
        int cnt;
 
        WARN_ON_ONCE(tgt->unit_size != c->unit_size);
+       WARN_ON_ONCE(tgt->percpu_size != c->percpu_size);
 
        do {
                inc_active(c, &flags);
@@ -365,6 +366,9 @@ static void __free_by_rcu(struct rcu_head *head)
        struct bpf_mem_cache *tgt = c->tgt;
        struct llist_node *llnode;
 
+       WARN_ON_ONCE(tgt->unit_size != c->unit_size);
+       WARN_ON_ONCE(tgt->percpu_size != c->percpu_size);
+
        llnode = llist_del_all(&c->waiting_for_gp);
        if (!llnode)
                goto out;