IB/core: Add ordered workqueue for RoCE GID management

Currently the RoCE GID management uses the ib_wq to do add and delete new GIDs
according to the netdev events.

The ib_wq isn't an ordered workqueue and thus two work elements can be executed
concurrently which will result in unexpected behavior and inconsistency of the
GIDs cache content.

Example:
ifconfig eth1 11.11.11.11/16 up

This command will invoke the following netdev events in the following order:
1. NETDEV_UP
2. NETDEV_DOWN
3. NETDEV_UP

If (2) and (3) will be executed concurrently or in reverse order, instead of
having a new GID with 11.11.11.11 IP, we will end up without any new GIDs.

Signed-off-by: Majd Dibbiny <majd@mellanox.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com>
Signed-off-by: Doug Ledford <dledford@redhat.com>

diff --git a/drivers/infiniband/core/roce_gid_mgmt.c b/drivers/infiniband/core/roce_gid_mgmt.c
index db958d3207efc9e13aa1ae2d05fbadce2732c0cd..94a9eefb3cfcad894d13b400a9d3a2281773b5b0 100644 (file)
--- a/drivers/infiniband/core/roce_gid_mgmt.c
+++ b/drivers/infiniband/core/roce_gid_mgmt.c
@@ -42,6 +42,8 @@
 #include <rdma/ib_cache.h>
 #include <rdma/ib_addr.h>
 
+static struct workqueue_struct *gid_cache_wq;
+
 enum gid_op_type {
        GID_DEL = 0,
        GID_ADD
@@ -560,7 +562,7 @@ static int netdevice_queue_work(struct netdev_event_work_cmd *cmds,
        }
        INIT_WORK(&ndev_work->work, netdevice_event_work_handler);
 
-       queue_work(ib_wq, &ndev_work->work);
+       queue_work(gid_cache_wq, &ndev_work->work);
 
        return NOTIFY_DONE;
 }
@@ -693,7 +695,7 @@ static int addr_event(struct notifier_block *this, unsigned long event,
        dev_hold(ndev);
        work->gid_attr.ndev   = ndev;
 
-       queue_work(ib_wq, &work->work);
+       queue_work(gid_cache_wq, &work->work);
 
        return NOTIFY_DONE;
 }
@@ -740,6 +742,10 @@ static struct notifier_block nb_inet6addr = {
 
 int __init roce_gid_mgmt_init(void)
 {
+       gid_cache_wq = alloc_ordered_workqueue("gid-cache-wq", 0);
+       if (!gid_cache_wq)
+               return -ENOMEM;
+
        register_inetaddr_notifier(&nb_inetaddr);
        if (IS_ENABLED(CONFIG_IPV6))
                register_inet6addr_notifier(&nb_inet6addr);
@@ -764,4 +770,5 @@ void __exit roce_gid_mgmt_cleanup(void)
         * ib-core is removed, all physical devices have been removed,
         * so no issue with remaining hardware contexts.
         */
+       destroy_workqueue(gid_cache_wq);
 }