RDS: Ensure non-zero SL uses correct path before lane 0 connection is dropped

RDS: Ensure non-zero SL uses correct path before lane 0 connection is dropped

There is an issue with the following scenario:
  * if non-zero lane is going down first with send completion error 12
  * Before lane 0 connection goes down, the peer initiates connection request with
    the non-zero lane
  * This non-zero lane connection request may be using old ARP entries of lane 0

This also fixes race condition between connection establishment and drop for
following scenario:
  * non-zero lane connection dropped
  * non-zero connection is initiated and this time it finds proper route and
    connection request goes through.
  * before non-zero lane connection is established at RDS layer,
    zero lane connection is getting dropped.
  * now this zero-lane connection will drop non-zero lane connection as well
    (with the assumption that non-zero lane did not find proper route).
  * when non-zero lane connection establishment event is received (REP packet),
    we have a race between connection establishment event on one CPU and
    connection drop on other CPU.

Orabug: 19133664

Signed-off-by: Ajaykumar Hotchandani <ajaykumar.hotchandani@oracle.com>
Signed-off-by: Chien-Hua Yen <chien.yen@oracle.com>
Reviewed-by: Bang Nguyen <bang.nguyen@oracle.com>
Signed-off-by: Guangyu Sun <guangyu.sun@oracle.com>
(cherry picked from commit 47d8e78f82872bbb9d709a0743ea2bdb2e9f6cbb)