ib/rds: runtime debuggability enhancement

ib/rds: runtime debuggability enhancement

This patch provides the ability to dynamically turn on or off various
types of debug/diag prints inside the RDS module.

The run-time debug prints are controlled by a rds module parameter,
rds_rt_debug_bitmap.

Here is the definition for different bits. We have implemented feature
related bits, such as Connection Management, Active Bonding, Error prints,
Send, Recv.

in net/rds/rds_rt_debug.h
...
enum {
        /* bit 0 ~ 19 are feature related bits */
        RDS_RTD_ERR                     = 1 << 0,       /* 0x1    */
        RDS_RTD_ERR_EXT                 = 1 << 1,       /* 0x2    */

        RDS_RTD_CM                      = 1 << 3,       /* 0x8    */
        RDS_RTD_CM_EXT                  = 1 << 4,       /* 0x10   */
        RDS_RTD_CM_EXT_P                = 1 << 5,       /* 0x20   */

        RDS_RTD_ACT_BND                 = 1 << 7,       /* 0x80   */
        RDS_RTD_ACT_BND_EXT             = 1 << 8,       /* 0x100  */

        RDS_RTD_RCV                     = 1 << 11,      /* 0x800  */
        RDS_RTD_RCV_EXT                 = 1 << 12,      /* 0x1000 */

        RDS_RTD_SND                     = 1 << 14,      /* 0x4000 */
        RDS_RTD_SND_EXT                 = 1 << 15,      /* 0x8000 */
...

In general, *EXTRA bits mean that you will get extra information but
possible flood prints as well. But every bit can be controlled by users
so users can decide how much information they want to see/collect. The
current embedded printk level used for this patch is KERN_INFO. Most
likely all the msgs will only go to /var/log/messages without showing up
on console if we use the default settings for /proc/sys/kernel/printk and
/etc/rsyslog.conf in ol6 environment.

E.g if we want to turn on RDS_RTD_ERR and RDS_RTD_CM bits. What we can
do is

echo 0x9 > /sys/module/rds/parameters/rds_rt_debug_bitmap

To turn on RDS_RTD_ERR(0x1), RDS_RTD_CM(0x8), and RDS_RTD_RCV(0x800) bits

echo 0x809 > /sys/module/rds/parameters/rds_rt_debug_bitmap

Performance penalty: with all the debug flag bits set to 0, there
should be no porformance impact in a kernel with this patch.

Orabug 21314268

Reviewed-by: Ajaykumar Hotchandani <ajaykumar.hotchandani@oracle.com>
Reviewed-by: Shamir Rabinovitch <shamir.rabinovitch@oracle.com>
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com>
Signed-off-by: Qing Huang <qing.huang@oracle.com>
Signed-off-by: Mukesh Kacker <mukesh.kacker@oracle.com>