fs: bump inode and dentry counters to long

This series reworks our current object cache shrinking infrastructure in
two main ways:

 * Noticing that a lot of users copy and paste their own version of LRU
   lists for objects, we put some effort in providing a generic version.
   It is modeled after the filesystem users: dentries, inodes, and xfs
   (for various tasks), but we expect that other users could benefit in
   the near future with little or no modification.  Let us know if you
   have any issues.

 * The underlying list_lru being proposed automatically and
   transparently keeps the elements in per-node lists, and is able to
   manipulate the node lists individually.  Given this infrastructure, we
   are able to modify the up-to-now hammer called shrink_slab to proceed
   with node-reclaim instead of always searching memory from all over like
   it has been doing.

Per-node lru lists are also expected to lead to less contention in the lru
locks on multi-node scans, since we are now no longer fighting for a
global lock.  The locks usually disappear from the profilers with this
change.

Although we have no official benchmarks for this version - be our guest to
independently evaluate this - earlier versions of this series were
performance tested (details at
http://permalink.gmane.org/gmane.linux.kernel.mm/100537) yielding no
visible performance regressions while yielding a better qualitative
behavior in NUMA machines.

With this infrastructure in place, we can use the list_lru entry point to
provide memcg isolation and per-memcg targeted reclaim.  Historically,
those two pieces of work have been posted together.  This version presents
only the infrastructure work, deferring the memcg work for a later time,
so we can focus on getting this part tested.  You can see more about the
history of such work at http://lwn.net/Articles/552769/

Dave Chinner (18):
  dcache: convert dentry_stat.nr_unused to per-cpu counters
  dentry: move to per-sb LRU locks
  dcache: remove dentries from LRU before putting on dispose list
  mm: new shrinker API
  shrinker: convert superblock shrinkers to new API
  list: add a new LRU list type
  inode: convert inode lru list to generic lru list code.
  dcache: convert to use new lru list infrastructure
  list_lru: per-node list infrastructure
  shrinker: add node awareness
  fs: convert inode and dentry shrinking to be node aware
  xfs: convert buftarg LRU to generic code
  xfs: rework buffer dispose list tracking
  xfs: convert dquot cache lru to list_lru
  fs: convert fs shrinkers to new scan/count API
  drivers: convert shrinkers to new count/scan API
  shrinker: convert remaining shrinkers to count/scan API
  shrinker: Kill old ->shrink API.

Glauber Costa (7):
  fs: bump inode and dentry counters to long
  super: fix calculation of shrinkable objects for small numbers
  list_lru: per-node API
  vmscan: per-node deferred work
  i915: bail out earlier when shrinker cannot acquire mutex
  hugepage: convert huge zero page shrinker to new shrinker API
  list_lru: dynamically adjust node arrays

This patch:

There are situations in very large machines in which we can have a large
quantity of dirty inodes, unused dentries, etc.  This is particularly true
when umounting a filesystem, where eventually since every live object will
eventually be discarded.

Dave Chinner reported a problem with this while experimenting with the
shrinker revamp patchset.  So we believe it is time for a change.  This
patch just moves int to longs.  Machines where it matters should have a
big long anyway.

Signed-off-by: Glauber Costa <glommer@openvz.org>
Cc: Dave Chinner <dchinner@redhat.com>
Cc: "Theodore Ts'o" <tytso@mit.edu>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Cc: Arve Hjønnevåg <arve@android.com>
Cc: Carlos Maiolino <cmaiolino@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Chuck Lever <chuck.lever@oracle.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Dave Chinner <dchinner@redhat.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Gleb Natapov <gleb@redhat.com>
Cc: Greg Thelen <gthelen@google.com>
Cc: J. Bruce Fields <bfields@redhat.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Jerome Glisse <jglisse@redhat.com>
Cc: John Stultz <john.stultz@linaro.org>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: Kent Overstreet <koverstreet@google.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Marcelo Tosatti <mtosatti@redhat.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Steven Whitehouse <swhiteho@redhat.com>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
Cc: Trond Myklebust <Trond.Myklebust@netapp.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>

diff --git a/fs/dcache.c b/fs/dcache.c
index 4d9df3c940e697603ca41aec60f7fc97c3361ada..6ef1c2e1bbc4a7a93a07a7a1f647ceca664b0778 100644 (file)
--- a/fs/dcache.c
+++ b/fs/dcache.c
@@ -146,13 +146,13 @@ struct dentry_stat_t dentry_stat = {
        .age_limit = 45,
 };
 
-static DEFINE_PER_CPU(unsigned int, nr_dentry);
+static DEFINE_PER_CPU(long, nr_dentry);
 
 #if defined(CONFIG_SYSCTL) && defined(CONFIG_PROC_FS)
-static int get_nr_dentry(void)
+static long get_nr_dentry(void)
 {
        int i;
-       int sum = 0;
+       long sum = 0;
        for_each_possible_cpu(i)
                sum += per_cpu(nr_dentry, i);
        return sum < 0 ? 0 : sum;
@@ -162,7 +162,7 @@ int proc_nr_dentry(ctl_table *table, int write, void __user *buffer,
                   size_t *lenp, loff_t *ppos)
 {
        dentry_stat.nr_dentry = get_nr_dentry();
-       return proc_dointvec(table, write, buffer, lenp, ppos);
+       return proc_doulongvec_minmax(table, write, buffer, lenp, ppos);
 }
 #endif
 

diff --git a/fs/inode.c b/fs/inode.c
index 93a0625b46e490252c03dab43310919763a8eca9..2a3c37ea823d5579175d0d0387112d47b264885a 100644 (file)
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -70,33 +70,33 @@ EXPORT_SYMBOL(empty_aops);
  */
 struct inodes_stat_t inodes_stat;
 
-static DEFINE_PER_CPU(unsigned int, nr_inodes);
-static DEFINE_PER_CPU(unsigned int, nr_unused);
+static DEFINE_PER_CPU(unsigned long, nr_inodes);
+static DEFINE_PER_CPU(unsigned long, nr_unused);
 
 static struct kmem_cache *inode_cachep __read_mostly;
 
-static int get_nr_inodes(void)
+static long get_nr_inodes(void)
 {
        int i;
-       int sum = 0;
+       long sum = 0;
        for_each_possible_cpu(i)
                sum += per_cpu(nr_inodes, i);
        return sum < 0 ? 0 : sum;
 }
 
-static inline int get_nr_inodes_unused(void)
+static inline long get_nr_inodes_unused(void)
 {
        int i;
-       int sum = 0;
+       long sum = 0;
        for_each_possible_cpu(i)
                sum += per_cpu(nr_unused, i);
        return sum < 0 ? 0 : sum;
 }
 
-int get_nr_dirty_inodes(void)
+long get_nr_dirty_inodes(void)
 {
        /* not actually dirty inodes, but a wild approximation */
-       int nr_dirty = get_nr_inodes() - get_nr_inodes_unused();
+       long nr_dirty = get_nr_inodes() - get_nr_inodes_unused();
        return nr_dirty > 0 ? nr_dirty : 0;
 }
 
@@ -109,7 +109,7 @@ int proc_nr_inodes(ctl_table *table, int write,
 {
        inodes_stat.nr_inodes = get_nr_inodes();
        inodes_stat.nr_unused = get_nr_inodes_unused();
-       return proc_dointvec(table, write, buffer, lenp, ppos);
+       return proc_doulongvec_minmax(table, write, buffer, lenp, ppos);
 }
 #endif
 

diff --git a/fs/internal.h b/fs/internal.h
index 2be46ea5dd0bf27af5002057f1b20399afee6e69..b6495659d6e84dd9d3734887404ce08e690f0c4e 100644 (file)
--- a/fs/internal.h
+++ b/fs/internal.h
@@ -121,7 +121,7 @@ extern void inode_add_lru(struct inode *inode);
  */
 extern void inode_wb_list_del(struct inode *inode);
 
-extern int get_nr_dirty_inodes(void);
+extern long get_nr_dirty_inodes(void);
 extern void evict_inodes(struct super_block *);
 extern int invalidate_inodes(struct super_block *, bool);
 

diff --git a/include/linux/dcache.h b/include/linux/dcache.h
index feaa8d88eef7171afb422208ceeb2ab7ad99fbcc..844a1ef387e46da67ba4575b1603c1a7a17ac95a 100644 (file)
--- a/include/linux/dcache.h
+++ b/include/linux/dcache.h
@@ -55,11 +55,11 @@ struct qstr {
 #define hashlen_len(hashlen)  ((u32)((hashlen) >> 32))
 
 struct dentry_stat_t {
-       int nr_dentry;
-       int nr_unused;
-       int age_limit;          /* age in seconds */
-       int want_pages;         /* pages requested by system */
-       int dummy[2];
+       long nr_dentry;
+       long nr_unused;
+       long age_limit;          /* age in seconds */
+       long want_pages;         /* pages requested by system */
+       long dummy[2];
 };
 extern struct dentry_stat_t dentry_stat;
 

diff --git a/include/linux/fs.h b/include/linux/fs.h
index 49e71b0f0e9f01823db69d455efd4a48b2203d50..3b3edac75df285d7a1fde4f3078d24d93f7366a6 100644 (file)
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1271,12 +1271,12 @@ struct super_block {
        struct list_head        s_mounts;       /* list of mounts; _not_ for fs use */
        /* s_dentry_lru, s_nr_dentry_unused protected by dcache.c lru locks */
        struct list_head        s_dentry_lru;   /* unused dentry lru */
-       int                     s_nr_dentry_unused;     /* # of dentry on lru */
+       long                    s_nr_dentry_unused;     /* # of dentry on lru */
 
        /* s_inode_lru_lock protects s_inode_lru and s_nr_inodes_unused */
        spinlock_t              s_inode_lru_lock ____cacheline_aligned_in_smp;
        struct list_head        s_inode_lru;            /* unused inode lru */
-       int                     s_nr_inodes_unused;     /* # of inodes on lru */
+       long                    s_nr_inodes_unused;     /* # of inodes on lru */
 
        struct block_device     *s_bdev;
        struct backing_dev_info *s_bdi;

diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h
index a4ed56cf0eac5f1e5c1b90edf2a1d3b6093d9e02..6c28b61bb69041b166b09698a05d8c0619122c1c 100644 (file)
--- a/include/uapi/linux/fs.h
+++ b/include/uapi/linux/fs.h
@@ -49,9 +49,9 @@ struct files_stat_struct {
 };
 
 struct inodes_stat_t {
-       int nr_inodes;
-       int nr_unused;
-       int dummy[5];           /* padding for sysctl ABI compatibility */
+       long nr_inodes;
+       long nr_unused;
+       long dummy[5];          /* padding for sysctl ABI compatibility */
 };
 
 

diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index 07f6fc468e1732734e7fcf5f73982a93670e4194..7822cd88a95c4a3673267b1725dec9fcfbdc4587 100644 (file)
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -1471,14 +1471,14 @@ static struct ctl_table fs_table[] = {
        {
                .procname       = "inode-nr",
                .data           = &inodes_stat,
-               .maxlen         = 2*sizeof(int),
+               .maxlen         = 2*sizeof(long),
                .mode           = 0444,
                .proc_handler   = proc_nr_inodes,
        },
        {
                .procname       = "inode-state",
                .data           = &inodes_stat,
-               .maxlen         = 7*sizeof(int),
+               .maxlen         = 7*sizeof(long),
                .mode           = 0444,
                .proc_handler   = proc_nr_inodes,
        },
@@ -1508,7 +1508,7 @@ static struct ctl_table fs_table[] = {
        {
                .procname       = "dentry-state",
                .data           = &dentry_stat,
-               .maxlen         = 6*sizeof(int),
+               .maxlen         = 6*sizeof(long),
                .mode           = 0444,
                .proc_handler   = proc_nr_dentry,
        },