mm-convert-pg_balloon-to-pg_offline-fix

fix comment text, per David

Cc: David Hildenbrand <david@redhat.com>
Cc: Alexander Duyck <alexander.h.duyck@linux.intel.com>
Cc: Alexey Dobriyan <adobriyan@gmail.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Baoquan He <bhe@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Christian Hansen <chansen3@cisco.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Juergen Gross <jgross@suse.com>
Cc: Julien Freche <jfreche@vmware.com>
Cc: Kairui Song <kasong@redhat.com>
Cc: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Cc: Konstantin Khlebnikov <koct9i@gmail.com>
Cc: "K. Y. Srinivasan" <kys@microsoft.com>
Cc: Len Brown <len.brown@intel.com>
Cc: Lianbo Jiang <lijiang@redhat.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Cc: Miles Chen <miles.chen@mediatek.com>
Cc: Nadav Amit <namit@vmware.com>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: Omar Sandoval <osandov@fb.com>
Cc: Pankaj gupta <pagupta@redhat.com>
Cc: Pavel Machek <pavel@ucw.cz>
Cc: Pavel Tatashin <pasha.tatashin@oracle.com>
Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Stephen Hemminger <sthemmin@microsoft.com>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Xavier Deguillard <xdeguillard@vmware.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

mm: convert PG_balloon to PG_offline

PG_balloon was introduced to implement page migration/compaction for pages
inflated in virtio-balloon.  Nowadays, it is only a marker that a page is
part of virtio-balloon and therefore logically offline.

We also want to make use of this flag in other balloon drivers - for
inflated pages or when onlining a section but keeping some pages offline
(e.g.  used right now by XEN and Hyper-V via set_online_page_callback()).

We are going to expose this flag to dump tools like makedumpfile.  But
instead of exposing PG_balloon, let's generalize the concept of marking
pages as logically offline, so it can be reused for other purposes later
on.

Rename PG_balloon to PG_offline.  This is an indicator that the page is
logically offline, the content stale and that it should not be touched
(e.g.  a hypervisor would have to allocate backing storage in order for
the guest to dump an unused page).  We can then e.g.  exclude such pages
from dumps.

We replace and reuse KPF_BALLOON (23), as this shouldn't really harm (and
for now the semantics stay the same).  In following patches, we will make
use of this bit also in other balloon drivers.  While at it, document
PGTABLE.

Link: http://lkml.kernel.org/r/20181119101616.8901-3-david@redhat.com
Signed-off-by: David Hildenbrand <david@redhat.com>
Acked-by: Konstantin Khlebnikov <koct9i@gmail.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Pankaj gupta <pagupta@redhat.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Alexey Dobriyan <adobriyan@gmail.com>
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Cc: Christian Hansen <chansen3@cisco.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Pavel Tatashin <pasha.tatashin@oracle.com>
Cc: Alexander Duyck <alexander.h.duyck@linux.intel.com>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: Miles Chen <miles.chen@mediatek.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Baoquan He <bhe@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Julien Freche <jfreche@vmware.com>
Cc: Kairui Song <kasong@redhat.com>
Cc: "K. Y. Srinivasan" <kys@microsoft.com>
Cc: Len Brown <len.brown@intel.com>
Cc: Lianbo Jiang <lijiang@redhat.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Nadav Amit <namit@vmware.com>
Cc: Omar Sandoval <osandov@fb.com>
Cc: Pavel Machek <pavel@ucw.cz>
Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Stephen Hemminger <sthemmin@microsoft.com>
Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: Xavier Deguillard <xdeguillard@vmware.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

mm: balloon: update comment about isolation/migration/compaction

Patch series "mm/kdump: allow to exclude pages that are logically offline"

Right now, pages inflated as part of a balloon driver will be dumped by
dump tools like makedumpfile.  While XEN is able to check in the crash
kernel whether a certain pfn is actuall backed by memory in the hypervisor
(see xen_oldmem_pfn_is_ram) and optimize this case, dumps of
virtio-balloon, hv-balloon and VMWare balloon inflated memory will
essentially result in zero pages getting allocated by the hypervisor and
the dump getting filled with this data.

The allocation and reading of zero pages can directly be avoided if a
dumping tool could know which pages only contain stale information not to
be dumped.

Also for XEN, calling into the kernel and asking the hypervisor if a pfn
is backed can be avoided if the duming tool would skip such pages right
from the beginning.

Dumping tools have no idea whether a given page is part of a balloon
driver and shall not be dumped.  Esp.  PG_reserved cannot be used for that
purpose as all memory allocated during early boot is also PG_reserved, see
discussion at [1].  So some other way of indication is required and a new
page flag is frowned upon.

We have PG_balloon (MAPCOUNT value), which is essentially unused now.  I
suggest renaming it to something more generic (PG_offline) to mark pages
as logically offline.  This flag can than e.g.  also be used by virtio-mem
in the future to mark subsections as offline.  Or by other code that wants
to put pages logically offline (e.g.  later maybe poisoned pages that
shall no longer be used).

This series converts PG_balloon to PG_offline, allows dumping tools to
query the value to detect such pages and marks pages in the hv-balloon and
XEN balloon properly as PG_offline.  Note that virtio-balloon already set
pages to PG_balloon (and now PG_offline).

Please note that this is also helpful for a problem we were seeing under
Hyper-V: Dumping logically offline memory (pages kept fake offline while
onlining a section via online_page_callback) would under some condicions
result in a kernel panic when dumping them.

As I don't have access to neither XEN nor Hyper-V nor VMWare
installations, this was only tested with the virtio-balloon and pages were
properly skipped when dumping.  I'll also attach the makedumpfile patch to
this series.

[1] https://lkml.org/lkml/2018/7/20/566

This patch (of 8):

b1123ea6d3b3 ("mm: balloon: use general non-lru movable page feature")
reworked balloon handling to make use of the general non-lru movable page
feature.  The big comment block in balloon_compaction.h contains quite
some outdated information.  Let's fix this.

Link: http://lkml.kernel.org/r/20181119101616.8901-2-david@redhat.com
Signed-off-by: David Hildenbrand <david@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Alexander Duyck <alexander.h.duyck@linux.intel.com>
Cc: Alexey Dobriyan <adobriyan@gmail.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Baoquan He <bhe@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Christian Hansen <chansen3@cisco.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Juergen Gross <jgross@suse.com>
Cc: Julien Freche <jfreche@vmware.com>
Cc: Kairui Song <kasong@redhat.com>
Cc: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Cc: Konstantin Khlebnikov <koct9i@gmail.com>
Cc: "K. Y. Srinivasan" <kys@microsoft.com>
Cc: Len Brown <len.brown@intel.com>
Cc: Lianbo Jiang <lijiang@redhat.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Cc: Miles Chen <miles.chen@mediatek.com>
Cc: Nadav Amit <namit@vmware.com>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: Omar Sandoval <osandov@fb.com>
Cc: Pankaj gupta <pagupta@redhat.com>
Cc: Pavel Machek <pavel@ucw.cz>
Cc: Pavel Tatashin <pasha.tatashin@oracle.com>
Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Stephen Hemminger <sthemmin@microsoft.com>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Xavier Deguillard <xdeguillard@vmware.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

mm/sparse: add common helper to mark all memblocks present

Presently the arches arm64, arm and sh have a function which loops through
each memblock and calls memory present. riscv will require a similar
function.

Introduce a common memblocks_present() function that can be used by
all the arches. Subsequent patches will cleanup the arches that
make use of this.

Link: http://lkml.kernel.org/r/20181107205433.3875-3-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Acked-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Oscar Salvador <osalvador@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

mm: Introduce common STRUCT_PAGE_MAX_SHIFT define

This define is used by arm64 to calculate the size of the vmemmap
region. It is defined as the log2 of the upper bound on the size
of a struct page.

We move it into mm_types.h so it can be defined properly instead of
set and checked with a build bug. This also allows us to use the same
define for riscv.

Link: http://lkml.kernel.org/r/20181107205433.3875-2-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Acked-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

locking/atomics: build atomic headers as required

Andrew and Ingo report that the check-atomics.sh script is simply too slow
to run for every kernel build, and it's impractical to make it faster
without rewriting it in something other than shell.

Rather than committing the generated headers, let's regenerate these
as-required for a pristine tree.

That ensures they're always up-to-date, allows them to be built in
parallel, and avoid redundant rebuilds, which is a 2-8s saving per
incremental build.  Since the results are not committed, it's very obvious
that they should not be modified directly.  If we need to generate more
headers in future, it's easy to extend Makefile.genheader to permit this.

I've verified that this works in the cases we previously had issues with
(out-of-tree builds and where scripts have no execute permissions), and
have tested these cases for both x86_64 and arm64.

The diffstat looks nice, at least...

Link: http://lkml.kernel.org/r/20181123153321.8561-1-mark.rutland@arm.com
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

kernel/signal.c: remove some duplicated includes

tty.h and fs.h are included twice.  It's unnecessary.  so remove them.

Link: http://lkml.kernel.org/r/20181126150900.32235-1-tiny.windzz@gmail.com
Signed-off-by: Yangtao Li <tiny.windzz@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

kernel/signal.c: remove some duplicated includes

We include workqueue.h twice.  It's unnecessary, so just remove them.

Link: http://lkml.kernel.org/r/20181126143710.31409-1-tiny.windzz@gmail.com
Signed-off-by: Yangtao Li <tiny.windzz@gmail.com>
Acked-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

scripts/atomic/check-atomics.sh: don't assume that scripts are executable

patch(1) doesn't set the x bit on files.  So if someone downloads and
applies patch-4.21.xz, their kernel won't build.  Fix that by executing
/bin/sh.

Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

Merge branch 'akpm-current/current'

Merge remote-tracking branch 'xarray/xarray'

Merge remote-tracking branch 'nvmem/for-next'

Merge remote-tracking branch 'slimbus/for-next'

Merge remote-tracking branch 'cisco/for-next'

Merge remote-tracking branch 'rtc/rtc-next'

Merge remote-tracking branch 'coresight/next'

Merge remote-tracking branch 'livepatching/for-next'

Merge remote-tracking branch 'y2038/y2038'

Merge remote-tracking branch 'kselftest/next'

Merge remote-tracking branch 'random/dev'

Merge remote-tracking branch 'pwm/for-next'

Merge remote-tracking branch 'pinctrl/for-next'

next-20181207/gpio

Merge remote-tracking branch 'rpmsg/for-next'

Merge remote-tracking branch 'vhost/linux-next'

Merge remote-tracking branch 'scsi-mkp/for-next'

Merge remote-tracking branch 'scsi/for-next'

Merge remote-tracking branch 'cgroup/for-next'

Merge remote-tracking branch 'slave-dma/next'

Merge remote-tracking branch 'icc/icc-next'

Merge remote-tracking branch 'mux/for-next'

Merge remote-tracking branch 'staging/staging-next'

Merge remote-tracking branch 'thunderbolt/next'

Merge remote-tracking branch 'extcon/extcon-next'

Merge remote-tracking branch 'char-misc/char-misc-next'

Merge remote-tracking branch 'tty/tty-next'

Merge remote-tracking branch 'phy-next/next'

Merge remote-tracking branch 'usb-serial/usb-next'

Merge remote-tracking branch 'usb-gadget/next'

Merge remote-tracking branch 'usb/usb-next'

Merge remote-tracking branch 'driver-core/driver-core-next'

Merge remote-tracking branch 'ipmi/for-next'

Merge remote-tracking branch 'leds/for-next'

Merge remote-tracking branch 'hsi/for-next'

Merge remote-tracking branch 'chrome-platform/for-next'

Merge remote-tracking branch 'drivers-x86/for-next'

Merge remote-tracking branch 'percpu/for-next'

Merge remote-tracking branch 'rcu/rcu/next'

Merge remote-tracking branch 'ftrace/for-next'

Merge remote-tracking branch 'edac-amd/for-next'

Merge remote-tracking branch 'clockevents/clockevents/next'

Merge remote-tracking branch 'tip/auto-latest'

Merge remote-tracking branch 'spi/for-next'

Merge remote-tracking branch 'devicetree/for-next'

Merge remote-tracking branch 'audit/next'

Merge remote-tracking branch 'iommu/next'

Merge remote-tracking branch 'watchdog/master'

Merge remote-tracking branch 'tpmdd/next'

Merge remote-tracking branch 'selinux/next'

Merge remote-tracking branch 'integrity/next-integrity'

Merge remote-tracking branch 'apparmor/apparmor-next'

Merge remote-tracking branch 'security/next-testing'

Merge remote-tracking branch 'regulator/for-next'

Merge remote-tracking branch 'battery/for-next'

Merge remote-tracking branch 'backlight/for-backlight-next'

Merge remote-tracking branch 'mfd/for-mfd-next'

Merge remote-tracking branch 'md/for-next'

Merge remote-tracking branch 'kgdb/kgdb-next'

Merge remote-tracking branch 'mmc/next'

Merge remote-tracking branch 'device-mapper/for-next'

Merge remote-tracking branch 'block/for-next'

Merge remote-tracking branch 'input/next'

Merge remote-tracking branch 'modules/modules-next'

Merge remote-tracking branch 'sound-asoc/for-next'

Merge remote-tracking branch 'sound/for-next'

Merge remote-tracking branch 'regmap/for-next'

Merge remote-tracking branch 'etnaviv/etnaviv/next'

Merge remote-tracking branch 'imx-drm/imx-drm/next'

Merge remote-tracking branch 'drm-msm/msm-next'

Merge remote-tracking branch 'drm-tegra/drm/tegra/for-next'

Merge remote-tracking branch 'drm-intel/for-linux-next'

Merge remote-tracking branch 'drm/drm-next'

Merge remote-tracking branch 'crypto/master'

Merge remote-tracking branch 'spi-nor/spi-nor/next'

Merge remote-tracking branch 'nand/nand/next'

Merge remote-tracking branch 'mtd/mtd/next'

Merge remote-tracking branch 'gfs2/for-next'

Merge remote-tracking branch 'mac80211-next/master'

Merge remote-tracking branch 'nfc-next/master'

Merge remote-tracking branch 'netfilter-next/master'

Merge remote-tracking branch 'mlx5-next/mlx5-next'

Merge remote-tracking branch 'ipsec-next/master'

Merge remote-tracking branch 'bpf-next/master'

Merge remote-tracking branch 'net-next/master'

next-20181203/rdma

Merge remote-tracking branch 'swiotlb/linux-next'

Merge remote-tracking branch 'dlm/next'

Merge remote-tracking branch 'ieee1394/for-next'

Merge remote-tracking branch 'thermal/next'

Merge remote-tracking branch 'opp/opp/linux-next'