]> www.infradead.org Git - users/hch/xfsprogs.git/log
users/hch/xfsprogs.git
20 months agoxfs_db: support dumping realtime superblocks
Darrick J. Wong [Sat, 13 Aug 2022 00:41:32 +0000 (17:41 -0700)]
xfs_db: support dumping realtime superblocks

Allow debugging of realtime superblocks, and add the relevant fields in
the fs superblock that point us at the existence and location of the rt
supers.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_db: listify the definition of enum typnm
Darrick J. Wong [Mon, 29 Aug 2022 22:45:45 +0000 (15:45 -0700)]
xfs_db: listify the definition of enum typnm

Convert the enum definition into a list so that future patches adding
things to enum typnm don't have to reflow the entire thing.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: support adding rtgroups to a filesystem
Darrick J. Wong [Thu, 10 Aug 2023 21:28:46 +0000 (14:28 -0700)]
xfs_repair: support adding rtgroups to a filesystem

Allow users to add the rtgroups feature to a filesystem if the
filesystem does not already have a realtime volume attached.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: repair rtsummary block headers
Darrick J. Wong [Mon, 19 Sep 2022 21:21:36 +0000 (14:21 -0700)]
xfs_repair: repair rtsummary block headers

Check and repair the new block headers attached to rtsummary blocks.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: repair rtbitmap block headers
Darrick J. Wong [Sat, 17 Sep 2022 00:10:10 +0000 (17:10 -0700)]
xfs_repair: repair rtbitmap block headers

Check and repair the new block headers attached to rtbitmap blocks.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: improve rtbitmap discrepancy reporting
Darrick J. Wong [Mon, 19 Sep 2022 16:49:44 +0000 (09:49 -0700)]
xfs_repair: improve rtbitmap discrepancy reporting

Improve the reporting of discrepancies in the realtime bitmap and
summary files by creating a separate helper function that will pinpoint
the exact (word) locations of mismatches.  This will help developers to
diagnose problems with the rtgroups feature and users to figure out
exactly what's bad in a filesystem.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: support realtime groups
Darrick J. Wong [Fri, 12 Aug 2022 23:58:20 +0000 (16:58 -0700)]
xfs_repair: support realtime groups

Support the realtime group feature.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agolibxfs: implement some sanity checking for enormous rgcount
Darrick J. Wong [Thu, 9 Nov 2023 23:34:10 +0000 (15:34 -0800)]
libxfs: implement some sanity checking for enormous rgcount

Similar to what we do for suspiciously large sb_agcount values, if
someone tries to get libxfs to load a filesystem with a very large
realtime group count, let's do some basic checks of the rt device to
see if it's really that large.  If the read fails, only load the first
rtgroup and warn the user.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agolibfrog: report rt groups in output
Darrick J. Wong [Fri, 12 Aug 2022 22:12:57 +0000 (15:12 -0700)]
libfrog: report rt groups in output

Report realtime group geometry.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: scrub each rtgroup's portion of the rtbitmap separately
Darrick J. Wong [Tue, 7 Mar 2023 03:55:29 +0000 (19:55 -0800)]
xfs: scrub each rtgroup's portion of the rtbitmap separately

Create a new scrub type code so that userspace can scrub each rtgroup's
portion of the rtbitmap file separately.  This reduces the long tail
latency that results from scanning the entire bitmap all at once, and
prepares us for future patchsets, wherein we'll need to be able to lock
a specific rtgroup so that we can rebuild that rtgroup's part of the
rtbitmap contents from the rtgroup's rmap btree.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: repair secondary realtime group superblocks
Darrick J. Wong [Tue, 7 Mar 2023 03:55:29 +0000 (19:55 -0800)]
xfs: repair secondary realtime group superblocks

Repair secondary realtime group superblocks.  They're not critical for
anything, but some consistency would be a good idea.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: scrub the realtime group superblock
Darrick J. Wong [Tue, 7 Mar 2023 03:55:28 +0000 (19:55 -0800)]
xfs: scrub the realtime group superblock

Enable scrubbing of realtime group superblocks.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: use an incore rtgroup rotor for rtpick
Darrick J. Wong [Thu, 9 Nov 2023 17:35:51 +0000 (09:35 -0800)]
xfs: use an incore rtgroup rotor for rtpick

During the 6.7 merge window, Linus noticed that the realtime allocator
was doing some sketchy things trying to encode a u64 sequence counter
into the rtbitmap file's atime.  The sketchy casting of a struct pointer
to a u64 pointer has subtly broken several times over the past decade as
the codebase has transitioned to using the VFS i_atime field and that
field has changed in size and layout over time.

Since the goal of the rtpick code is to _suggest_ a starting place for
new rt file allocations, the repeated breakage has not resulted in
inconsistent metadata.  IOWs, it's a hint.

For rtgroups, we don't need this complex code to cut the rtextents space
into fractions.  Add an rtgroup rotor and use that for rtpick, similar
to AG rotoring on the data device.  The new rotor does not persist,
which reduces the logging overhead slightly.

Link: https://lore.kernel.org/linux-xfs/CAHk-=wj3oM3d-Hw2vvxys3KCZ9De+gBN7Gxr2jf96OTisL9udw@mail.gmail.com/
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: store rtgroup information with a bmap intent
Darrick J. Wong [Tue, 7 Mar 2023 03:55:28 +0000 (19:55 -0800)]
xfs: store rtgroup information with a bmap intent

Make the bmap intent items take an active reference to the rtgroup
containing the space that is being mapped or unmapped.  We will need
this functionality once we start enabling rmap and reflink on the rt
volume.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: encode the rtsummary in big endian format
Darrick J. Wong [Tue, 7 Mar 2023 03:55:27 +0000 (19:55 -0800)]
xfs: encode the rtsummary in big endian format

Currently, the ondisk realtime summary file counters are accessed in
units of 32-bit words.  There's no endian translation of the contents of
this file, which means that the Bad Things Happen(tm) if you go from
(say) x86 to powerpc.  Since we have a new feature flag, let's take the
opportunity to enforce an endianness on the file.  Encode the summary
information in big endian format, like most of the rest of the
filesystem.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: add block headers to realtime summary blocks
Darrick J. Wong [Tue, 7 Mar 2023 03:55:27 +0000 (19:55 -0800)]
xfs: add block headers to realtime summary blocks

Upgrade rtsummary blocks to have self describing metadata like most
every other thing in XFS.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: encode the rtbitmap in little endian format
Darrick J. Wong [Tue, 7 Mar 2023 03:55:26 +0000 (19:55 -0800)]
xfs: encode the rtbitmap in little endian format

Currently, the ondisk realtime bitmap file is accessed in units of
32-bit words.  There's no endian translation of the contents of this
file, which means that the Bad Things Happen(tm) if you go from (say)
x86 to powerpc.  Since we have a new feature flag, let's take the
opportunity to enforce an endianness on the file.

The natural format of a bitmap is (IMHO) little endian, because the byte
offsets of the bitmap data should always increase in step with the
information being indexed.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: add block headers to realtime bitmap blocks
Darrick J. Wong [Tue, 7 Mar 2023 03:55:25 +0000 (19:55 -0800)]
xfs: add block headers to realtime bitmap blocks

Upgrade rtbitmap blocks to have self describing metadata like most every
other thing in XFS.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: export the geometry of realtime groups to userspace
Darrick J. Wong [Tue, 7 Mar 2023 03:55:25 +0000 (19:55 -0800)]
xfs: export the geometry of realtime groups to userspace

Create an ioctl so that the kernel can report the status of realtime
groups to userspace.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: define locking primitives for realtime groups
Darrick J. Wong [Tue, 7 Mar 2023 03:55:24 +0000 (19:55 -0800)]
xfs: define locking primitives for realtime groups

Define helper functions to lock all metadata inodes related to a
realtime group.  There's not much to look at now, but this will become
important when we add per-rtgroup metadata files and online fsck code
for them.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: record rt group superblock errors in the health system
Darrick J. Wong [Tue, 7 Mar 2023 03:55:24 +0000 (19:55 -0800)]
xfs: record rt group superblock errors in the health system

Record the state of per-rtgroup metadata sickness in the rtgroup
structure for later reporting.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: add frextents to the lazysbcounters when rtgroups enabled
Darrick J. Wong [Tue, 7 Mar 2023 03:55:23 +0000 (19:55 -0800)]
xfs: add frextents to the lazysbcounters when rtgroups enabled

Make the free rt extent count a part of the lazy sb counters when the
realtime groups feature is enabled.  This is possible because the patch
to recompute frextents from the rtbitmap during log recovery predates
the code adding rtgroup support, hence we know that the value will
always be correct during runtime.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: check that rtblock extents do not overlap with the rt group metadata
Darrick J. Wong [Tue, 7 Mar 2023 03:55:23 +0000 (19:55 -0800)]
xfs: check that rtblock extents do not overlap with the rt group metadata

The ondisk format specifies that the start of each realtime group must
have a superblock so that rt space mappings never cross an rtgroup
boundary.  Check that rt block pointers obey this.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: export realtime group geometry via XFS_FSOP_GEOM
Darrick J. Wong [Tue, 7 Mar 2023 03:55:22 +0000 (19:55 -0800)]
xfs: export realtime group geometry via XFS_FSOP_GEOM

Export the realtime geometry information so that userspace can query it.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: grow the realtime section when realtime groups are enabled
Darrick J. Wong [Tue, 7 Mar 2023 03:55:21 +0000 (19:55 -0800)]
xfs: grow the realtime section when realtime groups are enabled

Enable growing the rt section when realtime groups are enabled.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: write secondary realtime superblocks to disk
Darrick J. Wong [Tue, 7 Mar 2023 03:55:20 +0000 (19:55 -0800)]
xfs: write secondary realtime superblocks to disk

Create some library functions to make it easy to update all the
secondary realtime superblocks on disk; this will be used by growfs,
xfs_db, mkfs, and xfs_repair.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: update primary realtime super every time we update the primary fs super
Darrick J. Wong [Tue, 7 Mar 2023 03:55:20 +0000 (19:55 -0800)]
xfs: update primary realtime super every time we update the primary fs super

Every time we update parts of the primary filesystem superblock that are
echoed in the primary rt super, we should update that primary realtime
super.  Avoid an ondisk log format change by using ordered buffers to
write the primary rt super.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: define the format of rt groups
Darrick J. Wong [Tue, 7 Mar 2023 03:55:19 +0000 (19:55 -0800)]
xfs: define the format of rt groups

Define the ondisk format of realtime group metadata.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: create incore realtime group structures
Darrick J. Wong [Tue, 7 Mar 2023 03:55:18 +0000 (19:55 -0800)]
xfs: create incore realtime group structures

Create an incore object that will contain information about a realtime
allocation group.  This will eventually enable us to shard the realtime
section in a similar manner to how we shard the data section.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: remove XFS_ILOCK_RT*
Darrick J. Wong [Tue, 7 Mar 2023 03:55:18 +0000 (19:55 -0800)]
xfs: remove XFS_ILOCK_RT*

Now that we've centralized the realtime metadata locking routines, get
rid of the ILOCK subclasses since we now use explicit lockdep classes.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: refactor realtime inode locking
Darrick J. Wong [Tue, 7 Mar 2023 03:55:17 +0000 (19:55 -0800)]
xfs: refactor realtime inode locking

Create helper functions to deal with locking realtime metadata inodes.
This enables us to maintain correct locking order once we start adding
the realtime rmap and refcount btree inodes.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_db: allow setting current address to log blocks
Darrick J. Wong [Tue, 20 Sep 2022 20:38:44 +0000 (13:38 -0700)]
xfs_db: allow setting current address to log blocks

Add commands so that users can target blocks on an external log device.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_db: convert rtsummary geometry
Darrick J. Wong [Tue, 20 Sep 2022 23:07:55 +0000 (16:07 -0700)]
xfs_db: convert rtsummary geometry

Teach the rtconvert command to be able to convert realtime blocks and
extents to locations within the rt summary.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_db: convert rtbitmap geometry
Darrick J. Wong [Tue, 20 Sep 2022 23:01:06 +0000 (16:01 -0700)]
xfs_db: convert rtbitmap geometry

Teach the rtconvert command to be able to convert realtime blocks and
extents to locations within the rt bitmap.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_db: enable conversion of rt space units
Darrick J. Wong [Tue, 20 Sep 2022 22:17:22 +0000 (15:17 -0700)]
xfs_db: enable conversion of rt space units

Teach the xfs_db convert function about rt extents, rt block numbers,
and how to compute offsets within the rt bitmap and summary files.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_db: access arbitrary realtime blocks and extents
Darrick J. Wong [Tue, 20 Sep 2022 19:59:48 +0000 (12:59 -0700)]
xfs_db: access arbitrary realtime blocks and extents

Add two commands to xfs_db so that we can point ourselves at any
arbitrary realtime block or extent.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_db: access realtime file blocks
Darrick J. Wong [Tue, 20 Sep 2022 19:37:50 +0000 (12:37 -0700)]
xfs_db: access realtime file blocks

Now that we have the ability to point the io cursor at the realtime
device, let's make it so that the "dblock" command can walk the contents
of realtime files.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_db: make the daddr command target the realtime device
Darrick J. Wong [Tue, 20 Sep 2022 20:36:10 +0000 (13:36 -0700)]
xfs_db: make the daddr command target the realtime device

Make it so that users can issue the command "daddr -r XXX" to select
disk block XXX on the realtime device.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_db: report the realtime device when associated with each io cursor
Darrick J. Wong [Tue, 20 Sep 2022 20:10:35 +0000 (13:10 -0700)]
xfs_db: report the realtime device when associated with each io cursor

When db is reporting on an io cursor and the cursor points to the
realtime device, print that fact.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_db: support passing the realtime device to the debugger
Darrick J. Wong [Tue, 16 Aug 2022 16:39:37 +0000 (09:39 -0700)]
xfs_db: support passing the realtime device to the debugger

Create a new -R flag so that sysadmins can pass the realtime device to
the xfs debugger.  Since we can now have superblocks on the rt device,
we need this to be able to inspect/dump/etc.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agomkfs: add a utility to generate protofiles
Darrick J. Wong [Fri, 15 Jul 2022 21:22:06 +0000 (14:22 -0700)]
mkfs: add a utility to generate protofiles

Add a new utility to generate mkfs protofiles from a directory tree.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agomkfs.xfs: enable metadata directories
Darrick J. Wong [Fri, 15 Jul 2022 21:22:06 +0000 (14:22 -0700)]
mkfs.xfs: enable metadata directories

Enable formatting filesystems with metadata directories.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: allow sysadmins to add metadata directories
Darrick J. Wong [Fri, 15 Jul 2022 21:22:06 +0000 (14:22 -0700)]
xfs_repair: allow sysadmins to add metadata directories

Allow the sysadmin to use xfs_repair to upgrade an existing filesystem
to support metadata directories.  This will be needed to upgrade
filesystems to support realtime rmap and reflink.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: do not count metadata directory files when doing quotacheck
Darrick J. Wong [Thu, 29 Jun 2023 20:23:16 +0000 (13:23 -0700)]
xfs_repair: do not count metadata directory files when doing quotacheck

Previously, we stated that files in the metadata directory tree are not
counted in the dquot information.  Fix the offline quotacheck code in
xfs_repair and xfs_check to reflect this.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: truncate and unmark orphaned metadata inodes
Darrick J. Wong [Fri, 15 Jul 2022 21:22:06 +0000 (14:22 -0700)]
xfs_repair: truncate and unmark orphaned metadata inodes

If an inode claims to be a metadata inode but wasn't linked in either
directory tree, remove the attr fork and reset the data fork if the
contents weren't regular extent mappings before moving the inode to the
lost+found.

We don't ifree the inode, because it's possible that the inode was not
actually a metadata inode but simply got corrupted due to bitflips or
something, and we'd rather let the sysadmin examine what's left of the
file instead of photorec'ing it.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: drop all the metadata directory files during pass 4
Darrick J. Wong [Fri, 15 Jul 2022 21:22:06 +0000 (14:22 -0700)]
xfs_repair: drop all the metadata directory files during pass 4

Drop the entire metadata directory tree during pass 4 so that we can
reinitialize the entire tree in phase 6.  The existing metadata files
(rtbitmap, rtsummary, quotas) will be reattached to the newly rebuilt
directory tree.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: reattach quota inodes to metadata directory
Darrick J. Wong [Fri, 15 Jul 2022 21:22:05 +0000 (14:22 -0700)]
xfs_repair: reattach quota inodes to metadata directory

If the quota inodes came through unscathed, we should attach them to
the new metadata directory so that phase 7 can run quotacheck on them.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: metadata dirs are never plausible root dirs
Darrick J. Wong [Fri, 15 Jul 2022 21:22:05 +0000 (14:22 -0700)]
xfs_repair: metadata dirs are never plausible root dirs

Metadata directories are never candidates to be the root of the
user-accessible directory tree.  Update has_plausible_rootdir to ignore
them all, as well as detecting the case where the superblock incorrectly
thinks both trees have the same root.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: adjust keep_fsinos to handle metadata directories
Darrick J. Wong [Fri, 15 Jul 2022 21:22:05 +0000 (14:22 -0700)]
xfs_repair: adjust keep_fsinos to handle metadata directories

On a filesystem with metadata directories, we only want to automatically
mark the two root directories present because those are the only two
statically allocated inode numbers -- the rt summary inode is now just a
regular file in a directory.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: mark space used by metadata files
Darrick J. Wong [Fri, 15 Jul 2022 21:22:05 +0000 (14:22 -0700)]
xfs_repair: mark space used by metadata files

Track space used by metadata files as a separate incore extent type.
This ensures that we can warn about cross-linked metadata files, even
though we are going to rebuild the entire metadata directory tree in the
end.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: pass private data pointer to scan_lbtree
Darrick J. Wong [Fri, 15 Jul 2022 21:22:05 +0000 (14:22 -0700)]
xfs_repair: pass private data pointer to scan_lbtree

Pass a private data pointer through scan_lbtree.  We'll use this
later when scanning the rtrmapbt to keep track of scan state.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: update incore metadata state whenever we create new files
Darrick J. Wong [Fri, 15 Jul 2022 21:22:05 +0000 (14:22 -0700)]
xfs_repair: update incore metadata state whenever we create new files

Make sure that we update our incore metadata inode bookkeepping whenever
we create new metadata files.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: don't let metadata and regular files mix
Darrick J. Wong [Fri, 15 Jul 2022 21:22:05 +0000 (14:22 -0700)]
xfs_repair: don't let metadata and regular files mix

Track whether or not inodes thought they were metadata inodes.  We
cannot allow metadata inodes to appear in the regular directory tree,
and we cannot allow regular inodes to appear in the metadata directory
tree.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: rebuild the metadata directory
Darrick J. Wong [Fri, 15 Jul 2022 21:22:05 +0000 (14:22 -0700)]
xfs_repair: rebuild the metadata directory

Check the metadata directory for problems and rebuild it if necessary.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: check metadata inode flag
Darrick J. Wong [Fri, 15 Jul 2022 21:22:04 +0000 (14:22 -0700)]
xfs_repair: check metadata inode flag

Check whether or not the metadata inode flag is set appropriately.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: refactor grabbing realtime metadata inodes
Darrick J. Wong [Fri, 15 Jul 2022 21:22:04 +0000 (14:22 -0700)]
xfs_repair: refactor grabbing realtime metadata inodes

Create a helper function to grab a realtime metadata inode.  When
metadir arrives, the bitmap and summary inodes can float, so we'll
turn this function into a "load or allocate" function.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: refactor root directory initialization
Darrick J. Wong [Fri, 15 Jul 2022 21:22:04 +0000 (14:22 -0700)]
xfs_repair: refactor root directory initialization

Refactor root directory initialization into a separate function we can
call for both the root dir and the metadir.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: refactor marking of metadata inodes
Darrick J. Wong [Fri, 15 Jul 2022 21:22:04 +0000 (14:22 -0700)]
xfs_repair: refactor marking of metadata inodes

Refactor the mechanics of marking a metadata inode into a helper
function so that we don't have to open-code that for every single
metadata inode.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: refactor fixing dotdot
Darrick J. Wong [Fri, 15 Jul 2022 21:22:04 +0000 (14:22 -0700)]
xfs_repair: refactor fixing dotdot

Pull the code that fixes a directory's dot-dot entry into a separate
helper function so that we can call it on the rootdir and (later) the
metadir.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: dont check metadata directory dirent inumbers
Darrick J. Wong [Fri, 12 May 2023 23:54:36 +0000 (16:54 -0700)]
xfs_repair: dont check metadata directory dirent inumbers

Phase 6 always rebuilds the entire metadata directory tree, and repair
quietly ignores all the DIFLAG2_METADATA directory inodes that it finds.
As a result, none of the metadata directories are marked inuse in the
incore data.  Therefore, the is_inode_free checks are not valid for
anything we find in a metadata directory.

Therefore, avoid checking is_inode_free when scanning metadata directory
dirents.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: reject regular directory dirents that point to metadata fieles
Darrick J. Wong [Sat, 13 May 2023 00:00:43 +0000 (17:00 -0700)]
xfs_repair: reject regular directory dirents that point to metadata fieles

If a directory that's in the regular (non-metadata) directory tree has
an entry that points to a metadata file, trash the dirent.  Files are
not allowed to cross between the two trees.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: refactor metadata inode tagging
Darrick J. Wong [Fri, 15 Jul 2022 21:22:04 +0000 (14:22 -0700)]
xfs_repair: refactor metadata inode tagging

Refactor tagging of metadata inodes into a single helper function
instead of open-coding a if-else statement.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: don't zero the incore secondary super when zeroing
Darrick J. Wong [Tue, 16 Aug 2022 23:56:14 +0000 (16:56 -0700)]
xfs_repair: don't zero the incore secondary super when zeroing

If secondary_sb_whack detects nonzero bytes beyond the end of the ondisk
superblock, it will try to zero the end of the ondisk buffer as well as
the incore superblock prior to scan_ag using that incore super to
rewrite the ondisk super.

However, the metadata directory feature adds a sb_metadirino field to
the incore super.  On disk, this is stored in the same slot as
sb_rbmino, but we wanted to cache both inumbers incore to minimize the
churn.  Therefore, it is now only safe to zero the "end" of an xfs_dsb
buffer, and never an xfs_sb object.

Most of the XFS codebase moved off that second behavior long ago, with
the exception of this one part of repair.  The zeroing probably ought to
be turned into explicit logic to zero fields that weren't defined with
the featureset encoded in the primary superblock, but for now we'll
resort to always resetting the values from the xfs_mount's xfs_sb.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_scrub: re-run metafile scrubbers during phase 5
Darrick J. Wong [Thu, 17 Aug 2023 20:23:14 +0000 (13:23 -0700)]
xfs_scrub: re-run metafile scrubbers during phase 5

For metadata files on a metadir filesystem, re-run the scrubbers during
phase 5 to ensure that the metadata files are still connected.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_scrub: scan metadata directories during phase 3
Darrick J. Wong [Fri, 15 Jul 2022 21:22:04 +0000 (14:22 -0700)]
xfs_scrub: scan metadata directories during phase 3

Scan metadata directories for correctness during phase 3.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_spaceman: report health of metadir inodes too
Darrick J. Wong [Thu, 20 Jul 2023 00:22:16 +0000 (17:22 -0700)]
xfs_spaceman: report health of metadir inodes too

If the filesystem has a metadata directory tree, we should include those
inodes in the health report.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_io: support scrubbing metadata directory paths
Darrick J. Wong [Fri, 18 Aug 2023 17:06:28 +0000 (10:06 -0700)]
xfs_io: support scrubbing metadata directory paths

Support invoking the metadata directory path scrubber from xfs_io for
testing.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_io: support the bulkstat metadata directory flag
Darrick J. Wong [Fri, 15 Jul 2022 21:22:03 +0000 (14:22 -0700)]
xfs_io: support the bulkstat metadata directory flag

Support the new XFS_BULK_IREQ_METADIR flag for bulkstat commands.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_db: mask superblock fields when metadir feature is enabled
Darrick J. Wong [Fri, 15 Jul 2022 21:22:03 +0000 (14:22 -0700)]
xfs_db: mask superblock fields when metadir feature is enabled

When the metadata directory feature is enabled, mask the superblock
fields (rt, quota inodes) that got migrated to the directory tree.
Similarly, hide the 'metadirino' field when the feature is disabled.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_db: support metadata directories in the path command
Darrick J. Wong [Fri, 15 Jul 2022 21:22:03 +0000 (14:22 -0700)]
xfs_db: support metadata directories in the path command

Teach the path command to traverse the metadata directory tree by
passing a '\' as the first letter in the path.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_db: don't obfuscate metadata directories and attributes
Darrick J. Wong [Fri, 15 Jul 2022 21:22:03 +0000 (14:22 -0700)]
xfs_db: don't obfuscate metadata directories and attributes

Don't obfuscate the directory and attribute names of metadata inodes.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_db: report metadir support for version command
Darrick J. Wong [Fri, 15 Jul 2022 21:22:03 +0000 (14:22 -0700)]
xfs_db: report metadir support for version command

Report metadir support if we have it enabled.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_db: basic xfs_check support for metadir
Darrick J. Wong [Fri, 15 Jul 2022 21:22:03 +0000 (14:22 -0700)]
xfs_db: basic xfs_check support for metadir

Support metadata directories in xfs_check.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_io: support scrubbing metadata directory paths
Darrick J. Wong [Fri, 18 Aug 2023 17:15:31 +0000 (10:15 -0700)]
xfs_io: support scrubbing metadata directory paths

Support invoking the metadata directory path scrubber from xfs_io for
testing.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agolibfrog: allow METADIR in xfrog_bulkstat_single5
Darrick J. Wong [Tue, 25 Jul 2023 00:18:04 +0000 (17:18 -0700)]
libfrog: allow METADIR in xfrog_bulkstat_single5

This is a valid flag for a single-file bulkstat, so add that to the
filter.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agolibfrog: report metadata directories in the geometry report
Darrick J. Wong [Fri, 15 Jul 2022 21:22:03 +0000 (14:22 -0700)]
libfrog: report metadata directories in the geometry report

Report the presence of a metadata directory tree in the geometry report.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: check metadata directory file path connectivity
Darrick J. Wong [Fri, 18 Aug 2023 15:34:13 +0000 (08:34 -0700)]
xfs: check metadata directory file path connectivity

Create a new scrubber type that checks that well known metadata
directory paths are connected to the metadata inode that the incore
structures think is in use.  IOWs, check that "/quota/user" in the
metadata directory tree actually points to
mp->m_quotainfo->qi_uquotaip->i_ino.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: record health problems with the metadata directory
Darrick J. Wong [Tue, 7 Mar 2023 03:54:59 +0000 (19:54 -0800)]
xfs: record health problems with the metadata directory

Make a report to the health monitoring subsystem any time we encounter
something in the metadata directory tree that looks like corruption.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: adjust xfs_bmap_add_attrfork for metadir
Darrick J. Wong [Thu, 20 Jul 2023 00:25:43 +0000 (17:25 -0700)]
xfs: adjust xfs_bmap_add_attrfork for metadir

Online repair might use the xfs_bmap_add_attrfork to repair a file in
the metadata directory tree if (say) the metadata file lacks the correct
parent pointers.  In that case, it is not correct to check that the file
is dqattached -- metadata files must be not have /any/ dquot attached at
all.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: enable creation of dynamically allocated metadir path structures
Darrick J. Wong [Tue, 7 Mar 2023 03:54:58 +0000 (19:54 -0800)]
xfs: enable creation of dynamically allocated metadir path structures

Add a few helper functions so that it's possible to allocate
xfs_imeta_path objects dynamically, along with dynamically allocated
path components.  Eventually we're going to want to support paths of the
form "/realtime/$rtgroup.rmap", and this is necessary for that.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: allow bulkstat to return metadata directories
Darrick J. Wong [Tue, 7 Mar 2023 03:54:58 +0000 (19:54 -0800)]
xfs: allow bulkstat to return metadata directories

Allow the V5 bulkstat ioctl to return information about metadata
directory files so that xfs_scrub can find and scrub them, since they
are otherwise ordinary directories.

(Metadata files of course require per-file scrub code and hence do not
need exposure.)

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: advertise metadata directory feature
Darrick J. Wong [Tue, 7 Mar 2023 03:54:57 +0000 (19:54 -0800)]
xfs: advertise metadata directory feature

Advertise the existence of the metadata directory feature; this will be
used by scrub to decide if it needs to scan the metadir too.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: disable the agi rotor for metadata inodes
Darrick J. Wong [Tue, 7 Mar 2023 03:54:56 +0000 (19:54 -0800)]
xfs: disable the agi rotor for metadata inodes

Ideally, we'd put all the metadata inodes in one place if we could, so
that the metadata all stay reasonably close together instead of
spreading out over the disk.  Furthermore, if the log is internal we'd
probably prefer to keep the metadata near the log.  Therefore, disable
AGI rotoring for metadata inode allocations.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: ensure metadata directory paths exist before creating files
Darrick J. Wong [Tue, 7 Mar 2023 03:54:55 +0000 (19:54 -0800)]
xfs: ensure metadata directory paths exist before creating files

Since xfs_imeta_create can create new metadata files arbitrarily deep in
the metadata directory tree, we must supply a function that can ensure
that all directories in a path exist, and call it before the quota
functions create the quota inodes.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: read and write metadata inode directory
Darrick J. Wong [Tue, 7 Mar 2023 03:54:55 +0000 (19:54 -0800)]
xfs: read and write metadata inode directory

Plumb in the bits we need to look up metadata inode numbers from the
metadata inode directory and save them back.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: allow deletion of metadata directory files
Darrick J. Wong [Tue, 18 Apr 2023 23:28:35 +0000 (16:28 -0700)]
xfs: allow deletion of metadata directory files

Make it possible to free metadata files once we've unlinked them from
the directory structure.  We don't do this in the kernel, at least not
yet, but don't leave a logic bomb for later.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: enforce metadata inode flag
Darrick J. Wong [Tue, 7 Mar 2023 03:54:54 +0000 (19:54 -0800)]
xfs: enforce metadata inode flag

Add checks for the metadata inode flag so that we don't ever leak
metadata inodes out to userspace, and we don't ever try to read a
regular inode as metadata.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: convert metadata inode lookup keys to use paths
Darrick J. Wong [Tue, 7 Mar 2023 03:54:53 +0000 (19:54 -0800)]
xfs: convert metadata inode lookup keys to use paths

Convert the magic metadata inode lookup keys to use actual strings
for paths.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: load metadata directory root at mount time
Darrick J. Wong [Tue, 7 Mar 2023 03:54:53 +0000 (19:54 -0800)]
xfs: load metadata directory root at mount time

Load the metadata directory root inode into memory at mount time and
release it at unmount time.  We also make sure that the obsolete inode
pointers in the superblock are not logged or read from the superblock.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: update imeta transaction reservations for metadir
Darrick J. Wong [Tue, 7 Mar 2023 03:54:52 +0000 (19:54 -0800)]
xfs: update imeta transaction reservations for metadir

Update the new metadata inode transaction reservations to handle
metadata directories if that feature is enabled.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: define the on-disk format for the metadir feature
Darrick J. Wong [Tue, 7 Mar 2023 03:54:52 +0000 (19:54 -0800)]
xfs: define the on-disk format for the metadir feature

Define the on-disk layout and feature flags for the metadata inode
directory feature.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: iget for metadata inodes
Darrick J. Wong [Tue, 7 Mar 2023 03:54:51 +0000 (19:54 -0800)]
xfs: iget for metadata inodes

Create a xfs_iget_meta function for metadata inodes to ensure that we
always check that the inobt thinks a metadata inode is in use.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agomkfs: break up the rest of the rtinit() function
Darrick J. Wong [Fri, 15 Jul 2022 21:21:22 +0000 (14:21 -0700)]
mkfs: break up the rest of the rtinit() function

Break up this really long function into smaller functions that each do
one thing.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agolibxfs: convert all users to libxfs_imeta_create
Darrick J. Wong [Fri, 15 Jul 2022 21:21:22 +0000 (14:21 -0700)]
libxfs: convert all users to libxfs_imeta_create

Convert all open-coded sb metadata inode pointer logging to use
libxfs_imeta_create to create metadata inodes.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agomkfs: clean up the rtinit() function
Darrick J. Wong [Fri, 15 Jul 2022 21:21:21 +0000 (14:21 -0700)]
mkfs: clean up the rtinit() function

Clean up some of the warts in this function, like the inconsistent use
of @i for @error, missing comments, and make this more visually pleasing
by adding some whitespace between major sections.  Some things are left
untouched for the next patch.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: create transaction reservations for metadata inode operations
Darrick J. Wong [Tue, 7 Mar 2023 03:54:50 +0000 (19:54 -0800)]
xfs: create transaction reservations for metadata inode operations

Create transaction reservation types and block reservation helpers to
help us calculate transaction requirements.  Right now the reservations
are the same as always; we're just separating the symbols for a future
patch.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: create imeta abstractions to get and set metadata inodes
Darrick J. Wong [Tue, 7 Mar 2023 03:54:49 +0000 (19:54 -0800)]
xfs: create imeta abstractions to get and set metadata inodes

Create some helper routines to get and set metadata inode numbers
instead of open-coding them throughout xfs.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs: don't use the incore struct xfs_sb for offsets into struct xfs_dsb
Darrick J. Wong [Tue, 7 Mar 2023 03:54:48 +0000 (19:54 -0800)]
xfs: don't use the incore struct xfs_sb for offsets into struct xfs_dsb

Currently, the XFS_SB_CRC_OFF macro uses the incore superblock struct
(xfs_sb) to compute the address of sb_crc within the ondisk superblock
struct (xfs_dsb).  This is a landmine if we ever change the layout of
the incore superblock (as we're about to do), so redefine the macro
to use xfs_dsb to compute the layout of xfs_dsb.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: use library functions for orphanage creation
Darrick J. Wong [Fri, 15 Jul 2022 21:19:08 +0000 (14:19 -0700)]
xfs_repair: use library functions for orphanage creation

Use new library functions to create lost+found.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
20 months agoxfs_repair: use library functions to reset root/rbm/rsum inodes
Darrick J. Wong [Fri, 15 Jul 2022 21:15:16 +0000 (14:15 -0700)]
xfs_repair: use library functions to reset root/rbm/rsum inodes

Use the iroot reset function to reset root inodes instead of open-coding
the reset routine.  While we're at it, fix a longstanding memory leak if
the inode being reset actually had an xattr fork full of mappings.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>