Darrick J. Wong [Fri, 21 Mar 2025 21:28:26 +0000 (14:28 -0700)]
xfs/818: fix some design issues
While QA'ing zoned filesystem support, I discovered some design errors
in this test:
1) Since we're test formatting a sparse file on an xfs filesystem,
there's no need to play games with optimal device size; we can create
a totally sparse file that's the same size as SCRATCH_DEV.
2) mkfs.xfs cannot create realtime files, so if it fails with that,
there's no need to continue the test.
3) If mkfs -p fails for none of the proscribed reasons, it should exit
the test. The final cat $tmp.mkfs will take care of tweaking the golden
output to register the test failure for further investigation.
Cc: fstests@vger.kernel.org # v2025.03.09 Fixes: 6d39dc34e61e11 ("xfs: test filesystem creation with xfs_protofile") Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 21 Mar 2025 21:28:10 +0000 (14:28 -0700)]
common/populate: drop fallocate mode 0 requirement
None of the _scratch_$FSTYP_populate functions use fallocate mode 0 (aka
preallocation) to run, so drop the _require check. This enables xfs/349
and friends to work on always-cow xfs filesystems.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 21 Mar 2025 21:27:54 +0000 (14:27 -0700)]
generic/537: disable quota mount options for pre-metadir rt filesystems
Fix this regression in generic/537:
mount: /opt: permission denied.
dmesg(1) may have more information after failed mount system call.
mount -o uquota,gquota,pquota, -o ro,norecovery -ortdev=/dev/sdb4 /dev/sda4 /opt failed
mount -o uquota,gquota,pquota, -o ro,norecovery -ortdev=/dev/sdb4 /dev/sda4 /opt failed
(see /var/tmp/fstests/generic/537.full for details)
for reasons explained in the giant comment. TLDR: quota and rt aren't
compatible on older xfs filesystems so we have to work around that.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 21 Mar 2025 21:27:38 +0000 (14:27 -0700)]
xfs/614: determine the sector size of the fs image by doing a test format
In some cases (such as xfs always_cow=1), the configuration of the test
filesystem determines the sector size of the filesystem that we're going
to simulate formatting. Concretely, even if TEST_DEV is a block device
with 512b sectors, the directio geometry can specify 4k writes to avoid
nasty RMW cycles. When this happens, mkfs.xfs will set the sector size
to that 4k accordingly, but the golden output selection is wrong. Fix
this.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 21 Mar 2025 21:27:23 +0000 (14:27 -0700)]
generic/45[34]: add colored emoji variants to unicode tests
Ted told me this morning about a recent problem with kernel Unicode name
casefolding vs. emoji -- initially, someone decided that zero-width
joiners should be stripped out of filenames during comparisons, which
lead to malicious git pulls of branches containing "<zwj>.git/config"
files overwriting git repo config files. A quick fix was to stop
ignoring the "ignorable" code points, but that broke emoji in filenames,
because emoji use zero-width joiners to combine simpler emoji into more
complex ones, or alter skin tones, or colors, etc. Reportedly the
casefolding code will also fold a red heart into a black one.
So. To our filename support test, let's add various colors of heart
emoji and various skin tones of heart-hands; and compound emoji
consisting of multiple emoji glued together with zero width joiners.
This actually caused a buffer overflow in the string-escaping functions
of xfs_scrub phase 5 because I hadn't anticipated that we'd end up with
a filename consisting *entirely* of nonprinting bytes.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Chao Yu [Tue, 25 Mar 2025 08:34:28 +0000 (16:34 +0800)]
f2fs: test dirty status handling on database file
This is a regression testcase to check whether we will handle database
inode dirty status correctly:
1. create a regular file, and write data into the file
2. start transaction on the file (via F2FS_IOC_START_ATOMIC_WRITE)
3. write transaction data to the file
4. rename the file
5. commit and end the transaction (via F2FS_IOC_COMMIT_ATOMIC_WRITE)
6. drop caches in order to call f2fs_evict_inode()
It expects kernel panic will gone after we apply commit 03511e936916
("f2fs: fix inconsistent dirty state of atomic file").
Cc: Jaegeuk Kim <jaegeuk@kernel.org> Cc: Daeho Jeong <daehojeong@google.com> Cc: Jianan Huang <huangjianan@xiaomi.com> Signed-off-by: Chao Yu <chao@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Chao Yu [Tue, 25 Mar 2025 12:58:24 +0000 (20:58 +0800)]
f2fs: new test to detect and repair nlink corruption
This is a regression test to check whether fsck can handle corrupted
nlinks correctly, it uses inject.f2fs to inject nlinks w/ wrong value,
and expects fsck.f2fs can detect such corruption and do the repair.
Cc: Jaegeuk Kim <jaegeuk@kernel.org> Signed-off-by: Chao Yu <chao@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Chao Yu [Tue, 25 Mar 2025 12:58:23 +0000 (20:58 +0800)]
common/rc: introduce _check_f2fs_filesystem()
_check_generic_filesystem() will fail the test once it detects
corruption, let's introduce _check_f2fs_filesystem() to just check
filesystem w/ --dry-run option, and return the error number, then
let caller to decide whether the corruption is as expected or not.
Cc: Jaegeuk Kim <jaegeuk@kernel.org> Signed-off-by: Chao Yu <chao@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Chao Yu [Tue, 25 Mar 2025 12:58:20 +0000 (20:58 +0800)]
common/config: export F2FS_INJECT_PROG
export F2FS_INJECT_PROG w/ inject.f2fs, it can be used for fault injection.
Cc: Jaegeuk Kim <jaegeuk@kernel.org> Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Chao Yu <chao@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Chao Yu [Tue, 25 Mar 2025 12:58:19 +0000 (20:58 +0800)]
common/config: remove redundant export variables
F2FS_IO_PROG and TIMEOUT_PROG are exported twice, remove the redudant
one.
Cc: Jaegeuk Kim <jaegeuk@kernel.org> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Chao Yu <chao@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Thu, 20 Mar 2025 19:28:12 +0000 (03:28 +0800)]
fstests: remove run_setsid test way from check
This patch partially revert most of the 88d60f434 ("common: fix pkill
by running test program in a separate session"), it does:
1. Remove run_setsid script
2. Remove all run_setsid related things from check
3. Keep the change in _scratch_xfs_stress_scrub_cleanup() which is a
bug fix for xfs_scrub test.
Reviewed-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Thu, 20 Mar 2025 19:28:11 +0000 (03:28 +0800)]
fstests: remove privatens test way from check
This patch totally revert: ce7f796ad check: remove the deprecation of sessionid 336784e3d check: disable HAVE_PRIVATENS by default 949bdf8ea check: deprecate using process sessions to isolate test instances
and partially revert: 247ab01fa check: run tests in a private pid/mount namespace
So it does:
1. Remove "run_privatens" related things from xfstests/check, due to
check doesn't need that test way.
2. Keep run_privatens script and related changes for check-parallel.
Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Sandeen [Fri, 14 Mar 2025 18:37:45 +0000 (13:37 -0500)]
common/rc: explicitly test for engine availability in _require_fio
The current test in _require_fio (--warnings-fatal --showcmd) does not
fail if an invalid/unavailable io engine is specified.
Add an explicit test that every requested io engine in the job file
is actually available.
Also, remove the "ioe_e4defrag" entries in the [global] stanza of several
ext4 tests which use fio jobfiles. While ioengines can be set in the
[global] section, they can also be overridden in individual, subsequent
stanzas. In each affected test (ext4/301, ext4/302, ext4/303, and
ext4/304) every individual stanza after [global]re-specifies an ioengine;
either with ioengine=e4defrag or ioengine=libaio.
Because of this re-specification, the ioengine in the [global] section
is ignored. This is a good thing, because ioe_e4defrag is not a valid
ioengine, and would fail this new hand-rolled check, even though fio
did not complain.
So rather than over-complicate this new check, simply remove the unused,
invalid "ioengine=ioe_e4defrag" lines in these tests.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Sandeen [Mon, 10 Mar 2025 18:29:08 +0000 (13:29 -0500)]
lib: make a few symbols static
There are a few symbols in lib/tlibio.c which should be static,
and sparse notices this so fix it.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Sandeen [Mon, 10 Mar 2025 18:29:07 +0000 (13:29 -0500)]
lib: replace aiocb_t with struct aiocb
aiocb_t isn't defined anywhere, use struct aiocb instead,
to make sparse happy.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Sandeen [Mon, 10 Mar 2025 18:29:06 +0000 (13:29 -0500)]
lib: fix empty arg function prototypes
Several function prototypes used () when in fact they take
arguments. Fix those to make sparse happy.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Sandeen [Mon, 10 Mar 2025 18:29:05 +0000 (13:29 -0500)]
lib: Fix non-ANSI function declarations
lib/ was full of non-ANSI function declarations, fix them to make
sparse happier.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Sandeen [Tue, 11 Mar 2025 13:46:48 +0000 (08:46 -0500)]
treewide: check for #ifdef __linux__ not linux
There are several #ifdef linux guards in the code, which caused
a few sparse warnings, because while gcc defines both linux
and __linux__, sparse defines only __linux__. So, switch our
guards to check for __linux__ which is defined by both gcc
and sparse.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Sandeen [Mon, 10 Mar 2025 18:29:03 +0000 (13:29 -0500)]
fstests: enable sparse checking with make C=[12]
Enable "make C=1" sparse checking when files get rebuilt. To check
all files, run "make clean" first.
Enable "make C=2" sparse checking of all files without rebuilding them.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Sandeen [Mon, 10 Mar 2025 17:55:51 +0000 (12:55 -0500)]
common/rc: use getent when available
Zorro noticed that on systems using nss-altfiles, some tests failed when
trying to parse /etc/passwd directly. The "getent" command does the
right thing in this case, so let's use it as long as it's available.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Skip the test on older kernels that don't implement mount_setattr and
open_tree syscalls, instead of failing.
Signed-off-by: Anthony Iliopoulos <ailiop@suse.com> Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Luis Henriques [Tue, 4 Mar 2025 12:21:19 +0000 (12:21 +0000)]
src/locktest: add missing struct cast with syscall getsockname()
The usage of variable 'myAddr' (struct sockaddr_in) is being casted into a
'struct sockaddr' when used with bind() and with connect(). This patch
adds that cast when used with getsockname() as well, otherwise we'll get the
following error:
locktest.c: In function 'main':
locktest.c:1155:39: error: passing argument 2 of 'getsockname' from incompatible pointer type [-Wincompatible-pointer-types]
1155 | if (getsockname(s_fd, &myAddr, &addr_len)) {
| ^~~~~~~
| |
| struct sockaddr_in *
In file included from /usr/include/fortify/sys/socket.h:23,
from locktest.c:19:
/usr/include/sys/socket.h:391:23: note: expected 'struct sockaddr * restrict' but argument is of type 'struct sockaddr_in *'
391 | int getsockname (int, struct sockaddr *__restrict, socklen_t *__restrict);
| ^
Signed-off-by: Luis Henriques <luis@igalia.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Dave Chinner [Thu, 6 Mar 2025 21:34:50 +0000 (08:34 +1100)]
fstests: remove old tools from tools directory
ag-wipe, db-walk, fs-walk and interop are all old 2000s era SGI QA
scripts that are either not usable anymore of have been superceded
by much more recent functionality.
interop only runs on machines with the host named "bruce" or
"whack", requires NFS mounts from a machine named "snort" and access
to a specific user's home dir. I know what all these were; they were
test and build machines in SGI's engineering office in Melbourne,
Australia. That went away in ~2009....
ag-wipe is unused and has been superceded by multiple different sets of
functionality (e.g. ./check --large-fs).
db-walk is unused by fstests and now is native xfs_db functionality.
fs-walk is dumping the md5sum and stat info of every file in a
directory heirachy and only used by the interop script....
Remove them all.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Dave Chinner [Thu, 6 Mar 2025 21:34:49 +0000 (08:34 +1100)]
fstests: remove tools/auto-qa
This is a 2000s era SGI test script that no-one uses anymore. It
likely doesn't even work because it is dependent on CVS and/or
the SGI proprietary ptools revision control system for source
control. Hence it clearly hasn't been used for years, so remove it.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Dave Chinner [Thu, 6 Mar 2025 21:34:48 +0000 (08:34 +1100)]
fstests: remove crash script
This is an old, early 2000s SGI era script for crash and recovery
testing XFS filesystems. We have much better ways of doing this
today with tools like dm-error, dm-flakey, etc. Nobody has been
using this script for years, so remove it.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Dave Chinner [Thu, 6 Mar 2025 21:34:47 +0000 (08:34 +1100)]
fstests: remove old soak script
There is an old script for soak testing using check in the source
tree called "soak". All it does is run fsstress in a loop on the
scratch device, but to do that it needs to include the entire
fstests config/setup scripts.
These days we have much better soak testing capabilities (e.g. the
soak and stress groups, with controllable iterations, durations,
load, etc) so there's no point in keeping old scripts like this
around that add weird dependencies to the core infrastructure.
Remove it.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Dave Chinner [Thu, 6 Mar 2025 21:34:46 +0000 (08:34 +1100)]
fstests: remove setup script
THere is a script that will echo the current setup of the fstests
directory. This is largely the same output as running 'check -n' to
do a dry-run outputs. Hence this script is redundant, not likely to
be widely used and is easily replaced with check itself. Remove it.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Thu, 6 Mar 2025 09:49:23 +0000 (17:49 +0800)]
check: disable HAVE_PRIVATENS by default
Currently we have 3 ways to run a test case in _run_seq():
if [ -n "${HAVE_PRIVATENS}" ]; then
./tools/run_privatens "./$seq"
...
elif [ -n "${HAVE_SYSTEMD_SCOPES}" ]; then
systemd-run --quiet --unit "${unit}" --scope \
./tools/run_setsid "./$seq" &
...
else
./tools/run_setsid "./$seq" &
...
fi
The "privatens" way brings in some regressions. We need more time
to develop and test this way, it's not time let it to be the
first default choice, so isolate the HAVE_PRIVATENS initialization
by a TRY_PRIVATENS parameter, and disable it by default.
Set TRY_PRIVATENS=yes to give "privatens" a try, otherwise run in
old ways. This patch can be removed after "privatens" way is stable.
Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:15 +0000 (13:47 -0800)]
common/xfs: fix _xfs_get_file_block_size when rtinherit is set and no rt section
It's possible for the sysadmin to set rtinherit on the directory tree
even if there isn't a realtime section attached to the filesystem. When
this is the case, the realtime flag is /not/ passed to new files, and
file data is written to the data device. The file allocation unit for
the file is the fs blocksize, and it is not correct to use the rt
extent.
fstests can be fooled into doing the incorrect thing if test runner puts
'-d rtinherit=1 -r extsize=28k' into MKFS_OPTIONS without configuring a
realtime device. This causes many tests to do the wrong thing because
they think they must operate on units of 28k (and not 4k). Fix this.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:15 +0000 (13:47 -0800)]
generic/331,xfs/240: support files that skip delayed allocation
The goal of this test is to ensure that log recovery finishes a copy on
write operation in the event of temporary media errors. It's important
that the test observe some sort of IO error once we switch the scratch
device to fail all IOs, but regrettably the test encoded the specific
behavior of XFS and btrfs when the test was written -- the aio write
to the page cache doesn't have to touch the disk and succeeds, and the
fdatasync flushes things to disk and hits the IO error.
However, this is not how things work on the XFS realtime device. There
is no delalloc on realtime when the rt extent size > 1 fsblock (or on
any file with extent size hints), so the aio write allocates an
unwritten extent to stage the write. The allocation fails due to EIO,
so it's the write call that fails. Therefore, all we need to do is to
detect an IO error at any point between the write and the fdatasync call
to be satisfied that the test does what we want to do.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:13 +0000 (13:47 -0800)]
fuzzy: create missing fuzz tests for rt rmap btrees
Back when I first created the fuzz tests for the realtime rmap btree, I
forgot a couple of things. Add tests to fuzz rtrmap btree leaf records,
and node keys.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:13 +0000 (13:47 -0800)]
populate: adjust rtrmap calculations for rtgroups
Now that we've sharded the realtime volume and created per-group rmap
btrees, we need to adjust downward the size of rtrmapbt records since
the block counts are now 32-bit instead of 64-bit.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:12 +0000 (13:47 -0800)]
xfs/104: use _scratch_mkfs_sized
Use _scratch_mkfs_sized instead of opencoding the small fs creation
logic because the helper scales down the size of the rt volume to match
the data volume. This means the format won't fail if SCRATCH_RTDEV is
quite large.
Also fix some incorrect bash usage of "$@" and remove the leading
underscore because it's a private test function.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:12 +0000 (13:47 -0800)]
xfs/291: use _scratch_mkfs_sized instead of opencoding the logic
Now that _scratch_mkfs_xfs_sized accepts arbitrary mkfs arguments, this
test doesn't need to open-code the helper just to add the dir block and
log size arguments. Clean this up, which will also help us to avoid
problems with rtrmap if the rt devices is large enough to cause format
failures with the 133M data device size.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:12 +0000 (13:47 -0800)]
xfs/3{43,32}: adapt tests for rt extent size greater than 1
Both of these tests for the realtime volume can fail when the rt extent
size is larger than a single block.
332 is a read-write functionality test that encodes md5sum in the
output, so we need to skip it if $blksz isn't congruent with the extent
size, because the fcollapse call will fail.
343 is a test of the rmap btree, so the fix here is simpler -- make
$blksz the file allocation unit, and get rid of the md5sum in the
golden output.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:12 +0000 (13:47 -0800)]
xfs/341: update test for rtgroup-based rmap
Now that we're sharding the realtime volume into multiple allocation
groups, update this test to reflect the new reality. The realtime rmap
btree record and key sizes have shrunk, and we can't guarantee that a
quick file write actually hits the same rt group as the one we fuzzed,
so eliminate the file write test since we're really only curious if
xfs_repair will fix the problem.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:11 +0000 (13:47 -0800)]
xfs: fix various problems with fsmap detecting the data device
Various tests of realtime rmap functionality assumed that the data
device could be picked out from the GETFSMAP output by looking for
static fs metadata. This is no longer true, since rtgroups filesystems
write a superblock header at the start of the rt device, so update these
tests.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:11 +0000 (13:47 -0800)]
fuzz: for fuzzing the rtrmapbt, find the path to the rt rmap btree file
The fs population code creates a realtime rmap btree in /some/ realtime
group with at least two levels. This rmapbt file isn't necessarily the
one for group 0, so we need to find it programmatically.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
This is due to the fact that SCRATCH_DEV is temporarily reassigned to
the regular file. That path is passed straight through _scratch_mount
to _xfs_prepare_for_eio_shutdown, but that helper _fails because the
"dev" argument isn't actually a path to a block device.
Fix this by porting it to the new common/metadump code that we merged
last year.
Cc: <fstests@vger.kernel.org> # v2024.12.08 Fixes: 1a49022fab9b4d ("fstests: always use fail-at-unmount semantics for XFS") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:10 +0000 (13:47 -0800)]
xfs: fix tests that try to access the realtime rmap inode
The realtime rmap tests were added to fstests a long time ago. Since
they were added, we decided to create a metadata file directory
structure instead of adding more fields to the superblock. Therefore,
fix all the tests that try to access these paths.
While we're at it, fix xfs/409 to run the *online* scrub program like
it's supposed to. xfs/408 is the fuzzer for xfs_repair testing.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:10 +0000 (13:47 -0800)]
common: test statfs reporting with project quota
Create a test to check that statfs on a directory tree with a project
quota will report the quota limit and available blocks; and that the
available blocks reported doesn't exceed that of the whole filesystem.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:09 +0000 (13:47 -0800)]
common: enable testing of realtime quota when supported
If the kernel advertises realtime quota support, test it.
However, this has a plot twist -- because rt quota only works if the xfs
is formatted with rtgroups, we have to mount a filesystem to see if
rtquota is actually supported. Since it's time consuming to format and
mount the scratch filesystem, we'll assume that the test and scratch
fses have the same support.
This will cause problems if one sets SCRATCH_RTDEV but not TEST_RTDEV.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:09 +0000 (13:47 -0800)]
xfs: fix quota detection in fuzz tests
With metadir, quota options persist until they are changed by mount
options. Therefore, we can set the quota flags in MKFS_OPTIONS and
needn't supply them in MOUNT_OPTIONS. Unfortunately, this means that we
cannot grep the MOUNT_OPTIONS anymore; we must mount the fs and run
src/feature to determine if quotas are enabled.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:08 +0000 (13:47 -0800)]
common/xfs: capture realtime devices during metadump/mdrestore
If xfs_metadump supports the -r switch to capture the contents of
realtime devices and there is a realtime device, add the option to the
command line to enable preservation.
Similarly, if the dump file could restore to an external scratch rtdev,
pass the -r option to mdrestore so that we can restore rtdev contents.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:07 +0000 (13:47 -0800)]
xfs/449: update test to know about xfs_db -R
The realtime groups feature added a -R flag to xfs_db so that users can
pass in the realtime device. Since we've now modified the
_scratch_xfs_db to use this facility, we can update the test to do exact
comparisons of the xfs_db info command against the mkfs output.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:07 +0000 (13:47 -0800)]
xfs/185: update for rtgroups
This old test is a bit too fixated on exact rt allocator behavior. With
rtgroups enabled, we can end up with one large contiguous region that's
split into multiple bmbt mappings to avoid crossing rtgroup boundaries.
The realtime superblock throws another twist into the mix because the
first rtx will always be in use, which can shift the start of the
physical space mappings by up to 1 rtx.
Also fix a bug where we'd try to fallocate the total number of rtx,
whereas we should be asking for the number of free rtx to avoid ENOSPC
errors.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:06 +0000 (13:47 -0800)]
punch-alternating: detect xfs realtime files with large allocation units
For files on the XFS realtime volume, it's possible that the file
allocation unit (aka the minimum size we have to punch to deallocate
file blocks) could be greater than a single fs block. This utility
assumed that it's always possible to punch a single fs block, but for
these types of files, all that does is zeroes the page cache. While
that's what most *user applications* want, fstests uses punching to
fragment file mapping metadata and/or fragment free space, so adapt this
test for that purpose by detecting realtime files.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:06 +0000 (13:47 -0800)]
common/populate: use metadump v2 format by default for fs metadata snapshots
When we're snapshotting filesystem metadata after creating a populated
filesystem, force the creation of metadump v2 files by default to
exercise the new format, since xfs_metadump continues to use the v1
format unless told otherwise.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:06 +0000 (13:47 -0800)]
common/ext4: reformat external logs during mdrestore operations
The e2image file format doesn't support the capture of external log
devices, which means that mdrestore ought to reformat the external log
to get the restored filesystem to work again. The common/populate code
could already do this, so push it to the common ext4 helper.
While we're at it, fix the uncareful usage of SCRATCH_LOGDEV in the
populate code.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:05 +0000 (13:47 -0800)]
common/populate: refactor caching of metadumps to a helper
Hoist out of _scratch_populate_cached all the code that we use to save a
metadump of the populated filesystem. We're going to make this more
involved for XFS in the next few patches so that we can take advantage
of the new support for external devices in metadump/mdrestore.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:05 +0000 (13:47 -0800)]
fstests: test mkfs.xfs protofiles with xattr support
Make sure we can do protofiles with xattr support.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:03 +0000 (13:47 -0800)]
common/populate: label newly created xfs filesystems
When we're creating fully populated filesystems, add an obviously weird
label to the filesystem images so that it's obvious that it's a test
filesystem.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:03 +0000 (13:47 -0800)]
xfs/122: disable this test for any codebase that knows about metadir
All of the ondisk structure size checks from this test were copied to
the build time checks in xfs_ondisk.h. This means that the kernel and
xfsprogs build processes check the structure sizes, which means that
fstests no longer needs to do that.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:03 +0000 (13:47 -0800)]
xfs: create fuzz tests for metadata directories
Create fuzz tests to make sure that all the validation works for
metadata directories and subdirectories. The metadir fuzzer tests are
tagged 'realtime' to force creation of an interesting metadata directory
tree full of rtgroups inodes.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:02 +0000 (13:47 -0800)]
xfs/509: adjust inumbers accounting for metadata directories
The INUMBERS ioctl exports data from the inode btree directly -- the
number of inodes it reports is taken from ir_freemask and includes all
the files in the metadata directory tree. BULKSTAT, on the other hand,
only reports non-metadata files. When metadir is enabled, this will
(eventually) cause a discrepancy in the inode counts that is large
enough to exceed the tolerances, thereby causing a test failure.
Correct this by counting the files in the metadata directory and
subtracting that from the INUMBERS totals.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:02 +0000 (13:47 -0800)]
xfs/{050,144,153,299,330}: update quota reports to handle metadir trees
Prior to the new metadir feature in XFS, the rtbitmap and rtsummary
files were included in icount, though their bcount contribution is zero
due to rt and quota not being supported together. With the new metadir
feature in XFS, no files in the metadata directory tree are counted in
quota.
Hence we must adjust the icount of any quota report down by two to avoid
breaking golden outputs.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:02 +0000 (13:47 -0800)]
common/repair: patch up repair sb inode value complaints
Now that we've refactored xfs_repair to be more consistent in how it
reports unexpected superblock inode pointer values, we have to fix up
the fstests repair filters to emulate the old golden output.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:01 +0000 (13:47 -0800)]
xfs/{030,033,178}: forcibly disable metadata directory trees
The golden output for thests tests encode the xfs_repair output when we
fuzz various parts of the filesystem. With metadata directory trees
enabled, however, the golden output changes dramatically to reflect
reconstruction of the metadata directory tree.
To avoid regressions, add a helper to force metadata directories off via
MKFS_OPTIONS.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:01 +0000 (13:47 -0800)]
various: fix finding metadata inode numbers when metadir is enabled
There are a number of tests that use xfs_db to examine the contents of
metadata inodes to check correct functioning. The logic is scattered
everywhere and won't work with metadata directory trees, so make a
shared helper to find these inodes.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:01 +0000 (13:47 -0800)]
xfs/349: reclassify this test as not dangerous
This test creates a filesystem with all known types of metadata. It
doesn't fuzz anything, nor does it actually repair anything. Hence we
can put it in the auto group and drop the dangerous tags.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:01 +0000 (13:47 -0800)]
xfs/28[56],xfs/56[56]: add to the auto group
Enable /some/ testing of online fsck for everybody by adding these two
fsstress tests and two fsx tests to the auto group. At this time I
don't have any plans to do the same for the other scrub stress tests.
Q: Can you explain why only these four?
A: The rest of the stress tests pick /one/ metadata type and race
scrub/repair of it against fsstress by invoking xfs_io repeatedly.
xfs/28[56] runs the whole xfs_scrub program repeatedly so we get to
exercise all of them in a single test.
The more specific tests make it a lot easier to debug problems.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:01 +0000 (13:47 -0800)]
misc: remove the dangerous_scrub group
Now that online fsck has been in the upstream kernel for 8 months, I
think it's stabilized enough that the scrub functionality tests don't
need to hide behind the "dangerous" label anymore. Move them all to the
scrub group and delete the dangerous_scrub group.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:00 +0000 (13:47 -0800)]
misc: fix misclassification of xfs_repair fuzz tests
All the tests in the "fuzzers_repair" group actually test xfs_repair,
not scrub. Therefore, we should remove them all from 'dangerous_scrub'
and put them in the 'repair' group.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:46:59 +0000 (13:46 -0800)]
misc: rename the dangerous_norepair group to fuzzers_norepair
I've been running the norepair fuzzers (aka the verifier checks) for
most of the time that online fsck has been in development, and I haven't
seen any of the VMs crash in a couple of years. I think it's time that
these tests stopped hiding behind the "dangerous" label.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:46:59 +0000 (13:46 -0800)]
misc: rename the dangerous_bothrepair group to fuzzers_bothrepair
Now that online fsck has been in the upstream kernel for 8 months, I
think it's stabilized enough that the scrub + repair functionality tests
don't need to hide behind the "dangerous" label anymore.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
projects / users / hch / xfstests-dev.git / log