Darrick J. Wong [Thu, 20 Feb 2025 21:47:11 +0000 (13:47 -0800)]
fuzz: for fuzzing the rtrmapbt, find the path to the rt rmap btree file
The fs population code creates a realtime rmap btree in /some/ realtime
group with at least two levels. This rmapbt file isn't necessarily the
one for group 0, so we need to find it programmatically.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
This is due to the fact that SCRATCH_DEV is temporarily reassigned to
the regular file. That path is passed straight through _scratch_mount
to _xfs_prepare_for_eio_shutdown, but that helper _fails because the
"dev" argument isn't actually a path to a block device.
Fix this by porting it to the new common/metadump code that we merged
last year.
Cc: <fstests@vger.kernel.org> # v2024.12.08 Fixes: 1a49022fab9b4d ("fstests: always use fail-at-unmount semantics for XFS") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:10 +0000 (13:47 -0800)]
xfs: fix tests that try to access the realtime rmap inode
The realtime rmap tests were added to fstests a long time ago. Since
they were added, we decided to create a metadata file directory
structure instead of adding more fields to the superblock. Therefore,
fix all the tests that try to access these paths.
While we're at it, fix xfs/409 to run the *online* scrub program like
it's supposed to. xfs/408 is the fuzzer for xfs_repair testing.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:10 +0000 (13:47 -0800)]
common: test statfs reporting with project quota
Create a test to check that statfs on a directory tree with a project
quota will report the quota limit and available blocks; and that the
available blocks reported doesn't exceed that of the whole filesystem.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:09 +0000 (13:47 -0800)]
common: enable testing of realtime quota when supported
If the kernel advertises realtime quota support, test it.
However, this has a plot twist -- because rt quota only works if the xfs
is formatted with rtgroups, we have to mount a filesystem to see if
rtquota is actually supported. Since it's time consuming to format and
mount the scratch filesystem, we'll assume that the test and scratch
fses have the same support.
This will cause problems if one sets SCRATCH_RTDEV but not TEST_RTDEV.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:09 +0000 (13:47 -0800)]
xfs: fix quota detection in fuzz tests
With metadir, quota options persist until they are changed by mount
options. Therefore, we can set the quota flags in MKFS_OPTIONS and
needn't supply them in MOUNT_OPTIONS. Unfortunately, this means that we
cannot grep the MOUNT_OPTIONS anymore; we must mount the fs and run
src/feature to determine if quotas are enabled.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:08 +0000 (13:47 -0800)]
common/xfs: capture realtime devices during metadump/mdrestore
If xfs_metadump supports the -r switch to capture the contents of
realtime devices and there is a realtime device, add the option to the
command line to enable preservation.
Similarly, if the dump file could restore to an external scratch rtdev,
pass the -r option to mdrestore so that we can restore rtdev contents.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:07 +0000 (13:47 -0800)]
xfs/449: update test to know about xfs_db -R
The realtime groups feature added a -R flag to xfs_db so that users can
pass in the realtime device. Since we've now modified the
_scratch_xfs_db to use this facility, we can update the test to do exact
comparisons of the xfs_db info command against the mkfs output.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:07 +0000 (13:47 -0800)]
xfs/185: update for rtgroups
This old test is a bit too fixated on exact rt allocator behavior. With
rtgroups enabled, we can end up with one large contiguous region that's
split into multiple bmbt mappings to avoid crossing rtgroup boundaries.
The realtime superblock throws another twist into the mix because the
first rtx will always be in use, which can shift the start of the
physical space mappings by up to 1 rtx.
Also fix a bug where we'd try to fallocate the total number of rtx,
whereas we should be asking for the number of free rtx to avoid ENOSPC
errors.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:06 +0000 (13:47 -0800)]
punch-alternating: detect xfs realtime files with large allocation units
For files on the XFS realtime volume, it's possible that the file
allocation unit (aka the minimum size we have to punch to deallocate
file blocks) could be greater than a single fs block. This utility
assumed that it's always possible to punch a single fs block, but for
these types of files, all that does is zeroes the page cache. While
that's what most *user applications* want, fstests uses punching to
fragment file mapping metadata and/or fragment free space, so adapt this
test for that purpose by detecting realtime files.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:06 +0000 (13:47 -0800)]
common/populate: use metadump v2 format by default for fs metadata snapshots
When we're snapshotting filesystem metadata after creating a populated
filesystem, force the creation of metadump v2 files by default to
exercise the new format, since xfs_metadump continues to use the v1
format unless told otherwise.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:06 +0000 (13:47 -0800)]
common/ext4: reformat external logs during mdrestore operations
The e2image file format doesn't support the capture of external log
devices, which means that mdrestore ought to reformat the external log
to get the restored filesystem to work again. The common/populate code
could already do this, so push it to the common ext4 helper.
While we're at it, fix the uncareful usage of SCRATCH_LOGDEV in the
populate code.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:05 +0000 (13:47 -0800)]
common/populate: refactor caching of metadumps to a helper
Hoist out of _scratch_populate_cached all the code that we use to save a
metadump of the populated filesystem. We're going to make this more
involved for XFS in the next few patches so that we can take advantage
of the new support for external devices in metadump/mdrestore.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:05 +0000 (13:47 -0800)]
fstests: test mkfs.xfs protofiles with xattr support
Make sure we can do protofiles with xattr support.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:03 +0000 (13:47 -0800)]
common/populate: label newly created xfs filesystems
When we're creating fully populated filesystems, add an obviously weird
label to the filesystem images so that it's obvious that it's a test
filesystem.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:03 +0000 (13:47 -0800)]
xfs/122: disable this test for any codebase that knows about metadir
All of the ondisk structure size checks from this test were copied to
the build time checks in xfs_ondisk.h. This means that the kernel and
xfsprogs build processes check the structure sizes, which means that
fstests no longer needs to do that.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:03 +0000 (13:47 -0800)]
xfs: create fuzz tests for metadata directories
Create fuzz tests to make sure that all the validation works for
metadata directories and subdirectories. The metadir fuzzer tests are
tagged 'realtime' to force creation of an interesting metadata directory
tree full of rtgroups inodes.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:02 +0000 (13:47 -0800)]
xfs/509: adjust inumbers accounting for metadata directories
The INUMBERS ioctl exports data from the inode btree directly -- the
number of inodes it reports is taken from ir_freemask and includes all
the files in the metadata directory tree. BULKSTAT, on the other hand,
only reports non-metadata files. When metadir is enabled, this will
(eventually) cause a discrepancy in the inode counts that is large
enough to exceed the tolerances, thereby causing a test failure.
Correct this by counting the files in the metadata directory and
subtracting that from the INUMBERS totals.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:02 +0000 (13:47 -0800)]
xfs/{050,144,153,299,330}: update quota reports to handle metadir trees
Prior to the new metadir feature in XFS, the rtbitmap and rtsummary
files were included in icount, though their bcount contribution is zero
due to rt and quota not being supported together. With the new metadir
feature in XFS, no files in the metadata directory tree are counted in
quota.
Hence we must adjust the icount of any quota report down by two to avoid
breaking golden outputs.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:02 +0000 (13:47 -0800)]
common/repair: patch up repair sb inode value complaints
Now that we've refactored xfs_repair to be more consistent in how it
reports unexpected superblock inode pointer values, we have to fix up
the fstests repair filters to emulate the old golden output.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:01 +0000 (13:47 -0800)]
xfs/{030,033,178}: forcibly disable metadata directory trees
The golden output for thests tests encode the xfs_repair output when we
fuzz various parts of the filesystem. With metadata directory trees
enabled, however, the golden output changes dramatically to reflect
reconstruction of the metadata directory tree.
To avoid regressions, add a helper to force metadata directories off via
MKFS_OPTIONS.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:01 +0000 (13:47 -0800)]
various: fix finding metadata inode numbers when metadir is enabled
There are a number of tests that use xfs_db to examine the contents of
metadata inodes to check correct functioning. The logic is scattered
everywhere and won't work with metadata directory trees, so make a
shared helper to find these inodes.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:01 +0000 (13:47 -0800)]
xfs/349: reclassify this test as not dangerous
This test creates a filesystem with all known types of metadata. It
doesn't fuzz anything, nor does it actually repair anything. Hence we
can put it in the auto group and drop the dangerous tags.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:01 +0000 (13:47 -0800)]
xfs/28[56],xfs/56[56]: add to the auto group
Enable /some/ testing of online fsck for everybody by adding these two
fsstress tests and two fsx tests to the auto group. At this time I
don't have any plans to do the same for the other scrub stress tests.
Q: Can you explain why only these four?
A: The rest of the stress tests pick /one/ metadata type and race
scrub/repair of it against fsstress by invoking xfs_io repeatedly.
xfs/28[56] runs the whole xfs_scrub program repeatedly so we get to
exercise all of them in a single test.
The more specific tests make it a lot easier to debug problems.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:01 +0000 (13:47 -0800)]
misc: remove the dangerous_scrub group
Now that online fsck has been in the upstream kernel for 8 months, I
think it's stabilized enough that the scrub functionality tests don't
need to hide behind the "dangerous" label anymore. Move them all to the
scrub group and delete the dangerous_scrub group.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:47:00 +0000 (13:47 -0800)]
misc: fix misclassification of xfs_repair fuzz tests
All the tests in the "fuzzers_repair" group actually test xfs_repair,
not scrub. Therefore, we should remove them all from 'dangerous_scrub'
and put them in the 'repair' group.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:46:59 +0000 (13:46 -0800)]
misc: rename the dangerous_norepair group to fuzzers_norepair
I've been running the norepair fuzzers (aka the verifier checks) for
most of the time that online fsck has been in development, and I haven't
seen any of the VMs crash in a couple of years. I think it's time that
these tests stopped hiding behind the "dangerous" label.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:46:59 +0000 (13:46 -0800)]
misc: rename the dangerous_bothrepair group to fuzzers_bothrepair
Now that online fsck has been in the upstream kernel for 8 months, I
think it's stabilized enough that the scrub + repair functionality tests
don't need to hide behind the "dangerous" label anymore.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:46:59 +0000 (13:46 -0800)]
misc: rename the dangerous_online_repair group to fuzzers_online_repair
Now that online fsck has been in the upstream kernel for 8 months, I
think it's stabilized enough that the scrub functionality tests don't
need to hide behind the "dangerous" label anymore. It's time for more
widespread testing.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:46:58 +0000 (13:46 -0800)]
misc: drop the dangerous label from xfs_scrub fsstress tests
Now that online fsck has been in the upstream kernel for 8 months, I
think it's stabilized enough that we don't need to hide the stress tests
behind the "dangerous" label anymore.
Also rename fsstress_repair to fsstress_online_repair to be consistent
with the online_repair group.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:46:57 +0000 (13:46 -0800)]
dio_writeback_race: align the directio buffer to base page size
There's no need to align the memory buffer for the direcio write to the
file allocation unit size; base page size will do. This fixes the
following error when generic/761 is run against an XFS rt filesystem
with a 28k rtextsize:
QA output created by 761
+failed to allocate aligned memory
+cat: /opt/foobar: No such file or directory
Silence is golden
Cc: <fstests@vger.kernel.org> # v2025.02.16 Fixes: 17fb49493426ad ("fstests: add a generic test to verify direct IO writes with buffer contents change") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 20 Feb 2025 21:46:57 +0000 (13:46 -0800)]
dio-writeback-race: fix missing mode in O_CREAT
Fix this build error:
In file included from /usr/include/fcntl.h:314,
from dio-writeback-race.c:40:
In function 'open',
inlined from 'main' at dio-writeback-race.c:110:7:
/usr/include/x86_64-linux-gnu/bits/fcntl2.h:50:11: error: call to '__open_missing_mode' declared with attribute error: open with O_CREAT or O_TMPFILE in second argument needs 3 arguments
50 | __open_missing_mode ();
| ^~~~~~~~~~~~~~~~~~~~~~
Cc: <fstests@vger.kernel.org> # v2025.02.16 Fixes: 17fb49493426ad ("fstests: add a generic test to verify direct IO writes with buffer contents change") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Naohiro Aota [Mon, 3 Mar 2025 06:42:09 +0000 (15:42 +0900)]
tools/run_privatens: check if it is mount point for --make-private
While /tmp is mounted with tmpfs in the most setup, it still can be a non-mount
point. For example, I'm running the fstests in a container, which does not
mount /tmp inside the container.
Running any test case on such system results in having the following error
printed, which leads to all the test cases fail due to the output difference.
mount: /tmp: not mount point or bad option.
dmesg(1) may have more information after failed mount system call.
These lines are printed by the "mount --make-private" command. So, fix that by
using mountpoint command to check if the directory is a mount point or not.
Fixes: 247ab01fa227 ("check: run tests in a private pid/mount namespace") Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Fri, 28 Feb 2025 16:27:55 +0000 (16:27 +0000)]
btrfs/080: fix sporadic failures starting with kernel 6.13
Since kernel 6.13, namely since commit c87c299776e4 ("btrfs: make buffered
write to copy one page a time"), the test can sporadically fail with an
unexpected digests for files in snapshots. This is because after that
commit the pages for a buffered write are prepared and dirtied one by one,
and no longer in batches up to 512 pages (on x86_64), so a snapshot that
is created in the middle of a buffered write can result in a version of
the file where only a part of a buffered write got caught, for example if
a snapshot is created while doing the 32K write for file range [0, 32K),
we can end up a file in the snapshot where only the first 8K (2 pages) of
the write are visible, since the remaining 24K were not yet processed by
the write task. Before that kernel commit, this didn't happen since we
processed all the pages in a single batch and while holding the whole
range locked in the inode's io tree.
This is easy to observe by adding an "od -A d -t x1" call to the test
before the _fail statement when we get an unexpected file digest:
btrfs/080 32s ... [failed, exit status 1]- output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/080.out.bad)
--- tests/btrfs/080.out 2020-06-10 19:29:03.814519074 +0100
+++ /home/fdmanana/git/hub/xfstests/results//btrfs/080.out.bad 2025-02-27 17:12:08.410696958 +0000
@@ -1,2 +1,6 @@
QA output created by 080
-Silence is golden
+0000000 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa
+*
+0008192
+Unexpected digest for file /home/fdmanana/btrfs-tests/scratch_1/17_11_56_146646257_snap/foobar_50
+(see /home/fdmanana/git/hub/xfstests/results//btrfs/080.full for details)
...
(Run 'diff -u /home/fdmanana/git/hub/xfstests/tests/btrfs/080.out /home/fdmanana/git/hub/xfstests/results//btrfs/080.out.bad' to see the entire diff)
Ran: btrfs/080
Failures: btrfs/080
Failed 1 of 1 tests
The files are created like this while snapshots are created in parallel:
So with the kernel behaviour before 6.13 we expected the snapshot to
contain any of the following versions of the file:
1) Empty file;
2) 32K file reflecting only the first buffered write;
3) A file with a size of 49154 bytes reflecting both buffered writes;
4) A file with a size of 90123 bytes, reflecting the truncate operation
and both buffered writes.
So now the test can fail since kernel 6.13 due to snapshots catching
partial writes.
However the goal of the test when I wrote it was to verify that if the
snapshot version of a file gets the truncated size, then the buffered
writes are visible in the file, since they happened before the truncate
operation - that is, we can't get a file with a size of 90123 bytes that
doesn't have the range [0, 16K) full of bytes with a value of 0xaa and
the range [16K, 49154) full of bytes with the 0xbb value.
So update the test to the new kernel behaviour by verifying only that if
the file has the size we truncated to, then it has all the data we wrote
previously with the buffered writes.
See commit 3839d299 ("xfstests: mount xfs with a context when selinux is on")
fstests by default mount test and scratch devices with forced SELinux
context to get rid of the additional file attributes when SELinux is
enabled. When a test mounts additional devices from the pool, it may need
to honor this option to keep on par. Otherwise failures may be expected.
Moreover this test is perfectly fine labeling the files so let's just
disable the forced context for this one.
Signed-off-by: Daniel Vacek <neelx@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Thu, 27 Feb 2025 13:25:18 +0000 (13:25 +0000)]
btrfs/254: fix test failure due to already unmounted scratch device
If there are no failures in the middle of test while the 3rd scratch
device is mounted (at $seq_mnt), the unmount call in the _cleanup
function will result in a test failure since the unmount already
happened, making the test fail:
btrfs/254 2s ... - output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/254.out.bad)
--- tests/btrfs/254.out 2024-10-07 12:36:15.532225987 +0100
+++ /home/fdmanana/git/hub/xfstests/results//btrfs/254.out.bad 2025-02-27 12:53:30.848728429 +0000
@@ -3,3 +3,4 @@
Total devices <NUM> FS bytes used <SIZE>
devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
*** Some devices missing
+umount: /home/fdmanana/btrfs-tests/dev/254.mnt: not mounted.
...
(Run 'diff -u /home/fdmanana/git/hub/xfstests/tests/btrfs/254.out /home/fdmanana/git/hub/xfstests/results//btrfs/254.out.bad' to see the entire diff)
HINT: You _MAY_ be missing kernel fix: 770c79fb6550 btrfs: harden identification of a stale device
Ran: btrfs/254
Failures: btrfs/254
Failed 1 of 1 tests
This is a recent regression because the _unmount function used to redirect
stdout and stderr to $seqres.full, but not anymore since the recent commit
identified in the Fixes tag below. So redirect stdout and stderr of the
call to _unmount in the _cleanup function to /dev/null.
Fixes: f43da58ef936 ("unmount: resume logging of stdout and stderr for filtering") Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Sandeen [Fri, 28 Feb 2025 15:13:19 +0000 (09:13 -0600)]
generic: test zero-byte writes to file
A bug was recently fixed in exfat where attempting to do a zero-byte
write would yield -EFAULT; test for that here.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Mon, 10 Feb 2025 16:57:15 +0000 (00:57 +0800)]
fstests: use '-std=gnu11' to fix build errors with gcc-15
GCC-15 does a big change, it changes the default C compilation
language standard to -std=gnu23. That cause lots of "old style"
C codes hit building errors. On the other word, current xfstests
can't be used with GCC-15. So use a fixed language standard --
"-std=gnu11" (to match the kernel and xfsprogs).
Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Sterba [Wed, 19 Feb 2025 22:17:43 +0000 (23:17 +0100)]
fstests: remove reiserfs support
The linux kernel removed Reiserfs support in 6.13 so remove it from the
tests as well.
Signed-off-by: David Sterba <dsterba@suse.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Sterba [Wed, 19 Feb 2025 21:52:28 +0000 (22:52 +0100)]
fstests: remove ltp/growfiles
This utility is not used by any current test and seems that it's never
been used in xfstests, so remove it. Appending files can be simply done
by 'xfs_io' command too.
Signed-off-by: David Sterba <dsterba@suse.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jeff Layton [Thu, 20 Feb 2025 11:26:38 +0000 (06:26 -0500)]
generic/126: run it inside its own subdirectory
I had been seeing some failures in generic/126 when running on NFS under
kdevops. The problem turned out to be that kdevops makes the root of the
NFS export mode 01777 by default (i.e. with the sticky bit set). This
causes some of these permission tests to fail because the open() is
called with the O_CREAT bit set.
Make the outcome of this test not depend on the permissions on
$TEST_DIR. Create a new directory to do these tests in, and clean it up
when done.
Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Wed, 19 Feb 2025 18:19:15 +0000 (18:19 +0000)]
btrfs/254: fix test failure in case scratch devices are larger than 50G
If the devices in the scratch pool have a size larger than 50G, then the
test fails due to chunk allocation failure when attempting to create a
multidevice filesystem on $lvdev and $scratch_dev2. This happens because
the $lvdev device has a size of 1G and metadata chunks have a size of 1G
for filesystems with a size greater than 50G, so mkfs fails when it
attempts to allocate chunks since it needs to allocate a 1G metadata
chunk plus a system chunk and a data chunk.
Filipe Manana [Wed, 19 Feb 2025 18:19:14 +0000 (18:19 +0000)]
btrfs/254: don't leave mount on test fs in case of failure/interruption
If the test fails or is interrupted after mounting $scratch_dev3 inside
the test filesystem and before unmounting at test_add_device(), we leave
without being unable to unmount the test filesystem since it has a mount
inside it. This results in the need to manually unmount $scratch_dev3,
otherwise a subsequent run of fstests fails since the unmount of the
test device fails with -EBUSY.
Fix this by unmounting $scratch_dev3 ($seq_mnt) in the _cleanup()
function.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: David Sterba <dsterba@suse.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Anand Jain <anand.jain@oracle.com>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:32 +0000 (14:00 -0800)]
build: initialize stack variables to zero by default
Newer versions of gcc and clang can include the ability to zero stack
variables by default. Let's enable it so that we (a) reduce the risk of
writing stack contents to disk somewhere and (b) try to reduce
unpredictable program behavior based on random stack contents. The
kernel added this 6 years ago, so I think it's mature enough for
fstests.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:31 +0000 (14:00 -0800)]
common/config: add $here to FSSTRESS_PROG
In general we're supposed to specify full paths to fstests binaries with
$here so that subtests can change the current working directory without
issues.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:31 +0000 (14:00 -0800)]
misc: don't put nr_cpus into the fsstress -n argument
fsstress -n is the number of fs operations per process, not the total
number of operations. There's no need to factor nr_cpus into the -n
argument because that causes excess runtime as core count increases.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:31 +0000 (14:00 -0800)]
fsx: fix leaked log file pointer
Fix a resource leaks in fsx, where we fail to close the fsx logfile,
because the C library could have some buffered contents that aren't
flushed when the program terminates. glibc seems to do this for us, but
I wouldn't be so sure about the others.
Fixes: 3f742550dfed84 ("fsx: add support for recording operations to a file") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:31 +0000 (14:00 -0800)]
fsstress: fix a memory leak
Someone forgot to free the iovec that readv_f allocates.
Fixes: 80499d8f5f251e ("fsstress: new writev and readv operations test") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:30 +0000 (14:00 -0800)]
fix _require_scratch_duperemove ordering
Zorro complained that generic/559 stopped working, and I noticed that
the duperemove invocation in the _require_scratch_duperemove function
would fail with:
Error 2: No such file or directory while getting path to file /opt/file1. Skipping.
Error 2: No such file or directory while getting path to file /opt/file2. Skipping.
No dedupe candidates found.
Gathering file list...
The cause of this is the incorrect placement of _require_scratch_dedupe
after a _scratch_mount. _require_scratch_dedupe formats, mounts, tests,
and unmounts the scratch filesystem, which means that it should not come
between a _scratch_mount call and code that uses $SCRATCH_MNT.
Cc: <fstests@vger.kernel.org> # v2024.12.22 Fixes: 3b9f5fc7d7d853 ("common: call _require_scratch_dedupe from _require_scratch_duperemove") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:30 +0000 (14:00 -0800)]
fuzzy: port fsx and fsstress loop to use --duration
Quite a while ago, I added --duration= arguments to fsx and fsstress,
and apparently I forgot to update the scrub stress loops to use them.
Replace the usage of timeout(1) for the remount_period versions of the
loop to clean up that code; and convert the non-remount loop so that
they don't run over time.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:30 +0000 (14:00 -0800)]
fuzzy: always stop the scrub fsstress loop on error
Always abort the non-remount scrub fsstress loop in
__stress_scrub_fsstress_loop if fsstress returns a nonzero exit code.
Cc: <fstests@vger.kernel.org> # v2023.01.15 Fixes: 20df87599f66d0 ("fuzzy: make scrub stress loop control more robust") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:30 +0000 (14:00 -0800)]
fuzzy: don't use readarray for xfsfind output
Some of the scrub stress tests (e.g. xfs/796) walk the directory tree to
find filepaths to scrub, and load the entire list of paths into a bash
array. On a large filesystem or a long-running test this is hugely
wasteful of memory because we use each path exactly once.
Fix __stress_one_scrub_loop to avoid this by reading lines directly from
the output of the xfsfind utility. However, we play some games with fd
77 so that the processes in the loop body will use the same stdin as the
test and /not/ the piped stdout of xfsfind.
To avoid read(1) becoming confused by newlines in the file paths, adapt
xfsfind to print nulls between pathnames, and the bash code to recognize
them.
This was a debugging patch while I was trying to figure out why xfs/286
and other scrub soak tests started OOMing after the v2024.12.08 changes,
though in the end the OOMs were the result of memory leaks in fsstress.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:29 +0000 (14:00 -0800)]
generic/032: fix pinned mount failure
generic/032 now periodically fails with:
--- /tmp/fstests/tests/generic/032.out 2025-01-05 11:42:14.427388698 -0800
+++ /var/tmp/fstests/generic/032.out.bad 2025-01-06 18:20:17.122818195 -0800
@@ -1,5 +1,7 @@
QA output created by 032
100 iterations
-000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd >................<
-*
-100000
+umount: /opt: target is busy.
+mount: /opt: /dev/sda4 already mounted on /opt.
+ dmesg(1) may have more information after failed mount system call.
+cycle mount failed
+(see /var/tmp/fstests/generic/032.full for details)
The root cause of this regression is the _syncloop subshell. This
background process runs _scratch_sync, which is actually an xfs_io
process that calls syncfs on the scratch mount.
Unfortunately, while the test kills the _syncloop subshell, it doesn't
actually kill the xfs_io process. If the xfs_io process is in D state
running the syncfs, it won't react to the signal, but it will pin the
mount. Then the _scratch_cycle_mount fails because the mount is pinned.
Prior to commit 8973af00ec212f the _syncloop ran sync(1) which avoided
pinning the scratch filesystem.
Fix this by pgrepping for the xfs_io process and killing and waiting for
it if necessary.
Cc: <fstests@vger.kernel.org> # v2024.12.08 Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:29 +0000 (14:00 -0800)]
generic/650: revert SOAK DURATION changes
Prior to commit 8973af00ec21, in the absence of an explicit
SOAK_DURATION, this test would run 2500 fsstress operations each of ten
times through the loop body. On the author's machines, this kept the
runtime to about 30s total. Oddly, this was changed to 30s per loop
body with no specific justification in the middle of an fsstress process
management change.
On the author's machine, this explodes the runtime from ~30s to 420s.
Put things back the way they were.
Cc: <fstests@vger.kernel.org> # v2024.12.08 Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:29 +0000 (14:00 -0800)]
preamble: fix missing _kill_fsstress
Commit 8973af00ec212f added a _kill_fsstress to the standard _cleanup
function. However, if something breaks during test program
initialization before it gets to sourcing common/rc, then you get
failures that look like this:
--- /tmp/fstests/tests/generic/556.out 2024-09-25 12:09:52.938797554 -0700
+++ /var/tmp/fstests/generic/556.out.bad 2025-01-04 22:34:01.268327003 -0800
@@ -1,16 +1,3 @@
QA output created by 556
-SCRATCH_MNT/basic Casefold
-SCRATCH_MNT/basic
-SCRATCH_MNT/casefold_flag_removal Casefold
-SCRATCH_MNT/casefold_flag_removal Casefold
-SCRATCH_MNT/flag_inheritance/d1/d2/d3 Casefold
-SCRATCH_MNT/symlink/ind1/TARGET
-mv: 'SCRATCH_MNT/rename/rename' and 'SCRATCH_MNT/rename/RENAME' are the same file
-# file: SCRATCH_MNT/xattrs/x
-user.foo="bar"
-
-# file: SCRATCH_MNT/xattrs/x/f1
-user.foo="bar"
-
-touch: 'SCRATCH_MNT/strict/corac'$'\314\247\303': Invalid argument
-touch: 'SCRATCH_MNT/strict/cora'$'\303\247\303': Invalid argument
+./tests/generic/556: 108: common/config: Syntax error: "&" unexpected
+./tests/generic/556: 10: _kill_fsstress: not found
It's that last line that's unnecessary. Fix this by checking for the
presence of a _kill_fsstress before invoking it.
Cc: <fstests@vger.kernel.org> # v2024.12.08 Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:29 +0000 (14:00 -0800)]
mkfs: don't hardcode log size
Commit 000813899afb46 hardcoded a log size of 256MB into xfs/501,
xfs/502, and generic/530. This seems to be an attempt to reduce test
run times by increasing the log size so that more background threads can
run in parallel. Unfortunately, this breaks a couple of my test
configurations:
- External logs smaller than 256MB
- Internal logs where the AG size is less than 256MB
For example, here's seqres.full from a failed xfs/501 invocation:
** mkfs failed with extra mkfs options added to " -m metadir=2,autofsck=1,uquota,gquota,pquota, -d rtinherit=1," by test 501 **
** attempting to mkfs using only test 501 options: -l size=256m **
size 256m specified for log subvolume is too large, maximum is 32768 blocks
<snip>
mount -ortdev=/dev/sdb4 -ologdev=/dev/sdb2 /dev/sda4 /opt failed
umount: /dev/sda4: not mounted.
Note that there's some formatting error here, so we jettison the entire
rt configuration to force the log size option, but then mount fails
because we didn't edit out the rtdev option there too.
Fortunately, mkfs.xfs already /has/ a few options to try to improve
parallelism in the filesystem by avoiding contention on the log grant
heads by scaling up the log size. These options are aware of log and AG
size constraints so they won't conflict with other geometry options.
Use them.
Cc: <fstests@vger.kernel.org> # v2024.12.08 Fixes: 000813899afb46 ("fstests: scale some tests for high CPU count sanity") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:28 +0000 (14:00 -0800)]
unmount: resume logging of stdout and stderr for filtering
There's a number of places where a test program calls a variant of
_unmount but then pipes the output through a _filter script or
something. The new _unmount helper redirects stdout and stderr to
seqres.full, which means that those error messages (some of which are
encoded in the golden outputs) are suppressed. This leads to test
regressions in generic/050 and other places, so let's resume logging.
This also undoes all the changes that removed /dev/null redirection of
unmount calls.
Cc: <fstests@vger.kernel.org> # v2024.12.08 Fixes: 4c6bc4565105e6 ("fstests: clean up mount and unmount operations") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:28 +0000 (14:00 -0800)]
common/rc: don't copy fsstress to $TEST_DIR
Now that we can pkill only processes that were started by this test, we
don't need to copy the fsstress binary to $TEST_DIR to avoid killing the
wrong program instances. This avoids a whole slew of ETXTBSY problems
with scrub stress tests that run multiple copies of fsstress in the
background.
Revert most of the changes to generic/270, because it wants to do
something fancy with the fsstress binary, so it needs to control the
process directly.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 4 Feb 2025 20:17:11 +0000 (12:17 -0800)]
check: deprecate using process sessions to isolate test instances
As I've noted elsewhere, the use of process session ids to "isolate"
test instances from killing each other is kind of hacky and creates
other weird side effects. I'd rather everyone use the new code that
runs everything in proper isolation with private pid and mount
namespaces, but I don't know how many people this would break were it a
hard dependency.
Deprecate the process session handling immediately with a warning that
we're going to rip it out in a year.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:28 +0000 (14:00 -0800)]
check: run tests in a private pid/mount namespace
As mentioned in the previous patch, trying to isolate processes from
separate test instances through the use of distinct Unix process
sessions is annoying due to the many complications with signal handling.
Instead, we could just use nsexec to run the test program with a private
pid namespace so that each test instance can only see its own processes;
and private mount namespace so that tests writing to /tmp cannot clobber
other tests or the stuff running on the main system. Further, the
process created by the clone(CLONE_NEWPID) call is considered the init
process of that pid namespace, so all processes will be SIGKILL'd when
the init process terminates, so we no longer need systemd scopes for
externally enforced cleanup.
However, it's not guaranteed that a particular kernel has pid and mount
namespaces enabled. Mount (2.4.19) and pid (2.6.24) namespaces have
been around for a long time, but there's no hard requirement for the
latter to be enabled in the kernel. Therefore, this bugfix slips
namespace support in alongside the session id thing.
Declaring CONFIG_PID_NS=n a deprecated configuration and removing
support should be a separate conversation, not something that I have to
do in a bug fix to get mainline QA back up.
Note that the new helper cannot unmount the /proc it inherits before
mounting a pidns-specific /proc because generic/504 relies on being able
to read the init_pid_ns (aka systemwide) version of /proc/locks to find
a file lock that was taken and leaked by a process.
Cc: <fstests@vger.kernel.org> # v2024.12.08 Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:28 +0000 (14:00 -0800)]
common: fix pkill by running test program in a separate session
Run each test program with a separate session id so that we can tell
pkill to kill all processes of a given name, but only within our own
session id. This /should/ suffice to run multiple fstests on the same
machine without one instance shooting down processes of another
instance.
This fixes a general problem with using "pkill --parent" -- if the
process being targeted is not a direct descendant of the bash script
calling pkill, then pkill will not do anything. The scrub stress tests
make use of multiple background subshells, which is how a ^C in the
parent process fails to result in fsx/fsstress being killed.
This is necessary to fix SOAK_DURATION runtime constraints for all the
scrub stress tests. However, there is a cost -- the test program no
longer runs with the same controlling tty as ./check, which means that
^Z doesn't work and SIGINT/SIGQUIT are set to SIG_IGN. IOWs, if a test
wants to kill its subprocesses, it must use another signal such as
SIGPIPE. Fortunately, bash doesn't whine about children dying due to
fatal signals if the children run in a different session id.
Unfortunately we have to let it run the test as a background process for
bash to handle SIGINT, and SIGSTOP no longer works properly.
This solution is a bit crap, and I have a better solution for it in the
next patch that uses private pid and mount namespaces. Unfortunately,
that solution adds new minimum requirements for running fstests and
removing previously supported configurations abruptly during a bug fix
is not appropriate behavior.
I also explored alternate designs, and this was the least unsatisfying:
a) Setting the process group didn't work because background subshells
are assigned a new group id.
b) Constraining the pkill/pgrep search to a cgroup could work, but it
seems that procps has only recently (~2023) gained the ability to filter
on a cgroup. Furthermore, we'd have to set up a cgroup in which to run
the fstest. The last decade has been rife with user bug reports
complaining about chaos resulting from multiple pieces of software (e.g.
Docker, systemd, etc.) deciding that they own the entire cgroup
structure without having any means to enforce that statement. We should
not wade into that mess.
c) Putting test subprocesses in a systemd sub-scope and telling systemd
to kill the sub-scope could work because ./check can already use it to
ensure that all child processes of a test are killed. However, this is
an *optional* feature, which means that we'd have to require systemd.
d) Constraining the pkill/pgrep search to a particular pid namespace
could work, but we already have tests that set up their own mount
namespaces, which means the constrained pgrep will not find all child
processes of a test. Though this hasn't been born out through testing?
e) Constraining to any other type of namespace (uts, pid, etc) might not
work because those namespaces might not be enabled. However, combining
a private pid and mount namespace to isolate tests from each other seems
to work better than session ids. This is coming in a subsequent patch,
but to avoid breaking older systems, we will use this as an immediately
deprecated fallback.
f) Revert check-parallel and go back to one fstests instance per system.
Zorro already chose not to revert.
So. Change _run_seq to create a the ./$seq process with a new session
id, update _su calls to use the same session as the parent test, update
all the pkill sites to use a wrapper so that we only target processes
created by *this* instance of fstests, and update SIGINT to SIGPIPE.
Cc: <fstests@vger.kernel.org> # v2024.12.08 Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 14 Feb 2025 18:47:14 +0000 (10:47 -0800)]
tools: add a Makefile
Add a Makefile to the tools/ directory so that we can put helper scripts
in there and have them installed at make install time. The makefile
comes from a conversation that Zorro and I had over IRC, hence the RH
copyright.
Suggested-by: Zorro Lang <zlang@kernel.org> Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:27 +0000 (14:00 -0800)]
fuzzy: kill subprocesses with SIGPIPE, not SIGINT
The next patch in this series fixes various issues with the recently
added fstests process isolation scheme by running each new process in a
separate process group session. Unfortunately, the processes in the
session are created with SIGINT ignored by default because they are not
attached to the controlling terminal. Therefore, switch the kill signal
to SIGPIPE because that is usually fatal and not masked by default.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:27 +0000 (14:00 -0800)]
common/rc: create a wrapper for the su command
Create a _su wrapper around the su command so that the next patch can
fix all the pkill isolation code so that this test can only kill
processes started for this test.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 4 Feb 2025 19:15:49 +0000 (11:15 -0800)]
generic/759,760: skip test if we can't set up a hugepage for IO
On an arm64 VM with 64k base pages and a paltry 8G of RAM, this test
will frequently fail like this:
> QA output created by 759
> fsx -N 10000 -l 500000 -h
> -fsx -N 10000 -o 8192 -l 500000 -h
> -fsx -N 10000 -o 128000 -l 500000 -h
> +Seed set to 1
> +madvise collapse for buf: Cannot allocate memory
> +init_hugepages_buf failed for good_buf: Cannot allocate memory
This system has a 512MB hugepage size, which means that there's a good
chance that memory is so fragmented that we won't be able to create a
huge page (in 1/16th the available DRAM). Create a _run_hugepage_fsx
helper that will detect this situation at the start of the test and skip
it, having refactored run_fsx into a properly namespaced version that
won't exit the test on failure.
Cc: <fstests@vger.kernel.org> # v2025.02.02 Cc: joannelkoong@gmail.com Fixes: 627289232371e3 ("generic: add tests for read/writes from hugepages-backed buffers") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Joanne Koong <joannelkoong@gmail.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:27 +0000 (14:00 -0800)]
generic/759,760: fix MADV_COLLAPSE detection and inclusion
On systems with "old" C libraries such as glibc 2.36 in Debian 12, the
MADV_COLLAPSE flag might not be defined in any of the header files
pulled in by sys/mman.h, which means that the fsx binary might not get
built with any of the MADV_COLLAPSE code. If the kernel supports THP,
the test will fail with:
Fix both tests to detect fsx binaries that don't support MADV_COLLAPSE,
then fix fsx.c to include the mman.h from the kernel headers (aka
linux/mman.h) so that we can actually test IOs to and from THPs if the
kernel is newer than the rest of userspace.
Cc: <fstests@vger.kernel.org> # v2025.02.02 Cc: joannelkoong@gmail.com Fixes: 627289232371e3 ("generic: add tests for read/writes from hugepages-backed buffers") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Joanne Koong <joannelkoong@gmail.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Feb 2025 22:00:27 +0000 (14:00 -0800)]
common/populate: correct the parent pointer name creation formulae
The formulae used to compute the number of parent pointers that we have
to create in a child file in order to generate a particular xattr
structure are not even close to correct -- the first one needs a bit of
adjustment, but the second one is way off and creates far too many
files.
Fix the computation, and document where the magic numbers come from.
Cc: <fstests@vger.kernel.org> # v2024.06.27 Fixes: 0c02207d61af9a ("populate: create hardlinks for parent pointers") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
projects / users / hch / xfstests-dev.git / log