Christoph Hellwig [Tue, 23 Aug 2022 19:29:29 +0000 (21:29 +0200)]
btrfs/271: use the common fail_request setup helpers
Use the helpers from common/fail_make_request instead of open coding
them. This switches to using a higher error count than the existing
code, which was the intention from the very beginning (and doesn't
actuallt matter for the short sequences in this test).
Yang Xu [Thu, 4 Aug 2022 06:18:52 +0000 (14:18 +0800)]
src/t_ofd_locks.c: Reset errno to zero
It seems I met libcap errno bug again when using libcap-2.48-4.el8.x86_64.
But this time, errno is EINVAL if c program link with lcap.
Lastest upstream libcap doesn't have bug and it should be backport bug.
generic/478 will become not run because of this. To fix this that only
exists on some distributions, reset errno to zero.
Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Boris Burkov [Fri, 19 Aug 2022 19:34:05 +0000 (12:34 -0700)]
fstests: add btrfs fs-verity send/recv test
Test btrfs send/recv support for fs-verity. Includes tests for
signatures, salts, and interaction with chmod/caps. The last of those is
to ensure the various features that go in during inode_finalize interact
properly.
This depends on the kernel patch adding support for send:
btrfs: send: add support for fs-verity
And the btrfs-progs patch adding support for recv:
btrfs-progs: receive: add support for fs-verity
Signed-off-by: Boris Burkov <boris@bur.io> Reviewed-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 9 Aug 2022 21:01:06 +0000 (14:01 -0700)]
dmerror: support external log and realtime devices
Upgrade the dmerror code to coordinate making external scratch log and
scratch realtime devices error out along with the scratch device. Note
that unlike SCRATCH_DEV, we save the old rt/log devices in a separate
variable and overwrite SCRATCH_{RT,LOG}DEV so that all the helper
functions continue to work properly.
This is very similar to what we did for dm-flakey a while back.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 9 Aug 2022 21:00:58 +0000 (14:00 -0700)]
common/ext4: provide custom ext4 scratch fs options
Create a _scratch_options backend for ext* so that we can inject
pathnames to external log devices into the scratch fs mount options.
This enables common/dm* to install block device filters, e.g. dm-error
for stress testing.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 9 Aug 2022 21:00:52 +0000 (14:00 -0700)]
common/rc: move XFS-specific parts of _scratch_options into common/xfs
Move all the non-XFS code in _scratch_options into a
_scratch_xfs_options helper in common/xfs, in preparation to add ext4
bits.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Zorro zlang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 9 Aug 2022 21:00:46 +0000 (14:00 -0700)]
common/rc: move ext4-specific helpers into a separate common/ext4 file
Move the ext4-specific parts of common/rc into a separate file and
source it when we test that.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 9 Aug 2022 21:00:37 +0000 (14:00 -0700)]
xfs/{015,042,076}: fix mkfs failures with nrext64=1
One of my XFS fstests systems is configured with multiple 34G block
devicesm each with a 4k LBA size for the scratch filesystem. If I turn
on large extent counts with -i nrext64=1, I see the following failures
from each of these three tests:
-used inodes is in range
+max log size 4083 smaller than min log size 4287, filesystem is too small
Note that this particular output is dependent on having a recent
xfsprogs with "mkfs: complain about impossible log size constraints"
applied, else you get a far more obscure message about the log being too
small.
It turns out that you can simulate this pretty easily:
# truncate -s 34G /tmp/a; losetup -f -b 4096 /tmp/a
# truncate -s 34G /tmp/b; losetup -f -b 4096 /tmp/b
# mkfs.xfs -f -N /dev/loop0 -r rtdev=/dev/loop1 -d rtinherit=1,size=32m -i nrext64=1
max log size 4083 smaller than min log size 4287, filesystem is too small
So it's pretty obvious what happened here -- the AG size is 4096 blocks.
mkfs has to leave enough free space in each AG to handle the AG space
btree roots, the AG headers, and the AGFL, which means the log can't be
more than 4083 blocks. libxfs computes the minimum log size to be 4287
blocks, which is why the format fails.
Fix this problem by bumping the filesystems up to 96M, which provides
enough space in each AG to handle a ~20M internal log.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 5 Aug 2022 00:55:52 +0000 (17:55 -0700)]
xfs/018: fix LARP testing for small block sizes
Fix this test to work properly when the filesystem block size is less
than 4k. Tripping the error injection points on shape changes in the
xattr structure must be done dynamically.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Catherine Hoang <catherine.hoang@oracle.com> Reviewed-by: Allison Henderson <allison.henderson@oracle.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 4 Aug 2022 16:26:49 +0000 (09:26 -0700)]
xfs/533: fix false negatives for this test
xfsprogs 5.19 will change the error message that gets printed when the
primary superblock validation fails. Filter the _get_metadata_field
output so that the new message looks like the old message.
While we're at it, _notrun this test on V4 filesystems because the
validation messages are in the V5 superblock validation functions.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Yang Xu <xuyang2018.jy@fujitsu.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Convert this test to use _scratch_xfs_repair, since the only variance
from the standard usage is that it's called against a sparse file into
which the scratch filesystem has been metadumped and mdrestored.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 3 Aug 2022 04:21:26 +0000 (21:21 -0700)]
xfs/432: fix this test when external devices are in use
This program exercises metadump and mdrestore being run against the
scratch device. Therefore, the test must pass external log / rt device
arguments to xfs_repair -n to check the "restored" filesystem. Fix the
incorrect usage, and report repair failures, since this test has been
silently failing for a while now.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Sun, 31 Jul 2022 04:46:50 +0000 (12:46 +0800)]
fstests: add more dmerror related test cases into eio group
The g/484, g/628 and g/629 calls _dmerror_init and load error table
for testing, better to count them in eio test group, then we won't
miss them when we want to run all eio related cases.
Signed-off-by: Zorro Lang <zlang@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 28 Jul 2022 18:17:26 +0000 (11:17 -0700)]
seek_sanity_test: use XFS ioctls to determine file allocation unit size
liuyd.fnst@fujitsu.com reported that my recent change to the seek sanity
test broke NFS. I foolishly thought that st_blksize was sufficient to
find the file allocation unit size so that applications could figure out
the SEEK_HOLE granularity. Replace that with an explicit callout to XFS
ioctls so that xfs realtime will work again.
Fixes: e861a302 ("seek_sanity_test: fix allocation unit detection on XFS realtime") Reported-by: liuyd.fnst@fujitsu.com Tested-by: liuyd.fnst@fujitsu.com Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
/dev/block/8:3 corresponds to the SCRATCH_DEV on this system. Given the
failures in 322.out, I think this is caused by generic/322 calling
_init_flakey -> _dmsetup_create -> $DMSETUP_PROG create being unable to
open SCRATCH_DEV exclusively. Add a call to $UDEV_SETTLE_PROG prior to
the creation of the target to try to calm the system down sufficiently
that the test can proceed.
Note that I don't have any hard evidence that it's udev at fault here --
the few times I've caught this thing, udev *has* been active spraying
error messages for nonexistent sysfs paths to journald and adding a
'udevadm settle' seems to fix it... but that's still only
circumstantial. Regardless, it seems to have fixed the test failure.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 26 Jul 2022 19:48:51 +0000 (12:48 -0700)]
dmlogwrites: skip generic tests when external logdev in use
Currently, dm-logwrites and common/dmlogwrites don't seem to have any
means to coordinate the event numbers across multiple devices, and the
fstests setup code is sufficiently intense that it doesn't seem like
there's support for multi-disk filesystems. For now, we'll _notrun the
tests when we have external log devices, even though that seems like
something we'd _really_ want to test.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
btrfs: add test case to make sure btrfs can handle one corrupted device
The new test case will verify that btrfs can handle one corrupted device
without affecting the consistency of the filesystem.
Unlike a missing device, one corrupted device can return garbage to the fs,
thus btrfs has to utilize its data/metadata checksum to verify which
data is correct.
The test case will:
- Create a small fs
Mostly to speedup the test
- Fill the fs with a regular file
- Use fsstress to create some contents
- Save the fssum for later verification
- Corrupt one device with garbage but keep the primary superblock
untouched
- Run fssum verification
- Run scrub to fix the fs
- Run scrub again to make sure the fs is fine
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Boris Burkov <boris@bur.io> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
btrfs/253: skip on zoned mode as we cannot change the chunk size
On zoned mode, we have a fixed chunk size which is equal to the zone size.
So, we cannot change the chunk size, and running this test results in a
failure with below.
--- tests/btrfs/253.out 2021-12-10 04:33:53.000000000 +0000
+++ /host/results/btrfs/253.out.bad 2022-07-26 05:58:10.000000000 +0000
@@ -2,9 +2,16 @@
Capture default chunk sizes.
First allocation.
Second allocation.
+./common/rc: line 4670: echo: write error: Invalid argument
+./common/rc: line 4670: echo: write error: Invalid argument
Calculate request size so last memory allocation cannot be completely fullfilled.
Third allocation.
...
It is no use to test this feature on zoned mode. So, just skip it.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Yang Xu [Tue, 19 Jul 2022 08:31:13 +0000 (16:31 +0800)]
xfs: test xfs_db whether still can get corrupted field value
This is a regression test that xfs_db code doesn't clean up the
LIBXFS_READBUF_SALVAGE call sites in set_cur so that we can't use the
return value directly and scraping it out later.
Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Sun Ke [Wed, 27 Jul 2022 07:11:40 +0000 (15:11 +0800)]
ext4: resize an ext4 which resize_inode feature is disabled but has reserved GDT blocks.
A regression test for b55c3cd102a6 ("ext4: add reserved GDT blocks
check"). Make sure there's not kernel crash, if resize an ext4 which
resize_inode feature is disabled but has reserved GDT blocks.
Signed-off-by: Sun Ke <sunke32@huawei.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
The xunit-quiet format excludes the NNN.{full,dmesg,bad} files in
<system-out> and <system-err> nodes which are included in the xunit
report format.
For test runners that save the entire results directory to preserve
all of the test artifacts, capturing the NNN.{full,dmesg,bad} in the
results.xml file is redundant. In addition, if the NNN.bad is too
large, it can cause the junitparser python library to refuse to parse
the XML file to prevent potential denial of service attacks[1]. A
simple way to avoid this problem is to simply to omit the <system-out>
and <system-err> nodes in the results.xml file.
Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jeremy Bongio [Thu, 21 Jul 2022 22:39:30 +0000 (15:39 -0700)]
ext4: add test for ext4 uuid get/set ioctls during fsstress.
Adds a utility to get/set uuid through ext4 ioctl. Executes the ioctls
while running fsstress. These ioctls are used by tune2fs to safely change
the uuid without racing other filesystem modifications.
Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Jeremy Bongio <bongiojp@gmail.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 19 Jul 2022 21:38:01 +0000 (14:38 -0700)]
punch: skip fpunch tests when page size not congruent with file allocation unit
Skip the generic fpunch tests on a file when the file's allocation unit
size is not congruent with the system page size. This is needed for
testing swapfiles and mmap collisions wiht fallocate.
Assuming this edgecase configuration of an edgecase feature is
vanishingly rare, let's just _notrun the tests instead of rewriting a
ton of tests to do their integrity checking by hand.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 19 Jul 2022 21:37:55 +0000 (14:37 -0700)]
filter: report data block mappings and od offsets in multiples of allocation units
All the tests that use these two filter functions also make all of their
fallocate calls in units of file allocation units, not filesystem
blocks. Make them transform the file offsets to multiples of file
allocation units (via _get_file_block_size) so that xfs/242 and xfs/252
will work with XFS with a rt extent size set.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 19 Jul 2022 21:37:50 +0000 (14:37 -0700)]
punch: skip fpunch tests when op length not congruent with file allocation unit
Skip the generic fpunch tests on a file when the file's allocation unit
size is not congruent with the proposed testing operations.
This can be the case when we're testing reflink and fallocate on the XFS
realtime device. For those configurations, the file allocation unit is
a realtime extent, which can be any integer multiple of the block size.
If the request length isn't an exact multiple of the allocation unit
size, reflink and fallocate will fail due to alignment issues, so
there's no point in running these tests.
Assuming this edgecase configuration of an edgecase feature is
vanishingly rare, let's just _notrun the tests instead of rewriting a
ton of tests to do their integrity checking by hand.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 19 Jul 2022 21:37:44 +0000 (14:37 -0700)]
punch: use allocation unit to test punching holes
In step 17 of _test_generic_punch, we want to test that we can write
a file with a single block, use one of unresvsp, fpunch, or fzero to
modify the file, and then check that the file has one written block
followed by a hole.
Unfortunately, the test helper uses _get_block_size to determine how
much data to write to the test file. For filesystems with an allocation
unit size that is not the fs block size (e.g. XFS realtime with a rt
extent size), this produces unwritten extents in the fiemap output,
which causes test failures.
Fix step 17 to obtain the file allocation unit size with
_get_file_block_size.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 19 Jul 2022 21:37:39 +0000 (14:37 -0700)]
misc: avoid tests encoding FIEMAP/BMAP golden output with weird file blocksizes
Some tests encode FIEMAP/XFS_IOC_BMAP results in the golden output.
Typically these tests cannot handle a filesystem that chooses to
allocate extents that are much larger than the filesystem block size
(aka XFS rt extents and ext4 bigalloc). Since these tests are /never/
going to pass, disable them when these configurations are detected.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 19 Jul 2022 21:37:33 +0000 (14:37 -0700)]
misc: skip extent size hint tests when hint not congruent with file allocation unit
XFS files have an extent size hint, which tells the block allocator that
it should try to allocate larger aligned blocks when possible. These
hints must be some integer multiple of the allocation unit size, which
is one fs block for files on the data device, and one rt extent for
files on the realtime device. For tests that are hardwired to a static
extent size hint, the fssetxattr call will fail if the hint isn't
congruent, so just skip those tests.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 19 Jul 2022 21:37:27 +0000 (14:37 -0700)]
misc: skip remap/fallocate tests when op length not congruent with file allocation unit
Nearly all of the reflink and fpunch/fcollapse/finsert tests that I have
written assumed that it was ok to use 64k as the fundamental unit of
allocation. This works fine for testing the XFS data device, since the
file allocation unit is always a power of two, and never larger than
64k. Making this assumption allows those tests to encode md5sums in the
golden output for easy file data integrity checking.
Unfortunately, this isn't necessarily the case when we're testing
reflink and fallocate on the XFS realtime device. For those
configurations, the file allocation unit is a realtime extent, which can
be any integer multiple of the block size. If the request length isn't
an exact multiple of the allocation unit size, reflink and fallocate
will fail due to alignment issues, so there's no point in running these
tests.
Assuming this edgecase configuration of an edgecase feature is
vanishingly rare, let's just _notrun the tests instead of rewriting a
ton of tests to do their integrity checking by hand.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 19 Jul 2022 21:37:22 +0000 (14:37 -0700)]
misc: use _get_file_block_size for block (re)mapping tests
Tests that exercise block remapping functionality such as reflink,
hole punching, fcollapse, and finsert all require the input parameters
to be aligned to allocation unit size for regular files. This could be
different from the fundamental filesystem block size (think ext4
bigalloc or xfs realtime), so use the appropriate function here.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com>
signed-off-by: Zorro Lang <zlang@kernel.org>
Boris Burkov [Wed, 20 Jul 2022 00:49:50 +0000 (17:49 -0700)]
generic: test fs-verity EFBIG scenarios
btrfs, ext4, and f2fs cache the Merkle tree past EOF, which restricts
the maximum file size beneath the normal maximum. Test the logic in
those filesystems against files with sizes near the maximum.
To work properly, this does require some understanding of the practical
but not standardized layout of the Merkle tree. This is a bit unpleasant
and could make the test incorrect in the future, if the implementation
changes. On the other hand, it feels quite useful to test this tricky
edge case. It could perhaps be made more generic by adding some ioctls
to let the file system communicate the maximum file size for a verity
file or some information about the storage of the Merkle tree.
Signed-off-by: Boris Burkov <boris@bur.io> Reviewed-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Sweet Tea Dorminy <sweettea-kernel@dorminy.me> Signed-off-by: Zorro Lang <zlang@kernel.org>
Boris Burkov [Wed, 20 Jul 2022 00:49:49 +0000 (17:49 -0700)]
btrfs: test verity orphans with dmlogwrites
The behavior of orphans is most interesting across mounts, interrupted
at arbitrary points during fsverity enable. To cover as many such cases
as possible, use dmlogwrites and dmsnapshot as in
log-writes/replay-individual.sh. As we replay the log events, we run a
state machine with different invariants enforced at each state.
There are three possible states for a given point in the log:
0. Verity has not yet started
1. Verity has started but not finished
2. Verity has finished.
The possible transitions with causes are:
0->1: We see an orphan item for the file.
1->2: Running 'fsverity measure' succeeds on the file.
Each state has its own invariant for testing:
0: No verity items exist.
1: Mount should handle the orphan and blow away verity data: expect 0
Merkle items after mounting.
2: The orphan should be gone and mount should not blow away merkle
items. Expect the same number of merkle items before and after
mounting.
As a result, we can be confident that if the file system loses power at
any point during enabling verity on a file, the work is either completed,
or gets rolled-back by mount.
Signed-off-by: Boris Burkov <boris@bur.io> Reviewed-by: Sweet Tea Dorminy <sweettea-kernel@dorminy.me> Signed-off-by: Zorro Lang <zlang@kernel.org>
Boris Burkov [Wed, 20 Jul 2022 00:49:48 +0000 (17:49 -0700)]
btrfs: test btrfs specific fsverity corruption
There are some btrfs specific fsverity scenarios that don't map
neatly onto the tests in generic/574 like holes, inline extents,
and preallocated extents. Cover those in a btrfs specific test.
This test relies on the btrfs implementation of fsverity in the patch:
btrfs: initial fsverity support
and on btrfs-corrupt-block for corruption in the patches titled:
btrfs-progs: corrupt generic item data with btrfs-corrupt-block
btrfs-progs: expand corrupt_file_extent in btrfs-corrupt-block
Signed-off-by: Boris Burkov <boris@bur.io> Reviewed-by: Sweet Tea Dorminy <sweettea-kernel@dorminy.me> Signed-off-by: Zorro Lang <zlang@kernel.org>
Boris Burkov [Wed, 20 Jul 2022 00:49:47 +0000 (17:49 -0700)]
common/verity: support btrfs in generic fsverity tests
generic/572-579 have tests for fsverity. Now that btrfs supports
fsverity, make these tests function as well. For a majority of the tests
that pass, simply adding the case to mkfs a btrfs filesystem with no
extra options is sufficient.
However, generic/574 has tests for corrupting the merkle tree itself.
Since btrfs uses a different scheme from ext4 and f2fs for storing this
data, the existing logic for corrupting it doesn't work out of the box.
Adapt it to properly corrupt btrfs merkle items.
576 does not run because btrfs does not support transparent encryption.
This test relies on the btrfs implementation of fsverity in the patch:
btrfs: initial fsverity support
and on btrfs-corrupt-block for corruption in the patches titled:
btrfs-progs: corrupt generic item data with btrfs-corrupt-block
btrfs-progs: expand corrupt_file_extent in btrfs-corrupt-block
Signed-off-by: Boris Burkov <boris@bur.io> Reviewed-by: Sweet Tea Dorminy <sweettea-kernel@dorminy.me> Signed-off-by: Zorro Lang <zlang@kernel.org>
Boris Burkov [Wed, 20 Jul 2022 00:49:46 +0000 (17:49 -0700)]
common/verity: require corruption functionality
Corrupting ext4 and f2fs relies on xfs_io fiemap. Btrfs corruption
testing will rely on a btrfs specific corruption utility. Add the
ability to require corruption functionality to make this properly
modular. To start, just check for fiemap, as that is needed
universally for _fsv_scratch_corrupt_bytes.
Signed-off-by: Boris Burkov <boris@bur.io> Reviewed-by: Josef Bacik <josef@toxicpanda.com> Reviewed-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Sweet Tea Dorminy <sweettea-kernel@dorminy.me> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 19 Jul 2022 21:37:13 +0000 (14:37 -0700)]
generic/275: fix premature enospc errors when fs block size is large
When running this test on an XFS filesystem with a 64k block size, I
see this error:
generic/275 - output mismatch (see /var/tmp/fstests/generic/275.out.bad)
--- tests/generic/275.out 2021-05-13 11:47:55.694860280 -0700
+++ /var/tmp/fstests/generic/275.out.bad 2022-07-19 10:38:41.840000000 -0700
@@ -2,4 +2,7 @@
------------------------------
write until ENOSPC test
------------------------------
+du: cannot access '/opt/tmp1': No such file or directory
+stat: cannot statx '/opt/tmp1': No such file or directory
+/tmp/fstests/tests/generic/275: line 74: [: -lt: unary operator expected
done
...
(Run 'diff -u /tmp/fstests/tests/generic/275.out /var/tmp/fstests/generic/275.out.bad' to see the entire diff)
The 275.full file indicates that the test was unable to recreate the
$SCRATCH_MNT/tmp1 file after we freed all but the last 256K of free
space in the filesystem. I mounted the scratch fs, and df reported
exactly 256K of free space available, which means there are 4 blocks
left in the filesystem for user programs to use.
Unfortunately for this test, xfs_create requires sufficient free blocks
in the filesystem to handle full inode btree splits and the maximal
directory expansion for a new dirent. In other words, there must be
enough free space to handle the worst case space consumption. That
quantity is 26 blocks, hence the last dd in the test fails with ENOSPC,
which makes the test fail.
Fix all this by creating the file that we use to test the low-space file
write *before* we drain the free space down to 256K.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Josef Bacik [Tue, 19 Jul 2022 15:27:35 +0000 (11:27 -0400)]
btrfs/257: not run if we have compression enabled
This fails on all of our compression config variations because it
depends on specific file extent layout. Add _require_btrfs_no_compress
so we don't complain about failures that don't apply.
Signed-off-by: Josef Bacik <josef@toxicpanda.com> Reviewed-by: Sweet Tea Dorminy <sweettea-kernel@dorminy.me> Signed-off-by: Zorro Lang <zlang@kernel.org>
Christoph Hellwig [Tue, 19 Jul 2022 06:14:54 +0000 (08:14 +0200)]
btrfs/270: fix uncompressed data from previously bad mirror
The commited old version test the broken behavior of the current
upstream code that writes the uncompressed data into a previously
bad mirror. Fix the test to check that the compressed data gets
re-replicated and add it to the compress group while we're at it.
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Sweet Tea Dorminy <sweettea-kernel@dorminy.me> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 5 Jul 2022 22:02:08 +0000 (15:02 -0700)]
xfs/144: fix test mkfs.xfs sizing of internal logs that overflow the AG
Fix a few problems with this test -- one of the things we test require
mkfs to run in -N mode, so we need to have a certain amount of free
space, and fix that test not to use -N mode.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 5 Jul 2022 22:02:19 +0000 (15:02 -0700)]
xfs/547: fix problems with realtime
This test needs to fragment the free space on the data device so that
each block added to the attr fork gets its own mapping. If the test
configuration sets up a rt device and rtinherit=1 on the root dir, the
test will erroneously fragment space on the *realtime* volume. When
this happens, attr fork allocations are contiguous and get merged into
fewer than 10 extents and the test fails.
Fix this test to force all allocations to be on the data device, and fix
incorrect variable usage in the error messages.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Disseldorp [Fri, 8 Jul 2022 08:51:42 +0000 (10:51 +0200)]
check: add -L <n> parameter to rerun failed tests
If check is run with -L <n>, then a failed test will be rerun <n> times
before proceeding to the next test. Following completion of the rerun
loop, aggregate pass/fail statistics are printed.
Rerun tests will be tracked as a single failure in overall pass/fail
metrics (via @try and @bad), with .out.bad, .dmesg, .core, .hints,
.notrun and .full saved using a .rerun# suffix.
Suggested-by: Theodore Ts'o <tytso@mit.edu> Link: https://lwn.net/Articles/897061/ Signed-off-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Disseldorp [Fri, 8 Jul 2022 08:51:41 +0000 (10:51 +0200)]
check: append bad / notrun arrays in helper function
Currently the @try, @bad and @notrun arrays are appended with seqnum at
different points in the main run_section() loop:
- @try: shortly prior to test script execution
- @notrun: on list (check -n), or after .notrun flagged test completion
- @bad: at the start of subsequent test loop and loop exit
For future loop-test-following-failure functionality it makes sense to
combine some of these steps. This change moves both @notrun and @bad
appends into a helper function which is called at the end of each loop
iteration.
Signed-off-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Disseldorp [Fri, 8 Jul 2022 08:51:39 +0000 (10:51 +0200)]
report: pass through most details as function parameters
Report generation currently involves reaching into a whole bunch of
globals for things like section name and start/end times. Pass these
through as explicit function parameters to avoid unintentional breakage.
One minor fix included is the default xunit error message, which used
$sequm instead of $seqnum.
Signed-off-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 30 Jun 2022 00:48:14 +0000 (17:48 -0700)]
xfs: test xfs_copy doesn't do cached read before libxfs_mount
This is a regression test for an xfs_copy fix that ensures that it
doesn't perform a cached read of an XFS filesystem prior to initializing
libxfs, since the xfs_mount (and hence the buffer cache) isn't set up
yet.
[zlang: change the supported fs from generic to xfs]
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 28 Jun 2022 20:21:34 +0000 (13:21 -0700)]
xfs: test mkfs.xfs sizing of internal logs that
This is a regression test that exercises the mkfs.xfs code that creates
log sizes that are very close to the AG size when stripe units are in
play and/or when the log is forced to be in AG 0.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 28 Jun 2022 20:22:02 +0000 (13:22 -0700)]
xfs/166: fix golden output failures when multipage folios enabled
Beginning with 5.18, some filesystems support creating large folios for
the page cache. A system with 64k pages can create 256k folios, which
means that with the old file size of 1M, the last half of the file is
completely converted from unwritten to written by page_mkwrite. The
test encodes a translated version of the xfs_bmap output in the golden
output, which means that the test now fails on 64k pages. Fixing the
64k page case by increasing the file size to 2MB broke fsdax because
fsdax uses 2MB PMDs, hence 12MB.
Increase the size to prevent this from happening. This may require
further revision if folios get larger or fsdax starts supporting PMDs
that are larger than 2MB.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 28 Jun 2022 20:21:45 +0000 (13:21 -0700)]
check: document mkfs.xfs reliance on fstests exports
There are a number of fstests that employ special (and now unsupported)
XFS filesystem configurations to perform testing in a controlled
environment. The presence of the QA_CHECK_FS and MSGVERB variables are
used by mkfs.xfs to detect that it's running inside fstests, which
enables the unsupported configurations. Nobody else should be using
filesystems with tiny logs, non-redundant superblocks, or smaller than
the (new) minimum supported size.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 28 Jun 2022 20:21:23 +0000 (13:21 -0700)]
seek_sanity_test: fix allocation unit detection on XFS realtime
The seek sanity test tries to figure out a file space allocation unit by
calling stat and then using an iterative SEEK_DATA method to try to
detect a smaller blocksize based on SEEK_DATA's consultation of the
filesystem's internal block mapping. This was put in (AFAICT) because
XFS' stat implementation returns max(filesystem blocksize, PAGESIZE) for
most regular files.
Unfortunately, for a realtime file with an extent size larger than a
single filesystem block this doesn't work at all because block mappings
still work at filesystem block granularity, but allocation units do not.
To fix this, detect the specific case where st_blksize != PAGE_SIZE and
trust the fstat results.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Theodore Ts'o [Thu, 30 Jun 2022 14:46:37 +0000 (10:46 -0400)]
ext4: create file systems with the encrypt feature as necessary
The linux kernel commit 5f41fdaea63d ("ext4: only allow
test_dummy_encryption when supported") the kernel will reject mounts
with the test_dummy_encryption option if the ext4 file system does not
have the encrypt feature enabled.
There are a handful of tests (ext4/003, ext4/035, ext4/306, and
generic/260) which will format the scratch file system using a
hard-coded set of mkfs.ext4 parameters ignoring the MKFS_OPTION that
is set by the file system test config.
For file system configs which includes test_dummy_encryption in
MOUNT_OPTIONS and "-O encrypt" in MKFS_OPTIONS, we need to test for
test_dummy_encryption and force the hard-coded mkfs options to enable
the encrypt feature.
Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Theodore Ts'o [Thu, 30 Jun 2022 14:46:34 +0000 (10:46 -0400)]
ext4/054: skip test if the dax mount option is enabled
The ext4/054 test explicitly creates a file system with a 1k
blocksize. This can't possibly work on if the dax mount option is
enabled, so change ext4/054 to use _scratch_mkfs_blocksized, and and a
check to _scratch_mkfs_blocksized to _notrun the test if the block
size is less than the page size.
Also remove an unnecessary _require_test declaration since this test
does not use the test device.
Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Theodore Ts'o [Thu, 30 Jun 2022 14:46:32 +0000 (10:46 -0400)]
ext4/044: skip test if the file system does not have a journal
This test mounts the file system using "mount -t ext3"; if the file
system config creates the file system without the jbd2 journal, the
"mount -t ext3" will fail. So skip this test in that case.
Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Theodore Ts'o [Thu, 30 Jun 2022 14:46:31 +0000 (10:46 -0400)]
ext4/050: support indirect as well as extent mapped journals
Simplify the test and fix ext4/050 failures when running ext4 without
extents enabled (e.g., in ext3 emulation mode).
Instead of relying on parsing debugfs output's (which varies depending
on whether the journal inode is extent mapped or indirect block
mapped), use debugfs's "cat" command to get the contents of the
journal.
Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Luís Henriques [Mon, 27 Jun 2022 10:26:31 +0000 (11:26 +0100)]
ceph/005: verify correct statfs behaviour with quotas
When using a directory with 'max_bytes' quota as a base for a mount,
statfs shall use that 'max_bytes' value as the total disk size. That
value shall be used even when using subdirectory as base for the mount.
A bug was found where, when this subdirectory also had a 'max_files'
quota, the real filesystem size would be returned instead of the parent
'max_bytes' quota value. This test case verifies this bug is fixed.
Signed-off-by: Luís Henriques <lhenriques@suse.de> Reviewed-by: Xiubo Li <xiubli@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Amir Goldstein [Tue, 21 Jun 2022 17:37:29 +0000 (20:37 +0300)]
xfs/{422,517}: kill background jobs on test termination
Those tests failed to cleanup background jobs properly after test
is interrupted and even sometimes when it completed succefully.
xfs/517 would sometime fails randomally with this false positive error:
QA output created by 517
Format and populate
Concurrent fsmap and freeze
+Terminated
Test done
The tests have several background sub-shells that spawn short lived
programs in a loop. By killing the spawned programs using killall,
killall could find no process to kill and the sub-shell loop could still
spawn another process that is not going to be killed and in the worst
case, the freeze_loop() could spawn the xfs_io "freeze" command after
test has thawn the fs before exit, which leaves the fs frozen after the
test.
The "Terminated" output is emitted by the sub-shell when killing the
programs that it has spawned when the loop did not finish before test
timeout. By killing the sub-shell and not the spawned programs, we
avoid the false positive "Terminated" error.
Use a helper to perform this cleanup dance:
First kill and wait the freeze_loop so it won't try to freeze fs again
Then make sure fs is not frozen.
Then kill and wait for the rest of the sub-shells, because
if fs is frozen a killed writer process will never exit.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Thu, 23 Jun 2022 18:41:13 +0000 (11:41 -0700)]
ext4/053: test changing test_dummy_encryption on remount
The test_dummy_encryption mount option isn't supposed to be settable or
changeable via a remount, so add test cases for this. This is a
regression test for a bug that was introduced in Linux v5.17 and fixed
in v5.19-rc3 by commit 85456054e10b ("ext4: fix up test_dummy_encryption
handling for new mount API").
Reviewed-by: Lukas Czerner <lczerner@redhat.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Sandeen [Mon, 20 Jun 2022 21:43:31 +0000 (16:43 -0500)]
xfs/122: escape %zu in printf with %% not \\%
The standard way to escape % in a printf is with %%; although \\%zu
seems to have worked in awk until recently, an upgrade on Fedora 36
has started failing:
awk: cmd. line:1: (FILENAME=- FNR=1) fatal: not enough arguments to satisfy format string
'printf("sizeof(%s) = \%zu\n", sizeof(%s));
'
^ ran out for this one
Switching the escape to "%%" fixes this for me, and also works
on my very old RHEL7 mcahine.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Disseldorp [Mon, 20 Jun 2022 19:29:32 +0000 (21:29 +0200)]
report: fix xunit tests count
The xunit "section report" provides a tests attribute, which according
to https://llg.cubic.org/docs/junit/ represents:
tests="" <!-- The total number of tests in the suite, required. -->
The current value is generated as a sum of the $n_try and $n_notrun
counters. This is incorrect as the $n_try counter already includes tests
which are run but complete with _notrun.
One special case exists for $showme (check -n), where $n_try remains
zero, so $n_notrun can be used as-is.
Signed-off-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Luís Henriques [Mon, 13 Jun 2022 11:31:42 +0000 (12:31 +0100)]
generic/486: adjust the max xattr size
CephFS doesn't have a maximum xattr size. Instead, it imposes a maximum
size for the full set of xattrs names+values, which by default is 64K. And
since it reports 4M as the blocksize (the default ceph object size),
generic/486 will fail in ceph because the XATTR_SIZE_MAX value can't be used
in attr_replace_test.
The fix is to add a new argument to the test so that the max size can be
passed in instead of trying to auto-probe a value for it.
Signed-off-by: Luís Henriques <lhenriques@suse.de> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Chandan Babu R [Sat, 11 Jun 2022 11:10:34 +0000 (16:40 +0530)]
xfs/270: Fix ro mount failure when nrext64 option is enabled
With nrext64 option enabled at run time, the read-only mount performed by the
test fails because,
1. mkfs.xfs would have calculated log size based on reflink being enabled.
2. Clearing the reflink ro compat bit causes log size calculations to yield a
different value.
3. In the case where nrext64 is enabled, this causes attr reservation to be
the largest among all the transaction reservations.
4. This ends up causing XFS to require a larger ondisk log size than that
which is available.
This commit fixes the problem by setting features_ro_compat to the value
obtained by the bitwise-OR of features_ro_compat field with 2^31.
This commit includes changes suggested by Dave Chinner to replace bashisms
with invocations to inline awk scripts.
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Fri, 27 May 2022 12:11:15 +0000 (20:11 +0800)]
common/filter: filter out extra mount error output
The lastest mount command (from util-linux) merged below commit: 79534c0d7e0f ("mount: add hint about dmesg(8) to error messages")
which brought in a new error output when mount fails, no matter
ro/rw/busy mount fails.
That cause some cases (e.g. xfs/005) fail as:
mount: Structure needs cleaning
dmesg(1) may have more information after failed mount system call
More failed cases like generic/050, ext4/002, xfs/154, xfs/158 etc.
Especially xfs/154 and xfs/158, need to change their _filter_scratch
to _filter_error_mount.
Signed-off-by: Zorro Lang <zlang@kernel.org> Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com>
Zorro Lang [Fri, 3 Jun 2022 08:24:57 +0000 (16:24 +0800)]
xfs: corrupted xattr should not block removexattr
After we corrupted an attr leaf block (under node block), getxattr
might hit EFSCORRUPTED in xfs_attr_node_get when it does
xfs_attr_node_hasname. A known bug cause xfs_attr_node_get won't do
xfs_buf_trans release job, then a subsequent removexattr will hang.
This case covers a1de97fe296c ("xfs: Fix the free logic of state in
xfs_attr_node_hasname")
Signed-off-by: Zorro Lang <zlang@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
projects / users / hch / xfstests-dev.git / log