By specifying "xmlns=https://git.kernel.org/.../xfstests-dev.git",
this causes XML complaint parsers, such as the one used by the python
junitparser library, to put all of the XML elements into a namespace,
which then causes junitparser to toss its cookies.
This can be worked-around in a test runner script via:
sed -i.orig -e 's/xmlns=\".*\"//' "$RESULT_BASE/result.xml"
but it's better not to include the xmlns line at all in the first
place, since this may cause other users of fstests who are using
the Python junitparser library a lot of headaches.
Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
After every single test, we rewrite result.xml from scratch. This
ensures that the XML file is always in a valid, parseable state, even
if the check script is killed or the machine crashes in the middle of
a test.
If the test is being run in a Cloud VM as a "spot" (Amazon, Azure, or
GCE) or "preemptible" (Oracle) instance, the VM can be halted whenever
the Cloud provider needs the capacity for customers who are willing to
pay full price. ("Spot" instances can be 60% to 90% cheaper ---
allowing the frugal kernel developer to get up to 10 times more
testing for the same amount of money. :-)
Since a "spot" VM can get terminated at any time, it is possible for
the check script to be killed while it is in the middle of rewriting
the result.xml file. If the result.xml file is only partially
written, information regarding the tests run before VM termination
will be lost.
To address this race, write the new result.xml file as result.xml.new,
and only rename it to result.xml after the XML file is fully written
out.
Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 3 Jul 2023 17:03:44 +0000 (10:03 -0700)]
xfs/569: skip post-test fsck run
This test examines the behavior of mkfs.xfs with specific filesystem
configuration files by formatting the scratch device directly with those
exact parameters. IOWs, it doesn't include external log devices or
realtime devices. If external devices are set up, the post-test fsck
run fails because the filesystem doesnt' use the (allegedly) configured
external devices. Fix that by adding _require_scratch_nocheck.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
The test injects the bmap_alloc_minlen_extent error, which refuses to
allocate file space unless it's exactly minlen long. However, a
precondition of this injection point is that the free space on the data
device must be sufficiently fragmented that there are small free
extents.
However, if realtime and rtinherit are enabled, the punch-alternating
call will operate on a realtime file, which only serves to write 0x55
patterns into the realtime bitmap. Hence the test preconditions are not
satisfied, so the test is not serving its purpose.
Fix it by disabling rtinherit=1 on the rootdir so that we actually
fragment the bnobt/cntbt as required.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Qu Wenruo [Thu, 29 Jun 2023 00:10:10 +0000 (08:10 +0800)]
common/btrfs: handle dmdust as mounted device in _btrfs_buffered_read_on_mirror()
[BUG]
After commit ab41f0bddb73 ("common/btrfs: use _scratch_cycle_mount to
ensure all page caches are dropped"), the test case btrfs/143 can fail
like below:
btrfs/143 6s ... [failed, exit status 1]- output mismatch (see ~/xfstests/results//btrfs/143.out.bad)
--- tests/btrfs/143.out 2020-06-10 19:29:03.818519162 +0100
+++ ~/xfstests/results//btrfs/143.out.bad 2023-06-19 17:04:00.575033899 +0100
@@ -1,37 +1,6 @@
QA output created by 143
wrote 131072/131072 bytes
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa
................
-XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa
................
-XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa
................
-XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa
................
[CAUSE]
Test case btrfs/143 uses dm-dust device to emulate read errors, this
means we can not use _scratch_cycle_mount to cycle mount $SCRATCH_MNT.
As it would go mount $SCRATCH_DEV, not the dm-dust device to
$SCRATCH_MNT.
This prevents us to trigger read-repair (since no error would be hit)
thus fail the test.
[FIX]
Since we can mount whatever device at $SCRATCH_MNT, we can not use
_scratch_cycle_mount in this case.
Instead implement a small helper to grab the mounted device and its
mount options, and use the same device and mount options to cycle
$SCRATCH_MNT mount.
This would fix btrfs/143 and hopefully future test cases which use dm
devices.
Reported-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Tue, 20 Jun 2023 12:00:21 +0000 (13:00 +0100)]
btrfs: test activating swapfile in the presence of snapshots
Test that if we have a subvolume with a non-active swap file, we can not
activate it if there are any snapshots. Also test that after all the
snapshots are removed, we will be able to activate the swapfile.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Amir Goldstein [Sun, 18 Jun 2023 12:45:06 +0000 (15:45 +0300)]
generic/604: Fix for overlayfs
Since v6.3, I noticed that generic/604 does not run on overlayfs
because:
generic/604 -- upper fs needs to support d_type
This is odd because the base fs I am using (xfs) does support d_type.
The reason is that for overlayfs, this sequence run by this test:
_scratch_unmount &
_scratch_mount
Translates to:
umount $OVL_MNT; umount $BASE_MNT &
mount $BASE_MNT ...; mount $OVL_MNT ...
Which can end up reordred as:
umount $OVL_MNT;
mount $BASE_MNT ...
umount $BASE_MNT &
mount $OVL_MNT ...
and overlayfs is trying to use a non-existing upper fs.
Use UMOUNT_PROG directly instead of the _scratch_unmount
helper, to avoid unmounting the base fs.
Incidently, the only thing that has changed in overlayfs in v6.3
is idmapped mounts support and the test in question was run without
idmapped mounts enabled, so the cahnge in behavior must be related
to some subtle timing change.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Reviewed-by: Christian Brauner <brauner@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Yuezhang Mo [Tue, 4 Jul 2023 23:49:55 +0000 (07:49 +0800)]
check: fix excluded tests are only expunged in the first iteration
If iterating more than once and excluding some tests, the
excluded tests are expunged in the first iteration, but run in
subsequent iterations. This is not expected.
The problem was caused by the temporary file saving the excluded
tests being deleted by `rm -f $tmp.*` in _wrapup() at the end of
the first iteration.
This commit saves the excluded tests into a variable instead of a
temporary file.
Signed-off-by: Yuezhang Mo <Yuezhang.Mo@foxmail.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Amir Goldstein [Mon, 19 Jun 2023 10:50:58 +0000 (13:50 +0300)]
fstests: reduce runtime of check -n
kvm-xfstests invokes check -n twice to pre-process and generate the
tests-to-run list, which is then being passed as a long list of tests
for invkoing check in the command line.
check invokes dirname, basename and sed several times per test just
for doing basic string prefix/suffix trimming.
Use bash string pattern matching instead which is much faster.
Note that the following pattern matching expression change:
< test_dir=${test_dir#$SRC_DIR/*}
> t=${t#$SRC_DIR/}
does not change the meaning of the expression, because the
shortest match of "$SRC_DIR/*" that is being trimmed is "$SRC_DIR/"
and removing the tests/ prefix is what this code intended to do.
With check -n, there is no need to cleanup the results dir,
but check -n is doing that for every single listed test.
Move the cleanup of results dir to before actually running the test.
These improvements to check pre-test code cut down several minutes
from the time until tests actually start to run with kvm-xfstests.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Carlos Maiolino [Wed, 24 May 2023 13:42:07 +0000 (15:42 +0200)]
common/rc: Enable _test_mkfs to force a mkfs on a xfs filesystem
Calling _test_mkfs on an already initialized xfs FS will fail as the
initialization is not enforced by '-f' argument, unless it's included in
MKFS_OPTIONS.
So, adding 'RECREATE_TEST_DEV=true' to the config file end up being useless
for xfs filesystems.
So, adding the a specific xfs optiong in _test_mkfs using -f argument
makes RECREATE_TEST_DEV actually useful.
Signed-off-by: Carlos Maiolino <cmaiolino@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Xiubo Li [Thu, 1 Jun 2023 02:52:07 +0000 (10:52 +0800)]
common/rc: skip ceph-fuse when atime is required
Ceph won't maintain the atime, so just skip the tests when the atime
is required.
Fixes: https://tracker.ceph.com/issues/61551 Signed-off-by: Xiubo Li <xiubli@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Xiubo Li [Tue, 30 May 2023 07:15:52 +0000 (15:15 +0800)]
generic/020: add ceph-fuse support
For ceph fuse client the fs type will be "ceph-fuse".
Fixes: https://tracker.ceph.com/issues/61496 Signed-off-by: Xiubo Li <xiubli@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Qu Wenruo [Thu, 8 Jun 2023 11:48:36 +0000 (19:48 +0800)]
common/btrfs: use _scratch_cycle_mount to ensure all page caches are dropped
[BUG]
There is a chance that btrfs/266 would fail on aarch64 with 64K page
size. (No matter if it's 4K sector size or 64K sector size)
The failure indicates that one or more mirrors are not properly fixed.
[CAUSE]
I have added some trace events into the btrfs IO paths, including
__btrfs_submit_bio() and __end_bio_extent_readpage().
When the test failed, the trace looks like this:
112819.764977: __btrfs_submit_bio: r/i=5/257 fileoff=0 mirror=1 len=196608 pid=33663
^^^ Initial read on the full 192K file
112819.766604: __btrfs_submit_bio: r/i=5/257 fileoff=0 mirror=2 len=65536 pid=21806
^^^ Repair on the first 64K block
Which would success
112819.766760: __btrfs_submit_bio: r/i=5/257 fileoff=65536 mirror=2 len=65536 pid=21806
^^^ Repair on the second 64K block
Which would fail
112819.767625: __btrfs_submit_bio: r/i=5/257 fileoff=65536 mirror=3 len=65536 pid=21806
^^^ Repair on the third 64K block
Which would success
112819.770999: end_bio_extent_readpage: read finished, r/i=5/257 fileoff=0 len=196608 mirror=1 status=0
^^^ The repair succeeded, the
full 192K read finished.
But above read only happen for mirror 1 and mirror 3, mirror 2 is not
involved.
This means by somehow, the read on mirror 2 didn't happen, mostly
due to something wrong during the drop_caches call.
It may be an async operation or some hardware specific behavior.
On the other hand, for test cases doing the same operation but utilizing
direct IO, aka btrfs/267, it never fails as we never populate the page
cache thus would always read from the disk.
[WORKAROUND]
The root cause is in the "echo 3 > drop_caches", which I'm still
debugging.
But at the same time, we can workaround the problem by forcing a
cycle mount of scratch device, inside _btrfs_buffered_read_on_mirror().
By this we can ensure all page caches are dropped no matter if
drop_caches is working correctly.
With this patch, I no longer hit the failure on aarch64 with 64K page
size anymore, while before this the test case would always fail during a
-I 10 run.
[zlang: remove the duplicated drop_caches line]
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Christoph Hellwig <hch@infradead.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Qu Wenruo [Fri, 2 Jun 2023 10:24:53 +0000 (18:24 +0800)]
btrfs/106: avoid hard coded output to handle different page sizes
[BUG]
Test case btrfs/106 is known to fail if the system has a page size other
than 4K.
This test case can fail like this:
btrfs/106 5s ... - output mismatch (see ~/xfstests-dev/results//btrfs/106.out.bad)
--- tests/btrfs/106.out 2022-11-24 19:53:53.140469437 +0800
+++ ~/xfstests-dev/results//btrfs/106.out.bad 2023-06-02 16:12:57.014273249 +0800
@@ -5,19 +5,19 @@
File contents before unmount:
0 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa
*
-40
+1000
File contents after remount:
0 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa
...
(Run 'diff -u ~/xfstests-dev/tests/btrfs/106.out /home/adam/xfstests-dev/results//btrfs/106.out.bad' to see the entire diff)
This is particularly problematic for systems like Aarch64 or PPC64 which
supports 64K page size.
[CAUSE]
The test case is using page size to calculate the amount of data to
write, thus it doesn't support any page sizes other than 4K.
[FIX]
Instead of using the golden output, go with md5sum and compare them
before and after the remount.
The new md5sum would only go into $seqres.full for debugging, not into
golden output to avoid false alerts on different pages sizes.
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Anand Jain [Fri, 2 Jun 2023 11:38:54 +0000 (19:38 +0800)]
btrfs/122: fix nodesize option in mfks.btrfs
btrf/122 is failing on a system with 64k page size:
QA output created by 122
+ERROR: illegal nodesize 16384 (smaller than 65536)
+mount: /mnt/scratch: wrong fs type, bad option, bad superblock on /dev/vdb2, missing codepage or helper program, or other error.
+mount /dev/vdb2 /mnt/scratch failed
+(see /xfstests-dev/results//btrfs/122.full for details)
Mkfs.btrfs sets the default node size to 16K when the sector size is less
than 16K, and it matches the sector size when it's greater than 16K.
So, there's no need to explicitly set it.
Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Tested-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 6 Jun 2023 22:29:39 +0000 (15:29 -0700)]
common/xfs: compress online repair rebuild output by default
Force-repairing the filesystem after a test can fill up /tmp with quite
a lot of logging message. We don't have a better place to stash that
output in case the scrub fails and we need to analyze it later, so
compress it with gzip and only decompress it later.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 6 Jun 2023 22:29:34 +0000 (15:29 -0700)]
xfs/503: don't rebuild the fs metadata when testing metadump
This test exercises metadump with the standard populate image. There's
no need to test rebuilding the entire fs every step of the way since
we're just going to metadump over it.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 6 Jun 2023 22:29:28 +0000 (15:29 -0700)]
fuzzy: disallow post-test online rebuilds when testing online fsck
If we're testing the online fsck code or running fuzz tests of the
filesystem, don't let the post-test checks rebuild the filesystem
metadata, because this is redundant with the test and will disturb
the metadata if the tools fail.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 6 Jun 2023 22:29:19 +0000 (15:29 -0700)]
xfs/155: improve logging in this test
If this test fails after a certain number of writes, we should state
the exact number of writes so that we can coordinate with 155.full.
Instead, we state the pre-randomization number, which isn't all that
helpful.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Bill O'Donnell <bodonnel@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 6 Jun 2023 22:29:14 +0000 (15:29 -0700)]
xfs/155: discard stderr when checking for NEEDSREPAIR
This test deliberate crashes xfs_repair midway through writing metadata
to check that NEEDSREPAIR is always triggered by filesystem writes.
However, the subsequent scan for the NEEDSREPAIR feature bit prints
verifier errors to stderr.
On a filesystem with metadata directories, this leads to the test
failing with this recorded in the golden output:
+Metadata CRC error detected at 0x55c0a2dd0d38, xfs_dir3_block block 0xc0/0x1000
+dir block owner 0x82 doesnt match block 0xbb8cd37e44eb3623
This isn't specific to metadata directories -- any repair crash could
leave a metadata structure in a weird state such that starting xfs_db
will spray verifier errors. For _check_scratch_xfs_features here, we
don't care if the filesystem is corrupt; we /only/ care that the
superblock feature bit is set. Route all that noise to devnull.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 6 Jun 2023 22:29:08 +0000 (15:29 -0700)]
xfs/108: allow slightly higher block usage
With pmem and fsdax enabled, I occasionally see this test fail on XFS:
Mode: (0600/-rw-------) Uid: (1) Gid: (2)
Disk quotas for User #1 (1)
Filesystem Blocks Quota Limit Warn/Time Mounted on
-SCRATCH_DEV 48M 0 0 00 [------] SCRATCH_MNT
+SCRATCH_DEV 48.0M 0 0 00 [------] SCRATCH_MNT
Disk quotas for User #1 (1)
Filesystem Files Quota Limit Warn/Time Mounted on
SCRATCH_DEV 3 0 0 00 [------] SCRATCH_MNT
The cause of this failure is fragmentation in the file mappings that
results in a block mapping structure that no longer fits in the inode.
Hence the block usage is 49160K instead of the 49152K that was written.
Use some fugly sed duct tape to make this test accomodate this
possiblity.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Bill O'Donnell <bodonnel@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Qu Wenruo [Tue, 6 Jun 2023 11:01:23 +0000 (19:01 +0800)]
btrfs: add a test case to verify the scrub error reports
There is a regression in recent v6.4 cycle where a scrub rewrite changed
how we report errors, especially repairable errors.
Before the rewrite, we report the initial errors hit, and the amount of
repairable errors.
While after the rewrite, we no longer report the initial errors, but
only the number of repairable errors.
This behavior change is a regression, thus needs a test case to prevent
such problem from happening again.
The test case itself would:
- Create a btrfs using DUP data profile and 4K sector size
- Create a file with one 128K extent
- Corrupt the first mirror of that 128K extent
- Scrub and checks the detailed report
Both corrected errors and csum errors should be 32.
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Qu Wenruo [Tue, 6 Jun 2023 10:52:00 +0000 (18:52 +0800)]
btrfs: add a test case to verify read-only scrub
There is a regression on btrfs read-only scrub behavior.
The commit e02ee89baa66 ("btrfs: scrub: switch scrub_simple_mirror() to
scrub_stripe infrastructure") makes btrfs scrub to ignore the read-only
flag completely, causing scrub to always fix the corruption.
This test case would create an fs with repairable corruptions, then run
a read-only scrub, and finally to make sure the corruption is not
repaired.
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Andreas Gruenbacher [Wed, 7 Jun 2023 14:53:34 +0000 (16:53 +0200)]
generic: Add mmap + DIO write test
This is the same as generic/647, but with an additional test that writes
a memory-mapped page onto itself using direct I/O.
For direct I/O writes, the kernel will invalidate the page cache before
and after carrying out the write. This puts filesystems that fault in
pages on demand and then carry out the write with page faults disabled
into a position in which they will not be able to make any progress.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Anthony Iliopoulos [Mon, 22 May 2023 21:27:12 +0000 (23:27 +0200)]
fuzzy: skip online scrub and health checks if not supported
Commit a27e6e6f4c18 introduced xfs health checking on no-repair fuzz,
which in turn requires scrub to be run before that. The health checks
are done only if scrub returns with an error (which is expected as an
indication that fuzzed metadata errors were picked up), but the code
does not discern between xfs_scrub returning an error because of
uncorrected metadata vs failing because the kernel does not support
scrub at all.
This causes all tests that do fuzzing with no-repair strategy to fail on
kernels compiled without online scrub support (CONFIG_XFS_ONLINE_SCRUB).
Skip scrub and health checks altogether, if the kernel does not support
it, since the tests are still valuable.
Fixes: a27e6e6f4c18 ("common: check xfs health after doing an online scrub") Signed-off-by: Anthony Iliopoulos <ailiop@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Fri, 19 May 2023 09:57:01 +0000 (10:57 +0100)]
btrfs/213: avoid occasional failure due to already finished balance
btrfs/213 writes data, in 1M extents, for 4 seconds into a file, then
triggers a balance and then after 2 seconds it tries to cancel the
balance operation. More often than not, this works because the balance
is still running after 2 seconds. However it also fails sporadically
because balance has finished in less than 2 seconds, which is plausible
since data and metadata are cached or other factors such as virtualized
environment. When that's the case, it fails like this:
btrfs/213 51s ... - output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/213.out.bad)
--- tests/btrfs/213.out 2020-06-10 19:29:03.822519250 +0100
+++ /home/fdmanana/git/hub/xfstests/results//btrfs/213.out.bad 2023-05-17 15:39:32.653727223 +0100
@@ -1,2 +1,3 @@
QA output created by 213
+ERROR: balance cancel on '/home/fdmanana/btrfs-tests/scratch_1' failed: Not in progress
Silence is golden
...
(Run 'diff -u /home/fdmanana/git/hub/xfstests/tests/btrfs/213.out /home/fdmanana/git/hub/xfstests/results//btrfs/213.out.bad' to see the entire diff)
Ran: btrfs/213
Failures: btrfs/213
Failed 1 of 1 tests
To make it much less likely that balance has already finished before we
try to cancel it, unmount and mount again the filesystem before starting
balance, to clear cached metadata and data, and also double the time we
spend writing 1M data extents. Also make the test not run with an
informative message if we detect that balance finished before we could
cancel it.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Thu, 18 May 2023 11:05:55 +0000 (12:05 +0100)]
generic/708: fix commit subject and add its git hash
The test refers to a patch that ended up not committed to Linus' tree, as
the fix evolved through several patchset versions. The up to date fix
landed on kernel 6.4-rc1 and is the following:
b73a6fd1b1ef "btrfs: split partial dio bios before submit"
So updated the test to point to that.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Anna Schumaker [Tue, 16 May 2023 14:14:07 +0000 (10:14 -0400)]
generic: Add a test for xattr ctime updates
The NFS client wasn't updating ctime after a setxattr request. This is a
test written while fixing the bug.
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Andrey Albershteyn [Tue, 16 May 2023 10:23:03 +0000 (12:23 +0200)]
fstests: fix compilation error in splice2pipe on old (<C99) compilers
Compilation fails on system with compiler which uses older C dialect
(e.g. RHEL7; gcc with gnu90) with:
splice2pipe.c: In function 'prepare_pipe':
splice2pipe.c:54:2: error: 'for' loop initial declarations are only allowed in C99 mode
for (unsigned r = pipe_size; r > 0;) {
Fix it by declaring 'r' outside of the loop.
Signed-off-by: Andrey Albershteyn <aalbersh@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 2 May 2023 20:08:45 +0000 (13:08 -0700)]
generic/{094,225}: drop the file allocation unit requirements
Drop these two test precondition requirements now that we've fixed
fiemap-tester to handle unwritten preallocations that are mapped to
unwritten parts of files and/or mapped beyond EOF.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 2 May 2023 20:08:40 +0000 (13:08 -0700)]
fiemap: FIEMAP_EXTENT_LAST denotes the last record in the recordset
Remove this check because FIEMAP_EXTENT_LAST denotes the last space
mapping record in the recordset of space mappings attached to the file.
That last mapping might actually map space beyond EOF, in the case of
(a) speculative post-eof preallocations, (b) stripe-aligned allocations
on XFS, or (c) realtime files in XFS.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 2 May 2023 20:08:34 +0000 (13:08 -0700)]
fiemap-tester: holes can be backed by unwritten extents
Filesystem behavior is pretty open-ended for sparse ranges (i.e. holes)
of a file that have not yet been written to. That space can remain
unmapped, it can be mapped to written space that has been zeroed, or it
can be mapped to unwritten extents.
This program trips over that last condition on XFS. If the file is
located on a data device with a raid stripe geometry or on a realtime
device with a realtime extent size larger than 1 filesystem block, it's
possible for unwritten areas to be backed by unwritten preallocations or
unwritten rt blocks, respectively.
Fix the test to skip unwritten extents here.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 2 May 2023 20:08:28 +0000 (13:08 -0700)]
xfs/{243,245,272,274}: ignore raid alignment flags in bmap output
This test doesn't care about the RAID alignment status of the mappings
that it finds; it only cares about shared and unwritten. Ignore the
mapping stripe alignment flags in the bmapx output. This fixes this
test when the fs has su=128k,sw=4.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 2 May 2023 20:08:23 +0000 (13:08 -0700)]
generic/724,xfs/791: adjust test preconditions for post-EOF stripe zeroing
I recently introduced a new fstests config with explicitly specified
stripe geometry of 128k stripe units and a stripe width of 4. This
broke both of these tests because I hadn't counted on a few things:
1) The write to $SCRATCH_MNT/b at 768k would a 128k delalloc extent
2) This delalloc extent would extend beyond EOF
3) Increasing the file size from 832k to 1m would cause iomap to zero
the pagecache for the parts of the delalloc extent beyond EOF
4) The newly dirtied posteof delalloc areas would get written to disk
with a real space allocation
Under these circumstances, FIEXCHRANGE with SKIP_FILE1_HOLES sees a
written extent containing zeroes in file B between 832k and 1m. File
A has a written extent containing 'X' in the same range, so it exchanges
the two. When RAID geometry is disabled, the area between 832k and 1m
is usually a hole, so FIEXCHRANGE does nothing. This causes the md5sum
of the two files to be different, and the test fails.
Fix the test by truncating B to 1m before writing anything to it.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 2 May 2023 20:08:11 +0000 (13:08 -0700)]
fsx: fix indenting of columns in bad buffers report
When file corruption is detected, make the columns of the report line
up correctly even in the diff output. Although the .out.bad file
contains this (with spaces to demonstrate unequivocally what happens
when tabs are formatted as 8-column indent):
OFFSET GOOD BAD RANGE
0x2c000 0x0000 0xd6c1 0x00000
diffing the good and bad golden output yields poorly formatted output:
+OFFSET GOOD BAD RANGE
+0x2c000 0x0000 0xd6c1 0x00000
Replace the tabs with columns indented with printf width specifiers so
that the test output gets this:
OFFSET GOOD BAD RANGE
0x2c000 0x0000 0xd6c1 0x0
...which always lines up the columns regardless of the user's tab
display settings or diff inserting plus signs.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Wed, 10 May 2023 10:55:59 +0000 (11:55 +0100)]
btrfs/228: sync filesystem after creating subvolume
Test case btrfs/228 creates a subvolume and then calls the dump-tree
command from btrfs-progs. The tree dumping accesses the device directly
and therefore can only see committed metadata - this used to work because
subvolume creation used to commit the transaction that was used to create
the subvolume, however it is no longer the case after a recent patch that
currently is only on the btrfs integration branch "misc-next". That patch
has the following subject:
"btrfs: don't commit transaction for every subvol create"
So explicitly sync the filesystem before calling the dump-tree command,
commenting why we do it. This way the test works before and after that
patch, for any kernel release.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Tue, 9 May 2023 11:52:05 +0000 (12:52 +0100)]
btrfs: add a test case for the logical to ino ioctl
Add a test case to exercise the logical to ino ioctl, including the v2
version (which adds the ignore offset option). This is motivated by the
fact that we have no test cases giving full coverage to that ioctl, only
two test cases partially exercise it (btrfs/004 and btrfs/299) and
absolutely no coverage for the v2 ioctl. This resulted in a recent
regression fixed by the patch mentioned in the _fixed_by_kernel_commit
tag of the introduced test case.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Tue, 9 May 2023 11:52:04 +0000 (12:52 +0100)]
common/btrfs: add helper to get the bytenr for a file extent item
In upcoming changes there will be the need to find out the logical disk
address (bytenr) that a particular file extent item points to. This is
already implemented as local functions in the test btrfs/299, which is
a bit limited but works fine for that test. Some important or subtle
details why it works for this test:
1) It dumps all trees of the filesystem;
2) It relies on fsync'ing a file and then finding the desired file
extent item in the log tree from the dump;
3) There's a single subvolume, so it always finds the correct file extent
item. In case there were multiple subvolumes, it could pick the wrong
file extent item in case we have inodes with the same number on
multiple subvolumes (inode numbers are unique only within a subvolume,
they are not unique across an entire filesystem).
So add a helper to get the bytenr associated to a file extent item to
common/btrfs and use it at btrfs/299 and the upcoming changes.
This helper will dump only the tree of the default subvolume, will sync
the filesystem to commit any open transaction and works only in case the
filesystem is using the scratch device. This is explicitly documented.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 26 Apr 2023 00:14:34 +0000 (17:14 -0700)]
misc: add duration for long soak tests
Make it so that test runners can schedule long soak stress test programs
for an exact number of seconds by setting the SOAK_DURATION config
variable. Change the definition of the 'soak' test to specify that
these tests can be controlled via SOAK_DURATION.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Howells [Fri, 28 Apr 2023 20:06:17 +0000 (21:06 +0100)]
fstests: Doc changes for afs
Documentation changes for afs.
Signed-off-by: David Howells <dhowells@redhat.com>
cc: linux-afs@lists.infradead.org Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Howells [Mon, 24 Apr 2023 14:10:42 +0000 (15:10 +0100)]
generic/696: AFS doesn't support the "noacl" command line option
AFS doesn't support the "noacl" command line option. ACLs are mandatory
and interpreted by the server.
Signed-off-by: David Howells <dhowells@redhat.com>
cc: linux-afs@lists.infradead.org Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Howells [Mon, 24 Apr 2023 14:10:41 +0000 (15:10 +0100)]
generic/531: Check for O_TMPFILE
Make generic/531 check that the filesystem under test supports O_TMPFILE
before attempting to test it.
Signed-off-by: David Howells <dhowells@redhat.com>
cc: linux-afs@lists.infradead.org Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Howells [Mon, 24 Apr 2023 14:10:40 +0000 (15:10 +0100)]
generic/123, generic/128, afs: Allow for an fs that does its own perm management
The AFS filesystem has its own distributed permission management system
that's based on a per-cell user and group database used in conjunction with
ACLs. The user is determined by the authentication token acquired from the
kaserver or Kerberos, not by the local fsuid/fsgid. For the most part, the
uid, gid and mask on a file are ignored.
The generic/123 and generic/128 tests check that the UNIX permission bits do
what would normally be expected of them - but this fails on AFS. Using "su"
to change the user is not effective on AFS. Instead, "keyctl session" would
need to be used and an alternative authentication token would need to be
obtained.
Provide a "_require_unix_perm_checking" clause so that these tests can be
suppressed in cases such as AFS.
Signed-off-by: David Howells <dhowells@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Howells [Mon, 24 Apr 2023 14:10:39 +0000 (15:10 +0100)]
generic/317, afs: Allow for a filesystem not to honour the local uid/gid
Each AFS cell has it's own set of user IDs that is uses internally, in its
ACL system and in its protection management protocol. The user ID used by
the fileserver is selected from the set belonging to the fileserver's cell
according to the authentication token associated with an RPC operation -
and this is set as a file's user ID when it is created.
This means that tests that expect to set a UID and see the same UID still
set afterwards will fail.
Add a "_require_use_local_uidgid" clause to indicate that a test expects
internal UID/GID information to be seen in the stat output and should be
skipped if AFS's case.
Signed-off-by: David Howells <dhowells@redhat.com>
cc: linux-afs@lists.infradead.org Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Howells [Mon, 24 Apr 2023 14:10:38 +0000 (15:10 +0100)]
generic/314, afs: Allow for a filesystem that doesn't honour SGID inheritance
The AFS filesystem doesn't do any special handling for the SUID, SGID and
SVTX bits and doesn't perform any sort of propagation. Further, only a
user with cell admin rights can set non-0777 bits.
Handle this by adding a "_require_sgid_inheritance" clause and labelling
the test with it, thereby skipping for filesystems that don't support it.
Signed-off-by: David Howells <dhowells@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
cc: linux-afs@lists.infradead.org Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Howells [Mon, 24 Apr 2023 14:10:37 +0000 (15:10 +0100)]
fstests: add AFS support
Add support for the AFS filesystem. AFS is a network filesystem and there
are a number of features it doesn't support.
- No mkfs. (Kind of. An AFS volume server can be asked to create a new
volume, but that's probably best left to AFS-specific test suites.
Further, a volume would need to be destroyed before another of the same
name could be created; it's not simply a matter of overwriting the old
one as it is on a blockdev with a block-based filesystem.)
- No fsck. (Kind of - the server can be asked to salvage a volume, but it
may involve taking the server offline).
- No richacls. AFS has its own ACL system.
- No atimes.
Signed-off-by: David Howells <dhowells@redhat.com>
cc: linux-afs@lists.infradead.org Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Yang Xu [Tue, 25 Apr 2023 08:21:30 +0000 (16:21 +0800)]
src: Don't include <sys/mount.h> and <linux/mount.h> together
Newer glibc such as glibc 2.36 also defines 'struct mount_attr'
in addition to <linux/mount.h>.
Usually we should use glibc header instead of kernel header.
But now mount.h is a special case because both new glibc header and
kernel header all define "struct mount_attr'. They also define MS*
macro.
Since we have some syscall wrapper in vfs/missing.h, we can use
<linux.mount.h> directly instead of <sys/mount.h> for
detached_mounts_propagation.c.
For utils.c, it doesn't use the macro or function in <sys/mount.h>,
so remove it directly.
In fact, newer glibc(2.37-1)[1] has sloved conflict problem between
<sys/mount.h> and <linux/mount.h>. In the future(maybe ten years), we
can remove this kernel header and use glibc header.
Acked-by: Christian Brauner <brauner@kernel.org> Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Tested-by: Ziyang Zhang <ZiyangZhang@linux.alibaba.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Xiubo Li [Mon, 17 Apr 2023 02:41:34 +0000 (10:41 +0800)]
common/rc: skip ceph when atime is required
Ceph won't maintain the atime, so just skip the tests when the atime
is required.
URL: https://tracker.ceph.com/issues/53844 Signed-off-by: Xiubo Li <xiubli@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
check: _check_filesystems for errors even if test failed
Previously, we would only run _check_filesystems to ensure that a test
that appeared to pass did not have any filesystem corruption. However,
in _check_filesystems, we also repair any errors found in the filesystem.
Let's do this even if we already know the test failed so that subsequent
tests aren't affected.
Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
check: try to fix the test device if it gets corrupted
If the test device gets corrupted all subsequent tests will fail. To
prevent this from causing all subsequent tests to be useless, try
repair the file system on TEST_DEV if possible. We don't need to do
this with the scratch device since that file system gets recreated
each time anyway.
Signed-off-by: Theodore Ts'o <tytso@mit.edu> Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 13 Apr 2023 00:10:43 +0000 (17:10 -0700)]
xfs/517: add missing freeze command
This test is supposed to race fsstress, fsmap, and freezing for a while,
but when we converted it to use _scratch_xfs_stress_scrub, the freeze
loop fell off by accident. Add it back.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Tue, 4 Apr 2023 17:14:11 +0000 (01:14 +0800)]
fstests/MAINTAINERS: add a co-maintainer for btrfs testing part
Darrick J. Wong would like to nominate Anand Jain to help more on
btrfs testing part (tests/btrfs and common/btrfs). He would like to
be a co-maintainer of btrfs part, will help to review and test
fstests btrfs related patches, and I might merge from him if there's
big patchset. So CC him besides send to fstests@ list, when you have
a btrfs fstests patch.
Acked-by: David Sterba <dsterba@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Tue, 4 Apr 2023 17:14:10 +0000 (01:14 +0800)]
fstests/MAINTAINERS: add some specific reviewers
Some people contribute to someone specific fs testing mostly, record
some of them as Reviewer.
Acked-by: Christian Brauner <brauner@kernel.org> Acked-by: Amir Goldstein <amir73il@gmail.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Tue, 4 Apr 2023 17:14:09 +0000 (01:14 +0800)]
fstests/MAINTAINERS: add supported mailing list
The fstests supports different kind of fs testing, better to cc
specific fs mailing list for specific fs testing, to get better
reviewing points. So record these mailing lists and files related
with them in MAINTAINERS file.
Acked-by: Chao Yu <chao@kernel.org> Acked-by: Xiubo Li <xiubli@redhat.com> Acked-by: Jan Kara <jack@suse.cz> Acked-by: Christian Brauner <brauner@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Tue, 4 Apr 2023 17:14:07 +0000 (01:14 +0800)]
fstests: add MAINTAINERS and get_maintainer.pl files
As fstests covers more and more fs testing, so we always get help
from fs specific mailing list, due to they learn about their features
and bugs more. Besides that, some folks help to review patches
(relevant with them) more often.
So I'd like to bring in the similar way of linux/MAINTAINERS, records
fs relevant mailing lists, reviewers or co-maintainers. To recognize
the contribution from them, and help more users to know who or what
mailing list can be added in CC list of a patch.
The MAINTAINERS and get_maintainer.pl are copied from linux project,
then made some changes for fstests specially.
Darrick J. Wong [Tue, 4 Apr 2023 23:17:06 +0000 (16:17 -0700)]
populate: create fewer holes in directories and xattrs
The same study of aging filesystem metadumps also showed that the rate
of file and xattr deletion is a lot lower than what this script does.
Decrease the deletion interval from every other file/name to every 19th
item to get us to a more reasonable 5% deletion rate, but with a prime
number. ;)
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 4 Apr 2023 23:17:00 +0000 (16:17 -0700)]
populate: create fewer subdirs when constructing directories
Based on some surveys of aged filesystems, I've noticed that the
proportion of directory children that are subdirectories tends to be
more in the 5-10% range, not the 95% that the current code generates.
Rework popdir.pl so that we can specify arbitrary percentages of
children files, and lower the ratio dramatically.
This shouldn't have any substantive changes in the shape of the
directories that gets generated; it just gets us a more realistic
sample.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 4 Apr 2023 23:16:55 +0000 (16:16 -0700)]
common/populate: fix btree-format xattr creation on xfs
Currently, we set a large number of extended attributes when trying to
force the attr fork to be in BTREE format. This doesn't work reliably
because userspace has no control over where xattr leaf and dabtree
blocks are mapped, and contiguous mappings can prevent the file from
having a btree format attr fork.
However, we /do/ have one small knob for controlling attr fork mappings
in the form of creating remote value xattrs and then deleting them to
leave holes in the mappings. Create a separate helper function that
exploits this property to try to create a sparse attr fork with enough
mappings to give us the btree attr fork that we want.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 29 Mar 2023 00:58:16 +0000 (17:58 -0700)]
xfs/242: fix _filter_bmap for xfs_io bmap that does rt file properly
xfsprogs commit b1faed5f787 ("xfs_io: fix bmap command not detecting
realtime files with xattrs") fixed the xfs_io bmap output to display
realtime file columns for realtime files with xattrs. As a result, the
data and unwritten flags are in column 5 and not column 7.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 30 Mar 2023 16:46:17 +0000 (09:46 -0700)]
generic/{251,260}: compute maximum fitrim offset
FITRIM is a bizarre ioctl. Callers are allowed to pass in "start" and
"length" parameters, which are clearly some kind of range argument. No
means is provided to discover the minimum or maximum range. Although
regular userspace programs default to (start=0, length=-1ULL), this test
tries to exercise different parameters.
However, the test assumes that the "size" column returned by the df
command is the maximum value supported by the FITRIM command, and is
surprised if the number of bytes trimmed by (start=0, length=-1ULL) is
larger than this size quantity.
This is completely wrong on XFS with realtime volumes, because the
statfs output (which is what df reports) will reflect the realtime
volume if the directory argument is a realtime file or a directory
flagged with rtinherit. This is trivially reproducible by configuring a
rt volume that is much larger than the data volume, setting rtinherit on
the root dir at mkfs time, and running either of these tests.
Refactor the open-coded df logic so that we can determine the value
programmatically for XFS.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Anthony Iliopoulos [Wed, 29 Mar 2023 15:52:36 +0000 (17:52 +0200)]
common: switch _get_{available,total}_space accounting to units of single bytes
xfs/220 relies on detecting free space changes after truncation of
single filesystem blocks, and this fails when the fs block size is 512b.
By default df is counting 1024b block units and as such is not reporting
any change in the number of available blocks after freeing just a 512b
block.
Switch the _get_available_space df reporting block size from units of
1024b blocks directly to single bytes in order to make free space
accounting fine-grained and independent of any fs block size
assumptions.
Do the same for _get_available_space, since there is no reason for
having an additional conversion step.
Signed-off-by: Anthony Iliopoulos <ailiop@suse.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:53:55 +0000 (17:53 -0700)]
report: allow test runners to inject arbitrary values
Per Ted's request, add to the test section reporting code the ability
for test runners to point to a file containing colon-separated key value
pairs. These key value pairs will be recorded in the report file as
extra properties.
Requested-by: Theodore Ts'o <tytso@mit.edu> Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:53:32 +0000 (17:53 -0700)]
report: collect basic information about a test run
Record various generic information about an fstests run when generating
a junit xml report. This includes the cpu architecture, the kernel
revision, the CPU, memory, and numa node counts, and some information
about the block devices passed in.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:53:27 +0000 (17:53 -0700)]
report: encode xml entities in property values
Avoid trouble with the properties reported in the xml reports by
translating xml-tricky characters in the property values into their xml
entity equivalents.
IOWs, if someone sets a property "NAME" to the value 'BOBBY"; DROP TABLES;',
the xml will be formatted:
<property name="NAME" value="BOBBY"; DROP TABLES;"/>
Thus avoiding XML problems.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:53:21 +0000 (17:53 -0700)]
report: pass property value to _xunit_add_property
Change this helper to require the caller to pass the value as the second
parameter. This prepares us to start reporting a lot more information
about a test run, not all of which are encoded as bash variables.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:53:10 +0000 (17:53 -0700)]
report: encode the kernel log as a separate xml element
Record the .dmesg file in a new <kernel-log> element instead of
multiplexing it with <system-err>. This means that the xml report can
now capture kernel log and bad golden output.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:53:04 +0000 (17:53 -0700)]
report: encode cdata sections correctly
The XML report format captures the contents of .full and .out.bad files
in CDATA sections. CDATA sections are supposed to be a stream of
verbatim data, terminated with a "]]>". Hence XML entities such as
quotation marks and angle brackes should not be escaped, and an actual
bracket-bracket-gt sequence in those files /does/ need escaping.
Create a separate filtering function so that these files are encoded
properly.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:52:59 +0000 (17:52 -0700)]
report: record fstests start and report generation timestamps
Report two new timestamps in the xml report: the time that ./check was
started, and the time that the report was generated. We introduce new
timestamps to minimize breakage with parsing scripts.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:52:53 +0000 (17:52 -0700)]
report: clarify the meaning of the timestamp attribute
We've never specified what the timestamp attribute of the testsuite
element actually means, and it history is rather murky.
Prior to the introduction of the xml report format in commit f9fde7db2f,
the "date_time" variable was used only to scrape dmesg via the /dev/kmsg
device after each test. If /dev/kmsg was not a writable path, the
variable was not set at all. In this case, the report timestamp would
be blank.
In commit ffdecf7498a1, Ted changed the xunit report code to handle
empty date_time values by setting date_time to the time of report
generation. This change was done to handle the case where no tests are
run at all. However, it did not change the behavior that date_time is
not set if /dev/kmsg is not writable.
Clear up all this confusion by defining the timestamp attribute to
reflect the start time of the most recent test, regardless of the state
of /dev/kmsg. If no tests are run, then define the attribute to be the
time of report generation.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:52:47 +0000 (17:52 -0700)]
report: capture the time zone in the test report timestamp
Make sure we put the time zone of the system running the test in the
timestamp that is recorded in the xunit report. `date "+%F %T"' reports
the local time zone, not UTC.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:52:42 +0000 (17:52 -0700)]
report: derive an xml schema for the xunit report
The "xunit" report format emits an XML document that more or less
follows the junit xml schema. However, there are two major exceptions:
1. fstests does not emit an @errors attribute on the testsuite element
because we don't have the concept of unanticipated errors such as
"unchecked throwables".
2. The system-out/system-err elements sound like they belong under the
testcase element, though the schema itself imprecisely says "while the
test was executed". The schema puts them under the top-level testsuite
element, but we put them under the testcase element.
Define an xml schema for the xunit report format, and update the xml
headers to link to the schema file. This enables consumers of the
reports to check mechanically that the incoming document follows the
format.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:52:36 +0000 (17:52 -0700)]
check: generate section reports between tests
Generate the section report between tests so that the summary report
always reflects the outcome of the most recent test. Two usecases are
envisioned here -- if a cluster-based test runner anticipates that the
testrun could crash the VM, they can set REPORT_DIR to (say) an NFS
mount to preserve the intermediate results. If the VM does indeed
crash, the scheduler can examine the state of the crashed VM and move
the tests to another VM. The second usecase is a reporting agent that
runs in the VM to upload live results to a test dashboard.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Leah Rumancik <leah.rumancik@gmail.com> Tested-by: Leah Rumancik <leah.rumancik@gmail.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
projects / users / hch / xfstests-dev.git / log