Darrick J. Wong [Mon, 13 Nov 2023 17:08:24 +0000 (09:08 -0800)]
common: make helpers for ttyprintk usage
A handful of tests write things to /dev/ttyprintk to make it easier to
pinpoint where in a test something went wrong. This isn't entirely
robust, however, because ttyprintk is an optional feature. In the grand
tradition of kernel design there's also a /dev/kmsg that does nearly the
same thing, is also optional, and there's no documentation spelling out
when one is supposed to use one or the other.
So.
Create a pair of helpers to append messages to the kernel log. One
simply writes its arguments to the kernel log, and the other writes
stdin to the kernel log, stdout, and any other files specified as
arguments.
Underneath the covers, both functions will send the message to
/dev/ttyprintk if available. If it isn't but /dev/kmsg is, they'll
send the messages there, prepending a "[U]" to emulate the only
discernable difference between ttyprintk and kmsg.
If neither are available, then either /dev or the kernel aren't allowing
us to write to the kernel log, and the messages are not logged. The
second helper will still write the messages to stdout.
If this seems like overengineered nonsense, then yes it is.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 13 Nov 2023 17:08:39 +0000 (09:08 -0800)]
generic: test reads racing with slow reflink operations
XFS has a rather slow reflink operation. While a reflink operation is
running, other programs cannot read the contents of the source file,
which is causing latency spikes. Catherine Hoang wrote a patch to
permit reads, since the source file contents do not change. This is a
functionality test for that patch.
[zlang@ adds `_supported_fs generic`]
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Catherine Hoang <catherine.hoang@oracle.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Anand Jain [Wed, 8 Nov 2023 04:28:57 +0000 (12:28 +0800)]
common/btrfs: add _btrfs_get_fsid() helper
We have two instances of reading the btrfs fsid by using the command
'btrfs filesystem show <mnt>' turn this into an easy-to-use helper
function and also use it.
Suggested-by: David Sterba <dsterba@suse.cz> Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Anand Jain [Thu, 2 Nov 2023 11:28:22 +0000 (19:28 +0800)]
btrfs/219: add to the auto group
Add this test case back to the auto group which reverts the
commit e2e7b549380a ("fstests: btrfs/219: remove it from auto group") since
the previously missing kernel commit 5f58d783fd78 ("btrfs: free device in
btrfs_close_devices for a single device filesystem") has already been
integrated.
Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Anand Jain [Thu, 2 Nov 2023 11:28:21 +0000 (19:28 +0800)]
btrfs/219: cloned-device mount capability update
This test case checks for failure of the cloned device mounts, which
is no longer true after the commit a5b8a5f9f835 ("btrfs: support
cloned-device mount capability"). So check for the non-presence the
temp-fsid feature and do not test for the failure of the cloned device
mount.
Reported-by: kernel test robot <oliver.sang@intel.com> Closes: https://lore.kernel.org/oe-lkp/202310251645.5fe5495a-oliver.sang@intel.com Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Anand Jain [Thu, 2 Nov 2023 11:28:20 +0000 (19:28 +0800)]
btrfs/219: fix _cleanup() to successful release the loop-device
When we fail with the message 'We were allowed to mount when we should
have failed,' it will fail to clean up the loop devices, making it
difficult to run further test cases or the same test case again.
So we need a 2nd loop device local variable to release it. Let's
reorganize the local variables to clean them up in the _cleanup() function.
Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Anand Jain [Thu, 2 Nov 2023 11:28:18 +0000 (19:28 +0800)]
common/rc: _fs_sysfs_dname fetch fsid using btrfs tool
Currently _fs_sysfs_dname gets fsid from the findmnt command however
this command provides the metadata_uuid if the device is mounted with
temp-fsid. So instead, use btrfs filesystem show command to know the fsid.
Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Christoph Hellwig [Thu, 9 Nov 2023 16:49:47 +0000 (17:49 +0100)]
xfs/146: require fallocate support
xfs/146 uses the xfs_io falloc subcommand and thus kernel fallocate
support. Ensure the support is present.
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Bill O'Donnell [Fri, 1 Sep 2023 16:18:16 +0000 (11:18 -0500)]
fstests: generic/353 should accomodate other pwrite behaviors
xfs_io pwrite issues a series of block size writes, but there is no
guarantee that the resulting extent(s) will be singular or contiguous.
This behavior is acceptable, but the test is flawed in that it expects
a single extent for a pwrite.
Modify test to use actual blocksize for pwrite and reflink. Also
modify it to accommodate pwrite and reflink that produce different
mapping results.
Signed-off-by: Bill O'Donnell <bodonnel@redhat.com> Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Tue, 24 Oct 2023 11:23:46 +0000 (12:23 +0100)]
btrfs: test snapshotting a subvolume that was just created
Test that snapshotting a new subvolume (created in the current transaction)
that has a btree with a height > 1, works and does not result in a fs
corruption.
This exercises a regression introduced in kernel 6.5 by the kernel commit:
1b53e51a4a8f ("btrfs: don't commit transaction for every subvol create")
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Josef Bacik <josef@toxicpanda.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Amir Goldstein [Mon, 23 Oct 2023 16:32:59 +0000 (19:32 +0300)]
overlay: add test for lowerdir mount option parsing
Check parsing and display of spaces and escaped colons and commans in
lowerdir mount option.
This is a regression test for two bugs introduced in v6.5 with the
conversion to new mount api.
There is another regression of new mount api related to libmount parsing
of escaped commas, but this needs a fix in libmount - this test only
verifies the fixes in the kernel, so it uses LIBMOUNT_FORCE_MOUNT2=always
to force mount(2) and kernel pasring of the comma separated options list.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
The count of failure messages is the same as the count as the "-l 4k"
fstrim invocations. Since this is an 8k-block filesystem, the -l
parameter is clearly incorrect. The test computes random -m and -l
options.
Therefore, create helper functions to guess at the minimum and maximum
length and minlen parameters that can be used with the fstrim program.
In the inner loop of the test, make sure that our choices for -m and -l
fall within those constraints.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 26 Oct 2023 03:12:02 +0000 (20:12 -0700)]
generic/251: don't snapshot $here during a test
Zorro complained that the next patch caused him a regression:
generic/251 249s ... [failed, exit status 1]- output mismatch (see /root/git/xfstests/results//generic/251.out.bad)
--- tests/generic/251.out 2022-04-29 23:07:23.263498297 +0800
+++ /root/git/xfstests/results//generic/251.out.bad 2023-10-22 14:17:07.248059405 +0800
@@ -1,2 +1,5 @@
QA output created by 251
Running the test: done.
+5838a5839
+> aa60581221897d3d7dd60458e1cca2fa ./results/generic/251.full
+!!!Checksums has changed - Filesystem possibly corrupted!!!\n
...
(Run 'diff -u /root/git/xfstests/tests/generic/251.out /root/git/xfstests/results//generic/251.out.bad' to see the entire diff)
Ran: generic/251
Failures: generic/251
Failed 1 of 1 tests
The next patch writes some debugging information into $seqres.full,
which is a file underneat $RESULT_BASE. If the test operator does not
set RESULT_BASE, it will be set to a subdir of $here by default. Since
this test also snapshots the contents of $here before starting its loop,
any logging to $seqres.full on such a system will cause the post-copy
checksum to fail due to a mismatch.
Fix all this by copying $here to $SCRATCH_DEV and checksumming the copy
before the FITRIM stress test begins to avoid problems with $seqres.full.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Thu, 19 Oct 2023 10:06:14 +0000 (11:06 +0100)]
btrfs/298: fix failure when added device supports trim
A btrfs device add command issues a trim on the device if the device
supports trim, and then it outputs a message to stdout informing that it
performed a trim. If that happens it breaks the golden output and the
test fails like this:
btrfs/298 - output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/298.out.bad)
--- tests/btrfs/298.out 2023-10-18 23:29:06.029292800 +0100
+++ /home/fdmanana/git/hub/xfstests/results//btrfs/298.out.bad 2023-10-19 10:54:29.693210881 +0100
@@ -1,2 +1,3 @@
QA output created by 298
+Performing full device TRIM /dev/sdd (100.00GiB) ...
Silence is golden
...
(Run 'diff -u /home/fdmanana/git/hub/xfstests/tests/btrfs/298.out /home/fdmanana/git/hub/xfstests/results//btrfs/298.out.bad' to see the entire diff)
Ran: btrfs/298
Failures: btrfs/298
Failed 1 of 1 tests
Fix this by redirecting the device add's stdout to the $seqres.full file.
Any device add errors are sent to stderr, so we'll notice if errors happen
due to possible future regressions, as it will break the golden output.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Su Yue [Thu, 19 Oct 2023 01:53:56 +0000 (09:53 +0800)]
generic/245: Filter mv error message
Coreutils commit 3cb862ce5f10 ( mv: better diagnostic for 'mv dir x' failure)
was released in v9.4, changed the error message from
'mv: cannot move 'b/t' to 'a/t': Directory not empty' to
'mv: cannot overwrite 'a/t': Directory not empty' in case of
EDQUOT/EEXIST/EISDIR/EMLINK/ENOSPC/ENOTEMPTY/ETXTBSY.
The change breaks generic/245 due to the mismatched output:
generic/245 1s ... - output mismatch (see /root/xfstests-dev/results//generic/245.out.bad)
--- tests/generic/245.out 2023-10-05 11:15:21.124295738 +0800
+++ /root/xfstests-dev/results//generic/245.out.bad 2023-10-05 11:15:23.456315468 +0800
@@ -1,2 +1,2 @@
QA output created by 245
-mv: cannot move 'TEST_DIR/test-mv/ab/aa/' to 'TEST_DIR/test-mv/aa': File exists
+mv: cannot overwrite 'TEST_DIR/test-mv/aa': File exists
...
(Run 'diff -u /root/xfstests-dev/tests/generic/245.out /root/xfstests-dev/results//generic/245.out.bad' to see the entire diff)
Filter out and replace mv error messages to fix the test.
Signed-off-by: Su Yue <glass.su@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 9 Oct 2023 18:18:45 +0000 (11:18 -0700)]
generic/269,xfs/051: don't drop fsstress failures to stdout
Prior to commit f55e46d629, these two tests would run fsstress until it
hit a failure -- ENOSPC in the case of generic/269, and EIO in the case
of xfs/051. These errors are expected, which was why stderr was also
redirected to /dev/null. Commit f55e46d629 removed the stderr
redirection, which has resulted in a 100% failure rate.
Fix this regression by pushing stderr stream to $seqres.full.
Fixes: f55e46d629 ("fstests: redirect fsstress' stdout to $seqres.full instead of /dev/null") Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Mon, 9 Oct 2023 18:18:39 +0000 (11:18 -0700)]
generic/465: only complain about stale disk contents when racing directio
This test does a strange thing with directio -- it races a reader thread
with an appending aio writer thread and checks that the reader thread
only ever sees a (probably short) buffer containing the same contents
that are being read.
However, this has never worked correctly on XFS, which supports
concurrent readers and writers for directio. Say you already have a
file with a single written mapping A:
This implies separate bios for b and c. Both bios are issued, but c
completes first. The ioend for c will extend i_size all the way to
new_EOF. Extent b is still marked unwritten because it hasn't completed
yet.
Next, the test reader slips in and tries to read the range between the
old EOF and the new EOF. The file looks like this now:
So the reader sees "bbbbCCCCC" in the mapping, and the buffer returned
contains a range of zeroes followed by whatever was written to C.
For pagecache IO I would say that i_size should not be extended until
the extending write is fully complete, but the pagecache also
coordinates access so that reads and writes cannot conflict.
However, this is directio. Reads and writes to the storage device can
be issued and acknowledged in any order. I asked Ted and Jan about this
point, and they echoed that for directio it's expected that application
software must coordinate access themselves.
In other words, the only thing that the reader can check here is that
the filesystem is not returning stale disk contents. Amend the test so
that null bytes in the reader buffer are acceptable.
Cc: tytso@mit.edu Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
attempting to find secondary superblock...
found candidate secondary superblock...
+unable to verify superblock, continuing...
+found candidate secondary superblock...
+error reading superblock 1 -- seek to offset 584115421184 failed
+unable to verify superblock, continuing...
+found candidate secondary superblock...
+error reading superblock 1 -- seek to offset 584115421184 failed
+unable to verify superblock, continuing...
+found candidate secondary superblock...
+error reading superblock 1 -- seek to offset 584115421184 failed
+unable to verify superblock, continuing...
+found candidate secondary superblock...
+error reading superblock 1 -- seek to offset 584115421184 failed
+unable to verify superblock, continuing...
+found candidate secondary superblock...
+error reading superblock 1 -- seek to offset 584115421184 failed
+unable to verify superblock, continuing...
+found candidate secondary superblock...
+error reading superblock 1 -- seek to offset 584115421184 failed
+unable to verify superblock, continuing...
+found candidate secondary superblock...
verified secondary superblock...
writing modified primary superblock
sb root inode INO inconsistent with calculated value INO
Eventually I tracked this down to a mis-interaction between the test,
xfs_repair, and the storage device.
If the device doesn't support discard, _scratch_mkfs won't zero the
entire disk to remove old dead superblocks that might have been written
by previous tests. After we shatter the primary super, the xfs_repair
scanning code can still trip over those old supers when it goes looking
for secondary supers.
Most of the time it finds the actual AG 1 secondary super, but sometimes
it finds ghosts from previous formats. When that happens, xfs_repair
will talk quite a bit about those failed secondaries, even if it
eventually finds an acceptable secondary sb and completes the repair.
Filter out the messages about secondary supers.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
If we place /var/lib/xfstests on a read-only filesystem, commands in
_link_out_file_named() fail to modify the files. However, they won't fail
the test. As a result, the test case fails mysteriously with only "no
qualified output" printed.
Fix it by checking the error case.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: "Darrick J. Wong" <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 27 Sep 2023 01:42:58 +0000 (18:42 -0700)]
btrfs/300: check existence of unshare arguments
Make sure the installed unshare binary supports all the arguments that
it wants to use. The unshare program on my system (Ubuntu 22.04)
doesn't support --map-auto, so this test fails unnecessarily.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
_scratch_mkfs_geom: Fix regex used for matching block size option
The regular expression used by _scratch_mkfs_geom() to match mkfs.xfs' block
size argument interprets the character 'b' as optional. It should actually
interpret whitespace as optional.
This causes generic/223 to fail when testing an XFS filesystem which uses an
external log device along with the -lsize option. In this case, the original
value of -lsize is replaced with the value of $blocksize.
_scratch_mkfs_sized() also uses the same incorrect regex.
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org> Reviewed-by: "Darrick J. Wong" <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
btrfs/295: skip on zoned device as we cannot corrupt it directly
We use _pwrite_byte to corrupt the root node, but such overwrite won't work
on a sequential write required zone. So, skip the test on a zoned device.
Technically, we can run this test case by checking if the physical location
lands in a conventional zone. But, the logic should be no difference than
the regular mode and I don't think it's worth doing so.
btrfs: test scan but not register the single device fs
Recently, in the kernel commit 0d9436739af2 ("btrfs: scan but don't
register device on single device filesystem"), we adopted an approach
where we scan the device to validate it. However, we do not register
it in the kernel memory since it is not required to be remembered.
However, the seed device should continue to be registered because
otherwise, the mount operation for the sprout device will fail.
This patch ensures that we honor the mount requirements and do not break
anything while making changes in this part of the code.
Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Tue, 3 Oct 2023 11:57:45 +0000 (12:57 +0100)]
btrfs/192: use append operator to output log replay results to $seqres.full
After doing log replay, btrfs/192 is overwriting the $seqres.full file
because it uses the plain ">" redirect operator, instead of an append
">>" redirect operator. As a consequence it is overriding the file and
eliminating any previous output that may be useful to debug a test
failure (such as the fsstress seed or mkfs results). So use >> instead
of >.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Tue, 3 Oct 2023 11:57:44 +0000 (12:57 +0100)]
fstests: redirect fsstress' stdout to $seqres.full instead of /dev/null
Several tests are redirecting the output of fsstress to /dev/null and this
makes it harder to debug a test failure because we have no way of knowing
what was the seed used by fsstress, as fsstress outputs the seed it uses
to stdout. Very often when such a test fails, I have to go modify to
redirect stdout to the $seqres.full file and then run it in a loop until
I find a seed that causes a failure.
So modify all tests that redirect fsstress' output to /dev/null to instead
redirect it to the $seqres.full file. Note that for some tests I've added
the style ">> $seqres.full" (with a space after >>) while for others I did
">>$seqres.full" (no space) - the reason for this was to keep style
consistency within each test case.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Acked-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
btrfs/283: skip if we cannot write into one extent
On the zoned mode, the extent size is limited also by
queue/zone_append_max_bytes. This breaks the assumption that the file "foo"
has a single extent and corrupts the test output.
It is difficult to support the case, so let's just skip the test in this
case.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Boris Burkov [Thu, 28 Sep 2023 23:16:48 +0000 (16:16 -0700)]
btrfs: skip squota incompatible tests
These tests cannot succeed if mkfs enable squotas, as they either test
the specifics of qgroups behavior or they test *enabling* squotas. Skip
these in squota mode.
Signed-off-by: Boris Burkov <boris@bur.io> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Boris Burkov [Thu, 28 Sep 2023 23:16:46 +0000 (16:16 -0700)]
btrfs: quota rescan helpers
Many btrfs tests explicitly trigger quota rescan. This is not a
meaningful operation for simple quotas, so we wrap it in a helper that
doesn't blow up quite so badly and lets us run those tests where the
rescan is a qgroup detail.
Signed-off-by: Boris Burkov <boris@bur.io> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Boris Burkov [Thu, 28 Sep 2023 23:16:45 +0000 (16:16 -0700)]
btrfs/301: new test for simple quotas
Test some interesting basic and edge cases of simple quotas.
To some extent, this is redundant with the alternate testing strategy of
using MKFS_OPTIONS to enable simple quotas, running the full suite and
relying on kernel warnings and fsck to surface issues.
Signed-off-by: Boris Burkov <boris@bur.io> Signed-off-by: Zorro Lang <zlang@kernel.org>
Boris Burkov [Thu, 28 Sep 2023 23:16:44 +0000 (16:16 -0700)]
btrfs: quota mode helpers
To facilitate skipping tests depending on the qgroup mode after mkfs,
add support for figuring out the mode. This cannot just rely on the new
sysfs file, since it might not be present on older kernels.
Signed-off-by: Boris Burkov <boris@bur.io> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Fri, 22 Sep 2023 11:45:17 +0000 (12:45 +0100)]
btrfs/287: filter snapshot IDs to avoid failures when using some features
When running btrfs/287 with features that create extra trees or don't
the need to create some trees, such as when using the free space tree
(default for several btrfs-progs releases now) versus when not using
it (by passing -R ^free-space-tree in MKFS_OPTIONS), the test can fail
because the IDs for the two snapshots it creates changes, and the golden
output is requiring the numeric IDs of the snapshots.
For example, when disabling the free space tree, the test fails like this:
Filipe Manana [Fri, 22 Sep 2023 11:45:01 +0000 (12:45 +0100)]
btrfs: use full subcommand name at _btrfs_get_subvolid()
Avoid using the shortcut "sub" for the "subvolume" command, as this is the
standard practice because such shortcuts are not guaranteed to exist in
every btrfs-progs release (they may come and go). Also make the variables
local.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Thu, 21 Sep 2023 15:16:34 +0000 (16:16 +0100)]
generic: test new directory entries are returned after rewinding directory
Test that if names are added to a directory after an opendir(3) call and
before a rewinddir(3) call, future readdir(3) calls will return the names.
This is mandated by POSIX:
This exercises a regression in btrfs which is fixed by a kernel patch that
has the following subject:
""btrfs: refresh dir last index during a rewinddir(3) call""
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
btrfs/239: call fsync to create tree-log dedicated block group for zoned mode
Running btrfs/239 on a zoned device often fails with the following error.
btrfs/239 5s ... - output mismatch (see /host/btrfs/239.out.bad)
--- tests/btrfs/239.out 2023-09-21 16:56:37.735204924 +0900
+++ /host/btrfs/239.out.bad 2023-09-21 18:22:45.401433408 +0900
@@ -1,4 +1,6 @@
QA output created by 239
+/testdir/dira still exists
+/dira does not exists
File SCRATCH_MNT/testdir/file1 data: 0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab
*
...
This happens because "testdir" and "dira" are not logged on the first fsync
(fsync $SCRATCH_MNT/testdir), but are written as a full commit. That
prevents updating the log on "mv" time, leaving them pre-mv state.
The full commit is induced by the creation of a new block group. On the
zoned mode, we use a dedicated block group for tree-log. That block group
is created on-demand or assigned to a metadata block group if there is
none. On the first mount of a file system, we need to create one because
there is only one metadata block group available for the regular
metadata. That creation of a new block group forces tree-log to be a full
commit on that transaction, which prevents logging "testdir" and "dira".
Fix the issue by calling fsync before the first "sync", which creates the
dedicated block group and let the files be properly logged.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Amir Goldstein [Thu, 21 Sep 2023 14:31:02 +0000 (17:31 +0300)]
overlay: add test for rename of lower symlink with NOATIME attr
Test for a regression in copy up of symlink that has the S_NOATIME
inode flag.
This is a regression from v5.15 reported by Ruiwen Zhao:
https://lore.kernel.org/linux-unionfs/CAKd=y5Hpg7J2gxrFT02F94o=FM9QvGp=kcH1Grctx8HzFYvpiA@mail.gmail.com/
In the bug report, the symlink has the S_NOATIME inode flag because it is
on an NFS/FUSE filesystem that sets S_NOATIME for all inodes.
The reproducer uses another technique to create a symlink with
S_NOATIME inode flag by using chattr +A inheritance on filesystems
that inherit chattr flags to symlinks.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Fri, 15 Sep 2023 09:26:50 +0000 (10:26 +0100)]
btrfs: add missing commit ids for a few tests using _fixed_by_kernel_commit
The tests btrfs/288, btrfs/289 and btrfs/300 are using the "xxxx..." stub
for commit ids, as when they were submitted/merged the corresponding
btrfs patches were not yet in Linus' tree. So replace the stubs with the
commit ids.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Running btrfs/076 on a zoned null_blk device will fail with the following error.
- output mismatch (see /host/results/btrfs/076.out.bad)
--- tests/btrfs/076.out 2021-02-05 01:44:20.000000000 +0000
+++ /host/results/btrfs/076.out.bad 2023-09-15 01:49:36.000000000 +0000
@@ -1,3 +1,3 @@
QA output created by 076
-80
-80
+83
+83
...
This is because the default value of zone_append_max_bytes is 127.5 KB
which is smaller than BTRFS_MAX_UNCOMPRESSED (128K). So, the extent size is
limited to 126976 (= ROUND_DOWN(127.5K, 4096)), which makes the number of
extents larger, and fails the test.
Instead of hard-coding the number of extents, we can calculate it using the
max extent size of an extent. It is limited by either
BTRFS_MAX_UNCOMPRESSED or zone_append_max_bytes.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Anand Jain [Tue, 29 Aug 2023 12:32:40 +0000 (20:32 +0800)]
fstests: btrfs add more tests into the scrub group
I wanted to verify tests using the command "btrfs scrub start" and
found that there are many more test cases using "btrfs scrub start"
than what is listed in the group.list file. So, get them to the scrub
group.
Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
common/rc: make _get_max_file_size find file size on mount point
Currently, _get_max_file_size finds max file size on $TEST_DIR.
The tests/generic/692 uses this function to detect file size and
then tries to create a file on $SCRATCH_MNT.
This works fine when test and scratch filesystems have the same
block size. However, it will fail if they differ.
Make _get_max_file_size accept mount point on which to detect max
file size.
Signed-off-by: Andrey Albershteyn <aalbersh@redhat.com> Reviewed-by: "Darrick J. Wong" <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Some test cases lack executable permission ('x'). Before running each
test case, `./check` checks and grants them 'x' permission. However,
this always leads to a dirty git repo. And the absence of 'x' permission
in test cases is often overlooked during reviews.
Since maintainers use mvtest to assign new case, add this change for
convenience of maintainers.
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: "Darrick J. Wong" <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
fstests: btrfs/185 update for single device pseudo device-scan
As we are obliterating the need for the device scan for the single device,
which will return success if the basic superblock verification passes,
even for the duplicate device of the mounted filesystem, drop the check
for the return code in this testcase and continue to verify if the device
path of the mounted filesystem remains unaltered after the scan.
Also, if the test fails, it leaves the local non-standard mount point
remained mounted, leading to further test cases failing. Call unmount
in _cleanup().
Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Boris Burkov <boris@bur.io> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Amir Goldstein [Sun, 3 Sep 2023 07:54:11 +0000 (10:54 +0300)]
overlay: add test for persistent unique fsid
Test overlayfs fsid behavior with new mount options uuid=null/on
that were introduced in kernel v6.6:
- Test inherited upper fs fsid with mount option uuid=off/null
- Test uuid=null behavior for existing overlayfs by default
- Test persistent unique fsid with mount option uuid=on
- Test uuid=on behavior for new overlayfs by default
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Anand Jain [Tue, 29 Aug 2023 12:34:06 +0000 (20:34 +0800)]
fstests: btrfs/261 fix failure if /var/lib/btrfs isn't writable
We don't need scrub status; it is okay to ignore the warnings due to
the readonly /var/lib/btrfs if any. Redirect stderr to seqres.full.
We check the scrub return status.
+WARNING: failed to open the progress status socket at /var/lib/btrfs/scrub.progress.42fad803-d505-48f4-a04d-612dbf8bd724: Read-only file system. Progress cannot be queried
+WARNING: failed to write the progress status file: Read-only file system. Status recording disabled
Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Anand Jain [Mon, 21 Aug 2023 09:05:09 +0000 (17:05 +0800)]
fstests: use btrfs check repair for repairing btrfs filesystems
There are two repair functions: _repair_scratch_fs() and
_repair_test_fs(). As the names suggest, these functions are designed to
repair the filesystems SCRATCH_DEV and TEST_DEV, respectively. However,
these functions never called proper comamnd for the filesystem type btrfs.
This patch fixes it. Thx.
Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 29 Aug 2023 23:03:49 +0000 (16:03 -0700)]
xfs/559: adapt to kernels that use large folios for writes
The write invalidation code in iomap can only be triggered for writes
that span multiple folios. If the kernel reports a huge page size,
scale up the write size.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 29 Aug 2023 23:03:37 +0000 (16:03 -0700)]
common: split _get_hugepagesize into detection and actual query
This helper has two parts -- querying the value, and _notrun'ing the
test if huge pages aren't turned on. Break these into the usual
_require_hugepages and _get_hugepagesize predicates so that we can adapt
xfs/559 to large folios being used for writes.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 24 Aug 2023 23:47:14 +0000 (16:47 -0700)]
btrfs/282: skip test if /var/lib/btrfs isnt writable
I run fstests in a readonly container, and accidentally uninstalled the
btrfsprogs package. When I did, this test started faililng:
--- btrfs/282.out
+++ btrfs/282.out.bad
@@ -1,3 +1,7 @@
QA output created by 282
wrote 2147483648/2147483648 bytes at offset 0
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+WARNING: cannot create scrub data file, mkdir /var/lib/btrfs failed: Read-only file system. Status recording disabled
+WARNING: failed to open the progress status socket at /var/lib/btrfs/scrub.progress.3e1cf8c6-8f8f-4b51-982c-d6783b8b8825: No such file or directory. Progress cannot be queried
+WARNING: cannot create scrub data file, mkdir /var/lib/btrfs failed: Read-only file system. Status recording disabled
+WARNING: failed to open the progress status socket at /var/lib/btrfs/scrub.progress.3e1cf8c6-8f8f-4b51-982c-d6783b8b8825: No such file or directory. Progress cannot be queried
Skip the test if /var/lib/btrfs isn't writable, or if /var/lib isn't
writable, which means we cannot create /var/lib/btrfs.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jeff Layton [Fri, 1 Sep 2023 17:39:57 +0000 (13:39 -0400)]
generic/187: don't run this test on NFS
This test is unreliable on NFS. It fails consistently when run vs. a
server exporting btrfs, but passes when the server exports xfs. Since we
don't have any sort of attribute that we can require to test this, just
skip this one on NFS.
Also, subsume the check for btrfs into the _supported_fs check, and add
a comment for it.
Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jeff Layton [Fri, 1 Sep 2023 17:39:56 +0000 (13:39 -0400)]
generic/357: don't run this test on NFS
NFS doesn't keep track of whether a file is reflinked or not, so it
doesn't prevent this behavior. It shouldn't be a problem for NFS anyway,
so just skip this test there.
Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jeff Layton [Fri, 1 Sep 2023 17:39:55 +0000 (13:39 -0400)]
generic/294: don't run this test on NFS
When creating a new dentry (of any type), NFS will optimize away any
on-the-wire lookups prior to the create since that means an extra
round trip to the server. Because of that, it consistently fails this
test.
Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jeff Layton [Wed, 30 Aug 2023 10:58:52 +0000 (06:58 -0400)]
generic/*: add a check for security attrs
There are several generic tests that require "setcap", but don't check
whether the underlying fs supports security attrs. Add the appropriate
checks.
Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jeff Layton [Wed, 30 Aug 2023 10:58:51 +0000 (06:58 -0400)]
generic/578: add a check to ensure that fiemap is supported
This test requires FIEMAP support.
Suggested-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jeff Layton [Wed, 30 Aug 2023 10:58:50 +0000 (06:58 -0400)]
common/attr: fix the _require_acl test
_require_acl tests whether you're able to fetch the ACL from a file
using chacl, and then tests for an -EOPNOTSUPP error return.
Unfortunately, filesystems that don't support them (like NFSv4) just
return -ENODATA when someone calls getxattr for the POSIX ACL, so the
test doesn't work.
Fix the test to have chacl set an ACL on the file instead, which should
reliably fail on filesystems that don't support them.
Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 1 Sep 2023 14:53:31 +0000 (07:53 -0700)]
generic/61[67]: support SOAK_DURATION
Now that I've finally gotten liburing installed on my test machine, I
can actually test io_uring. Adapt these two tests to support
SOAK_DURATION so I can add it to that too.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 29 Aug 2023 23:07:58 +0000 (16:07 -0700)]
generic/650: add SOAK_DURATION controls
Make this test controllable via SOAK_DURATION, for anyone who wants to
perform a long soak test of filesystem vs. cpu hotplug.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Naohiro Aota [Wed, 30 Aug 2023 02:17:52 +0000 (11:17 +0900)]
btrfs/237: kick reclaim process with a small filesystem
Since commit 3687fcb0752a ("btrfs: zoned: make auto-reclaim less
aggressive"), the reclaim process won't run unless the 75% (by default) of
the filesystem volume is allocated as block groups. As a result, btrfs/237
won't success when it is run with a large volume.
To run the reclaim process, we need to either fill the FS to the desired
level, or make a small FS so that the test write can go over the level.
Since the current test code expects the FS has only one data block group,
filling the FS is both cumbersome and need effort to rewrite the test code.
So, we take the latter method. We create a small (16 * zone size) FS. The
size is chosen to hold a minimal FS with DUP metadata setup.
However, creating a small FS is not enough. With SINGLE metadata setup, we
allocate 3 zones (one for each DATA, METADATA and SYSTEM), which is less
than 75% of 16 zones. We can tweak the threshold to 51% on regular btrfs
kernel config (!CONFIG_BTRFS_DEBUG), but that is still not enough to start
the reclaim process. So, this test requires CONFIG_BTRFS_DEBUG to set the
threshold to 1%.
Bill O'Donnell [Mon, 28 Aug 2023 17:23:07 +0000 (12:23 -0500)]
fstests: generic/352 should accomodate other pwrite behaviors
xfs_io pwrite issues a series of block size writes, but there is no
guarantee that the resulting extent(s) will be singular or contiguous.
This behavior is acceptable, but the test is flawed in that it expects
a single extent for a pwrite.
Modify test to limit pwrite and reflink to a single block.
Signed-off-by: Bill O'Donnell <bodonnel@redhat.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 23 Aug 2023 01:02:39 +0000 (18:02 -0700)]
fstests: test fix for an agbno overflow in __xfs_getfsmap_datadev
Dave Chinner reported that xfs/273 fails if the AG size happens to be an
exact power of two. I traced this to an agbno integer overflow when the
current GETFSMAP call is a continuation of a previous GETFSMAP call, and
the last record returned was non-shareable space at the end of an AG.
This is the regression test for that bug.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Naohiro Aota [Tue, 22 Aug 2023 07:28:50 +0000 (16:28 +0900)]
aio-dio-write-verify: check for the IO errors
The async write IOs can return some errors, which may lead to a short read or
corruption in io_verify() stage. Catch an error early to identify the root
cause easily.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Yang Xu [Mon, 14 Aug 2023 14:41:41 +0000 (22:41 +0800)]
generic/471: Remove this broken case
I remember this case fails on last year becuase of
kernel commit cae2de69 ("iomap: Add async buffered write support")
kernel commit 1aa91d9 ("xfs: Add async buffered write support").
as below:
pwrite: Resource temporarily unavailable
wrote 8388608/8388608 bytes at offset 0
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-RWF_NOWAIT time is within limits.
+pwrite: Resource temporarily unavailable
+(standard_in) 1: syntax error
+RWF_NOWAIT took seconds
So For async buffered write requests, the request will return -EAGAIN
if the ilock cannot be obtained immediately.
Here also a discussion[1] that seems generic/471 has been broken.
Now, I met this problem in my linux distribution, then I found the above
discussion. IMO, remove this case is ok and then we can avoid to meet this
false report again.
[Additional information from Dave Chinner]
We changed how timestamps are updated so that they are aware of
IOCB_NOWAIT. If the IOCB_NOWIAT DIO write now needs to update the
inode timestamps, it will return -EAGAIN instead of doing
potentially blocking operations that require IO to complete (i.e.
taking a transaction reservation).
Hence the first time we go to do a DIO read an inode, it's going to
do an atime update, which now occurrs from an IOCB_NOWAIT context
and we return -EAGAIN....
Yes, we added non-blocking timestamp updates as part of the async
buffered write support, but this was a general XFS IO path change of
behaviour to address a potential blocking point in *all* IOCB_NOWAIT
reads and writes, buffered or direct.
The test is not validating that RWF_NOWAIT is behaving correctly - it
just was a simple operation that kinda exercised RWF_NOWAIT semantics
when we had no other way to test this code. It has outlived it's
original purpose, so it should be removed...
Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com> Reviewed-by: Bill O'Donnell <bodonnel@redhat.com> Reviewed-by: Jens Axboe <axboe@kernel.dk> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Note the 0/33, when the data csum mismatch triggered, it always fail
with -4 (-EINTR).
It looks like with lucky enough concurrency, we can get to the following
situation inside fsstress:
Process A | Process B
-----------------------------------+---------------------------------------
do_aio_rw() |
|- io_sumit(); |
|- io_get_events(); |
| Returned -EINTR, but IO hasn't |
| finished. |
`- free(buf); | malloc();
| Got the same memory of @buf from
| thread A.
| Modify the memory
| Now the buffer is changed while
| still under IO
This is the typical buffer modification during direct IO, which is going
to cause csum mismatch for btrfs, and btrfs properly detects it.
This is the direct cause of the problem.
The root cause is that, io_uring would use signals to handle
submission/completion of IOs.
Thus io_uring operations would interrupt AIO operations, thus causing
the above problem.
[FIX]
To fix the problem, we can just retry io_getevents() so that we can
properly wait for the IO.
This prevents us to modify the IO buffer before writeback really
finishes.
With this fixes, I can no longer reproduce the data corruption.
Naohiro Aota [Mon, 21 Aug 2023 07:12:11 +0000 (16:12 +0900)]
common/rc: introduce _random_file() helper
Currently, we use "ls ... | sort -R | head -n1" (or tail) to choose a
random file in a directory.It sorts the files with "ls", sort it randomly
and pick the first line, which wastes the "ls" sort.
Also, using "sort -R | head -n1" is inefficient. For example, in a
directory with 1000000 files, it takes more than 15 seconds to pick a file.
$ time bash -c "ls -U | sort -R | head -n 1 >/dev/null"
bash -c "ls -U | sort -R | head -n 1 >/dev/null" 15.38s user 0.14s system 99% cpu 15.536 total
$ time bash -c "ls -U | shuf -n 1 >/dev/null"
bash -c "ls -U | shuf -n 1 >/dev/null" 0.30s user 0.12s system 138% cpu 0.306 total
So, we should just use "ls -U" and "shuf -n 1" to choose a random file.
Introduce _random_file() helper to do it properly.
Lee Trager [Fri, 18 Aug 2023 08:11:56 +0000 (01:11 -0700)]
fstests: Verify dir permissions when creating a stub subvolume
btrfs supports creating nesting subvolumes however snapshots are not
recurive. When a snapshot is taken of a volume which contains a subvolume
the subvolume is replaced with a stub subvolume which has the same name and
uses inode number 2. This test validates that the stub volume copies
permissions of the original volume.
Signed-off-by: Lee Trager <lee@trager.us> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Disseldorp [Wed, 16 Aug 2023 10:33:30 +0000 (12:33 +0200)]
common/rc: drop 'fsck -f' parameter from _repair_test_fs
The '-f' parameter is fsck.ext# specific, where it's documented to:
Force checking even if filesystem is marked clean
_repair_test_fs() is only called on _check_test_fs() failure, so
dropping the parameter should be possible without changing ext#
behaviour.
Doing so fixes _repair_test_fs() on exfat, where fsck.exfat doesn't
support '-f'.
Signed-off-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Amir Goldstein [Tue, 15 Aug 2023 08:28:35 +0000 (11:28 +0300)]
generic/{175,297,298}: fix use of uninitialized var
The truncate command in those tests use an uninitialized variable i.
in kdevops, i must contain some leftover, so we get errors like:
/data/fstests-install/xfstests/tests/generic/298: line 45: /dev/loop12):
syntax error: operand expected (error token is "/dev/loop12)")
Apparently, noone including the author of the tests knows why this
truncate command is in the test, so remove the wrong truncate command.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Amir Goldstein [Mon, 14 Aug 2023 17:31:32 +0000 (20:31 +0300)]
check: fix parsing expunge file with comments
commit 60054d51 ("check: fix excluded tests are only expunged in the
first iteration") change to use exclude_tests array instead of file.
The check if a test is in expunge file was using grep -q $TEST_ID FILE
so it was checking if the test was a non-exact match to one of the
lines, for a common example: "generic/001 # exclude this test" would be
a match to test generic/001.
The commit regressed this example, because the new code checks for exact
match of [ "generic/001" == "generic/001 " ]. Change the code to match a
regular expression to deal with this case and any other suffix correctly.
NOTE that the original code would have matched test generic/100 with lines
like "generic/1000" when we get to 4 digit seqnum, so the regular
expression does an exact match to the first word of the line.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Shiyang Ruan [Mon, 14 Aug 2023 03:55:03 +0000 (11:55 +0800)]
fsx: tidy options usage and format
1. Add missing options and wrap the cli example line.
2. Cleanup and also add missing "-K" operation for options description
part.
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Stas Sergeev [Tue, 1 Aug 2023 17:52:20 +0000 (22:52 +0500)]
t_ofd_locks: fix sem initialization sequence
The locker was waiting for sem_otime on sem0 to became non-zero after
incrementing sem0 himself. So sem_otime was never 0 at the time of
checking it, so the check was redundant/wrong.
This patch:
- moves the increment of sem1 to the lock-tester site
- lock-setter waits for that sem1 event, for which this patch replaces
the wait loop on sem_otime with GETVAL loop, adding a small sleep
- increment of sem0 to 2 moved past that sem1 event. That sem0 event
is currently not used/waited.
This guarantees that the lock-setter is working only after lock-getter
is fully initialized.
CC: fstests@vger.kernel.org CC: Murphy Zhou <xzhou@redhat.com> CC: Jeff Layton <jlayton@kernel.org> CC: Zorro Lang <zlang@redhat.com> Signed-off-by: Stas Sergeev <stsp2@yandex.ru> Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Stas Sergeev [Tue, 1 Aug 2023 17:52:19 +0000 (22:52 +0500)]
t_ofd_locks: fix stalled semaphore handling
Currently IPC_RMID was attempted on a semid returned after failed
semget() with flags=IPC_CREAT|IPC_EXCL. So nothing was actually removed.
This patch introduces the much more reliable scheme where the wrapper
script creates and removes semaphores, passing a sem key to the test
binary via new -K option.
This patch speeds up the test ~5 times by removing the sem-awaiting
loop in a lock-getter process. As the semaphore is now created before
the test process started, there is no need to wait for anything.
CC: fstests@vger.kernel.org CC: Murphy Zhou <xzhou@redhat.com> CC: Jeff Layton <jlayton@kernel.org> CC: Zorro Lang <zlang@redhat.com> Signed-off-by: Stas Sergeev <stsp2@yandex.ru> Reviwed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 4 Aug 2023 21:34:19 +0000 (14:34 -0700)]
xfs: skip fragmentation tests when alwayscow mode is enabled, part 2
If the always_cow debugging flag is enabled, all file writes turn into
copy writes. This dramatically ramps up fragmentation in the filesystem
(intentionally!) so there's no point in complaining about fragmentation.
I missed these two in the original commit because readahead for md5sum
would create large folios at the start of the file. This resulted in
the fdatatasync after the random writes issuing writeback for the whole
large folio, which reduced file fragmentation to the point where this
test started passing.
With Ritesh's patchset implementing sub-folio dirty tracking, this test
goes back to failing due to high fragmentation (as it did before large
folios) so we need to mask these off too.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 4 Aug 2023 21:17:48 +0000 (14:17 -0700)]
generic/642: fix SOAK_DURATION usage in generic/642
Misspelled variable name. Yay bash.
Fixes: 3e85dd4fe4 ("misc: add duration for long soak tests") Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Luis Chamberlain [Thu, 3 Aug 2023 19:18:41 +0000 (12:18 -0700)]
fstests: add helper to canonicalize devices used to enable persistent disks
The filesystem configuration file does not allow you to use symlinks to
real devices given the existing sanity checks verify that the target end
device matches the source. Device mapper links work but not symlinks for
real drives do not.
Using a symlink is desirable if you want to enable persistent tests
across reboots. For example you may want to use /dev/disk/by-id/nvme-eui.*
so to ensure that the same drives are used even after reboot. This
is very useful if you are testing for example with a virtualized
environment and are using PCIe passthrough with other qemu NVMe drives
with one or many NVMe drives.
To enable support just add a helper to canonicalize devices prior to
running the tests.
This allows one test runner, kdevops, which I just extended with
support to use real NVMe drives it has support now to use nvme EUI
symlinks and fallbacks to nvme model + serial symlinks as not all
NVMe drives support EUIs. The drives it uses for the filesystem
configuration optionally is with NVMe eui symlinks so to allow
the same drives to be used over reboots.
For instance this works today with real nvme drives:
generic/110 2s
Ran: generic/110
Passed all 1 tests
But this does not:
TEST_DIR=/mnt TEST_DEV=/dev/disk/by-id/nvme-eui.0035385411904c1e FSTYP=xfs ./check generic/110
mount: /mnt: /dev/disk/by-id/nvme-eui.0035385411904c1e already mounted on /mnt.
common/rc: retrying test device mount with external set
mount: /mnt: /dev/disk/by-id/nvme-eui.0035385411904c1e already mounted on /mnt.
common/rc: could not mount /dev/disk/by-id/nvme-eui.0035385411904c1e on /mnt
umount /mnt
TEST_DIR=/mnt TEST_DEV=/dev/disk/by-id/nvme-eui.0035385411904c1e FSTYP=xfs ./check generic/110
TEST_DEV=/dev/disk/by-id/nvme-eui.0035385411904c1e is mounted but not on TEST_DIR=/mnt - aborting
Already mounted result:
/dev/disk/by-id/nvme-eui.0035385411904c1e /mnt
This fixes this. This allows the same real drives for a test to be
used over and over after reboots.
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 26 Jul 2023 01:56:51 +0000 (18:56 -0700)]
check: generate gcov code coverage reports at the end of each section
Support collecting kernel code coverage information as reported in
debugfs. At the start of each section, we reset the gcov counters;
during the section wrapup, we'll collect the kernel gcov data.
If lcov is installed and the kernel source code is available, it will
also generate a nice html report. If a CLI web browser is available, it
will also format the html report into text for easy grepping.
This requires the test runner to set REPORT_GCOV=1 explicitly and gcov
to be enabled in the kernel.
Cc: tytso@mit.edu Cc: kent.overstreet@linux.dev Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Thu, 3 Aug 2023 11:37:46 +0000 (12:37 +0100)]
btrfs/276: make test accurate regarding number of expected extents
btrfs/276 creates a 16G file with compression enabled in order to quickly
and efficiently create a file with many extents and have a fs tree with a
height of 3 (root node at level 2), so that it can test that fiemap is
correctly reporting extent sharedness when we have shared subtrees of the
fs tree due to a snapshot.
Compression results in extents with a maximum size of 128K and the test
is expecting only extents of 128K, which normally happens if the machine
has a large amount of RAM and writeback is not triggered before the xfs_io
command finishes. However if writeback is triggered in the meanwhile, due
to memory pressure for example, then we can get end up with some extents
that are smaller than 128K, therefore increasing the total number of
extents in the test file and make the test fail.
This seems to happen often on test machines with small amounts of RAM,
such as 4G, as reported by Qu in the following thread:
So to address this create a file with holes and direct IO to make sure we
always get a specific number of extents in the test file. To speedup the
test create 2000 64K extents, with holes in between them, so that it works
on a fs with any sector size, and then create a bunch of files with large
xattrs to quickly bump the fs tree height to 3 for any node size (4K to
64K). This also guarantees that the file extent items are spread over
multiples leaves, in order to exercise fiemap's correctness when reporting
shared extents due to shared subtrees.
Reported-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Qu Wenruo <wqu@suse.com> Tested-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Thu, 27 Jul 2023 18:53:14 +0000 (02:53 +0800)]
fstests: add smoketest group
Darrick suggests that fstests can provide a simple smoketest, by
running several generic filesystem smoke testing for five minutes
apiece (SOAK_DURATION="5m"). Since there are only five smoke tests,
this is effectively a 20min super-quick test.
With gcov enabled, running these tests yields about ~75% coverage for
iomap and ~60% for xfs; or ~50% for ext4 and ~75% for ext4; and ~45%
for btrfs. Coverage was about ~65% for the pagecache.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
projects / users / hch / xfstests-dev.git / log