]> www.infradead.org Git - users/hch/xfstests-dev.git/log
users/hch/xfstests-dev.git
14 months agobtrfs: new test for devt change between mounts
Boris Burkov [Mon, 11 Mar 2024 19:13:44 +0000 (12:13 -0700)]
btrfs: new test for devt change between mounts

It is possible to confuse the btrfs device cache (fs_devices) by
starting with a multi-device filesystem, then removing and re-adding a
device in a way which changes its dev_t while the filesystem is
unmounted. After this procedure, if we remount, then we are in a funny
state where struct btrfs_device's "devt" field does not match the bd_dev
of the "bdev" field. I would say this is bad enough, as we have violated
a pretty clear invariant.

But for style points, we can then remove the extra device from the fs,
making it a single device fs, which enables the "temp_fsid" feature,
which permits multiple separate mounts of different devices with the
same fsid. Since btrfs is confused and *thinks* there are different
devices (based on device->devt), it allows a second redundant mount of
the same device (not a bind mount!). This then allows us to corrupt the
original mount by doing stuff to the one that should be a bind mount.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
[ use _create_loop_device, renamed $MNT $BIND and rm them before mkdir ]
Signed-off-by: Zorro Lang <zlang@kernel.org>
[ update the commit id of _fixed_by_kernel_commit ]

15 months agoxfs: don't run tests that require v4 file systems when not supported
Christoph Hellwig [Thu, 18 Apr 2024 07:40:46 +0000 (09:40 +0200)]
xfs: don't run tests that require v4 file systems when not supported

Add a _require_xfs_nocrc helper that checks that we can mkfs and mount
a crc=0 file systems before running tests that rely on it to avoid failures
on kernels with CONFIG_XFS_SUPPORT_V4 disabled.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agoxfs/{158,160}: split out v4 tests
Christoph Hellwig [Thu, 18 Apr 2024 07:40:45 +0000 (09:40 +0200)]
xfs/{158,160}: split out v4 tests

Move the subtests that check we can't upgrade v4 file systems to a
separate test.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agoxfs/512: split out v4 specific tests
Christoph Hellwig [Thu, 18 Apr 2024 07:40:44 +0000 (09:40 +0200)]
xfs/512: split out v4 specific tests

Split the v4-specific tests into a new xfs/613.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agoxfs/263: split out the v4 test
Christoph Hellwig [Thu, 18 Apr 2024 07:40:43 +0000 (09:40 +0200)]
xfs/263: split out the v4 test

Move the v4-specific test into a separate test case so that we can still
run the tests on a kernel without v4 support.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agoxfs/045: don't force v4 file systems
Christoph Hellwig [Thu, 18 Apr 2024 07:40:42 +0000 (09:40 +0200)]
xfs/045: don't force v4 file systems

xfs_db can change UUIDs on v5 filesystems now, so we don't need the
-mcrc=0 in this test.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agocommon/quota: update keywords of quota feature in _require_prjquota() for f2fs
Chao Yu [Tue, 16 Apr 2024 10:16:50 +0000 (18:16 +0800)]
common/quota: update keywords of quota feature in _require_prjquota() for f2fs

Previously, in f2fs, sysfile quota feature has different name:
- "quota" in mkfs.f2fs
- and "quota_ino" in dump.f2fs

Now, it has unified the name to "quota" since commit 92cc5edeb7
("f2fs-tools: reuse feature_table to clean up print_sb_state()").

It needs to update keywords "quota" in _require_prjquota() for f2fs,
Otherwise, quota testcase will fail as below.

generic/383 1s ... [not run] quota sysfile not enabled in this device /dev/vdc

This patch keeps keywords "quota_ino" in _require_prjquota() to
keep compatibility for old f2fs-tools.

Cc: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: Chao Yu <chao@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agogeneric/645: Add hint for expected failure with old kernel
Taylor Jackson [Wed, 17 Apr 2024 19:34:24 +0000 (19:34 +0000)]
generic/645: Add hint for expected failure with old kernel

The following hint is added to reflect that any old kernel
without kernel commit dacfd001eaf2 (“fs/mnt_idmapping.c: Return
-EINVAL when no map is written”) is expected to fail this generic
645 test since without that commit, mount_setattr won’t return
EINVAL when attempting to create an idmapped mount using a user
namespace with no mappings.

Reported-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Taylor Jackson <tjackson9431@gmail.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Christian Brauner <brauner@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agogeneric/732: don't run it on overlayfs
Amir Goldstein [Wed, 17 Apr 2024 13:01:02 +0000 (16:01 +0300)]
generic/732: don't run it on overlayfs

The test tries to mount with same mount options on two different
mount points.

Overlayfs does not support doing that.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agoxfs/522: use reflink instead of crc as test feature
Christoph Hellwig [Mon, 8 Apr 2024 13:32:42 +0000 (15:32 +0200)]
xfs/522: use reflink instead of crc as test feature

Replace crc as the main test feature with reflink so that this test
do not require v4 file system support.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agoxfs/078: remove the 512 byte block size sub-case
Christoph Hellwig [Mon, 8 Apr 2024 13:32:40 +0000 (15:32 +0200)]
xfs/078: remove the 512 byte block size sub-case

512 byte block sizes are only supported for v4 file systems, and
xfs/078 crudely forces use of v4 file systems for it.  This doesn't
work if the kernel is built without v4 support.  Given that v4
support is slowly being phased out and 512 byte block sizes have never
been common, drop this part of the test.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agoremove xfs/096
Christoph Hellwig [Mon, 8 Apr 2024 13:32:39 +0000 (15:32 +0200)]
remove xfs/096

This test exercises mkfs error handling before strict validation was added
and thus is useless for xfsprogs > 4.5.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agoxfs: remove support for tools and kernels without v5 support
Christoph Hellwig [Mon, 8 Apr 2024 13:32:38 +0000 (15:32 +0200)]
xfs: remove support for tools and kernels without v5 support

v5 file systems have been the default for more than 10 years.  Drop
support for non-v5 enabled kernels and xfsprogs.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agoext4/01{2,9}: fix invalid filesystem option 'journal'
Luis Henriques (SUSE) [Mon, 1 Apr 2024 09:57:08 +0000 (10:57 +0100)]
ext4/01{2,9}: fix invalid filesystem option 'journal'

Creating an ext4 filesystem using '-O journal' will fail with:

    Invalid filesystem option set: journal

Fix it by replacing it by '-O has_journal', which ensures the filesystem
(ext3 or ext4) is created with a journal.  While there, also redirect stderr
and stdout to the full log.

Signed-off-by: "Luis Henriques (SUSE)" <luis.henriques@linux.dev>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agocommon/btrfs: lookup running processes using pgrep
Anand Jain [Mon, 1 Apr 2024 05:56:03 +0000 (13:56 +0800)]
common/btrfs: lookup running processes using pgrep

Certain helper functions and the testcase btrfs/132 use the following
script to find running processes:

while ps aux | grep "balance start" | grep -qv grep; do
<>
done

Instead, using pgrep is more efficient.

while pgrep -f "btrfs balance start" > /dev/null; do
<>
done

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
15 months agobtrfs/06[0-9]..07[0-4]: kill all background tasks when test is killed/interrupted
Filipe Manana [Wed, 27 Mar 2024 17:11:44 +0000 (17:11 +0000)]
btrfs/06[0-9]..07[0-4]: kill all background tasks when test is killed/interrupted

Test cases btrfs/06[0-9] and btrfs/07[0-4] exercise multiple concurrent
operations while fsstress is running in parallel, and all these are left
as child processes running in the background, which are correctly stopped
if the tests are not interrupted/killed. However if any of these tests is
interrupted/killed, it often leaves child processes still running in the
background, which prevent further running fstests again. For example:

  $ /check -g auto
  (...)
  btrfs/060 394s ...  264s
  btrfs/061 83s ...  69s
  btrfs/062 109s ...  105s
  btrfs/063 52s ...  67s
  btrfs/064 53s ...  51s
  btrfs/065 88s ...  271s
  btrfs/066 127s ...  241s
  btrfs/067 435s ...  248s
  btrfs/068 161s ... ^C^C
  ^C

  $ ./check btrfs/068
  FSTYP         -- btrfs
  PLATFORM      -- Linux/x86_64 debian0 6.8.0-rc7-btrfs-next-153+ #1 SMP PREEMPT_DYNAMIC Mon Mar  4 17:19:19 WET 2024
  MKFS_OPTIONS  -- /dev/sdb
  MOUNT_OPTIONS -- /dev/sdb /home/fdmanana/btrfs-tests/scratch_1

  our local _scratch_mkfs routine ...
  btrfs-progs v6.6.2
  See https://btrfs.readthedocs.io for more information.

  ERROR: unable to open /dev/sdb: Device or resource busy
  check: failed to mkfs $SCRATCH_DEV using specified options
  Interrupted!
  Passed all 0 tests

In this case there was still a process running _btrfs_stress_subvolume()
from common/btrfs.

This is a bit annoying because it requires manually finding out which
process is preventing unmounting the scratch device and then properly
stop/kill it.

So fix this by adding a _cleanup() function to all these tests and then
making sure it stops all the child processes it spawned and are running
in the background.

All these tests have the same structure as they were part of the same
patchset and from the same author.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
15 months agobtrfs: remove stop file early at _btrfs_stress_subvolume
Filipe Manana [Wed, 27 Mar 2024 17:11:43 +0000 (17:11 +0000)]
btrfs: remove stop file early at _btrfs_stress_subvolume

Instead of having every test case that uses _btrfs_stress_subvolume()
removing the stop file before calling that function, do the file
remove at _btrfs_stress_subvolume(). There's no point in doing it in
every single test case.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
15 months agobtrfs: add helper to stop background process running _btrfs_stress_subvolume
Filipe Manana [Wed, 27 Mar 2024 17:11:42 +0000 (17:11 +0000)]
btrfs: add helper to stop background process running _btrfs_stress_subvolume

We have this logic to stop a process running _btrfs_stress_subvolume()
spread in several test cases:

   touch $stop_file
   wait $subvol_pid

Add a helper to encapsulate that logic and also remove the stop file after
the process terminated as there's no point having it around anymore.

This will help to avoid repeating the same code again several times in
upcoming changes.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
15 months agobtrfs: add helper to kill background process running _btrfs_stress_replace
Filipe Manana [Wed, 27 Mar 2024 17:11:41 +0000 (17:11 +0000)]
btrfs: add helper to kill background process running _btrfs_stress_replace

Killing a background process running _btrfs_stress_replace() is not as
simple as sending a signal to the process and waiting for it to die.
Therefore we have the following logic to terminate such process:

   kill $pid
   wait $pid
   while ps aux | grep "replace start" | grep -qv grep; do
      sleep 1
   done

Since this is repeated in several test cases, move this logic to a common
helper and use it in all affected test cases. This will help to avoid
repeating the same code again several times in upcoming changes.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
15 months agobtrfs: add helper to kill background process running _btrfs_stress_remount_compress
Filipe Manana [Wed, 27 Mar 2024 17:11:40 +0000 (17:11 +0000)]
btrfs: add helper to kill background process running _btrfs_stress_remount_compress

Killing a background process running _btrfs_stress_remount_compress() is
not as simple as sending a signal to the process and waiting for it to
die. Therefore we have the following logic to terminate such process:

    kill $pid
    wait $pid
    while ps aux | grep "mount.*$SCRATCH_MNT" | grep -qv grep; do
        sleep 1
    done

Since this is repeated in several test cases, move this logic to a common
helper and use it in all affected test cases. This will help to avoid
repeating the same code again several times in upcoming changes.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
[ Restore 'wait $fsstress_pid' before 'kill $replace_pid' ]

15 months agobtrfs: add helper to kill background process running _btrfs_stress_defrag
Filipe Manana [Wed, 27 Mar 2024 17:11:39 +0000 (17:11 +0000)]
btrfs: add helper to kill background process running _btrfs_stress_defrag

Killing a background process running _btrfs_stress_defrag() is not as
simple as sending a signal to the process and waiting for it to die.
Therefore we have the following logic to terminate such process:

       kill $pid
       wait $pid
       while ps aux | grep "btrfs filesystem defrag" | grep -qv grep; do
           sleep 1
       done

Since this is repeated in several test cases, move this logic to a common
helper and use it in all affected test cases. This will help to avoid
repeating the same code again several times in upcoming changes.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
15 months agobtrfs: add helper to kill background process running _btrfs_stress_scrub
Filipe Manana [Wed, 27 Mar 2024 17:11:38 +0000 (17:11 +0000)]
btrfs: add helper to kill background process running _btrfs_stress_scrub

Killing a background process running _btrfs_stress_scrub() is not as
simple as sending a signal to the process and waiting for it to die.
Therefore we have the following logic to terminate such process:

   kill $pid
   wait $pid
   while ps aux | grep "scrub start" | grep -qv grep; do
       sleep 1
   done

Since this is repeated in several test cases, move this logic to a common
helper and use it in all affected test cases. This will help to avoid
repeating the same code again several times in upcoming changes.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
15 months agobtrfs/028: removed redundant sync and scratch filesystem unmount
Filipe Manana [Wed, 27 Mar 2024 17:11:37 +0000 (17:11 +0000)]
btrfs/028: removed redundant sync and scratch filesystem unmount

There's no need to have an explicit scratch filesystem sync and unmount
at the of the test, as the fstests framework automatically unmounts the
filesystem and the unmount naturally syncs any data and metadata.

So remove them and update the comment to be more clear.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
15 months agobtrfs/028: use the helper _btrfs_kill_stress_balance_pid
Filipe Manana [Wed, 27 Mar 2024 17:11:36 +0000 (17:11 +0000)]
btrfs/028: use the helper _btrfs_kill_stress_balance_pid

Now that there's a helper to kill a background process that is running
_btrfs_stress_balance(), use it in btrfs/028. It's equivalent to the
existing code in btrfs/028.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
15 months agobtrfs: add helper to kill background process running _btrfs_stress_balance
Filipe Manana [Wed, 27 Mar 2024 17:11:35 +0000 (17:11 +0000)]
btrfs: add helper to kill background process running _btrfs_stress_balance

Killing a background process running _btrfs_stress_balance() is not as
simple as sending a signal to the process and waiting for it to die.
Therefore we have the following logic to terminate such process:

   kill $pid
   wait $pid
   # Wait for the balance operation to finish.
   while ps aux | grep "balance start" | grep -qv grep; do
       sleep 1
   done

Since this is repeated in several test cases, move this logic to a common
helper and use it in all affected test cases. This will help to avoid
repeating the same code again several times in upcoming changes.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
15 months agogeneric: test MADV_POPULATE_READ with IO errors
Darrick J. Wong [Wed, 27 Mar 2024 02:43:41 +0000 (19:43 -0700)]
generic: test MADV_POPULATE_READ with IO errors

This is a regression test for "mm/madvise: make
MADV_POPULATE_(READ|WRITE) handle VM_FAULT_RETRY properly".

Cc: David Hildenbrand <david@redhat.com>
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agoxfs/176: fix stupid failure
Darrick J. Wong [Wed, 27 Mar 2024 02:43:30 +0000 (19:43 -0700)]
xfs/176: fix stupid failure

Create the $SCRATCH_MNT/urk directory before we fill the filesystem so
that its creation won't fail and result in find spraying ENOENT errors
all over the golden output.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agoxfs/270: fix rocompat regex
Darrick J. Wong [Wed, 27 Mar 2024 02:43:23 +0000 (19:43 -0700)]
xfs/270: fix rocompat regex

This test fails with the fsverity patchset because the rocompat feature
bit for verity is 0x10.  The regular expression used to check if the
output is hexadecimal requires a single-digit answer, which is no longer
the case.

Fixes: 5bb78c56ef ("xfs/270: Fix ro mount failure when nrext64 option is enabled")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agogeneric/735: improve test by incorporating extra hints
Disha Goel [Tue, 19 Mar 2024 11:16:13 +0000 (16:46 +0530)]
generic/735: improve test by incorporating extra hints

On power systems with 64k block size (where default page size is 64k) we
encountered a kernel oops due to an integer overflow issue when writing
near the last logical block of a file. The allocator could allocate a
range where the end exceeds the maximum supported logical block
(UINT32_MAX), leading to a subsequent BUG_ON. This issue has been
addressed in the upstream kernel with commit 2dcf5fde6dff
("ext4: prevent the normalized size from exceeding EXT_MAX_BLOCKS").

==================================================
kernel BUG at fs/ext4/mballoc.c:4448!
Oops: Exception in kernel mode, sig: 5 [#1]
CPU: 8 PID: 2880554 Comm: xfs_io
NIP ext4_mb_use_inode_pa+0x110/0x160 [ext4]
LR ext4_mb_use_inode_pa+0xac/0x160 [ext4]
Call Trace:
ext4_mb_new_inode_pa+0x134/0x3a0 [ext4]
ext4_mb_try_best_found+0x158/0x280 [ext4]
ext4_mb_regular_allocator+0x16c/0x940 [ext4]
ext4_mb_new_blocks+0x610/0x960 [ext4]
ext4_ext_map_blocks+0x858/0xa90 [ext4]
ext4_map_blocks+0x218/0x800 [ext4]
ext4_iomap_alloc+0x10c/0x260 [ext4]
ext4_iomap_begin+0xfc/0x1f0 [ext4]
iomap_iter+0xf0/0x190
__iomap_dio_rw+0x208/0x690
iomap_dio_rw+0x20/0x80
ext4_dio_write_iter+0x210/0x4d0 [ext4]
vfs_write+0x364/0x4e0
sys_pwrite64+0xd4/0x120
system_call_exception+0x164/0x310
system_call_vectored_common+0xe8/0x278
==================================================

This test has been extended to provide a hint about the relevant fix
in case of failure.

Signed-off-by: Disha Goel <disgoel@linux.ibm.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agoext4/006: take into account updates to _scratch_fuzz_modify()
Luis Henriques (SUSE) [Fri, 15 Mar 2024 17:13:25 +0000 (17:13 +0000)]
ext4/006: take into account updates to _scratch_fuzz_modify()

Test ext4/006 takes into account the number of lines produced by its own
output.  However, changes introduced to function _scratch_fuzz_modify() by
commit 9bab148bb3c7 ("common/fuzzy: exercise the filesystem a little harder
after repairing"), modified the output.  Namely, the following three lines
were removed:

echo "+++ touch ${nr} files"
echo "+++ create files"
echo "+++ remove files"

And a new one was added:

echo "+++ stressing filesystem"

However, the usage of 'fsstress' also added an extra line with:

printf("seed = %ld\n", seed);

So the delta is one line (-3 + 2).

Modify test ext4/006 to take this change into account.

Signed-off-by: "Luis Henriques (SUSE)" <luis.henriques@linux.dev>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agocommon/fuzzy: make _scratch_fuzz_modify work for non-xfs filesystems
Luis Henriques (SUSE) [Fri, 15 Mar 2024 17:13:24 +0000 (17:13 +0000)]
common/fuzzy: make _scratch_fuzz_modify work for non-xfs filesystems

Since commit 9bab148bb3c7 ("common/fuzzy: exercise the filesystem a little
harder after repairing") funtion _scratch_fuzz_modify() has become
xfs-specific due to the use of some functions that assume this filesytem,
namely _xfs_force_bdev() and _xfs_has_feature().

Ensure _scratch_fuzz_modify() works again with other filesystems by using
these functions only when testing xfs.

Signed-off-by: "Luis Henriques (SUSE)" <luis.henriques@linux.dev>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agogeneric: add a regression test for fiemap into an mmap range
Josef Bacik [Wed, 20 Mar 2024 15:46:50 +0000 (11:46 -0400)]
generic: add a regression test for fiemap into an mmap range

Btrfs had a deadlock that you could trigger by mmap'ing a large file and
using that as the buffer for fiemap.  This test adds a c program to do
this, and the fstest creates a large enough file and then runs the
reproducer on the file.  Without the fix btrfs deadlocks, with the fix
we pass fine.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agovfs/idmapped_mounts.c: Change mount_setattr expected output
Taylor Jackson [Tue, 26 Mar 2024 20:33:52 +0000 (20:33 +0000)]
vfs/idmapped_mounts.c: Change mount_setattr expected output

In kernel commit dacfd001eaf2 (“fs/mnt_idmapping.c: Return -EINVAL
when no map is written”), the behavior of mount_setattr changed to
return EINVAL when attempting to create an idmapped mount when using
a user namespace with no mappings. The following commit updates the test
to expect no mount to be created in that case. And since no mount is created,
this commit also removes the check for overflow IDs because it does not make
sense to check for overflow IDs for a mount that was not created.

Signed-off-by: Taylor Jackson <tjackson9431@gmail.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Christian Brauner <brauner@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agovfs/idmapped_mounts.c: Incorrect array index for nested user ns
Taylor Jackson [Tue, 26 Mar 2024 20:33:51 +0000 (20:33 +0000)]
vfs/idmapped_mounts.c: Incorrect array index for nested user ns

Within the vfs test for idmapped mounts, the function nested_userns()
is using an incorrect array index when attempting to set up the mapping
for the 4th nested user ns within hierarchy[4]. The correct index that
belongs to the 4th nested user ns is actually hierarchy[3].
And hierarchy[4] is reserved for the dummy entry that marks the end
of the array.

Signed-off-by: Taylor Jackson <tjackson9431@gmail.com>
Reviewed-by: Christian Brauner <brauner@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agoxfs/558: scale blk IO size based on the filesystem blksz
Pankaj Raghav [Wed, 13 Mar 2024 20:38:18 +0000 (21:38 +0100)]
xfs/558: scale blk IO size based on the filesystem blksz

This test fails for 64k filesystem block size on a 4k PAGE_SIZE
system. Scale the `blksz` based on the filesystem block size instead of
fixing it as 64k so that we do get some iomap invalidations while doing
concurrent writes.

Cap the blksz to be at least 64k to retain the same behaviour as before
for smaller filesystem blocksizes.

This fixes the "Expected to hear about writeback iomap invalidations?"
message for 64k filesystems.

Signed-off-by: Pankaj Raghav <p.raghav@samsung.com>
Tested-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agocommon/rc: fix unknown _xfs_repair_test_fs function name
Zorro Lang [Sun, 24 Mar 2024 14:17:07 +0000 (22:17 +0800)]
common/rc: fix unknown _xfs_repair_test_fs function name

Sometimes I hit below errors:
  ./common/rc: line 1293: _xfs_repair_test_fs: command not found
  ./common/rc: line 1298: _xfs_repair_test_fs: command not found

The _repair_test_fs trys to call _xfs_repair_test_fs(), but there's
not that function in fstests. According to commit c7d81cdecbef,
it brought in _test_xfs_repair, but called wrong name. So fix it.

Fixes: c7d81cdecbef ("check: try to fix the test device if it gets corrupted")
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
15 months agofstests: btrfs/195: skip raid setups not in the profile configs
Josef Bacik [Tue, 19 Mar 2024 16:55:57 +0000 (12:55 -0400)]
fstests: btrfs/195: skip raid setups not in the profile configs

You can specify a custom BTRFS_PROFILE_CONFIGS to skip certain raid
configurations in the tests, however btrfs/195 doesn't honor this
currently.  Fix this up by getting the profile configs and skipping any
configurations that are not listed in BTRFS_PROFILE_CONFIGS.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
15 months agogeneric: test mount fails on physical device with configured dm volume
Anand Jain [Sat, 16 Mar 2024 17:02:34 +0000 (22:32 +0530)]
generic: test mount fails on physical device with configured dm volume

When a dm Flakey device is configured, (or similar dm where both physical
and dm devices are accessible) we have access to both the physical device
and the dm flakey device, ensure that the physical device mount fails.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
15 months agocommon/rc: use proper temporary file path in _repair_test_fs()
David Sterba [Tue, 19 Mar 2024 18:12:07 +0000 (19:12 +0100)]
common/rc: use proper temporary file path in _repair_test_fs()

The path /tmp.repair would be on the system root that could not be
writable, the temporary files are available at $tmp .

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
15 months agobtrfs/330: add test to validate ro/rw subvol mounting
Josef Bacik [Tue, 19 Mar 2024 18:12:05 +0000 (19:12 +0100)]
btrfs/330: add test to validate ro/rw subvol mounting

Btrfs has had the ability for almost a decade to allow ro and rw
mounting of subvols.  This behavior specifically

mount -o subvol=foo,ro /some/dir
mount -o subvol=bar,rw /some/other/dir

This seems simple, but because of the limitations of how we did mounting
in ye olde days we would mark the super block as RO and the mount if we
mounted RO first.  In the case above /some/dir would instantiate the
super block as read only and the mount point.  So the second mount
command under the covers would convert the super block to RW, and then
allow the mount to continue.

The results were still consistent, /some/dir was still read only because
the mount was marked read only, but /some/other/dir could be written to.

This is a test to make sure we maintain this behavior, as I almost
regressed this behavior while converting us to the new mount API.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
[ Fixed mkfs.btrfs redirect. Removed common/filter. ]

15 months agobtrfs/131,btrfs/172,btrfs/206: add check for block-group-tree feature in btrfs
Josef Bacik [Tue, 19 Mar 2024 18:12:03 +0000 (19:12 +0100)]
btrfs/131,btrfs/172,btrfs/206: add check for block-group-tree feature in btrfs

A new disk format option will make the no-holes option a requirement, so
add a helper to make sure that we aren't creating a fs with
BLOCK_GROUP_TREE by default, and skip the tests that require turning off
no-holes.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs/316: use rescan wrapper
Boris Burkov [Wed, 13 Mar 2024 23:46:30 +0000 (16:46 -0700)]
btrfs/316: use rescan wrapper

btrfs/316 is broken on the squota configuration because it uses a raw
rescan call which fails, instead of using the rescan wrapper. The test
passes with squota, so run it (instead of requiring rescan) though I
suspect it isn't the most meaningful test.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs/277: specify protocol version 3 for verity send
Boris Burkov [Wed, 13 Mar 2024 23:46:29 +0000 (16:46 -0700)]
btrfs/277: specify protocol version 3 for verity send

This test uses btrfs send with fs-verity which relies on protocol
version 3. The default in progs is version 2, so we need to explicitly
specify the protocol version. Note that the max protocol version in
progs is also currently broken (not properly gated by EXPERIMENTAL) so
that needs fixing as well.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
[ added _require_btrfs_send_version 3 ]

16 months agobtrfs/320: skip -O squota runs
Boris Burkov [Wed, 13 Mar 2024 19:51:17 +0000 (19:51 +0000)]
btrfs/320: skip -O squota runs

This test makes assumptions about the shared usage under snapshots which
are not valid when using squotas. Skip squotas for this test.

Also, make it use the rescan wrapper, just for uniformity and since it
doesn't hurt.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
[ added _require_qgroup_rescan ]

16 months agocommon/btrfs: introduce _require_btrfs_send_version
Anand Jain [Thu, 14 Mar 2024 06:37:40 +0000 (14:37 +0800)]
common/btrfs: introduce _require_btrfs_send_version

Rename _require_btrfs_send_v2() to _require_btrfs_send_version() and
check if the Btrfs kernel supports the v3 stream.

Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agofstests: add missing commit IDs to some tests
Filipe Manana [Wed, 13 Mar 2024 15:41:36 +0000 (15:41 +0000)]
fstests: add missing commit IDs to some tests

Some tests are still using a 'xxx...' commit ID but the respective patches
were already merged to Linus' tree or btrfs-progs, so update them with the
correct commit IDs and in two cases update the subject as well, because it
was modified after the test case was added and before being sent to Linus
(btrfs/317 and generic/707).

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agogeneric/574: don't fail the test on intentional coredump
Darrick J. Wong [Tue, 12 Mar 2024 14:57:20 +0000 (07:57 -0700)]
generic/574: don't fail the test on intentional coredump

Don't fail this test just because the mmap read of a corrupt verity file
causes xfs_io to segfault and then dump core.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Bill O'Donnell <bodonnel@redhat.com>
Reviewed-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agomisc: fix test that fail formatting with 64k blocksize
Darrick J. Wong [Tue, 27 Feb 2024 17:39:45 +0000 (18:39 +0100)]
misc: fix test that fail formatting with 64k blocksize

There's a bunch of tests that fail the formatting step when the test run
is configured to use XFS with a 64k blocksize.  This happens because XFS
doesn't really support that combination due to minimum log size
constraints. Fix the test to format larger devices in that case.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Co-developed-by: Pankaj Raghav <p.raghav@samsung.com>
Signed-off-by: Pankaj Raghav <p.raghav@samsung.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agocommon/rc: notrun if io_uring is disabled by sysctl
Zorro Lang [Mon, 11 Mar 2024 16:20:29 +0000 (00:20 +0800)]
common/rc: notrun if io_uring is disabled by sysctl

If kernel supports io_uring, userspace still can/might disable that
supporting by set /proc/sys/kernel/io_uring_disabled=2. Let's notrun
if io_uring is disabled by that way.

Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agofsstress: bypass io_uring testing if io_uring_queue_init returns EPERM
Zorro Lang [Mon, 11 Mar 2024 16:20:28 +0000 (00:20 +0800)]
fsstress: bypass io_uring testing if io_uring_queue_init returns EPERM

I found the io_uring testing still fails as:
  io_uring_queue_init failed
even if kernel supports io_uring feature.

That because of the /proc/sys/kernel/io_uring_disabled isn't 0.

Different value means:
  0 All processes can create io_uring instances as normal.
  1 io_uring creation is disabled (io_uring_setup() will fail with
    -EPERM) for unprivileged processes not in the io_uring_group
    group. Existing io_uring instances can still be used.  See the
    documentation for io_uring_group for more information.
  2 io_uring creation is disabled for all processes. io_uring_setup()
    always fails with -EPERM. Existing io_uring instances can still
    be used.

So besides the CONFIG_IO_URING kernel config, there's another switch
can on or off the io_uring supporting. And the "2" or "1" might be
the default on some systems.

On this situation the io_uring_queue_init returns -EPERM, so I change
the fsstress to ignore io_uring testing if io_uring_queue_init returns
-ENOSYS or -EPERM. And print different verbose message for debug.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Jeff Moyer <jmoyer@redhat.com>
16 months agofsstress: check io_uring_queue_init errno properly
Zorro Lang [Mon, 11 Mar 2024 16:20:27 +0000 (00:20 +0800)]
fsstress: check io_uring_queue_init errno properly

As the manual of io_uring_queue_init says "io_uring_queue_init(3)
returns 0 on success and -errno on failure". We should check if the
return value is -ENOSYS, not the errno.

Fixes: d15b1721f284 ("ltp/fsstress: don't fail on io_uring ENOSYS")
Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Jeff Moyer <jmoyer@redhat.com>
16 months agogeneric/392: stop checking st_blocks
Christoph Hellwig [Mon, 26 Feb 2024 10:03:19 +0000 (11:03 +0100)]
generic/392: stop checking st_blocks

st_blocks is a rather vaguely defined field.  To quote the Linux stat(2)
man page:

    Use of the st_blocks and st_blksize fields may be less portable.
    (They were introduced in BSD.  The interpretation differs between
    systems, and possibly on a single system when NFS mounts are
    involved.)

or the FreeBSD one:

    st_blocks   Actual number of blocks allocated for the file in
512-byte units.  As short symbolic links are stored in
the inode, this number may be zero.

and at least for XFS they include speculative preallocations and
in-flight COW fork allocations, and the numbers can change when the way
how data is stored is reorganized.  Because of that it doesn't make sense
to require st_blocks to not change after a crash even when fsync or
fdatasync was involved.

Remove the st_blocks checks and the now superfluous XFS always_cow
workaround.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agobtrfs/172,206: call _log_writes_cleanup in _cleanup
Su Yue [Thu, 7 Mar 2024 04:04:08 +0000 (12:04 +0800)]
btrfs/172,206: call _log_writes_cleanup in _cleanup

Because block group tree requires require no-holes feature,
_log_writes_mkfs "-O ^no-holes" fails when "-O block-group-tree" is
given in MKFS_OPTION.
Without explicit _log_writes_cleanup, the two tests fail with
logwrites-test device left. And all next tests will fail due to
SCRATCH DEVICE EBUSY.

Fix it by overriding _cleanup to call _log_writes_cleanup.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Su Yue <glass.su@suse.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agofstests: btrfs/121: allow snapshot with invalid qgroup to return error
Qu Wenruo [Sun, 3 Mar 2024 06:52:51 +0000 (17:22 +1030)]
fstests: btrfs/121: allow snapshot with invalid qgroup to return error

[BUG]
After incoming kernel commit "btrfs: qgroup: verify btrfs_qgroup_inherit
parameter", test case btrfs/121 would fail like this:

btrfs/121 1s ... [failed, exit status 1]- output mismatch (see /xfstests/results//btrfs/121.out.bad)
    --- tests/btrfs/121.out 2022-05-11 09:55:30.739999997 +0800
    +++ /xfstests/results//btrfs/121.out.bad 2024-03-03 13:33:38.076666665 +0800
    @@ -1,2 +1,3 @@
     QA output created by 121
    -Silence is golden
    +failed: '/usr/bin/btrfs subvolume snapshot -i 1/10 /mnt/scratch /mnt/scratch/snap1'
    +(see /xfstests/results//btrfs/121.full for details)
    ...
    (Run 'diff -u /xfstests/tests/btrfs/121.out /xfstests/results//btrfs/121.out.bad'  to see the entire diff)

[CAUSE]
The incoming kernel commit would do early qgroups validation before
subvolume/snapshot creation, and reject invalid qgroups immediately.

Meanwhile that test case itself still assume the ioctl would go on
without any error, thus the new behavior would break the test case.

[FIX]
Instead of relying on the snapshot creation ioctl return value, we just
completely ignore the output of that snapshot creation.
Then manually check if the fs is still read-write.

For different kernels (3 cases), they would lead to the following
results:

- Older unpatched kernel
  The filesystem would trigger a transaction abort (would be caught by
  dmesg filter), and also fail the "touch" command.

- Older but patched kernel
  The filesystem continues to create the snapshot, while still keeps the
  fs read-write.

- Latest kernel with qgroup validation
  The filesystem refuses to create the snapshot, while still keeps the
  fs read-write.

Both "older but patched" and "latest" kernels would still pass the test
case, even with different behaviors.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agoxfs: Fix no executable permission
Li Zhijian [Mon, 15 Jan 2024 00:59:13 +0000 (08:59 +0800)]
xfs: Fix no executable permission

Give them the executable permission, this also make the git status keep clean
after `./check`

Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agocommon: dm-error now supports zoned devices
Christoph Hellwig [Wed, 21 Feb 2024 06:37:41 +0000 (07:37 +0100)]
common: dm-error now supports zoned devices

Since kernel commit a951104333bd ("dm error: Add support for zoned block
devices") dm-error fully supports zoned devices.  Make use of that to
also run error injection tests for zoned device.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agoshared/298: run xfs_db against the loop device instead of the image file
Christoph Hellwig [Fri, 1 Mar 2024 15:28:20 +0000 (08:28 -0700)]
shared/298: run xfs_db against the loop device instead of the image file

xfs_db fails to properly detect the device sector size and thus segfaults
when run again an image file with 4k sector size.  While that's something
we should fix in xfs_db it will require a fair amount of refactoring of
the libxfs init code.  For now just change shared/298 to run xfs_db
against the loop device created on the image file that is used for I/O,
which feels like the right thing to do anyway to avoid cache coherency
issues.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agoshared/298: call fs commands on the loop device
Christoph Hellwig [Wed, 6 Mar 2024 01:22:46 +0000 (18:22 -0700)]
shared/298: call fs commands on the loop device

In general calling fs tools is best done on the block device used for
the file system and not the backing device of a loop file.  Thus switch
shared/298 to call all fs commands on the loop device.  Also add a
common on why the xfs_io fiemap command is called on the backing file,
and to have a good place for the comment stop passing the backing file
as the argument to get_holes function and just use it implicitly as
the other helpers to with the loop device.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agoxfstest: add detection for ext4.h presence in configure.ac
Disha Goel [Fri, 1 Mar 2024 08:15:40 +0000 (13:45 +0530)]
xfstest: add detection for ext4.h presence in configure.ac

In some distributions, __u64 is already defined in system header files,
causing compilation errors when building xfstest.

        # make
            [CC]    ext4_resize
        ext4_resize.c:17:28: error: conflicting types for '__u64'
         typedef unsigned long long __u64;
                                    ^~~~~
        In file included from /usr/include/asm/types.h:26:0,
                         from /usr/include/linux/types.h:5,
                         from /usr/include/linux/mount.h:4,
                         from /usr/include/sys/mount.h:32,
                         from ext4_resize.c:15:
        /usr/include/asm-generic/int-l64.h:30:23: note: previous declaration of '__u64' was here
         typedef unsigned long __u64;
                       ^~~~~

To address this issue, configure.ac now checks for the presence and
compilability of <linux/ext4.h>. If found and compilable, the macro
HAVE_LINUX_EXT4_H is defined. The commit also updates src/ext4_resize.c
to conditionally include <linux/ext4.h> based on the presence of the
header, ensuring compatibility with systems where ext4.h is either
present or not. Also include <linux/types.h> which gets __u64
definition on systems where ext4.h is not present. This change
enhances the configure process and improves code consistency.

The changes were tested on various distributions on Power
architecture, by successfully compiling xfstest. Additionally,
verified the compatibility by running ext4/033 and ext4/056
tests, both of which use ext4_resize and observed successful
test execution.

        # make
checking linux/ext4.h usability... yes
        checking linux/ext4.h presence... yes
        checking for linux/ext4.h... yes
            [CC]    detached_mounts_propagation
            [CC]    ext4_resize
            [CC]    t_readdir_3

        # make
        checking linux/ext4.h usability... no
        checking linux/ext4.h presence... no
        checking for linux/ext4.h... no
            [CC]    detached_mounts_propagation
            [CC]    ext4_resize
            [CC]    t_snapshot_deleted_subvolume

Signed-off-by: Disha Goel <disgoel@linux.ibm.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agoxfs: test for premature ENOSPC with large cow delalloc extents
Darrick J. Wong [Thu, 7 Mar 2024 23:22:55 +0000 (15:22 -0800)]
xfs: test for premature ENOSPC with large cow delalloc extents

On a higly fragmented filesystem a Direct IO write can fail with -ENOSPC error
even though the filesystem has sufficient number of free blocks.

This occurs if the file offset range on which the write operation is being
performed has a delalloc extent in the cow fork and this delalloc extent
begins much before the Direct IO range.

In such a scenario, xfs_reflink_allocate_cow() invokes xfs_bmapi_write() to
allocate the blocks mapped by the delalloc extent. The extent thus allocated
may not cover the beginning of file offset range on which the Direct IO write
was issued. Hence xfs_reflink_allocate_cow() ends up returning -ENOSPC.

This test addresses this issue.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agoxfs/43[4-6]: make module reloading optional
Darrick J. Wong [Fri, 1 Mar 2024 17:51:24 +0000 (09:51 -0800)]
xfs/43[4-6]: make module reloading optional

These three tests examine two things -- first, can xfs CoW staging
extent recovery handle corruptions in the refcount btree gracefully; and
second, can we avoid leaking incore inodes and dquots.

The only cheap way to check the second condition is to rmmod and
modprobe the XFS module, which triggers leak detection when rmmod tears
down the caches.  Currently, the entire test is _notrun if module
reloading doesn't work.

Unfortunately, these tests never run for the majority of XFS developers
because their testbeds either compile the xfs kernel driver into vmlinux
statically or the rootfs is xfs so the module cannot be reloaded.  The
author's testbed boots from NFS and does not have this limitation.

Because we've had repeated instances of CoW recovery regressions not
being caught by testing until for-next hits my machine, let's make the
module reloading optional in all three tests to improve coverage.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agoxfs/122: update test to pick up rtword/suminfo ondisk unions
Darrick J. Wong [Tue, 27 Feb 2024 02:02:05 +0000 (18:02 -0800)]
xfs/122: update test to pick up rtword/suminfo ondisk unions

Update this test to check that the ondisk unions for rt bitmap word and
rt summary counts are always the correct size.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agoxfs/599: reduce the amount of attrs created here
Darrick J. Wong [Tue, 27 Feb 2024 02:01:50 +0000 (18:01 -0800)]
xfs/599: reduce the amount of attrs created here

Luis Chamberlain reported insane runtimes in this test:

"xfs/599 takes a long time on LBS, but it passes. The amount of time it
takes, however, begs the question if the test is could be trimmed to do
less work because the larger the block size the larger the number of
dirents and xattrs are used to create. The large dirents are not a
problem. The amount of time it takes to create xattrs with hashcol
however grows exponentially in time.

"n=16k   takes 5   seconds
"n=32k   takes 30  seconds
"n=64k     takes 6-7 minutes
"n=1048576 takes 30 hours

"n=1048576 is what we use for block size 32k.

"Do we really need so many xattrs for larger block sizes for this test?"

No, we don't.  The goal of this test is to create a two-level dabtree of
xattrs having identical hashes.  However, the test author (me)
apparently forgot that if a dabtree is created in the attr fork, there
will be a dabtree entry for each extended attribute, not each attr leaf
block.  Hence it's a waste of time to multiply da_records_per_block by
attr_records_per_block.

Reported-by: Luis Chamberlain <mcgrof@kernel.org>
Fixes: 1cd6b61299 ("xfs: add a couple more tests for ascii-ci problems")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agogeneric/491: increase test timeout
Darrick J. Wong [Tue, 27 Feb 2024 02:01:34 +0000 (18:01 -0800)]
generic/491: increase test timeout

Bump the read timeout in this test to a few seconds just in case it
actually takes the IO system more than a second to retrieve the data
(e.g. cloud storage network lag).

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agogeneric/192: fix spurious timeout
Darrick J. Wong [Tue, 27 Feb 2024 02:01:19 +0000 (18:01 -0800)]
generic/192: fix spurious timeout

I have a theory that when the nfs server that hosts the root fs for my
testing VMs gets backed up, it can take a while for path resolution and
loading of echo, cat, or tee to finish.  That delays the test enough to
result in:

  --- /tmp/fstests/tests/generic/192.out 2023-11-29 15:40:52.715517458 -0800
  +++ /var/tmp/fstests/generic/192.out.bad 2023-12-15 21:28:02.860000000 -0800
  @@ -1,5 +1,6 @@
   QA output created by 192
   sleep for 5 seconds
   test
  -delta1 is in range
  +delta1 has value of 12
  +delta1 is NOT in range 5 .. 7
   delta2 is in range

Therefore, invoke all these utilities with --help before the critical
section to make sure they're all in memory.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agoxfs/155: fail the test if xfs_repair hangs for too long
Darrick J. Wong [Tue, 27 Feb 2024 04:41:00 +0000 (20:41 -0800)]
xfs/155: fail the test if xfs_repair hangs for too long

There are a few hard to reproduce bugs in xfs_repair where it can
deadlock trying to lock a buffer that it already owns.  These stalls
cause fstests never to finish, which is annoying!  To fix this, set up
the xfs_repair run to abort after 10 minutes, which will affect the
golden output and capture a core file.

This doesn't fix xfs_repair, obviously.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agogeneric/604: try to make race occur reliably
Darrick J. Wong [Tue, 27 Feb 2024 04:40:21 +0000 (20:40 -0800)]
generic/604: try to make race occur reliably

This test will occasionaly fail like so:

  --- /tmp/fstests/tests/generic/604.out 2024-02-03 12:08:52.349924277 -0800
  +++ /var/tmp/fstests/generic/604.out.bad 2024-02-05 04:35:55.020000000 -0800
  @@ -1,2 +1,5 @@
   QA output created by 604
  -Silence is golden
  +mount: /opt: /dev/sda4 already mounted on /opt.
  +       dmesg(1) may have more information after failed mount system call.
  +mount -o usrquota,grpquota,prjquota, /dev/sda4 /opt failed
  +(see /var/tmp/fstests/generic/604.full for details)

As far as I can tell, the cause of this seems to be _scratch_mount
getting forked and exec'd before the backgrounded umount process has a
chance to enter the kernel.  When this occurs, the mount() system call
will return -EBUSY because this isn't an attempt to make a bind mount.
Slow things down slightly by stalling the mount by 10ms.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agobtrfs: test normal qgroup operations in a compress friendly way
Josef Bacik [Tue, 5 Mar 2024 18:52:24 +0000 (19:52 +0100)]
btrfs: test normal qgroup operations in a compress friendly way

btrfs/022 currently fails if you are testing with -o compress because it
does a limit exceed test which will pass with compression on.

However the other functionality this test tests is completely acceptable
with compression enabled.  Handle this by breaking the test into two
tests, one that simply tests the qgroup exceed limits test that requires
no compression, and the rest of the tests that do not have the no
compression restriction.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs/291: remove image file after teardown
Josef Bacik [Tue, 5 Mar 2024 18:52:22 +0000 (19:52 +0100)]
btrfs/291: remove image file after teardown

LVM doesn't like it when you remove the file out from underneath the
backing device.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs/287,btrfs/293: filter all btrfs subvolume delete calls
Josef Bacik [Tue, 5 Mar 2024 18:52:19 +0000 (19:52 +0100)]
btrfs/287,btrfs/293: filter all btrfs subvolume delete calls

Some of our btrfs subvolume delete calls get put into the golden output,
and many of them simply _filter_scratch.  This works fine, but we
recently changed btrfs subvolume delete output, and it would have been
nice to simply filter this in one place.  We have a
_filter_btrfs_subvol_delete helper, but it's only used in one place.
Fix all of these uses to call _filter_btrfs_subvol_delete, this will
allow for follow up fixes against _filter_btrfs_subvol_delete itself to
deal with changed output.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs/271: adjust failure condition
Josef Bacik [Tue, 5 Mar 2024 18:52:17 +0000 (19:52 +0100)]
btrfs/271: adjust failure condition

btrfs/271 was failing with the subpage blocksize VM's.  This is because
there's an assumption made that the device error counters are
per-sector, but they're per-io.  With a 16kib pagesize and a 4k
sectorsize/nodesize the threshold was expecting 16 failed IO's, but
instead we were getting 5.

This other gotcha here is that with the tree log we will write the log
tree first, and then update the log root tree with the location of the
log tree root node.  With pagesize == nodesize this is fine, we will
only write the log tree root node.  However with subpage blocksize both
of these nodes could be on the same page, and thus they are both written
out during that initial write.  When we update the pointer for the log
root tree we will COW the log root tree root node and submit another IO,
resulting in 3 metadata IO's instead of 2.

Fix the failure case to be < 4 blocks, which is the minimum number of
IO's we should be seeing.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs/213: make the test more reliable
Josef Bacik [Tue, 5 Mar 2024 18:52:14 +0000 (19:52 +0100)]
btrfs/213: make the test more reliable

This test will write for 8 seconds and then try to balance, but for some
setups 8 seconds may be enough to fill the disk.  Instead figure out
what half the size of the disk is and write at most that many bytes, or
for 8 seconds, whichever comes first.  Then use the amount of time it
took to do the write to determine how long we should allow the balance
to continue before we attempt to cancel it.

Additionally the macro is '_notrun' not '_not_run'.  With this change
this test now does the correct thing on my ARM CI VM.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs/131: don't run with subpage blocksizes
Josef Bacik [Tue, 5 Mar 2024 18:52:12 +0000 (19:52 +0100)]
btrfs/131: don't run with subpage blocksizes

This test requires a feature that is incompatible with subpage
blocksizes.  Check to see if that's what we're testing and simply skip
this test.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs/012: adjust how we populate the fs to convert
Josef Bacik [Tue, 5 Mar 2024 18:52:06 +0000 (19:52 +0100)]
btrfs/012: adjust how we populate the fs to convert

/lib/modules/$(uname -r)/ can have symlinks to the source tree where the
kernel was built from, which can have all sorts of stuff, which will
make the runtime for this test exceedingly long.  We're just trying to
copy some data into our tree to test with, we don't need the entire
devel tree of whatever we're doing.  Additionally VM's that aren't built
with modules will fail this test.

Update the test to use /etc, which will always exist.  Additionally use
timeout just in case there's large files or some other shenanigans so
the test doesn't run forever copying large amounts of files.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
[ fix use $TIMEOUT_PROG ]

16 months agobtrfs/011: increase the runtime for replace cancel
Josef Bacik [Tue, 5 Mar 2024 18:52:03 +0000 (19:52 +0100)]
btrfs/011: increase the runtime for replace cancel

This test exercises the btrfs replace cancel path, but in order to do
this we have to have enough work to do in order to successfully cancel
the balance, otherwise the test fails because the operation has
completed before we're able to cancel.  This test has a very low pass
rate because we do not generate a large enough file system for replace
to have enough work, passing around 5% of the time.  Increase the time
spent to 10x the time we wait for the replace to start its work before
we cancel, this allows us to consistently pass this test.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agocommon/rc: specify required device size
Anand Jain [Thu, 7 Mar 2024 11:41:19 +0000 (19:41 +0800)]
common/rc: specify required device size

The current _notrun call states that the scratch device is too small but
does not specify the required size. Simply update the _notrun messages.

Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs: fix grep warning at _require_btrfs_mkfs_uuid_option()
Filipe Manana [Wed, 6 Mar 2024 15:01:57 +0000 (15:01 +0000)]
btrfs: fix grep warning at _require_btrfs_mkfs_uuid_option()

When running _require_btrfs_mkfs_uuid_option(), some grep versions
complain about escaping the dash characters and make the tests that
use this function fail like this:

  btrfs/313       2s - output mismatch (see /root/fstests/results//btrfs_normal/btrfs/313.out.bad)
      --- tests/btrfs/313.out 2024-03-05 18:48:34.929372495 +0000
      +++ /root/fstests/results//btrfs_normal/btrfs/313.out.bad 2024-03-05 20:52:27.745166101 +0000
      @@ -1,5 +1,8 @@
       QA output created by 313
       ---- clone_uuids_verify_tempfsid ----
      +grep: warning: stray \ before -
      +grep: warning: stray \ before -
      +grep: warning: stray \ before -
       Mounting original device
       On disk fsid: FSID
      ...
      (Run 'diff -u /root/fstests/tests/btrfs/313.out /root/fstests/results//btrfs_normal/btrfs/313.out.bad'  to see the entire diff)
  btrfs/314       3s - output mismatch (see /root/fstests/results//btrfs_normal/btrfs/314.out.bad)
      --- tests/btrfs/314.out 2024-03-05 18:48:34.929372495 +0000
      +++ /root/fstests/results//btrfs_normal/btrfs/314.out.bad 2024-03-05 20:52:32.880237216 +0000
      @@ -1,6 +1,9 @@
       QA output created by 314

       From non-tempfsid SCRATCH_MNT to tempfsid TEST_DIR/314/tempfsid_mnt
      +grep: warning: stray \ before -
      +grep: warning: stray \ before -
      +grep: warning: stray \ before -
       wrote 9000/9000 bytes at offset 0
      ...

So fix this by not escaping anymore the dashes and using the -- separator
before the regex pattern parameter.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs/016: fix a false alert due to xattrs mismatch
Qu Wenruo [Mon, 19 Feb 2024 08:00:07 +0000 (18:30 +1030)]
btrfs/016: fix a false alert due to xattrs mismatch

[BUG]
When running btrfs/016 after any other test case, it would fail on a
SELinux enabled environment:

  btrfs/015 1s ...  0s
  btrfs/016 1s ... [failed, exit status 1]- output mismatch (see ~/xfstests-dev/results//btrfs/016.out.bad)
      --- tests/btrfs/016.out 2023-12-28 10:39:36.481027970 +1030
      +++ ~/xfstests-dev/results//btrfs/016.out.bad 2023-12-28 15:53:10.745436664 +1030
      @@ -1,2 +1,3 @@
       QA output created by 016
      -Silence is golden
      +fssum failed
      +(see ~/xfstests-dev/results//btrfs/016.full for details)
      ...
      (Run 'diff -u ~/xfstests-dev/tests/btrfs/016.out ~/xfstests-dev/results//btrfs/016.out.bad'  to see the entire diff)
  Ran: btrfs/015 btrfs/016
  Failures: btrfs/016
  Failed 1 of 2 tests

[CAUSE]
The test case itself would try to use a blank SELinux context for the
SCRATCH_MNT, to control the xattrs.

But the initial send stream is generated from $TEST_DIR, which may still
have the default SELinux mount context.

And such mismatch in the SELinux xattr (source on $TEST_DIR still has
the extra xattr, meanwhile the receve end on $SCRATCH_MNT doesn't) would
lead to above mismatch.

[FIX]
Fix the false alerts by disable XATTR checks.

Furthermore instead of doing all the edge juggling using $TEST_DIR, this
time we do all the work on $SCRATCH_MNT.

This means we would generate the initial send stream from $SCRATCH_MNT,
then reformat the fs, mount scratch again, receive and verify.

We no longer needs to cleanup the extra file for the initial send
stream, as they are on the scratch device and would be formatted anyway.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agocommon/config: fix CANON_DEVS=yes when file does not exist
Luis Chamberlain [Wed, 14 Feb 2024 17:42:08 +0000 (09:42 -0800)]
common/config: fix CANON_DEVS=yes when file does not exist

CANON_DEVS=yes allows you to use symlinks for devices, so fstests
resolves them back to the real backing device. The iteration for
resolving the backing device works obviously if you have the file
present, but if one was not present there is a parsing error. Fix
this parsing error introduced by a0c36009103b8 ("fstests: add helper
to canonicalize devices used to enable persistent disks").

Fixes: a0c36009103b8 ("fstests: add helper to canonicalize devices used to enable persistent disks"
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agogeneric/736: fix a buffer overflow in readdir-while-renames.c
Filipe Manana [Sun, 11 Feb 2024 11:28:26 +0000 (11:28 +0000)]
generic/736: fix a buffer overflow in readdir-while-renames.c

The test is using a 32 characters buffer to print the full path for each
file name, which in some setups it's not enough because $TEST_DIR can
point to a path name longer than that, or even smaller but then the buffer
is still not large enough after appending a file name. When that's the
case it results in a core dump like this:

  generic/736       QA output created by 736
  *** buffer overflow detected ***: terminated
  /opt/xfstests/tests/generic/736: line 32:  9217 Aborted                 (core dumped) $here/src/readdir-while-renames $target_dir
  Silence is golden
  - output mismatch (see /opt/xfstests/results//generic/736.out.bad)
      --- tests/generic/736.out 2024-01-14 12:01:35.000000000 -0500
      +++ /opt/xfstests/results//generic/736.out.bad 2024-01-23 18:58:37.990000000 -0500
      @@ -1,2 +1,4 @@
       QA output created by 736
      +*** buffer overflow detected ***: terminated
      +/opt/xfstests/tests/generic/736: line 32:  9217 Aborted                 (core dumped) $here/src/readdir-while-renames $target_dir
       Silence is golden
      ...
      (Run diff -u /opt/xfstests/tests/generic/736.out /opt/xfstests/results//generic/736.out.bad  to see the entire diff)
  Ran: generic/736
  Failures: generic/736
  Failed 1 of 1 tests

We don't actually need to print the full path into the buffer, because we
have previously set the current directory (chdir) to the path pointed by
"dir_path". So fix this by printing only the relative path name which
uses at most 5 characters (NUM_FILES is 5000 plus the nul terminator).

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agotests/*/Makefile: make sure group.list DIRT exists before install
Martin Jansa [Thu, 8 Feb 2024 22:52:41 +0000 (23:52 +0100)]
tests/*/Makefile: make sure group.list DIRT exists before install

* sometimes make install was failing with:
  cp: cannot stat 'group.list': No such file or directory
  and bunch of non-fatal messages:
  mv: failed to preserve ownership for 'group.list': Invalid argument

* this was when tools/mkgroupfile did
  mv -f "$new_groups" "$groupfile"
  overwritting the group.list file while install-sh was already
  copying it to output

* in the end easily reproducible by
  1) removing tests/*/group.list before each make install
  2) adding some sleep in mkgroupfile before the mv call

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agobtrfs: require no nodatacow for tests that exercise read repair
Filipe Manana [Thu, 1 Feb 2024 18:03:50 +0000 (18:03 +0000)]
btrfs: require no nodatacow for tests that exercise read repair

Several test cases that exercise the ability to detect corrupted data and
repair it, fail when "-o nodatacow" is passed to MOUNT_OPTIONS, because
that ability requires the existence of data checksums, and those are
disabled in nodatacow mode. So skip the tests when "-o nodatacow" is
present.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agobtrfs/299: skip test if we were mounted with nodatacow
Filipe Manana [Thu, 1 Feb 2024 18:03:49 +0000 (18:03 +0000)]
btrfs/299: skip test if we were mounted with nodatacow

The test requires the ability to create an inline extent in a file with
a prealloced extent, created with fallocate's FALLOC_FL_KEEP_SIZE mode,
which can only happen when COW is enabled. If the test is run with
MOUNT_OPTIONS="-o nodatacow", then COW never happens as all writes end
up using the preallocated extent. This results in the logical-resolve
command to return one file path when it should return none, since the
base logical address of the prealloc extent is still in use unless COW
happens.

So make the test not run if nodatacow is specified in MOUNT_OPTIONS.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agobtrfs/173: make the test work when mounting with nodatacow
Filipe Manana [Thu, 1 Feb 2024 18:03:48 +0000 (18:03 +0000)]
btrfs/173: make the test work when mounting with nodatacow

Currently btrfs/173 fails when passing "-o nodatacow" to MOUNT_OPTIONS
because it assumes that when creating a file it does not have the
nodatacow flag set, which is obviously not true if the fs is mounted with
"-o nodatacow". To allow the test to run successfully with nodatacow,
just make sure it clears the nodatacow flag from the file if the fs was
mounted with "-o nodatacow".

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agobtrfs: require no nodatacow for tests that exercise compression
Filipe Manana [Thu, 1 Feb 2024 18:03:47 +0000 (18:03 +0000)]
btrfs: require no nodatacow for tests that exercise compression

Several test cases fail when running with MOUNT_OPTIONS="-o nodatacow"
because they attempt to use compression and compression can not be
enabled on nodatacow files (it fails with -EINVAL). So make sure those
tests are not run if nodatacow is specified in MOUNT_OPTIONS.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agogeneric/682: update and fix-up golden output
Anthony Iliopoulos [Thu, 1 Feb 2024 16:17:32 +0000 (17:17 +0100)]
generic/682: update and fix-up golden output

coreutils v9.4 introduced a change in the error output of mv under
certain errno values via commit 3cb862ce5f10 ("mv: better diagnostic for
'mv dir x' failure"), which broke the golden output.

Update golden output to match the change, and further add an output
filter to avoid having the test fail on environments that ran with an
older coreutils release, taken from commit d9323ad7a05e ("generic/245:
Filter mv error message").

Signed-off-by: Anthony Iliopoulos <ailiop@suse.com>
Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agogeneric/68[12]: use the dir blocksize for xfs filesystems
Anthony Iliopoulos [Thu, 1 Feb 2024 16:17:31 +0000 (17:17 +0100)]
generic/68[12]: use the dir blocksize for xfs filesystems

The tests are using the filesystem block size for calculating the number
of dirents required to fill a 2-block directory. For v4 xfs filesystems
formatted with fs blocksize of 512 bytes this is failing, as the tests
do not take into account that the directory block size is not always
equal to the filesystem block size. As such, the tests never go over
quota, and even if they did there is no hard block limit being set (due
to 512 / 1024 = 0 calculation in setquota).

Use the directory blocksize instead of the filesystem blocksize, when
the fstype under test is xfs.

Signed-off-by: Anthony Iliopoulos <ailiop@suse.com>
Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
16 months agobtrfs: check conversion of zoned fileystems
Johannes Thumshirn [Tue, 27 Feb 2024 12:25:31 +0000 (13:25 +0100)]
btrfs: check conversion of zoned fileystems

Recently we had a bug where a zoned filesystem could be converted to a
higher data redundancy profile than supported.

Add a test-case to check the conversion on zoned filesystems.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agofilter.btrfs: add filter for btrfs device add
Johannes Thumshirn [Tue, 27 Feb 2024 12:25:30 +0000 (13:25 +0100)]
filter.btrfs: add filter for btrfs device add

Add a filter for the output of btrfs device add.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agofilter.brtfs: add filter for conversion
Johannes Thumshirn [Tue, 27 Feb 2024 12:25:29 +0000 (13:25 +0100)]
filter.brtfs: add filter for conversion

Add a filter for the output of btrfs-balance with a convert argument.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs: detect regular qgroup for older kernels correctly
Qu Wenruo [Tue, 20 Feb 2024 04:01:34 +0000 (14:31 +1030)]
btrfs: detect regular qgroup for older kernels correctly

[BUG]
When running an older (vendoer v6.4) kernel, some qgroup test cases
would be skipped:

  btrfs/017 1s ... [not run] not running normal qgroups

[CAUSE]
With the introduce of simple quota mode, there is a new sysfs interface,
/sys/fs/btrfs/<uuid>/qgroups/mode to indicate the currently running
qgroup modes.

And _qgroup_mode() from `common/btrfs` is using that new interface to
detect the mode.

Unfortuantely for older kernels without simple quota support,
_qgroup_mode() would return "disabled" directly, causing those test case
to be skipped.

[FIX]
Fallback to regular qgroup if that sysfs interface is not accessible, as
qgroup is introduced from the very beginning of btrfs, thus the regular
qgroup is always supported.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs: btrfs/224 do not assign snapshot to a subvolume qgroup
Qu Wenruo [Mon, 26 Feb 2024 04:02:34 +0000 (14:32 +1030)]
btrfs: btrfs/224 do not assign snapshot to a subvolume qgroup

For "btrfs subvolume snapshot -i <qgroupid>", we only expect the target
qgroup to be a higher level one.

Assigning a 0 level qgroup to another 0 level qgroup is only going to
cause confusion, and I'm planning to do extra sanity checks both in
kernel and btrfs-progs to reject such behavior.

So change the test case to do regular higher level qgroup assignment
only.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs: validate inconsitent qgroup won't leak reserved data space
Qu Wenruo [Fri, 23 Feb 2024 09:35:47 +0000 (20:05 +1030)]
btrfs: validate inconsitent qgroup won't leak reserved data space

There is a kernel regression caused by commit e15e9f43c7ca ("btrfs:
introduce BTRFS_QGROUP_RUNTIME_FLAG_NO_ACCOUNTING to skip qgroup
accounting"), where if qgroup is inconsistent (not that hard to trigger)
btrfs would leak its qgroup data reserved space, and cause a warning at
unmount time.

The test case would verify the behavior by:

- Enable qgroup first

- Intentionally mark qgroup inconsistent
  This is done by taking a snapshot and assign it to a higher level
  qgroup, meanwhile the source has no higher level qgroup.

- Trigger a large enough write to cause qgroup data space leak

- Unmount and check the dmesg for the qgroup rsv leak warning

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs: test tempfsid with device add, seed, and balance
Anand Jain [Mon, 19 Feb 2024 19:48:50 +0000 (03:48 +0800)]
btrfs: test tempfsid with device add, seed, and balance

Make sure that basic functions such as seeding and device add fail,
while balance runs successfully with tempfsid.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs: validate send-receive operation with tempfsid.
Anand Jain [Mon, 19 Feb 2024 19:48:49 +0000 (03:48 +0800)]
btrfs: validate send-receive operation with tempfsid.

Given concurrent mounting of both the original and its clone device on
the same system, this test confirms the integrity of send and receive
operations in the presence of active tempfsid.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs: verify tempfsid clones using mkfs
Anand Jain [Mon, 19 Feb 2024 19:48:48 +0000 (03:48 +0800)]
btrfs: verify tempfsid clones using mkfs

Create appearing to be a clone using the mkfs.btrfs option and test if
the tempfsid is active.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs: introduce helper for creating cloned devices with mkfs
Anand Jain [Mon, 19 Feb 2024 19:48:47 +0000 (03:48 +0800)]
btrfs: introduce helper for creating cloned devices with mkfs

Use newer mkfs.btrfs option to generate two cloned devices,
used in test cases.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs: test case prerequisite _require_btrfs_mkfs_uuid_option
Anand Jain [Mon, 19 Feb 2024 19:48:46 +0000 (03:48 +0800)]
btrfs: test case prerequisite _require_btrfs_mkfs_uuid_option

For easier and more effective testing of btrfs tempfsid, newer versions
of mkfs.btrfs contain options such as --device-uuid. Check if the
currently running mkfs.btrfs contains this option.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs: check if cloned device mounts with tempfsid
Anand Jain [Mon, 19 Feb 2024 19:48:45 +0000 (03:48 +0800)]
btrfs: check if cloned device mounts with tempfsid

If another device with the same fsid and uuid would mount then verify if
it mounts with a temporary fsid.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
16 months agobtrfs: verify that subvolume mounts are unaffected by tempfsid
Anand Jain [Mon, 19 Feb 2024 19:48:44 +0000 (03:48 +0800)]
btrfs: verify that subvolume mounts are unaffected by tempfsid

The tempfsid logic must determine whether the incoming mount request
is for a device already mounted or a new device mount. Verify that it
recognizes the device already mounted well by creating reflink across
the subvolume mount points.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>