]> www.infradead.org Git - users/hch/xfstests-dev.git/log
users/hch/xfstests-dev.git
9 months agofsx: support unshare range fallocate mode
Brian Foster [Thu, 26 Sep 2024 14:41:46 +0000 (10:41 -0400)]
fsx: support unshare range fallocate mode

The fallocate unshare mode flag modifies traditional preallocate
mode to unshare any shared extents backing the target range. Without
the unshare flag, preallocate mode simply assures that blocks are
physically allocated, regardless of whether they might be shared.
Unshare mode behaves the same as preallocate mode outside of the
shared extent case.

Since unshare is fundamentally a modifier to preallocate mode,
enable it via an operation flag. Similar to keep size mode, select
it randomly for fallocate operations and track it via a flag and
string combination for operation logging and replay.

Unshare is mainly used for filesystems that support reflink, but the
operation is equivalent to preallocate mode for non-shared ranges,
so enable it by default. Filesystems that do not support the
fallocate flag (such as those that might not support reflink) will
fail the test operation and disable unshare calls at runtime. Also
provide a new command line option to explicitly disable unshare
calls.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agobtrfs/321: make the test work when compression is enabled
Filipe Manana [Thu, 12 Sep 2024 12:45:00 +0000 (13:45 +0100)]
btrfs/321: make the test work when compression is enabled

When running btrfs/321 with compression enabled it fails like this:

  $ MOUNT_OPTIONS="-o compress" ./check btrfs/321
  FSTYP         -- btrfs
  PLATFORM      -- Linux/x86_64 debian0 6.11.0-rc7-btrfs-next-174+ #1 SMP PREEMPT_DYNAMIC Tue Sep 10 17:11:38 WEST 2024
  MKFS_OPTIONS  -- /dev/sdc
  MOUNT_OPTIONS -- -o compress /dev/sdc /home/fdmanana/btrfs-tests/scratch_1

  btrfs/321 2s ... [failed, exit status 1]- output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/321.out.bad)
      --- tests/btrfs/321.out 2024-09-12 12:12:11.259272125 +0100
      +++ /home/fdmanana/git/hub/xfstests/results//btrfs/321.out.bad 2024-09-12 13:18:40.231120012 +0100
      @@ -1,2 +1,5 @@
       QA output created by 321
      -Silence is golden
      +mount: /home/fdmanana/btrfs-tests/scratch_1: can't read superblock on /dev/sdc.
      +       dmesg(1) may have more information after failed mount system call.
      +mount -o compress -o ro /dev/sdc /home/fdmanana/btrfs-tests/scratch_1 failed
      +(see /home/fdmanana/git/hub/xfstests/results//btrfs/321.full for details)
      ...
      (Run 'diff -u /home/fdmanana/git/hub/xfstests/tests/btrfs/321.out /home/fdmanana/git/hub/xfstests/results//btrfs/321.out.bad'  to see the entire diff)

  HINT: You _MAY_ be missing kernel fix:
        10d9d8c3512f btrfs: fix a use-after-free bug when hitting errors inside btrfs_submit_chunk()

  Ran: btrfs/321
  Failures: btrfs/321
  Failed 1 of 1 tests

This is because with compression enabled we get a csum tree that has only
one leaf, and that leaf is the root of the csum tree. That means that
after the test corrupts the leaf, the next mount will fail since an error
loading the root is critical and makes the mount operation fail.

Fix this by creating a file with 128M of data instead of 32M, as this
guarantees that even if compression is enabled, and even with the maximum
allowed leaf size (64K), we still get a csum tree with multiple leaves,
making the test work.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agogeneric: add a regression test for sub-block fsmap queries
Darrick J. Wong [Wed, 18 Sep 2024 20:57:19 +0000 (13:57 -0700)]
generic: add a regression test for sub-block fsmap queries

Zizhi Wo found some bugs in the GETFSMAP implementation if it is fed
sub-fsblock ranges.  Add a regression test for this.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agofstests: fix min_dio_alignment logic for getting device block size
Filipe Manana [Mon, 16 Sep 2024 12:03:12 +0000 (13:03 +0100)]
fstests: fix min_dio_alignment logic for getting device block size

If we failed to get the dio alignment from statx we try to get the
device's block size using the BLKSSZGET ioctl, however we failed to
return it because we don't check if the ioctl succeeded (returned 0).
Furthermore in case the ioctl returned an error, we end up returning an
undefined value since the 'logical_block_size' variable ends up not
being initialized.

This was causing some tests to be skipped on btrfs after commit
ee799a0cf1d4 ("replace _min_dio_alignment with calls to
src/min_dio_alignment"), like generic/240 for example:

  $ ./check generic/240
  FSTYP         -- btrfs
  PLATFORM      -- Linux/x86_64 debian0 6.11.0-rc7-btrfs-next-174+ #1 SMP PREEMPT_DYNAMIC Tue Sep 10 17:11:38 WEST 2024
  MKFS_OPTIONS  -- /dev/sdc
  MOUNT_OPTIONS -- /dev/sdc /home/fdmanana/btrfs-tests/scratch_1

  generic/240 1s ... [not run] fs block size must be larger than the device block size.  fs block size: 4096, device block size: 4096
  Ran: generic/240
  Not run: generic/240
  Passed all 1 tests

Where before that commit the test ran.

Fix this by checking that the ioctl succeeded.

Fixes: 0e5f196d0a6a ("add a new min_dio_alignment helper")
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agofstests: add missing kernel git commit IDs to some tests
Filipe Manana [Thu, 12 Sep 2024 11:26:38 +0000 (12:26 +0100)]
fstests: add missing kernel git commit IDs to some tests

Three tests (btrfs/321, generic/364 and xfs/608) refer to kernel patches
that are now in Linus' git kernel tree, so update the tests to include
the commit IDs.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agooverlay: create a variant to syncfs error test xfs/546
Amir Goldstein [Fri, 30 Aug 2024 18:08:44 +0000 (20:08 +0200)]
overlay: create a variant to syncfs error test xfs/546

Test overlayfs over xfs with and without "volatile" mount option.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agoext4: Regression test for corruption during on-line resize
Srivathsa Dara [Tue, 17 Sep 2024 13:58:13 +0000 (13:58 +0000)]
ext4: Regression test for corruption during on-line resize

Regression test for:
a6b3bfe176e8 ext4: fix corruption during on-line resize

Signed-off-by: Srivathsa Dara <srivathsa.d.dara@oracle.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agobtrfs/319: make the test work when compression is used
Filipe Manana [Thu, 5 Sep 2024 15:38:18 +0000 (16:38 +0100)]
btrfs/319: make the test work when compression is used

Currently btrfs/319 assumes there is no compression and that the files
get a single extent (1 fiemap line) with a size of 1048581 bytes. However
when testing with compression, for example by passing "-o compress" to
MOUNT_OPTIONS environment variable, we get several extents and two lines
of fiemap output, which makes the test fail since it hardcodes the fiemap
output:

  $ MOUNT_OPTIONS="-o compress" ./check btrfs/319
  FSTYP         -- btrfs
  PLATFORM      -- Linux/x86_64 debian0 6.11.0-rc6-btrfs-next-173+ #1 SMP PREEMPT_DYNAMIC Tue Sep  3 17:40:24 WEST 2024
  MKFS_OPTIONS  -- /dev/sdc
  MOUNT_OPTIONS -- -o compress /dev/sdc /home/fdmanana/btrfs-tests/scratch_1

  btrfs/319 1s ... - output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/319.out.bad)
      --- tests/btrfs/319.out 2024-08-12 14:16:55.653383284 +0100
      +++ /home/fdmanana/git/hub/xfstests/results//btrfs/319.out.bad 2024-09-05 15:24:53.323076548 +0100
      @@ -6,11 +6,13 @@
       e61178ee0288ebe3fa36a3c975b02c94  SCRATCH_MNT/snap/foo
       e61178ee0288ebe3fa36a3c975b02c94  SCRATCH_MNT/snap/bar
       File bar fiemap in the original filesystem:
      -0: [0..2055]: shared|last
      +0: [0..2047]: shared
      +1: [2048..2055]: shared|last
       Creating a new filesystem to receive the send stream...
      ...
      (Run 'diff -u /home/fdmanana/git/hub/xfstests/tests/btrfs/319.out /home/fdmanana/git/hub/xfstests/results//btrfs/319.out.bad'  to see the entire diff)

  HINT: You _MAY_ be missing kernel fix:
        46a6e10a1ab1 btrfs: send: allow cloning non-aligned extent if it ends at i_size

  Ran: btrfs/319
  Failures: btrfs/319
  Failed 1 of 1 tests

So change the test to not rely on the fiemap output in its golden output
and instead just check if all the extents reported by fiemap have the
shared flag set (failing if there are any without the shared flag).

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agogeneric/756: test name_to_handle_at(AT_HANDLE_MNT_ID_UNIQUE) explicitly
Aleksa Sarai [Wed, 4 Sep 2024 19:48:23 +0000 (05:48 +1000)]
generic/756: test name_to_handle_at(AT_HANDLE_MNT_ID_UNIQUE) explicitly

In order to make sure we are actually testing AT_HANDLE_MNT_ID_UNIQUE,
add a test (based on generic/426) which runs the open_by_handle in a
mode where it will error out if there is a problem with getting mount
IDs. The test is skipped if the kernel doesn't support the necessary
features.

Suggested-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agoopen_by_handle: verify u32 and u64 mount IDs
Aleksa Sarai [Wed, 4 Sep 2024 19:48:22 +0000 (05:48 +1000)]
open_by_handle: verify u32 and u64 mount IDs

Now that open_by_handle_at(2) can return u64 mount IDs, do some tests to
make sure they match properly as part of the regular open_by_handle
tests. Also, add automatic tests for the old u32 mount IDs as well.

By default, we do mount ID checks but silently skip the tests if the
syscalls are not supported by the running kernel (to ensure the tests
continue to work for old kernels). We will add some tests explicitly
checking the new features (with no silent skipping) in a future patch.

The u32 mount ID tests require STATX_MNT_ID (Linux 5.8), while the u64
mount ID tests require STATX_MNT_ID_UNIQUE (Linux 6.9) and
AT_HANDLE_MNT_ID_UNIQUE (linux-next).

Link: https://lore.kernel.org/all/20240828-exportfs-u64-mount-id-v3-0-10c2c4c16708@cyphar.com/
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agogeneric/362: skip test on NFS mount
Chen Hanxiao [Tue, 3 Sep 2024 02:18:09 +0000 (10:18 +0800)]
generic/362: skip test on NFS mount

xfstests complains:

FSTYP         -- nfs
PLATFORM      -- Linux/x86_64 r95b-1 5.14.0-496.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Aug 12 18:50:44 EDT 2024
MKFS_OPTIONS  -- 192.168.122.42:/nfsscratch
MOUNT_OPTIONS -- -o vers=4.2 192.168.122.42:/nfsscratch /mnt/scratch

generic/362       QA output created by 362
Failed to open/create file: Invalid argument
Silence is golden
- output mismatch (see /var/lib/xfstests/results//generic/362.out.bad)
    --- tests/generic/362.out   2024-09-02 14:27:09.162636093 -0400
    +++ /var/lib/xfstests/results//generic/362.out.bad  2024-09-02 14:33:36.167636093 -0400
    @@ -1,2 +1,3 @@
     QA output created by 362
    +Failed to open/create file: Invalid argument
     Silence is golden
    ...
    (Run 'diff -u /var/lib/xfstests/tests/generic/362.out /var/lib/xfstests/results//generic/362.out.bad'  to see the entire diff)
Ran: generic/362
Failures: generic/362
Failed 1 of 1 tests

NFS commit 9597c13b forbade open with O_APPEND|O_DIRECT

strace show that dio-append-buf-fault use (O_APPEND|O_DIRECT):

 mount -o vers=4.2 192.168.122.42:/nfstest /mnt/scratch/
 strace ./src/dio-append-buf-fault /mnt/scratch/111
..
  openat(AT_FDCWD, "/mnt/scratch/111", O_WRONLY|O_CREAT|O_TRUNC|O_APPEND|O_DIRECT, 0666) = 3

So skip generic/362 on NFS

Signed-off-by: Chen Hanxiao <chenhx.fnst@fujitsu.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agogeneric: test concurrent direct IO writes and fsync using same fd
Filipe Manana [Thu, 29 Aug 2024 23:10:21 +0000 (00:10 +0100)]
generic: test concurrent direct IO writes and fsync using same fd

Test that a program that has 2 threads using the same file descriptor and
concurrently doing direct IO writes and fsync doesn't trigger any crash
or deadlock.

This is motivated by a bug found in btrfs fixed by the following patch:

  "btrfs: fix race between direct IO write and fsync when using same fd"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agoxfs: functional testing for filesystem properties
Darrick J. Wong [Tue, 27 Aug 2024 18:47:26 +0000 (11:47 -0700)]
xfs: functional testing for filesystem properties

Make sure that fs property storage and retrieval actually work.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agocommon/xfs: FITRIM now supports realtime volumes
Darrick J. Wong [Tue, 27 Aug 2024 18:47:10 +0000 (11:47 -0700)]
common/xfs: FITRIM now supports realtime volumes

XFS now supports FITRIM to the realtime volume.  Detect this support and
enable it.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agoxfs: refactor statfs field extraction
Darrick J. Wong [Tue, 27 Aug 2024 18:46:54 +0000 (11:46 -0700)]
xfs: refactor statfs field extraction

Prepare for the next patch by refactoring the open-coded bits that call
statfs on a mounted xfs filesystem to extract a status field.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agoxfs/004: fix column extraction code
Darrick J. Wong [Tue, 27 Aug 2024 18:46:39 +0000 (11:46 -0700)]
xfs/004: fix column extraction code

Now that the xfs_db freesp command prints a CDF of the free space
histograms, fix the pct column extraction code to handle the two
new columns by <cough> using awk.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agoxfs: test xfs_scrub services
Darrick J. Wong [Tue, 27 Aug 2024 18:46:23 +0000 (11:46 -0700)]
xfs: test xfs_scrub services

Create a pair of new tests that check that xfs_scrub and xfs_scrub_all
will find and test mounted filesystems.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agogeneric/453: check xfs_scrub detection of confusing job offers
Darrick J. Wong [Tue, 27 Aug 2024 18:46:07 +0000 (11:46 -0700)]
generic/453: check xfs_scrub detection of confusing job offers

Earlier this year, ESET revealed that Linux users had been tricked into
opening executables containing malware payloads.  The trickery came in
the form of a malicious zip file containing a filename with the string
"job offer․pdf".  Note that the filename does *not* denote a real pdf
file, since the last four codepoints in the file name are "ONE DOT
LEADER", p, d, and f.  Not period (ok, FULL STOP), p, d, f like you'd
normally expect.

Now that xfs_scrub can look for codepoints that could be confused with a
period followed by alphanumerics, let's make sure it actually works.

Link: https://www.welivesecurity.com/2023/04/20/linux-malware-strengthens-links-lazarus-3cx-supply-chain-attack/
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agogeneric/453: test confusable name detection with 32-bit unicode codepoints
Darrick J. Wong [Tue, 27 Aug 2024 18:45:52 +0000 (11:45 -0700)]
generic/453: test confusable name detection with 32-bit unicode codepoints

Test the confusable name detection when there are 32-bit unicode
sequences in use.  In other words, emoji.  Change the xfs_scrub test to
dump the output to a file instead of passing huge echo commands around.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agobtrfs/319: add git commit ID
Filipe Manana [Mon, 26 Aug 2024 12:22:57 +0000 (13:22 +0100)]
btrfs/319: add git commit ID

The kernel patch was merged into Linus' tree, so update the 'xxxxxxxxxxxx'
stub with the respective git commit ID.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agofstests: btrfs: test reading data with a corrupted checksum tree leaf
Qu Wenruo [Tue, 27 Aug 2024 00:13:54 +0000 (09:43 +0930)]
fstests: btrfs: test reading data with a corrupted checksum tree leaf

[BUG]
There is a bug report that, KASAN get triggered when:

- A read bio needs to be split
  This can happen for profiles with stripes, including
  RAID0/RAID10/RAID5/RAID6.

- An error happens before submitting the new split bio
  This includes:
  * chunk map lookup failure
  * data csum lookup failure

Then during the error path of btrfs_submit_chunk(), the original bio is
fully freed before submitted range has a chance to call its endio
function, resulting a use-after-free bug.

[NEW TEST CASE]
Introduce a new test case to verify the specific behavior by:

- Create a btrfs with enough csum leaves with data RAID0 profile
  To bump the csum tree level, use the minimal nodesize possible (4K).
  Writing 32M data which needs at least 8 leaves for data checksum

  RAID0 profile ensures the data read bios will get split.

- Find the last csum tree leave and corrupt it

- Read the data many times until we trigger the bug or exit gracefully
  With an x86_64 VM with KASAN enabled, it can trigger the KASAN report in
  just 4 iterations (the default iteration number is 32).

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agofstests: btrfs/125: do not use raid5 for metadata
Qu Wenruo [Mon, 26 Aug 2024 22:47:08 +0000 (08:17 +0930)]
fstests: btrfs/125: do not use raid5 for metadata

[BUG]
There are several bug reports of btrfs/125 failure recently, either
causing balance failure (-EIO), or even kernel crash.

The balance failure looks like this:

     Mount normal and balance
    +ERROR: error during balancing '/mnt/scratch': Input/output error
    +There may be more info in syslog - try dmesg | tail
    +md5sum: /mnt/scratch/tf2: Input/output error

The test case btrfs/125 is not reliable in the past, and has been
discussed several times:

https://lore.kernel.org/linux-btrfs/CAL3q7H4oa70DUhOFE7kot62KjxcbvvZKxu62VfLpAcmgsinBFw@mail.gmail.com/
https://lore.kernel.org/linux-btrfs/53f7bace2ac75d88ace42dd811d48b7912647301.1654672140.git.wqu@suse.com/#t

[CAUSE]
There are several different factors involved.

1. RMW mix the old and new metadata, causing unrepairable corruption
   E.g. with the following layout:

   data 1   |<- Stale metadata ->| (from the out-of-date device)
   data 2   |     Unused         |
   parity   |PPPPPPPPPPPPPPPPPPPP|

   In above case, although metadata on data 1 is out-of-date, we can
   still rebuild the correct data from parity and data 2.

   But if we have new metadata writes into the data 2 stripe, an RMW
   will screw up the whole situation:

   data 1   |<- Stale metadata ->| (from the out-of-date device)
   data 2   |<-  New metadata  ->|
   parity   |XXXXXXXXXXXXXXXXXXXX|

   The RMW will use the stale metadata and new metadata to calculate new
   parity.
   The resulted new parity will no longer be able to recover the old
   data 1.

   This is a known bug, thus our documentation is already recommending
   to avoid RAID56 for metadata usage.

   > Metadata
   >    Do not use raid5 nor raid6 for metadata. Use raid1 or raid1c3
   >    respectively.

   And this is very hard to fix, unlike data we can fetch the
   data csum and verify during RMW, we can not do that during RMW.

   At the timing of RMW, we're holding the rbio lock for the full
   stripe.
   If the extent tree search requires a read-recover, it will generate
   another rbio, which may cover the same full stripe we're working on,
   leading to a deadlock.

   Furthermore the current RAID56 repair code is all based on veritical
   sectors, but metadata can cross several horizontal sectors.
   This will require multiple combinations to repair a metadata.

2. Crash caused by double freeing a bio
   By chance if the above RMW corrupted csum tree, then during
   btrfs_submit_chunk() we will hit an error path that leads to double
   freeing of a bio, resulting crash or a KASAN report.

   Thankfully the patch fixing the use-after-free is already sent to the
   mailing list:
   https://lore.kernel.org/linux-btrfs/f4f916352ddf3f80048567ec7d8cc64cb388dc09.1724493430.git.wqu@suse.com/

[WORKAROUND]
Since it's very hard to fix the RAID56 metadata problem without a
deadlock or a huge code rework, for now just use RAID1 for the metadata
of this particular test case.

There may be a chance to fix the situation by properly marking the
missing-then-reappear device as out-of-date, so no direct read from that
device.

But that will also be a huge new feature, not something can be done
immediately.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agogeneric: test to run fsx eof pollution
Brian Foster [Wed, 28 Aug 2024 18:15:34 +0000 (14:15 -0400)]
generic: test to run fsx eof pollution

Filesystem regressions related to partial page zeroing can go
unnoticed for a decent amount of time. A recent example is the issue
of iomap zero range not handling dirty pagecache over unwritten
extents, which leads to wrong behavior on certain file extending
operations (i.e. truncate, write extension, etc.).

fsx does occasionally uncover these sorts of problems, but failures
can be rare and/or require longer running tests outside what is
typically run via full fstests regression runs. fsx now supports a
mode that injects post-eof data in order to explicitly test partial
eof zeroing behavior. This uncovers certain problems more quickly
and applies coverage more broadly across size changing operations.

Add a new test that runs an fsx instance (modeled after generic/127)
with eof pollution mode enabled. While the test is generic, it is
currently limited to XFS as that is currently the only known major
fs that does enough zeroing to satisfy the strict semantics expected
by fsx. The long term goal is to uncover and fix issues so more
filesystems can enable this test.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agofsx: support eof page pollution for eof zeroing test coverage
Brian Foster [Wed, 28 Aug 2024 18:15:33 +0000 (14:15 -0400)]
fsx: support eof page pollution for eof zeroing test coverage

File ranges that are newly exposed via size changing operations are
expected to return zeroes until written to. This behavior tends to
be difficult to regression test as failures can be racy and
transient. fsx is probably the best tool for this type of test
coverage, but uncovering issues can require running for a
significantly longer period of time than is typically invoked
through fstests tests. As a result, these types of regressions tend
to go unnoticed for an unfortunate amount of time.

To facilitate uncovering these problems more quickly, implement an
eof pollution mode in fsx that opportunistically injects post-eof
data prior to operations that change file size. Since data injection
occurs immediately before the size changing operation, it can be
used to detect problems in partial eof page/block zeroing associated
with each relevant operation.

The implementation takes advantage of the fact that mapped writes
can place data beyond eof so long as the page starts within eof. The
main reason for the isolated per-operation approach (vs. something
like allowing mapped writes to write beyond eof, for example) is to
accommodate the fact that writeback zeroes post-eof data on the eof
page. The current approach is therefore not necessarily guaranteed
to detect all problems, but provides more generic and broad test
coverage than the alternative of testing explicit command sequences
and doesn't require significant changes to how fsx works. If this
proves useful long term, further enhancements can be considered that
might facilitate the presence of post-eof data across operations.

Enable the feature with the -e command line option. It is disabled
by default because zeroing behavior is inconsistent across
filesystems. This can also be revisited in the future if zeroing
behavior is refined for the major filesystems that rely on fstests
for regression testing.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agofsx: factor out a file size update helper
Brian Foster [Wed, 28 Aug 2024 18:15:32 +0000 (14:15 -0400)]
fsx: factor out a file size update helper

In preparation for support for eof page pollution, factor out a file
size update helper. This updates the internally tracked file size
based on the upcoming operation and zeroes the appropriate range in
the good buffer for extending operations.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agofsx: don't skip file size and buf updates on simulated ops
Brian Foster [Wed, 28 Aug 2024 18:15:31 +0000 (14:15 -0400)]
fsx: don't skip file size and buf updates on simulated ops

fsx supports the ability to skip through a certain number of
operations of a given command sequence before beginning full
operation. The way this works is by tracking the operation count,
simulating minimal side effects of skipped operations in-memory, and
then finally writing out the in-memory state to the target file when
full operation begins.

Several fallocate() related operations don't correctly track
in-memory state when simulated, however. For example, consider an
ops file with the following two operations:

  zero_range 0x0 0x1000 0x0
  read 0x0 0x1000 0x0

... and an fsx run like so:

  fsx -d -b 2 --replay-ops=<opsfile> <file>

This simulates the zero_range operation, but fails to track the file
extension that occurs as a side effect such that the subsequent read
doesn't occur as expected:

  Will begin at operation 2
  skipping zero size read

The read is skipped in this case because the file size is zero.  The
proper behavior, and what is consistent with other size changing
operations, is to make the appropriate in-core changes before
checking whether an operation is simulated so the end result of
those changes can be reflected on-disk for eventual non-simulated
operations. This results in expected behavior with the same ops file
and test command:

  Will begin at operation 2
  2 read  0x0 thru        0xfff   (0x1000 bytes)

Update zero, copy and clone range to do the file size and EOF change
related zeroing before checking against the simulated ops count.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agof2fs/003: add missing _fixed_by_kernel_commit line
Chao Yu [Mon, 2 Sep 2024 10:18:44 +0000 (18:18 +0800)]
f2fs/003: add missing _fixed_by_kernel_commit line

The bug related to this regression testcase has been fixed by commit
b40a2b003709 ("f2fs: use meta inode for GC of atomic file"), let's
add missing _fixed_by_kernel_commit line for this testcase.

Cc: Jaegeuk Kim <jaegeuk@kernel.org>
Cc: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agooverlay: deprecate test t_truncate_self
Amir Goldstein [Fri, 30 Aug 2024 19:45:46 +0000 (21:45 +0200)]
overlay: deprecate test t_truncate_self

Since kernel commit 2a010c412853 ("fs: don't block i_writecount during
exec"), truncating an executable file while it is being executed is
allowed. Therefore, the test t_truncate_self now fails, so remove it.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agogeneric/615: add to the quick group
Christoph Hellwig [Wed, 21 Aug 2024 04:47:43 +0000 (06:47 +0200)]
generic/615: add to the quick group

This is a simple and very quick test, add it to the quick group.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
10 months agoxfs: add a test for v1 inodes with nonzero nlink and onlink fields
Darrick J. Wong [Thu, 22 Aug 2024 05:05:40 +0000 (22:05 -0700)]
xfs: add a test for v1 inodes with nonzero nlink and onlink fields

Add a regression test for XFS V1 inodes that have both nlink and onlink
fields set to 1.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agocommon/config: Correctly ignore {TEST|SCRATCH}_DEV for tmpfs
André Almeida [Wed, 21 Aug 2024 15:57:46 +0000 (12:57 -0300)]
common/config: Correctly ignore {TEST|SCRATCH}_DEV for tmpfs

As per commit 264e5358e2c2 ("tmpfs: don't require {TEST|SCRATCH}_DEV"),
tmpfs doesn't need TEST or SCRATCH devices due to being a RAM-based
filesystem.

Fix the check by comparing the content of the variable TEST_DEV, instead
of comparing with the string TEST_DEV. Same for SCRATCH_DEV.

Fixes: 264e5358e2c2 ("tmpfs: don't require {TEST|SCRATCH}_DEV")
Signed-off-by: André Almeida <andrealmeid@igalia.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agogeneric/755: test that inode's ctime is updated on unlink
Jeff Layton [Tue, 20 Aug 2024 19:48:25 +0000 (15:48 -0400)]
generic/755: test that inode's ctime is updated on unlink

I recently found and fixed a bug in btrfs where it wasn't updating the
citme on the target inode when unlinking [1]. Add a fstest for this.

[1]: https://lore.kernel.org/linux-btrfs/20240812-btrfs-unlink-v1-1-ee5c2ef538eb@kernel.org/

Signed-off-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agoreplace _min_dio_alignment with calls to src/min_dio_alignment
Christoph Hellwig [Wed, 14 Aug 2024 04:52:14 +0000 (06:52 +0200)]
replace _min_dio_alignment with calls to src/min_dio_alignment

Use the min_dio_alignment C tool to check the minimum alignment.
This allows using the values obtained from statx instead of just guessing
based on the sector size and page size.

For tests using the scratch device this sometimes required moving code
around a bit to ensure the scratch device is actually mounted before
querying the alignment.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agogeneric: don't use _min_dio_alignment without a device argument
Christoph Hellwig [Wed, 14 Aug 2024 04:52:13 +0000 (06:52 +0200)]
generic: don't use _min_dio_alignment without a device argument

Replace calls to _min_dio_alignment that do not provide a device to
check with calls to the feature utility to query the page size, as that
is what these calls actually do.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agoxfs/424: don't use _min_dio_alignment
Christoph Hellwig [Wed, 14 Aug 2024 04:52:12 +0000 (06:52 +0200)]
xfs/424: don't use _min_dio_alignment

xfs/424 tests xfs_db and not the kernel xfs code, and really wants
the device sector size and not the minimum direct I/O alignment.

Switch to a direct call of the blockdev utility.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agoadd a new min_dio_alignment helper
Christoph Hellwig [Wed, 14 Aug 2024 04:52:11 +0000 (06:52 +0200)]
add a new min_dio_alignment helper

Add a new C program to find the minimum direct I/O alignment.  This
uses the statx stx_dio_offset_align field if provided, then falls
back to the BLKSSZGET ioctl for block backed file systems and finally
the page size.  It is intended as a more capable replacement for the
_min_dio_alignment bash helper.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agostatx.h: update to latest kernel UAPI
Christoph Hellwig [Wed, 14 Aug 2024 04:52:10 +0000 (06:52 +0200)]
statx.h: update to latest kernel UAPI

Update the localy provided statx definition to the latest kernel UAPI,
and use it unconditionally instead only if no kernel version is provided.

This allows using more recent additions than provided in the system
headers.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agobtrfs: test send clones extents with unaligned end offset ending at i_size
Filipe Manana [Mon, 12 Aug 2024 13:51:09 +0000 (14:51 +0100)]
btrfs: test send clones extents with unaligned end offset ending at i_size

Test that a send operation will issue a clone operation for a shared
extent of a file if the extent ends at the i_size of the file and the
i_size is not sector size aligned.

This verifies an improvement to the btrfs send feature implemented by
the following kernel patch:

  "btrfs: send: allow cloning non-aligned extent if it ends at i_size"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agoxfs/348: add helper tags
Ma Xinjian [Fri, 9 Aug 2024 08:17:22 +0000 (16:17 +0800)]
xfs/348: add helper tags

This test requires a kernel patch since 3bf963a6c6 ("xfs/348: partially revert
dbcc549317"), so note that in the test.

Signed-off-by: Ma Xinjian <maxj.fnst@fujitsu.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agoxfs/516: use _scratch_mkfs_xfs
Christoph Hellwig [Wed, 7 Aug 2024 14:35:10 +0000 (07:35 -0700)]
xfs/516: use _scratch_mkfs_xfs

Use _scratch_mkfs_xfs instead of _scratch_mkfs to get _notrun handling
for unsupported option combinations.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agoxfs/432: use _scratch_mkfs_xfs
Christoph Hellwig [Wed, 7 Aug 2024 14:35:09 +0000 (07:35 -0700)]
xfs/432: use _scratch_mkfs_xfs

Use _scratch_mkfs_xfs instead of _scratch_mkfs to get _notrun handling
for unsupported option combinations.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agocommon: _notrun if _scratch_mkfs_xfs failed
Christoph Hellwig [Wed, 7 Aug 2024 14:35:08 +0000 (07:35 -0700)]
common: _notrun if _scratch_mkfs_xfs failed

If we fail to create a file system with specific passed in options, that
that these options conflict with other options $MKFS_OPTIONS. Don't
fail the test case for that, but instead _norun it and display the options
that caused it to fail.

Add a lower-level _try_scratch_mkfs_xfs helper for those places that want
to check the return value.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agocommon: _notrun if _scratch_mkfs_sized failed
Christoph Hellwig [Wed, 7 Aug 2024 14:35:07 +0000 (07:35 -0700)]
common: _notrun if _scratch_mkfs_sized failed

If we fail to create a file system of a specific size that means it can't
work with some of the options in $MKFS_OPTIONS like the log size.  Don't
fail the test case for that, but instead _norun it and display the options
that caused it to fail.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agoxfs: test online repair when xfiles consists of THPs
Darrick J. Wong [Tue, 6 Aug 2024 22:55:23 +0000 (15:55 -0700)]
xfs: test online repair when xfiles consists of THPs

Fork xfs/286 so that we can ensure that the xfile and xmbuf code in
fsck can handle THPs and large folios.  This actually caused a
regression in the mm code during 6.10.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agoxfs: remove all traces of xfs_check
Darrick J. Wong [Tue, 6 Aug 2024 22:56:06 +0000 (15:56 -0700)]
xfs: remove all traces of xfs_check

xfsprogs stopped shipping xfs_check (the wrapper script) in May 2014.
It's now been over a decade since it went away, and its replacements
(xfs_repair and xfs_scrub) now detect a superset of the problems that
check can find.

There is no longer any point in invoking xfs_check, so let's remove it
from fstests completely.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agobtrfs: add test for btrfstune squota enable/disable
Boris Burkov [Fri, 12 Jul 2024 18:52:36 +0000 (11:52 -0700)]
btrfs: add test for btrfstune squota enable/disable

btrfstune supports enabling simple quotas on a fleshed out filesystem
(without adding owner refs) and clearing squotas entirely from a
filesystem that ran under squotas (clearing the incompat bit)

Test that these operations work on a relatively complicated filesystem
populated by fsstress while preserving fssum.

Signed-off-by: Boris Burkov <boris@bur.io>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agogeneric/732: don't run it on tmpfs
Yang Erkun [Mon, 29 Jul 2024 08:47:15 +0000 (16:47 +0800)]
generic/732: don't run it on tmpfs

Like what 4fd042e0465c("generic/732: don't run it on NFS") say, the same
options for tmpfs won't share the same backend. Skip it for tmpfs.

Besides, add some explanation for why we should skip tmpfs.

Signed-off-by: Yang Erkun <yangerkun@huawei.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agogeneric/732: remove unused code
Yang Erkun [Mon, 29 Jul 2024 08:47:14 +0000 (16:47 +0800)]
generic/732: remove unused code

After commit 4fd042e0465c("generic/732: don't run it on NFS"), we will
not run this case for nfs, so remove this unused code.

Signed-off-by: Yang Erkun <yangerkun@huawei.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agoxfs/233: don't require rmap
Christoph Hellwig [Mon, 29 Jul 2024 14:20:27 +0000 (07:20 -0700)]
xfs/233: don't require rmap

Nothing in xfs/233 requires an rmap, it can run on any file system.
And it is a very useful test because it starts out with a very small
file system (or RT subvolume), which exercise some code paths no other
test does.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agoadd more tests to the growfs group
Christoph Hellwig [Mon, 29 Jul 2024 14:20:26 +0000 (07:20 -0700)]
add more tests to the growfs group

xfs/127 xfs/233 exercises growfs behavior, add them to the growfs group.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agobtrfs/287: wait for subvolume deletion to complete
Filipe Manana [Wed, 31 Jul 2024 12:41:24 +0000 (13:41 +0100)]
btrfs/287: wait for subvolume deletion to complete

The test deletes the subvolume and then immediately calls the logical
resolve ioctl to confirm the extent is not referenced by the subvolume
anymore. This however may often fail because the subvolume delete only
makes the subvolume not accessible to user space anymore, but the actual
deletion of the subvolume tree, and all its data references, happens in
the background in the cleaner kthread running in kernel space.

So if by the time we do the query the cleaner kthread has not yet deleted
the subvolume tree, the test fails like this:

  $ ./check btrfs/287
  FSTYP         -- btrfs
  PLATFORM      -- Linux/x86_64 debian0 5.14.0-btrfs-next-22 #1 SMP Tue Jul 30 16:31:55 WEST 2024
  MKFS_OPTIONS  -- /dev/sdc
  MOUNT_OPTIONS -- /dev/sdc /home/fdmanana/btrfs-tests/scratch_1

  btrfs/287 0s ... - output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/287.out.bad)
      --- tests/btrfs/287.out 2024-07-30 17:40:49.037599612 +0100
      +++ /home/fdmanana/git/hub/xfstests/results//btrfs/287.out.bad 2024-07-31 13:06:28.275728352 +0100
      @@ -82,12 +82,18 @@
       inode 257 offset 20971520 snap2
       inode 257 offset 12582912 snap2
       inode 257 offset 4194304 snap2
      +inode 257 offset 20971520 snap1
      +inode 257 offset 12582912 snap1
      +inode 257 offset 4194304 snap1
       inode 257 offset 20971520 root 5
      ...
      (Run 'diff -u /home/fdmanana/git/hub/xfstests/tests/btrfs/287.out /home/fdmanana/git/hub/xfstests/results//btrfs/287.out.bad'  to see the entire diff)

  HINT: You _MAY_ be missing kernel fix:
        0cad8f14d70c btrfs: fix backref walking not returning all inode refs

Fix this by using the "subvolume sync" command to wait for the subvolume
to be deleted by the cleaner kthread before doing logical resolve queries.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agobtrfs/312: add git commit ID
Filipe Manana [Mon, 5 Aug 2024 15:03:47 +0000 (16:03 +0100)]
btrfs/312: add git commit ID

The corresponding kernel fix was merged into Linus' tree, so specify its
ID in the test.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agofstests: remove unnecessary stdout/stderr redirection for run_check calls
Filipe Manana [Wed, 31 Jul 2024 15:59:58 +0000 (16:59 +0100)]
fstests: remove unnecessary stdout/stderr redirection for run_check calls

It's pointless, and confusing, to have calls to run_check redirect the
stdout and/or stderr, because run_check already redirects it:

   $ cat common/rc
   (...)
   run_check()
   {
       echo "# $@" >> $seqres.full 2>&1
       "$@" >> $seqres.full 2>&1 || _fail "failed: '$@'"
   }
   (...)

So remove those redirections.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agogeneric/019: redirect fsstress output to log file instead
Filipe Manana [Wed, 31 Jul 2024 15:59:24 +0000 (16:59 +0100)]
generic/019: redirect fsstress output to log file instead

Currently we are redirecting stdout and stderr of fsstress to /dev/null.
In case we have a test failure, it's useful to know the seed that
fsstress was using because it might be helpful to run it again with that
same seed in the hope of being able to reproduce the failure.

So redirect stdout/stderr to the log file $seqres.full instead, so
that we'll see a line like this after running the test:

    seed = 1722389488

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agogeneric: test page fault during direct IO write with O_APPEND
Filipe Manana [Sat, 27 Jul 2024 11:28:16 +0000 (12:28 +0100)]
generic: test page fault during direct IO write with O_APPEND

Test that doing a direct IO append write to a file when the input buffer
was not yet faulted in, does not result in an incorrect file size.

This exercises a bug on btrfs reported by users and which is fixed by
the following kernel patch:

   "btrfs: fix corruption after buffer fault in during direct IO append write"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agogeneric/754: fix _fixed_by tags
Darrick J. Wong [Fri, 26 Jul 2024 16:51:07 +0000 (09:51 -0700)]
generic/754: fix _fixed_by tags

This test requires an xfs_repair patch, so note that in the test.  Also
update the kernel git hash since we now have one.

Reported-by: maxj.fnst@fujitsu.com
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agoxfs: add a few more tests to the repair group
Christoph Hellwig [Mon, 15 Jul 2024 05:38:37 +0000 (07:38 +0200)]
xfs: add a few more tests to the repair group

Add a bunch of tests that test repair for the RT subvolume to the repair
group.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agoxfs/011: support byte-based grant heads are stored in bytes now
Christoph Hellwig [Mon, 15 Jul 2024 06:24:53 +0000 (08:24 +0200)]
xfs/011: support byte-based grant heads are stored in bytes now

New kernels where reservation grant track the actual reservation space
consumed in bytes instead of LSNs in cycle/block tuples export different
sysfs files for this information.

Adapt the test to detect which version is exported, and simply check
for a near-zero reservation space consumption for the byte based version.

Based on work from Dave Chinner.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agof2fs: test for race condition in between atomic_write and dio
Chao Yu [Sat, 20 Jul 2024 07:45:40 +0000 (15:45 +0800)]
f2fs: test for race condition in between atomic_write and dio

Test that we will simulate sqlite atomic write logic w/ below steps:
1. create a regular file, and initialize it w/ 0xff data
2. start transaction (via F2FS_IOC_START_ATOMIC_WRITE) on it
3. write transaction data
4. trigger direct read/write IO to check whether it fails or not
5. commit and end the transaction (via F2FS_IOC_COMMIT_ATOMIC_WRITE)
This is a regression test to check handling of race condition in
between atomic_write and direct IO.

Cc: Jaegeuk Kim <jaegeuk@kernel.org>
Cc: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agof2fs: test for race condition in between atomic_write and gc
Chao Yu [Sat, 20 Jul 2024 07:43:16 +0000 (15:43 +0800)]
f2fs: test for race condition in between atomic_write and gc

Test that we will simulate sqlite atomic write logic w/ below steps:
1. create a regular file, and initialize it w/ 0xff data
2. start transaction (via F2FS_IOC_START_ATOMIC_WRITE) on it
3. write transaction data
4. trigger foreground GC to migrate data block of the file
5. commit and end the transaction
6. check consistency of transaction
This is a regression test to check handling of race condition in
between atomic_write and GC.

Cc: Jaegeuk Kim <jaegeuk@kernel.org>
Cc: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
11 months agobtrfs: properly shutdown subvolume stress worker to avoid umount failures
Filipe Manana [Wed, 24 Jul 2024 13:58:52 +0000 (14:58 +0100)]
btrfs: properly shutdown subvolume stress worker to avoid umount failures

When killing a test that is using the subvolume stress worker, we may end
up in a situation where we end up leaving a subvolume mounted which makes
the shutdown sequence fail. Example when killing a script that keeps
running fstests in a loop:

   FSTYP         -- btrfs
   PLATFORM      -- Linux/x86_64 debian0 6.10.0-rc7-btrfs-next-167+ #1 SMP PREEMPT_DYNAMIC Thu Jul 11 17:54:07 WEST 2024
   MKFS_OPTIONS  -- /dev/sdc
   MOUNT_OPTIONS -- /dev/sdc /home/fdmanana/btrfs-tests/scratch_1

   (...)
   btrfs/065 23s ... ^C^C^C
   Iteration 134, errors 1, leaks 0, Wed Jul 24 12:14:33 PM WEST 2024, flakey errors: 0 MKFS_OPTIONS="" MOUNT_OPTIONS=""

   SCRATCH_DEV=/dev/sdc is mounted but not on SCRATCH_MNT=/home/fdmanana/btrfs-tests/scratch_1 - aborting
   Already mounted result:
   /dev/sdc /home/fdmanana/btrfs-tests/scratch_1 /dev/sdc /home/fdmanana/btrfs-tests/dev/065.mnt
   grep: results/btrfs/065.out.bad: No such file or directory
   Error iteration 134, total errors 2, leaks 0
   'results/btrfs/065.full' -> '/home/fdmanana/failures/btrfs_065/134/065.full'

Running 'mount' to see what's going on:

   $ mount
   (...)
   /dev/sdb on /home/fdmanana/btrfs-tests/dev type btrfs (rw,relatime,discard=async,space_cache=v2,subvolid=5,subvol=/)
   /dev/sdc on /home/fdmanana/btrfs-tests/scratch_1 type btrfs (rw,relatime,discard=async,space_cache=v2,subvolid=5,subvol=/)
   /dev/sdc on /home/fdmanana/btrfs-tests/dev/065.mnt type btrfs (rw,relatime,discard=async,space_cache=v2,subvolid=2627,subvol=/subvol_3395330)

Then this makes the next attempt to run a test (./check) always fail due
to the extra mount of the subvolume, requiring one to manually umount the
subvolume before running fstests again.

So update _btrfs_kill_stress_subvolume_pid() to also unmount the subvolume.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
12 months agofstests: btrfs/012: fix a false alert due to socket/pipe files
Qu Wenruo [Wed, 17 Jul 2024 22:54:49 +0000 (08:24 +0930)]
fstests: btrfs/012: fix a false alert due to socket/pipe files

[BUG]
On my Archlinux VM, the test btrfs/012 always fail with the following
output diff:

     QA output created by 012
    +File /etc/pacman.d/gnupg/S.dirmngr is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.dirmngr is a socket
    +File /etc/pacman.d/gnupg/S.gpg-agent is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.gpg-agent is a socket
    +File /etc/pacman.d/gnupg/S.gpg-agent.browser is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.gpg-agent.browser is a socket
    +File /etc/pacman.d/gnupg/S.gpg-agent.extra is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.gpg-agent.extra is a socket
    +File /etc/pacman.d/gnupg/S.gpg-agent.ssh is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.gpg-agent.ssh is a socket
    +File /etc/pacman.d/gnupg/S.keyboxd is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.keyboxd is a socket
    ...

[CAUSE]
It's a false alerts.

When diff hits two files which are not directory/softlink/regular files
(aka, socket/pipe/char/block files), they are all treated as
non-comparable.
In that case, diff would just do the above message.

And with Archlinux, pacman (the package manager) maintains its gpg
directory inside "/etc/pacman.d/gnupg", and the test case uses
"/etc" as the source directory to populate the target ext4 fs.

And the socket files inside "/etc/pacman.d/gnupg" is causing the false
alerts.

[FIX]
- Use fsstress to populate the fs
  That covers all kind of operations, including creating special files.
  And fsstress is very reproducible, with the seed saved to the full
  log, it's much easier to reproduce than using the distro dependent
  "/etc/" directory.

- Use fssum to save the digest and later verify the contents
  It does not only verify the contents but also other things like
  timestamps/xattrs/uid/gid/mode/etc.
  And it's more comprehensive than the content oriented diff tool.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
12 months agobtrfs: test a compressed send stream scenario that triggered a read corruption
Filipe Manana [Tue, 16 Jul 2024 10:14:45 +0000 (11:14 +0100)]
btrfs: test a compressed send stream scenario that triggered a read corruption

Test a scenario of a compressed send stream that triggered a bug in the
extent map merging code introduced in the merge window for 6.11.

The commit that introduced the bug is on its way to Linus' tree and its
subject is:

   "btrfs: introduce new members for extent_map"

The corresponding fix was submitted to the btrfs mailing list, with the
subject:

  "btrfs: fix corrupt read due to bad offset of a compressed extent map"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
12 months agobtrfs/220: remove integrity checker bits
David Sterba [Thu, 18 Jul 2024 15:57:54 +0000 (17:57 +0200)]
btrfs/220: remove integrity checker bits

We've deleted the integrity checker code in 6.8, no point testing it.

Signed-off-by: David Sterba <dsterba@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
12 months agobtrfs/081: wait for reader process to exit before cycle mounting
Filipe Manana [Fri, 28 Jun 2024 17:04:49 +0000 (18:04 +0100)]
btrfs/081: wait for reader process to exit before cycle mounting

We send a kill signal to the reader process, check the md5sum of the
files and then cycle mount the scratch device. Most of the time the
reader process has already terminated before we attempt the cycle mount,
but sometimes it may still be alive in which case the cat command
executed by the reader process may fail because the scratch fs was
unmounted and the target file doesn't exist. This makes the cat command
print an error message and the test fail like this:

     Verifying file digests after cloning
     14968c092c68e32fa35e776392d14523  SCRATCH_MNT/foo
     14968c092c68e32fa35e776392d14523  SCRATCH_MNT/bar
    +cat: /opt/scratch/bar: No such file or directory
    +cat: /opt/scratch/bar: No such file or directory
    +cat: /opt/scratch/bar: No such file or directory
    +cat: /opt/scratch/bar: No such file or directory
    ...
    (Run diff -u /opt/xfstests/tests/btrfs/081.out

Fix this by making the test wait for the reader to terminate after
sending it the kill signal.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Boris Burkov <boris@bur.io>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agofstests: btrfs/029: add fixes for the kernel behavior change
Qu Wenruo [Wed, 10 Jul 2024 23:42:55 +0000 (09:12 +0930)]
fstests: btrfs/029: add fixes for the kernel behavior change

Since fstests commit 866948e00073 ("btrfs/029: change the cross vfsmount
reflink test"), the test case will fail for older kernels (e.g. 5.14
kernels from SLE).

The failure is a false alert, but it would still take some time to
figure it out.
So add the fixes tag to make it more clear that it's a kernel behavior
change.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agobtrfs: fix _require_btrfs_send_version to detect btrfs-progs support
Filipe Manana [Fri, 12 Jul 2024 09:54:24 +0000 (10:54 +0100)]
btrfs: fix _require_btrfs_send_version to detect btrfs-progs support

Commit 199d0a992536df3702a0c4843d2a449d54f399c2 ("common/btrfs: introduce
_require_btrfs_send_version") turned _require_btrfs_send_v2 into a generic
helper to detect support for any send stream version, however it's only
working for detecting kernel support, it misses detecting the support from
btrfs-progs - it always checks only that it supports v2 (the send command
supports the --compressed-data option).

Fix that by verifying that btrfs-progs supports the requested version.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
12 months agobtrfs: add test for subvolid reuse with squota
Boris Burkov [Tue, 9 Jul 2024 17:51:04 +0000 (10:51 -0700)]
btrfs: add test for subvolid reuse with squota

Squotas are likely to leave qgroups that outlive deleted subvolids.
Before the kernel patch
2b8aa78cf127 ("btrfs: qgroup: fix qgroup id collision across mounts")
this would lead to a repeated subvolid which would collide on an
existing qgroup id and error out with EEXIST. In snapshot creation, this
would lead to a read only fs.

Add a test which exercises the path that could create duplicate
subvolids but with squotas enabled, which should avoid the trap.

Signed-off-by: Boris Burkov <boris@bur.io>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
[zlang: Remove _supported_fs line]

12 months agobtrfs: update golden output of RST test cases
Johannes Thumshirn [Mon, 1 Jul 2024 08:01:09 +0000 (10:01 +0200)]
btrfs: update golden output of RST test cases

Starting with kernel patch "btrfs: remove raid-stripe-tree
encoding field from stripe_extent" and btrfs-progs commit
7c549b5f7cc0 ("btrfs-progs: remove raid stripe encoding"), the on-disk
format of the raid stripe tree got changed.

As the feature is still experimental and not to be used in production, it
is OK to do a on-disk format change.

Update the golden output of the RAID stripe tree test cases after the
on-disk format and print format changes.

Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
12 months agoremove spurious _supported_fs calls
Christoph Hellwig [Sun, 23 Jun 2024 12:10:37 +0000 (14:10 +0200)]
remove spurious _supported_fs calls

Remove _supported_fs calls for generic in the generic directory
or for $FSTYP in the $FSTYP directory.

This leaves us with the negative checks, and the overloaded ext4
directory where some tests can also be run for ext2 and ext3.

While at it also remove the pointless "real QA test starts here"
usually placed right next to it.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agogeneric/746: clean up fs support
Christoph Hellwig [Sun, 23 Jun 2024 12:10:36 +0000 (14:10 +0200)]
generic/746: clean up fs support

Use a single case statement for fs-specific options and to check if
this test is supported at all.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agogeneric/745: rework support fs checking
Christoph Hellwig [Sun, 23 Jun 2024 12:10:35 +0000 (14:10 +0200)]
generic/745: rework support fs checking

To prepare for deprecating _supported_fs use a case statement and
_notrun.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agogeneric/740: enable by default
Christoph Hellwig [Sun, 23 Jun 2024 12:10:34 +0000 (14:10 +0200)]
generic/740: enable by default

Instead of limiting this test to a few file systems, opt out the
file systems supported in common/rc that don't support overwrite
checking at all, and those like extN that support it, but only when
run interactively.

Also remove support for really old mkfs.btrfs versions that lack
the overwrite check.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agogeneric/740: skip jffs2 as foreign fs earlier
Christoph Hellwig [Sun, 23 Jun 2024 12:10:33 +0000 (14:10 +0200)]
generic/740: skip jffs2 as foreign fs earlier

Commit a633d252e3c4 ("shared/032: add options for jffs2") added a
check to skip checking the overwrite of jffs2, but only after
adding specific mkfs options for it and zeroing part of the device.

Switch to skipping it earlier in a more obvious place.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agogeneric/740: pass the --quick option to mkfs.ntfs
Christoph Hellwig [Sun, 23 Jun 2024 12:10:32 +0000 (14:10 +0200)]
generic/740: pass the --quick option to mkfs.ntfs

Without --quick mkfs.ntfs will zero the entire device, which can take
a very long time.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agogeneric/740: clean up handling of mkfs options
Christoph Hellwig [Sun, 23 Jun 2024 12:10:31 +0000 (14:10 +0200)]
generic/740: clean up handling of mkfs options

Use a single case statement instead of lots of conditionals.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoremove support for ext4dev
Christoph Hellwig [Sun, 23 Jun 2024 12:10:30 +0000 (14:10 +0200)]
remove support for ext4dev

ext4dev is a long deprecated alias for ext4 that was used during early
ext4 development.  Drop support for it.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoxfs/242: fix test failure due to incorrect filtering in _filter_bmap
Long Li [Fri, 12 Jul 2024 06:47:16 +0000 (14:47 +0800)]
xfs/242: fix test failure due to incorrect filtering in _filter_bmap

I got a failure in xfs/242 as follows, it can be easily reproduced
when I run xfs/242 as a cyclic test.

  13. data -> unwritten -> data
  0: [0..127]: data
  -1: [128..511]: unwritten
  -2: [512..639]: data
  +1: [128..639]: unwritten

The root cause, as Dave pointed out in previous email [1], is that
_filter_bmap may incorrectly match the AG-OFFSET in column 5 for datadev
files. On the other hand, _filter_bmap missing a "next" to jump out when
it matches "data" in the 5th column, otherwise it might print the result
twice. The issue was introduced by commit 7d5d3f77154e ("xfs/242: fix
_filter_bmap for xfs_io bmap that does rt file properly"). The failure
disappeared when I retest xfs/242 by reverted commit 7d5d3f77154e.

Fix it by matching the 7th column first and then the 5th column in
_filter_bmap, because the rtdev file only has 5 columns in the `bmap -vp`
output.

[1] https://lore.kernel.org/all/Zh9UkHEesvrpSQ7J@dread.disaster.area/

Signed-off-by: Long Li <leo.lilong@huawei.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoxfs/016: fix test fail when head equal to near_end_min
Long Li [Fri, 12 Jul 2024 06:47:15 +0000 (14:47 +0800)]
xfs/016: fix test fail when head equal to near_end_min

xfs/016 checks for corruption in the log when it wraps. It looks for a log
head that is at or above the minimum log size. If the final position of
the log head equals near_end_min, the test will fail. Under these
conditions, we should let the test continue.

Signed-off-by: Long Li <leo.lilong@huawei.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agofiexchange.h: update XFS_IOC_EXCHANGE_RANGE again
Darrick J. Wong [Wed, 3 Jul 2024 21:36:45 +0000 (14:36 -0700)]
fiexchange.h: update XFS_IOC_EXCHANGE_RANGE again

We corrected the definition for XFS_IOC_EXCHANGE_RANGE towards the end
of 6.10, so do it again.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoxfs/444: fix agfl reset warning detection for small log buffers
Darrick J. Wong [Wed, 3 Jul 2024 21:36:05 +0000 (14:36 -0700)]
xfs/444: fix agfl reset warning detection for small log buffers

Collectively, the ten subtests in xfs/444 can generate a lot of kernel
log data.  If the amount of log data is enough to overflow the kernel
log buffers, the AGFL reset warning generated by fix_start and fix_wrap
might have been overwritten by subsequent log data.  Fix this by
checking for the reset warning after each test and only complaining if
at the end if we have /never/ seen the warning.

Found by running on a kernel configured with CONFIG_LOG_BUF_SHIFT=14
(16K).  This happened to be a Raspberry Pi, but in principle this can
happen to anyone.  I'd never noticed this before because x86 helpfully
sets it to 17 (128K) by default.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agogeneric: test creating and removing symlink xattrs
Darrick J. Wong [Thu, 20 Jun 2024 21:01:06 +0000 (14:01 -0700)]
generic: test creating and removing symlink xattrs

This began as a regression test for the issues identified in "xfs: allow
symlinks with short remote targets".  To summarize, the kernel XFS code
does not convert a remote symlink back to a shortform symlink after
deleting the attr fork.  Recent attempts to tighten validation have
flagged this incorrectly, so we need a regression test to focus on this
dusty corner of the codebase.

However, there's nothing in here that's xfs-specific so it's a generic
test.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoxfs/348: partially revert dbcc549317 ("xfs/348: golden output is not correct")
Dave Chinner [Thu, 20 Jun 2024 21:00:50 +0000 (14:00 -0700)]
xfs/348: partially revert dbcc549317 ("xfs/348: golden output is not correct")

In kernel commit 1eb70f54c445f ("xfs: validate inode fork size against
fork format"), we incorrectly started flagging as corrupt symlinks with
short targets that would fit in the inode core but are remote.  The
kernel has historically written out symlinks this way and read them back
in, so we're fixing that.

The 1eb70 change came with change dbcc to fstests to adjust the golden
output; since we're adjusting the kernel back to old behavior, we need
to adjust the test too.

Fixes: dbcc549317 ("xfs/348: golden output is not correct")
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoxfs/122: update for vectored scrub
Darrick J. Wong [Thu, 20 Jun 2024 21:00:35 +0000 (14:00 -0700)]
xfs/122: update for vectored scrub

Add the two new vectored scrub structures.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoscrub: test correction of directory tree corruptions
Darrick J. Wong [Thu, 20 Jun 2024 21:00:19 +0000 (14:00 -0700)]
scrub: test correction of directory tree corruptions

Make sure that we can fix directory tree loops and multiply-owned dirs.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agocommon/fuzzy: stress directory tree modifications with the dirtree tester
Darrick J. Wong [Thu, 20 Jun 2024 21:00:03 +0000 (14:00 -0700)]
common/fuzzy: stress directory tree modifications with the dirtree tester

Stress test the directory tree corruption detector by racing it with
fsstress.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoxfs: add parent pointer inject test
Allison Henderson [Thu, 20 Jun 2024 20:59:48 +0000 (13:59 -0700)]
xfs: add parent pointer inject test

Add a test to verify parent pointers after an error injection and log
replay.

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Catherine Hoang <catherine.hoang@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoxfs: add multi link parent pointer test
Allison Henderson [Thu, 20 Jun 2024 20:59:32 +0000 (13:59 -0700)]
xfs: add multi link parent pointer test

Add a test to verify parent pointers while multiple links to a file are
created and removed.

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Catherine Hoang <catherine.hoang@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoxfs: add parent pointer test
Allison Henderson [Thu, 20 Jun 2024 20:59:16 +0000 (13:59 -0700)]
xfs: add parent pointer test

Add a test to verify basic parent pointers operations (create, move, link,
unlink, rename, overwrite).

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Catherine Hoang <catherine.hoang@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
[djwong: test the xfs_io parent -p argument too]
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agocommon: add helpers for parent pointer tests
Allison Henderson [Thu, 20 Jun 2024 20:59:01 +0000 (13:59 -0700)]
common: add helpers for parent pointer tests

Add helper functions in common/parent to parse and verify parent
pointers. Also add functions to check that mkfs, kernel, and xfs_io
support parent pointers.

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Catherine Hoang <catherine.hoang@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
[djwong: add license and copyright, dont _fail tests immediately, make
 sure the pptr-generated paths match the dir-generated paths]
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoxfs/306: fix formatting failures with parent pointers
Darrick J. Wong [Thu, 20 Jun 2024 20:58:45 +0000 (13:58 -0700)]
xfs/306: fix formatting failures with parent pointers

The parent pointers feature isn't supported on tiny 20MB filesystems
because the larger directory transactions result in larger minimum log
sizes, particularly with nrext64 enabled:

** mkfs failed with extra mkfs options added to " -m rmapbt=0, -i nrext64=1, -n parent=1," by test 306 **
** attempting to mkfs using only test 306 options: -d size=20m -n size=64k **
max log size 5108 smaller than min log size 5310, filesystem is too small

We don't support 20M filesystems anymore, so bump the filesystem size up
to 100M and skip this test if we can't actually format the filesystem.
Convert the open-coded punch-alternating logic into a call to that
program to reduce execve overhead, which more than makes up having to
write 5x as much data to fragment the free space.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoxfs/{018,191,288}: disable parent pointers for these tests
Darrick J. Wong [Thu, 20 Jun 2024 20:58:29 +0000 (13:58 -0700)]
xfs/{018,191,288}: disable parent pointers for these tests

These tests depend heavily on the xattr formats created for new files.
Parent pointers break those assumptions, so force parent pointers off.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoxfs/021: adapt golden output files for parent pointers
Darrick J. Wong [Thu, 20 Jun 2024 20:58:13 +0000 (13:58 -0700)]
xfs/021: adapt golden output files for parent pointers

Parent pointers change the xattr structure dramatically, so fix this
test to handle them.  For the most part we can get away with filtering
out the parent pointer fields (which xfs_db decodes for us), but the
namelen/valuelen/attr_filter fields still show through.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agopopulate: create hardlinks for parent pointers
Darrick J. Wong [Thu, 20 Jun 2024 20:57:58 +0000 (13:57 -0700)]
populate: create hardlinks for parent pointers

Create some hardlinked files so that we can exercise parent pointers.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoxfs/122: update for parent pointers
Darrick J. Wong [Thu, 20 Jun 2024 20:57:42 +0000 (13:57 -0700)]
xfs/122: update for parent pointers

Update test for parent pointers.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoxfs/206: filter out the parent= status from mkfs
Darrick J. Wong [Thu, 20 Jun 2024 20:57:27 +0000 (13:57 -0700)]
xfs/206: filter out the parent= status from mkfs

Filter out the parent pointer bits from the mkfs output so that we don't
cause a regression in this test.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agogeneric: test recovery of extended attribute updates
Darrick J. Wong [Thu, 20 Jun 2024 20:57:11 +0000 (13:57 -0700)]
generic: test recovery of extended attribute updates

Fork generic/475 to test recovery of extended attribute modifications
and log recovery.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoexchangerange: make sure that we don't swap unwritten extents unless they're part...
Darrick J. Wong [Thu, 20 Jun 2024 20:56:55 +0000 (13:56 -0700)]
exchangerange: make sure that we don't swap unwritten extents unless they're part of a rt extent

By default, the FILE1_WRITTEN flag for the EXCHANGERANGE ioctl isn't
supposed to touch anything except for written extents.  In other words,
it shouldn't exchange delalloc reservations, unwritten preallocations,
or holes.  The XFS implementation flushes dirty pagecache to disk so
there should never be delalloc reservations running through the
exchangerange machinery, but there can be unwritten extents.

Hence, write a test to make sure that unwritten extents don't get moved
around.  This test opts itself out for realtime filesystems where the
allocation unit is larger than 1 fsblock because xfs has to move full
allocation units, and that requires exchanging of partially written rt
extents.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoxfs/206: screen out exchange-range from golden output
Darrick J. Wong [Thu, 20 Jun 2024 20:56:40 +0000 (13:56 -0700)]
xfs/206: screen out exchange-range from golden output

Fix this so that exchange-range doesn't trigger test failures.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>
12 months agoxfs/122: fix for exchrange conversion
Darrick J. Wong [Thu, 20 Jun 2024 20:56:24 +0000 (13:56 -0700)]
xfs/122: fix for exchrange conversion

Fix this test for the swapext -> exchrange conversion.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zorro Lang <zlang@kernel.org>