Jon Derrick [Fri, 3 May 2019 18:28:28 +0000 (12:28 -0600)]
blktests: Use old variable check for Bash <4.2
Bash 4.2 and above supports -v variable checks, which returns true for
set or null. Instead use an older bashism that is compatible with bash
4.1 and earlier but only returns true if the variable is set non-null.
This inherently adds a sanity check in case of null variables.
Signed-off-by: Jon Derrick <jonathan.derrick@intel.com>
Ming Lei [Fri, 26 Apr 2019 02:39:37 +0000 (10:39 +0800)]
block/028: check if T10 verification fails
When T10 verification fails, the error code of BLK_STS_PROTECTION
may not be propagated to user space, see mpage_end_io().
So seems the only reliable way for detecting the failure is to
check dmesg.
Cc: Martin K . Petersen <martin.petersen@oracle.com> Signed-off-by: Ming Lei <ming.lei@redhat.com>
[Omar: use DMESG_FILTER] Signed-off-by: Omar Sandoval <osandov@fb.com>
Ming Lei [Mon, 15 Apr 2019 01:22:29 +0000 (09:22 +0800)]
nvme/012 & 013: avoid extremely slow xfs IO
It is observed that nvme/012 may take ~17 minutes to complete on aarch64,
even worse it may trigger IO timeout on nvme-loop.
Eric and Dave replied that it is because of too small log size on small
disk.
So pass '-l size=32m' to avoid the issue.
With this patch, nvme/012 can be completed in one minute.
Cc: Eric Sandeen <esandeen@redhat.com> Cc: Dave Chinner <dchinner@redhat.com> Cc: "Darrick J. Wong" <darrick.wong@oracle.com> Cc: linux-xfs@vger.kernel.org Signed-off-by: Ming Lei <ming.lei@redhat.com>
Ming Lei [Tue, 5 Mar 2019 01:50:26 +0000 (09:50 +0800)]
blktests: add userspace IO test
Add one test to cover changes on block passthrough IO interface,
such as blk_rq_map_user(), blk_rq_map_user_iov(), blk_rq_unmap_user()
and blk_rq_map_kern().
Dongli Zhang [Thu, 14 Mar 2019 11:45:17 +0000 (19:45 +0800)]
loop/001: verify all partitions are removed
loop/001 does not test whether all partitions are removed successfully
during loop device partition scanning. As a result, the regression
introduced by 0da03cab87e6 ("loop: Fix deadlock when calling
blkdev_reread_part()") can not be detected.
The regression will generate below message in dmesg:
[ 464.414043] __loop_clr_fd: partition scan of loop0 failed (rc=-22)
and leave orphan partitions like below:
- /dev/loop0p1
- /sys/block/loop0/loop0p1
This patch verifies all partitions are removed by checking if there is
/sys/block/loopX/loopXpY left. The expected number of partitions left is 0.
Signed-off-by: Dongli Zhang <dongli.zhang@oracle.com>
[Omar: check devices and sysfs for both before and after] Signed-off-by: Omar Sandoval <osandov@fb.com>
Shin'ichiro Kawasaki [Thu, 14 Mar 2019 03:46:33 +0000 (12:46 +0900)]
zbd/006: Test revalidate during other I/O requests
Since SCSI scanning occurs asynchronously, the kernel function
blk_revalidate_disk_zones() called from sd_revalidate_disk() may be
executed while write I/Os are ongoing. As a result,
blk_revalidate_disk_zones() must not cause write I/O errors by changing
zone related parameters of the device queue unless the disk has changed.
This patch allows checking this behavior and catch regressions such as
fixed by commit ccce20fc7968 ("scsi: sd_zbc: Avoid that resetting a zone
fails sporadically").
To trigger disk revalidate, fio is executed with the --loops option
causing the target device file to be closed and reopen at each loop. The
file close triggers disk revalidate and fio starts issuing I/Os before
revalidate completes, resulting in the desired simultaneous parallel
execution of write I/Os and blk_revalidate_disk_zones().
Also move the _find_first_sequential_zone() helper function from zbd/005
to zbd/rc and reuse it in the new test case for target zone selection.
Shin'ichiro Kawasaki [Tue, 5 Mar 2019 05:24:47 +0000 (14:24 +0900)]
zbd: Change sysfs path for partition devices
zbd/001 and zbd/002 test cases fail for partition devices because of
sysfs path difference between partition devices and their holder
devices. The size parameter in sysfs path is different between the
partition devices and their holder devices. The holder devices have
nr_zones parameter in sysfs but the partition devices do not.
Utilize _test_dev_is_partition() helper function and TEST_DEV_PART_SYSFS
variable to refer correct sysfs size parameter for the partition devices.
Do not refer sysfs nr_zones parameter for the partition devices. Instead,
calculate the expected nr_zones from device capacity and zone size.
Shin'ichiro Kawasaki [Tue, 5 Mar 2019 05:24:46 +0000 (14:24 +0900)]
common: Add _test_dev_is_partition() helper function
To control test conditions unique for partition devices, introduce the
_test_dev_is_partition() helper function. Refer TEST_DEV_PART_SYSFS
variable to tell if the TEST_DEV is a partition device or not.
Shin'ichiro Kawasaki [Tue, 5 Mar 2019 05:24:45 +0000 (14:24 +0900)]
check: Add TEST_DEV_PART_SYSFS variable
When partition devices are specified in TEST_DEV, TEST_DEV_SYSFS
variable points to the sysfs paths of holder devices of the partition
devices (e.g., /sys/block/sda). This sysfs path is different from the
sysfs path of the partition devices (e.g., /sys/block/sda/sda1). For
example, size parameters exist in both the holder device sysfs and
the partition device sysfs with different values.
To allow test cases to access sysfs path of the partition devices,
add TEST_DEV_PART_SYSFS variable. TEST_DEV_SYSFS is set as is to refer
the sysfs path of the holder devices. If the TEST_DEV is not a partition
device, an empty string is set to the TEST_DEV_PART_SYSFS variable.
Rename _find_sysfs_dir() function to _find_sysfs_dirs() and make it find
holder device sysfs as well as the partition device sysfs. The function
obtains the canonical sysfs path, and if the device is a partition
device, the function cuts the last device name in the canonical sysfs
path to obtain the holder device sysfs path.
Instead of trying to log in to each target port from each initiator
port, only log in to the target ports associated with the RDMA HCA
from which the log in request is sent. This avoids that the SRP
initiator driver logs the following error message when e.g.
attempting to log in to interface lo from an Ethernet interface:
"QP creation failed".
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Bart Van Assche [Fri, 22 Feb 2019 22:21:10 +0000 (14:21 -0800)]
tests/srp/rc: Consider all network interfaces
Instead of giving up if a login attempt fails, try all network interfaces.
This patch ensures that login succeeds if the first network interface that
is encountered is down and another network interface is up.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Bart Van Assche [Fri, 22 Feb 2019 22:05:47 +0000 (14:05 -0800)]
tests/srp/rc: Simplify log_in()
Since the 'sysfsdir' variable is identical to the 'd' variable, leave
out 'sysfsdir' and use 'd' instead. This patch does not change any
functionality.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Bart Van Assche [Thu, 21 Feb 2019 23:18:58 +0000 (15:18 -0800)]
nvmeof-mp, srp: Check multipathd version
The srp tests reliably report data corruption with multipathd versions
before 0.7.0 due to a multipathd bug. Hence verify that the multipathd
version is at least 0.7.0.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
zhengbin [Wed, 20 Feb 2019 09:38:06 +0000 (17:38 +0800)]
loop/004: Need to wait for drop caches if block_size is changed
When i test blktests, loop/004 will be fail. The Key test steps
are as follows:
1. losetup -f --show /dev/sda
2. src/loblksize /dev/loop0 4096
step 1 will create /dev/loop0. after that, daemon systemd-udevd
will visit and close /dev/loop0 who will add and delete
i_mapping->nrpages.
PS: the operation of systemd-udevd is in the background.
step 2 will set /dev/loop0 block size, linux kernel function
loop_set_block_size has been changed since commit 5db470e229e2
("loop: drop caches if offset or block_size are changed")
+ if (lo->lo_queue->limits.logical_block_size != arg &&
+ lo->lo_device->bd_inode->i_mapping->nrpages) {
+ err = -EAGAIN;
+ pr_warn("%s: loop%d (%s) has still dirty pages\n",
+ __func__, lo->lo_number, lo->lo_file_name,
+ lo->lo_device->bd_inode->i_mapping->nrpages);
+ goto out_unfreeze;
+ }
--->maybe systemd-udevd just visited /dev/loop0, this leads to failure
Shin'ichiro Kawasaki [Wed, 20 Feb 2019 08:12:28 +0000 (17:12 +0900)]
zbd/004: Add zone condition "Closed" for sequential write required zones
The test case zbd/004 executes write operations for two sequential write
required zones across the zone boundary between them. After the write
operation, the second zone has non-zero write pointer. At that status,
the zone can have not only "Implicit Open" condition but also "Closed"
condition based on zone status management logic of the block zoned device.
Add "Closed" condition to the zone condition check logic in zbd/004. Add
ZONE_COND_CLOSED constant definition in zbd/rc.
Masato Suzuki [Wed, 20 Feb 2019 08:12:27 +0000 (17:12 +0900)]
block/024: Increase I/O time
The total time spent on write I/Os for the 3 dd executions will be at most
1500 * 3 * 0.5ms = 2250ms, that is, 2.25s. Due to system overhead and
timer triggers, a total write I/O time larger than this value is generally
reported through I/O stat. However, for a system with very low overhead
and/or very precise timers, the reported value can be under 2.5s, leading
to a rounded value equal to 2s, and not the 3s expected.
To avoid the test to fail due to this problem, increase the number of write
I/Os executed so that the total exact I/O time exceeds 2.5s, leading to a
rounded value equal to 3s. The change increases the number of I/Os of the
second and third dd calls to 1800, leading to an exact write I/O time of
(1500 + 1800 * 2) * 0.5 ms = 2.55s.
Omar Sandoval [Wed, 13 Feb 2019 20:56:48 +0000 (12:56 -0800)]
travis: streamline config
- Reorder the testing matrix to more or less the order of importance.
- Don't install libc and libstdc++ explicitly; it's not necessary and
for some reason it installs a bunch of other packages.
- We only test on Linux, so get rid of the conditional.
- Don't error out if apt update fails, since that seems to happen every
now and then because some repository changed keys.
Omar Sandoval [Wed, 13 Feb 2019 20:05:51 +0000 (12:05 -0800)]
travis: run make check
We want to run shellcheck and the other sanity checks on all pushes.
Install the latest stable version of shellcheck, as old versions are
known to have false positives.
Bart Van Assche [Wed, 2 Jan 2019 03:14:37 +0000 (19:14 -0800)]
tests/srp/rc: Increase maximum request size
A new feature for the SRP initiator and target drivers in kernel v5.0-rc1
is that both the initiator and target drivers support immediate data.
Since the default maximum immediate data size is 8 KB, increase the target
maximum request size such that the SRP initiator can send write requests
with up to 8 KB of immediate data.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Bart Van Assche [Thu, 13 Dec 2018 21:49:20 +0000 (13:49 -0800)]
common/multipath-over-rdma: Skip teardown if $debug has been set
A convenient way to debug the srp and nvmeof-mp tests scripts is to set
the debug variable in common/multipath-over-rdma. Examining what went
wrong during a test is only possible if teardown() is skipped after a
test failure. Make sure that this is the case.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Bart Van Assche [Mon, 11 Feb 2019 16:24:39 +0000 (08:24 -0800)]
Unbreak the continuous integration build
The Travis environment in which the continous integration build is run
does not have the <linux/blkzoned.h> header file. Make sure that the
zbdioctl.c source file builds fine without that header file.
Masato Suzuki [Mon, 28 Jan 2019 13:14:55 +0000 (22:14 +0900)]
zbd/005: Test write ordering
Run a high queue depth direct sequential write fio job to check that
write requests are not being reordered when the deadline scheduler is
used. This test allowed to catch a bug fixed with commit 80e02039721 "block: mq-deadline: Fix write completion handling".
Masato Suzuki [Mon, 28 Jan 2019 13:14:53 +0000 (22:14 +0900)]
zbd/003: Test sequential zones reset
Test zone reset operation to make sure that the BLKRESETZONE ioctl call
works as expected but also that the zone sector remapping that may be
done for logical devices (partitions or dm-linear devices) is correct.
Masato Suzuki [Mon, 28 Jan 2019 13:14:50 +0000 (22:14 +0900)]
tests: Introduce zbd test group
The zoned block device (zbd) test group is used to gather all tests
specific to zoned block devices (null_blk device with zoned mode enabled,
SMR disks, dm-linear on top of zoned devices, etc). Execution of this group
requires that the kernel be compiled with the block layer
CONFIG_BLK_DEV_ZONED option enabled and also requires the null_blk driver
to have zoned mode support (added in kernel 4.19).
This group rc script implements _fallback_null_blk_zoned() helper function
which initailize a null_blk device with zoned mode. Each of the zbd group
test cases calls it in fallback_device() function. This allows the zbd
group test cases fallback to the null_blk device even if the TEST_DEVS
is empty. With this, all tests scripts can be written by only defining
the test_device() function while allowing operation on both null_blk and
user specified devices.
Shin'ichiro Kawasaki [Mon, 28 Jan 2019 13:14:49 +0000 (22:14 +0900)]
src: Introduce zbdioctl program
zbdioctl implements calls to zoned block devices ioctls that are not
supported currently by sys-utils blkzone utility, namely BLKGETZONESZ
and BLKGETNRZONES.
Shin'ichiro Kawasaki [Mon, 28 Jan 2019 13:14:48 +0000 (22:14 +0900)]
check: Introduce fallback_device() and cleanup_fallback_device()
These optional functions can be defined by a test case script. When
defined and TEST_DEVS is empty, the fallback_device() is executed before
runing the test case. The fallback_device() function intializes a virtual
device to run the test case and return the device to be set in TEST_DEVS.
After running the test case, cleanup_fallback_device() is executed to
clean up the device.
This feature allows to run test cases with test_device() function even if
TEST_DEVS is not set in the config, using virtaul devices such as null_blk.
Define CAN_BE_ZONED=1 in block/005, block/006, block/010, block/011,
block/016, block/017, block/020, block/021 and block/023 as all these
tests should execute without any problem against null_blk with zoned
mode enabled or zoned block devices specified in TEST_DEVS.
Shin'ichiro Kawasaki [Mon, 28 Jan 2019 13:14:46 +0000 (22:14 +0900)]
block/004: Adjust fio conditions for zoned block devices
For a random write pattern to a zoned block device, fio requires --direct=1
and --zonemode=zbd options as well as deadline I/O scheduler to be
specified. Specify these options and set the I/O scheduler if the target
device is a zoned block device. Before doing that, also make sure that the
deadline scheduler is available and that fio supports the zbd zone mode.
Set CAN_BE_ZONED flag to run this test case for zoned block devices.
Shin'ichiro Kawasaki [Mon, 28 Jan 2019 13:14:45 +0000 (22:14 +0900)]
common: Introduce _have_fio_zbd_zonemode() helper function
Fio zbd zone mode is necessary for zoned block devices. Introduce the
helper function _have_fio_zbd_zonemode() to check that the installed
fio version supports the option --zonemode=zbd.
Shin'ichiro Kawasaki [Mon, 28 Jan 2019 13:14:44 +0000 (22:14 +0900)]
config: Introduce RUN_ZONED_TESTS variable and CAN_BE_ZONED flag
To allow running tests using a null_blk device with the zoned mode
disabled (current setup) as well as enabled, introduce the config
the RUN_ZONED_TESTS config variable and the per-test flag CAN_BE_ZONED.
RUN_ZONED_TESTS=1 indicates that tests run against null_blk will be
executed twice, first with null_blk as a regular block device
(RUN_FOR_ZONED=0) and a second time with null_blk set as a zoned block
device (RUN_FOR_ZONED=1). This applies only to tests cases that have the
variable CAN_BE_ZONED set to 1, indicating that the test case applies to
zoned block devices. If CAN_BE_ZONED is not defined by a test case, the
test is executed only with the regular null_blk device.
_init_null_blk is modified to prepare null_blk as a zoned blocked device
if RUN_FOR_ZONED is set and as a regular block device otherwise. To avoid
"modprobe -r null_blk" failures, rmdir calls on all sysfs nullbX
directories is added.
When a zoned block device is specified in TEST_DEVS, failures of test
cases that do not set CAN_BE_ZONED are avoided by automatically skipping
the test. The new helper function _test_dev_is_zoned() is introduced to
implement this.
The use of the RUN_ZONED_TESTS variable requires that the kernel be
compiled with CONFIG_BLK_DEV_ZONED enabled.
Jan Kara [Mon, 21 Jan 2019 12:02:03 +0000 (13:02 +0100)]
loop: Add test for changing capacity when filesystem is mounted
Add test for changing capacity of a loop device when a filesystem with
non-default block size is mounted on it. This is a regression test for
"blockdev: Fix livelocks on loop device".
Signed-off-by: Jan Kara <jack@suse.cz> Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
[Omar: mount under $TMPDIR] Signed-off-by: Omar Sandoval <osandov@fb.com>
Theodore Ts'o [Mon, 7 Jan 2019 21:09:31 +0000 (16:09 -0500)]
src/sg/syzkaller1.c: fix portability problem for syscall(__NR_mmap, ...)
How mmap is mapped to a raw system call varies across different
architectures. On some architectures (such as 32-bit ARM), __NR_mmap
may not exist at all; glibc will use __NR_mmap2 to implement mmap(2).
Syzkaller is using mmap() as a non-portable version of malloc(3), so
it should be safe to use the glibc's mmap wrapper instead of trying to
directly call the system call.
Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Dennis Zhou [Thu, 20 Dec 2018 18:18:26 +0000 (12:18 -0600)]
blktests: add Ming Lei's scsi-stress-remove
This test exposed a race condiiton when shutting down a request_queue
with active IO against it and blkg association for the IOs [1]. The
issue ended up being that while the request_queue will just start
failing requests, blkg destruction sets the q->root_blkg to %NULL. This
caused a NPE. This was fixed in [2].
So to help prevent this from happening again, integrate Ming's test into
blktests so that it can more easily be ran. Here I've ported it to fit
better into the blktests framework.
Dennis Zhou [Thu, 20 Dec 2018 18:18:25 +0000 (12:18 -0600)]
blktests: split out cgroup2 controller and file check
This is a prep patch for a new test that will race blkg association and
request_queue cleanup. As blkg association is a underlying cgroup io
controller feature, we need the ability to check if the controller is
available.
Josef Bacik [Wed, 5 Dec 2018 15:34:03 +0000 (10:34 -0500)]
blktests: add cgroup2 infrastructure
In order to test io.latency and other cgroup related things we need some
supporting helpers to setup and tear down cgroup2. This adds support
for checking that we can even configure cgroup2 things, set them up if
need be, and then add the cleanup stuff to the main cleanup function so
everything is always in a clean state.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
[Omar: split into separate file, fix shellcheck errors, rework
cleanup/exit] Signed-off-by: Omar Sandoval <osandov@fb.com>
Omar Sandoval [Tue, 18 Dec 2018 20:15:56 +0000 (12:15 -0800)]
scsi/006: allow changing cache_type to fail
Some devices don't support all cache types. Allow setting the cache type
to fail with EINVAL. On success, make sure it was changed to the desired
value.
Bart Van Assche [Tue, 27 Nov 2018 20:57:11 +0000 (12:57 -0800)]
common/multipath-over-rdma: Retry unloading rdma_rxe if necessary
If any context, e.g. queued work, holds a reference on an rdma_rxe
device then it can happen that the first unload attempt fails. Try
several times to unload that kernel module if the first unload
attempt fails.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Bart Van Assche [Thu, 1 Nov 2018 15:25:27 +0000 (08:25 -0700)]
src/sg/syzkaller1.c: Fix a 32-bit compiler warning
Avoid that clang reports the following warning when building in 32-bit mode:
sg/syzkaller1.c:405:34: error: implicit conversion from 'unsigned long long' to
'uintptr_t' (aka 'unsigned int') changes value from 18446744073709551615
to 4294967295 [-Werror,-Wconstant-conversion]
0x32ul, 0xfffffffffffffffful, 0x0ul, 0, 0, 0);
^~~~~~~~~~~~~~~~~~~~
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Bart Van Assche [Mon, 29 Oct 2018 21:34:00 +0000 (14:34 -0700)]
src/discontiguous-io: Do not shadow variables
Avoid using variables in an inner scope with the same name as variables in
an outer scope. Enable the -Wshadow compiler flag for C and C++ source
files such that in the future the compiler will complain about shadowing.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Theodore Ts'o [Tue, 30 Oct 2018 14:36:49 +0000 (10:36 -0400)]
Fix build failure for discontiguous-io on 32-bit platforms
Avoid that building with a 32-bit compiler fails as follows:
discontiguous-io.cpp:294:34: error: no matching function for call to 'min(long unsigned int, size_t)'
std::min(4ul, len - i * 4));
^ Signed-off-by: Theodore Ts'o <tytso@mit.edu>
[bvanassche: elaborated commit message] Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Jan Kara [Thu, 18 Oct 2018 10:31:47 +0000 (12:31 +0200)]
loop/006: Add test for oops during backing file verification
Add regression test for patch "block/loop: Use global lock for ioctl()
operation." where we can oops while traversing list of loop devices
backing newly created device.
Signed-off-by: Jan Kara <jack@suse.cz>
[Omar: rename to 006, change description] Signed-off-by: Omar Sandoval <osandov@fb.com>
Jens Axboe [Thu, 25 Oct 2018 20:49:04 +0000 (14:49 -0600)]
blktest: remove instances of null_blk queue_mode=1
This is no longer supported in recent kernels, get rid of any testing of
queue_mode=1. queue_mode=1 tested the legacy IO path, which is going
away completely. As such, there's no point in doing anymore testing with
it.
Add a series of tests for the NVMeOF drivers on top of the dm-mpath
driver. These tests are similar to the tests under tests/srp. Both
tests use the dm-mpath driver for multipath and the loopback
functionality of the rdma_rxe driver. The only difference is that the
nvmeof-mp tests use the NVMeOF initiator and target drivers instead
of the SRP initiator and target drivers.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
projects / users / hch / blktests.git / log