Artem Bityutskiy [Wed, 25 May 2011 15:22:51 +0000 (18:22 +0300)]
fs-tests: integck: make directory checking work
This patch makes the directory checking in case of power cut emulation
actually work. There were many bugs. Basically, we cannot rely on anything
unless the directory is marked as clean.
Artem Bityutskiy [Tue, 24 May 2011 14:46:11 +0000 (17:46 +0300)]
fs-tests: integck: always check clean data
Check the clean data after every emulated power cut, not only when
'create_test_data()'/'update_test_data()' succeed. This required some minor
re-work.
Artem Bityutskiy [Mon, 23 May 2011 11:30:01 +0000 (14:30 +0300)]
fs-tests: integck: EIO can also be return in power-cut testing
Currently when we are doing UBIFS power cut emulation testing we assume that
EROFS is returned in case of an emulated power cut event. However, this is
not true and sometimes Linux returns EIO to user-space instead. Namely, this
happens when we are waiting for write-back to finish. Here is a call trace
form kernel 2.6.39-rc7:
Artem Bityutskiy [Fri, 20 May 2011 15:27:36 +0000 (18:27 +0300)]
fs-tests: integck: implement MTD reattaching
This patch adds -m <mtdnum> option to integck and teaches integck to re-attach
the MTD device to UBI in case of an emulated power cut event. This is needed
for the new UBI power cut and unstable bits emulation infrastructure: when UBI
emulates a power cut the only way to recover form this is to re-attach the MTD
device.
Matthew L. Creech [Wed, 11 May 2011 21:02:28 +0000 (17:02 -0400)]
mkfs.ubifs: add "-F" option for "free-space fixup"
This adds a superblock flag indicating that "free-space fixup" is needed, and
allows it to be set by the user via the "-F" command-line option. The first
time the filesystem is mounted, this flag will trigger a one-time re-mapping of
all LEBs containing free space. This fixes problems seen on some NAND flashes
when a non-UBIFS-aware flash programmer is used.
Artem: add a bit more help text, tweaked the patch a bit.
Thanks to Ben Gardiner <bengardiner@nanometrics.ca> for help.
Signed-off-by: Matthew L. Creech <mlcreech@gmail.com> Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Artem Bityutskiy [Tue, 17 May 2011 07:28:00 +0000 (10:28 +0300)]
fs-tests: integck: read all files in power cut mode
When we are in power cut mode read all files after re-mounting. This will
check that FS recovery worked well and all files are readable after recovery.
Artem Bityutskiy [Mon, 16 May 2011 14:04:49 +0000 (17:04 +0300)]
fs-tests: integck: make -v work -e
Make the verify option -v work with power cut emulation option -e. We can
verify all operations except of writing to the file, because of write-back.
Indeed, even if we write successfully, the pages may still be in the page-cache
and if an emulated power cut happens they will be dropped, and our verification
will fail.
I am not 100% sure it is save to verify non-write operation in other FSes, but
only UBIFS has power cut emulation mode now, so we do not really care about
other FSes at this point.
Artem Bityutskiy [Mon, 16 May 2011 13:57:05 +0000 (16:57 +0300)]
fs-tests: integck: check direntries in power cut mode
Enable directory entry checking in power cut mode. This means that with this
patch integck will check that all directory entries in the file-system were
created by integck.
Artem Bityutskiy [Tue, 3 May 2011 12:02:58 +0000 (15:02 +0300)]
fs-tests: integck: check errno in pvc macro
All errors but write errors have corresponding CHECK(), so if something
gets wrong we fail. But in case of I/O errors we do not fail if an
unexpected error happens, which makes it difficult to trace issues.
Inject the errno checking to the pvc macro to make sure we fail if
errno is not EROFS.
Artem Bityutskiy [Mon, 2 May 2011 15:19:47 +0000 (18:19 +0300)]
fs-tests: integck: fix segfault
Commit 1394a46213e8180e1233ca6d4811e7c77d49b1e0 was a good idea, but it
introduced a segfault - files can be unlinked and 'file->links' is NULL, it is
bad idea to unconditionally dereference it with 'file->links->name'. This patch
introduces a helper function which takes care of the situation with unlinked
files, plus it adds few assertions.
fs-tests: integck: re-structure file_write a little
This is just a preparation - re-structure 'file_write()' to make one "success"
return point - we'll add synchronization stuff there. Also, while on it, make
it delete the file errored in truncation, not only in write. Also, move
the fsync() call to this function because it is common to all write paths.
I observes segfaults in integck test, and unfortunately I do not have the core
file to investigate the problem. But I see one possibility for the test to
segfault - it has unbounded recursion. Limit the maximum recursion depth.
fs-tests: integck: fail loudly on unexpected EROFS
If we are not in power cut testing mode (-p was not used) and we get an
unexpected EROFS, we silently exit. This patch fixes this behavior and make
the test loudly fail instead.
The test sometimes dies due to SIGBUS because a power cut event happens when
we are writing to an mmap()'ed area. SIGBUS is very difficult to ignore, so
fix the issue simply by avoiding doing 'mmap()' operations when in power cut
testing mode.
fs-tests: integck: fix line number in CHECK() output
In 'check_failed()' use the file and line number provided by the CHECK()
macro via the function parameters, rather than using __LINE__ directly.
Otherwise the funtion prints its own line, instead of the line where the
CHECK() failed.
fs-tests: integck: do not run checking when in power cut mode
Do not check the file-system when we are in power cut mode. Currently
we still check the FS if we did not encounter any error, and the test
fails sometimes because meanwhile write-back encounters an error and
drops pages. Disable the checking so far. It should be enabled when
we implement the 'clean' flag for files.
The integck test assumes that if the FS is rootfs then it cannot
re-mount it, otherwise it can. However, this is not true because the
tested FS can be not remountable even if it is not rootfs, e.g., if
'integck' lives on this FS and is executed from it.
This patch improves the integck logic and actually checks wheter it
is possible to re-mount the FS before starting the test.
Also, it makes sure that power cut testing is run only on re-mountable
FS, because we have to be able to clean the "corrupted" state of the
file-system.
-v (verbose) switch is supposed to turn power cut testing error
messages on/off. However, it does not work because the messages
are printed even if -v was not specified. This patch fixes it.
Instead of using rand_r with own seed for operations and rand() with
global state for data it is cleaner to use rand_r with own different
states for these things. This patch eliminates 'srand()' and 'rand()'
usage.
fs-tests: use independent random generators for ops and data
Currently integck uses the same global random generator for everything -
for choosing the operation, generating the data, and for checking. This
makes integck to become stuck sometimes. My guess this is because of
we somehow re-set it back with srand() when checking files.
This patch makes integck use different generators for data and for
choosing operations by using rand_r() with own seed for operations.
I often need to use gdb to debug integck, or sometimes even attach
already running integck. But when it is compiled with optimizations, it
is nearly impossible to use gdb. Introduce 'debug' target to compile
integck without optimizations.
Additionally, to make the stack backtraces work, add a dirty hack to
integck.c to remove all the static keywords - this turns static
functions to non-static and makes them appear in stack backtraces.
fs-tests: integck: add -e option to verify operations
Improve the test and add -e option which forces integck to verify all
operation it performs. This patch does not implement file writes and
holes operations so far - will be done in subsequent patches.
fs-tests: integck: print backtrace in case of failure
This patch teaches the 'CHECK()' macro to print the back-trace in
case of failure which is sometimes very useful for debugging. This
patch also adds a helper function for the 'CHECK()' macro because
otherwise it becomes too large.
Note, since all functions in the test are static, the stack trace
does not really show symbols.
This is a dirty patch - it makes many changes, but I simply had no
time to split it. And because this test is something no one except me
and Adrian cares about, I do not want to spend too much time creating
nice patches. Anyway, this patch make the power cut testing mode work.
Now you can run the test like "./integck /mnt/ubifs -p" and it will
run recover from emulated power cut failures and continue forever,
untill an unexpected issue is met.
fs-tests: integck: do not assume the FS is mounted after failure
If a power cut emulation error happens and we are trying to re-mount
the file-system, do not assume it is mounted because the failure could
have happened after it had had been unmounted but before it was
successfully mounted.
fs-tests: integck: add recovery remounting function
When re-mountig to recover from the emulated power-cut failure then we should
do a bit less fancy things, e.g., we should not try to re-mount to R/O and R/W
modes forth and back, but we should instead just unmount directly.
Additionally, we should not die if mount fails because there may be a power
cut emulation during mounting. Thus, create a separate function for recovering
and try to recover in a loop.
Teach 'add_dir_entry()' to allocate the file/dir/symlink object for
the client. This simplifies client's code. The only place where we
do not want to do this is when we are creating a hardling pointing
to an existing file.
Teach 'remove_dir_entry()' to free the file/dir/symlink object for
the client. Similarly, this simplifies client's code. The only place
where we do not want to do this is the rename function - when we
rename an object we want to only change its direntry, but not the
object itself.
But the main motivation for this change is not just to clean-up, but
to fix another memory leak. In the re-name function we free the
'rename_entry', but we did not free the corresponding object. This
patch fixes the situation.
fs-tests: integck: lessen memory consumption even more
This patch kills the "name" field from 'struct file_info' where we stored
the original file name. We do not really need to keep this name around,
becaus it might have long gone and it is not very interesting. Besides,
we have several memory leaks where we leak 'file->name' strings, so this
is the easiest way to fix those leaks as well.
We do not need to store directory name in the 'struct dir_info' objects,
because we already have it in 'struct dir_entry_info'. So we duplicate
the names for directories in 'dir->name' and 'dir->entry->name'.
Make 'add_dir_entry()' consistent and allocate name for all types,
not only for 'd' and 's'. Also, since 'add_dir_entry()' sets the
parent - do not do this in the calling functions.
This is a clean-up which makes 'add_dir_entry()' more consistent.
When we create a symlink we also allocate a symlink_info structure
and the target path name. But when we remove a symlink - we do not
delete that memory. This patch fixes the issue.
fs-tests: integck: teach dir_remove return error and fix a memory leak
Teach 'dir_remove()' to return an error to the caller if it fails to
remove the directory. Also, there was a memory leak - the 'path'
string was not freed - fix it as well.
fs-tests: integck: make operate_on_dir propagate errors up
Make 'operate_on_dir()' functions propogate errors up if any of
the functions it calls failed. This patch also makes 'operate_on_entry()'
return an error code, but without actual implementation.
fs-tests: integck: check deletion errors in rm_minus_rf_dir
Modify the 'rm_minus_rf_dir()' function to return -1 in case of any
errors during deletions. Make 'integck()' handle the errors. Also
introduce a 'pcv()' function to print error message if -v command
line option was specified.
fs-tests: integck: introduce power cut testing arguments
Introduce new command line arguments for power cut testing:
-p to enable the power cut testing mode and -v to be verbose
about the errors. The real functionality is not implemented
so far.
Remove the final dependency on the common code - the CHECK() macro.
Now we are finally independent. The reason we needed this is that
we are going to improve integck to handle errors more gracefully,
in order to support failure testing better. And this requires changes
in the common code, but it is ancient and shared by many tests, and
we do not want to touch it. So we better make integck independent
and concentrate on integck only.
This patch also removes Makefile dependencies. And while on it, remove
the broken 'tests' target.
Do not use the shared 'tests_random_no()' function but use our own
simple implementation instead. We do not need those rather complex
and difficult to understand computations.
Do not use the shared 'tests_fs_is_rootfs()' function - introduce
'fsinfo.is_rootfs' flag and utilize it instead. This is another step
forward the goal to make integck independent of the common code.
Do not use shared 'tests_clear_dir()' function which removes a directory
tree recursively, but instead use own implementation. This is because
I'm trying to make integck independend on the shared code because I
need this to do further improvements.
projects / mtd-utils.git / log