[MTD] UBI: rework of off-line UBI analysis tool

The new version can create a gnuplot graph of the erase count statistics.
It can also extract UBI volumes and single blocks with a preanalysis of
the EC as well as the VID header. It has a manual page too ;-).

Signed-off-by: Frank Haverkamp <haver@vnet.ibm.com>

diff --git a/ubi-utils/Makefile b/ubi-utils/Makefile
index 2776c07001a0b4b6f6959016e4d4786a26870f08..0818a9bef93ee65646ddc5af9f2b2bd61f3b05f2 100644 (file)
--- a/ubi-utils/Makefile
+++ b/ubi-utils/Makefile
@@ -71,7 +71,7 @@ ubigen: ubigen.o libubigen.o crc32.o
 mkbootenv: mkbootenv.o bootenv.o hashmap.o error.o crc32.o
        $(CC) $(LDFLAGS) -o $@ $^
 
-unubi: unubi.o crc32.o
+unubi: unubi.o crc32.o unubi_analyze.o eb_chain.o
        $(CC) $(LDFLAGS) -o $@ $^
 
 pfi2bin: pfi2bin.o peb.o error.o list.o crc32.o libubigen.o bootenv.o \

diff --git a/ubi-utils/doc/unubi.roff b/ubi-utils/doc/unubi.roff
new file mode 100644 (file)
index 0000000..6cebc46
--- /dev/null
+++ b/ubi-utils/doc/unubi.roff
@@ -0,0 +1,123 @@
+.TH UNUBI 1 "NOVEMBER 2006" FSP "FSP Flashutils"
+.SH NAME
+unubi \- extract volumes/eraseblocks from a raw\-UBI image
+.SH SYNOPSIS
+\fBunubi [\-aevEV] [\-d \fIout\-dir\fB] [\-r \fIvolume\-id\fB]
+[\-b \fIblock\-size\fB] \fIimage\-file
+.SH DESCRIPTION
+.PP
+\fBunubi\fR reads an image file containing blocks of UBI headers and data
+(such as produced from \fBnand2bin\fR) and rebuilds the volumes within.
+The default operation (when no flags are given) is to rebuild all valid
+volumes found in the image. \fBunubi\fR can also read straight from the
+onboard MTD device (ex. /dev/mtdblock/NAND).
+.SH OPTIONS
+.IP "\-a, \-\-analyze"
+When flagged, analysis files are generated within the output directory. These
+may include tables and or graphs detailing statistics gathered from the
+eraseblock data. Files are prefixed `analysis_'.
+
+See \fBANALYSIS\fR.
+.IP "\-b, \-\-blocksize \fIblock\-size\fR"
+Specify in bytes the \fIimage\-file\fR eraseblock size. Sizes may be
+postfixed with `KiB' or `MiB' to indicate mebibytes or kibibytes
+respectively. Default is 128KiB.
+.IP "\-d, \-\-dir \fIoutput\-dir\fR"
+Specify the output directory. If no directory is specified, the default
+is `unubi_\fIimage\-file\fR' within the curent working directory. If the
+attempt to create the output directory fails,
+.B unubi
+will try to create it in /tmp before aborting.
+.IP "\-e, \-\-eb\-split"
+When flagged, images are created for each eraseblock in \fIimage\-file\fR
+regardless of its validity. Each image is the complete eraseblock, including
+headers and any space to the end of the eraseblock after where the data may
+end.
+
+Invalid images are named `ebEEEE', where EEEE is the physical index of the
+eraseblock in the image. Valid images are named `ebEEEE_VVV_NNN_RRR' where
+VVV is the known volume ID, NNN is the logical number and RRR is the version
+of the eraseblock data. Note that the version number is in hexadecimal.
+
+Invalid images may also contain this postfix, if the data in the header
+could be valid (ie. the header contains a resonable volume ID, but the
+header and/or data CRCs are not valid). If this is the case, images are named
+`ebEEEE_VVV_NNN_RRR.reason', so as to distinguish known values from
+non\-definite ones.
+
+See \fBREASON SUFFIXES\fR.
+.IP "\-r, \-\-rebuild \fIvolume\-id\fR"
+Specify a volume to rebuild. Can be used successively to specify
+several volumes to be rebuilt.
+
+Images are named `volumeVVV' where VVV is the volume ID. For each missing
+eraseblock, an error message will be printed.
+.IP "\-v, \-\-vol\-split"
+When flagged, images are created for each valid eraseblock in
+\fIimage\-file\fR. Since a vaild eraseblock will have a defined data start and
+data length, only this range will make up the image.
+
+Images are named `volVVV_NNN_RRR_EEEE', where, for the data in the eraseblock,
+VVV is the volume ID, NNN is the logical number, RRR is the version and EEEE
+is the phyisical index of the eraseblock in the image.
+.IP "\-V, \-\-vol\-split!"
+Same as above, only all images are the complete eraseblock (including headers,
+and raw data, even past the point where the data is supposed to end).
+Overrides \-v when both \-v and \-V are flagged.
+.SH ANALYSIS
+The following files will be generated during the analysis:
+.IP "analysis_ec_hdr.data"
+A space delimited table with these two columns for each eraseblock: the
+eraseblock's index or physical position in the image, and the eraseblock's
+erase count. The third column contains the erase count data sorted.
+.IP "analysis_vid_hdr.data"
+A space delimited table with these four colums for each eraseblock: the
+volume ID, the volume logical number, the leb version, and the data size.
+In addition there are a normalized column representing the volume ID and
+volume logical number, a normalized column representing the leb version, and
+a normalized column representing the data_size. These normalized columns are
+used to better draw the the gnuplot image.
+.IP "analysis_ec_hdr.plot"
+A gnuplot script for quickly viewing a sample output from the respective .data
+file.
+.IP "analysis_vid_hdr.plot"
+A gnuplot script for quickly viewing a sample output from the respective .data
+file.
+.SH REASONS SUFFIXES
+When \-\-eb\-split produces possibly invalid, though usable, eraseblocks, the
+known reason suffixes are:
+.IP ".ec_magic"
+The erase counter header did not contain a valid magic field.
+.IP ".ec_hdr_crc"
+The erase counter header did not contain a vaild header CRC field.
+.IP ".vid_magic"
+The volume ID header did not contain a valid magic field.
+.IP ".vid_hdr_crc"
+The volume ID header did not contain a valid header CRC field.
+.IP ".data_crc"
+The volume ID header did not contain a valid data CRC field.
+.SH EXAMPLES
+To extract and rebuild all valid volumes from demo.img (note the output
+directory will be /home/user/unubi_demo.img):
+.sp 1
+.RS
+.B /home/user# unubi demo.img
+.sp 1
+.RE
+To analyze demo.img as well as extract and rebuild volume 7:
+.sp 1
+.RS
+.B /home/user# unubi \-a \-r 7 demo.img
+.sp 1
+.RE
+To split demo.img into raw images for each eraseblock into the folder
+/var/eraseblocks:
+.sp 1
+.RS
+.B /home/user# unubi \-e \-d /var/eraseblocks demo.img
+.SH AUTHORS
+Frank Haverkamp <haver@vnet.ibm.com>
+.sp 0
+Drake Dowsett <dowsett@de.ibm.com>
+.SH CONTACT
+Andreas Arnez <arnez@de.ibm.com>

diff --git a/ubi-utils/src/eb_chain.c b/ubi-utils/src/eb_chain.c
new file mode 100644 (file)
index 0000000..501a838
--- /dev/null
+++ b/ubi-utils/src/eb_chain.c
@@ -0,0 +1,287 @@
+/*
+ * Copyright (c) International Business Machines Corp., 2006
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
+ * the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+/*
+ * Author:  Drake Dowsett, dowsett@de.ibm.com
+ * Contact: Andreas Arnez, arnez@de.ibm.com
+ */
+
+/* see eb_chain.h */
+
+#include <errno.h>
+#include <stdlib.h>
+#include <string.h>
+#include "eb_chain.h"
+
+#define COPY(dst, src)                                                 \
+       do                                                              \
+       {                                                               \
+               dst = malloc(sizeof(*dst));                             \
+               if (dst == NULL)                                        \
+                       return -ENOMEM;                                 \
+               memcpy(dst, src, sizeof(*dst));                         \
+       } while (0)
+
+
+/**
+ * inserts an eb_info into the chain starting at head, then searching
+ * linearly for the correct position;
+ * new should contain valid vid and ec headers and the data_crc should
+ * already have been checked before insertion, otherwise the chain
+ * could be have un an undesired manner;
+ * returns -ENOMEM if alloc fails, otherwise SHOULD always return 0,
+ * if not, the code reached the last line and returned -EAGAIN,
+ * meaning there is a bug or a case not being handled here;
+ **/
+int
+eb_chain_insert(eb_info_t *head, eb_info_t new)
+{
+       uint32_t vol, num, ver;
+       uint32_t new_vol, new_num, new_ver;
+       eb_info_t prev, cur, hist, ins;
+       eb_info_t *prev_ptr;
+
+       if ((head == NULL) || (new == NULL))
+               return 0;
+
+       if (*head == NULL)
+       {
+               COPY(*head, new);
+               (*head)->next = NULL;
+               return 0;
+       }
+
+       new_vol = ubi32_to_cpu(new->inner.vol_id);
+       new_num = ubi32_to_cpu(new->inner.lnum);
+       new_ver = ubi32_to_cpu(new->inner.leb_ver);
+
+       /** TRAVERSE HORIZONTALY **/
+
+       cur = *head;
+       prev = NULL;
+
+       /* traverse until vol_id/lnum align */
+       vol = ubi32_to_cpu(cur->inner.vol_id);
+       num = ubi32_to_cpu(cur->inner.lnum);
+       while ((new_vol > vol) || ((new_vol == vol) && (new_num > num)))
+       {
+               /* insert new at end of chain */
+               if (cur->next == NULL)
+               {
+                       COPY(ins, new);
+                       ins->next = NULL;
+                       cur->next = ins;
+                       return 0;
+               }
+
+               prev = cur;
+               cur = cur->next;
+               vol = ubi32_to_cpu(cur->inner.vol_id);
+               num = ubi32_to_cpu(cur->inner.lnum);
+       }
+
+       if (prev == NULL)
+               prev_ptr = head;
+       else
+               prev_ptr = &(prev->next);
+
+       /* insert new into the middle of chain */
+       if ((new_vol != vol) || (new_num != num))
+       {
+               COPY(ins, new);
+               ins->next = cur;
+               *prev_ptr = ins;
+               return 0;
+       }
+
+       /** TRAVERSE VERTICALY **/
+
+       hist = cur;
+       prev = NULL;
+
+       /* traverse until versions align */
+       ver = ubi32_to_cpu(cur->inner.leb_ver);
+       while (new_ver < ver)
+       {
+               /* insert new at bottom of history */
+               if (hist->older == NULL)
+               {
+                       COPY(ins, new);
+                       ins->next = NULL;
+                       ins->older = NULL;
+                       hist->older = ins;
+                       return 0;
+               }
+
+               prev = hist;
+               hist = hist->older;
+               ver = ubi32_to_cpu(hist->inner.leb_ver);
+       }
+
+       if (prev == NULL)
+       {
+               /* replace active version */
+               COPY(ins, new);
+               ins->next = hist->next;
+               *prev_ptr = ins;
+
+               /* place cur in vertical histroy */
+               ins->older = hist;
+               hist->next = NULL;
+               return 0;
+       }
+       else
+       {
+               /* insert between versions, beneath active version */
+               COPY(ins, new);
+               ins->next = NULL;
+               ins->older = prev->older;
+               prev->older = ins;
+               return 0;
+       }
+
+       /* logically impossible to reach this point... hopefully */
+       return -EAGAIN;
+}
+
+
+/**
+ * sets the pointer at pos to the position of the first entry in the chain
+ * with of vol_id and, if given, with the same lnum as *lnum;
+ * if there is no entry in the chain, then *pos is NULL on return;
+ * always returns 0;
+ **/
+int
+eb_chain_position(eb_info_t *head, uint32_t vol_id, uint32_t *lnum,
+                 eb_info_t *pos)
+{
+       uint32_t vol, num;
+       eb_info_t cur;
+
+       if ((head == NULL) || (*head == NULL) || (pos == NULL))
+               return 0;
+
+       *pos = NULL;
+
+       cur = *head;
+       while (cur != NULL)
+       {
+               vol = ubi32_to_cpu(cur->inner.vol_id);
+               num = ubi32_to_cpu(cur->inner.lnum);
+
+               if (vol_id == vol)
+                       if ((lnum == NULL) || (*lnum == num))
+                       {
+                               *pos = cur;
+                               return 0;
+                       }
+
+               cur = cur->next;
+       }
+
+       return 0;
+}
+
+
+/**
+ * prints to stream, the vol_id, lnum and leb_ver for each entry in the
+ * chain, starting at head;
+ * this is intended for debuging purposes;
+ * always returns 0;
+ **/
+int
+eb_chain_print(FILE* stream, eb_info_t *head)
+{
+       eb_info_t cur;
+
+       if (head == NULL)
+               return 0;
+
+       if (stream == NULL)
+               stream = stdout;
+
+       if (*head == NULL)
+       {
+               fprintf(stream, "EMPTY\n");
+               return 0;
+       }
+
+       cur = *head;
+       while (cur != NULL)
+       {
+               eb_info_t hist;
+
+               fprintf(stream, "  VOL %4u-%04u | VER 0x%8x\n",
+                       ubi32_to_cpu(cur->inner.vol_id),
+                       ubi32_to_cpu(cur->inner.lnum),
+                       ubi32_to_cpu(cur->inner.leb_ver));
+
+               hist = cur->older;
+               while (hist != NULL)
+               {
+                       fprintf(stream, "+ VOL %4u-%04u | VER 0x%8x\n",
+                               ubi32_to_cpu(hist->inner.vol_id),
+                               ubi32_to_cpu(hist->inner.lnum),
+                               ubi32_to_cpu(hist->inner.leb_ver));
+
+                       hist = hist->older;
+               }
+
+               cur = cur->next;
+       }
+
+       return 0;
+}
+
+
+/**
+ * frees the memory of the entire chain, starting at head;
+ * head will be NULL on return;
+ * always returns 0;
+ **/
+int
+eb_chain_destroy(eb_info_t *head)
+{
+       if (head == NULL)
+               return 0;
+
+       while (*head != NULL)
+       {
+               eb_info_t cur;
+               eb_info_t hist;
+
+               cur = *head;
+               *head = (*head)->next;
+
+               hist = cur->older;
+               while (hist != NULL)
+               {
+                       eb_info_t temp;
+
+                       temp = hist;
+                       hist = hist->older;
+
+                       free(temp);
+               }
+
+               free(cur);
+       }
+
+       return 0;
+}
+

diff --git a/ubi-utils/src/eb_chain.h b/ubi-utils/src/eb_chain.h
new file mode 100644 (file)
index 0000000..f640c54
--- /dev/null
+++ b/ubi-utils/src/eb_chain.h
@@ -0,0 +1,73 @@
+#ifndef __EB_CHAIN_H__
+#define __EB_CHAIN_H__
+
+/*
+ * Copyright (c) International Business Machines Corp., 2006
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
+ * the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+/*
+ * Author:  Drake Dowsett
+ * Contact: Andreas Arnez (arnez@de.ibm.com)
+ *
+ * Eraseblock Chain
+ *
+ * A linked list structure to order eraseblocks by volume and logical number
+ * and to update by version number. Doesn't contain actual eraseblock data
+ * but rather the erasecounter and volume id headers as well as a position
+ * indicator.
+ *
+ * Diagram Example:
+ *
+ * [V1.0v0]->[V1.1v2]->[V1.2v1]->[V2.0v2]->[V2.1v0]->[V2.2v1]->NULL
+ *     |         |         |         |         |         |
+ *   NULL    [V1.1v1]  [V1.2v0]  [V2.0v1]    NULL    [V2.2v0]
+ *               |         |         |                   |
+ *           [V1.1v0]    NULL    [V2.0v0]              NULL
+ *               |                   |
+ *             NULL                NULL
+ *
+ * [VA.BvC] represents the eb_info for the eraseblock with the vol_id A,
+ * lnum B and leb_ver C
+ * -> represents the `next' pointer
+ * | represents the `older' pointer
+ */
+
+#include <stdio.h>
+#include <stdint.h>
+#include <mtd/ubi-header.h>
+
+typedef struct eb_info *eb_info_t;
+struct eb_info {
+       struct ubi_ec_hdr outer;
+       struct ubi_vid_hdr inner;
+       fpos_t eb_top;
+       uint32_t linear;
+
+       eb_info_t next;
+       eb_info_t older;
+};
+
+int eb_chain_insert(eb_info_t *head, eb_info_t item);
+
+int eb_chain_position(eb_info_t *head, uint32_t vol_id, uint32_t *lnum,
+                     eb_info_t *pos);
+
+int eb_chain_print(FILE *stream, eb_info_t *head);
+
+int eb_chain_destroy(eb_info_t *head);
+
+#endif /* __EB_CHAIN_H__ */

diff --git a/ubi-utils/src/unubi.c b/ubi-utils/src/unubi.c
index 6d877e70f52d64748084fec40b4d09d6ea6ac534..4b9ddfd06ab1652039df0577b515d4135f1a1120 100644 (file)
--- a/ubi-utils/src/unubi.c
+++ b/ubi-utils/src/unubi.c
@@ -14,13 +14,23 @@
  * You should have received a copy of the GNU General Public License
  * along with this program; if not, write to the Free Software
  * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
- *
- * Author: Frank Haverkamp
- *
- * An utility to decompose UBI images. Not yet finished ...
  */
 
-#include <config.h>
+/*
+ * Authors: Frank Haverkamp, haver@vnet.ibm.com
+ *         Drake Dowsett, dowsett@de.ibm.com
+ */
+
+/*
+ * unubi  reads  an  image  file containing blocks of UBI headers and data
+ * (such as produced from nand2bin) and rebuilds the volumes within.   The
+ * default  operation  (when  no  flags are given) is to rebuild all valid
+ * volumes found in the image. unubi  can  also  read  straight  from  the
+ * onboard MTD device (ex. /dev/mtdblock/NAND).
+ */
+
+/* TODO: consideration for dynamic vs. static volumes */
+
 #include <argp.h>
 #include <errno.h>
 #include <fcntl.h>
@@ -34,359 +44,802 @@
 #include <sys/ioctl.h>
 #include <sys/stat.h>
 #include <sys/types.h>
+#include <mtd/ubi-header.h>
 
 #include "crc32.h"
-#include <mtd/ubi-header.h>
+#include "eb_chain.h"
+#include "unubi_analyze.h"
+
+#define EXEC           "unubi"
+#define CONTACT                "haver@vnet.ibm.com"
+#define VERSION                "0.9"
+
+static char doc[] = "\nVersion: " VERSION "\n\t"
+       BUILD_OS" "BUILD_CPU" at "__DATE__" "__TIME__"\n"
+       "\nAnalyze raw flash containing UBI data.\n";
+
+#define ERR_MSG(fmt...)                                                        \
+       fprintf(stderr, EXEC ": " fmt)
 
-#define MAXPATH                1024
-#define MIN(x,y)       ((x)<(y)?(x):(y))
+#define SPLIT_DATA     1
+#define SPLIT_RAW      2
+
+#define DIR_FMT                "unubi_%s"
+#define KIB            1024
+#define MIB            (KIB * KIB)
+#define MAXPATH                KIB
+
+/* filenames */
+#define FN_INVAL       "%s/eb%04u%s"                   /* invalid eraseblock */
+#define FN_NSURE       "%s/eb%04u_%03u_%03u_%03x%s"    /* unsure eraseblock */
+#define FN_VALID       "%s/eb%04u_%03u_%03u_%03x%s"    /* valid eraseblock */
+#define FN_VOLSP       "%s/vol%03u_%03u_%03u_%04u"     /* split volume */
+#define FN_VOLWH       "%s/volume%03u"                 /* whole volume */
 
 static uint32_t crc32_table[256];
 
+/* struct args:
+ *     bsize           int, blocksize of image blocks
+ *     analyze         flag, when non-zero produce analysis
+ *     eb_split        flag, when non-zero output eb####
+ *                     note: SPLIT_DATA vs. SPLIT_RAW
+ *     vol_split       flag, when non-zero output vol###_####
+ *                     note: SPLIT_DATA vs. SPLIT_RAW
+ *     odir_path       string, directory to place volumes in
+ *     img_path        string, file to read as ubi image
+ *     vols            int array of size UBI_MAX_VOLUMES, where a 1 can be
+ *                     written for each --rebuild flag in the index specified
+ *                     then the array can be counted and collapsed using
+ *                     count_set() and collapse()
+ */
 struct args {
-       const char *output_dir;
-       uint32_t hdr_offs;
-       uint32_t data_offs;
-       uint32_t blocksize;
-
-       /* special stuff needed to get additional arguments */
-       char *arg1;
-       char **options;         /* [STRING...] */
+       uint32_t bsize;
+       int analyze;
+       int eb_split;
+       int vol_split;
+       char *odir_path;
+       char *img_path;
+       uint32_t *vols;
+
+       char **options;
 };
 
-static struct args myargs = {
-       .output_dir = "unubi",
-       .hdr_offs = 64,
-       .data_offs = 128,
-       .blocksize = 128 * 1024,
-       .arg1 = NULL,
-       .options = NULL,
-};
+static error_t parse_opt(int key, char *arg, struct argp_state *state);
 
-static error_t parse_opt (int key, char *arg, struct argp_state *state);
-
-const char *argp_program_bug_address =
-       "... uuuh, lets wait until it looks nicer";
-
-static char doc[] = "\nVersion: " PACKAGE_VERSION "\n\t"
-       BUILD_OS" "BUILD_CPU" at "__DATE__" "__TIME__"\n"
-       "\nWrite to UBI Volume.\n";
+const char *argp_program_version = VERSION;
+const char *argp_program_bug_address = CONTACT;
 
 static struct argp_option options[] = {
-       { .name = "dir",
-         .key = 'd',
-         .arg = "<output-dir>",
-         .flags = 0,
-         .doc = "output directory",
-         .group = OPTION_ARG_OPTIONAL },
-
-       { .name = "blocksize",
-         .key = 'b',
-         .arg = "<blocksize>",
-         .flags = 0,
-         .doc = "blocksize",
-         .group = OPTION_ARG_OPTIONAL },
-
-       { .name = "data-offs",
-         .key = 'x',
-         .arg = "<data-offs>",
-         .flags = 0,
-         .doc = "data offset",
-         .group = OPTION_ARG_OPTIONAL },
-
-       { .name = "hdr-offs",
-         .key = 'x',
-         .arg = "<hdr-offs>",
-         .flags = 0,
-         .doc = "hdr offset",
-         .group = OPTION_ARG_OPTIONAL },
-
-       { .name = NULL, .key = 0, .arg = NULL, .flags = 0,
-         .doc = NULL, .group = 0 },
+       {
+               name: NULL, key: 0, arg: NULL,
+               flags: 0, group: 0, doc: "OPTIONS",
+       },
+       {
+               name: "rebuild", key: 'r', arg: "<volume-id>",
+               flags: 0, group: 1, doc: "Extract and rebuild volume",
+       },
+       {
+               name: "dir", key: 'd', arg: "<output-dir>",
+               flags: 0, group: 2, doc: "Specify output directory",
+       },
+       {
+               name: "analyze", key: 'a', arg: NULL,
+               flags: 0, group: 3, doc: "Analyze image",
+       },
+       {
+               name: "blocksize", key: 'b', arg: "<block-size>",
+               flags: 0, group: 4, doc: "Specify size of eraseblocks "
+                                        "in image in bytes (default "
+                                        "128KiB)",
+       },
+       {
+               name: "eb-split", key: 'e', arg: NULL,
+               flags: 0, group: 5, doc: "Generate individual eraseblock "
+                                        "images (all eraseblocks)",
+       },
+       {
+               name: "vol-split", key: 'v', arg: NULL,
+               flags: 0, group: 5, doc: "Generate individual eraseblock "
+                                        "images (valid eraseblocks only)",
+       },
+       {
+               name: "vol-split!", key: 'V', arg: NULL,
+               flags: 0, group: 5, doc: "Raw split by eraseblock "
+                                        "(valid eraseblocks only)",
+       },
+       {
+               name: NULL, key: 0, arg: NULL,
+               flags: 0, group: 0, doc: NULL,
+       },
 };
 
 static struct argp argp = {
        .options = options,
        .parser = parse_opt,
        .args_doc = "image-file",
-       .doc =  doc,
+       .doc = doc,
        .children = NULL,
        .help_filter = NULL,
        .argp_domain = NULL,
 };
 
-/*
- * str_to_num - Convert string into number and cope with endings like
- *              k, K, kib, KiB for kilobyte
- *              m, M, mib, MiB for megabyte
- */
-uint32_t str_to_num(char *str)
+
+/**
+ * parses out a numerical value from a string of numbers followed by:
+ *     k, K, kib, KiB for kibibyte
+ *     m, M, mib, MiB for mebibyte
+ **/
+static uint32_t
+str_to_num(char *str)
 {
-       char *s = str;
-       ulong num = strtoul(s, &s, 0);
+       char *s;
+       ulong num;
+
+       s = str;
+       num = strtoul(s, &s, 0);
 
        if (*s != '\0') {
-               if (strcmp(s, "KiB") == 0)
-                       num *= 1024;
-               else if (strcmp(s, "MiB") == 0)
-                       num *= 1024*1024;
-               else {
-                       fprintf(stderr, "WARNING: Wrong number format "
-                               "\"%s\", check your paramters!\n", str);
-               }
+               if ((strcmp(s, "KiB") == 0) || (strcmp(s, "K") == 0) ||
+                   (strcmp(s, "kib") == 0) || (strcmp(s, "k") == 0))
+                       num *= KIB;
+               if ((strcmp(s, "MiB") == 0) || (strcmp(s, "M") == 0) ||
+                   (strcmp(s, "mib") == 0) || (strcmp(s, "m") == 0))
+                       num *= MIB;
+               else
+                       ERR_MSG("couldn't parse '%s', assuming %lu\n",
+                               s, num);
        }
        return num;
 }
 
-/*
- * @brief Parse the arguments passed into the test case.
- *
- * @param key            The parameter.
- * @param arg            Argument passed to parameter.
- * @param state          Location to put information on parameters.
- *
- * @return error
- *
- * Get the `input' argument from `argp_parse', which we know is a
- * pointer to our arguments structure.
- */
+
+/**
+ * parses the arguments passed into the program
+ * get the input argument from argp_parse, which we know is a
+ * pointer to our arguments structure;
+ **/
 static error_t
 parse_opt(int key, char *arg, struct argp_state *state)
 {
+       uint32_t i;
        struct args *args = state->input;
 
        switch (key) {
-       case 'b': /* --blocksize<blocksize> */
-               args->blocksize = str_to_num(arg);
+       case 'a':
+               args->analyze = 1;
                break;
-
-       case 'x': /* --data-offs=<data-offs> */
-               args->data_offs = str_to_num(arg);
+       case 'b':
+               args->bsize = str_to_num(arg);
                break;
-
-       case 'y': /* --hdr-offs=<hdr-offs> */
-               args->hdr_offs = str_to_num(arg);
+       case 'd':
+               args->odir_path = arg;
                break;
-
-       case 'd': /* --dir=<output-dir> */
-               args->output_dir = arg;
+       case 'e':
+               args->eb_split = SPLIT_RAW;
+               break;
+       case 'r':
+               i = str_to_num(arg);
+               if (i < UBI_MAX_VOLUMES)
+                       args->vols[str_to_num(arg)] = 1;
+               else {
+                       ERR_MSG("volume-id out of bounds\n");
+                       return ARGP_ERR_UNKNOWN;
+               }
+               break;
+       case 'v':
+               if (args->vol_split != SPLIT_RAW)
+                       args->vol_split = SPLIT_DATA;
+               break;
+       case 'V':
+               args->vol_split = SPLIT_RAW;
                break;
-
        case ARGP_KEY_NO_ARGS:
-               /* argp_usage(state); */
                break;
-
        case ARGP_KEY_ARG:
-               args->arg1 = arg;
-               /* Now we consume all the rest of the arguments.
-                   `state->next' is the index in `state->argv' of the
-                   next argument to be parsed, which is the first STRING
-                   we're interested in, so we can just use
-                   `&state->argv[state->next]' as the value for
-                   arguments->strings.
-
-                   _In addition_, by setting `state->next' to the end
-                   of the arguments, we can force argp to stop parsing
-                   here and return. */
-
+               args->img_path = arg;
                args->options = &state->argv[state->next];
                state->next = state->argc;
                break;
-
        case ARGP_KEY_END:
-               /* argp_usage(state); */
                break;
-
        default:
-               return(ARGP_ERR_UNKNOWN);
+               return ARGP_ERR_UNKNOWN;
        }
 
        return 0;
 }
 
-static inline void
-hexdump(FILE *fp, const void *p, ssize_t size)
+
+/**
+ * counts the number of indicies which are flagged in full_array;
+ * full_array is an array of flags (1/0);
+ **/
+static size_t
+count_set(uint32_t *full_array, size_t full_len)
 {
-       int k;
-       const uint8_t *buf = p;
+       size_t count, i;
 
-       for (k = 0; k < size; k++) {
-               fprintf(fp, "%02x ", buf[k]);
-               if ((k & 15) == 15)
-                       fprintf(fp, "\n");
-       }
+       if (full_array == NULL)
+               return 0;
+
+       for (i = 0, count = 0; i < full_len; i++)
+               if (full_array[i] != 0)
+                       count++;
+
+       return count;
 }
 
-/*
- * This was put together in 1.5 hours and this is exactly how it looks
- * like! FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME
- * FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME
- * FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME
- * FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME
- * FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME
- * FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME
- * FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME
- * FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME
- * FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME
- * FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME
- * FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME FIXME
- * FIXME FIXME FIXME FIXME FIXME FIXME!
- */
-static int extract_volume(struct args *args, const uint8_t *buf,
-                         int len, int volume, FILE *fp)
+
+/**
+ * generates coll_array from full_array;
+ * full_array is an array of flags (1/0);
+ * coll_array is an array of the indicies in full_array which are flagged (1);
+ **/
+static size_t
+collapse(uint32_t *full_array, size_t full_len,
+        uint32_t *coll_array, size_t coll_len)
 {
-       int i, rc;
-       int nrblocks = len / args->blocksize;
-       int max_lnum = -1, lnum = 0;
-       const uint8_t *ptr;
-       uint8_t **vol_tab;
-       int *vol_len;
+       size_t i, j;
 
-       vol_tab = calloc(nrblocks, sizeof(uint8_t *));
-       vol_len = calloc(nrblocks, sizeof(int));
+       if ((full_array == NULL) || (coll_array == NULL))
+               return 0;
 
-       if (!buf || !vol_tab || !vol_len)
-               exit(EXIT_FAILURE);
+       for (i = 0, j = 0; (i < full_len) && (j < coll_len); i++)
+               if (full_array[i] != 0) {
+                       coll_array[j] = i;
+                       j++;
+               }
 
-       for (i = 0, ptr = buf; i < nrblocks; i++, ptr += args->blocksize) {
-               uint32_t crc;
-               struct ubi_ec_hdr *ec_hdr = (struct ubi_ec_hdr *)ptr;
-               struct ubi_vid_hdr *vid_hdr = NULL;
-               uint8_t *data;
+       return j;
+}
+
+
+/**
+ * header_crc: calculate the crc of EITHER a eb_hdr or vid_hdr
+ * one of the first to args MUST be NULL, the other is the header
+ *     to caculate the crc on
+ * always returns 0
+ **/
+static int
+header_crc(struct ubi_ec_hdr *ebh, struct ubi_vid_hdr *vidh, uint32_t *ret_crc)
+{
+       uint32_t crc = UBI_CRC32_INIT;
+
+       if (ret_crc == NULL)
+               return 0;
+
+       if ((ebh != NULL) && (vidh == NULL))
+               crc = clc_crc32(crc32_table, crc, ebh, UBI_EC_HDR_SIZE_CRC);
+       else if ((ebh == NULL) && (vidh != NULL))
+               crc = clc_crc32(crc32_table, crc, vidh, UBI_VID_HDR_SIZE_CRC);
+       else
+               return 0;
+
+       *ret_crc = crc;
+       return 0;
+}
 
-               /* default */
-               vol_len[lnum] = args->blocksize - (2 * 1024);
 
-               /* Check UBI EC header */
-               crc = clc_crc32(crc32_table, UBI_CRC32_INIT, ec_hdr,
-                               UBI_EC_HDR_SIZE_CRC);
-               if (crc != ubi32_to_cpu(ec_hdr->hdr_crc))
-                       continue;
+/**
+ * data_crc: save the FILE* position, calculate the crc over a span,
+ *     reset the position
+ * returns non-zero when EOF encountered
+ **/
+static int
+data_crc(FILE* fpin, size_t length, uint32_t *ret_crc)
+{
+       int rc;
+       size_t i;
+       char buf[length];
+       uint32_t crc;
+       fpos_t start;
+
+       rc = fgetpos(fpin, &start);
+       if (rc < 0)
+               return -1;
+
+       for (i = 0; i < length; i++) {
+               int c = fgetc(fpin);
+               if (c == EOF) {
+                       ERR_MSG("unexpected EOF\n");
+                       return -1;
+               }
+               buf[i] = (char)c;
+       }
 
-               vid_hdr = (struct ubi_vid_hdr *)
-                       (ptr + ubi32_to_cpu(ec_hdr->vid_hdr_offset));
-               data = (uint8_t *)(ptr + ubi32_to_cpu(ec_hdr->data_offset));
+       rc = fsetpos(fpin, &start);
+       if (rc < 0)
+               return -1;
 
-               crc = clc_crc32(crc32_table, UBI_CRC32_INIT, vid_hdr,
-                               UBI_VID_HDR_SIZE_CRC);
-               if (crc != ubi32_to_cpu(vid_hdr->hdr_crc))
-                       continue;
+       crc = clc_crc32(crc32_table, UBI_CRC32_INIT, buf, length);
+       *ret_crc = crc;
+       return 0;
+}
+
+
+/**
+ * reads data of size len from fpin and writes it to path
+ **/
+static int
+extract_data(FILE* fpin, size_t len, const char *path)
+{
+       int rc;
+       size_t i;
+       FILE* fpout;
+
+       rc = 0;
+       fpout = NULL;
+
+       fpout = fopen(path, "wb");
+       if (fpout == NULL) {
+               ERR_MSG("couldn't open file for writing: %s\n", path);
+               rc = -1;
+               goto err;
+       }
 
-               if (volume == (int)ubi32_to_cpu(vid_hdr->vol_id)) {
+       for (i = 0; i < len; i++) {
+               int c = fgetc(fpin);
+               if (c == EOF) {
+                       ERR_MSG("unexpected EOF while writing: %s\n", path);
+                       rc = -2;
+                       goto err;
+               }
+               c = fputc(c, fpout);
+               if (c == EOF) {
+                       ERR_MSG("couldn't write: %s\n", path);
+                       rc = -3;
+                       goto err;
+               }
+       }
 
-                       printf("****** block %4d volume %2d **********\n",
-                              i, volume);
+ err:
+       if (fpout != NULL)
+               fclose(fpout);
+       return rc;
+}
 
-                       hexdump(stdout, ptr, 64);
 
-                       printf("--- vid_hdr\n");
-                       hexdump(stdout, vid_hdr, 64);
+/**
+ * using eb chain, tries to rebuild the data of volume at vol_id, or for all
+ * the known volumes, if vol_id is NULL;
+ **/
+static int
+rebuild_volume(FILE* fpin, uint32_t *vol_id, eb_info_t *head, const char* path)
+{
+       char filename[MAXPATH];
+       int rc;
+       uint32_t vol, num;
+       FILE* fpout;
+       eb_info_t cur;
 
-                       printf("--- data\n");
-                       hexdump(stdout, data, 64);
+       rc = 0;
 
-                       lnum = ubi32_to_cpu(vid_hdr->lnum);
-                       vol_tab[lnum] = data;
-                       if (max_lnum < lnum)
-                               max_lnum = lnum;
-                       if (vid_hdr->vol_type == UBI_VID_STATIC)
-                               vol_len[lnum] =
-                                       ubi32_to_cpu(vid_hdr->data_size);
+       if ((fpin == NULL) || (head == NULL) || (*head == NULL))
+               return 0;
 
+       /* when vol_id is null, then do all  */
+       if (vol_id == NULL) {
+               cur = *head;
+               vol = ubi32_to_cpu(cur->inner.vol_id);
+       }
+       else {
+               vol = *vol_id;
+               eb_chain_position(head, vol, NULL, &cur);
+               if (cur == NULL) {
+                       ERR_MSG("no valid volume %d was found\n", vol);
+                       return -1;
                }
        }
 
-       for (lnum = 0; lnum <= max_lnum; lnum++) {
-               if (vol_tab[lnum]) {
-                       rc = fwrite(vol_tab[lnum], 1, vol_len[lnum], fp);
-                       if (ferror(fp) || (vol_len[lnum] != rc)) {
-                               perror("could not write file");
-                               exit(EXIT_FAILURE);
+       num = 0;
+       snprintf(filename, MAXPATH, FN_VOLWH, path, vol);
+       fpout = fopen(filename, "wb");
+       if (fpout == NULL) {
+               ERR_MSG("couldn't open file for writing: %s\n", filename);
+               return -1;
+       }
+
+       while (cur != NULL) {
+               size_t i;
+
+               if (ubi32_to_cpu(cur->inner.vol_id) != vol) {
+                       /* close out file */
+                       fclose(fpout);
+
+                       /* only stay around if that was the only volume */
+                       if (vol_id != NULL)
+                               goto out;
+
+                       /* begin with next */
+                       vol = ubi32_to_cpu(cur->inner.vol_id);
+                       num = 0;
+                       snprintf(filename, MAXPATH, FN_VOLWH, path, vol);
+                       fpout = fopen(filename, "wb");
+                       if (fpout == NULL) {
+                               ERR_MSG("couldn't open file for writing: %s\n",
+                                       filename);
+                               return -1;
                        }
-               } else {
-                       /* Fill up empty areas by 0xff, for static
-                        * volumes this means they are broken!
-                        */
-                       for (i = 0; i < vol_len[lnum]; i++) {
-                               if (fputc(0xff, fp) == EOF) {
-                                       perror("could not write char");
-                                       exit(EXIT_FAILURE);
-                               }
+               }
+
+               if (ubi32_to_cpu(cur->inner.lnum) != num) {
+                       ERR_MSG("missing valid block %d for volume %d\n",
+                               num, vol);
+               }
+
+               rc = fsetpos(fpin, &(cur->eb_top));
+               if (rc < 0)
+                       goto out;
+               fseek(fpin, ubi32_to_cpu(cur->outer.data_offset), SEEK_CUR);
+
+               for (i = 0; i < ubi32_to_cpu(cur->inner.data_size); i++) {
+                       int c = fgetc(fpin);
+                       if (c == EOF) {
+                               ERR_MSG("unexpected EOF while writing: %s\n",
+                                       filename);
+                               rc = -2;
+                               goto out;
+                       }
+                       c = fputc(c, fpout);
+                       if (c == EOF) {
+                               ERR_MSG("couldn't write: %s\n", filename);
+                               rc = -3;
+                               goto out;
                        }
                }
+
+               cur = cur->next;
+               num++;
        }
 
-       free(vol_tab);
-       free(vol_len);
-       return 0;
+ out:
+       if (vol_id == NULL)
+               fclose(fpout);
+       return rc;
 }
 
-int
-main(int argc, char *argv[])
+
+/**
+ * traverses FILE* trying to load complete, valid and accurate header data
+ * into the eb chain;
+ **/
+static int
+unubi_volumes(FILE* fpin, uint32_t *vols, size_t vc, struct args *a)
 {
-       int len, rc;
-       FILE *fp;
-       struct stat file_info;
-       uint8_t *buf;
-       int i;
+       char filename[MAXPATH + 1];
+       char reason[MAXPATH + 1];
+       int rc;
+       size_t i, count;
+       /* relations:
+        * cur ~ head
+        * next ~ first */
+       eb_info_t head, cur, first, next;
+       eb_info_t *next_ptr;
+
+       rc = 0;
+       count = 0;
+       head = NULL;
+       first = NULL;
+       next = NULL;
+       cur = malloc(sizeof(*cur));
+       if (cur == NULL) {
+               ERR_MSG("out of memory\n");
+               rc = -ENOMEM;
+               goto err;
+       }
+       memset(cur, 0, sizeof(*cur));
 
-       init_crc32_table(crc32_table);
+       fgetpos(fpin, &(cur->eb_top));
+       while (1) {
+               const char *raw_path;
+               uint32_t crc;
+
+               memset(filename, 0, MAXPATH + 1);
+               memset(reason, 0, MAXPATH + 1);
+
+               /* in case of an incomplete ec header */
+               raw_path = FN_INVAL;
+
+               /* read erasecounter header */
+               rc = fread(&cur->outer, 1, sizeof(cur->outer), fpin);
+               if (rc == 0)
+                       goto out; /* EOF */
+               if (rc != sizeof(cur->outer)) {
+                       ERR_MSG("reading ec-hdr failed rc=%d\n", rc);
+                       rc = -1;
+                       goto err;
+               }
+
+               /* check erasecounter header magic */
+               if (ubi32_to_cpu(cur->outer.magic) != UBI_EC_HDR_MAGIC) {
+                       snprintf(reason, MAXPATH, ".invalid.ec_magic");
+                       goto invalid;
+               }
+
+               /* check erasecounter header crc */
+               header_crc(&(cur->outer), NULL, &crc);
+
+               if (ubi32_to_cpu(cur->outer.hdr_crc) != crc) {
+                       snprintf(reason, MAXPATH, ".invalid.ec_hdr_crc");
+                       goto invalid;
+               }
+
+               /* read volume id header */
+               rc = fsetpos(fpin, &(cur->eb_top));
+               if (rc != 0)
+                       goto err;
+               fseek(fpin, ubi32_to_cpu(cur->outer.vid_hdr_offset), SEEK_CUR);
+
+               /* read erasecounter header */
+               rc = fread(&cur->inner, 1, sizeof(cur->inner), fpin);
+               if (rc == 0)
+                       goto out; /* EOF */
+               if (rc != sizeof(cur->inner)) {
+                       ERR_MSG("reading vid-hdr failed rc=%d\n", rc);
+                       rc = -1;
+                       goto err;
+               }
+
+               /* empty? */
+               if (ubi32_to_cpu(cur->inner.magic) == 0xffffffff) {
+                       snprintf(reason, MAXPATH, ".empty");
+                       goto invalid;
+               }
+
+               /* vol_id should be in bounds */
+               if ((ubi32_to_cpu(cur->inner.vol_id) >= UBI_MAX_VOLUMES) &&
+                   (ubi32_to_cpu(cur->inner.vol_id) <
+                    UBI_INTERNAL_VOL_START)) {
+                       snprintf(reason, MAXPATH, ".invalid");
+                       goto invalid;
+               } else
+                       raw_path = FN_NSURE;
+
+               /* check volume id header magic */
+               if (ubi32_to_cpu(cur->inner.magic) != UBI_VID_HDR_MAGIC) {
+                       snprintf(reason, MAXPATH, ".invalid.vid_magic");
+                       goto invalid;
+               }
+
+               /* check volume id header crc */
+               header_crc(NULL, &(cur->inner), &crc);
+               if (ubi32_to_cpu(cur->inner.hdr_crc) != crc) {
+                       snprintf(reason, MAXPATH, ".invalid.vid_hdr_crc");
+                       goto invalid;
+               }
+
+               /* check data crc */
+               rc = data_crc(fpin, ubi32_to_cpu(cur->inner.data_size), &crc);
+               if (rc < 0)
+                       goto err;
+               if (ubi32_to_cpu(cur->inner.data_crc) != crc) {
+                       snprintf(reason, MAXPATH, ".invalid.data_crc");
+                       goto invalid;
+               }
 
-       argp_parse(&argp, argc, argv, ARGP_IN_ORDER, 0, &myargs);
+               /* enlist this vol, it's valid */
+               raw_path = FN_VALID;
+               cur->linear = count;
+               rc = eb_chain_insert(&head, cur);
+               if (rc < 0) {
+                       if (rc == -ENOMEM) {
+                               ERR_MSG("out of memory\n");
+                               goto err;
+                       }
+                       ERR_MSG("unknown and unexpected error, please contact "
+                               CONTACT "\n");
+                       goto err;
+               }
+
+               if (a->vol_split) {
+                       size_t size = 0;
 
-       if (!myargs.arg1) {
-               fprintf(stderr, "Please specify input file!\n");
-               exit(EXIT_FAILURE);
+                       rc = fsetpos(fpin, &(cur->eb_top));
+                       if (rc != 0)
+                               goto err;
+
+                       if (a->vol_split == SPLIT_DATA) {
+                               /* write only data section */
+                               size = ubi32_to_cpu(cur->inner.data_size);
+                               fseek(fpin,
+                                     ubi32_to_cpu(cur->outer.data_offset),
+                                     SEEK_CUR);
+                       }
+                       else if (a->vol_split == SPLIT_RAW)
+                               /* write entire eraseblock */
+                               size = a->bsize;
+
+                       snprintf(filename, MAXPATH, FN_VOLSP,
+                                a->odir_path,
+                                ubi32_to_cpu(cur->inner.vol_id),
+                                ubi32_to_cpu(cur->inner.lnum),
+                                ubi32_to_cpu(cur->inner.leb_ver), count);
+                       rc = extract_data(fpin, size, filename);
+                       if (rc < 0)
+                               goto err;
+               }
+
+ invalid:
+               if (a->eb_split) {
+                       /* jump to top of block */
+                       rc = fsetpos(fpin, &(cur->eb_top));
+                       if (rc != 0)
+                               goto err;
+
+                       if (strcmp(raw_path, FN_INVAL) == 0)
+                               snprintf(filename, MAXPATH, raw_path,
+                                        a->odir_path, count, reason);
+                       else
+                               snprintf(filename, MAXPATH, raw_path,
+                                        a->odir_path,
+                                        count,
+                                        ubi32_to_cpu(cur->inner.vol_id),
+                                        ubi32_to_cpu(cur->inner.lnum),
+                                        ubi32_to_cpu(cur->inner.leb_ver),
+                                        reason);
+
+                       rc = extract_data(fpin, a->bsize, filename);
+                       if (rc < 0)
+                               goto err;
+               }
+
+               /* append to simple linked list */
+               if (first == NULL)
+                       next_ptr = &first;
+               else
+                       next_ptr = &next->next;
+
+               *next_ptr = malloc(sizeof(**next_ptr));
+               if (*next_ptr == NULL) {
+                       ERR_MSG("out of memory\n");
+                       rc = -ENOMEM;
+                       goto err;
+               }
+               memset(*next_ptr, 0, sizeof(**next_ptr));
+
+               next = *next_ptr;
+               memcpy(next, cur, sizeof(*next));
+               next->next = NULL;
+
+               count++;
+               rc = fsetpos(fpin, &(cur->eb_top));
+               if (rc != 0)
+                       goto err;
+               fseek(fpin, a->bsize, SEEK_CUR);
+               memset(cur, 0, sizeof(*cur));
+
+               fgetpos(fpin, &(cur->eb_top));
        }
 
-       fp = fopen(myargs.arg1, "r");
-       if (!fp) {
-               perror("Cannot open file");
-               exit(EXIT_FAILURE);
+ out:
+       for (i = 0; i < vc; i++) {
+               rc = rebuild_volume(fpin, &vols[i], &head, a->odir_path);
+               if (rc < 0)
+                       goto err;
        }
-       if (fstat(fileno(fp), &file_info) != 0) {
-               fprintf(stderr, "Cannot fetch file size "
-                       "from input file.\n");
+
+       /* if there were no volumes specified, rebuild them all,
+        * UNLESS eb_ or vol_ split or analyze was specified */
+       if ((vc == 0) && (!a->eb_split) && (!a->vol_split) && (!a->analyze)) {
+               rc = rebuild_volume(fpin, NULL, &head, a->odir_path);
+               if (rc < 0)
+                       goto err;
        }
-       len = file_info.st_size;
-       buf = malloc(len);
-       if (!buf) {
-               perror("out of memory!");
-               exit(EXIT_FAILURE);
+
+ err:
+       free(cur);
+
+       if (a->analyze)
+               unubi_analyze(&head, first, a->odir_path);
+       eb_chain_destroy(&head);
+       eb_chain_destroy(&first);
+
+       return rc;
+}
+
+
+/**
+ * handles command line arguments, then calls unubi_volumes
+ **/
+int
+main(int argc, char *argv[])
+{
+       int rc, free_a_odir;
+       size_t vols_len;
+       uint32_t *vols;
+       FILE* fpin;
+       struct args a;
+
+       rc = 0;
+       free_a_odir = 0;
+       vols_len = 0;
+       vols = NULL;
+       fpin = NULL;
+       init_crc32_table(crc32_table);
+
+       /* setup struct args a */
+       memset(&a, 0, sizeof(a));
+       a.bsize = 128 * KIB;
+       a.vols = malloc(sizeof(*a.vols) * UBI_MAX_VOLUMES);
+       if (a.vols == NULL) {
+               ERR_MSG("out of memory\n");
+               rc = ENOMEM;
+               goto err;
        }
-       rc = fread(buf, 1, len, fp);
-       if (ferror(fp) || (len != rc)) {
-               perror("could not read file");
-               exit(EXIT_FAILURE);
+       memset(a.vols, 0, sizeof(*a.vols) * UBI_MAX_VOLUMES);
+
+       /* parse args and check for validity */
+       argp_parse(&argp, argc, argv, ARGP_IN_ORDER, 0, &a);
+       if (a.img_path == NULL) {
+               ERR_MSG("no image file specified\n");
+               rc = EINVAL;
+               goto err;
        }
-       if (!myargs.output_dir) {
-               fprintf(stderr, "No output directory specified!\n");
-               exit(EXIT_FAILURE);
+       else if (a.odir_path == NULL) {
+               char *ptr;
+               int len;
+
+               ptr = strrchr(a.img_path, '/');
+               if (ptr == NULL)
+                       ptr = a.img_path;
+               else
+                       ptr++;
+
+               len = strlen(DIR_FMT) + strlen(ptr);
+               free_a_odir = 1;
+               a.odir_path = malloc(sizeof(*a.odir_path) * len);
+               if (a.odir_path == NULL) {
+                       ERR_MSG("out of memory\n");
+                       rc = ENOMEM;
+                       goto err;
+               }
+               snprintf(a.odir_path, len, DIR_FMT, ptr);
+       }
+
+       fpin = fopen(a.img_path, "rb");
+       if (fpin == NULL) {
+               ERR_MSG("couldn't open file for reading: "
+                       "%s\n", a.img_path);
+               rc = EINVAL;
+               goto err;
        }
 
-       rc = mkdir(myargs.output_dir, 0777);
-       if (rc && errno != EEXIST) {
-               perror("Cannot create output directory");
-               exit(EXIT_FAILURE);
+       rc = mkdir(a.odir_path, 0777);
+       if ((rc < 0) && (errno != EEXIST)) {
+               ERR_MSG("couldn't create ouput directory: "
+                       "%s\n", a.odir_path);
+               rc = -rc;
+               goto err;
        }
-       for (i = 0; i < 32; i++) {
-               char fname[1024];
-               FILE *fpout;
-
-               printf("######### VOLUME %d ############################\n",
-                      i);
-
-               sprintf(fname, "%s/ubivol_%d.bin", myargs.output_dir, i);
-               fpout = fopen(fname, "w+");
-               if (!fpout) {
-                       perror("Cannot open file");
-                       exit(EXIT_FAILURE);
+
+       /* fill in vols array */
+       vols_len = count_set(a.vols, UBI_MAX_VOLUMES);
+       if (vols_len > 0) {
+               vols = malloc(sizeof(*vols) * vols_len);
+               if (vols == NULL) {
+                       ERR_MSG("out of memory\n");
+                       rc = ENOMEM;
+                       goto err;
                }
-               extract_volume(&myargs, buf, len, i, fpout);
-               fclose(fpout);
+               collapse(a.vols, UBI_MAX_VOLUMES, vols, vols_len);
+       }
+
+       /* unubi volumes */
+       rc = unubi_volumes(fpin, vols, vols_len, &a);
+       if (rc < 0) {
+               ERR_MSG("error encountered while working on image file: "
+                       "%s\n", a.img_path);
+               rc = -rc;
+               goto err;
        }
 
-       fclose(fp);
-       free(buf);
-       exit(EXIT_SUCCESS);
+ err:
+       free(a.vols);
+       if (free_a_odir != 0)
+               free(a.odir_path);
+       if (fpin != NULL)
+               fclose(fpin);
+       if (vols_len > 0)
+               free(vols);
+       return rc;
 }

diff --git a/ubi-utils/src/unubi_analyze.c b/ubi-utils/src/unubi_analyze.c
new file mode 100644 (file)
index 0000000..2e94ca9
--- /dev/null
+++ b/ubi-utils/src/unubi_analyze.c
@@ -0,0 +1,435 @@
+/*
+ * Copyright (c) International Business Machines Corp., 2006
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
+ * the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+/*
+ * Authors: Drake Dowsett, dowsett@de.ibm.com
+ * Contact: Andreas Arnez, arnez@de.ibm.com
+ *
+ * unubi uses the following functions to generate analysis output based on
+ * the header information in a raw-UBI image
+ */
+
+/*
+ * TODO: use OOB data to check for eraseblock validity in NAND images
+ */
+
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+#include <stdio.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+
+#include "eb_chain.h"
+#include "crc32.h"
+
+#define MAXPATH                1024
+#define EC_X_INT       50
+
+#define FN_STATS       "analysis_stats.txt"
+#define FN_EH_DATA     "analysis_ec_hdr.data"
+#define FN_EH_PLOT     "analysis_ec_hdr.plot"
+#define FN_VH_DATA     "analysis_vid_hdr.data"
+#define FN_VH_PLOT     "analysis_vid_hdr.plot"
+
+
+/**
+ * intcmp - function needed by qsort to order integers
+ **/
+int intcmp(const void *a, const void *b)
+{
+       int A = *(int *)a;
+       int B = *(int *)b;
+       return A - B;
+}
+
+int longcmp(const void *a, const void *b)
+{
+       long long A = *(long long *)a;
+       long long B = *(long long *)b;
+       return A - B;
+}
+
+
+/**
+ * unubi_analyze_group_index - finds the normalized index in an array
+ * item:       look for this item in the array
+ * array:      array to search through
+ * size:       length of the array
+ * array should be sorted for this algorithm to perform properly;
+ * if the item is not found returns -1, otherwise return value is the
+ * index in the array (note this contricts the array size to 2^32-1);
+ **/
+int
+norm_index(uint32_t item, uint32_t *array, size_t length)
+{
+       size_t i, index;
+
+       for (index = 0, i = 0; i < length; i++) {
+               if ((i != 0) && (array[i] != array[i - 1]))
+                       index++;
+
+               if (item == array[i])
+                       return index;
+       }
+
+       return -1;
+}
+
+
+/**
+ * unubi_analyze_ec_hdr - generate data table and plot script
+ * first:      head of simple linked list
+ * path:       folder to write into
+ * generates a data file containing the eraseblock index in the image
+ * and the erase counter found in its ec header;
+ * if the crc check fails, the line is commented out in the data file;
+ * also generates a simple gnuplot sript for quickly viewing one
+ * display of the data file;
+ **/
+int
+unubi_analyze_ec_hdr(eb_info_t first, const char *path)
+{
+       char filename[MAXPATH + 1];
+       size_t count, eraseblocks;
+       uint32_t crc, crc32_table[256];
+       uint64_t *erase_counts;
+       FILE* fpdata;
+       FILE* fpplot;
+       eb_info_t cur;
+
+       /* crc check still needed for `first' linked list */
+       init_crc32_table(crc32_table);
+
+       /* prepare output files */
+       memset(filename, 0, MAXPATH + 1);
+       snprintf(filename, MAXPATH, "%s/%s", path, FN_EH_DATA);
+       fpdata = fopen(filename, "w");
+       if (fpdata == NULL)
+               return -1;
+
+       memset(filename, 0, MAXPATH + 1);
+       snprintf(filename, MAXPATH, "%s/%s", path, FN_EH_PLOT);
+       fpplot = fopen(filename, "w");
+       if (fpplot == NULL)
+               return -1;
+
+       chmod(filename, 0755);
+
+       /* first run: count elements */
+       count = 0;
+       cur = first;
+       while (cur != NULL) {
+               cur = cur->next;
+               count++;
+       }
+       eraseblocks = count;
+
+       erase_counts = malloc(eraseblocks * sizeof(*erase_counts));
+       memset(erase_counts, 0, eraseblocks * sizeof(*erase_counts));
+
+       /* second run: populate array to sort */
+       count = 0;
+       cur = first;
+       while(cur != NULL) {
+               erase_counts[count] = ubi64_to_cpu(cur->outer.ec);
+               cur = cur->next;
+               count++;
+       }
+       qsort(erase_counts, eraseblocks, sizeof(*erase_counts),
+             (void *)longcmp);
+
+       /* third run: generate data file */
+       count = 0;
+       cur = first;
+       fprintf(fpdata, "# eraseblock_no actual_erase_count "
+                       "sorted_erase_count\n");
+       while (cur != NULL) {
+               crc = clc_crc32(crc32_table, UBI_CRC32_INIT, &cur->outer,
+                               UBI_EC_HDR_SIZE_CRC);
+
+               if ((ubi32_to_cpu(cur->outer.magic) != UBI_EC_HDR_MAGIC) ||
+                   (crc != ubi32_to_cpu(cur->outer.hdr_crc)))
+                       fprintf(fpdata, "# ");
+
+               fprintf(fpdata, "%u %llu %llu", count,
+                       ubi64_to_cpu(cur->outer.ec),
+                       erase_counts[count]);
+
+               if (ubi32_to_cpu(cur->outer.magic) != UBI_EC_HDR_MAGIC)
+                       fprintf(fpdata, " ## bad magic: %08x",
+                               ubi32_to_cpu(cur->outer.magic));
+
+               if (crc != ubi32_to_cpu(cur->outer.hdr_crc))
+                       fprintf(fpdata, " ## CRC mismatch: given=%08x, "
+                               "calc=%08x", ubi32_to_cpu(cur->outer.hdr_crc),
+                               crc);
+
+               fprintf(fpdata, "\n");
+
+               cur = cur->next;
+               count++;
+       }
+       fclose(fpdata);
+
+       fprintf(fpplot, "#!/usr/bin/gnuplot -persist\n");
+       fprintf(fpplot, "set xlabel \"eraseblock\"\n");
+
+       /* fourth run: generate plot file xtics */
+       count = 0;
+       cur = first;
+       fprintf(fpplot, "set xtics (");
+       while (cur != NULL) {
+               if ((count % EC_X_INT) == 0) {
+                       if (count > 0)
+                               fprintf(fpplot, ", ");
+                       fprintf(fpplot, "%d", count);
+               }
+
+               cur = cur->next;
+               count++;
+       }
+       fprintf(fpplot, ")\n");
+
+       fprintf(fpplot, "set ylabel \"erase count\"\n");
+       fprintf(fpplot, "set xrange [-1:%u]\n", eraseblocks + 1);
+       fprintf(fpplot, "# set yrange [-1:%llu]\n",
+               erase_counts[eraseblocks - 1] + 1);
+       fprintf(fpplot, "plot \"%s\" u 1:2 t \"unsorted: %s\" with boxes\n",
+               FN_EH_DATA, FN_EH_DATA);
+       fprintf(fpplot, "# replot \"%s\" u 1:3 t \"sorted: %s\" with lines\n",
+               FN_EH_DATA, FN_EH_DATA);
+       fprintf(fpplot, "pause -1 \"press ENTER\"\n");
+
+       fclose(fpplot);
+
+       return 0;
+}
+
+
+/**
+ * unubi_analyze_vid_hdr - generate data table and plot script
+ * head:       head of complex linked list (eb_chain)
+ * path:       folder to write into
+ * generates a data file containing the volume id, logical number, leb version,
+ * and data size from the vid header;
+ * all eraseblocks listed in the eb_chain are valid (checked in unubi);
+ * also generates a simple gnuplot sript for quickly viewing one
+ * display of the data file;
+ **/
+int
+unubi_analyze_vid_hdr(eb_info_t *head, const char *path)
+{
+       char filename[MAXPATH + 1];
+       int y1, y2;
+       size_t count, step, breadth;
+       uint32_t *leb_versions, *data_sizes;
+       FILE* fpdata;
+       FILE* fpplot;
+       eb_info_t cur;
+
+       /* prepare output files */
+       memset(filename, 0, MAXPATH + 1);
+       snprintf(filename, MAXPATH, "%s/%s", path, FN_VH_DATA);
+       fpdata = fopen(filename, "w");
+       if (fpdata == NULL)
+               return -1;
+
+       memset(filename, 0, MAXPATH + 1);
+       snprintf(filename, MAXPATH, "%s/%s", path, FN_VH_PLOT);
+       fpplot = fopen(filename, "w");
+       if (fpplot == NULL)
+               return -1;
+
+       chmod(filename, 0755);
+
+       /* first run: count elements */
+       count = 0;
+       cur = *head;
+       while (cur != NULL) {
+               cur = cur->next;
+               count++;
+       }
+       breadth = count;
+
+       leb_versions = malloc(breadth * sizeof(*leb_versions));
+       memset(leb_versions, 0, breadth * sizeof(*leb_versions));
+
+       data_sizes = malloc(breadth * sizeof(*data_sizes));
+       memset(data_sizes, 0, breadth * sizeof(*data_sizes));
+
+       /* second run: populate arrays to sort */
+       count = 0;
+       cur = *head;
+       while (cur != NULL) {
+               leb_versions[count] = ubi32_to_cpu(cur->inner.leb_ver);
+               data_sizes[count] = ubi32_to_cpu(cur->inner.data_size);
+               cur = cur->next;
+               count++;
+       }
+       qsort(leb_versions, breadth, sizeof(*leb_versions), (void *)intcmp);
+       qsort(data_sizes, breadth, sizeof(*data_sizes), (void *)intcmp);
+
+       /* third run: generate data file */
+       count = 0;
+       cur = *head;
+       fprintf(fpdata, "# x_axis vol_id lnum   y1_axis leb_ver   "
+               "y2_axis data_size\n");
+       while (cur != NULL) {
+               y1 = norm_index(ubi32_to_cpu(cur->inner.leb_ver), leb_versions,
+                               breadth);
+               y2 = norm_index(ubi32_to_cpu(cur->inner.data_size), data_sizes,
+                               breadth);
+
+               if ((y1 == -1) || (y2 == -1))
+                       return -1;
+
+               fprintf(fpdata, "%u %u %u   %u %u   %u %u\n",
+                       count,
+                       ubi32_to_cpu(cur->inner.vol_id),
+                       ubi32_to_cpu(cur->inner.lnum),
+                       y1,
+                       ubi32_to_cpu(cur->inner.leb_ver),
+                       y2,
+                       ubi32_to_cpu(cur->inner.data_size));
+               cur = cur->next;
+               count++;
+       }
+       fclose(fpdata);
+
+       fprintf(fpplot, "#!/usr/bin/gnuplot -persist\n");
+       fprintf(fpplot, "set xlabel \"volume\"\n");
+
+       /* fourth run: generate plot file xtics */
+       count = 0;
+       step = 0;
+       cur = *head;
+       fprintf(fpplot, "set xtics (");
+       while (cur != NULL) {
+               if (count > 0)
+                       fprintf(fpplot, ", ");
+               if (step != ubi32_to_cpu(cur->inner.vol_id)) {
+                       step = ubi32_to_cpu(cur->inner.vol_id);
+                       fprintf(fpplot, "\"%d\" %d 0", step, count);
+               }
+               else
+                       fprintf(fpplot, "\"%d\" %d 1",
+                               ubi32_to_cpu(cur->inner.lnum), count);
+               cur = cur->next;
+               count++;
+       }
+       fprintf(fpplot, ")\n");
+       fprintf(fpplot, "set nox2tics\n");
+
+       /* fifth run: generate plot file ytics */
+       count = 0;
+       cur = *head;
+       fprintf(fpplot, "set ylabel \"leb version\"\n");
+       fprintf(fpplot, "set ytics (");
+       while (cur != NULL) {
+               y1 = norm_index(ubi32_to_cpu(cur->inner.leb_ver), leb_versions,
+                               breadth);
+
+               if (y1 == -1)
+                       return -1;
+
+               if (count > 0)
+                       fprintf(fpplot, ", ");
+
+               fprintf(fpplot, "\"%u\" %u", ubi32_to_cpu(cur->inner.leb_ver),
+                       y1);
+
+               cur = cur->next;
+               count++;
+       }
+       fprintf(fpplot, ")\n");
+
+       /* sixth run: generate plot file y2tics */
+       count = 0;
+       cur = *head;
+       fprintf(fpplot, "set y2label \"data size\"\n");
+       fprintf(fpplot, "set y2tics (");
+       while (cur != NULL) {
+               y2 = norm_index(ubi32_to_cpu(cur->inner.data_size),
+                               data_sizes, breadth);
+
+               if (y2 == -1)
+                       return -1;
+
+               if (count > 0)
+                       fprintf(fpplot, ", ");
+
+               fprintf(fpplot, "\"%u\" %u", ubi32_to_cpu(cur->inner.data_size),
+                       y2);
+
+               cur = cur->next;
+               count++;
+       }
+       fprintf(fpplot, ")\n");
+
+       y1 = norm_index(leb_versions[breadth - 1], leb_versions, breadth);
+       y2 = norm_index(data_sizes[breadth - 1], data_sizes, breadth);
+       fprintf(fpplot, "set xrange [-1:%u]\n", count + 1);
+       fprintf(fpplot, "set yrange [-1:%u]\n", y1 + 1);
+       fprintf(fpplot, "set y2range [-1:%u]\n", y2 + 1);
+       fprintf(fpplot, "plot \"%s\" u 1:4 t \"leb version: %s\" "
+                       "axes x1y1 with lp\n", FN_VH_DATA, FN_VH_DATA);
+       fprintf(fpplot, "replot \"%s\" u 1:6 t \"data size: %s\" "
+                       "axes x1y2 with lp\n", FN_VH_DATA, FN_VH_DATA);
+       fprintf(fpplot, "pause -1 \"press ENTER\"\n");
+
+       fclose(fpplot);
+
+       free(data_sizes);
+       free(leb_versions);
+
+       return 0;
+}
+
+
+/**
+ * unubi_analyze - run all analyses
+ * head:       eb_chain head
+ * first:      simple linked list of eraseblock headers (use .next)
+ * path:       directory (without trailing slash) to output to
+ * returns 0 upon successful completion, or -1 otherwise
+ **/
+int
+unubi_analyze(eb_info_t *head, eb_info_t first, const char *path)
+{
+       int rc;
+
+       if (path == NULL)
+               return -1;
+
+       if (first == NULL)
+               return -1;
+
+       if ((head == NULL) || (*head == NULL))
+               return -1;
+
+       rc = unubi_analyze_ec_hdr(first, path);
+       if (rc < 0)
+               return -1;
+
+       rc = unubi_analyze_vid_hdr(head, path);
+       if (rc < 0)
+               return -1;
+
+       return 0;
+}

diff --git a/ubi-utils/src/unubi_analyze.h b/ubi-utils/src/unubi_analyze.h
new file mode 100644 (file)
index 0000000..ac01a44
--- /dev/null
+++ b/ubi-utils/src/unubi_analyze.h
@@ -0,0 +1,26 @@
+/*
+ * Copyright (c) International Business Machines Corp., 2006
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
+ * the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+/*
+ * Authors: Drake Dowsett, dowsett@de.ibm.com
+ */
+
+#include "eb_chain.h"
+
+int
+unubi_analyze(eb_info_t *head, eb_info_t first, const char *path);