Jacob Keller [Mon, 8 Sep 2014 23:53:06 +0000 (16:53 -0700)]
aiaiai: cleanup display of randconfig builds
This patch removes some code attempting to show the random configuration
generated for the build. It does not correctly show up in the email, so
I need to spend some time figuring out where to display the
configuration file.
Additionally, clean up some display about generating the random config
so that we don't see the errors. Also we do a make mrproper to cleanup
any generated files so that builds will be clean when we start.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Jacob Keller [Thu, 10 Jul 2014 23:36:37 +0000 (16:36 -0700)]
test-patchset: allow proper support for randconfig
Make randconfig pre-generate a configuration so that the pre and post
patch series builds use the same configuration (vs using random configs
each time). In addition, append the random configuration whenever there
is a build diff, so that the user can see.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Jacob Keller [Wed, 18 Jun 2014 20:43:52 +0000 (13:43 -0700)]
libshell: use a git submodule to track upstream libshell project
Rather than inlining the various libshell code we need, just add a git
submodule pointing to the upstream tree. This makes it easier to manage
changes rather than having to hand merge them. This brings with it
several bug fixes and changes from the upstream project, so may require
some patience to work out any subtle changes.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Jacob Keller [Tue, 27 May 2014 21:54:02 +0000 (14:54 -0700)]
email-test-configuration: add helper program to debug configuration
This script is very useful for debugging the configuration file. It
takes a snapshot of the environment before and after parse_config and
after each parse_prj_config, and then uses diff to display the output.
This makes it very easy to see if the configuration is being read as
desired. This script has already helped debug an issue in the
shell_ini_config parser.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Jacob Keller [Thu, 17 Apr 2014 23:27:35 +0000 (16:27 -0700)]
aiaiai-test-patchset: make checkpatch optional like other checkers
This modifies behavior of aiaiai such that checkpatch will be an option
much like sparse and coccinelle. In addition, only work if checkpatch.pl
is found in the project at the requested location. Will disable
checkpatch if it cannot be found.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Jacob Keller [Thu, 17 Apr 2014 23:26:19 +0000 (16:26 -0700)]
aiaiai-test-patchset: check for coccinelle scripts first
Rather than always archiving the coccinelle scripts, check to ensure
they exist first. This prevents issues when attempting to build a
non-kernel tree, (ie: ethtool). Will display a warning if coccinelle was
requested but no scripts were found.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Jacob Keller [Tue, 27 May 2014 18:02:59 +0000 (11:02 -0700)]
aiaiai: merge development branch to master
Finally merge the development changes after some fairly serious use in
the email environment. I have appended the shortlog below, as well as a
helpful short summary of the large changes in no particular order.
* Moved as much as possible for configuring email setup to the cfgfile
* Added several options previously not configurable at all
* Add support for "hooks" framework, including a simple dispatcher hook
* Add hook which automatically detects project based on git-index info
* Add systemd scripts for the email setup
* Allow default values for project configuration
* Add script to help keep project repos up to date
These changes hopefully make configuration and setup of the email
workflow easier. In addition, the hook enabling automatic project
detection makes handling multiple project kernel trees much easier.
Artem Bityutskiy (6):
test-pattchet: add a couple quotes
aiaiai-test-patchset: move --targets option down
aiaiai-diff-log-helper: fix logs diffing for gcc-4.8
aiaiai-checker: use --no-data for smatch
aiaiai-checker: ignore debugging output from spatch
TODO: remove a completed item
Jacob Keller (51):
Add support for choosing build targets
aiaiai-sh-functions: remove extra quotes in call to ini_config_get
aiaiai-test-patchset: use merge_addresses only once
aiaiai-sh-functions: remove [email] section from cfgfile
aiaiai-sh-functions: re-order options to match example config
aiaiai-email: add [defaults] section to config
aiaiai-email: move test-patchset options into the configuration file
aiaiai: don't allow arbitrary validator command
aiaiai-email: move -J bigjobs into configuration file
aiaiai-test-patchset: add missing colon on targets option
aiaiai-email-dispatcher-helper: fix typo on aiaiai
aiaiai-email-test-patchset: fix parameter substitution
aiaiai-email-test-patchset: move -- to end of all options
email: add "debug" section to configuration file
email-test-patchset: parse general configuration settings earlier
sh-function: add insert_header function
doc: add CONFIGURATION file describing the cfgfile
aiaiai: add initial support for dispatcher hook
hook: add support for X-Aiaiai-Project
aiaiai: fixup srcdir in email and gerrit scripts
dispatcher: add aiaiai-email-sh-functions to source list
doc: rename CONFIGURATION to configuration.txt
email: don't allow pcfg_branch to default
email: add canonical_url variable for list_projects
aiaiai: split sed project listing from list_projects
email: rename error_test_patchset_failed
aiaiai: change dispatcher hook to email hook
aiaiai: add git-find-base utility for finding base commit from mbox
email: example hook for aiaiai-email-test-patchset
aiaiai: use fail_usage on incorrect number of arguments
apply-patch: prefix diff output with '> '
autodetect-project: fix change of function name
systemd: add scripts for running the aiaiai processes
email-hook: remove options that can't be used
git-find-base: Add authorship, copyright, and license notice
git-find-base: add detection for possible duplicate patches
git-find-base: add help text to the program
send-mail-on-failure: generalize email address
aiaiai-project-update: fix srcdir line from location change
aiaiai-project-update: fix typo of printf in usage statement
aiaiai-project-update: correct function to upstream change
git-find-base: rewritten to use newer design
aiaiai-email-test-patchset: correct hook calling to actually grab error
aiaiai: extract patches from email prior to using git-am
aiaiai-diff-log-helper: explicitely call out python2
aiaiai: add checks to configuration file
aiaiai-email-lda: move reap options to cfgfile
email: fix incorrect variable name in email for hook rejection
aiaiai: don't display log output from git-find-base
aiaiai: don't cat the $mbox file at begining of email-testpatchset
aiaiai: don't hardcode KCFLAGS or W=1
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Jacob Keller [Wed, 9 Apr 2014 22:26:13 +0000 (15:26 -0700)]
aiaiai: don't hardcode KCFLAGS or W=1
This patch fixes the TODO item regarding hardcoded kernel make
parameters. We now default -M to be "KCFLAGS='-Wno-missing-initializers
-Wno-sign-compare' W=1", and allow redefining the option via the email
configuration line or the command line setting for aiaiai-test-patchset
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Wed, 9 Apr 2014 22:26:12 +0000 (15:26 -0700)]
aiaiai: don't cat the $mbox file at begining of email-testpatchset
This patch corrects an inadverdent addition of cat "$mbox" which apears
to come from commit bbf0119cbfc0. Possibly originating as some sort of
debugging code, it has essentially been lost in a rebase most likely.
However, this cat of the mbox contents is quite annoying. Possibly add
this back as a feature, since it sometimes proves useful, but generally
slows down aiaiai due to sometimes printing an exceptional amount of
data to stdout! Also fixes a typo of private.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Fixes: bbf0119cbfc0 ("email-lda: use cover letter subject if available") Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Wed, 9 Apr 2014 22:26:11 +0000 (15:26 -0700)]
aiaiai: don't display log output from git-find-base
git-find-base outputs a large amount of data, given the number of
projects in a cfgfile. This can cause systemd to overload with too much
information. This patch alleviates the issue by only displaying the
output of git-find-base if it fails.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Wed, 9 Apr 2014 22:26:09 +0000 (15:26 -0700)]
aiaiai-email-lda: move reap options to cfgfile
This patch moves the last few options of the LDA email program into the
configuration file, so that all email scripts don't require options for
configuration.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Wed, 9 Apr 2014 22:26:08 +0000 (15:26 -0700)]
aiaiai: add checks to configuration file
This patch adds configuration check functions and performs some basic
checks against the configuration file. A future effort may be to clean
up these checks so that it outputs a bit more error cases.
The main value of these checks is to prevent weird errors later when we
assume that values have some standard meaning. In addition, it allows
cleaning up boolean values to always be canonicalized into 0 or 1.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Wed, 9 Apr 2014 22:26:07 +0000 (15:26 -0700)]
aiaiai-diff-log-helper: explicitely call out python2
Since this script is not a python3 script, (it runs in python2!) then we
shouldn't rely on "/usr/bin/env python" to always land us on a python2
installation, as some machines may have installed python3, and
configured their default python to be the python3 executable.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Wed, 9 Apr 2014 22:26:06 +0000 (15:26 -0700)]
aiaiai: extract patches from email prior to using git-am
Some patch authors submit patches directly attached via git-format-patch
output as an attachment. Sometimes they include both the patch and the
inline diff. This is problematic because it causes aiaiai to fail to
apply the patch, due to git being confused about the attached patch.
This patch modifies how aiaiai reads the commits, using a python script
which is capable of extracting the patches from the mbox (including
attachments) before calling git-am.
This helps aiaiai more easily handle this weird patch case, and still
works great for regular patch submissions. In addition, turn on 3-way
merge of git-am, so that duplicate patches will be properly ignored (as
in the case with inlined and attached patches)
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Wed, 9 Apr 2014 22:26:05 +0000 (15:26 -0700)]
aiaiai-email-test-patchset: correct hook calling to actually grab error
We can't grab the $? value from inside an if block, as the if statement
already changed the return code. Instead, we just call the hook script
without an error section, and then check for errors afterwards. This
corrects an issue where the return code would always be 0, even if the
command failed.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
aiaiai-checker: ignore debugging output from spatch
spatch is a great tool, but it is still not very stable and crashes sometimes,
in which case it prints a lot of debugging stuff to stderr, which we do not
want in out logs. Normal warnings go to stdout. Therefore, ignore stderr output
of spatch.
And switch to the new style options format instead of using the old style.
This patch has no functional changes. It is just a clean-up which makes the
help output a tiny bit easier to follow by moving --target out of the
block of options related to checkers. Also, it changes the orger of option
handling to match the order in the help output.
Jacob Keller [Fri, 4 Apr 2014 22:06:52 +0000 (15:06 -0700)]
git-find-base: rewritten to use newer design
Instead of just checking a few extra headers, add support for all header
types that git can output, and be a bit more robust about renames and
deletes. For now, assume a rename won't attempt to overwrite an existing
file.. Ideally we should do a full check on the base as well..
This model should allow support of all git commit types, not just simple
ones. In addition, in order to allow a certain patch format, we add an
option to remove duplicates. Do it by default, but enable option to keep
duplicate diff chunks. Primarily useful for patches which contain both
an inline and attachment.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Fri, 4 Apr 2014 22:06:49 +0000 (15:06 -0700)]
aiaiai-project-update: correct function to upstream change
Upstream used get_cfgfile_projects_list as the name of the function that
determined the full project listing. aiaiai-project-update was missed
in the change.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Fri, 4 Apr 2014 22:06:46 +0000 (15:06 -0700)]
send-mail-on-failure: generalize email address
Rather than using a personal email address (oops!) generalize the
address in the script as an argument. In addition, modify the systemd
script to use root@localhost. The user *is* expected to modify the
systemd script in their own copy to point to the correct address.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Mon, 31 Mar 2014 22:24:02 +0000 (15:24 -0700)]
git-find-base: add detection for possible duplicate patches
Some users send a patch which includes both an attachment and inline of
the diff. This confuses git-find-base, because it will see equivalent
patch blobs. If we see the same diff hunk with identical git-index
information, print a warning, and ignore it.
This patch also rewords some of the other warnings to be more consistent
and always show the intial -> modified blobs for possible debugging.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Mon, 31 Mar 2014 22:24:00 +0000 (15:24 -0700)]
email-hook: remove options that can't be used
Hooks don't really support options, so just remove them. Also, use
fail_usage instead of die for the exit message, so that we actually show
the usage when wrong number of arguments is given.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Mon, 31 Mar 2014 22:23:59 +0000 (15:23 -0700)]
systemd: add scripts for running the aiaiai processes
This patch adds 3 service files for use with aiaiai
1) aiaiai.service which runs the aiaiai email dispatcher
2) aiaiai-project-update.service which runs aiaiai-project-update which
will update remotes for all projects in the configuration file
3) send-mail-on-failure for notifying an administrator of service errors
The services are primarily useful for tracking aiaiai output as well as
keeping remotes that aiaiai needs up to date.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Fri, 28 Mar 2014 16:08:32 +0000 (09:08 -0700)]
apply-patch: prefix diff output with '> '
When aiaiai responds to a mailing list that is monitored by patchwork, a
failure to apply a diff can cause patchwork to generate false positive
"patches" due to the diff in the email. This patch modifies aiaiai so
that it prefixes the output of the command with a '> ' which will
prevent patchwork from recognizing this as a diff with a patch inlined
to the reply.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Fri, 28 Mar 2014 16:08:31 +0000 (09:08 -0700)]
aiaiai: use fail_usage on incorrect number of arguments
We should display the full usage with an error message when we have an
incorrect number of arguments, rather than just die. The advantage is
that the user will see what the actual usage of the program is, rather
than just a somewhat cryptic error message.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Thu, 27 Mar 2014 18:40:08 +0000 (11:40 -0700)]
email: example hook for aiaiai-email-test-patchset
This commit introduces an example hook which uses git-find-base to
determine what project (and even what commit inside that project!) the
patch is based on. This can be used so that patches automatically go to
the correct project, without having to specify the project in the email
address.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Thu, 27 Mar 2014 18:40:07 +0000 (11:40 -0700)]
aiaiai: add git-find-base utility for finding base commit from mbox
This patch adds a helper utility for git, git-find-base which works by
checking the index information provided by git-diff's output. It works
by parsing the mbox for diff lines, and extracting the index line. This
is the shortblob of the file in git's database that this patch
originally applied to.
The general flow of this perl tool is:
1) Extract all the files and base blob indexes that the patch modifies,
using a regular expression.
2) Run git log with all arguments directly passed through
3) For each commit returned by git log, check the ls-tree output for the
modified file. If any of them fail to match, then this patch cannot
apply cleanly to the commit.
4) print the sha1sum of the first commit which passes the check blobs
test.
This tool can be used to automatically determine if a commit can be
applied cleanly to a branch, and even detect which commit it should be
applied to. Various options passed to the log can change how much
detection it does.
If a patch series is applied, it will check each modification of a file,
and ensure that it could apply in sequence generating the final result.
This allows finding the first commit at which a series will apply. If a
series is not self-consistent, then git-find-base will fail to supply
the base commit, and may output errors regarding the nature of the
inconsistency.
The primary intention for this tool will be an example hook for aiaiai
which automatically detects which project a patch was originally based
on.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Thu, 27 Mar 2014 18:40:06 +0000 (11:40 -0700)]
aiaiai: change dispatcher hook to email hook
This patch modifies the current dispatcher hook to the new model of hook
(reading hook contents from stdout of hook). In addition, it also moves
the hook into aiaiai-email-test-patchset instead of putting it in the
dispatcher-helper. This helps simplify the hook process overall.
Amended by Artem.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Thu, 27 Mar 2014 18:40:05 +0000 (11:40 -0700)]
email: rename error_test_patchset_failed
Since this function could be used to generically refer to any internal
error, modify the name and description. We will be using this function
in a later patch to indicate an internal error in a hook as well.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Thu, 27 Mar 2014 18:40:04 +0000 (11:40 -0700)]
aiaiai: split sed project listing from list_projects
In order to allow other places such as hooks to list the projects,
seperate out the sed script into its own function. This enables less
code duplication if another script wants to list all projects in a
cfgfile.
Artem: re-named the function for a bit better readability.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Thu, 27 Mar 2014 18:40:03 +0000 (11:40 -0700)]
email: add canonical_url variable for list_projects
If supplied, this url can be used to add a [git clone -b <branch> <url>]
section to the list_projects output, which indicates to the user how to
clone the specified project. This can be useful to enable users to more
easily see how to start working on a project.
The canonical url is not used by aiaiai as an access point. It mainly is
useful if the local clones that aiaiai uses are setup as --mirror
clones.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Thu, 27 Mar 2014 18:40:02 +0000 (11:40 -0700)]
email: don't allow pcfg_branch to default
Since we don't allow pcfg_path to default, it makes little sense to
allow pcfg_branch to default. Also, it would be better to have a die
check on branch, so that we fail if the project doesn't specify the
branch.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Thu, 27 Mar 2014 18:39:59 +0000 (11:39 -0700)]
aiaiai: fixup srcdir in email and gerrit scripts
$srcdir should represent the current directory where the actual shell
file is located. This requires also updating $PATH so that it points to
the correct directories. This is somewhat confusing since we have to add
'..' to a section, but it fixes the switch to dash check for email
scripts which previously was broken. It also makes everything more
consistent.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Sat, 15 Mar 2014 00:16:01 +0000 (17:16 -0700)]
hook: add support for X-Aiaiai-Project
Rather than determining the project strictly from the +project portion
of the aiaiai address, also allow the dispatcher hook to insert the
project hook into the mbox header. This could be used to allow advanced
configuration of the project, such as by checking patch diffs to see
which files are modified.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Sat, 15 Mar 2014 00:16:00 +0000 (17:16 -0700)]
aiaiai: add initial support for dispatcher hook
This patch adds some initial support for a dispatcher hook which could
be used to add custom headers, which will (later) be interpreted by
aiaiai to include specialized settings. A follow-on patch will add the
first initial custom header.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Sat, 15 Mar 2014 00:15:59 +0000 (17:15 -0700)]
doc: add CONFIGURATION file describing the cfgfile
This patch adds a CONFIGURATION file which describes the aiaiai
configuration file. This will be expanded upon in future commits which
introduce the aiaiai hooks.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Sat, 15 Mar 2014 00:15:58 +0000 (17:15 -0700)]
sh-function: add insert_header function
Since we use a special trick to insert header information into the mbox,
this could be extraced into a function in aiaiai-sh-functions. This will
help future work which will want to add multiple headers into this mbox
file, using the same file descriptor trick.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Mon, 10 Mar 2014 23:46:14 +0000 (16:46 -0700)]
email-test-patchset: parse general configuration settings earlier
This patch modifies the email patch testing program so that it parses
the general configuration settings as soon as it reads the configuration
file location. It also modifies the script so that it will use readlink
on the path instead of just blindly using the path. This is taken from
the LDA which does this as well.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Mon, 10 Mar 2014 23:46:13 +0000 (16:46 -0700)]
email: add "debug" section to configuration file
This patch further extends the configuration file so that it has a debug
section which currently includes replacements for test-mode and preserve
options. These options are very useful for debugging aiaiai, but today
cannot easily be changed without modifying the command line. Instead,
remove them from the options and include them via the configuration
file.
I chose not to add "verbose" to this list, because we actually use
verbose earlier than we parse the configuration. It may still be worth
coming up with some interface where command line options overwrite
configuration options, but I wasn't sure the best way to handle this
yet.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Mon, 10 Mar 2014 23:46:12 +0000 (16:46 -0700)]
aiaiai-email-test-patchset: move -- to end of all options
A previous patch added kmake_opts to the configuration file, and when
passing this into aiaiai-test-patchset, the -- field which ends option
scanning was not correctly moved to the end of the options arguments.
This would result in incorrect behavior if anyone tried to use this
option.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
The targets parameter was incorrectly failing to add the --targets,
because the substitution did not have a +. This patch corrects the
parameter substitution so that the targets will correctly include the
required option for use with aiaiai-test-patchset directly.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Mon, 10 Mar 2014 23:46:09 +0000 (16:46 -0700)]
aiaiai-test-patchset: add missing colon on targets option
The targets option recently added was broken. It did not properly handle
a value for the argument, because no colon was present. This patch fixes
the getopt line so that target will infact select an option.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Sat, 1 Mar 2014 00:00:19 +0000 (16:00 -0800)]
aiaiai: don't allow arbitrary validator command
Since aiaiai-email-test-patchset is the only validator that makes sense,
and we should extend it if new features are desired, we don't need to
support multiple validators, as this is clunky, and difficult to
configure for the user. Now that the configuration file supports all the
options from the aiaiai-email-test-patchset, we can just directly call
it instead of having to use a passed in parameter.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Jacob Keller [Fri, 7 Mar 2014 01:12:51 +0000 (17:12 -0800)]
aiaiai-email: move test-patchset options into the configuration file
This patch modifies the configuration file to include most of the
options around the validator command, so that those no longer have to be
specified manually. This does break compatability with the current
validator setup, since those old options must now be specified on the
command line.
- v2
* Fix typo in coccinelle variable
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Jacob Keller [Fri, 7 Mar 2014 00:31:12 +0000 (16:31 -0800)]
aiaiai-email: add [defaults] section to config
This patch adds a [defaults] section for the configuration file which
allows setting shared project settings which might be overridden
per-project. This will be used more in a follow-on patch which adds
additional configuration options.
- v2
* put ini_config_is_set into the libshell file
* use __ instead of _ to indicate internal variable
* update comment to better reflect per-project and default variable
interaction
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Jacob Keller [Fri, 7 Mar 2014 00:22:58 +0000 (16:22 -0800)]
aiaiai-sh-functions: re-order options to match example config
This patch re-orders the parsing for the standard configuration
variables, in order to match what is in our example configuration. It
also updates the header comment to match.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Jacob Keller [Thu, 6 Mar 2014 22:08:48 +0000 (14:08 -0800)]
aiaiai-sh-functions: remove [email] section from cfgfile
This patch is part of a series to improve user friendliness regarding
options for aiaiai-email-test-patchset, and improving use of the
configuration file. This patch removes the seperate [email] section, as
it really doesn't have any use. Just put these inside the global
section, since email and global really have no different meaning.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Jacob Keller [Thu, 27 Feb 2014 21:24:16 +0000 (13:24 -0800)]
Add support for choosing build targets
Since aiaiai-make-kernel has support for multiple targets, we should
extend the aiaiai-test-patchset to have a parameter which enables
selection of targets. By default, we should use the "all" target, but
enable the user to select what target they want. An example of a
different or additional target includes "namespacecheck"
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Thu, 13 Feb 2014 22:28:19 +0000 (14:28 -0800)]
email-test-patchset: fix pcfg_reply_to_all
Through the course of several refactors, one of the patches
accidentallly broke reply_to_all functionality. In the older code, we
did a negative check, and if reply_to_all was not set, we would then
unset $to in order to prevent the compose_email function/code from
adding any Cc addresses.
However, one of the refactors missed this negative check, assumed it was
positive, and then changed the semantics so that reply_to_all = 0
caused reply behavior rather than otherwise.
This patch corrects the given commit by making it check for 1, instead
of not 1.
Artem: amend an weird comment while on it
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Fixes: 092f2ca246 "(email-test-patchset: further improve readability)" Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Tue, 11 Feb 2014 17:50:38 +0000 (09:50 -0800)]
email-test-patchset: grab all To and Cc headers.
This change modifies email-test-patchset to grab every To and Cc header
instead of just the first. Because refernces to to and cc except email
lists (comma seperated), we use merge_addresses with an empty second
parameter. This works due to recent changes to the merge_addresses
function see in a previous patch in this series.
Artem: amend the patch a tiny bit.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Tue, 11 Feb 2014 17:50:33 +0000 (09:50 -0800)]
sh-functions: add fetch_all_headers function
This patch adds a fetch_all_headers function which is used to get every
occurrence of an mbox header, rather than just the first. In combination
with the change to merge_addresses, this can be used to grab every Cc,
and To address.
Artem: massage comments
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Jacob Keller [Tue, 11 Feb 2014 17:50:28 +0000 (09:50 -0800)]
email-sh-functions: make merge addresses more flexible
This patch combines effort from a previous patch, as well as worm from
Artem to make merge_addresses more flexible. First, it enables multiple
line inputs, so that it can convert a list of addresses per line into a
single comma seperated string. In addition, make the arguments optional,
so it will merge as many argumements as it is passed instead of limiting
to 1 or 2.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Artem Bityutskiy [Fri, 7 Feb 2014 15:24:12 +0000 (17:24 +0200)]
Add some missing quotes
We are trying to use quotes every time we expand variables. Let's be consistent
and do this everywhere. This patch adds quotes to several paces where we missed
them.
Artem Bityutskiy [Fri, 7 Feb 2014 15:04:26 +0000 (17:04 +0200)]
Start using cocci scripts from the kernel tree
Start using coccinelle scripts from the kernel tree we build against. This is
better than carrying our own copies of coccinelle scripts, since they tend to
become out-of-date.
Artem Bityutskiy [Fri, 7 Feb 2014 13:32:32 +0000 (15:32 +0200)]
Compile helper tools if necessary
Some of our tools like 'remap-log' have to be compiled. This is not a problem
when using Aiaiai from a package, because it has the compiled version of the
tool, but this is a problem when using Aiaiai from sources.
Introduce a function which tries to compile the internal tools. This is
supposed to improve user experience with Aiaiai.
Remove the top-level Makefile since it is not really needed after this change.
Artem Bityutskiy [Fri, 7 Feb 2014 13:00:36 +0000 (15:00 +0200)]
Move internal scripts to the "helper" subdirectory
This patch implements the following TODO entry:
Re-arrange the source code structure so that the internal scripts would all sit
in a separate sub-directory. Internal scripts are those which users are not
supposed to run directly. For example, 'aiaiai-locker' is one of them.
Artem Bityutskiy [Fri, 7 Feb 2014 12:20:35 +0000 (14:20 +0200)]
Rename 'external' to 'helpers'
I would like to move all the internal tools to a separate directory. Let's call
it 'helpers'. The 'external' directory now contains all the tools which came
from an external project. Probably this is not the best way to lay-out our
files. Let's just re-name it to helpers. Later we'll move more stuff to
'helpers'.
Jacob Keller [Thu, 9 Jan 2014 23:28:37 +0000 (15:28 -0800)]
email-lda: process all series directories at the end
Sometimes the email-lda program can misbehave and a series could end up
becoming "complete" but the LDA does not queue it when the last patch was
added. This patch modifies the LDA so that it will check all series at the end
of each mail and queue them, rather than only checking a series when some
thing is added.
The primary benefit of this is if a series does not properly get queued, the
administrator can come in and manually add the patches to the series
directory. Then the series will properly be "queued" after the next email
triggers the LDA.
This does come at a slight hit in terms of performance as we now have to check
whether series are complete for every email. However, the gain in being able
to easily fixup and get a series queued again when errors occur is much more
useful.
Artem: this patch did not really work properly, because it was missing the
"series_is_complete" check. I also did some re-factoring, and tested the final
version. Seems to work fine. I also got rid of bash-only constructs.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Artem Bityutskiy [Thu, 6 Feb 2014 14:59:34 +0000 (16:59 +0200)]
email-lda: make 'mbox' to be a local variable
Make 'mbox' to be a local variable in all the functions. This is nicer and
saver, and makes the code easier to change. The only plase where we use the
global variable is the clean-up handler.
Artem Bityutskiy [Thu, 6 Feb 2014 14:13:11 +0000 (16:13 +0200)]
email-lda: fix copy-paste error
The 'reject_and_reply' function accepts the mbox to act upon as the parameter,
and internally names it "file". However, in few places it actually uses "mbox"
instead of "file", which happens to work fine, but is generally a bug.
Artem Bityutskiy [Thu, 6 Feb 2014 14:08:12 +0000 (16:08 +0200)]
email-lda: improve readability some more
This patch adds some useful commentaries. Also, it partially makes the mbox
parameter to be non-global. Next patches will do more in this direction. This
idea is to improve code readability but making it using less global variables.
Ideally there should be none, but I am not sure I'll have time to do that.
Let's do this for mbox and probably "m" and "n". See next patches.
Artem Bityutskiy [Thu, 6 Feb 2014 13:48:52 +0000 (15:48 +0200)]
email-lda: centralise naming decision
The 'sec_name' function basically defines the name of the mbox file or a
directory for use. Currently we use the date as the prefix. Later we may change
this.
Anyway, make 'sec_name' always use the current date, isntead of getting it from
the caller. This should make things a bit easier to understand.
Artem Bityutskiy [Thu, 6 Feb 2014 13:40:22 +0000 (15:40 +0200)]
email-lda: fix name generator
This patch fixes the 'seq_name' finction which generates a file name to use for
storing an mbox. Currently it just appends a counter to the end. And it uses
the "-" for this, like something-counter, but sometimes it uses "_", which is a
bug. Let's fix it and use "-" everiwhere.
Artem Bityutskiy [Thu, 6 Feb 2014 13:28:37 +0000 (15:28 +0200)]
email-lda: make the queued file name saner
When aiaiai-email-lda collects all patches in the series, it concatenates them
into a single file, and calls 'queue_mboxfile'. This function basically saves
a copy of the mbox in the "queue_saved" directory, and then moves it to the
queue directory.
The name of the file will be like this: date_m_of_n-counter.
However, the "m" part makes no sense at all. It is equivalent to the mumber of
the patch which came last. For example, if we have a series of 2 patches, and
path 1 came last, the name will be:
date_1_of_2-counter
Or if patch 2 came last, the name will be
date_2_of_2-counter
And this makes no sense. Just remove the "m" part from the name and make it to
be:
Artem Bityutskiy [Thu, 6 Feb 2014 13:08:21 +0000 (15:08 +0200)]
email-lda: use verbose output for everything
The current split does not make much sence: we always print some cruft, but
with --verbose we print more of it. Let's just make all the cruft be under the
--verbose option.
Artem Bityutskiy [Thu, 6 Feb 2014 08:59:04 +0000 (10:59 +0200)]
email-lda: always use the cover letter
The previous patch from Jacob only works when there is more than one patch in
the series, but did not work for the situation when there is a cover letter
(0/1) and a single patch (1/1). This patch fixes the situation.
Jacob Keller [Thu, 9 Jan 2014 23:28:32 +0000 (15:28 -0800)]
email-lda: use cover letter subject if available
Often, a patch series will be sent with a cover letter which describes the
series and is given the '0/n' patch number. Rather than dropping these
patches, keep track of them in series. We do this by checking whether the 1/n
patch has an "In-Reply-To" header. This usually means that their will be an
associated cover letter. Once this is found, rather than sending the patch
directly on to the test program, we use the cover letter's subject by using
formail to add a special "Series-Subject". This makes it so that the
aiaiai-email will end up as a reply to the cover letter which is a bit more
aesthetically pleasing to users.
To help this process, a new "series_is_complete" function is added in place of
the original series complete check. This function reports whether the series
is completely queued, for both cases of cover letter and no cover letter.
Artem:
I've modified this patch.
1. Removed a couple of '[[' bash constructs
2. Also preserved the Message ID of the cover letter to make the reply not only
have the subject of the cover letter, but also refer the cover letter
properly.
3. Use common prefix for the special cover letter e-mail headers that we add:
X-Aiaiai-Cover-Letter-Subject
X-Aiaiai-Cover-Letter-Message-Id
4. Dropped sponge dependency. Generally, this is a nice tool, and I did not
hear about it before. I'll definitely start using it when constructing pipes
in the console. But this tool is not that well-known, so it is not typically
present in systems. E.g., mine do not have it. And I decided to avoid adding
another dependency, we already require a lot of tools. Instead, I used
another trick, which is actual just as elegant as using sponge. Well, needs
more lines of code, but it is really nice trick anyway, I like it.
I just open the mbox file, then unlink it, which means it won't be visible
in the file-system, but won't be deleted since we have the file descriptor
opened. Then I pipe the data from the file descriptor to 'formail', and
redirect the results to the mbox file. So a new mbox file will be created
and all the new contents will be there. I think this is elegant, and does
not add a new dependency.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
projects / users / dedekind / aiaiai.git / log