Unit tests can be run by running ``nosetests`` in the Wizard root
directory (the directory that contains a :file:`wizard` and :file:`bin`
-directory). These should run and pass on all platforms.
+directory). These should run and pass on all platforms. Test files
+tend to live in :file:`tests` directories under :file:`wizard`.
Doctests
--------
Certain pure functions contain doctests for their functionality. You
can run them using ``sphinx-build -b doctest doc doc/_build`` in
the Wizard root directory. These should run and pass on all platforms.
+These are found inside of docstrings and are noted by a ``>>>`` marker.
+They should be written for simple, self-contained functions that would
+benefit from an example code usage.
Acceptance tests
----------------
they run Wizard commands and try to see if any of the commands return
a non-zero exit code (future development may allow for pre-conditions and
post-conditions to be checked). The test scripts are located in the
-:file:`tests` directory, and are identifiably by their prefix ``test-``
-and their suffix ``.sh``. They should run out of the box on
+:file:`tests` directory, and are identifiably by their suffix
+``-test.sh``. They should run out of the box on
scripts servers, so long as the Wizard source tree is inside of your
:file:`web_scripts` directory, and require some configuration if you
plan on running them locally.
-Custom configuration can be specified in the :file:`config` file (located
-at :file:`tests/config`. This is actually a Bash script to be sourced
-by the real test script (:file:`tests/setup`), which exports various
-environment variables that Wizard will use during installation.
+These test scripts are the key to ensuring that Wizard is functioning
+properly, and exercise most of the common code paths (and a few, though
+not all, of the uncommon ones). They're organized by application name
+and then test name. Every application should have a :file:`appname-install-test.sh`
+which can be run without any parameters; you can also specify a version
+number to test installing a particular version installation. There is
+also :file:`appname-upgrade-test.sh`, which requires specifying a version
+number and tests upgrading from that version to the latest version in
+the repository.
-Here is a sample file::
+.. note::
+
+ There are substantially more MediaWiki test scripts than for other applications,
+ a mixture of MediaWiki-specific tests and more general tests of functionality.
+ There may be some possible restructuring to reduce duplication for the
+ general tests.
+
+There are also utility scripts :file:`all.sh`, which takes a application
+name and a version number as parameters, and runs all of the corresponding
+test scripts, as well as :file:`clean.sh` which removes all test directories.
+
+Custom configuration can be specified in the :file:`config` file (located at
+:file:`tests/config`). This is actually a Bash script to be sourced by the
+real test script (:file:`tests/setup`), which exports various environment
+variables that Wizard will use during installation. You should only need to
+set these variables if you're attempting to run these tests off of a
+non-scripts server.
+
+Here is a sample :file:`config` file::
+
+ MYSQL_ARGS="-uroot -ppassword"
export WIZARD_WEB_HOST="localhost"
export WIZARD_WEB_PATH="/wizard/tests/$TESTDIR"
- export WIZARD_MYSQL_HOST="localhost"
- export WIZARD_MYSQL_USER="root"
- export WIZARD_MYSQL_PASSWORD="password"
- export WIZARD_MYSQL_DB="wizard_test_$TESTID"
+ export WIZARD_DSN="mysql://root:password@localhost/wizard_test_$TESTID"
export WIZARD_EMAIL="bob@example.com"
-You will need to specify all of these environment variables. :envvar:`WIZARD_WEB_HOST`
-and :envvar:`WIZARD_WEB_PATH` indicate Wizard's configuration with respect to
-your web server. ``http://$WIZARD_WEB_HOST/$WIZARD_WEB_PATH`` will be the location
-that a newly installed application will be accessible. You will notice that
-we used :envvar:`TESTDIR`; this is the directory that will be created in
-:file:`tests` for the application. :envvar:`WIZARD_MYSQL_HOST`,
-:envvar:`WIZARD_MYSQL_USER`, :envvar:`WIZARD_MYSQL_PASSWORD` and :envvar:`WIZARD_MYSQL_DB`
-are standard configuration variables for accessing a local MySQL database. You can use
-:envvar:`TESTID` to uniquely identify any particular test. Finally, :envvar:`WIZARD_EMAIL`
-is any email address you own that will be configured as an administrative email.
-
-It may be useful to include a little bit of code to handle dropping and creating
-databases. Here is a sample::
-
- MYSQL_ARGS="-uroot -prootpassword"
- mysql $MYSQL_ARGS -e "DROP DATABASE \`$WIZARD_MYSQL_DB\`;" || true
- mysql $MYSQL_ARGS -e "CREATE DATABASE \`$WIZARD_MYSQL_DB\`;"
+You will need to specify all of the environment variables in the
+:file:`config`. Those prefixed with ``WIZARD`` are directly used by Wizard,
+while the ``MYSQL`` environment variables are used if a test script wants to
+interactive directly with a MYSQL database. We don't quite have a good story
+for alternative databases in test scripts.
+
+* :envvar:`WIZARD_WEB_HOST` and :envvar:`WIZARD_WEB_PATH` indicate Wizard's
+ configuration with respect to your web server.
+ ``http://$WIZARD_WEB_HOST/$WIZARD_WEB_PATH`` will be the location that a newly
+ installed application will be accessible. You will notice that we used
+ :envvar:`TESTDIR`; this is the directory that will be created in :file:`tests`
+ for the application.
+* :envvar:`WIZARD_DSN` is the database source name, which
+ should be used to access a MySQL database (as Wizard does not contain support
+ for any other database system yet.) You can use :envvar:`TESTID` to uniquely
+ identify any particular test.
+* :envvar:`WIZARD_EMAIL` is any email address you
+ own that will be configured as an administrative email.
scripts.mit.edu / wizard.git / blobdiff