Add detectVersionFromGit utility method

[wizard.git] / doc / testing.rst

diff --git a/doc/testing.rst b/doc/testing.rst
index 81d91fb6bae049a69738a90fa65e6a208fdd6aa1..6a4b8e0f4bbcd1c84c0c4e30f81ea2e765c59d56 100644 (file)
--- a/doc/testing.rst
+++ b/doc/testing.rst
@@ -12,7 +12,8 @@ Unit tests
 
 Unit tests can be run by running ``nosetests`` in the Wizard root
 directory (the directory that contains a :file:`wizard` and :file:`bin`
-directory).  These should run and pass on all platforms.
+directory).  These should run and pass on all platforms.  Test files
+tend to live in :file:`tests` directories under :file:`wizard`.
 
 Doctests
 --------
@@ -20,6 +21,9 @@ Doctests
 Certain pure functions contain doctests for their functionality.  You
 can run them using ``sphinx-build -b doctest doc doc/_build`` in
 the Wizard root directory.  These should run and pass on all platforms.
+These are found inside of docstrings and are noted by a ``>>>`` marker.
+They should be written for simple, self-contained functions that would
+benefit from an example code usage.
 
 Acceptance tests
 ----------------
@@ -28,12 +32,33 @@ These are the most useful metrics of whether or not Wizard is working:
 they run Wizard commands and try to see if any of the commands return
 a non-zero exit code (future development may allow for pre-conditions and
 post-conditions to be checked).  The test scripts are located in the
-:file:`tests` directory, and are identifiably by their prefix ``test-``
-and their suffix ``.sh``.  They should run out of the box on
+:file:`tests` directory, and are identifiably by their suffix
+``-test.sh``.  They should run out of the box on
 scripts servers, so long as the Wizard source tree is inside of your
 :file:`web_scripts` directory, and require some configuration if you
 plan on running them locally.
 
+These test scripts are the key to ensuring that Wizard is functioning
+properly, and exercise most of the common code paths (and a few, though
+not all, of the uncommon ones).  They're organized by application name
+and then test name.  Every application should have a :file:`appname-install-test.sh`
+which can be run without any parameters; you can also specify a version
+number to test installing a particular version installation.  There is
+also :file:`appname-upgrade-test.sh`, which requires specifying a version
+number and tests upgrading from that version to the latest version in
+the repository.
+
+.. note::
+
+    There are substantially more MediaWiki test scripts than for other applications,
+    a mixture of MediaWiki-specific tests and more general tests of functionality.
+    There may be some possible restructuring to reduce duplication for the
+    general tests.
+
+There are also utility scripts :file:`all.sh`, which takes a application
+name and a version number as parameters, and runs all of the corresponding
+test scripts, as well as :file:`clean.sh` which removes all test directories.
+
 Custom configuration can be specified in the :file:`config` file (located at
 :file:`tests/config`).  This is actually a Bash script to be sourced by the
 real test script (:file:`tests/setup`), which exports various environment