Edward Z. Yang [Sat, 3 Oct 2009 05:18:38 +0000 (01:18 -0400)]
Implement web verification for pre-upgrade and post-upgrade.
* checkWeb and verifyWeb for Application and Deployment
* Fix bug if MediaWiki update.php doesn't return any output
* Return backup name from d.backup(), and print it from 'wizard backup'
* Ignore keyboard interrupts when running upgrades
* Created get_web_host_and_path() function in wizard.scripts
* Created fetch() in wizard.util
Edward Z. Yang [Sat, 3 Oct 2009 01:55:15 +0000 (21:55 -0400)]
Backup on upgrade, add tests.
* Fix bug in 'wizard backup'
* Make 'wizard install' perform a commit
* Make 'wizard install app-oldversion' do a hard reset
* Add backup() and restore() to Deployment
* Have sql credentials read environmental variables
Edward Z. Yang [Sat, 3 Oct 2009 00:04:52 +0000 (20:04 -0400)]
Implement 'wizard backup'. Other minor refactorings:
* Moved SQL credential detection to new wizard.scripts module
* Made error messages for wizard install a little better
* Make help message not as long for environment variables
* Move safe_unlink to wizard.util
Edward Z. Yang [Tue, 25 Aug 2009 03:42:56 +0000 (23:42 -0400)]
Various refinements from our mass-upgrade run.
* Add missing command entry for mass-upgrade
* Increase concurrent processes by default
* Add UpgradeFailure checking for MediaWiki
* Change install and upgrade to take a distutils.version.LooseVersion
* Propagate --force from mass-upgrade to upgrade
Edward Z. Yang [Fri, 21 Aug 2009 06:04:00 +0000 (02:04 -0400)]
Refactor, increase error coverage, more sanity checks.
* Deal with when source is a binary for test scripts.
* Move chdir from command to util module
* Reclassify util.chdir exceptions to be subclasses of IOError
* Migrate forces earlier (for the sake of better code)
* Deploy scripts now have verify methods which throw exceptions
on event of errors. This includes newly implemented
DAG validation (verifyGit)
* Compactify some open().read()/write() calls
* Add tag property to ApplicationVersion
* Change shell.drop_priviledges() API to require a directory
as first parameter.
Edward Z. Yang [Thu, 20 Aug 2009 16:34:13 +0000 (12:34 -0400)]
Enhancements from our first migration.
* Make MediaWiki regex more lenient with trailing newlines
* Add check to see if application is configured; if it isn't
bail out and don't migrate.
* Perform increments early enough so that numeric IDs are unique
* Fix bug in increment printing
* fdopen() doesn't work, so simply create a blank lockfile
* Make force removal log message more descriptive
* Remove .scripts/old-version code
* Prefer git describe method, but if it doesn't work switch
to .scripts-version
* Make shell return appropriate values during dry runs
* Make call error more descriptive (this messes with exception
counting, however.)
Edward Z. Yang [Wed, 19 Aug 2009 22:34:50 +0000 (18:34 -0400)]
Fix bugs, add better logging.
* dry_run semantics were improperly being triggered
due to a logic bug.
* Created warnings.log and errors.log files, which
are machine friendly lists of warning'd and error'd
installs.
* Add the shorten number to make it easy for someone
to grab a specific logfile.
* Add locking to migration; this means better error
messages too
* Make call errors more expressive.
Edward Z. Yang [Wed, 19 Aug 2009 21:43:05 +0000 (17:43 -0400)]
Fix exceptions, quiet subprocesses, change logging and order.
* old_excepthook() didn't work, so print out the error using logging.
* Fix bug in --limit
* Change log file name generation algo, to be 0001-afs-etc
* Pass --quiet to subprocesses, in view of future change of checking
for non-empty stderr, and taking this to mean, "There were warnings."
* Sort directories before iterating threw them
Edward Z. Yang [Wed, 19 Aug 2009 18:58:03 +0000 (14:58 -0400)]
Update migration semantics to work better.
* .scripts-version is kept in its original place to
make things more robust
* Backing up of .scripts and .git directories actually works
* Attempt to do all validation before doing any --force
actions, to reduce broken migrations
* Remove .scripts/version generation
* More robust already migrated checks (check for .git and .scripts
independently)
* Fix error in error handling code
* Turn on logging of effectful sh.eval() calls in upgrade
* Make parametrization more resistant against missing files
* Add log parameter, remove differing log messages based on eval
* Make pristine_tag and scripts_tag not braindead, fixing a bunch
of bugs.
Edward Z. Yang [Wed, 19 Aug 2009 06:01:54 +0000 (02:01 -0400)]
Fix numerous bugs from our test runs.
- Only create subdirectories if we're forcing.
- Add support for --limit on mass-migrate
- Match for AlreadyMigratedError as well as fully qualified name.
- Add --force on mass-migrate
- Improve logging messages to remove unnecessary cruft
and add important information
- Move .git to .git.bak, for record-keeping purposes
- Be more selective about logging for shell
- Resolve error case for exception parsing
- Make exceptions show up by removing --quiet
Edward Z. Yang [Wed, 12 Aug 2009 22:45:20 +0000 (18:45 -0400)]
Implement test suite for Wizard; includes numerous API changes.
* Added environment variables for all installation arguments.
* Added --srv-path option, for specifying local repositories.
* Added environment variables for some common command options,
namely WIZARD_SRV_PATH, WIZARD_VERBOSE and WIZARD_DEBUG.
* API change: 'wizard install --app APP DIR' to 'wizard
install APP DIR'
* Changed Application.repository to be a method that takes
srv_path as an argument (since this can now vary). Also
look for "$APP/.git" if "$APP.git" doesn't exist.
* Added AppVersion.pristine_tag
* Change Deployment.app_version algorithm to use
'git describe' over .scripts/version
* Miscellaneous typo fixes
Edward Z. Yang [Fri, 7 Aug 2009 06:37:46 +0000 (02:37 -0400)]
Implement 'wizard install', with other improvements.
* Chmod config directory to be 777 for mediawiki autoinstall
* Rewrite configure to use applications() function
* Replace wizard_bin with "wizard", which then gets converted
* Add support for interactive mode
* Remove wizard.util hack by splitting out into git module
* Allow PythonCallError to work even when stderr is None
* Update TODO
scripts.mit.edu / wizard.git / log
