Edward Z. Yang [Mon, 12 Oct 2009 20:32:50 +0000 (16:32 -0400)]
More bugfixes from running live.
* Add a few more wgCacheEpoch resolutions
* Make prepareMerge work for \r\r\r\n
* Add flush and fails to Report. This needs further work.
* Require reason for wizard blacklist
* Cleanup /dev/shm/wizard at the start of mass runs
* Special case a few more errors
* Report reason for blacklisting in stdout.
* Add global finally for removing /dev/shm directories
* Add QuotaParseError
Edward Z. Yang [Mon, 12 Oct 2009 02:52:52 +0000 (22:52 -0400)]
Major updates to resolution code from runs.
* Augment MediaWiki conflict resolution with correct version for
end of line
* Add prepareMerge to application for pre-merge resolution
helpers (mostly for line-ending problems)
* Fix MediaWiki resolveConflicts parsing of ls-files output,
and another unrelated bug
* Add space between logging and summary outputs
* Move backup back to correct location, and perform size calculation
twice
* UpgradeVerificationFailure gets useful output
* /dev/shm/wizard gets chmod'ed 0777
* Fix resolution algorithm to handle each conflict block individually.
Test cases augmented accordingly.
Edward Z. Yang [Sat, 3 Oct 2009 05:18:38 +0000 (01:18 -0400)]
Implement web verification for pre-upgrade and post-upgrade.
* checkWeb and verifyWeb for Application and Deployment
* Fix bug if MediaWiki update.php doesn't return any output
* Return backup name from d.backup(), and print it from 'wizard backup'
* Ignore keyboard interrupts when running upgrades
* Created get_web_host_and_path() function in wizard.scripts
* Created fetch() in wizard.util
Edward Z. Yang [Sat, 3 Oct 2009 01:55:15 +0000 (21:55 -0400)]
Backup on upgrade, add tests.
* Fix bug in 'wizard backup'
* Make 'wizard install' perform a commit
* Make 'wizard install app-oldversion' do a hard reset
* Add backup() and restore() to Deployment
* Have sql credentials read environmental variables
Edward Z. Yang [Sat, 3 Oct 2009 00:04:52 +0000 (20:04 -0400)]
Implement 'wizard backup'. Other minor refactorings:
* Moved SQL credential detection to new wizard.scripts module
* Made error messages for wizard install a little better
* Make help message not as long for environment variables
* Move safe_unlink to wizard.util
Edward Z. Yang [Tue, 25 Aug 2009 03:42:56 +0000 (23:42 -0400)]
Various refinements from our mass-upgrade run.
* Add missing command entry for mass-upgrade
* Increase concurrent processes by default
* Add UpgradeFailure checking for MediaWiki
* Change install and upgrade to take a distutils.version.LooseVersion
* Propagate --force from mass-upgrade to upgrade
Edward Z. Yang [Fri, 21 Aug 2009 06:04:00 +0000 (02:04 -0400)]
Refactor, increase error coverage, more sanity checks.
* Deal with when source is a binary for test scripts.
* Move chdir from command to util module
* Reclassify util.chdir exceptions to be subclasses of IOError
* Migrate forces earlier (for the sake of better code)
* Deploy scripts now have verify methods which throw exceptions
on event of errors. This includes newly implemented
DAG validation (verifyGit)
* Compactify some open().read()/write() calls
* Add tag property to ApplicationVersion
* Change shell.drop_priviledges() API to require a directory
as first parameter.
Edward Z. Yang [Thu, 20 Aug 2009 16:34:13 +0000 (12:34 -0400)]
Enhancements from our first migration.
* Make MediaWiki regex more lenient with trailing newlines
* Add check to see if application is configured; if it isn't
bail out and don't migrate.
* Perform increments early enough so that numeric IDs are unique
* Fix bug in increment printing
* fdopen() doesn't work, so simply create a blank lockfile
* Make force removal log message more descriptive
* Remove .scripts/old-version code
* Prefer git describe method, but if it doesn't work switch
to .scripts-version
* Make shell return appropriate values during dry runs
* Make call error more descriptive (this messes with exception
counting, however.)
Edward Z. Yang [Wed, 19 Aug 2009 22:34:50 +0000 (18:34 -0400)]
Fix bugs, add better logging.
* dry_run semantics were improperly being triggered
due to a logic bug.
* Created warnings.log and errors.log files, which
are machine friendly lists of warning'd and error'd
installs.
* Add the shorten number to make it easy for someone
to grab a specific logfile.
* Add locking to migration; this means better error
messages too
* Make call errors more expressive.
Edward Z. Yang [Wed, 19 Aug 2009 21:43:05 +0000 (17:43 -0400)]
Fix exceptions, quiet subprocesses, change logging and order.
* old_excepthook() didn't work, so print out the error using logging.
* Fix bug in --limit
* Change log file name generation algo, to be 0001-afs-etc
* Pass --quiet to subprocesses, in view of future change of checking
for non-empty stderr, and taking this to mean, "There were warnings."
* Sort directories before iterating threw them
Edward Z. Yang [Wed, 19 Aug 2009 18:58:03 +0000 (14:58 -0400)]
Update migration semantics to work better.
* .scripts-version is kept in its original place to
make things more robust
* Backing up of .scripts and .git directories actually works
* Attempt to do all validation before doing any --force
actions, to reduce broken migrations
* Remove .scripts/version generation
* More robust already migrated checks (check for .git and .scripts
independently)
* Fix error in error handling code
* Turn on logging of effectful sh.eval() calls in upgrade
* Make parametrization more resistant against missing files
* Add log parameter, remove differing log messages based on eval
* Make pristine_tag and scripts_tag not braindead, fixing a bunch
of bugs.
Edward Z. Yang [Wed, 19 Aug 2009 06:01:54 +0000 (02:01 -0400)]
Fix numerous bugs from our test runs.
- Only create subdirectories if we're forcing.
- Add support for --limit on mass-migrate
- Match for AlreadyMigratedError as well as fully qualified name.
- Add --force on mass-migrate
- Improve logging messages to remove unnecessary cruft
and add important information
- Move .git to .git.bak, for record-keeping purposes
- Be more selective about logging for shell
- Resolve error case for exception parsing
- Make exceptions show up by removing --quiet
scripts.mit.edu / wizard.git / log
