wizard.app

Plumbing object model for representing applications we want to install. This module does the heavy lifting, but you probably want to use wizard.deploy.Deployment which is more user-friendly. You’ll need to know how to overload the Application class and use some of the functions in this module in order to specify new applications.

To specify custom applications as plugins, add the following entry_points configuration:

[wizard.app]
yourappname = your.module:Application
otherappname = your.other.module:Application

Note

Wizard will complain loudly if yourappname conflicts with an application name defined by someone else.

There are some submodules for programming languages that define common functions and data that may be used by applications in that language. See:

Classes

class wizard.app.Application(name)

Represents an application, i.e. mediawiki or phpbb.

Note

Many of these methods assume a specific working directory; prefer using the corresponding methods in wizard.deploy.Deployment and its subclasses.

backup(deployment, outdir, options)

Run for ‘wizard backup’ and upgrades to backup database schemas and other non-versioned data in an application. outdir is the directory that backup files should be placed. This assumes that the current working directory is the deployment. Subclasses should provide an implementation, even if it is a no-op.

Note

Static user files may not need to be backed up, since in many applications upgrades do not modify static files.

checkConfig(deployment)

Checks whether or not an autoinstall has been configured/installed for use. Assumes that the current working directory is the deployment. Subclasses should provide an implementation.

checkDatabase(deployment)

Checks if the database is accessible.

checkWeb(deployment)

Checks if the autoinstall is viewable from the web. Subclasses should provide an implementation.

Note

Finding a reasonable heuristic that works across skinning choices can be difficult. We’ve had reasonable success searching for metadata. Be sure that the standard error page does not contain the features you search for. Try not to depend on pages that are not the main page.

checkWebPage(deployment, page, outputs=[], exclude=[])

Checks if a given page of an autoinstall contains a particular string.

database

Name of the database that this application uses, i.e. mysql or postgres. If we end up supporting multiple databases for a single application, there should also be a value for this in wizard.deploy.Deployment; the value here is merely the preferred value.

deprecated_keys

Keys that are used in older versions of the application, but not for the most recent version.

detectVersion(deployment)

Checks source files to determine the version manually. This assumes that the current working directory is the deployment. Subclasses should provide an implementation.

detectVersionFromFile(filename, regex)

Helper method that detects a version by using a regular expression from a file. The regexed value is passed through shlex. This assumes that the current working directory is the deployment.

detectVersionFromGit(tagPattern, preStrip='')

Helper method that detects a version by using the most recent tag in git that matches the specified pattern. This assumes that the current working directory is the deployment.

download(version)

Returns a URL that can be used to download a tarball of version of this application.

dsn(deployment)

Returns the deployment specific database URL. Uses the override file in .wizard if it exists, and otherwise attempt to extract the variables from the source files.

Under some cases, the database URL will contain only the database property, and no other values. This indicates that the actual DSN should be determined from the environment.

This function might return None.

Note

We are allowed to batch these two together, because the full precedence chain for determining the database of an application combines these two together. If this was not the case, we would have to call databaseUrlFromOverride() and databaseUrlFromExtract() manually.

dsnFromExtract(deployment)

Extracts database URL from a deployment, and returns them as a sqlalchemy.engine.url.URL. Returns None if we can’t figure it out: i.e. the conventional variables are not defined for this application.

dsnFromOverride(deployment)

Extracts database URL from an explicit dsn override file.

extract(deployment)

Extracts wizard variables from a deployment. Default implementation uses extractors.

extractors

Dictionary of variable names to extractor functions. These functions take a wizard.deploy.Deployment as an argument and return the value of the variable, or None if it could not be found. See also filename_regex_extractor().

install(version, options)

Run for ‘wizard configure’ (and, by proxy, ‘wizard install’) to configure an application. This assumes that the current working directory is a deployment. (Unlike its kin, this function does not take a wizard.deploy.Deployment as a parameter.) Subclasses should provide an implementation.

install_schema

Instance of wizard.install.ArgSchema that defines the arguments this application requires.

makeVersion(version)

Creates or retrieves the ApplicationVersion singleton for the specified version.

name

String name of the application

needs_web_stub

Indicates whether or not a web stub is necessary.

parametrize(deployment, ref_deployment)

Takes a generic source checkout and parametrizes it according to the values of deployment. This function operates on the current working directory. deployment should not be the same as the current working directory. See parametrizeWithVars() for details on the parametrization.

parametrizeWithVars(variables)

Takes a generic source checkout and parametrizes it according to the values of variables. Default implementation uses parametrized_files and a simple search and replace on those files.

parametrized_files

List of files that need to be modified when parametrizing. This is a class-wide constant, and should not normally be modified.

prepareConfig(deployment)

Takes a deployment and replaces any explicit instances of a configuration variable with generic WIZARD_* constants. The default implementation uses substitutions, and emits warnings when it encounters keys in deprecated_keys.

prepareMerge(deployment)

Performs various edits to files in the current working directory in order to make a merge go more smoothly. This is usually used to fix botched line-endings. If you add new files, you have to ‘git add’ them; this is not necessary for edits. By default this is a no-op; subclasses should replace this with useful behavior.

random_blacklist

Values that are not sufficiently random for a random key. This can include default values for a random configuration option,

random_keys

Keys that we can simply generate random strings for if they’re missing

remove(deployment, options)

Run for ‘wizard remove’ to delete all database and non-local file data. This assumes that the current working directory is the deployment. Subclasses should provide an implementation.

repository(srv_path)

Returns the Git repository that would contain this application. srv_path corresponds to options.srv_path from the global baton.

researchFilter(filename, added, deleted)

Allows an application to selectively ignore certain diffstat signatures during research; for example, configuration files will have a very specific set of changes, so ignore them; certain installation files may be removed, etc. Return True if a diffstat signature should be ignored,

researchVerbose(filename)

Allows an application to exclude certain dirty files from the output report; usually this will just be parametrized files, since those are guaranteed to have changes. Return True if a file should only be displayed in verbose mode.

resolutions

Dictionary of file names to a list of resolutions, which are tuples of a conflict marker string and a result list. See wizard.resolve for more information.

resolveConflicts(deployment)

Resolves conflicted files in the current working directory. Returns whether or not all conflicted files were resolved or not. Fully resolved files are added to the index, but no commit is made. The default implementation uses resolutions.

restore(deployment, backup_dir, options)

Run for ‘wizard restore’ and failed upgrades to restore database and other non-versioned data to a backed up version. This assumes that the current working directory is the deployment. Subclasses should provide an implementation.

substitutions

Dictionary of variable names to substitution functions. These functions take a wizard.deploy.Deployment as an argument and modify the deployment such that an explicit instance of the variable is released with the generic WIZARD_* constant. See also filename_regex_substitution().

upgrade(deployment, version, options)

Run for ‘wizard upgrade’ to upgrade database schemas and other non-versioned data in an application after the filesystem has been upgraded. This assumes that the current working directory is the deployment. Subclasses should provide an implementation.

url(deployment)

Returns the deployment specific web URL. Uses the override file in .wizard if it exists, and otherwise attempt to extract the variables from the source files.

This function might return None, which indicates we couldn’t figure it out.

urlFromExtract(deployment)

Extracts URL from a deployment, and returns None if we can’t figure it out. Default implementation is to fail; we might do something clever with extractable variables in the future.

urlFromOverride(deployment)

Extracts URL from explicit url override file.

versions

Dictionary of version strings to ApplicationVersion. See also makeVersion().

class wizard.app.ApplicationVersion(version, application)

Represents an abstract notion of a version for an application, where version is a distutils.version.LooseVersion and application is a Application.

application

The Application of this instance.

static make(app, version)

Makes/retrieves a singleton ApplicationVersion from a``app`` and version string.

static parse(value)

Parses a line from the versions store and return ApplicationVersion.

Use this only for cases when speed is of primary importance; the data in version is unreliable and when possible, you should prefer directly instantiating a wizard.deploy.Deployment and having it query the autoinstall itself for information.

The value to parse will vary. For old style installs, it will look like:

/afs/athena.mit.edu/contrib/scripts/deploy/APP-x.y.z

For new style installs, it will look like:

APP-x.y.z-scripts
pristine_tag

Returns the name of the Git tag for the pristine version corresponding to this version.

tag

Returns the name of the git describe tag for the commit the user is presently on, something like mediawiki-1.2.3-scripts-4-g123abcd

version

The distutils.version.LooseVersion of this instance.

wizard_tag

Returns the name of the Git tag for this version.

Functions

wizard.app.getApplication(appname)

Retrieves application instance given a name

wizard.app.applications()

Hash table for looking up string application name to instance

wizard.app.expand_re(val)

Takes a tree of values (implement using nested lists) and transforms them into regular expressions.

>>> expand_re('*')
'\\*'
>>> expand_re(['a', 'b'])
'(?:a|b)'
>>> expand_re(['*', ['b', 'c']])
'(?:\\*|(?:b|c))'
wizard.app.make_extractors(seed)

Take a dictionary of key to (file, regex) tuples and convert them into extractor functions (which take a wizard.deploy.Deployment and return the value of the second subpattern of regex when matched with the contents of file).

wizard.app.make_substitutions(seed)

Take a dictionary of key to (file, regex) tuples and convert them into substitution functions (which take a wizard.deploy.Deployment, replace the second subpattern of regex with key in file, and returns the number of substitutions made.)

wizard.app.filename_regex_extractor(file, regex)

Given a relative file name file, a regular expression regex, and a wizard.deploy.Deployment extracts a value out of the file in that deployment. This function is curried, so you pass just file and regex, and then pass deployment to the resulting function.

Its Haskell-style type signature would be:

Filename -> Regex -> (Deployment -> String)

The regular expression requires a very specific form, essentially ()()() (with the second subgroup being the value to extract). These enables the regular expression to be used equivalently with filename

For convenience purposes, we also accept [Filename], in which case we use the first entry (index 0). Passing an empty list is invalid.

>>> open("test-settings.extractor.ini", "w").write("config_var = 3\n")
>>> f = filename_regex_extractor('test-settings.extractor.ini', re.compile('^(config_var\s*=\s*)(.*)()$'))
>>> f(deploy.Deployment("."))
'3'
>>> os.unlink("test-settings.extractor.ini")

Note

The first application of regex and file is normally performed at compile-time inside a submodule; the second application is performed at runtime.

wizard.app.filename_regex_substitution(key, files, regex)

Given a Wizard key (WIZARD_*), a list of files, a regular expression regex, and a wizard.deploy.Deployment performs a substitution of the second subpattern of regex with key. Returns the number of replacements made. This function is curried, so you pass just key, files and regex, and then pass deployment to the resulting function.

Its Haskell-style type signature would be:

Key -> ([File], Regex) -> (Deployment -> IO Int)

For convenience purposes, we also accept Filename, in which case it is treated as a single item list.

>>> open("test-settings.substitution.ini", "w").write("config_var = 3")
>>> f = filename_regex_substitution('WIZARD_KEY', 'test-settings.substitution.ini', re.compile('^(config_var\s*=\s*)(.*)()$'))
>>> f(deploy.Deployment("."))
1
>>> print open("test-settings.substitution.ini", "r").read()
config_var = WIZARD_KEY
>>> os.unlink("test-settings.substitution.ini")
wizard.app.throws_database_errors(f)

Decorator that takes database errors from wizard.sql and converts them into application script failures from wizard.app. We can’t throw application errors directly from wizard.sql because that would result in a cyclic import; also, it’s cleaner to distinguish between a database error and an application script failure.

Exceptions

exception wizard.app.Error

Generic error class for this module.

exception wizard.app.NoRepositoryError(app)

Application does not appear to have a Git repository in the normal location.

app

The name of the application that does not have a Git repository.

exception wizard.app.DeploymentParseError(value)

Could not parse value from versions store.

location

The location of the autoinstall that threw this variable. This should be set by error handling code when it is available.

value

The value that failed to parse.

exception wizard.app.NoSuchApplication(app)

You attempted to reference a Application named app, which is not recognized by Wizard.

app

The name of the application that does not exist.

location

The location of the autoinstall that threw this variable. This should be set by error handling code when it is availble.

Failures

exception wizard.app.Failure

Represents a failure when performing some double-dispatched operation such as an installation or an upgrade. Failure classes are postfixed with Failure, not Error.

exception wizard.app.InstallFailure

Installation failed for unknown reason.

exception wizard.app.RecoverableInstallFailure(errors)

Bases: wizard.app.InstallFailure

Installation failed, but we were able to determine what the error was, and should give the user a second chance if we were running interactively.

errors

List of the errors that were found.

exception wizard.app.UpgradeFailure(details)

Upgrade script failed.

details

String details of failure (possibly stdout or stderr output)

exception wizard.app.UpgradeVerificationFailure

Upgrade script passed, but website wasn’t accessible afterwards

exception wizard.app.BackupFailure(details)

Backup script failed.

details

String details of failure

exception wizard.app.RestoreFailure(details)

Restore script failed.

details

String details of failure

exception wizard.app.RemoveFailure(details)

Remove script failed.

details

String details of failure

Table Of Contents

Previous topic

wizard

Next topic

wizard.app.php

This Page