Plumbing object model for representing applications we want to install. This module does the heavy lifting, but you probably want to use wizard.deploy.Deployment which is more user-friendly. You’ll need to know how to overload the Application class and use some of the functions in this module in order to specify new applications.
To specify custom applications as plugins, add the following entry_points configuration:
[wizard.app]
yourappname = your.module:Application
otherappname = your.other.module:Application
Note
Wizard will complain loudly if yourappname conflicts with an application name defined by someone else.
There are some submodules for programming languages that define common functions and data that may be used by applications in that language. See:
Represents an application, i.e. mediawiki or phpbb.
Note
Many of these methods assume a specific working directory; prefer using the corresponding methods in wizard.deploy.Deployment and its subclasses.
Run for ‘wizard backup’ and upgrades to backup database schemas and other non-versioned data in an application. outdir is the directory that backup files should be placed. This assumes that the current working directory is the deployment. Subclasses should provide an implementation, even if it is a no-op.
Note
Static user files may not need to be backed up, since in many applications upgrades do not modify static files.
Checks whether or not an autoinstall has been configured/installed for use. Assumes that the current working directory is the deployment. Subclasses should provide an implementation.
Checks if the database is accessible.
Checks if the autoinstall is viewable from the web. Subclasses should provide an implementation.
Note
Finding a reasonable heuristic that works across skinning choices can be difficult. We’ve had reasonable success searching for metadata. Be sure that the standard error page does not contain the features you search for. Try not to depend on pages that are not the main page.
Checks if a given page of an autoinstall contains a particular string.
Name of the database that this application uses, i.e. mysql or postgres. If we end up supporting multiple databases for a single application, there should also be a value for this in wizard.deploy.Deployment; the value here is merely the preferred value.
Keys that are used in older versions of the application, but not for the most recent version.
Checks source files to determine the version manually. This assumes that the current working directory is the deployment. Subclasses should provide an implementation.
Helper method that detects a version by using a regular expression from a file. The regexed value is passed through shlex. This assumes that the current working directory is the deployment.
Helper method that detects a version by using the most recent tag in git that matches the specified pattern. This assumes that the current working directory is the deployment.
Returns a URL that can be used to download a tarball of version of this application.
Returns the deployment specific database URL. Uses the override file in .wizard if it exists, and otherwise attempt to extract the variables from the source files.
Under some cases, the database URL will contain only the database property, and no other values. This indicates that the actual DSN should be determined from the environment.
This function might return None.
Note
We are allowed to batch these two together, because the full precedence chain for determining the database of an application combines these two together. If this was not the case, we would have to call databaseUrlFromOverride() and databaseUrlFromExtract() manually.
Extracts database URL from a deployment, and returns them as a sqlalchemy.engine.url.URL. Returns None if we can’t figure it out: i.e. the conventional variables are not defined for this application.
Extracts database URL from an explicit dsn override file.
Extracts wizard variables from a deployment. Default implementation uses extractors.
Dictionary of variable names to extractor functions. These functions take a wizard.deploy.Deployment as an argument and return the value of the variable, or None if it could not be found. See also filename_regex_extractor().
Human-readable name of the application
Run for ‘wizard configure’ (and, by proxy, ‘wizard install’) to configure an application. This assumes that the current working directory is a deployment. (Unlike its kin, this function does not take a wizard.deploy.Deployment as a parameter.) Subclasses should provide an implementation.
Instance of wizard.install.ArgSchema that defines the arguments this application requires.
Creates or retrieves the ApplicationVersion singleton for the specified version.
String name of the application
Indicates whether or not a web stub is necessary.
Takes a generic source checkout and parametrizes it according to the values of deployment. This function operates on the current working directory. deployment should not be the same as the current working directory. See parametrizeWithVars() for details on the parametrization.
Takes a generic source checkout and parametrizes it according to the values of variables. Default implementation uses parametrized_files and a simple search and replace on those files.
List of files that need to be modified when parametrizing. This is a class-wide constant, and should not normally be modified.
Takes a deployment and replaces any explicit instances of a configuration variable with generic WIZARD_* constants. The default implementation uses substitutions, and emits warnings when it encounters keys in deprecated_keys.
Performs various edits to files in the current working directory in order to make a merge go more smoothly. This is usually used to fix botched line-endings. If you add new files, you have to ‘git add’ them; this is not necessary for edits. By default this is a no-op; subclasses should replace this with useful behavior.
Values that are not sufficiently random for a random key. This can include default values for a random configuration option,
Keys that we can simply generate random strings for if they’re missing
Run for ‘wizard remove’ to delete all database and non-local file data. This assumes that the current working directory is the deployment. Subclasses should provide an implementation.
Returns the Git repository that would contain this application. srv_path corresponds to options.srv_path from the global baton.
Allows an application to selectively ignore certain diffstat signatures during research; for example, configuration files will have a very specific set of changes, so ignore them; certain installation files may be removed, etc. Return True if a diffstat signature should be ignored,
Allows an application to exclude certain dirty files from the output report; usually this will just be parametrized files, since those are guaranteed to have changes. Return True if a file should only be displayed in verbose mode.
Dictionary of file names to a list of resolutions, which are tuples of a conflict marker string and a result list. See wizard.resolve for more information.
Resolves conflicted files in the current working directory. Returns whether or not all conflicted files were resolved or not. Fully resolved files are added to the index, but no commit is made. The default implementation uses resolutions.
Run for ‘wizard restore’ and failed upgrades to restore database and other non-versioned data to a backed up version. This assumes that the current working directory is the deployment. Subclasses should provide an implementation.
Dictionary of variable names to substitution functions. These functions take a wizard.deploy.Deployment as an argument and modify the deployment such that an explicit instance of the variable is released with the generic WIZARD_* constant. See also filename_regex_substitution().
Run for ‘wizard upgrade’ to upgrade database schemas and other non-versioned data in an application after the filesystem has been upgraded. This assumes that the current working directory is the deployment. Subclasses should provide an implementation.
Returns the deployment specific web URL. Uses the override file in .wizard if it exists, and otherwise attempt to extract the variables from the source files.
This function might return None, which indicates we couldn’t figure it out.
Extracts URL from a deployment, and returns None if we can’t figure it out. Default implementation is to fail; we might do something clever with extractable variables in the future.
Extracts URL from explicit url override file.
Dictionary of version strings to ApplicationVersion. See also makeVersion().
Represents an abstract notion of a version for an application, where version is a distutils.version.LooseVersion and application is a Application.
The Application of this instance.
Makes/retrieves a singleton ApplicationVersion from a``app`` and version string.
Parses a line from the versions store and return ApplicationVersion.
Use this only for cases when speed is of primary importance; the data in version is unreliable and when possible, you should prefer directly instantiating a wizard.deploy.Deployment and having it query the autoinstall itself for information.
The value to parse will vary. For old style installs, it will look like:
/afs/athena.mit.edu/contrib/scripts/deploy/APP-x.y.z
For new style installs, it will look like:
APP-x.y.z-scripts
Returns the name of the Git tag for the pristine version corresponding to this version.
Returns the name of the git describe tag for the commit the user is presently on, something like mediawiki-1.2.3-scripts-4-g123abcd
The distutils.version.LooseVersion of this instance.
Returns the name of the Git tag for this version.
Retrieves application instance given a name
Hash table for looking up string application name to instance
Takes a tree of values (implement using nested lists) and transforms them into regular expressions.
>>> expand_re('*')
'\\*'
>>> expand_re(['a', 'b'])
'(?:a|b)'
>>> expand_re(['*', ['b', 'c']])
'(?:\\*|(?:b|c))'
Take a dictionary of key to (file, regex) tuples and convert them into extractor functions (which take a wizard.deploy.Deployment and return the value of the second subpattern of regex when matched with the contents of file).
Take a dictionary of key to (file, regex) tuples and convert them into substitution functions (which take a wizard.deploy.Deployment, replace the second subpattern of regex with key in file, and returns the number of substitutions made.)
Given a relative file name file, a regular expression regex, and a wizard.deploy.Deployment extracts a value out of the file in that deployment. This function is curried, so you pass just file and regex, and then pass deployment to the resulting function.
Its Haskell-style type signature would be:
Filename -> Regex -> (Deployment -> String)
The regular expression requires a very specific form, essentially ()()() (with the second subgroup being the value to extract). These enables the regular expression to be used equivalently with filename
For convenience purposes, we also accept [Filename], in which case we use the first entry (index 0). Passing an empty list is invalid.
>>> open("test-settings.extractor.ini", "w").write("config_var = 3\n")
>>> f = filename_regex_extractor('test-settings.extractor.ini', re.compile('^(config_var\s*=\s*)(.*)()$'))
>>> f(deploy.Deployment("."))
'3'
>>> os.unlink("test-settings.extractor.ini")
Note
The first application of regex and file is normally performed at compile-time inside a submodule; the second application is performed at runtime.
Given a Wizard key (WIZARD_*), a list of files, a regular expression regex, and a wizard.deploy.Deployment performs a substitution of the second subpattern of regex with key. Returns the number of replacements made. This function is curried, so you pass just key, files and regex, and then pass deployment to the resulting function.
Its Haskell-style type signature would be:
Key -> ([File], Regex) -> (Deployment -> IO Int)
For convenience purposes, we also accept Filename, in which case it is treated as a single item list.
>>> open("test-settings.substitution.ini", "w").write("config_var = 3")
>>> f = filename_regex_substitution('WIZARD_KEY', 'test-settings.substitution.ini', re.compile('^(config_var\s*=\s*)(.*)()$'))
>>> f(deploy.Deployment("."))
1
>>> print open("test-settings.substitution.ini", "r").read()
config_var = WIZARD_KEY
>>> os.unlink("test-settings.substitution.ini")
Decorator that takes database errors from wizard.sql and converts them into application script failures from wizard.app. We can’t throw application errors directly from wizard.sql because that would result in a cyclic import; also, it’s cleaner to distinguish between a database error and an application script failure.
Generic error class for this module.
Application does not appear to have a Git repository in the normal location.
The name of the application that does not have a Git repository.
Could not parse value from versions store.
The location of the autoinstall that threw this variable. This should be set by error handling code when it is available.
The value that failed to parse.
You attempted to reference a Application named app, which is not recognized by Wizard.
The name of the application that does not exist.
The location of the autoinstall that threw this variable. This should be set by error handling code when it is availble.
Represents a failure when performing some double-dispatched operation such as an installation or an upgrade. Failure classes are postfixed with Failure, not Error.
Installation failed for unknown reason.
Bases: wizard.app.InstallFailure
Installation failed, but we were able to determine what the error was, and should give the user a second chance if we were running interactively.
List of the errors that were found.
Upgrade script failed.
String details of failure (possibly stdout or stderr output)
Upgrade script passed, but website wasn’t accessible afterwards
Backup script failed.
String details of failure