TODO NOW:
-* Migrate and upgrade the lone mediawiki-1.5.6 install
-* Create the migration script
+- Fix mediawiki repository (has lots of stale files and a weird
+ history. This also means that the ichuang wiki will need to be
+ remigrated; frob .scripts-version to make it acceptable, but
+ otherwise should be fairly trivial). When I say "fix", I mean
+ "redo". Thorough documentation would be good too. Some bits
+ can be automated and/or have tools to assist it
+- Whiteboard the flow for performing an upgrade on a single
+ install. How assisted does it need to be?
+- Conduct migration tool testing
+- Create mass-migration tool (should be able to limit on mediawiki)
+- Run parallel-find.pl
+- Migrate all mediawikis
OVERALL PLAN:
nice, but is priority. For the long term, seeing this scripts
be packaged with rest of our code would be optimal.
-* The new procedure for generating an update is as follows:
+* The new procedure for generating an update is as follows (this is
+ also similar to procedure for creating these repositories):
1. Have the Git repository and working copy for the project on hand.
- 2. Download the new tarball
+ 2. Checkout the pristine branch
- 3. Extract the tarball over the working copy (`cp -R a/. b` works well)
+ 3. Remove all files from the working copy (rm -Rf *, and then delete
+ any dot stragglers. A script to do this would be handy)
- 4. Check if there are any special update procedures, and update the
- .scripts/update shell script as necessary (this means that any
- application specific update logic will be kept with the actual
- source code. The language of this update script will vary
- depending on context.)
+ 4. Download the new tarball
- X. Check for empty directories and add stub files as necessary
- (use preserve-empty-dir)
-
- 5. Commit your changes, and tag as v1.2.3-scripts
+ 5. Extract the tarball over the working copy (`cp -R a/. b` works well,
+ remember that the working copy is empty)
- 6. Run the "dry-run script", which uses Git commands to check how many
- working copies apply the change cleanly, and writes out a logfile
- with the working copies that don't apply cleanly.
+ 6. Check for empty directories and add stub files as necessary
+ (use preserve-empty-dir)
- 7. Run the "limited run" script, which applies the update to our
- test-bed, and lets us check the basic functionality of the update.
+ 7. Git add it all, and then commit as a new pristine version (v1.2.3)
- 8. Run the "deploy" script, which applies the update to all working
- copies possible, and sends mail to users to whom the working copy
- did not apply cleanly. (It also frobs .scripts/version)
+ 8. Checkout the master branch
- Note: The last three scripts will need to be implemented, with an
- eye towards speed.
+ 9. [FOR EXISTING REPOSITORIES]
+ Merge the pristine branch in. Resolve any conflicts that our
+ patches have with new changes. Do NOT let Git auto-commit it
+ with --no-commit (otherwise, you want to git commit --amend
+ to keep our history clean
-* How to migrate an old autoinstaller to the new autoinstaller
+ [FOR THE FIRST TIME]
+ Apply the scripts patch that was used for that version here
+ (usually patch -p1 < patch)
- - Find the oldest tarball/patch set for the application that still
- is in use and upgradable.
+ 10. Check if there are any special update procedures, and update the
+ .scripts/update shell script as necessary (this means that any
+ application specific update logic will be kept with the actual
+ source code. The language of this update script will vary
+ depending on context.)
- - Untar, apply patch, place in a directory (unfurl) and git init
+ 11. Commit your changes, and tag as v1.2.3-scripts
- - Commit this as the "pristine" version (branch "pristine")
+ If you're setting up a repository from scratch, stop here, and
+ repeat as necessary
- - Apply scripts patches
+ XXX: Should we force people to push to the real repository at
+ this point, or just make the repository that the script pulls
+ stuff out of configurable? (Twiddling origin can get you a
+ devel setup with no code changes)
- - Create the .scripts directory and populate it with the interesting
- information (see below)
+ 12. Run the "dry-run script", which uses Git commands to check how many
+ working copies apply the change cleanly, and writes out a logfile
+ with the working copies that don't apply cleanly.
- - Commit this as the "scripts" version (branch "master")
+ 13. Run the "limited run" script, which applies the update to our
+ test-bed, and lets us check the basic functionality of the update.
+ This can include a script that lets us update a single directory
+ with verbose output.
-* How to update the autoinstaller repository
+ 14. Run the "deploy" script, which applies the update to all working
+ copies possible, and sends mail to users to whom the working copy
+ did not apply cleanly. It also frobs .scripts/version for successful
+ upgrades.
+
+ 15. Run parallel-find.pl
+
+* For mass importing into the repository, the steps are:
+
+[TO SET IT UP]
+# let app-1.2.3 be the scripts folder originally in deploydev
+# let this folder be srv/
+mkdir app
+cd app
+git init
+cd ..
+unfurl app-1.2.3 app
+# NOTE: contents of application are now in app directory
+cd app
+git add .
+git commit -s -m "App 1.2.3"
+git tag v1.2.3
+git branch pristine
+# NOTE: you're still on master branch
+mkdir .scripts
+echo "Deny from all" > .scripts/.htaccess
+touch .scripts/update
+chmod a+x .scripts/update
+# OPERATION: create the update script
+# WARNING: the following operation might require -p1
+patch -p0 < ../app-1.2.3/app-1.2.3.patch
+# NOTE: please sanity check the patch!
+git add .
+# NOTE: -a flag is to handle if the patch deleted something
+git commit -as -m "App 1.2.3-scripts"
+git tag v1.2.3-scripts
+
+[TO ADD AN UPDATE]
+# let this folder be srv/app.git
+git checkout pristine
+# NOTE: this preserves your .git folder, but removes everything
+wipe-working-dir .
+cd ..
+unfurl app-1.2.3 app
+cd app
+# NOTE: please sanity check app directory
+git add .
+# NOTE: -a is to take care of deletions
+git commit -as -m "App 1.2.3"
+git tag v1.2.3
+[IF THE PATCH HAS CHANGED]
+ # NOTE: Now, the tricky part (this is different from a real update)
+ git symbolic-ref HEAD refs/heads/master
+ # NOTE: Now, we think we're on the master branch, but we have
+ # pristine copy checked out
+ # NOTE: -p0 might need to be twiddled
+ patch -p0 < ../app-1.2.3/app-1.2.3.patch
+ git add .
+ # NOTE: DON'T FORGET THIS STEP!!! Otherwise fixing this is going
+ # to be painful later down the line
+ git checkout .scripts
+ # OPERATION: Check if the directory needs an updated update script
+ # NOTE: Fake the merge
+ git rev-parse pristine > .git/MERGE_HEAD
+[IF THE PATCH HASN'T CHANGED]
+ git checkout master
+ git merge --no-commit pristine
+git commit -as -m "App 1.2.3-scripts"
+git tag v1.2.3-scripts
- - Checkout pristine branch
- - Delete contents of pristine branch (excluding .git, try rm -Rf * and remove stragglers)
- - Unfurl tarball
- - Commit as new pristine branch
- - Checkout scripts branch
- - Merge pristine branch
- - Fix as necessary
* The repository for a given application will contain the following files:
* .scripts/database (generated) contains the database the
user installed the script to, so scripts-remove can clean it
+ XXX: Could cause problems if a user copies the autoinstall,
+ fiddles with the DB credentials, and then scripts-remove's
+ the autoinstall. Possible fix is to add the original
+ directory as a sanity check. Additionally, we could have
+ the application read out of this file.
+
* .scripts/version (generated) which contains the version
last autoinstalled (as distinct from the actual version
the script is) (This is the same as .scripts-version right
now; probably want to keep that for now)
- - Because there will be no .gitignore file, you *must not* run
- `git add .` on an actual running copy of the application.
- `git add -u .` will generally be safe, but preferred mode
- of operation is to operate on a clean install.
-
-* The migration process shall be as such:
-
- 1. git init
-
- 2. git remote add origin /foo
-
- 3. git config branch.master.merge refs/heads/master
-
- 4. git fetch origin
-
- 5. git reset v1.2.3-scripts
-
- 5. git checkout .scripts
+ XXX: It's unclear if we want to move to this wholesale, or
+ delay this indefinitely.
- 6. Setup .scripts/version (probably pipe the output of real-version)
- UNCLEAR if this is a good thing; if it is, make sure we add
- a .gitignore to the .scripts directory
+* The migration process has been implemented, see 'wizard migrate'.
-* We will not add special code to handle .htaccess; thus the kernel patch
- for allowing Apache access to .htaccess sent to scripts-team@mit.edu
- must be handled first.
+ XXX: We have not decided what migration should do to .scripts-version;
+ if it does move it to .scripts, repositories should have a .gitignore
+ in those directories
* The autoupgrade shall be the process of:
(with some more robust error checking)
-* Make install-statistics generate nice pretty graphs of installs by date
+* Make 'wizard summary' generate nice pretty graphs of installs by date
(more histograms, will need to check actual .scripts-version files.)
scripts.mit.edu / wizard.git / blobdiff