X-Git-Url: https://scripts.mit.edu/gitweb/wizard.git/blobdiff_plain/ab821ded0093db5f0f2b70e161542292c4f5c7c5..009abb4c8dc4cb426cce626c3c960edf776ac2c5:/TODO diff --git a/TODO b/TODO index 93444a8..21ab933 100644 --- a/TODO +++ b/TODO @@ -2,109 +2,10 @@ The Git Autoinstaller TODO NOW: -- Make it faster - - Wipe temp directories if the upgrade succeeds - - Put temp directories on tmpfs before merging, then move to disk - if it fails and needs resolution. /var/run is a pretty good - choice if your running as root, not so good if you're not. Find - other common tmpfs locations (mount | grep tmpfs) and perhaps - check some common ones. - - Certain classes of error will continually fail, so they should - put in a different "seen" file which also skips them, unless - we have some sort of gentle force - - Keep my sanity when upgrading 1000 installs - - Distinguish between errors(?) - - Custom merge algo: absolute php.ini symlinks to relative symlinks (this - does not seem to have been a problem in practice) - - Custom merge algo: check if it's got extra \r's in the file, - and dos2unix it if it does, before performing the merge - - `vos exa` in order to check what a person's quota is. We can - figure out roughly how big the upgrade is going to be by - doing a size comparison of the tars: `git pull` MUST NOT - fail, otherwise things are left conflicted, and not easy to fix. - - Prune -7 call errors and automatically reprocess them (with a - strike out counter of 3) - - Snap-in conflict resolution teaching: - 1. View the merge conflicts after doing a short run - 2. Identify common merge conflicts - 3. Copypaste the conflict markers to the application. Scrub - user-specific data; this may mean removing the entire - upper bit which is the user-version. - 4. Specify which section to keep. /Usually/ this means - punting the new change, but if the top was specified - it means we get a little more flexibility. Try to - minimize wildcarding: those things need to be put into - subpatterns and then reconstituted into the output. - Example: - Input: - <<<<<<< - ***1*** - ======= - upstream - >>>>>>> - Output: - [1] # discard system string - Input: - <<<<<<< - old upstream - ======= - new upstream - >>>>>>> - Output: - ['R'] # keep the new upstream string - # This would be useful if a particular upstream change - # is really close to where user changes are, so that - # the conflict pops up a lot and it's actually spurious - Input: - <<<<<<< - ***1*** - old upstream - ***2*** - old upstream - ***3*** - ======= - new upstream - >>>>>>> - Output: - ['R', 1, 2, 3] # should be evident - # it's not actually clear to me if this is useful - To resolve: do we need the power of regexes? This might suck - because it means we need to implement escaping. We might want - simple globbing to the end of line since that's common in - configuration files. - -- Distinguish from logging and reporting (so we can easily send mail - to users) - - Remove "already migrated" cruft that will accumulate if we do small - --limit and then increase. - - Logs aren't actually useful, /because/ most operations are idempotent. - Thus, scratch logfile and make our report files more useful: error.log - needs error information; we don't care too much about machinability. - All report files should be overwritten on the next run, since we like - using --limit to incrementally increase the number of things we run. Note - that if we add soft ignores, you /do/ lose information, so there needs - to be some way to also have the soft ignore report a "cached error" - - Report the identifier number at the beginning of all of the stdout logs - - Log files that already exist should be initialized with some sort - of separator THAT CONTAINS THE LOCATION OF THE INSTALL - - Don't really care about having the name in the logfile name, but - have a lookup txt file - - Figure out a way of collecting blacklist data from .scripts/blacklisted - and aggregate it together - - Failed migrations should be wired to have wizard commands in them - automatically log to the relevant file. In addition, the seen file - should get updated when one of them gets fixed. - - Log files need to have dates, since it looks like upgrades will be - multi-day affairs - - Failed migration should report how many unmerged files there are - (so we can auto-punt if it's over a threshold) - - Verification failures should be written to a report file, possibly - with short HTML fingerprints so we can inspect them easily and - numbers to look at the log files - -- Let users use Wizard when ssh'ed into Scripts - - Make single user mass-migrate work when not logged in as root + - Replace gaierror with a more descriptive name (this is a DNS error) + - Stronger skips means that backup failures should also be avoided + - Distinguish between types of backup failures - Make the rest of the world use Wizard - Make parallel-find.pl use `sudo -u username git describe --tags` @@ -127,8 +28,24 @@ TODO NOW: output summary charts when I increase specificity - Summary script should do something intelligent when distinguishing between old-style and new-style installs + - Report code in wizard/command/__init__.py is ugly as sin. Also, + the Report object should operate at a higher level of abstraction + so we don't have to manually increment fails. (in fact, that should + probably be called something different). The by-percent errors should + also be automated. + - Indents in upgrade.py are getting pretty ridiculous; more breaking + into functions is probably a good idea + - Move resolutions in mediawiki.py to a text file? (the parsing overhead + may not be worth it) + - Investigate QuotaParseErrors + - If a process is C-ced, it can result in a upgrade that has + an updated filesystem but not updated database. Make this more + resilient + - PHP end of file allows omitted semicolon, can result in parse error + if merge resolutions aren't careful. - Other stuff + - Make single user mass-migrate work when not logged in as root - Don't use the scripts heuristics unless we're on scripts with the AFS patch. Check with `fs sysname` - Make 'wizard summary' generate nice pretty graphs of installs by date