scripts test suite

[andersk]

It would be useful to have a scripts.mit.edu test suite, that lets us run one command to find out everything we break on one of the servers when we upgrade it.

[ezyang]

I've thrown this at a prospective once at a hackathon, they made some progress but it wasn't particularly successful: it was too big a project. I think it would be better if we had specific suggestions for things to check in this ticket.

[adehnert]

• Django
  • Do an autoinstall
  • Run through the steps of http://scripts.mit.edu/faq/127/how-can-i-import-an-outside-django-project
  • Run ./manage.py shell
  • Visit the site's initial page and the admin
• Autoinstallers
  • Run each autoinstaller
  • Visit the sites produced and verify that they at least load (how much more to do is unclear)

[adehnert]

I'm kinda dubious that a single-command test suite is worth the effort. A good sized checklist of things to verify, on the other hand, would probably be worthwhile.