Systematically track alert frequency

[adehnert]

A lot of nagios alerts fire, and some people are probably experiencing "alert fatigue". Some of them don't necessarily indicate an actual problem that we can and/or need to address (eg, the sql.mit.edu connection count alerts or the mailq alerts). To help evaluate whether those alerts are useful, fixing them should be automated, thresholds should be increased, and so forth, it would be nice systematically track and possibly graph how often those alerts occur.

zlogs definitely have the requisite data (and I can supply data from mongodb if that's better than parsing normal zlogs), or sipb-noc might be able supply this data.