Opened 13 years ago

Last modified 6 years ago

#49 new enhancement

Make logview work

Reported by: price Owned by: geofft
Priority: major Milestone:
Component: web Keywords:

Description (last modified by ezyang)

logview suffers from two problems that sharply limit its value to users:

  • its dependence on grep makes it miss out many errors entirely;
  • we don't have a good setup for getting logs from both servers.

The second problem is relatively easy to fix; #23 describes one approach that should be easy to implement and solve that problem.

The first is harder. Ideally Apache would put a request's logs someplace corresponding to the locker. Maybe this could be at suexec time. But, e.g., mod_rewrite errors, which are common causes of pulled-out user hair, happen before suexec time, so ideally we'd do something earlier.

Here is a proposal, which I may have the good properties we want.

  1. Apache based errors continue to go to /home/logview/error_log.
  2. Errors from suexec'd binaries now have their errors go to per-user files. This is arranged for by suexec, which replaces the stderr fd with an fd to a file it opened.
  3. error_log now is watched by a zephyr daemon, which has the following behavior. By default, it ignores all entries in the log. However, it is subscribed to -c scripts-logs; when it receives an authenticated zephyr from a user requesting logs for a locker, it checks if the user is admof locker. If so, it sets up a new rule, which sends class'd personals (name?) to that user of error logs corresponding to that locker. We remove rules when the zephyr notifies us no one is listening to our zephyrs.
  4. We setup a custom BarnOwl?, so users can run 'athrun scripts logview locker', and they get a BarnOwl? which sends the control zephyr and subscribes to the relevant class to view errors for the class. It might even be nifty if we have some commands for subscribing to more errors, though not strictly necessary.
  5. Setup similar daemons for the per-user stderr logs.

Note that it is important that the daemon rate limits message sending, so we don't flood the zephyr servers.

Attachments (1) (2.3 KB) - added by broder 12 years ago.

Download all attachments as: .zip

Change History (11)

comment:1 Changed 13 years ago by andersk

  • Priority changed from major to critical

nelhage says this is why he never deploys anything on scripts, and I don’t blame him. This is a hard problem but it shouldn’t be intractable.

Changed 12 years ago by broder

comment:2 Changed 12 years ago by broder

  • Owner set to geofft

comment:3 Changed 9 years ago by ezyang

  • Type changed from defect to enhancement

comment:4 Changed 9 years ago by ezyang

  • Description modified (diff)

comment:5 Changed 9 years ago by adehnert

I think (2) and (3) are independently useful, and we should not block implementing one on implementing the other, or implementing (4) or (5).

comment:6 Changed 9 years ago by ezyang

  • Description modified (diff)

comment:7 Changed 8 years ago by adehnert

There was a suggestion on -c scripts -i logview on 2013-02-18 at about 1PM that maybe doing this with journald would be cool? Apparently we should have suexec fork some proxy process.

comment:8 Changed 6 years ago by achernya

Fixed in r2602 and r2603 through the glory of suexec and journald.

comment:9 Changed 6 years ago by geofft

Is there outstanding work to do here? Did we want to restore the ability for logview to look at .htaccess errors?

comment:10 Changed 6 years ago by geofft

Looks like that is in fact the pending work, and we'll either need to patch Apache or run something to watch error_log (as described-ish in 3 above, although ignore the bits about barnowl) to route errors from .htaccess files into users' journals.

Note: See TracTickets for help on using tickets.