Better package management for eggs, gems, and other vaguely spherical-shaped objects

[andersk]

We currently install a bunch of Perl CPAN modules, Python eggs, and Ruby gems behind RPM’s back. But we have no plan in place for upgrading them, making sure they don’t conflict with RPMs, or making sure they get superseded by new Fedora packages when they become available. This is a disaster waiting to discovered.

Everything should be an RPM. There are tools for automatically converting these creatures into RPMs, such as cpanspec, setup.py bdist_rpm, and gem2rpm. We should start using them and get rid of all the unpackaged modules lying around.

[mitchb]

Actually, "secretly" we do have a plan to make sure they get superseded by new Fedora rpms. Which is to say that part of my as-yet-unpublished (no comment on how much is written) how-to on porting to a new release involves checking each of those things to see if there's a Fedora package for it. Actually, our server install directions already take care of that for the CPAN modules.

[geofft]

It would also be nice to have a utility to watch for upstream updates of things we package, and either ask us to rebuild them or autobuild them with the same specfile. (See also autodebathenificator.)

[ezyang]

Would be nice to see this happen in time for Fedora 17; would remove a bunch of steps at least from the transition.

[andersk]

scripts / trac-#131 / andersk  14:22  (Anders Kaseorg)
    Am I crazy to suggest that, possibly even as soon as the F15
    transition, we go move all the non-RPM-installed modules to a
    separate installation prefix that is not in any languages’ default
    search path?  Then,

    • We don’t have to worry about extra modules conflicting with RPMs;
    • We can more easily check what we have installed and what needs to
      be upgraded;
    • We don’t need to immediately flip out and RPMify everything, but we
      can start doing so on a per-module basis if we want;
    • Users can opt-in to getting extra modules they need on particular
      projects;
    • Users don’t have to worry about accidentally pulling in our
      poorly-maintained and rarely updated libraries for projects that
      don’t need them.

See zlogs in /mit/sipbzlog/scripts-by-instance/trac-#131 (2011-09-08).

[ezyang]

We punted the alternative paths idea for F17, because we hadn't figured out all the details in time for testing period. Let's try again later.

[ezyang]

Notes from Zephyr: one problem with achieving this plan is dealing with the deluge of RPMs this will result in. Attempting to upstream all of these RPMs is unlikely to win us the praise of Fedora maintainers. A better idea is to have a system like packagefab/koji which will handle this RPMs for us.

We also want to get better workflow auditing packages before we put them in standard locations which will be automatically looked at (and possibly run) by users. Geoffrey suggests some light static analysis to ensure that files are placed in the right location and "active" files like pth files are reasonable. Given the relative rarity this operation happens I would personally like it if this process is lightweight.

[andersk]

Scripts F20 is now sphereoid-free.