Changeset 2046 for branches


Ignore:
Timestamp:
Nov 11, 2011, 2:07:27 AM (11 years ago)
Author:
ezyang
Message:
Massive doc updates.
Location:
branches/fc15-dev/server/doc
Files:
2 edited

Legend:

Unmodified
Added
Removed
  • branches/fc15-dev/server/doc/install-howto.sh

    r1985 r2046  
    77# [WIZARD]     Semi-production server that will only have
    88#              daemon.scripts-security-upd bits, among other
    9 #              restricted permissions bits, among other
    10 #              restricted permissions bits, among other
    11 #              restricted permissions bits, among other
    129#              restricted permissions
    1310# [TESTSERVER] Completely untrusted server
    14 
    15 set -e -x
    16 
    17 # Some commands should be run as the scripts-build user, not root.
    18 
    19 alias asbuild="sudo -u scripts-build"
    20 
    21 # Old versions of this install document advised setting
    22 # NSS_NONLOCAL_IGNORE=1 anytime you're setting up anything, e.g. using
    23 # yum, warning that useradd will query LDAP in a stupid way that makes
    24 # it hang forever.  As of Fedora 13, this does not seem to be a problem,
    25 # so it's been removed from the instructions.  If an install is hanging,
    26 # though, try adding NSS_NONLOCAL_IGNORE.
    2711
    2812# This is actually just "pick an active scripts server".  It can't be
     
    4125server=YOUR-SERVER-NAME-HERE
    4226
    43 # Start with a Scripts kickstarted install of Fedora (install-fedora)
    44 
    45 # Take updates, reboot if there's a kernel update.
    46     yum update -y
    47 
    48 # Get rid of network manager
    49     yum remove NetworkManager
    50 
    51 # Copy over root's dotfiles from one of the other machines.
    52 # Perhaps a useful change is to remove the default aliases
    53     cd /root
    54     ls -l .bashrc
    55     ls -l .screenrc
    56     ls -l .ssh
    57     ls -l .vimrc
    58     ls -l .k5login
    59     # [PRODUCTION] This rc file has sensitive data on it and should only
    60     # be pushed onto production servers.
    61     ls -l .ldapvirc
    62     # Trying to scp from server to server won't work, as scp
    63     # will attempt to negotiate a server-to-server connection.
    64     # Instead, scp to your trusted machine as a temporary file,
    65     # and then push to the other server
    66 scp -r root@$source_server:~/{.bashrc,.screenrc,.ssh,.vimrc,.k5login} .
    67 scp -r {.bashrc,.screenrc,.ssh,.vimrc,.k5login} root@$server:~
    68 # [PRODUCTION]
    69 scp root@$source_server:~/.ldapvirc .
    70 scp .ldapvirc root@$server:~
    71 
    72 # Install the initial set of credentials (to get Kerberized logins once
    73 # krb5 is installed).  Otherwise, SCP'ing things in will be annoying.
    74 # Note that if you don't feel like forcefully ignoring "public key
    75 # changed" errors, all you need to transfer is /etc/ssh/ssh_host_rsa_key
    76 # initially.
    77 #   o Install the machine keytab.
    78     ls -l /etc/krb5.keytab
    79 #     Use ktutil to combine the host/scripts.mit.edu and
    80 #     host/scripts-vhosts.mit.edu keys with host/this-server.mit.edu in
    81 #     the keytab.  Do not use 'k5srvutil change' on the combined keytab
    82 #     or you'll break the other servers. (real servers only).  Be
    83 #     careful about writing out the keytab: if you write it to an
    84 #     existing file the keys will just get appended.  The correct
    85 #     credential list should look like:
    86 #       ktutil:  l
    87 #       slot KVNO Principal
    88 #       ---- ---- ---------------------------------------------------------------------
    89 #          1    5 host/old-faithful.mit.edu@ATHENA.MIT.EDU
    90 #          2    3 host/scripts-vhosts.mit.edu@ATHENA.MIT.EDU
    91 #          3    2      host/scripts.mit.edu@ATHENA.MIT.EDU
    92 #   o [PRODUCTION] Replace the ssh host keys with the ones common to all
    93 #     scripts servers (real servers only)
    94     ls -l /etc/ssh/*key*
    95 #     You can do that with:
    96 scp root@$source_server:/etc/ssh/*key* .
    97 scp *key* root@$server:/etc/ssh/
    98     # Actually, this appears to be unnecessary
    99     service sshd reload
    100 
    101 # Check out the scripts /etc configuration
    102     # backslash to make us not use the alias
    103     cd /root
    104     \cp -a etc /
    105     chmod 0440 /etc/sudoers
    106 
     27# ----------------------------->8--------------------------------------
     28#                       FIRST TIME INSTRUCTIONS
     29#
    10730# [PRODUCTION] If this is the first time you've installed this hostname,
    10831# you will need to update a bunch of files to add support for it. These
     
    12447#   o Set up Nagios monitoring on sipb-noc for the host
    12548#   o Set up the host as in the pool on r-b/r-b /etc/heartbeat/ldirectord.cf
    126     XXX TODO COMMANDS
    127 
    128 # NOTE: You will have just lost DNS resolution and the ability
    129 # to do password SSH in.  If you managed to botch this step without
    130 # having named setup, you can do a quick fix by frobbing /etc/resolv.conf
    131 # with a non 127.0.0.1 address for the DNS server.  Be sure to revert it once
    132 # you have named.
    133 
    134 # NOTE: You can get password SSH back by editing /etc/ssh/sshd_config (allow
    135 # password auth) and /etc/pam.d/sshd (comment out the first three auth
    136 # lines).  However, you should have the Kerberos credentials in place
    137 # so as soon as you install the full set of Scripts packages, you'll get
    138 # Kerberized logins.
    139 
    140 # Make sure network is working.  If this is a new server name, you'll
    141 # need to add it to /etc/hosts and
    142 # /etc/sysconfig/network-scripts/route-eth1.  Kickstart should have
     49#   o Update locker/etc/known_hosts
     50#
     51# You will also need to prepare the keytabs for credit-card.  In particular,
     52# use ktutil to combine the host/scripts.mit.edu and
     53# host/scripts-vhosts.mit.edu keys with host/this-server.mit.edu in
     54# the keytab.  Do not use 'k5srvutil change' on the combined keytab
     55# or you'll break the other servers. (real servers only).  Be
     56# careful about writing out the keytab: if you write it to an
     57# existing file the keys will just get appended.  The correct
     58# credential list should look like:
     59#   ktutil:  l
     60#   slot KVNO Principal
     61#   ---- ---- ---------------------------------------------------------------------
     62#      1    5 host/old-faithful.mit.edu@ATHENA.MIT.EDU
     63#      2    3 host/scripts-vhosts.mit.edu@ATHENA.MIT.EDU
     64#      3    2      host/scripts.mit.edu@ATHENA.MIT.EDU
     65#
     66# The LDAP keytab should be by itself, so be sure to delete it and
     67# put it in its own file.
     68# ----------------------------->8--------------------------------------
     69
     70# Start with a Scripts kickstarted install of Fedora (install-fedora)
     71
     72# Take updates, reboot if there's a kernel update.
     73    yum update -y
     74
     75# Get rid of network manager
     76    yum remove NetworkManager
     77
     78# This is superseded by credit-card, but only for [PRODUCTION]
     79# Don't use credit-card on [WIZARD]: it will put in the wrong creds!
     80#
     81#   # All types of servers will have an /etc/daemon.keytab file, however,
     82#   # different types of server will have different credentials in this
     83#   # keytab.
     84#   #   [PRODUCTION] daemon.scripts
     85#   #   [WIZARD]     daemon.scripts-security-upd
     86#   #   [TESTSERVER] daemon.scripts-test
     87
     88# Check out the scripts /etc configuration
     89    cd /root
     90    \cp -a etc /
     91    chmod 0440 /etc/sudoers
     92
     93# Make sure network is working.  Kickstart should have
    14394# configured eth0 and eth1 correctly; use service network restart
    144 # to add the new routes in route-eth1.
    145     service network restart
     95# to add the new routes from etc in route-eth1.
     96    systemctl restart network.service
     97    # Check everything worked:
    14698    route
    14799    ifconfig
     
    160112    rpm -e --nodeps rsyslog
    161113    yum install -y syslog-ng
    162     chkconfig syslog-ng on
    163 
    164 # [PRODUCTION/WIZARD] Fix the openafs /usr/vice/etc <-> /etc/openafs
    165 # mapping.
    166     echo "/afs:/usr/vice/cache:10000000" > /usr/vice/etc/cacheinfo
    167     echo "athena.mit.edu" > /usr/vice/etc/ThisCell
    168 
    169 # [TESTSERVER] If you're installing a test server, this needs to be
    170 # much smaller; the max filesize on XVM is 10GB.  Pick something like
    171 # 500000. Also, some of the AFS parameters are kind of retarded (and if
    172 # you're low on disk space, will actually exhaust our inodes).  Edit
    173 # these parameters in /etc/sysconfig/openafs
    174     echo "/afs:/usr/vice/cache:500000" > /usr/vice/etc/cacheinfo
    175     XXX TODO COMMANDS
    176 
    177 # Test that zephyr is working
    178     chkconfig zhm on
    179     service zhm start
    180     echo 'Test!' | zwrite -d -c scripts -i test
     114    systemctl enable syslog-ng.service
    181115
    182116# Install the full list of RPMs that users expect to be on the
     
    216150    rpm -i ghc-cgi*1.8.1*.rpm
    217151
    218 # Check out the scripts /usr/vice/etc configuration
    219     cd /root/vice
    220     \cp -a etc /usr/vice
     152# Note: Since ultimately we'd like to move away from using per-language
     153# package manager and all of these be RPMs, it is of questionable
     154# importance how much /good/ automation for these is necessary.
     155
     156# Warning: For a new release, we're supposed to check if Fedora has
     157# packaged up the RPM.  Unfortunately we don't really have good incants
     158# for this.
    221159
    222160# Install the full list of perl modules that users expect to be on the
     
    246184#   want to be able to write to ~/.python-eggs.  (Also makes sourcediving
    247185#   easier.)
    248 cat /usr/lib/python2.6/site-packages/easy-install.pth | grep "^./" | cut -c3- | cut -f1 -d- > egg.txt
     186# 'easy_install AuthKit jsonlib2 pygit'
     187cat /usr/lib/python2.7/site-packages/easy-install.pth | grep "^./" | cut -c3- | cut -f1 -d- > egg.txt
    249188    cat egg.txt | xargs easy_install -Z
     189
    250190# - Look at `gem list` for Ruby gems.
    251191#   Again, use 'yum search' and prefer RPMs, but failing that, 'gem install'.
    252192#       ezyang: rspec-rails depends on rspec, and will override the Yum
    253193#       package, so... don't use that RPM yet
     194# XXX This doesn't do the right thing for old version gems
    254195gem list --no-version > gem.txt
    255196    gem install $(gem list --no-version | grep -Fxvf - gem.txt)
     197    # Also, we need to install the old rails version
     198
    256199# - Look at `pear list` for Pear fruits (or whatever they're called).
    257200#   Yet again, 'yum search' for RPMs before resorting to 'pear install'.  Note
     
    262205    pear channel-update pear.php.net
    263206    pear install $(pear list | tail -n +4 | cut -f 1 -d " " | grep -Fxvf - pear.txt)
     207
    264208# - Look at `pecl list` for PECL things.  'yum search', and if you must,
    265209#   'pecl install' needed items. If it doesn't work, try 'pear install
     
    268212    pecl install --nodeps $(pecl list | tail -n +4 | cut -f 1 -d " " | grep -Fxvf - pecl.txt)
    269213
    270 # [PRODUCTION] Install the credentials.  There are a lot of things to
    271 # remember here.  Be sure to make sure the permissions match up (ls -l
    272 # on an existing server!).
    273 scp root@$source_server:{/etc/{sql-mit-edu.cfg.php,pki/tls/private/scripts.key,signup-ldap-pw,whoisd-password},/home/logview/.k5login} .
    274 scp signup-ldap-pw whoisd-password sql-mit-edu.cfg.php root@$server:/etc
    275 scp scripts.key root@$server:/etc/pki/tls/private
    276 scp .k5login root@$server:/home/logview
    277 #   o The SSL cert private key (real servers only)
    278     ls -l /etc/pki/tls/private/scripts.key
    279 #   o The LDAP password for the signup process (real servers only)
    280     ls -l /etc/signup-ldap-pw
    281 #   o The whoisd password (real servers only)
    282     ls -l /etc/whoisd-password
    283 #   o Make sure logview's .k5login is correct (real servers only)
    284     cat /home/logview/.k5login
    285 
    286 # All types of servers will have an /etc/daemon.keytab file, however,
    287 # different types of server will have different credentials in this
    288 # keytab.
    289 #   [PRODUCTION] daemon.scripts
    290 #   [WIZARD]     daemon.scripts-security-upd
    291 #   [TESTSERVER] daemon.scripts-test
    292 k5srvutil list -f daemon.keytab
    293 scp daemon.keytab root@$server:/etc
    294     chown afsagent:afsagent /etc/daemon.keytab
    295 #   o The daemon.scripts keytab (will be daemon.scripts-test for test)
    296     ls -l /etc/daemon.keytab
    297 
    298 # Spin up OpenAFS.  This will fail if there's been a new kernel since
    299 # when you last tried.  In that case, you can hold on till later to
    300 # start OpenAFS.  This will take a little bit of time;
    301     service openafs-client start
    302 # Then, check that fs sysname is correct.  You should see, among others,
    303 # 'amd64_fedoraX_scripts' (vary X) and 'scripts'. If it's not, you
    304 # probably did a distro upgrade and should update /etc/sysconfig/openafs.
     214# ----------------------------->8--------------------------------------
     215#                       INFINITE CONFIGURATION
     216
     217# Run credit-card to clone in credentials and make things runabble
     218python host.py push $server
     219
     220# [PRODUCTION/WIZARD] Fix the openafs /usr/vice/etc <-> /etc/openafs
     221# mapping.
     222    echo "/afs:/usr/vice/cache:10000000" > /usr/vice/etc/cacheinfo
     223    echo "athena.mit.edu" > /usr/vice/etc/ThisCell
     224# [TESTSERVER] If you're installing a test server, this needs to be
     225# much smaller; the max filesize on XVM is 10GB.  Pick something like
     226# 500000. Also, some of the AFS parameters are kind of retarded (and if
     227# you're low on disk space, will actually exhaust our inodes).  Edit
     228# these parameters in /etc/sysconfig/openafs
     229    echo "/afs:/usr/vice/cache:500000" > /usr/vice/etc/cacheinfo
     230    XXX TODO COMMANDS
     231
     232# Test that zephyr is working
     233    systemctl enable zhm.service
     234    systemctl start zhm.service
     235    echo 'Test!' | zwrite -d -c scripts -i test
     236
     237# Check out the scripts /usr/vice/etc configuration
     238    cd /root/vice
     239    \cp -a etc /usr/vice
     240
     241# [PRODUCTION] Set up replication (see ./install-ldap).
     242# You'll need the LDAP keytab for this server: be sure to chown it
     243# fedora-ds after you create the fedora-ds user
     244    ls -l /etc/dirsrv/keytab
     245    cat install-ldap
     246
     247    systemctl enable openafs-client.service
     248    systemctl enable dirsrv.service
     249    systemctl enable nslcd.service
     250    systemctl enable nscd.service
     251    systemctl enable postfix.service
     252    systemctl enable nrpe.service
     253    systemctl enable httpd.service # not for [WIZARD]
     254
     255    systemctl start openafs-client.service
     256    systemctl start dirsrv.service
     257    systemctl start nslcd.service
     258    systemctl start nscd.service
     259    systemctl start postfix.service
     260    systemctl start nrpe.service
     261    systemctl start httpd.service # not for [WIZARD]
     262
     263# Note about OpenAFS: Check that fs sysname is correct.  You should see,
     264# among others, 'amd64_fedoraX_scripts' (vary X) and 'scripts'. If it's
     265# not, you probably did a distro upgrade and should update
     266# /etc/sysconfig/openafs (XXX this is wrong: figuring out new
     267# systemd world order).
    305268    fs sysname
     269
     270# Postfix doesn't actually deliver mail; fix this
     271    cd /etc/postfix
     272    postmap virtual
     273
     274# Munin might not be monitoring packages that were installed after it
     275    munin-node-configure --suggest --shell | sh
     276
     277# Run fmtutil-sys --all, which does something that makes TeX work.
     278# (Note: this errors on XeTeX which is ok.)
     279    fmtutil-sys --all
     280
     281# Ensure that PHP isn't broken:
     282    mkdir /tmp/sessions
     283    chmod 01777 /tmp/sessions
     284    # XXX: this seems to get deleted if tmp gets cleaned up, so we
     285    # might need something a little better (maybe init script.)
     286
     287# Fix etc by making sure none of our config files got overwritten
     288    cd /etc
     289    svn status -q
     290    # Some usual candidates for clobbering include nsswitch.conf and
     291    # sysconfig/openafs
     292    # [WIZARD/TEST] Remember that changes you made should not get
     293    # reverted!
     294
     295# ThisCell got clobbered, replace it with athena.mit.edu
     296    echo "athena.mit.edu" > /usr/vice/etc/ThisCell
     297
     298# Reboot the machine to restore a consistent state, in case you
     299# changed anything. (Note: Starting kdump fails (this is ok))
     300
     301# When all is said and done, fix up the Subversion checkouts
     302    cd /etc
     303    svn switch --relocate svn://$source_server/ svn://scripts.mit.edu/
     304    cd /usr/vice/etc
     305    svn switch --relocate svn://$source_server/ svn://scripts.mit.edu/
     306    cd /srv/repository
     307    # Some commands should be run as the scripts-build user, not root.
     308    alias asbuild="sudo -u scripts-build"
     309    asbuild svn switch --relocate svn://$source_server/ svn://scripts.mit.edu/
     310    asbuild svn up # verify scripts.mit.edu works
     311
     312# ------------------------------->8-------------------------------
     313#                ADDENDA AND MISCELLANEOUS THINGS
     314
     315# [OPTIONAL] Your machine's hostname is baked in at install time;
     316# in the rare case you need to change it: it appears to be in:
     317#   o /etc/sysconfig/network
     318#   o your lvm thingies; probably don't need to edit
    306319
    307320# [WIZARD/TESTSERVER] If you are setting up a non-production server,
     
    338351    vim /home/afsagent/renew # replace all mentions of daemon.scripts.mit.edu
    339352
    340 # [PRODUCTION] Set up replication (see ./install-ldap).
    341 # You'll need the LDAP keytab for this server: be sure to chown it
    342 # fedora-ds after you create the fedora-ds user
    343     ls -l /etc/dirsrv/keytab
    344     cat install-ldap
    345 
    346 # Make the services dirsrv, nslcd, nscd, postfix, and httpd start at
    347 # boot. Run chkconfig to make sure the set of services to be run is
    348 # correct.
    349     service nslcd start
    350     service nscd start
    351     service postfix start
    352     chkconfig nslcd on
    353     chkconfig nscd on
    354     chkconfig postfix on
    355 
    356 # [PRODUCTION]
    357     chkconfig dirsrv on
    358 
    359 # [PRODUCTION/TESTSERVER]
    360 # (Maybe WIZARD too once we start doing strange things to autoupgrade
    361 # installs behind firewalls.)
    362     service httpd start # will fail if AFS is not running
    363     chkconfig httpd on
    364 
    365 # nrpe is required for nagios alerts
    366     chkconfig nrpe on
    367 
    368 # [PRODUCTION] Check sql user credentials (needs to be done after LDAP
    369 # is setup)
    370     chown sql /etc/sql-mit-edu.cfg.php
    371 
    372 # Postfix doesn't actually deliver mail; fix this
    373     cd /etc/postfix
    374     postmap virtual
    375 
    376 # Munin might not be monitoring packages that were installed after it
    377     munin-node-configure --suggest --shell | sh
    378 
    379 # Run fmtutil-sys --all, which does something that makes TeX work.
    380 # (Note: this errors on XeTeX which is ok.)
    381     fmtutil-sys --all
    382 
    383 # Ensure that PHP isn't broken:
    384     mkdir /tmp/sessions
    385     chmod 01777 /tmp/sessions
    386     # XXX: this seems to get deleted if tmp gets cleaned up, so we
    387     # might need something a little better (maybe init script.)
    388 
    389 # Ensure fcgid isn't broken (should be 755)
    390     ls -ld /var/run/mod_fcgid
    391 
    392 # Fix etc by making sure none of our config files got overwritten
    393     cd /etc
    394     svn status -q
    395     # Some usual candidates for clobbering include nsswitch.conf and
    396     # sysconfig/openafs
    397     # [WIZARD/TEST] Remember that changes you made should not get
    398     # reverted!
    399 
    400 # ThisCell got clobbered, replace it with athena.mit.edu
    401     echo "athena.mit.edu" > /usr/vice/etc/ThisCell
    402 
    403 # Reboot the machine to restore a consistent state, in case you
    404 # changed anything. (Note: Starting kdump fails (this is ok))
    405 
    406 # [OPTIONAL] Your machine's hostname is baked in at install time;
    407 # in the rare case you need to change it: it appears to be in:
    408 #   o /etc/sysconfig/network
    409 #   o your lvm thingies; probably don't need to edit
    410 
    411353# [TESTERVER]
    412354#   - You need a self-signed SSL cert or Apache will refuse to start
     
    421363#     be an accepted vhost name
    422364#   - Look at the old test server and see what config changes are floating around
    423 
    424 # XXX: our SVN checkout should be updated to use scripts.mit.edu
    425 # (repository and etc) once serving actually works.
    426     cd /etc
    427     svn switch --relocate svn://$source_server/ svn://scripts.mit.edu/
    428     cd /usr/vice/etc
    429     svn switch --relocate svn://$source_server/ svn://scripts.mit.edu/
    430     cd /srv/repository
    431     asbuild svn switch --relocate svn://$source_server/ svn://scripts.mit.edu/
    432     asbuild svn up # verify scripts.mit.edu works
  • branches/fc15-dev/server/doc/upgrade-tips

    r2018 r2046  
    201201extensions other than the few we’ve whitelisted should be disabled by
    202202emptying their .ini files in /etc/php.d.
     203
     2048. Sending announcements
     205------------------------
     206
     207Once development work has finished, we need to allow users to test
     208their websites on the new servers.
     209
     210    SIPB Internal Testing: Send an email to scripts-team@mit.edu
     211    and -c sipb notifying them of testing procedure and known
     212    issues.
     213
     214    General Testing:
Note: See TracChangeset for help on using the changeset viewer.