scripts.mit.edu

MIT SIPB Script Services for Athena

Is scripts.mit.edu appropriate for my site?

From section "General Questions"

In general, scripts.mit.edu is well-suited for dynamic sites (including blogs and wikis, as well as custom apps) for individuals and small groups, but its volunteer-run nature may make it poorly suited for larger groups (such as labs). IS&T has a page detailing other options for making a website at MIT, which can help you decide where to host your website.

scripts.mit.edu is run by volunteers from SIPB, MIT’s student computing group. It is neither run nor supported by IS&T. We provide best-effort support via email at scripts@mit.edu, but we can’t guarantee that we will be able to help with your problem or that we will have any particular response time, even for urgent requests. Please keep this in mind before using scripts.mit.edu for a mission-critical application.

scripts.mit.edu uses multiple servers for load-balancing and reliability, and has historically maintained very high uptime. However, we can make no guarantees about future uptime, and there are many Athena services outside our control whose outages can impact scripts.mit.edu sites.

One particular concern around around the best-effort support policy involves security issues. Scripts policy when we discover that a scripts account has been compromised and is running exploit code is to disable the account. We will only reenable it once the account owners confirm they have fixed the site, and a Scripts maintainer has verified that the exploit code and original vulnerability have been fixed. This can sometimes take a large amount of maintainer time, and therefore there may be significant delay before our volunteers have a chance to reenable the site. As a result, if your site being down for at least several days after a security issue is prohibitive, you should use a different hosting solution.

Also, scripts.mit.edu is based on MIT’s infrastructure, and thus the MITnet Rules of Use apply to all sites hosted on the scripts.mit.edu platform. In particular, use for private financial gain is prohibited (so don’t run your startup on scripts.mit.edu).

Can my group use scripts.mit.edu or sql.mit.edu with our Athena locker?

From section "General Questions"

Yes! All scripts.mit.edu services support both groups and individual users. Any administrator of your Athena locker can act on behalf of your group. The quick-start installer system may be a good way of getting started.

If you need to ssh to scripts.mit.edu in order to run commands on behalf of a group locker, you should do so using the special instructions provided on our ssh documentation page.

You can sign up for sql.mit.edu by following our instructions.

If you have any questions, feel free to contact scripts@mit.edu.

What are the names of the various scripts.mit.edu servers?

From section "General Questions"

scripts.mit.edu makes use of load balancing across several different servers to handle the enormous number of requests that are served each day. It is very important that you not hardcode the names of the servers into your scripts, because different servers may be in the load balancer pool at any given time.

You can find out the current list of active servers by visiting this URL at any time:
https://scripts.mit.edu/servers

You may also obtain this information via finger:

athena% finger @scripts.mit.edu

The section of the finger output that corresponds to balancing of web requests to ports 80, 443, and 444 is the part under the heading:
FWM 2 wlc persistent 600

How can I sign up for Scripts if my locker is not publicly listable?

From section "General Questions"

Note: We strongly recommend that most users keep the default permissions that their account came with (which includes a publicly-listable home directory). The advice below is intended for advanced users who are sure they wish to block public listing of their home directory, and makes it much more difficult for the Scripts Team and other support groups around MIT to debug user problems. Please carefully consider whether the difficulty you may experience when requesting assistance in the future is outweighed by the benefit of preventing your locker from being listed.


If you have removed permissions for your locker to be publicly listable and you try to sign up for scripts.mit.edu, you will receive the following error message:





ERROR:
The scripts servers cannot verify the permissions of the locker YOURLOCKER. This is probably because your locker is not publicly listable. You can remedy this signup problem and make your locker publicly listable by running “fs setacl /mit/YOURLOCKER system:anyuser l” (that’s a lowercase L at the end).


NOTE: This will make it possible for the public (including anyone viewing http://web.mit.edu/YOURLOCKER) to see the names of your files and the list of people who have access to them, though it will not cause the contents of your files to be publicly readable. If you are unwilling to have your locker listable by the public, please contact scripts@mit.edu for information about other ways to work around the problem.



The AFS kernel module on the scripts.mit.edu servers prevents the permissions that are granted to daemon.scripts (which is the AFS username that the scripts servers authenticate with) from being exercised except by the scripts.mit.edu user corresponding to the locker. This means that only the scripts user for your locker can see or write files in your locker that daemon.scripts has AFS permissions on. This is the basic way that we isolate users from each other.



Unfortunately, that means that until your user has been created by the signup process, if your locker is only listable to daemon.scripts, the scripts servers still cannot read it. The usual way to work around this is to run ‘fs setacl /mit/YOURLOCKER system:anyuser l’ if you’re willing to have your locker be world-listable. For the rare case where someone is unwilling to have their locker publicly listable, there is one alternative involving granting one of the site-defined capital letter AFS permissions (“E”) to daemon.scripts that tells the kernel module to allow our signup user to exercise the daemon.scripts permissions. We don’t generally recommend this because it’s fairly confusing and makes debugging problems rather difficult.



If you are absolutely certain that you need this configuration, you should grant the “l” and “E” permissions to daemon.scripts by running a command like this on Athena:


athena% fs setacl /mit/YOURLOCKER daemon.scripts lE

(That’s a lowercase “L” and an uppercase “E”.)

Can my scripts write data somewhere?

From section "General Questions"

You can give your scripts permission to write data to subdirectories of your Athena home directory by granting appropriate AFS access to daemon.scripts (we provide sample commands below).

Each scripts.mit.edu user can also write up to 10MB (or 10k inodes) to /tmp. You should only write truly temporary data to /tmp.

Here are some commands that might be helpful:

The following command will grant the script server write access to
your current directory (cd into the directory that you want to write to and
then run this command):

fs sa . daemon.scripts write

The following command will grant the script server write access to all subdirectories of your current directory (you could cd into a particular directory and then run this command in order to grant the script server write access to all subdirectories of that directory):

athrun scripts fssar daemon.scripts write

The following command will list the access control list for the current directory:

fs la .

Who is a locker administrator?

From section "General Questions"

For technical reasons (part of the signup process involves giving the scripts server access to the web_scripts directory) as well as practical ones (only authorized users should be able to act on behalf of the locker), we require that anyone who is setting up scripts services for a locker be a locker administrator.

You can view the locker administrators with the command fs la (filesystem list access), e.g.:

athena% attach scripts
athena% fs la /mit/scripts
Access list for /mit/scripts is
Normal rights:
  system:scripts-root rlidwka
  system:anyuser rl

Any entry with lowercase “a” at the end indicates a locker administrator. If the entry starts with “system:”, it refers to a Moira list; otherwise, it is a username. (The notable exceptions are system:anyuser, which refers to the public, and system:administrators, which includes only IS&T staff.) You can usually find out the members of a list with the command blanche, e.g., blanche scripts-root or using WebMoira. (Generally, you can also just e-mail the list.)

In general, we recommend maintaining a list (which must be marked as an “AFS group”) that has admininstrative access, rather than individually granting access to users. A list owner can use blanche or WebMoira (https://groups.mit.edu/webmoira/list/listname) to update the membership of lists.

Any existing locker administrator can make you directly or a list containing you an administrator using the fs sa (filesystem set access) command:

athena% attach locker
athena% fs sa /mit/locker system:listname all

(Or:
athena% fs sa /mit/locker username all
to grant access directly to a user.)

If you are unable to contact any locker administrators, you should have someone appropriate (e.g., an officer of the student group), which may be you, e-mail accounts@mit.edu.

What does "You cannot administer the locker!" mean?

From section "General Questions"

If you get the following error in Pony:

You cannot administer the $locker_name locker!

Then one of the following applies to you and you should rectify the problem with the given solution.

1. You are not an administrator of the locker (see Who is a locker administrator? for what this means and how to check). If this is the case, contact the administrator(s) of the locker to be added to the ACL of the locker.

2. A locker administrator removed system:anyuser l from the AFS ACL for the root of the locker (/mit/lockername/). If this is the case, please add it back. You can read this guide on AFS by SIPB.

Can I ssh to scripts.mit.edu to test my scripts?

From section "General Questions"

Yes, although you must sign up for at least one of the script services before you can connect. You can SSH to scripts.mit.edu from any computer supporting Kerberized SSH, which includes all Athena workstation or dialups. From Athena, you can use the following command:

athena% ssh -k scripts

If you want to log in to access a locker other than your own, you should log in with your locker name specified as the “username” for the login:

athena% ssh -k LOCKERNAME@scripts

(with LOCKERNAME replaced by the name of your locker)

If you’re trying to log into scripts.mit.edu from home and you don’t have an SSH client on your computer, you can log into athena.dialup.mit.edu from https://athena.dialup.mit.edu/, and log into scripts from there.

You can also log in directly from most personal computers if you install a Kerberized SSH client. Other authentication mechanisms (such as direct password-based logins) are not supported for security reasons.

Windows

To connect to scripts.mit.edu from a Windows computer, install MIT SecureCRT and Kerberos For Windows from http://web.mit.edu/software/win.html.

Then, open MIT SecureCRT (from the Start menu). Click the “Quick Connect” button in the toolbar of the Sessions dialog box that appears (the second button from the left). Fill out the dialog that appears, as follows:

Protocol: SSH2
Hostname: scripts.mit.edu
Port: 22
Firewall: None
Username: [your Athena user name, or locker name]

In the “Authentication” selectbox, scroll down and click on GSSAPI to highlight it. Make sure that the checkbox beside it is checked. Then, use the black arrows to the right of the box to move “GSSAPI” to the top of the list of Authentication methods.

Then, click Connect to connect to scripts.mit.edu. If you are prompted for a username and password, please make sure you installed Kerberos for Windows and selected GSSAPI; you cannot log in to scripts.mit.edu with a username and password.

From a Mac

To connect to scripts.mit.edu from a Mac, download and install the MIT Kerberos Extras from https://ist.mit.edu/kerberos/extras/mac/recommended. Then, open “Terminal” (in /Applications/Utilities/), and type:

kinit AthenaUsername
ssh -k LockerName@scripts.mit.edu

The “-k” flag to ssh doesn’t exist for older Mac OS X versions. For these versions and with Apple’s default ssh configuration, it is safe to not use this flag. If you have customized your ssh configuration, make sure you have “GSSAPIAuthentication yes” and “GSSAPIDelegateCredentials no” set for scripts.mit.edu.

From Linux or other UNIX

To connect from a Linux (or other UNIX) computer, install ssh and Kerberos, and set up Kerberos to use the ATHENA.MIT.EDU realm. Many Linux distributions provide packages that can do this for you. (If you are using Debian or Ubuntu, you may want to consider using Debathena, which will allow you to access many Athena services from your own computer.) Then, run the two Mac command-line commands listed above.

Why does reading files cause 'permission denied' errors on scripts.mit.edu?

From section "General Questions"

If you get this error when ssh’d to scripts and trying to access another locker, please make sure you’ve followed the directions to ssh to scripts. If you are getting this error when running scripts, or when accessing files under the locker you’re logged in as, please continue reading.

On scripts.mit.edu, your scripts are served using special Athena filesystem privileges associated with your account on scripts.mit.edu. These privileges allow your scripts.mit.edu account to access any file that is both 1) inside your AFS locker and 2) accessible by “daemon.scripts” (a special set of AFS credentials owned by the script server).

The signup process takes care of setting up your web_scripts directory so that daemon.scripts has access to your web_scripts directory and its subdirectories. To give daemon.scripts read access to other parts of your home directory, you can do, for example:

athena% add consult
athena% fsr sa ~/my-scripts-data/ daemon.scripts read

Certain kinds of “non-executable content” (for example, images) underneath your web_scripts directory are also served to the web using your scripts.mit.edu credentials. Here is the complete list of the file extensions served directly to the web using your scripts.mit.edu account.

Why isn't the 'add' command working?

From section "General Questions"

There are two cases where you can run into this problem: either on Athena, while signing up, or on scripts.mit.edu itself.

The command “add scripts” gives an error

The add command is a standard Athena command to manage lockers. If you’re having trouble getting add scripts to work, check the following:

  • Make sure you’re on a standard Athena Linux or Athena Solaris computer connected to the network, either a workstation (e.g., in the clusters), or a dialup such as athena.dialup.mit.edu.
  • If the prompt does not start with athena% or athena$, try opening a new terminal with the “Prompt” button, or try a different workstation.
  • If you get the error “add: Command not found,” you’re probably missing the login settings files (“dotfiles”), which set up the add command. Try the following:

    1. If you’re on a cluster machine (not a dialup), log out, click “Login Options”, select “Ignore customizations”, and then log in. This should attempt to avoid customized dotfiles.
    2. If this still doesn’t work, you’re probably missing your dotfiles entirely. Check that the files .login and .cshrc are present in your home directory. (These are hidden files, so you will have to use ls -a to list them.) If they aren’t there, copy them (as well as any other missing dotfiles) with the command
      cp /usr/prototype_user/.* ~
      Then log out and back in.

If you’ve determined you’re missing your dotfiles and you’re having trouble restoring them, you can ask IS&T for help.

Adding a locker from a script doesn’t work

scripts.mit.edu doesn’t support the add command, because it’s difficult to make it work correctly outside of the shell. As a replacement, however, the /mit directory is automounted, so using a file in /mit/lockername implicitly attaches the locker. This way, you can run software in lockers by giving its full path, e.g. /mit/moira/bin/blanche for the blanche client.

(The full path of locker software is generally /mit/locker/bin/program or /mit/locker/arch/@sys/bin/program. If you’re having trouble, add the locker on Athena and run type program or which program.

Do you enforce any resource limits?

From section "General Questions"

Yes. The server enforces the following resource limits on web scripts:

core file size          (blocks) 0
data seg size           (kbytes) 1048576
file size               (blocks) unlimited
pending signals                  32255
max locked memory       (kbytes) 32
max memory size         (kbytes) 524268
open files                       1024
pipe size            (512 bytes) 8
POSIX message queues     (bytes) 819200
stack size              (kbytes) 10240
cpu time               (seconds) 300
max user processes               4096
virtual memory          (kbytes) 524288
file locks                       unlimited

These limits exist primarily to protect the server from runaway scripts.

There are also some timeouts in the Apache configuration that apply to web scripts. In particular, if a CGI or FastCGI application goes 30 seconds without writing and flushing any data back to Apache, the request will be aborted.

Finally, there are also looser limits placed on processes launched from a non-web context:

core file size          (blocks) 0
data seg size           (kbytes) 1048576
file size               (blocks) unlimited
pending signals                  32255
max locked memory       (kbytes) 32
max memory size         (kbytes) 524268
open files                       1024
pipe size            (512 bytes) 8
POSIX message queues     (bytes) 819200
stack size              (kbytes) 10240
cpu time               (seconds) unlimited
max user processes               1024
virtual memory          (kbytes) 1048576
file locks                       unlimited

If you find yourself running into these limits on a regular basis, please contact us to let us know what is triggering the limits.

Can my scripts access the network?

From section "General Questions"

Yes.

Can my scripts send zephyrs?

From section "General Questions"

Yes. You can send unauthenticated zephyrs using “zwrite -d”.

How do I make a file accessible to Apache? (I'm getting a 403 Forbidden error)

From section "General Questions"

To mark a single file accessible to Apache

You can run

chmod 777 FILENAME
to make a single file named FILENAME accessible to the Apache web server.

To mark an entire directory world-readable

If you want to make an entire directory world-readable, you can change its AFS access control list in order to do so. You can do so by “cd”-ing to the directory and then running

fs sa . system:anyuser read
(You should run these commands on an Athena workstation or dialup, instead of directly on the scripts servers.)

It may take some time for Scripts to pick up the permission changes; you can speed up the process by running

athrun scripts for-each-server fs flushv

If you would like to make programs that are normally executable downloadable by the user, you will also need to create a .htaccess file containing the following:

<Files *>
SetHandler none
</Files>

This instructs Apache to pass the files through directly, rather than attempting to interpret them as code.

Technical Information

The file system on Athena (AFS) usually ignores chmod modes, but scripts.mit.edu normally restricts AFS access to a home directory to the owner of that home directory. However, the special mode 777 has special meaning on scripts.mit.edu (we run a modified AFS client), and indicates that the Apache web server may directly access the file. No other chmod modes will have the same effect.

This is usually not needed for actual scripts or even most static content (such as .html files), since those files are accessed via your user account. It was previously needed for .htaccess files, but now (as of June 2009) Apache is given implicit read permission on all files whose names begin with “.ht”, such as .htaccess and .htpasswd.

What language should I write my program in?

From section "Languages and Libraries"

We recommend that Python be used for new code, since it is a popular choice beyond MIT, much of the scripts.mit.edu team uses it, and any future maintainers of are relatively likely to know it (as MIT teaches many classes in Python). Popular Python frameworks include Django and flask with sqlalchemy.

PHP is probably the mostly commonly used language on scripts.mit.edu (WordPress and Mediawiki are written in it).

A wide variety of other languages also work, including Ruby on Rails.

Is language "foo" or software package "bar" installed?

From section "Languages and Libraries"

If you’d like to check whether the service has certain software installed, you can ssh to scripts.mit.edu from an Athena computer (you must first have signed up for one of the script services).

The service supports essentially any programming language with a Linux implementation. If you want to use a language (or language version) that is not installed on the server, you can either contact us at scripts@mit.edu (we might be able to install what you want) or you can use a shebang line (and provide your own binary in AFS). Since the server is running Linux, custom shebang lines must point to Linux-compatible executables.

Here’s a shebang line demo: code | output

We also have instructions for installing Python libraries into your locker and a more detailed FAQ about Java.

How can I install Python modules in my locker?

From section "Languages and Libraries"

First, note that many Python modules are already installed on scripts.mit.edu, and we’re generally pretty willing to install generally-useful software globally; email us at scripts@mit.edu to ask. This has the advantage that you won’t have to worry about keeping the packages up to date; in some cases, using out-of-date modules could lead to security vulnerabilities in your site.

Sometimes, however, you may want to install a specific Python module from PyPI in your locker, say because you want to modify it or because you need a different version than what’s installed globally. This is pretty straight forward. First, you need to create the directory to install into and give scripts.mit.edu permission to it. From an Athena prompt, do the following, where lockername is the name of your locker:

% add consult
% mkdir -p /mit/lockername/.local/lib/python2.7/site-packages/
% fsr sa /mit/lockername/.local/lib/python2.7/site-packages/ \
    daemon.scripts write

Then, to install a package, ssh into scripts.mit.edu as your locker and run

$ easy_install --user -Z packagename

where packagename is the package you want to install. To upgrade something you already have installed, simply do:

$ easy_install --user -Z -U packagename

These packages will be automatically available to Python scripts you run on scripts.mit.edu.

Note that because the modules are tied to the minor version of Python they were built with, you will need to install the modules again if the system Python is upgraded. Be sure to add yourself to the scripts-announce mailing list so you will be notified about such transitions.

How do I make IPython or manage.py shell work on scripts.mit.edu?

From section "Languages and Libraries"

IPython, a nicer interactive Python shell that is, among other things, used by Django’s “manage.py shell” command, doesn’t work by default on scripts.mit.edu because it isn’t able to write to your ~/.ipython directory. Assuming you don’t already use IPython on Athena, you can create a ~/.ipython directory that will work on scripts as follows, replacing LOCKER with the name of your locker:

  • Create a temporary ~/.ipython directory and make it accessible to scripts via
    mkdir /mit/LOCKER/.ipython
    fs sa /mit/LOCKER/.ipython daemon.scripts write
  • Log in to scripts (ssh LOCKER@scripts) and run ipython. Because the .ipython directory exists and is writable, ipython will be able to start, but will not configure itself completely.
  • Run the command __IP.user_setup('/mit/LOCKER/web_scripts/.ipython','','install') to create a completely-configured default IPython setup in your web_scripts directory, which is writable from scripts.mit.edu.
  • Log out of scripts. Remove the temporary ~/.ipython directory and move the real one in place with
    rm -r /mit/LOCKER/.ipython
    mv /mit/LOCKER/web_scripts/.ipython /mit/LOCKER

How do I enable additional PHP extensions?

From section "Languages and Libraries"

Several PHP extensions are installed on the scripts servers, but not enabled by default. (Since most scripts do not use these extensions, this saves startup time and memory.) You can load any of these extensions by creating a text file called php.ini in the same directory as your PHP script, and adding a line with:

extension = [extension name].so

one line for each extension you want to enable.

Note that this will override some settings in the global php.ini — most importantly, this will cause magic_quotes_gpc to turn on! You should make sure that your application unsets this option, or set magic_quotes_gpc = no in your own php.ini file.

If your site is composed of several subdirectories, instead of creating a php.ini for each one you can add scripts and then fix-php-ini in the root directory of your site where your php.ini is located. fix-php-ini will traverse the directory and place symlinks as needed.

The following are the extensions installed on the scripts servers, with links to more information about them from the PHP website.

Why does my PEAR channel say it does "not have REST dependency information available"

From section "Languages and Libraries"

PEAR writes out .txt files which, for historical reasons, are not served by default from Scripts to the world. Run fs sa PEAR_DIR/get system:anyuser read and athrun consult fsr sa PEAR_DIR/rest system:anyuser read. See also How do I make a file accessible to Apache

How can I install Ruby gems in my locker?

From section "Languages and Libraries"

First, note that many Ruby gems are already installed on scripts.mit.edu, and we’re generally pretty willing to install generally-useful software globally; email us at scripts@mit.edu to ask. This has the advantage that you won’t have to worry about keeping the packages up to date; in some cases, using out-of-date modules could lead to security vulnerabilities in your site. Furthermore, a frequent practice in the Ruby community is to have sets of Ruby packages associated with deployments, in which case installing a gem in your locker might be too global.

In order to install a Ruby gem to your locker, you will first need to create the directory to install into and give scripts.mit.edu permission to it. From an Athena prompt, do the following, where lockername is the name of your locker:

add consult
mkdir /mit/lockername/.gem
mkdir /mit/lockername/.bundle
mkdir /mit/lockername/bin
fsr sa /mit/lockername/.gem daemon.scripts write
fsr sa /mit/lockername/.bundle daemon.scripts write
fsr sa /mit/lockername/bin daemon.scripts write

To tell a bundler configured app to use this local Gem store, run this command on a Scripts server, in the directory of your application:

bundle config path ~/.gem/ruby/1.9.1

Then, to install a gem, SSH into Scripts and run:

gem install -l gemname

Does scripts.mit.edu support Ruby on Rails?

From section "Languages and Libraries"

Yes! We now have an autoinstaller for Ruby on Rails. Just run these commands on Athena to get started:

athena% add scripts
athena% scripts-rails

Does scripts.mit.edu support Jifty?

From section "Languages and Libraries"

You can install Jifty apps on scripts using the jifty locker.

How do I use ASP.Net Applications on Scripts?

From section "Languages and Libraries"

We do not recommend using ASP.NET on scripts.mit.edu, as few if any scripts team members and scripts-hosted sites use it, making it likely to break during server changes, and making you less likely to receive help with issues. We have a list of some recommended languages that you can look over instead.

That said, you may be able to get ASP.Net applications to work on Scripts with Mono.
To try, add the following to your .htaccess file:

DirectoryIndex index.aspx
Options ExecCGI FollowSymLinks
SetEnv MONO_SHARED_DIR "/tmp/$USERNAME"
AddHandler fcgid-script .asax .ashx .asmx .aspx .axd .config .cs .dll .rem .soap

FCGIWrapper "/usr/bin/fastcgi-mono-server2 /applications=/test:/mit/pweaver/web_scripts/test" .asax
FCGIWrapper "/usr/bin/fastcgi-mono-server2 /applications=/test:/mit/pweaver/web_scripts/test" .ashx
FCGIWrapper "/usr/bin/fastcgi-mono-server2 /applications=/test:/mit/pweaver/web_scripts/test" .asmx
FCGIWrapper "/usr/bin/fastcgi-mono-server2 /applications=/test:/mit/pweaver/web_scripts/test" .aspx
FCGIWrapper "/usr/bin/fastcgi-mono-server2 /applications=/test:/mit/pweaver/web_scripts/test" .ascx
FCGIWrapper "/usr/bin/fastcgi-mono-server2 /applications=/test:/mit/pweaver/web_scripts/test" .axd
FCGIWrapper "/usr/bin/fastcgi-mono-server2 /applications=/test:/mit/pweaver/web_scripts/test" .config
FCGIWrapper "/usr/bin/fastcgi-mono-server2 /applications=/test:/mit/pweaver/web_scripts/test" .cs
FCGIWrapper "/usr/bin/fastcgi-mono-server2 /applications=/test:/mit/pweaver/web_scripts/test" .dll
FCGIWrapper "/usr/bin/fastcgi-mono-server2 /applications=/test:/mit/pweaver/web_scripts/test" .rem
FCGIWrapper "/usr/bin/fastcgi-mono-server2 /applications=/test:/mit/pweaver/web_scripts/test" .soap

Replace /test:/mit/pweaver/web_scripts/test with the application’s name and path of you application in your web_scripts directory.

Is Java supported?

From section "Languages and Libraries"

Short answer: No.

There are two aspects to this question: general Java programs (short answer: we try) and Java Servlets (short answer: we’d like to).

General Java programs

We try to allow running Java programs on Scripts (such as cron jobs or CGI programs). However, keeping Java programs working requires ongoing work from the maintainers and the maintainers don’t use Java and therefore often won’t notice when it breaks. Consequently, we recommend picking a better-supported language (such as Python) if you have a choice.

The issue with Java programs is that by default Java will attempt to allocate a large fraction of the system’s memory when it starts up. On most systems this is fine, because the operating system will not reserve the memory until Java writes to it, rather than merely allocates it. However, to ensure that Scripts remains responsive for our many users, we limit how much memory a single process can allocate (unfortunately, Linux does not allow limiting how much memory a process can use, except by limiting the amount a process can allocate). We attempt to configure Java and these memory limits such that Java can run within them, but sometimes part of this configuration breaks without our noticing.

Java Servlets

We do not currently support running Java Servlets. We would like to, and would welcome SIPB prospectives interested in adding support, but expect that it would take significant work.

How can I set a custom error page on Scripts?

From section "Web Script Service"

If you want to make Scripts use a customized error page for a 404 (File Not Found) error, you need to place the following lines in your .htaccess file:


ErrorDocument 404 /~lockername/404.html

Remember to replace lockername with the name of your locker, and 404.html with the location of the customized error page.

If you have additional error pages, create an additional line for each error code. Typical error codes include 401 (Unauthorized) and 403 (Forbidden).

What are scripts.mit.edu security updates?

From section "Web Script Service"

If you use the scripts.mit.edu automatic installers, the software that you install will be set up so that it can receive automatic security updates from the scripts.mit.edu maintainers. If you plan to manage security updates for your software on your own, you should remove system:scripts-security-upd from the AFS ACLs of the directories of your automatically-installed software.

(We will attempt to deal with security problems in the automatically-installed software, but we cannot realistically make meaningful guarantees about security, as we are volunteer-run.).

How do I redirect visitors from my web.mit.edu or www.mit.edu URL to my scripts.mit.edu website?

From section "Web Script Service"

URLs on web.mit.edu (e.g., http://web.mit.edu/jbarnold/ or http://web.mit.edu/~jbarnold/, the tilde is optional) point to the top level of your Athena locker (/mit/jbarnold in this case).

URLs on www.mit.edu (e.g., http://www.mit.edu/~jbarnold/, the tilde is required) point to the www subdirectory of your Athena locker (/mit/jbarnold/www).

If you want visitors to your web.mit.edu or www.mit.edu URL to see your scripts site instead, first, cd to the directory corresponding to your web.mit.edu or www.mit.edu URL (for example, my directory /mit/jbarnold/www corresponds to the URL http://www.mit.edu/~jbarnold):

athena% attach jbarnold
athena% cd /mit/jbarnold/www

Next, create a .htaccess.mit file in this directory and add rewrite rules to it. For example, if I wanted to redirect both http://www.mit.edu/~jbarnold/ and http://web.mit.edu/jbarnold/www to https://jbarnold.scripts.mit.edu, I would add the following two lines:

Redirect 301 /~jbarnold https://jbarnold.scripts.mit.edu
Redirect 301 /jbarnold/www https://jbarnold.scripts.mit.edu

The first line handles redirecting http://www.mit.edu/~jbarnold to https://jbarnold.scripts.mit.edu and the second line (which is completely independent) handles redirecting http://web.mit.edu/~jbarnold to https://jbarnold.scripts.mit.edu.

Finally, if you placed this in your locker, rather in the www subdirectory, you may need to give the web.mit.edu server access to your locker, so it can read your .htaccess.mit file (warning: this will also cause all the files in your locker to be readable over web.mit.edu):

athena% fs sa . system:htaccess.mit read

You should check that the redirect is working, and that people cannot read your personal files via web.mit.edu (e.g., http://web.mit.edu/jbarnold/).

You can find more information about .htaccess files and Apache Redirect rules via Google. IS&T has a website about .htaccess.mit files, but it focuses on restricting content, not redirects. If you have questions about how you should set up your .htaccess.mit file, feel free to contact us by e-mailing scripts@mit.edu.

You can also redirect visitors from your scripts.mit.edu site to another website.

Can I set up my scripts site to forward/redirect to another website?

From section "Web Script Service"

Yes. If you would like to redirect all web requests for a given URL on scripts.mit.edu to another website (such as web.mit.edu), follow these instructions:

Note: These directions are only appropriate for redirecting a URL on your scripts.mit.edu site to another URL that is not also on your scripts site. If you want to, for example, cause https://mylocker.scripts.mit.edu to take people to https://mylocker.scripts.mit.edu/wiki, please skip to the directions at the end of the page.

1. Create a directory for the redirect. In these examples, we will be redirecting https://mygroup.scripts.mit.edu/subpage/ to https://www.example.com/other/website/, so create the “subpage” directory:
athena% attach mygroup
athena% mkdir /mit/mygroup/web_scripts/subpage

2. Create a text file named .htaccess within your the new directory, e.g.:
athena% cd /mit/mygroup/web_scripts/subpage
athena% emacs .htaccess

3. Place two lines like the following in the .htaccess file:
RedirectPermanent /~mygroup/subpage/ https://www.example.com/other/site/
RedirectPermanent /subpage/ https://www.example.com/other/site/
This will send requests to either https://scripts.mit.edu/~mygroup/subpage/ or https://mygroup.scripts.mit.edu/subpage/ to the right address. It will also redirect, e.g., https://mygroup.scripts.mit.edu/subpage/foo.html to https://www.example.com/other/site/subpage/foo.html.

If you would like the redirect not to be cached by web browsers (e.g.,if you’re planning to change the redirect or set up a scripts site at this address), use “RedirectTemp” instead of “RedirectPermanent.”

If you’re also planning to get a mygroup.mit.edu-style address (a CNAME) to redirect to an outside site, then add at the bottom
RedirectPermanent / https://www.example.com/other/site/

If, instead of redirecting visitors to a different website, you wish to cause visitors who go to https://mylocker.scripts.mit.edu to be redirected to https://mylocker.scripts.mit.edu/wiki (or something similar), you can do the following:

1. Create a text file named .htaccess in your locker’s web_scripts directory (or edit an existing one), e.g.:
athena% cd /mit/mylocker/web_scripts
athena% emacs .htaccess

2. Place a line like the following near the top of the .htaccess file:
RedirectMatch ^/?$ /wiki
This will only send requests to https://mylocker.scripts.mit.edu to the initial address for the wiki. It will not interfere with the way the rest of your existing URLs work.

More information about redirect rules is available in the Apache documentation.

You can also set up your web.mit.edu or www.mit.edu site to forward to scripts.

How do I move a scripts.mit.edu site from one locker to another?

From section "Web Script Service"

If you are trying to move a MediaWiki, please use the detailed instructions here.

If you have a custom hostname that needs to be moved, perform the steps below and then contact scripts@mit.edu.

In this example, Ben Bitdiddle will be move a wiki to the 6.001 locker. First he makes sure the 6.001 locker is signed up for the scripts and SQL services:

athena% add scripts
athena% signup-web
...
athena% signup-sql
...

Then he copies over the files:

athena% attach bitdiddl 6.001
athena% cp -a /mit/bitdiddl/web_scripts/wiki /mit/6.001/web_scripts

Then he dumps the SQL database, creates a database named wiki for 6.001 by logging into sql.mit.edu as 6.001, and restores the dump into the 6.001 locker. A database dump can be created and restored either with phpMyAdmin, or with the command line:

athena% ssh -k scripts
[bitdiddl@scripts]$ mysqldump bitdiddl+wiki > database-dump.sql
[bitdiddl@scripts]$ exit
athena% mv database-dump.sql /mit/6.001/web_scripts
athena% ssh -k 6.001@scripts
[6.001@scripts]$ mysql 6.001+wiki < database-dump.sql

Finally, he edits the configuration files in /mit/6.001/web_scripts/wiki to use the 6.001 locker's username and password. For MediaWiki, this is done in the file LocalSettings.php.

How do I access the Authorization headers from a FastCGI?

From section "Web Script Service"

Apache by default does not pass the Authorization header through to FastCGI scripts. If you want access to it (perhaps for use with Ruby on Rails), you need to add the following to a .htaccess file:

RewriteCond %{HTTP:Authorization} ^(.+)
RewriteRule ^(.*)$ $1 [E=HTTP_AUTHORIZATION:%1,PT]

Does scripts.mit.edu publish information about who runs which websites?

From section "Web Script Service"

We don’t go out of our way to make this information available, nor to
protect it.

There is no listing of websites hosted on scripts.mit.edu or their
maintainers. If we’re asked to by IS&T staff (e.g., IT Security or User
Accounts) or law enforcement, we will identify the maintainer of a single
website. We may also do so if needed in the process of answering a
good-faith support ticket about the maintenance of that website.
Otherwise, we avoid giving this information out to the public for the
asking.

That said, for multiple technical reasons ranging from the configuration
of our servers to the properties of various parts of Athena infrastructure
we build upon, it’s generally possible to determine the ownership of a
website with some concerted effort, including which users or groups have
the ability to publish the site and who was the original creator of a file
in a locker. Most of this is due to the Athena locker model itself, and
applies equally to sites hosted on other services such as web.mit.edu.

Most other MIT hosting options will also reveal the owner’s identity
publicly through Moira’s hostname registry. If you wish to have a truly
anonymous website, you’ll need to get hosting independent of the MIT
infrastructure.

Can my group obtain a shorter URL?

From section "Web Script Service"

If you have a reasonable reason for wanting a short URL (in specific technical jargon, a DNS CNAME pointing to the script server and a corresponding virtual host configuration on the server), you can request one using our hostname management system. Note that this step is absolutely not required to use scripts.mit.edu; the default LOCKER.scripts.mit.edu URL is usually sufficient.

There are several types of hostname you can request:

  • By default, you are assigned a LOCKER.scripts.mit.edu host name; you can request arbitrary subdomains of this address (for example, blog.LOCKER.scripts.mit.edu). These host names are activated instantaneously, and do not require approval.
  • If you own an external hostname (e.g. example.com), you can point it at the Scripts server. Configure your hostname with your registrar or DNS hosting as an A record for 18.181.0.46, then add your hostname using the online interface as usual. Alternatively, you can configure it as a CNAME for “scripts-vhosts.mit.edu.”, which is less cryptic but interferes with using your hostname with mail_scripts, and also doesn’t work for top-level URLs (i.e., you won’t be able to do it for example.com, but you could for www.example.com).
  • For high visibility websites, you can request a .mit.edu hostname. You can use the stella command on Athena or via the web to check if the hostname you would like is available. You can also transfer hostnames that you or your group owns. These host names must be manually approved, and we recommend they be requested sparingly.

If our hostname management system doesn’t work for some reason, you can also request a hostname by e-mailing scripts@mit.edu. Please include the name you would like, and the directory you would like the host to point to (e.g., group.scripts.mit.edu or group.scripts.mit.edu/wiki/).

Can I add a vhost before I point my domain at scripts?

From section "Web Script Service"

The easiest approach is simply to point the domain at scripts before you set it up, but if you wish to migrate a currently active website, you may want to have the vhost set up on scripts before changing the DNS to avoid interruption.

If you want to do this, scripts will require that you create a file in the root directory of your existing website. For instance, if you are trying to add the domain “example.com” and it asks for the file “scripts_870shdfs8.html”, you should make sure that http://example.com/scripts_870shdfs8.html exists (though its content is irrelevant).

If you want to use a wildcard (e.g. *.example.com), you should make sure that all subdomains matching the wildcard point to the same place, and contain the file listed – for instance, for *.example.com, a.example.com, b.example.com and c.example.com should all contain the file.

Can I limit access to my scripts and/or other content using passwords?

From section "Web Script Service"

Yes. Follow these instructions to limit web access to a subdirectory of your web_scripts directory:

1. Decide upon one or more usernames and passwords that will be allowed to access the content. Do not use your Athena password!
2. On Athena, cd to the directory to which you want to limit access.
3. Run “add scripts”.
4. Run “webaccess allow USER_NAME” and enter the desired password for USER_NAME upon request.
5. Repeat step #4 for every username that you want to be able to access the content.
6. Use a web browser to confirm that access is being restricted correctly.

“webaccess remove USER_NAME” will remove a user’s access.
“webaccess reset” will clear all access restrictions on a directory.

(webaccess works by creating .htaccess and .htpasswd files for you)

You can also limit access using MIT certificates.

Can I authenticate people using MIT personal certificates?

From section "Web Script Service"

Yes. There are two ways to do so: one will limit access to an entire directory (with an .htaccess file), and the second will provide the information in the MIT personal certificate to your scripts.

Limiting access with .htaccess

Create a plain-text file called .htaccess in the directory to which you want to restrict access (for example, in the base directory of your wiki or other piece of software).

  • To allow anybody with an MIT certificate to access your site (i.e., to make your site MIT-only), add the following lines to the .htaccess file:
    • AuthType SSLCert
      Require valid-user
      ErrorDocument 401 /__scripts/needcerts
  • To restrict access to a particular set of one or more usernames:
    • AuthType SSLCert
      Require user username1 username2
      ErrorDocument 401 /__scripts/needcerts
  • To restrict access to members of a Moira list (aka traditional or blanche list) that is an AFS group:
    • AuthType SSLCert
      Require afsgroup system:group1 system:group2
      ErrorDocument 401 /__scripts/needcerts
  • If you want to allow access to members of either an existing list or a list of usernames, you’ll need to create a new list containing the existing list and any additional usernames.

(The ErrorDocument 401 /__scripts/needcerts line ensures that users are redirected to an HTTPS connection on port 444, which is configured to accept certificates. If the user has no certificates, they will be presented with a default error page; to use a custom error page, you can instead use ErrorDocument 401 /__scripts/needcerts/~username/customerror.cgi.)

If you are restricting a WordPress installation, you may need to edit the WordPress address and/or site address in the WordPress admin interface to add :444 after the hostname, to avoid an infinite redirect loop.

Note that the list in question must be visible and an AFS group. You can check the status by running blanche group -i at an Athena prompt: it should say “visible” under flags and “is a group” on the next line. (If you own the list, you can change these by running blanche group -V to make it visible and blanche group -G to make it an AFS group.)

Please double-check that the restrictions are enforced properly, e.g., by using a computer that doesn’t have certificates or asking someone else to try accessing the page. Note that scripts does not support .htaccess.mit files.

If you have any questions about using this system for restricting access, please contact scripts@mit.edu.

Finding certificate information from CGI scripts

If you access your website with HTTPS on port 444 (https://username.scripts.mit.edu:444/), the scripts server will request a certificate from the user. It will then create special SSL environment variables based on the user’s identity. The most useful is the variable SSL_CLIENT_S_DN_Email, which can be used to determine the Athena username of the individual currently accessing your script.

Here’s an example of listing these environment variables with Perl:
code | output without SSL | output with SSL

Here’s another example, using PHP to detect whether you’re using certificates:
code | output without SSL | output with SSL

If you’re using Django, a certificate authentication backend is available:
Details on the web | Code: git://snippets.scripts.mit.edu/.git or /afs/sipb.mit.edu/project/snippets/

You can also limit access using passwords.

Do my scripts need to be world-readable?

From section "Web Script Service"

No. By default, your scripts are not readable to other users.

How do I enable FastCGI?

From section "Web Script Service"

CGI vs FastCGI

Normal CGI scripts are initialized, run, and destroyed for every single browser request. The advantage of this model is that it is very simple, and thus robust.

If performance or latency becomes an issue, you can enable FastCGI. FastCGI is a web server module that speeds up your applications by keeping one interpreter running for multiple requests, instead of launching a separate interpreter for each request made while loading your page.

When FastCGI can help

This can lead to improvements in latency (responsiveness), for example if:

  • The language you are using takes a long time to start an interpreter, or to load libraries you are importing
  • You have a script which has significant repetitive launching overhead
  • Your scripts make the same, very large, database requests which (while cached by MySQL) still have to be read from memory and possibly cross an internal network
  • You are using a web framework like CherryPy, Turbogears, Pylons, or Ruby on Rails which prefers to run like a daemon

Caveats

There are some complications however:

  • The script in not guaranteed to die unless it kills itself
  • If you update your fcgi script, the old process (using the old script) may still be running with outdated code
  • The Scripts service automatically load-balances scripts across servers, just like regular CGI; new scripts may not even be launched on the servers the old scripts are running on, and thus cannot “kill all older versions of myself” since they may be on different servers

The canonical way around these issues is to have the script watch the timestamps of the files it has loaded, and kill itself if a modification timestamp changes. If you are using a framework such as CherryPy, this is already handled for you.

(Minor note: This method lends itself to race conditions on very short timescales during an update. It is also expensive if you are watching many files. Other solutions include updating a single “watch me” file with a version or hash after every update, or using transactional semantics. This is almost never an issue however.)

Enabling FastCGI

FastCGI is enabled by default for files with the .fcgi extension. To enable FastCGI for another extension, such as .rb, add the following lines to a plain text file named .htaccess in the top directory of your website:

<Files *.rb>
  SetHandler fcgid-script
</Files>

To enable FastCGI for another languages, change rb to your language’s extension (e.g., php, pl, etc.)

Are .htaccess.mit files supported?

From section "Web Script Service"

scripts.mit.edu uses .htaccess files instead of .htaccess.mit files.

You can use .htaccess files in order to restrict access to your website using MIT certificates.

You can also protect content using passwords.

Why isn't my symlink working?

From section "Web Script Service"

Although you can use symlinks underneath your web_scripts directory to point to other files to point to other files underneath your web_scripts directory, you cannot use symlinks to point to executables outside of your web_scripts directory (for example, you cannot point at executables in another Athena locker). scripts.mit.edu expects all of your web scripts to reside underneath your web_scripts directory.

Can I use scripts.mit.edu to generate an Apache-style log file for my web.mit.edu site?

From section "Web Script Service"

Yes. The combination of Server Side Includes on web.mit.edu, a Perl script, and the SIPB web script service allows you to get a log file similar to the data you would be able to get if you were hosting the web pages on your own server.

How to set it up:

1. Sign up to use scripts.mit.edu.
2. Save the fake_log.pl script into your ~/web_scripts directory with a name that ends in .pl.
3. Make sure your web pages have the extension .shtml, and insert the following code into your web pages, replacing USERNAME with your username (and, if you called fake_log.pl something else, use the name that you chose):

<!-- Generate data that looks like an Apache log -->
<img src="https://USERNAME.scripts.mit.edu/~USERNAME/fake_log.pl?uri=
<!--#echo var="DOCUMENT_URI" -->&amp;ref=
<!--#echo var="HTTP_REFERER" -->&amp;met=
<!--#echo var="REQUEST_METHOD" -->&amp;pro=
<!--#echo var="SERVER_PROTOCOL" -->" 
width="1" height="1" alt="fake image for tracking"/>

4. The fake_log.pl script logs each hit to a file called fake.log in your ~/web_scripts directory. This log is designed to be compatible with Apache log files, so that you can use any of the myriad tools for analyzing and visualizing data from Apache log files.

Some questions and answers:

How does this work?

The fake_log.pl script works by taking the data returned from several Server Side Includes that work on web.mit.edu and using that and other data to write out an entry with the same format that Apache uses.

Since you’re just feeding information into a Perl script instead of getting the real server logs, couldn’t someone feed the Perl script garbage, or misleading information?

Yes. Invalid information will probably make the Perl script just plain fail. This method of logging data does not prevent people from deliberately feeding you bad data, but in most cases, no one will care to take the time to do so.

Will this log hits from text-only browsers?
Since this tracking method relies on the browser loading images, this will not log hits from text-only browsers.

What file extensions are served directly to the web using my scripts.mit.edu account?

From section "Web Script Service"

When a file with one of the following extensions 1) is underneath your web_scripts directory and 2) is not protected by a .htaccess file, it will be served directly to the web using your scripts.mit.edu account (you may want to read this description of what files your account can access). In other words, you do not need to make files with the following extensions world-readable in order for them to be served directly to the web:

avi
css
doc
docm
docx
dot
dotm
dotx
eot
gif
htm
html
ico
il
jar
jpeg
jpg
js
mid
midi
mov
mp3
mpeg
mpg
odb
odc
odf
odg
odi
odm
odp
ods
odt
otf
otg
oth
otp
ots
ott
pdf
png
pot
potm
potx
ppa
ppam
pps
ppsm
ppsx
ppt
pptm
pptx
ps
svg
swf
tar
tgz
tif
tiff
ttf
wav
wmv
woff
woff2
xaml
xap
xhtml
xla
xlam
xls
xlsb
xlsm
xlsx
xlt
xltm
xltx
xml
xsl
zip

May I include that stylish "powered by scripts" image on my website?

From section "Web Script Service"

You are indeed welcome to include this “powered by scripts” image on any website served by scripts.mit.edu. If you include this image, we recommend that you make it a link to scripts.mit.edu using the following HTML:

<a href="https://scripts.mit.edu/">
<img alt="powered by scripts.mit.edu"
src="https://scripts.mit.edu/media/powered_by.gif" /></a>

powered by scripts.mit.edu

Why shouldn't I use scripts.mit.edu/~locker URLs?

From section "Web Script Service"

Early versions of the scripts.mit.edu webhosting platform used URLs of the form http://scripts.mit.edu/~locker/ (or https://scripts-cert.mit.edu/~locker/). Those URLs still work, but have long since been deprecated in favor of URLs like https://locker.scripts.mit.edu/. Current autoinstallers and documentation all use the new form of the URLs (if you find exceptions, please contact us).

The older ~ form of URLs, in addition to being aesthetically unpleasing, is incompatible with the modern web’s security model. Under the “same-origin policy”, browsers consider a site at https://scripts.mit.edu/~locker/ to be run by the same group as one at https://scripts.mit.edu/~evil/, and allow JavaScript from the latter to interfere with the former. (You can read more at Wikipedia or the Mozilla Developer Network.) As a result, using ~ URLs is discouraged and will stop working entirely at some point in the future.

How do I enable file uploads to my MediaWiki?

From section "MediaWiki"

To enable uploads, edit the LocalSettings.php file in your MediaWiki install directory, and change the line reading $wgEnableUploads = false; to $wgEnableUploads = true;.

MediaWiki by default allows uploads of images with the following extensions: png, gif, jpg, and jpeg. To upload more file types, you will also need to add lines such as

$wgFileExtensions[] = 'doc';
$wgFileExtensions[] = 'pdf';

for each extension (.doc and .pdf in this example) you would like to allow your site to upload. You can add these lines anywhere after the line starting with require_once and before the ?> at the end of the file.

Certain file types, such as .zip, cannot be properly uploaded with the default file-type detection system. If you have problems uploading certain file types, you may need to use an additional file-type detector, named “file”, with this command:

$wgMimeDetectorCommand = 'file -bi';

If you run into trouble, please check the MediaWiki web site or contact scripts@mit.edu.

How do I reset my password for MediaWiki?

From section "MediaWiki"

The easiest way to avoid this problem is to set an e-mail address in your account, so you can use the “Mail me a new password” feature. However, if you find that you’ve forgotten the password to an account on your wiki, and you don’t have an e-mail address set for it, you can reset its password as follows:

  1. Log in to phpMyAdmin with your SQL password.
  2. From the menu on the left, select the database corresponding to your wiki, and select the “user” table within it.
  3. Click “Browse” at the top, and note down the user ID for the account whose password you want to reset. (This will be 1 for the first account.)
  4. Click “SQL” at the top, and run the following command, replacing “uid” with the user ID and “newpassword” with the new password for the account:
    • UPDATE user SET user_password = MD5)) WHERE user_id = uid

How do I move my MediaWiki to a new locker?

From section "MediaWiki"

Unfortunately, it is a multi-step process to move a MediaWiki installation to a new locker.

To move a MediaWiki located in directory /mit/A/web_scripts/index to a new directory /mit/B/web_scripts/index:

1. Attach the two directories so as to make administrative tasks easier:
athena% attach A
athena% attach B

2. To sign up the second locker (if necessary)
athena% add scripts
athena% signup-web B

3. Copy the Databases using SQL.
3b) While in Locker B signup for sql
athena% signup-sql
3c) Sign into phpMyAdmin web interface at https://sql.mit.edu/
3d) You will need the username and password located in the .my.cnf file of each locker (i.e. /mit/A/.my.cnf).
3e) Sign into the SQL account for locker A using phpMyAdmin first.
3f) Click on A+index
3g) Use the export button to download a copy of the database to your computer.
3h) Exit and Sign into the SQL account for locker B (NOT phpMyAdmin)
3i) Create a new database using the “add” field in the bottom right of the list of databases.
3j) Sign into phpMyAdmin for locker B
3k) Click on B+index
3l) Use the import tool to import the file you just exported from your
desktop. Be sure to select the correct character set.

4. Recursively copy the software,
athena% cp -r /mit/A/web_scripts/index /mit/B/web_scripts

5. Update settings:
5a) Update the /mit/B/web_scripts/index/LocalSettings.php with the wgSitename,
wgScriptPath, wgDBname, wgDBuser, and wbDBpassword lines to reflect
the new locker

5b) Update the /mit/B/web_scripts/index/php.ini file with the correct locker username for the
upload_tmp_dir and the session.save_path (otherwise the computer will not
remember that you are logged in)

6) Be happy.

Helpful site: https://www.mediawiki.org/wiki/Manual:Moving_a_wiki

How do I enable math support in my MediaWiki wiki?

From section "MediaWiki"

Here is the procedure for enabling math support for a scripts.mit.edu wiki:

1) ssh to scripts.mit.edu

2) cd to your wiki’s math directory and run make

3) Edit the LocalSettings.php file in the base directory of your wiki so that the line

$wgUseTeX = false;

reads

$wgUseTeX = true;

You will then need to re-save any existing pages that contain equations in order to have the equations appear correctly.

If you encounter trouble with this procedure, please contact scripts@mit.edu.

How can I move a WordPress install?

From section "WordPress"

WordPress saves the URL to use to access it in its configuration. If you move the directory you installed WordPress into or start using a short URL without first updating that URL, you may accidentally lock yourself out of the WordPress admin interface.

To update the stored URL:

  1. Wait until you’re ready to start using the new URL. In particular, if you’re requesting a new hostname (like MY_GROUP.mit.edu), wait until the Scripts team contacts you to tell you that the new hostname is working, and confirm that http://MY_GROUP.mit.edu/ works. If you don’t wait, you will lock yourself out of the WordPress admin interface.
  2. Log into the WordPress admin interface using the old URL
  3. Select the “Settings” link in the sidebar
  4. Make “Blog address (URL)” blank and change “WordPress address (URL)” to the address you want to be able to access it at.
  5. You may get a “404 Not Found” message when you save the changes. This is likely not a problem, and you should see if finishing the move fixes it.
  6. Once you move the PHP scripts in AFS or your new hostname gets activated, you should be able to use the new URL

How can I restrict my Trac instance to only certain users?

From section "Trac"

If you wish to restrict access to your Trac instance to only certain people with MIT certificates, you may do so by editing the .htaccess file in the directory you installed Trac into. Note: There is also a .htaccess file in the tracdata subdirectory, and that file should not be changed.

Initially, when you've used the Trac autoinstaller, the .htaccess file will look similar this:

AuthType SSLCert
AuthSSLCertAuthoritative off
AuthSSLCertVar SSL_CLIENT_S_DN_Email
AuthSSLCertStripSuffix @MIT.EDU
Require valid-user
AuthOptional on

RewriteEngine on

RewriteCond %{HTTPS} =on
RewriteRule ^logout http://%{SERVER_NAME}%{REQUEST_URI} [R,L]

RewriteCond %{REQUEST_URI} !^/~yourlocker/trac-path/trac.fcgi
RewriteRule ^(.*)$ /~yourlocker/trac-path/trac.fcgi/$1 [L]

<Files login>
AuthOptional off
ErrorDocument 401 /__scripts/needcerts
</Files>

You will want to change it to be similar to the following (changes are highlighted in red):

AuthType SSLCert
#AuthSSLCertAuthoritative off
AuthSSLCertVar SSL_CLIENT_S_DN_Email
AuthSSLCertStripSuffix @MIT.EDU
Require user moe larry curly
#AuthOptional on
ErrorDocument 401 /__scripts/needcerts

RewriteEngine on

RewriteCond %{HTTPS} =on
RewriteRule ^logout http://%{SERVER_NAME}%{REQUEST_URI} [R,L]

RewriteCond %{REQUEST_URI} !^/~yourlocker/tracpath/trac.fcgi
RewriteRule ^(.*)$ /~yourlocker/trac-path/trac.fcgi/$1 [L]

<Files login>
AuthOptional off
ErrorDocument 401 /__scripts/needcerts
</Files>

How can I import an outside Django project?

From section "Django"

Configuring an outside Django project to run on scripts can be a bit tricky if you’ve only before had experience with “./manage.py runserver”. Here’s one way to get a Django project running on scripts.mit.edu:

  1. Start with the regular scripts Django autoinstaller
  2. Move the created /mit/lockername/Scripts/django/projname aside to /mit/lockername/Scripts/django/projname.auto
  3. Copy in your application’s code to /mit/lockername/Scripts/django/projname
  4. Update the paths in /mit/lockername/web_scripts/progname/index.fcgi
    1. Make sure that the sys.path.insert line points to the directory containing projname (likely /mit/lockername/Scripts/django/)
    2. Possibly add a sys.path.insert line that points to /mit/lockername/Scripts/django/projname, if your includes are appname.models, not projname.appname.models (or similar)
    3. Make sure the chdir line is right —- it should point to the directory containing your project code
  5. Add a symbolic link from /mit/lockername/web_scripts/projname/media/ to /mit/lockername/Scripts/django/projname/media/
  6. Merge the autoinstalled settings.py (that is, /mit/lockername/Scripts/django/projname.auto/settings.py) into your settings.py
    1. Copy the database information
    2. Copy the SECRET_KEY, or make your own new one
    3. Copy the ADMIN_MEDIA_PREFIX —- for scripts.mit.edu, this should be “/__scripts/django/static/admin/”
    4. Make sure the value of MEDIA_URL is reasonable (generally “/projname/media/”)
  7. Make sure to install any necessary dependencies (see How can I install Python modules in my locker? for instructions how to do so with easy_install)
  8. Set up your database appropriately, either by running “./manage.py syncdb” or by installing a database dump from a running copy of your project
  9. Once you’ve confirmed that your Django project works, delete /mit/lockername/Scripts/django/projname.auto/

How do I do certificate authentication with Django?

From section "Django"

Using certificate authentication with Django probably requires configuring another authentication backend. SIPB’s snippets collection has code and limited documentation on how to do this. If you have issues with those instructions, please let us know.

How can I signup my personal or group locker for the MySQL service?

From section "MySQL Service"

If your group does not already have a locker, start by mailing accounts@mit.edu to get one.

Please note: MySQL has hard-coded a 16 character limit for usernames into its server and client. Since all MySQL accounts are linked to Athena lockers by name, your locker must be no more than 16 characters to use the sql.mit.edu service.

Individuals or groups that have lockers can easily register for a MySQL account at an Athena prompt:

athena$ add scripts
athena$ signup-sql

Individuals wishing to register an account for themselves should select option #1 at the first prompt. To register an account for a group, select option #2 at the first prompt.

You can then read your password from /mit/locker/.sql/my.cnf. You can use this to log in to sql.mit.edu under the “SQL Password” option. You will need to create and drop databases from sql.mit.edu, but you can do all other operations from any MySQL client. We recommend using phpMyAdmin.

How can I access my MySQL databases?

From section "MySQL Service"

First you’ll need to sign up for sql.mit.edu.

After that, you must use the sql.mit.edu web interface to create and drop databases. A regular MySQL client, such as phpMyAdmin (described below), cannot be used for these operations.

All other SQL commands can be issued from any host, client, and/or script of your choice; simply connect to the MySQL server at sql.mit.edu using your new MySQL username and password. You can also access your databases using an online web interface called phpMyAdmin.

What is .my.cnf? What is my sql.mit.edu password?

From section "MySQL Service"

Your sql.mit.edu login information is in your locker, in the file “.my.cnf”. This is a hidden file (dotfile), so you will need to use ls -a instead of ls to see it. You can still view and edit the file just by specifying the filename.

athena% *cd /mit/*_locker_
athena% cat .my.cnf
[mysql]
host=sql.mit.edu
user=username
password=password

This file should be a link to “.sql/my.cnf” in your locker, and only accessible by the locker owner and the scripts server (regardless of whether the root of the locker is publicly readable or not).

If you update your MySQL password on the MIT SIPB MySQL interface, you will need to update this file manually with your new MySQL password. Otherwise, the scripts auto-installers, as well as the mysql command-line client, will not be able to log in properly. If you have already installed software, you will need to update the password there too (LocalSettings.php for MediaWiki, wp-config.php for WordPress, etc.)

If you have lost your MySQL password and, it’s not in your .my.cnf file, you can reset it for your personal account by logging into sql.mit.edu with your MIT certificate. If you have lost the password for a group locker, please e-mail sql@mit.edu for help getting it reset.

How can I backup and restore MySQL data?

From section "MySQL Service"

You can back up your SQL data in two ways: via the command-line, or using the phpMyAdmin graphical user interface. Note that phpMyAdmin cannot import large databases (to get around this limitation, you can import and export databases using the command-line).


Backing up and restoring with the command-line client

First, perform a kerberised ssh to scripts.mit.edu after sshing to an Athena workstation (lockername@scripts if your website is in a locker). Then, to export the database:

mysqldump user+database > filename.sql

To import a database dump, make sure the database exists (you may need to create it from sql.mit.edu) and run the command

mysql user+database < filename.sql


Backing up and restoring with phpMyAdmin

You can use phpMyAdmin to backup and restore data. However, importing will not work for large databases.

To backup your data using phpMyAdmin:

  1. Log in to phpMyAdmin with your SQL password.
  2. Select the database or table you want to export (if you want to export just one database or table), and click the link at the top-right labeled Export.
  3. If a list of databases or tables displays at the top, select the databases or tables you want to export: either use “Select All”, or Ctrl-click / Cmd-click to select multiple items. (There is no need to backup information_schema, since this is a read-only system table.)
  4. Ensure the export type is “SQL”. Check the “Save as file” box near the bottom, and select “gzipped” if you want to compress your exported data.
  5. Click Go and save the downloaded file somewhere safe.

To restore your data using phpMyAdmin:

  1. Make sure the database that you’re restoring into exists by creating it on the sql.mit.edu web interface (not phpMyAdmin) if necessary.
  2. Log in to phpMyAdmin with your SQL password.
  3. If you want to import a single database, choose that database from the left menu.
  4. Click Import at the top.
  5. Select the backup you downloaded previously (or any SQL-formatted database dump) and click Go.

How can I make Dreamweaver work with my MySQL databases?

From section "MySQL Service"

Dreamweaver’s MySQL interface interprets a “+” character as a space, so when looking for database “USERNAME+DBNAME” it looks for “USERNAME DBNAME”.

CGI scripts on the web have a long history of using + as a substitute for space.

To prevent Dreamweaver from doing this, percent-encode the “+” sign as “%2B”. Ask it to use a database name like:

  • USERNAME%2BDBNAME

for example,

  • presbrey%2Bwiki

Why do my mail scripts not generate bounces?

From section "Mail Script Service"

The scripts server interprets procmail failures as temporary failures, queueing the mail until the procmailrc is fixed. This is useful if your procmailrc is unreadable or broken, but it can make debugging difficult. If you want to explicitly bounce a message, you can use a stanza similar to this in your procmailrc file:

:0:
* ^Delivered-To:.*USER_NAME\+TAG@.*
{
        EXITCODE=67
        HOST
}

The list of recognized exit codes can be found in the sysexits(3) manpage.

How do I list my current crontab? How do I remove my crontab?

From section "Cron Service"

ssh to scripts.mit.edu from an Athena computer. “crontab -l” will list your current crontab. “crontab -r” will remove your crontab. Note that you must ssh from an Athena computer; no password will be required.

Can I serve Subversion or Git repositories on scripts.mit.edu?

From section "Version Control"

Yes, scripts supports virtual hosting for Subversion and Git repositories.

git autoinstaller

For git, we now provide an autoinstaller that will create a git repository and set it up for access using git’s smart HTTP transport, the git daemon (read-only), ssh (for locker administrators only), or standard filesystem access. To use the autoinstaller:

athena% add scripts
athena% scripts-git

To configure write access to your repository, edit the .htaccess file in the ~/web_scripts/repo folder that was setup. For example, you can setup client auth or you can restrict both read/write access by moving the access control lines outside of the <Files> block.

Manual setup for Subversion or git

If you want to use Subversion or set up git by hand, the process is more involved.

Create a folder named ~/Scripts/git or ~/Scripts/svn:

athena% add scripts
athena% signup-minimal
athena% mkdir -p ~/Scripts/git
athena% mkdir -p ~/Scripts/svn

Then either place your repository in that directory or create a symbolic link to your repository:

athena% ln -s ~/cool-repo.git ~/Scripts/git/

Make sure that your repository is readable by scripts, including all its subdirectories. For example, run:

athena% add consult
athena% fsr sa ~/Scripts/svn/repository/ daemon.scripts read

If you are using git, you need to make sure that your repository is marked as exportable:

athena% touch ~/Scripts/git/cool-repo.git/git-daemon-export-ok

Users can checkout the repository using:

git clone git://lockername.scripts.mit.edu/cool-repo.git
svn co svn://lockername.scripts.mit.edu/repo

scripts currently doesn’t support pushing via the git protocol. The smart HTTP transport will work, though to use that you should read git’s documentation or use our autoinstaller (described above). Subversion does support unencrypted commits, which you can configure with the files in the conf/ subdirectory of your repository. For this to work, scripts must also have write access to your repository, e.g.:

athena% athrun consult fsr sa ~/Scripts/svn/repository/ daemon.scripts write

You will also need to enable password authentication by removing the # from the start of the “password-db = passwd” line in ~/Scripts/svn/repository/conf/svnserve.conf, and by adding passwords to ~/Scripts/svn/repository/conf/passwd.

How can I avoid typing my password repeatedly when using git with the smart HTTP transport?

From section "Version Control"

It is possible to use git with client certificates. This avoids needing to repeatedly type passwords when using git with the smart HTTP transport. First, you will need a git repository on scripts using the smart HTTP transport. Our autoinstaller (add scripts; scripts-git) will do this for you.

Extracting a certificate

Next, you need to get a copy of your certificate. The easiest way to do this is probably to extract it from your browser.

In Chromium, go to Preferences, search for “SSL”, pick “Manage Certificates” -> “Your Certificates”, and then export your certificate. In Firefox, go to Preferences -> Advanced -> Encryption -> View Certificates and then export your certificates.

This will export your certificate as a PKCS#12 file, which git doesn’t know how to handle. To convert it to something git does know how to handle, run openssl pkcs12 -in cert.p12 -out cert.pem -nodes (leave off the -nodes if you want your certificate to be encrypted, but be aware that you’ll need to type your password repeatedly in that case).

Telling git to use your cert

To tell git about your certificate and the server, you’ll want to run

  git config http.sslCert ~/path/to/ssl/cert.pem
  git remote set-url origin https://locker.scripts.mit.edu/repo/repo.git
  git remote set-url --push origin https://locker.scripts.mit.edu:444/repo/repo.git

In particular, note the :444 in the latter URL, which causes scripts.mit.edu to use certificates.

Configuring the server to accept client auth

You’ll also need to make sure that the server accepts you when identified by cert. Replace the indented lines between “<Files _git-auth.cgi>” and “</Files>” in the .htaccess with one of the snippets from the certificate authorization FAQ.

Previous:
Next:
© 2004-2017, the SIPB scripts.mit.edu project.
These pages may be reused under either the GFDL 1.2 or CC-BY-SA 3.0.
Questions? Contact scripts@mit.edu.

You are currently connected to pancake-bunny.mit.edu.