System Installation Policies / opt_depot


Contents


Preface

The goals of these policies are to make any installed software easy to locate, easy to maintain and use, and easy to de-install. The purpose of this document is to set some guidelines as to where and how software should be installed on the Computer Center systems. When this document speaks of "software", the reference is to third-party or locally written software that has been added to the computer system and which is not part of the operating system as released by the vendor.

Also, the /admin directory is an CSD convention and was created to house security-sensitive or other specialized software that can only be run by root, i.e. the nightly backup program. This document will not focus on /admin, but will address the system administration policies as they relate to software to be installed in /v/site and/opt.


Background Information

Any software (as defined above) installed by CSD staff should reside in one of two locations:

/v/site/packages

/opt/depot

The /v/site filesystem is available to all computer systems via NFS, and should only contain software that is meant to be shared. There are two main subdirectories in /v/site: /v/site/packages and /v/site/os. The directory /v/site/packages is the one that most people will be using, and is the focus of this document. It contains a subdirectory for each installed product that is freely available, i.e. elm, perl, etc. The /v/site/os directory contains software that must be handled in a special way, and will be described later in the section entitled Exceptions.

The /opt filesystem is not available via NFS, and should contain software that is meant to be run only on the SPARCcenter 2000. There are several directories under /opt. The one of most concern in this document is /opt/depot which, like /v/site/packages, contains a subdirectory for each installed product. However, in addition to the subdirectories for the products which are specific to the SPARCcenter 2000, i.e. matlab, /opt/depot also contains soft-links to each product directory found in /v/site/packages. For example, /opt/depot/elm is a soft-link to /v/site/packages/elm. Therefore, /opt/depot contains an entry for each and every product installed on the SPARCcenter2000.

Each product is to be compiled and installed using /opt/depot/ package_name as the target directory, even if the product is physically located in the directory /v/site/packages/package_name. Furthermore, the product's executables should be installed in a bin subdirectory, any libraries should be installed in a lib subdirectory, and any man pages should be installed in a man subdirectory. For example, the product elm is installed into the directories/opt/depot/elm/bin, /opt/depot/elm/lib, and opt/depot/elm/man. It bears emphasizing that there is a difference between the physical allocation of software and the logical location of software. While sharable software is physically located in /v/site/packages, all software is logically installed in /opt/depot.

By utilizing some locally-written perl scripts, collectively known as opt_depot, the executables, libraries, and man pages that are generated as a result of compiling each individual product are soft-linked together into the directories /opt/bin, /opt/lib, and /opt/man. That is, all executables from each product can be found in /opt/bin, all libraries from each product are found in /opt/lib, and all man pages from each product are found in /opt/man. There will be other directories created in /opt as needed (/opt/info and /opt/include), depending on the products and the files they create.

Thus, /opt logically contains all software that has been added to the computer system. In this way, a single directory, /opt/bin, can be placed in the default pathname, PATH, in order to use all products that have been added to the system. The same is true for the directories /opt/lib and /opt/man, which are used in the LD_LIBRARY_PATH and MAN_PATH variables.


Installing New Software

  1. Determine where the software package should physically reside. The decision on which location (/v/site/packages or/opt/depot) is the appropriate choice is based on the following criteria:

    If the software is freely distributable, site licensed, or written by CSD and meant to be shared with other systems, then it should physically reside in /v/site/packages.

    If the software is not freely distributable, site licensed, or meant to be shared with other systems, then it should physically reside in /opt/depot.

  2. Create a directory, either in /v/site/packages or /opt/depot, for the package to be installed, e.g. /v/site/packages/gopher. Under that, create the appropriate subdirectory structure for the package, such as bin, lib, and man. For man pages, create the appropriate further subdirectory, such as manl. The permissions on each directory created should be 755. When installing a package that produces a single executable, it does not merit a directory to itself. In this case, use one of the following locations as the destination: /v/site/packages/site.misc or /opt/depot/opt.misc.

  3. If the software is to be installed into /v/site/packages, then run the script called opt_link (the absolute pathname is /v/site/opt_link). This script will use the contents of /v/site/packages as the basis for creating soft links in /opt/depot. That is, if there is a /v/site/packages/gopher, then it will create a soft link to it called /opt/depot/gopher.

    If the software is to be installed into /opt/depot, as opposed to /v/site/packages, then running opt_link is not necessary (but will not hurt).

  4. Install the software using /opt/depot/package_name as the directory in which to install the software. In many cases, the software's makefile will need to be modified to make /opt/depot/package_name the target directory. Executables generated should be installed in /opt/depot/package_name/bin, libraries should be installed in /opt/depot/package_name/lib, man pages should be installed in /opt/depot/package_name/man, and so on. Man pages should be installed under /opt/depot/package_name/man using the traditional man page structure, for example /opt/depot/X11R5/man/man1/mwm.1 and /opt/depot/X11R5/man/mann/xload.n.

    Remember: It is crucial that the software is compiled and installed so that it looks for all of its files using the path /opt/depot/<packagename>. This should be the only hard coded path in your package.

    Note that installing directly into a /opt/depot/<packagename> linked to your Central Package Archive only works if the machine on which you compiled the package has write access to your Central Package Archive. If the local machine compiling the package does not have write access to the Central Package Archive, you can make a directory anywhere on the local machine and make a symbolic link from /opt/depot/<packagename> to your new directory, run your installation process, make a tar file of /opt/depot/<packagename>, ftp this tar file over to your Central Package Archive, and de-tar. You can then run opt_setup to take linking everything into your Local Installation Directory.

    More concretely,

    1. mkdir /tmp/packagename
    2. ln -s /tmp/packagename /opt/depot/packagename
    3. make install
    4. tar cvf /tmp/packagename.tar /opt/depot/packagename
    5. ftp server, put packagename.tar
    6. (on server) tar xvf packagename.tar
    7. (on local machine) rm /opt/depot/packagename
    8. (on local machine) opt_setup -v

  5. Execute the script opt_depot -v (the absolute pathname is /v/site/opt_depot). This will scan the contents of /opt/depot and generate all the appropriate links in /opt/bin, /opt/lib, /opt/man, etc. The -v parameter tells opt_depot to be verbose. A log file will automatically be generated in /logs/opt_depot/opt.

    NOTE: The opt_depot script uses a lock file called lock.opt_depot to prevent two users from running opt_depot at the same time. This lock file is created in the destination directory (i.e. /opt) and is automatically removed when the run is completed. However, if the user CTRL C's out of opt_depot, the lock file is not removed and the next time opt_depot is invoked, an error message will be displayed indicating that someone else might still be running opt_depot. Remove the file /opt/lock.opt_depot and run opt_depot again.

  6. Finally, if any new man pages were installed into /opt/man as a result of running opt_depot, execute the command catman -w /opt/man to rebuild the /opt/man/windex database, which is used for the whatis command. Otherwise, man won't be able to find any new man pages that were installed.


Modifying Software

  1. When installing a new version of some software for which it is desirable to keep the older version around as well, create a new directory in the appropriate directory (/v/site/packages or /opt/depot) using the tilde (~) in the directory name. For example, one might have /v/site/packages/elm~1 and /v/site/packages/elm~2. Then, to specify which version of the software is preferred, modify the file /opt/.priority by placing the desired version before all other versions in the list. The highest priority version found will be the one linked in /opt/bin, /opt/lib, /opt/man, etc. The /opt/.priority file can contain either individual file names or software package names. See the section Dealing With Name Conflicts for more information on the priority file.

  2. Run opt_depot -v (the absolute pathname is /v/site/opt_depot) to make sure that all the appropriate links to /opt/bin, /opt/lib,/opt/man, etc. are there. In principle, this step may not be necessary if the new version is a strict replacement for the old version and all the links already exist. However, it never hurts to run opt_depot to be sure. The -v parameter tells opt_depot to be verbose. A log file will automatically be generated in /logs/opt_depot/opt.


De-Installing Software

  1. When ready to de-install some piece of software, remove the appropriate directory in /v/site/packages or /opt/depot, i.e. /v/site/packages/elm~1. If the software resides in /v/site/packages/site.misc/bin or /opt/depot/opt.misc/bin, then simply remove the single executable file and leave the directory structure intact.

  2. If the software had resided in /v/site/packages then it will be necessary to run the script opt_link (the absolute pathname is /v/site/opt_link), which will remove the appropriate soft link in /opt/depot for the package.

  3. Run the script opt_clean (the absolute pathname is /v/site/opt_clean) which will remove all soft links for the package in /opt/bin, /opt/lib, /opt/man, etc.


Dealing With Name Conflicts

Occasionally, there are files and packages to be linked into /opt which share the same name. When opt_depot is run, it will generate a "name conflict" error message when it has already created a link of a certain file name, and then finds a second file by the same name that needs to be linked. There are three ways to resolve file name conflicts when running opt_depot. The choice as to which is most practical depends on the individual situation.

  1. If two files share the same name but have no resemblance to each other, it might be practical to change the name of one of the files. This will prevent the conflict from occurring again.

  2. If the files are nearly identical but from different packages (e.g. license manager files like idl/bin/lmstat and imsl/bin/lmstat), then one of the files can be given a higher priority by listing it before the other conflicting files of the same name in /opt/.priority. By the same token, if there are several versions of a package and one wants to insure that a certain one will be linked in /opt, then the version to be linked should be specified in the /opt/.priority file.

    The /opt/.priority file can contain either individual file names or software package names, specified either by absolute pathname or pathname relative to the /opt/depot directory. Each entry should be separated by a newline character, and comments are denoted by a pound sign (#) as the first character on a line. The order in which file names or package names are placed in the file determines the priority of the file or package and the one with the highest prioritywill be the one linked in /opt/bin, /opt/lib, /opt/man, etc. If there are two files by the same name, and one is listed in the priority file and one is not, then the one listed has the highest priority. If neither of the files are listed in the priority file, then the first one linked remains.

    An example /opt/.priority file follows:

    # This is the /opt/.priority file for determining opt_depot linking priorities

    # Version 19.19 of gnuemacs will have priority over version 18.57

    gnuemacs~19.19/bin/emacs

    gnuemacs~18.57/bin/emacs

    # The lmstat file in idl will have priority over all other files by that name

    /opt/depot/idl/bin/lmstat

  3. If a file name or software package conflicts with another package in many different ways and is a rarely accessed package, then it may be best to totally exclude it from the opt_depot linking process by placing it in the /opt/.exclude file list. The /opt/.exclude file is formatted exactly the same as the /opt/.priority file; however, its function is to simply exclude files or packages from being linked into /opt.

    An example /opt/.exclude file follows:

    # This is the /opt/.exclude file to keep opt_depot from linking files

    # X Windows software X11R4 will not be linked into /opt

    /opt/depot/X11R4


Disaster Recovery

  1. If /opt/bin, /opt/lib, opt/man, etc. are hopelessly corrupted, run the script opt_nuke (the absolute pathname is /v/site/opt_nuke). Running this script will remove all the links in /opt, allowing the user to start all over again.

  2. Run the script called opt_link (the absolute pathname is /v/site/opt_link). This script will use the contents of /v/site/packages as the basis for creating soft links in /opt/depot. That is, if there is a /v/site/packages/gopher, then it will create a soft link to it called /opt/depot/gopher. In actuality, running this script may not be necessary if you trust the integrity of /opt/depot, but it won't hurt to run it and it will insure that /opt/depot is up-to-date.

  3. Execute the script opt_depot -v (the absolute pathname is /v/site/opt_depot). This will scan the contents of /opt/depot and generate all the appropriate links in /opt/bin, /opt/lib, /opt/man,etc. The -v parameter tells opt_depot to be verbose. A log file will automatically be generated in /logs/opt_depot/opt.


Exceptions

The exception to the policies described above is software that is a replacement for an executable that comes with the operating system, i.e. /usr/bin/login. In that case, the software should be installed in the directory that the operating system expects to find it, and the original version of the software should be renamed with an extension of .dist, i.e. /usr/bin/login.dist. In addition, a copy of the new executable should be kept in /v/site/os. In this way, the software is made available to others via NFS, but they must take some positive action in order to use it, such as copy it into their own /usr/bin directory. It will not be available in their default pathway.


Documentation

Every directory in /v/site/packages and /opt/depot should have a file called ARLUT_README, which should be an on-line version of what is kept in /sysmods.jnl and the Software Maintenance Logs. This file is to include (but is not limited to) such information as to what the package is and what it does, the name of the person installing it, the date of installation, where the sources were obtained, any special information about how the software was installed, and anything else that might be helpful to someone else. By no means should this file be considered a replacement for /sysmods.jnl or the Software Maintenance Logs, which are kept by the system administrators of each system.


Sources

All sources are to be kept in the /sources directory. There are three primary subdirectories: opt, site, and os. Sources for software that physically resides in /v/site/packages should be kept in /sources/site/package_name. Sources for software that physically resides in /opt/depot should be kept in /sources/opt/package_name. In cases of software that have version numbers associated with them, it is also appropriate to have a further subdirectory that indicates the version number. For example, one might have /sources/site/elm/elm1.0 and /sources/site/elm/elm2.0. Sources for software that is are placement for standard operating system software (such as /usr/bin/login), should be kept in /sources/os.


Closing

There is one more script in the opt_depot suite, called opt_setup. This script is not normally run on the Computer Center host machines. It was written for those systems which NFS mount the /v/site volume for the purpose of using the software located there. The process is fully documented elsewhere, but the basic steps required are that the system administrator NFS mount the /v/site volume and then run the script /v/site/opt_setup, which then calls opt_link,opt_clean, and opt_depot.


Server Side Directory Structure

In this diagram, /v/site is the OS specific NFS mounted site volume that contains all the site-wide packages for the laboratory. /v/site/os contains all the scripts that are used to manage /opt on the client side. /v/site/packages contains all of the installed packages in separate directories. These will be linked under /opt/depot as shown on the client side diagram.

Client Directory Structure