Software package management and Debian
New World hosts always run a supported version of Debian and always with as few changes and tweaks as necessary to achieve the required functionality.
This implies that the set of software packages installed on any New World host is a selection from the standard, supported pool of packages maintained by Debian themselves. This would be correct.
Because there are two groups of people in CSE who take an interest in exactly which packages are installed on a host -- namely CSG and teaching staff -- separate lists of packages are maintained by the two groups. The configuration manager (cfengine) ensures that each host has the appropriate list(s) of packages to be installed on it. A separate action ensures that new packages are installed as the lists change.
Initial installation of a New World host
Generally speaking, the process looks like this:
- Add a line for the host in
hostlist.csvand runupdate_hosts, - Update the CSE DHCP server if you're planning on a dynamic, as opposed to static, network configuration,
- Boot the host using the current Debian installation DVD and do a minimal install. Reboot when done,
- Initial install and setup of cfengine on a New World host,
- Go away and have a cake or some chocolate while cfengine does the rest,
- Come back and it'll all be done. Rebooting generally not required.
How the correct package lists are distributed to hosts
After the initial installation of the operating system, additional Debian packages are installed by cfengine when it detects changes to the lists of required packages. These lists of required packages are maintained by two sources:
- CSG-managed lists,
- Teaching staff-managed lists — see extra packages (and extra files while you're at it).
CSG-maintained package lists
Each package list maintained by CSG generally has an idea of some particular functionality behind it.
These packages are selected for a particular host by the contents of /var/lib/cfengine3/masterfiles/debianpackages.inc.
For example, hosts which are members of the class "vlabserver" will have the following package lists copied to them:
000vlabserver1.pkglist000vlabserver2.pkglist
Which contain the necessary packages to support the multi-user VNC login functionality of VLAB servers.
Similarly, hosts in the class "userlogin" will have the following package list copied to them:
000userlogin.pkglist
The above contains packages which include Kerberos and LDAP support so users can log in, both via VNC/VLAB and SSH.
The above mentioned debianpackages.inc also contains the trigger mechanism to cause package lists to be read after changes. Namely, in cfengine, when a package list is copied or updated the class "pkglist_changed" is created and this causes the command /usr/local/infrastructure/bin/debpkgs_install to be executed.
Teaching staff-managed package lists
See:
/var/lib/cfengine3/masterfiles/extrafiles.inc/var/lib/cfengine3/masterfiles/extraversions.inc/var/lib/cfengine3/masterfiles/extrapackages.inc
Similiar to CSG-managed lists, teaching package lists are copied due to promises in extrapackages.inc. List changes cause the class "extrapackages_changed" to be created by cfengine which, in turn, causes the scripts "/usr/local/infrastructure/bin/extrapkgs_install and /usr/local/infrastructure/bin/extrapkgs_purge to run.
How hosts update their installed packages when the list(s) change
The package lists mentioned above are plain text files consisting of one package name per line. Anything other than just package names is ignored, including package names containing version and/or repository specifications.
When one of the package list install scripts is run it merges and deduplicates all the package names and then uses apt-get to attempt an install of each package in sort order. Already-installed packages will be skipped by apt-get. If a named package does not exist or an error occurs during its installation that package only will be skipped and the installation process will continue.
Note that CSG-maintained package lists and teaching-maintained package lists are treated separately.
Auto-updating of installed packages
The standard Debian unattended-upgrades package is installed on all New World hosts. It is run daily by cron via /etc/cron.daily/apt-compat which, in turn, runs /usr/lib/apt/apt.systemd.daily.
You can find traces of its actions in:
/var/log/apt/history.log/var/log/unattended-upgrades/unattended-upgrades*.log
See also:
/etc/apt/apt.conf.d/21auto-upgradesetc/apt/apt.conf.d/50unattended-upgrades
Purging (removal) of installed packages
Lists of packages to install are contained in files ending in .list, but as a special case teaching-maintained package lists can end in .purge and, instead of installing, named packages listed in these files will be purged by apt-get. That is, completely removed. See extrapkgs_purge mentioned above and apt-get purge.