Guide to TechDocs for administrators and content creators

Key details
URL: https://techdocs.cseunsw.tech Access initially only from UNSW subnetworks All wiki pages are public Only authorised users can edit SMB content creation share: smb://techdocs.cseunsw.tech/techdocs SSH login access available pandoc is installed
Creating and editing teaching content
Updating the MediaWiki software

This is the UNSW CSE TechDocs MediaWiki server. It's:

A MediaWiki server delivering technical information on CSE's computer systems to CSE students and other users,
A Linux server providing a collaborative content creation platform for authorised users,
A SAMBA server for file sharing,
An SSH server, and
A documentation automation platform for creating, managing and populating MediaWiki pages.

Style guide

Style guide

MediaWiki help pages

pandoc

pandoc is installed. Amongst other things, it can be used to convert from HTML (e.g., old, handwitten web pages) to MediaWiki wikitext by doing something like this:

$ pandoc -f html -t mediawiki -o newpage.wiki oldpage.html

The result of the conversion can then be pasted in to a TechDocs wiki page.

MediaWiki maintenance scripts

The maintenance directory in the untarred MediaWiki directory (i.e., /usr/local/mediawiki) contains a large number of PHP scripts which can be used for maintenance and administration. See the file list below from v1.38.1, ponder the README and then maybe look at nukePage or getText.php).

Usually, if you run the PHP scripts in the maintenence directory without parameters you'll get a usage message. E.g.:

# php undelete.php
Argument <pagename> required!

Undelete a page

Usage: php undelete.php blah blah blah… ("blah blah blah…" added by the editor)

<lots more about options>

MediaWiki maintenance directory file list

CodeCleanerGlobalsPass.php          findClasses.php                   populateInterwiki.php
CommandLineInc.php                  findDeprecated.php                populateIpChanges.php
Doxyfile                            findMissingActors.php             populatePPSortKey.php
Maintenance.php                     findMissingFiles.php              populateRecentChangesSource.php
Makefile                            findOrphanedFiles.php             populateRevisionLength.php
README                              fixDefaultJsonContentPages.php    populateRevisionSha1.php
Sqlite.php                          fixDoubleRedirects.php            postgres
SqliteMaintenance.php               fixExtLinksProtocolRelative.php   preprocessorFuzzTest.php
TableCleanup.php                    fixMergeHistoryCorruption.php     protect.php
abstractSchemaChanges               fixTimestamps.php                 pruneFileCache.php
addChangeTag.php                    fixUserRegistration.php           purgeChangedFiles.php
addRFCandPMIDInterwiki.php          formatInstallDoc.php              purgeChangedPages.php
addSite.php                         generateConfigDoc.php             purgeExpiredBlocks.php
archives                            generateConfigSchema.php          purgeExpiredUserrights.php
attachLatest.php                    generateJsonI18n.php              purgeExpiredWatchlistItems.php
benchmarks                          generateLocalAutoload.php         purgeList.php
blockUsers.php                      generateSchemaChangeSql.php       purgeMessageBlobStore.php
categoryChangesAsRdf.php            generateSchemaSql.php             purgeModuleDeps.php
changePassword.php                  generateSitemap.php               purgeOldText.php
checkBadRedirects.php               getConfiguration.php              purgePage.php
checkComposerLockUpToDate.php       getLagTimes.php                   purgeParserCache.php
checkDependencies.php               getReplicaServer.php              reassignEdits.php
checkImages.php                     getText.php                       rebuildFileCache.php
checkUsernames.php                  grep.php                          rebuildImages.php
cleanupBlocks.php                   importDump.php                    rebuildLocalisationCache.php
cleanupCaps.php                     importImages.php                  rebuildall.php
cleanupEmptyCategories.php          importSiteScripts.php             rebuildmessages.php
cleanupImages.php                   importSites.php                   rebuildrecentchanges.php
cleanupInvalidDbKeys.php            importTextFiles.php               rebuildtextindex.php
cleanupPreferences.php              includes                          recountCategories.php
cleanupRemovedModules.php           initEditCount.php                 refreshExternallinksIndex.php
cleanupRevActorPage.php             initSiteStats.php                 refreshFileHeaders.php
cleanupSpam.php                     initUserPreference.php            refreshImageMetadata.php
cleanupTitles.php                   install.php                       refreshLinks.php
cleanupUploadStash.php              interwiki.list                    removeInvalidEmails.php
cleanupUsersWithNoId.php            invalidateBotPasswords.php        removeUnusedAccounts.php
cleanupWatchlist.php                invalidateUserSessions.php        renameDbPrefix.php
clearInterwikiCache.php             jsduck                            renameRestrictions.php
compareParserCache.php              jsparse.php                       renderDump.php
compareParsers.php                  lag.php                           resetAuthenticationThrottle.php
convertExtensionToRegistration.php  language                          resetPageRandom.php
copyFileBackend.php                 locking                           resetUserEmail.php
copyJobQueue.php                    makeTestEdits.php                 resetUserTokens.php
createAndPromote.php                manageForeignResources.php        rollbackEdits.php
createBotPassword.php               manageJobs.php                    runBatchedQuery.php
deduplicateArchiveRevId.php         mcc.php                           runJobs.php
deleteArchivedFiles.php             mctest.php                        runScript.php
deleteArchivedRevisions.php         mediawiki.Title                   shell.php
deleteAutoPatrolLogs.php            mergeMessageFileList.php          showJobs.php
deleteBatch.php                     migrateActors.php                 showSiteStats.php
deleteDefaultMessages.php           migrateArchiveText.php            sql.php
deleteEqualMessages.php             migrateComments.php               sqlite
deleteLocalPasswords.php            migrateFileRepoLayout.php         storage
deleteOldRevisions.php              migrateImageCommentTemp.php       tables-generated.sql
deleteOrphanedRevisions.php         migrateRevisionActorTemp.php      tables.json
deleteSelfExternals.php             migrateUserGroup.php              tables.sql
deleteTag.php                       moveBatch.php                     term
deleteUserEmail.php                 mwdoc-filter.php                  tidyUpT39714.php
dev                                 mwdocgen.php                      undelete.php
doMaintenance.php                   mwjsduck-gen                      update-keys.sql
dumpBackup.php                      mysql.php                         update.php
dumpCategoriesAsRdf.php             namespaceDupes.php                updateArticleCount.php
dumpIterator.php                    nukeNS.php                        updateCollation.php
dumpLinks.php                       nukePage.php                      updateCredits.php
dumpTextPass.php                    pageExists.php                    updateExtensionJsonSchema.php
dumpUploads.php                     parse.php                         updateRestrictions.php
edit.php                            patchSql.php                      updateSearchIndex.php
emptyUserGroup.php                  populateArchiveRevId.php          updateSpecialPages.php
eraseArchivedFile.php               populateBacklinkNamespace.php     uppercaseTitlesForUnicodeTransition.php
eval.php                            populateChangeTagDef.php          userOptions.php
exportSites.php                     populateContentTables.php         validateRegistrationFile.php
fetchText.php                       populateExternallinksIndex60.php  version.php
fileOpPerfTest.php                  populateFilearchiveSha1.php       view.php
findBadBlobs.php                    populateImageSha1.php             wrapOldPasswords.php

Upgrading the MediaWiki software

The usual upgrade procedure within the SAME MAJOR VERSION is like the below. BUT, check the update procedure at http://www.mediawiki.org/ and VERIFY THE BELOW BEFORE PROCEEDING.

As root on the server…
In /usr/local/mediawiki/maintenance: php runJobs.php
In /root (the commands could still be in root's history):
- mysqldump --user=wikiuser --password='NOTPASSWORD' techdocs > file.sql
- mysqldump --user=wikiuser --password='NOTPASSWORD' techdocs --xml > file.xml
systemctl stop apache2
Untar new version in /usr/local
Delete old /usr/local/mediawiki symlink to the previous version
In /usr/local: ln -s mediawiki-<newversion> mediawiki
In /usr/local/mediawiki: ln -s ../mediawiki_config/LocalSettings.php .
In /usr/local/mediawiki/maintenance: php update.php
systemctl start apache2

Actions for site administrators

Note: There is no central account database (like LDAP or the UDB). Accounts for the wiki, SMB and SSH are all handled separately and manually.

Granting access

Creating a user account on the wiki with create and edit privileges

As an administrator…
Left-hand panel
Special pages
Login / create account
Create account
Enter details (no privileges need to be assigned as a user account automatically has the required privileges). Similar user name as CSE account is recommended
Create account
Done

Create local login account

Required for SSH or SMB access. Not required for wiki-only access

Note: All SMB user accounts automatically have access to the "techdocs" share.

As root on the host…
If only SMB access required (no SSH):
- useradd -s /usr/sbin/nologin <username> (an /etc/passwd entry for the user must exist)
If SMB and SSH access required:
- useradd -s /bin/bash -m -p x <username>
- Set up user's SSH key in the account's .ssh/authorized_keys file
smbpasswd -a <username> (supply desired password)

Edit access restricted by IPv4 address blocks

Although users can login from any IPv4 address, edits, new page creation and other changes are restricted to authorised users connecting from UNSW IPv4 addresses, namely 149.171/16 and 129.94/16.

Explanation: MediaWiki has the ability to block access by user name or IP address range. These blocks can be set up manually via the Special:Block page. However, the actual blocks discussed here are imposed by the script /usr/local/mediawiki_config/limit_edit_access_to_ipv4_address_ranges which, in turn, uses the Tcl script generate_ip_blocks in the same directory. The PHP maintenance script blockUsers.php is invoked by limit_edit_access_to_ipv4_address_ranges to make the actual database changes.

See the scripts for more detail.

See Help:Blocking users and Special:Block.

How techdocs was installed and configured

Install and configure techdocs server

Script to backup techdocs

Backup script

#!/bin/bash

# 1 MySQL dump
# 2 LocalSettings.php
# 3 /usr/local/mediawiki_images
# 4 /etc/apache2/apache2.conf
# 5 /var/www/html
# 6 /etc/samba/smb.conf
# 7 /home -> workarea
# 8 /var/lib/samba/private/passdb.tdb
# 9 CREDENTIALS.txt

SERVER="techdocs.cseunsw.tech"
DIVIDER="=============================="

cd "`dirname "$0"`"                                                             || exit 1

trap "rm -f techdocs.sql.tmp*" exit

echo "$DIVIDER"
ssh -l root -n -C $SERVER mysqldump                                             \
        --hex-blob                                                              \
        --add-drop-database                                                     \
        --databases techdocs > techdocs.sql.tmp                                 || exit 1
grep -v -- '^-- Dump completed on ' techdocs.sql.tmp > techdocs.sql.tmpx
if [ $? -gt 1 ]; then                                                           exit 1; fi
grep -v -- '^-- Dump completed on ' techdocs.sql > techdocs.sql.tmpz
if [ $? -gt 1 ]; then                                                           exit 1; fi
diff -q techdocs.sql.tmpx techdocs.sql.tmpz
case "$?" in
        0)      echo "No DB change"
                rm techdocs.sql.tmp*                                            || exit 1
                ;;
        1)      echo "DB changed"
                mv techdocs.sql.tmp techdocs.sql                                || exit 1
                rm techdocs.sql.tmp*
                ls -al techdocs.sql
                ;;
        *)      echo "Ooops" >&2
                exit 1
esac

echo "$DIVIDER"
rsync -vaz root@$SERVER:/usr/local/mediawiki_config/LocalSettings.php .         || exit 1

echo "$DIVIDER"
rsync -vaz --delete root@$SERVER:/usr/local/mediawiki_images .                  || exit 1

echo "$DIVIDER"
rsync -vaz root@$SERVER:/etc/apache2/sites-available/001-techdocs.conf .        || exit 1

echo "$DIVIDER"
rsync -vaz --delete root@$SERVER:/var/www/html/ var_www_html                    || exit 1

echo "$DIVIDER"
rsync -vaz root@$SERVER:/etc/samba/smb.conf .                                   || exit 1

echo "$DIVIDER"
rsync -vaz --exclude=".*" --delete root@$SERVER:/home/ workarea/                || exit 1

echo "$DIVIDER"
rsync -vaz root@$SERVER:/var/lib/samba/private/passdb.tdb .                     || exit 1

echo "$DIVIDER"
rsync -vaz root@$SERVER:CREDENTIALS.txt .                                       || exit 1

echo "$DIVIDER"

Forcibly changing a user's password as root

$ ssh -l root -A techdocs.cseunsw.tech
root@techdocs:~# cd /usr/local/mediawiki/maintenance/
root@techdocs:/usr/local/mediawiki/maintenance# php changePassword.php --user=TheUser --password=TheNewPassword
Password set for TheUser
root@techdocs:/usr/local/mediawiki/maintenance# history -c

The last step removes the password-changing command from the bash history so subsequent root sessions won't be able to see the new password.