User virtual machines on VLAB servers

Draft/test user-defined virtual machine setup for VLAB servers. Operational on vx01.

Virtual machine and virtual network configurations created by vmexec

Implement and run user-defined virtual machines (VM).

TL;DR

• VMs run as x86_64 Kernel Virtual Machines (KVM) under qemu.
• Can install VMs from scratch using ISO files as boot media.
• Virtual hard disk files and and CD/ISO image files *must* be in user subdirectories located somewhere under the directory listed below.
• Virtual hard disk files must be in qcow2 format.
• Three network configurations (see diagram):
  • Standalone VM using NAT to communicate with the outside world (-z0),
  • Gateway VM with one network interface using NAT to communicate with the outside world (as above), plus second network interface connected to a dedicated logical bridge creating a private virtual network (-z1), and
  • Private VM connecting to the dedicated logical bridge (see previous) (-z2).
  • The standalone and gateway configurations support using port forwarding to allow connections from the outside world to the VM's (such as to SSH servers or web servers running on the VM's).
• qemu-img is used to create virtual hard disk files.
• vmexecmkdir is used to create a user subdirectory on a host if one doesn't already exist.
• vmexec runs VMs on a host.

CD/ISO files and virtual hard disk files location

The CD/ISO files and virtual hard disk files used by the user-created virtual machines must actually be located somewhere under:

• /usr/local/uservms

on the host where the VMs will be run. If not, vmexec will give an error when trying to run the VM. Symlink-ing to locations outside of this directory will cause the same error.

Typically, users will have their own subdirectories under this directory.

Programs

Program name	Description
qemu-img	Create a virtual hard disk (file). Can use a pre-existing disk file, say from a class account, as a starting point.
vmexecmkdir	Create a user subdirectory for VM disk images and ISO files.
vmexec	Run a virtual machine, booting either from a supplied CD/ISO image or from a given virtual hard disk.

qemu-img

See the man page, but...

qemu-img -f qcow2 <disk_file_path> 20G

will create an empty 20G qcow2 disk.

And:

qemu-img -f qcow2 -b <base_file_path> <disk_file_path>

creates a disk whose initial contents are based on the given base disk image (such as in a class account). The size of the created disk will be that specified when the base file was created.

vmexecmkdir

Create a user subdirectory on a host for ISO files and virtual hard disk files (see location above).

Usage: vmexecmkdir [options]

-h  Display this help
-c  Create user directory in <see above>

vmexec

Usage: vmexec [options] [<disk0> [<disk1> ...]]

-h               Display this help
-m <n>           RAM megabytes (128/2048 - default 1024)
-n <n>           Number of virtual CPUs (1/2 - default 1)
-k <kernelpath>  Path to kernel image
-i <isopath>     CD/ISO file path
-d               Boot from CD/ISO
-x               Set display to "none"
-z <0|1|2>       Set network configuration (0=default/NAT, 1=gateway, or 2=internal)
-f <port1:port2> Set TCP port forwarding from external/host
                 port <port1> to internal/VM port <port2>.
                 May be used more than once. <port1> must be
                 between 8000 and 8999, inclusive.
                 Cannot be used with -z2

vmexec runs a user-defined virtual machine using qemu/KVM.

• The machine will be configured for "user" networking, meaning it will be on its own private subnetwork. It will be able to connect externally using Network Address Translation (NAT) but will not be reachable itself from external hosts, including from the host the virtual machine is running on. The VM will see network interface as an Intel E1000 card, and the hard disks connected via virtio block interface.
• It will have a graphical console.
• The [virtual] CPU will be Intel. There can be a maximum of TWO per VM.
• Disk and ISO paths can be absolute, or relative to the user's subdirectory (e.g. my-vms/disk0.img, ../comp1511/example.img, etc.)

Virtual networking

See also the diagram above.

The VM's are created with one of three possible networking configurations:

0	The default configuration. The VM is standalone (cannot communicate with other VM's belonging to the same user) and connects to the outside world through the physical host's own networking connection using NAT. Port forwarding can be configured (-f) to allow TCP connections from the outside world to the VM.
1	This is the gateway configuration where the VM has two virtual network interfaces. The first, like in the default configuration above, uses NAT to communicate with the outside world. The second network interface connects to a dedicated, private, logical network bridge belonging to the user (created at the same time the VM is created). This second network interface can be used as the gateway for the user's private VM's (see next configuration) to connect to the outside world.
2	This is the private VM configuration where the VM's network interface connects only to the private network created for a gateway VM (see previous).

Scribbles

Installing qemu-system-x86_64 package on nw-syd-armvx1:

Start-Date: 2023-06-01  14:04:59
Commandline: apt-get install qemu-system-x86
Install: qemu-system-x86:arm64 (1:5.2+dfsg-11+deb11u2), ovmf:arm64 (2020.11-2+deb11u1, automatic), libcapstone4:arm64 (4.0.2-3, automatic), libvdeplug2:arm64 (4.0.1-2, automatic), libbrlapi0.8:arm64 (6.3+dfsg-1+deb11u1, automatic), libexecs0:arm64 (1.3-1, automatic), libspice-server1:arm64 (0.14.3-2.1, automatic), ipxe-qemu:arm64 (1.0.0+git-20190125.36a4c85-5.1, automatic), libusbredirparser1:arm64 (0.8.0-1+b1, automatic), seabios:arm64 (1.14.0-2, automatic), libcacard0:arm64 (1:2.8.0-3, automatic), qemu-system-common:arm64 (1:5.2+dfsg-11+deb11u2, automatic), libvirglrenderer1:arm64 (0.8.2-5+deb11u1, automatic), qemu-system-data:arm64 (1:5.2+dfsg-11+deb11u2, automatic), qemu-system-gui:arm64 (1:5.2+dfsg-11+deb11u2, automatic)
End-Date: 2023-06-01  14:05:04