[pve-devel] [RFC qemu-server] add config to command tests
Thomas Lamprecht
t.lamprecht at proxmox.com
Mon Nov 19 12:52:52 CET 2018
To have a better safety net for not introducing regressions and also
some functional checks as the QEMU command defines the layout
behavior of the VM.
Signed-off-by: Thomas Lamprecht <t.lamprecht at proxmox.com>
---
Lacks test cases, obviously, but sending this out to find out if it's OK in
general. Once we have the infrastructure in place it's easier to add some cases
anyway.
test/Makefile | 5 +-
test/cfg2cmd/README.adoc | 85 ++++++++++++
test/cfg2cmd/minimal-defaults.conf | 2 +
test/cfg2cmd/minimal-defaults.conf.cmd | 24 ++++
test/cfg2cmd/simple1.conf | 15 +++
test/cfg2cmd/simple1.conf.cmd | 32 +++++
test/run_config2command_tests.pl | 173 +++++++++++++++++++++++++
7 files changed, 335 insertions(+), 1 deletion(-)
create mode 100644 test/cfg2cmd/README.adoc
create mode 100644 test/cfg2cmd/minimal-defaults.conf
create mode 100644 test/cfg2cmd/minimal-defaults.conf.cmd
create mode 100644 test/cfg2cmd/simple1.conf
create mode 100644 test/cfg2cmd/simple1.conf.cmd
create mode 100755 test/run_config2command_tests.pl
diff --git a/test/Makefile b/test/Makefile
index 6777f4a..48ea0a4 100644
--- a/test/Makefile
+++ b/test/Makefile
@@ -1,6 +1,6 @@
all: test
-test: test_snapshot test_ovf
+test: test_snapshot test_ovf test_cfg_to_cmd
test_snapshot: run_snapshot_tests.pl
./run_snapshot_tests.pl
@@ -8,3 +8,6 @@ test_snapshot: run_snapshot_tests.pl
test_ovf: run_ovf_tests.pl
./run_ovf_tests.pl
+
+test_cfg_to_cmd: run_config2command_tests.pl cfg2cmd/*.conf
+ perl -I../ ./run_config2command_tests.pl
diff --git a/test/cfg2cmd/README.adoc b/test/cfg2cmd/README.adoc
new file mode 100644
index 0000000..fd9923a
--- /dev/null
+++ b/test/cfg2cmd/README.adoc
@@ -0,0 +1,85 @@
+QemuServer Config 2 Command Test
+================================
+Thomas Lamprecht <t.lamprecht at proxmox.com>
+
+Overview
+--------
+
+This is a relatively simple configuration to command test program.
+It's main goals are to better enforce stability of commands, thus reducing
+the likelihood that, for example, a migration breaking change which forgot to
+bump/check the KVM/QEMU version, slips through
+
+Further you get a certain regression and functional test coverage. You get a
+safety net against breaking older or not often (manual) tested setups and
+features.
+
+NOTE: The safety net is only as good as the test count *and* quality.
+
+
+Test Specification
+------------------
+
+A single test consists of two files, the input VM config `FILE.conf` and the
+expected output command `FILE.conf.cmd`
+
+Input
+~~~~~
+
+The `FILE.conf` are standard Proxmox VE VM configuration files, so you can just
+copy over a config file from `/etc/pve/qemu-server` to add a configuration you
+want to have tested.
+
+Output
+~~~~~~
+
+For the expected output `FILE.conf.cmd` we check the KVM/QEMU command produced.
+As a single long line would be pretty hard to check for (problematic) changes
+by humans, we use a pretty format, i.e., where each key value pair is on it's
+own line. With this approach we can just diff expected and actual command and
+one can pin point pretty fast in which component (e.g., net, drives, CPU, ...)
+the issue is, if any. Such an output would look like:
+
+----
+/usr/bin/kvm \
+ -id 101 \
+ -name vm101 \
+ ...
+----
+
+TIP: If the expected output file does not exist we have nothing to check, but
+for convenience we will write it out. This should happen from clean code, and
+the result should not get applied blindly, but only after a (quick) sanity
+check.
+
+
+Environment
+~~~~~~~~~~~
+
+It makes sense to have a stable and controlled environment for tests, thus you
+one can use the 'description' in VM configurations to control this. The
+description consists of all lines beginning with a '#' as first non-whitespace
+character. Any environment variable follows the following format:
+
+----
+# NAME: VALUE
+... rest of config...
+----
+
+There are the following variables you can control:
+
+* *TEST*: a one line description for your test, gets outputted one testing and
+ should described in a short way any specialty about this specific test,
+ i.e., what does this test wants to ensure.
+
+* *QEMU_VERSION*: the version we fake for this test, if not set we use the
+ actual one installed on the host.
+
+* *HOST_ARCH*: the architecture we should fake for the test (aarch64 or x86_64),
+ defaults to `x86_64` to allow making this optional and still guarantee
+ stable tests
+
+The storage environment is currently hardcoded in the test code, you can
+extend it there if it's needed.
+
+// vim: noai:tw=78
diff --git a/test/cfg2cmd/minimal-defaults.conf b/test/cfg2cmd/minimal-defaults.conf
new file mode 100644
index 0000000..bb22d05
--- /dev/null
+++ b/test/cfg2cmd/minimal-defaults.conf
@@ -0,0 +1,2 @@
+# TEST: minimal configuration to detect default changes
+smbios1: uuid=6cf17dc3-8341-4ecc-aebd-7503f2583fb3
diff --git a/test/cfg2cmd/minimal-defaults.conf.cmd b/test/cfg2cmd/minimal-defaults.conf.cmd
new file mode 100644
index 0000000..6c1f9d6
--- /dev/null
+++ b/test/cfg2cmd/minimal-defaults.conf.cmd
@@ -0,0 +1,24 @@
+/usr/bin/kvm \
+ -id 8006 \
+ -name vm8006 \
+ -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server,nowait' \
+ -mon 'chardev=qmp,mode=control' \
+ -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \
+ -mon 'chardev=qmp-event,mode=control' \
+ -pidfile /var/run/qemu-server/8006.pid \
+ -daemonize \
+ -smbios 'type=1,uuid=6cf17dc3-8341-4ecc-aebd-7503f2583fb3' \
+ -smp '1,sockets=1,cores=1,maxcpus=1' \
+ -nodefaults \
+ -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
+ -vnc unix:/var/run/qemu-server/8006.vnc,x509,password \
+ -cpu kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce \
+ -m 512 \
+ -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \
+ -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \
+ -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \
+ -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' \
+ -device 'VGA,id=vga,bus=pci.0,addr=0x2' \
+ -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \
+ -iscsi 'initiator-name=iqn.1993-08.org.debian:01:a1d15f6610fd' \
+ -machine 'type=pc'
diff --git a/test/cfg2cmd/simple1.conf b/test/cfg2cmd/simple1.conf
new file mode 100644
index 0000000..10a4c31
--- /dev/null
+++ b/test/cfg2cmd/simple1.conf
@@ -0,0 +1,15 @@
+# TEST: Simple test for a basic configuration with no special things
+# QEMU_VERSION: 2.12.1
+bootdisk: scsi0
+cores: 3
+ide2: none,media=cdrom
+memory: 768
+name: simple
+net0: virtio=A2:C0:43:77:08:A0,bridge=vmbr0
+numa: 0
+ostype: l26
+scsi0: local:8006/vm-8006-disk-0.qcow2,discard=on,size=104858K
+scsihw: virtio-scsi-pci
+smbios1: uuid=7b10d7af-b932-4c66-b2c3-3996152ec465
+sockets: 1
+vmgenid: c773c261-d800-4348-9f5d-167fadd53cf8
diff --git a/test/cfg2cmd/simple1.conf.cmd b/test/cfg2cmd/simple1.conf.cmd
new file mode 100644
index 0000000..f202ea8
--- /dev/null
+++ b/test/cfg2cmd/simple1.conf.cmd
@@ -0,0 +1,32 @@
+/usr/bin/kvm \
+ -id 8006 \
+ -name simple \
+ -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server,nowait' \
+ -mon 'chardev=qmp,mode=control' \
+ -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \
+ -mon 'chardev=qmp-event,mode=control' \
+ -pidfile /var/run/qemu-server/8006.pid \
+ -daemonize \
+ -smbios 'type=1,uuid=7b10d7af-b932-4c66-b2c3-3996152ec465' \
+ -smp '3,sockets=1,cores=3,maxcpus=3' \
+ -nodefaults \
+ -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
+ -vnc unix:/var/run/qemu-server/8006.vnc,x509,password \
+ -cpu kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce \
+ -m 768 \
+ -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \
+ -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \
+ -device 'vmgenid,guid=c773c261-d800-4348-9f5d-167fadd53cf8' \
+ -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \
+ -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' \
+ -device 'VGA,id=vga,bus=pci.0,addr=0x2' \
+ -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \
+ -iscsi 'initiator-name=iqn.1993-08.org.debian:01:a1d15f6610fd' \
+ -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' \
+ -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' \
+ -device 'virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5' \
+ -drive 'file=/var/lib/vz/images/8006/vm-8006-disk-0.qcow2,if=none,id=drive-scsi0,discard=on,format=qcow2,cache=none,aio=native,detect-zeroes=unmap' \
+ -device 'scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=100' \
+ -netdev 'type=tap,id=net0,ifname=tap8006i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' \
+ -device 'virtio-net-pci,mac=A2:C0:43:77:08:A0,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' \
+ -machine 'type=pc'
diff --git a/test/run_config2command_tests.pl b/test/run_config2command_tests.pl
new file mode 100755
index 0000000..00dbcbb
--- /dev/null
+++ b/test/run_config2command_tests.pl
@@ -0,0 +1,173 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use lib qw(..);
+
+use Test::More;
+use Test::MockModule;
+
+use PVE::Tools qw(file_get_contents file_set_contents run_command);
+use PVE::QemuConfig;
+use PVE::QemuServer;
+
+my $base_env = {
+ storage_config => {
+ ids => {
+ local => {
+ content => {
+ images => 1,
+ },
+ path => '/var/lib/vz',
+ type => 'dir',
+ shared => 0,
+ },
+ 'cifs-store' => {
+ shared => 1,
+ path => '/mnt/pve/cifs-store',
+ username => 'guest',
+ server => '127.0.0.42',
+ type => 'cifs',
+ share => 'CIFShare',
+ content => {
+ images => 1
+ },
+ },
+ 'rbd-store' => {
+ monhost => '127.0.0.42,127.0.0.21,::1',
+ content => {
+ images => 1
+ },
+ type => 'rbd',
+ pool => 'cpool',
+ username => 'admin',
+ shared => 1
+ },
+ 'local-lvm' => {
+ vgname => 'pve',
+ bwlimit => 'restore=1024',
+ type => 'lvmthin',
+ thinpool => 'data',
+ content => {
+ images => 1,
+ }
+ }
+ }
+ },
+ vmid => 8006,
+ real_qemu_version => PVE::QemuServer::kvm_user_version(), # not yet mocked
+};
+
+my $current_test; # = {
+# description => 'Test description', # if available
+# qemu_version => '2.12',
+# host_arch => 'HOST_ARCH',
+# config => { config hash },
+# expected => [ expected outcome cmd line array ],
+# };
+
+# use the config description to allow changing environment, fields are:
+# TEST: A single line describing the test, gets outputted
+# QEMU_VERSION: \d+\.\d+(\.\d+)? (defaults to current version)
+# HOST_ARCH: x86_64 | aarch64 (default to x86_64, to make tests stable)
+# all fields are optional
+sub parse_test($) {
+ my ($config_fn) = @_;
+
+ $current_test = {}; # reset
+
+ my $fake_config_fn ="$config_fn/qemu-server/8006.conf";
+ my $config_raw = file_get_contents($config_fn);
+ my $config = PVE::QemuServer::parse_vm_config($fake_config_fn, $config_raw);
+
+ $current_test->{config} = $config;
+
+ my $description = $config->{description} // '';
+
+ while ($description =~ /^\h*(.*?)\h*$/gm) {
+ my $line = $1;
+ next if !$line || $line =~ /^#/;
+ $line =~ s/^\s+//;
+ $line =~ s/\s+$//;
+
+ if ($line =~ /^TEST:\s*(.*)\s*$/) {
+ $current_test->{description} = "$1";
+ } elsif ($line =~ /^QEMU_VERSION:\s*(.*)\s*$/) {
+ $current_test->{qemu_version} = "$1";
+ } elsif ($line =~ /^HOST_ARCH:\s*(.*)\s*$/) {
+ $current_test->{host_arch} = "$1";
+ }
+ }
+}
+
+my $qemu_server_module;
+$qemu_server_module = Test::MockModule->new('PVE::QemuServer');
+$qemu_server_module->mock(
+ kvm_user_version => sub {
+ return $current_test->{qemu_version} // $base_env->{real_qemu_version};
+ },
+ get_host_arch => sub() {
+ return $current_test->{host_arch} // 'x86_64';
+ },
+);
+
+my $qemu_server_config;
+$qemu_server_config = Test::MockModule->new('PVE::QemuConfig');
+$qemu_server_config->mock(
+ load_config => sub {
+ my ($class, $vmid, $node) = @_;
+
+ return $current_test->{config};
+ },
+);
+
+sub do_test($) {
+ my ($config_fn) = @_;
+
+ die "no such input test config: $config_fn\n" if ! -f $config_fn;
+
+ parse_test $config_fn;
+
+ $config_fn =~ /([^\/]+)$/;
+ my $testname = "$1";
+ if (my $desc = $current_test->{description}) {
+ $testname = "'$testname' - $desc";
+ }
+
+ my ($vmid, $storecfg) = $base_env->@{qw(vmid storage_config)};
+
+ my $cmdline = PVE::QemuServer::vm_commandline($storecfg, $vmid);
+
+ $cmdline =~ s/ -/ \\\n -/g; # same as qm showcmd --pretty
+ $cmdline .= "\n";
+
+ my $cmd_fn = "$config_fn.cmd";
+
+ if (-f $cmd_fn) {
+ my $cmdline_expected = file_get_contents($cmd_fn);
+
+ my $cmd_expected = [ sort split /\s*\\?\n\s*/, $cmdline_expected ];
+ my $cmd = [ sort split /\s*\\?\n\s*/, $cmdline ];
+
+ # comment out for easier debugging
+ #file_set_contents("$cmd_fn.tmp", $cmdline);
+
+ is_deeply($cmd, $cmd_expected, "$testname")
+ } else {
+ file_set_contents($cmd_fn, $cmdline);
+ }
+}
+
+print "testing config to command stabillity\n";
+
+# exec tests
+if (my $file = shift) {
+ do_test $file;
+} else {
+ foreach my $file (<cfg2cmd/*.conf>) {
+ do_test $file;
+ }
+}
+
+done_testing();
--
2.19.1
More information about the pve-devel
mailing list