[pve-devel] [PATCH qemu-server v9 2/3] config: add AMD SEV support

[Fiona Ebner]

Am 26.04.24 um 11:58 schrieb Markus Frank:
> This patch is for enabling AMD SEV (Secure Encrypted Virtualization)
> support in QEMU.
> 
> VM-Config-Examples:
> amd_sev: type=std,no-debug=1,no-key-sharing=1
> amd_sev: es,no-debug=1,kernel-hashes=1
> 
> kernel-hashes, reduced-phys-bios & cbitpos correspond to the variables
> with the same name in qemu.

s/reduced-phys-bios/reduced-phys-bits/
s/qemu/QEMU/

> 
> kernel-hashes=1 adds kernel-hashes to enable measured linux kernel

The second time it should be "kernel hashes" instead of "kernel-hashes"

> launch since it is per default off for backward compatibility.
> 

---snip---

> diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
> index 2a349c8..c29809d 100644
> --- a/PVE/API2/Qemu.pm
> +++ b/PVE/API2/Qemu.pm
> @@ -4512,6 +4512,11 @@ __PACKAGE__->register_method({
>  	    push $local_resources->@*, "clipboard=vnc";
>  	}
>  
> +	# do not allow live migration with AMD SEV enabled

The comment does not add any information (it's already obvious from the
code). Maybe mention the migration attack instead?

> +	if ($res->{running} && $vmconf->{amd_sev}) {
> +	    push $local_resources->@*, "amd_sev";
> +	}
> +
>  	# if vm is not running, return target nodes where local storage/mapped devices are available
>  	# for offline migration
>  	if (!$res->{running}) {
> @@ -5192,6 +5197,12 @@ __PACKAGE__->register_method({
>  	die "unable to use snapshot name 'pending' (reserved name)\n"
>  	    if lc($snapname) eq 'pending';
>  
> +	my $conf = PVE::QemuConfig->load_config($vmid);
> +	if ($param->{vmstate} && $conf->{amd_sev}) {
> +	    die "Snapshots that include memory are not supported while memory"
> +		." is encrypted by AMD SEV.\n"
> +	}
> +
>  	my $realcmd = sub {
>  	    PVE::Cluster::log_msg('info', $authuser, "snapshot VM $vmid: $snapname");
>  	    PVE::QemuConfig->snapshot_create($vmid, $snapname, $param->{vmstate},

What about hibernate? That uses the very same mechanism under the hood
(savevm-start QMP command), so that should be prevented as well, right?

A helper would be good for the common checks. Could be called
check_non_migratable_resources() and start out with the checks for
clipboard and hostpci devices (currently present for hibernation). Your
series could then add the AMD-SEV check.

The helper can then be called by check_local_resources() (although then
we should avoid adding hostpci devices twice to the list of local
resources), as well as the snapshot and hibernation API calls (when
state is included).

Off-topic: I noticed the clipboard check is also missing from snapshot
and hibernate API calls, but it's not 100% clear to me if they should be
added right now or if we should wait for a (minor) release, see:
https://lists.proxmox.com/pipermail/pve-devel/2024-May/063896.html

So you could start out with just the AMD-SEV check until we decide to
enforce the VNC clipboard check for snapshot/hibernate and how to
properly avoid duplicates with the hostpci check. (A TODO comment for
those would be good).

> diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
> index 8d9b35a..340402a 100644
> --- a/PVE/QemuMigrate.pm
> +++ b/PVE/QemuMigrate.pm
> @@ -260,6 +260,10 @@ sub prepare {
>  	die "VMs with 'clipboard' set to 'vnc' are not live migratable!\n";
>      }
>  
> +    if ($running && $conf->{'amd_sev'}) {
> +	die "cannot live-migrate VM when AMD SEV is enabled.\n";
> +    }

Then you could also re-use the helper here :)

> +
>      my $vollist = PVE::QemuServer::get_vm_volumes($conf);
>  
>      my $storages = {};
> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
> index 82e7d6a..3417a86 100644
> --- a/PVE/QemuServer.pm
> +++ b/PVE/QemuServer.pm
> @@ -177,6 +177,37 @@ my $agent_fmt = {
>      },
>  };
>  
> +my $sev_fmt = {
> +    type => {
> +	description => "Enable standard SEV with type='std' or enable"
> +	    ." experimental SEV-ES with the 'es' option.",
> +	type => 'string',
> +	default_key => 1,
> +	format_description => "sev-type",

You don't need a format_description if there is an enum.

> +	enum => ['std', 'es'],
> +	maxLength => 3,

You don't need a maxLenght if there is an enum.

> +    },
> +    'no-debug' => {
> +	description => "Sets policy bit 0 to 1 to disallow debugging of guest",
> +	type => 'boolean',
> +	default => 0,
> +	optional => 1,
> +    },
> +    'no-key-sharing' => {
> +	description => "Sets policy bit 1 to 1 to disallow key sharing with other guests",
> +	type => 'boolean',
> +	default => 0,
> +	optional => 1,
> +    },
> +    "kernel-hashes" => {
> +	description => "Add kernel hashes to guest firmware for measured linux kernel launch",
> +	type => 'boolean',
> +	default => 0,
> +	optional => 1,
> +    },
> +};
> +PVE::JSONSchema::register_format('pve-qemu-sev-fmt', $sev_fmt);
> +
>  my $vga_fmt = {
>      type => {
>  	description => "Select the VGA type.",
> @@ -358,6 +389,12 @@ my $confdesc = {
>  	description => "Memory properties.",
>  	format => $PVE::QemuServer::Memory::memory_fmt
>      },
> +    amd_sev => {
> +	description => "Secure Encrypted Virtualization (SEV) features by AMD CPUs",
> +	optional => 1,
> +	format => 'pve-qemu-sev-fmt',
> +	type => 'string',
> +    },
>      balloon => {
>  	optional => 1,
>  	type => 'integer',
> @@ -4091,6 +4128,39 @@ sub config_to_command {
>  	}
>      }
>  
> +    if ($conf->{amd_sev}) {

Please factor this out into a helper function instead of adding a big
block to config_to_command. It's already huge enough.

> +	if ($conf->{bios} && $conf->{bios} ne 'ovmf') {
> +	    die "For using SEV you need to change your guest bios to ovmf.\n";

s/bios/BIOS/

> +	}
> +
> +	my $amd_sev_conf = parse_property_string($sev_fmt, $conf->{amd_sev});
> +	my $sev_hw_caps = get_hw_capabilities()->{'amd-sev'};

Maybe error out if the parsed caps are not a hash like we expect (before
accessing 'amd-sev')? And error out if the keys we expect do not exist
in the result.

> +
> +	if (!$sev_hw_caps->{'sev-support'}) {
> +	    die "Your CPU does not support AMD SEV!\n";
> +	}
> +	if ($amd_sev_conf->{type} eq 'es' && !$sev_hw_caps->{'sev-support-es'}) {
> +	    die "Your CPU does not support AMD SEV-ES!\n";
> +	}
> +
> +	my $sev_mem_object = 'sev-guest,id=sev0'
> +	    .',cbitpos='.$sev_hw_caps->{cbitpos}
> +	    .',reduced-phys-bits='.$sev_hw_caps->{'reduced-phys-bits'};
> +
> +	my $policy = 0b0;
> +	$policy += 0b1 if ($amd_sev_conf->{'no-debug'});
> +	$policy += 0b10 if ($amd_sev_conf->{'no-key-sharing'});
> +	$policy += 0b100 if ($amd_sev_conf->{type} eq 'es');
> +	# disable migration with bit 3 nosend to prevent amd-sev-migration-attack
> +	$policy += 0b1000;
> +
> +	$sev_mem_object .= ',policy='.sprintf("%#x", $policy);
> +	$sev_mem_object .= ',kernel-hashes=on' if ($amd_sev_conf->{'kernel-hashes'});

Style nit: superfluous parentheses for post-if

> +
> +	push @$devices, '-object' , $sev_mem_object;
> +	push @$machineFlags, 'confidential-guest-support=sev0';
> +    }
> +
>      push @$cmd, @$devices;
>      push @$cmd, '-rtc', join(',', @$rtcFlags) if scalar(@$rtcFlags);
>      push @$cmd, '-machine', join(',', @$machineFlags) if scalar(@$machineFlags);
> @@ -4134,6 +4204,15 @@ sub check_rng_source {
>      }
>  }
>  
> +sub get_hw_capabilities {
> +    # Get reduced-phys-bits & cbitpos from host-hw-capabilities.json
> +    my $filename = '/run/qemu-server/host-hw-capabilities.json';
> +    my $json_text = PVE::Tools::file_get_contents($filename);
> +    ($json_text) = $json_text =~ /(.*)/; # untaint json text> +    my $hw_capabilities = decode_json($json_text);

Maybe eval and use a nice error message/prefix if the decoding dies here?

> +    return $hw_capabilities;
> +}
> +
>  sub spice_port {
>      my ($vmid) = @_;
>