[pve-devel] spice problems

[Fabian Grünbichler]

On Tue, Jan 10, 2017 at 01:50:35AM +0100, Michael Rasmussen wrote:
> On Mon, 9 Jan 2017 08:08:54 +0100
> Fabian Grünbichler <f.gruenbichler at proxmox.com> wrote:
> 
> > 
> > so you should now have
> > 
> > /etc/pve/pve-root-ca.pem (self-signed cluster CA cert generated by pvecem)
> > /etc/pve/priv/pve-root-ca.key (private key for ^^^)
> > 
> check
> > and for each node
> > 
> > /etc/pve/nodes/NODE/pve-ssl.pem (node cert signed by cluster CA)
> > /etc/pve/nodes/NODE/pve-ssl.key (private key for ^^^)
> > /etc/pve/nodes/NODE/pveproxy-ssl.pem (custom cert chain for pveproxy)
> > /etc/pve/nodes/NODE/pveproxy-ssl.pem (private key for node cert from ^^^)
> > 
> Check (I guess pveproxy-ssl.pem (private key for node cert from ^^^)
> should be pveproxy-ssl.key?)
> 
> > could you verify that the correct files are in the correct place with
> > the following command for each of the certificates?
> > 
> > # openssl x509 -in PATH -noout -subject -issuer
> > 
> openssl x509 -in /etc/pve/pve-root-ca.pem -noout -subject -issuer
> subject= /CN=Proxmox Virtual Environment/OU=d43de9da-071f-4ae5-b05b-a31593bb5668/O=PVE Cluster Manager CA
> issuer= /CN=Proxmox Virtual Environment/OU=d43de9da-071f-4ae5-b05b-a31593bb5668/O=PVE Cluster Manager CA
> 
> openssl x509 -in /etc/pve/nodes/esx1/pve-ssl.pem -noout -subject -issuer
> subject= /OU=PVE Cluster Node/O=Proxmox Virtual Environment/CN=esx1.datanom.net
> issuer= /CN=Proxmox Virtual Environment/OU=d43de9da-071f-4ae5-b05b-a31593bb5668/O=PVE Cluster Manager CA
> 
> openssl x509 -in /etc/pve/nodes/esx2/pve-ssl.pem -noout -subject -issuer
> subject= /OU=PVE Cluster Node/O=Proxmox Virtual Environment/CN=esx2.datanom.net
> issuer= /CN=Proxmox Virtual Environment/OU=d43de9da-071f-4ae5-b05b-a31593bb5668/O=PVE Cluster Manager CA
> 
> openssl x509 -in /etc/pve/nodes/esx1/pveproxy-ssl.pem -noout -subject -issuer
> subject= /OU=Domain Control Validated/CN=*.datanom.net
> issuer= /C=BE/O=GlobalSign nv-sa/CN=AlphaSSL CA - SHA256 - G2
> 
> openssl x509 -in /etc/pve/nodes/esx2/pveproxy-ssl.pem -noout -subject -issuer
> subject= /OU=Domain Control Validated/CN=*.datanom.net
> issuer= /C=BE/O=GlobalSign nv-sa/CN=AlphaSSL CA - SHA256 - G2
> 
> > if you restart the pveproxy service, the log should indicate that the
> > pveproxy-ssl.pem certificate chain file is used, and not display any
> > errors, could you verify this as well?
> > 
> systemctl restart pveproxy
> systemctl status -l pveproxy
> ● pveproxy.service - PVE API Proxy Server
>    Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled)
>    Active: active (running) since Tue 2017-01-10 01:31:51 CET; 13s ago
>   Process: 31999 ExecStop=/usr/bin/pveproxy stop (code=exited, status=0/SUCCESS)
>   Process: 32020 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
>  Main PID: 32025 (pveproxy)
>    CGroup: /system.slice/pveproxy.service
>            ├─32025 pveprox
>            ├─32026 pveproxy worke
>            ├─32027 pveproxy worke
>            └─32028 pveproxy worke
> 
> Jan 10 01:31:51 esx1 pveproxy[32020]: Using '/etc/pve/local/pveproxy-ssl.pem' as certificate for the web interface.
> Jan 10 01:31:51 esx1 pveproxy[32025]: starting server
> Jan 10 01:31:51 esx1 pveproxy[32025]: starting 3 worker(s)
> Jan 10 01:31:51 esx1 pveproxy[32025]: worker 32026 started
> Jan 10 01:31:51 esx1 pveproxy[32025]: worker 32027 started
> Jan 10 01:31:51 esx1 pveproxy[32025]: worker 32028 started
> Jan 10 01:31:51 esx1 systemd[1]: Started PVE API Proxy Server.
> 
> > the GUI and API should return the same certificate (they are handled by
> > the same web server after all, and the GUI is just some static files +
> > calls to the API), can you verify which one it is? e.g., using curl:
> > 
> > $ curl -k -v https://YOURNODE:8006/
> > 
> curl -k -v https://esx1.datanom.net:8006/
> *   Trying 172.16.3.8...
> * TCP_NODELAY set
> * Connected to esx1.datanom.net (172.16.3.8) port 8006 (#0)
> * found 176 certificates in /etc/ssl/certs/ca-certificates.crt
> * found 1117 certificates in /etc/ssl/certs
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * SSL connection using TLS1.2 / ECDHE_RSA_AES_256_GCM_SHA384
> * 	 server certificate verification SKIPPED
> * 	 server certificate status verification SKIPPED
> * 	 common name: *.datanom.net (matched)
> * 	 server certificate expiration date OK
> * 	 server certificate activation date OK
> * 	 certificate public key: RSA
> * 	 certificate version: #3
> * 	 subject: OU=Domain Control Validated,CN=*.datanom.net
> * 	 start date: Fri, 30 Dec 2016 20:47:18 GMT
> * 	 expire date: Tue, 31 Dec 2019 20:47:18 GMT
> * 	 issuer: C=BE,O=GlobalSign nv-sa,CN=AlphaSSL CA - SHA256 - G2
> * 	 compression: NULL
> * ALPN, server did not agree to a protocol
> 
> The same certificate shown in browser.
> 
> > last but not least, could you verify that the node cert in pve-ssl.pem
> > is actually signed by the cluster CA? example:
> > 
> > # openssl verify -CAfile /etc/pve/pve-root-ca.pem /etc/pve/nodes/nora/pve-ssl.pem
> > /etc/pve/nodes/nora/pve-ssl.pem: OK
> > 
> openssl verify -CAfile /etc/pve/pve-root-ca.pem /etc/pve/nodes/esx1/pve-ssl.pem
> /etc/pve/nodes/esx1/pve-ssl.pem: OK
> openssl verify -CAfile /etc/pve/pve-root-ca.pem /etc/pve/nodes/esx2/pve-ssl.pem
> /etc/pve/nodes/esx2/pve-ssl.pem: OK
> 
> > note that you don't need to include the actual host names or other
> > sensitive information when replying.
> It's ok since the are all running on a closed network without access to
> the outside world.
> 

thanks, I will try to reproduce and see what I can find! possibly the
recently introduced fingerprint pinning is the culprit here..