[PVE-User] PVE behind reverse proxy: different webroot possible?

[Uwe Sauter]

Hi Thomas,

I've been working on this and found half of a solution. Using Nginx' sub_filter rules I was able to get all the static stuff to be
displayed under a new webroot "/pve".

This is the relevant parts of my Nginx configuration:

/etc/ngnix/conf.d/default
####
upstream proxmox-cluster {
  ip_hash;
  server pve-host1:8006;
  server pve-host2:8006:
  server pve-host3:8006;
}

server {
  server_name                myserver.example.org;
  listen                     443 ssl;

  add_header X-Content-Type-Options nosniff;
  add_header X-Frame-Options "SAMEORIGIN";
  add_header X-XSS-Protection "1; mode=block";
  add_header X-Robots-Tag none;
  add_header X-Download-Options noopen;
  add_header X-Permitted-Cross-Domain-Policies none;

#[…] other locations

  location /pve/ {
    proxy_pass https://proxmox-cluster/;

    gzip off;

# filter to rewrite href anchors in content
    sub_filter               'href="/'       'href="/pve/';
# filter to rewrite src anchors in content
    sub_filter               'src="/'        'src="/pve/';
# filters to add prefix to urls (e.g. in pvemanagerlib.js)
    sub_filter               'url: "/api2'   'url: "/pve/api2';
    sub_filter               "url: '/api2"   "url: '/pve/api2";
    sub_filter               'url = "/api2'  'url = "/pve/api2';
    sub_filter               "url = '/api2"  "url = '/pve/api2";
# needed to load js files used by console window
    sub_filter               'url = "/api2'  'url = "/pve/api2';
    sub_filter               "url = '/api2"  "url = '/pve/api2";
#
    sub_filter_last_modified on;
# replace every occasion not only first
    sub_filter_once          off;
# use filter on all types, not only on text/html
    sub_filter_types         *;

    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

# necessary to tell pveproxy not to compress responses or else sub_filter will not work (as it does not decompress)
    proxy_set_header Accept-Encoding "";

    proxy_redirect off;

    # really needed?
    proxy_buffering off;
    client_max_body_size 0;
    proxy_connect_timeout 60s;
    proxy_read_timeout 60s;
    proxy_send_timeout 60s;
    send_timeout 60s;
  }
}
####


With this configuration I had partial success. What is not working currently:

* Notes on the summary tab of virtual machines. Error displayed is:
  "Error Method 'GET /pve/api2/extjs/nodes/hlrs-pxmx-02/qemu/101/config' not implemented (501)"

* Separate console window does not load. It seems that the URL needed to access a console is constructed by several function calls
and I don't know where I need to search for the string I need to replace. It tries to load
"https://myserver.example.org/api2/json/nodes/pve-host1/qemu/101/vncproxy" while it should load
"https://myserver.example.org/pve/api2/json/nodes/pve-host1/qemu/101/vncproxy".

* Inline console view: result is that I get displayed the root of my server: "https://myserver.example.org/index.html" (though
index.html is added by Nginx directive "index" for location "/").

* There might be more things I missed. Not even tested is containers (I don't have any).


Do you have any suggestions on those issues?

Regards,

	Uwe

PS: Please also note that the use of single and double quotes inside pvemangerib.js is inconsistet (compare lines 4296 and 33465).
This is the reason why I have 2 sub_filters for basically the same replacement.



Am 09.05.2017 um 11:01 schrieb Thomas Lamprecht:
> Hi,
> 
> On 05/05/2017 06:18 PM, Uwe Sauter wrote:
>> Hi,
>>
>> I've seen the wiki page [1] that explains how to operate a PVE host behind a reverse proxy.
>>
>> I'm currently in the situation that I have several services already behind a rev proxy that are accessible with different
>> webroots, e.g.
>>
>> https://example.com/dashboard
>> https://example.com/owncloud
>> https://example.com/nagios
>>
>> What changes would be needed so that I could reach the PVE host with
>>
>> https://example.com/pve ?
>>
>> Is it even possible?
> 
> Hmm, there are some problems as we mostly set absolute paths on resources (images, JS and CSS files)
> so the loading fails...
> I.e., pve does not knows that it is accessed from https://example.com/pve-node/ and tries to load the resources from the absolute
> path /pve/foo.js
> but then https://example.com/pve/foo.js results in a 404/501 error.
> Same happens for api calls, AFAIK.
> Normally some webapps allow to set a "ROOT_URL" config entry, where the access URL can be set.
> As there are many places where this would need to be changed it is not just a quick fix, though.
> 
> But you could work with sub-domains and achieve the same, e.g. a rever proxy entry for:
> https://pve-node.example.com
> should work.
> 
> Tested with a default setup and the following nginx configuration:
> 
> ----
> server {
>     listen 443;
>     server_name test.localhost; # <- FIXME, change
>     ssl on;
>     ssl_certificate /etc/pve/local/pve-ssl.pem;# OPTIONAL FIXME, change if you want other certs
>     ssl_certificate_key /etc/pve/local/pve-ssl.key;# or proxy and PVE are on separated machines
> 
>     location / {
>         proxy_pass https://localhost:8006/;
> 
>         proxy_set_header Host $host;
>         proxy_set_header X-Forwarded-Proto https;
>         proxy_set_header X-Real-IP $remote_addr;
>         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
>         proxy_redirect off;
> 
>         # really needed?
>         proxy_buffering off;
>         client_max_body_size 0;
>         proxy_connect_timeout 60s;
>         proxy_read_timeout 60s;
>         proxy_send_timeout 60s;
>         send_timeout 60s;
>     }
> }
> ----
> 
> With this I can access my cluster at https://test.localhost/ just fine.
> 
> Change server_name respectively and if you run the proxy on another server than the PVE host adapt also the proxy_pass entry and
> the ssl certs for that matter.
> I did only tested the situation where the nginx runned on the PVE host but it should work the same.
> 
> AFAICT, the "upstream" config entry described in the wiki is not really needed. Also the redirect from port 80 HTTP to 443 HTTPS
> is just convenience.
> 
> I'll update the wiki article a bit :)
>> Also is it possible to make a whole PVE cluster available behind a rev proxy using [2]?
> 
> How do you mean that? It should be possible to add multiple redirects for multiple nodes so it should work.
> 
> cheers,
> Thomas
>