From korobkovserg at gmail.com Tue Feb 2 15:50:45 2021 From: korobkovserg at gmail.com (Sergey Korobkov) Date: Tue, 2 Feb 2021 17:50:45 +0300 Subject: [PVE-User] Live migration fails with "Mismatched RAM page size ram-node0 (local) 2097152 != 1526773257204281392" Message-ID: <9b662ee8-10c1-9a71-e598-674538535c51@gmail.com> Hello, Two exactly the same machines ( except ram size 380G and 1.5T ). Upgraded on Debian 10.7 from: pve-manager/6.1-5/9bf06119 Linux 5.3.13-1-pve #1 SMP PVE 5.3.13-1 (Thu, 05 Dec 2019 07:18:14 +0100) QEMU emulator version 4.1.1 (pve-qemu-kvm_4.1.1) to: pve-manager/6.3-3/eee5f901 Linux 5.4.78-2-pve #1 SMP PVE 5.4.78-2 (Thu, 03 Dec 2020 14:26:17 +0100) QEMU emulator version 5.1.0 (pve-qemu-kvm_5.1.0) We had enabled hugepages for virtual machines( "hugepages: 2" specified in virtual machine description). Live migration fails with errors like this: Feb 02 16:26:13 QEMU[12090]: kvm7: load of migration failed: Invalid argument Feb 02 16:26:13 QEMU[12090]: kvm7: error while loading state for instance 0x0 of device 'ram' Feb 02 16:26:13 QEMU[12090]: kvm7: Mismatched RAM page size ram-node0 (local) 2097152 != 1526773257204281392 We think it's some overflow issue. ------------------ Sincerely yours, Sergey. From s.reiter at proxmox.com Thu Feb 4 17:59:26 2021 From: s.reiter at proxmox.com (Stefan Reiter) Date: Thu, 4 Feb 2021 17:59:26 +0100 Subject: [PVE-User] Live migration fails with "Mismatched RAM page size ram-node0 (local) 2097152 != 1526773257204281392" In-Reply-To: <9b662ee8-10c1-9a71-e598-674538535c51@gmail.com> References: <9b662ee8-10c1-9a71-e598-674538535c51@gmail.com> Message-ID: <71fdbe8b-6212-e355-fdd3-ae2d69908307@proxmox.com> On 02/02/2021 15:50, Sergey Korobkov wrote: > Hello, > > Two exactly the same machines ( except ram size 380G and 1.5T ). > > Upgraded on Debian 10.7 from: > pve-manager/6.1-5/9bf06119 > Linux 5.3.13-1-pve #1 SMP PVE 5.3.13-1 (Thu, 05 Dec 2019 07:18:14 +0100) > QEMU emulator version 4.1.1 (pve-qemu-kvm_4.1.1) > > to: > pve-manager/6.3-3/eee5f901 > Linux 5.4.78-2-pve #1 SMP PVE 5.4.78-2 (Thu, 03 Dec 2020 14:26:17 +0100) > QEMU emulator version 5.1.0 (pve-qemu-kvm_5.1.0) > > We had enabled hugepages for virtual machines( "hugepages: 2" specified > in virtual machine description). > > Live migration fails with errors like this: > > Feb 02 16:26:13 QEMU[12090]: kvm7: load of migration failed: Invalid > argument > Feb 02 16:26:13 QEMU[12090]: kvm7: error while loading state for > instance 0x0 of device 'ram' > Feb 02 16:26:13 QEMU[12090]: kvm7: Mismatched RAM page size ram-node0 > (local) 2097152 != 1526773257204281392 > > We think it's some overflow issue. > Hi! After looking carefully I believe to have found the root cause of this issue in an upstream bug that we run into since pve-qemu-kvm 5.1.0-4, where we started migrating dirty bitmaps. I have sent a potential fix to the upstream qemu-devel mailing list: https://lists.nongnu.org/archive/html/qemu-devel/2021-02/msg01711.html If the resident experts on there agree that this is indeed the solution, we will most likely ship it once we release our QEMU 5.2 build :) Thanks for the report! From ralf.storm at konzept-is.de Fri Feb 5 10:43:34 2021 From: ralf.storm at konzept-is.de (Ralf Storm) Date: Fri, 5 Feb 2021 10:43:34 +0100 Subject: [PVE-User] Backup Failing sporadically due to "PBS backups are not supported by the running QEMU version" Message-ID: Hello, I have this Error first time ca. 2 weeks ago when backing up the biggest of our 70 vms : "PBS backups are not supported by the running QEMU version. Please make sure you've installed the latest version and the VM has been restarted" We have a Subscription and are on the newest updates. The only thing thats helps ist to shutdown the VM and reboot the whole Node on which the VM is runninng. Shutdown, waiting for the quemu to stop and then restart does not help, migrating to another Node does not help. Shutdown and migrate to another Node does not help. And this happens onyto this one VM, the other more than 70 VMs are not affected. The storage is ceph in Proxmox. -- Ralf Storm IT Administrator Konzept Informationssysteme GmbH ... From mszamocki at cirrus.pl Mon Feb 8 08:11:18 2021 From: mszamocki at cirrus.pl (=?utf-8?Q?Micha=C5=82?= Szamocki) Date: Mon, 8 Feb 2021 08:11:18 +0100 (CET) Subject: [PVE-User] missing /proc/sys/net mount in LXC Message-ID: <1336869185.3855.1612768278982.JavaMail.zimbra@cirrus.pl> Hi, since last update from pve-no-subscription repository # pveversion pve-manager/6.3-3/eee5f901 (running kernel: 5.4.78-2-pve) My containers don't have: # mount | grep proc/sys/net proc on /proc/sys/net type proc (rw,nosuid,nodev,noexec,relatime) (This mount point I found in container on Proxmox 5.4) How to create this mountpoint in Proxmox 6.3-3? BR, Micha? Szamocki Cirrus From f.gruenbichler at proxmox.com Mon Feb 8 09:21:26 2021 From: f.gruenbichler at proxmox.com (Fabian =?iso-8859-1?q?Gr=FCnbichler?=) Date: Mon, 08 Feb 2021 09:21:26 +0100 Subject: [PVE-User] Backup Failing sporadically due to "PBS backups are not supported by the running QEMU version" In-Reply-To: References: Message-ID: <1612772213.gdeednkn6n.astroid@nora.none> On February 5, 2021 10:43 am, Ralf Storm wrote: > Hello, > > > I have this Error first time ca. 2 weeks ago when backing up the biggest > of our 70 vms : "PBS backups are not supported by the running QEMU > version. Please make sure you've installed the latest version and the VM > has been restarted" > > We have a Subscription and are on the newest updates. that does sound rather strange. could you post the following information (feel free to censort parts you are not comfortable with sharing publicly) pveversion -v qm config XXX qm status XXX --verbose (where XXX is the VMID of the problematic VM) journal output from around the attempted backup and the full backup task log would also be interesting. > The only thing thats helps ist to shutdown the VM and reboot the whole > Node on which the VM is runninng. Shutdown, waiting for the quemu to > stop and then restart does not help, migrating to another Node does not > help. Shutdown and migrate to another Node does not help. that sounds strange as well - a reboot of the node and a fresh start of the VM (via migration or shutdown and start) are equivalent w.r.t. what state of the old running VM is carried over.. > And this happens onyto this one VM, the other more than 70 VMs are not > affected. what does biggest mean in the context of this node? biggest disk? highest memory? ... ? thanks! From f.gruenbichler at proxmox.com Mon Feb 8 09:21:26 2021 From: f.gruenbichler at proxmox.com (Fabian =?iso-8859-1?q?Gr=FCnbichler?=) Date: Mon, 08 Feb 2021 09:21:26 +0100 Subject: [PVE-User] Backup Failing sporadically due to "PBS backups are not supported by the running QEMU version" In-Reply-To: References: Message-ID: <1612772213.gdeednkn6n.astroid@nora.none> On February 5, 2021 10:43 am, Ralf Storm wrote: > Hello, > > > I have this Error first time ca. 2 weeks ago when backing up the biggest > of our 70 vms : "PBS backups are not supported by the running QEMU > version. Please make sure you've installed the latest version and the VM > has been restarted" > > We have a Subscription and are on the newest updates. that does sound rather strange. could you post the following information (feel free to censort parts you are not comfortable with sharing publicly) pveversion -v qm config XXX qm status XXX --verbose (where XXX is the VMID of the problematic VM) journal output from around the attempted backup and the full backup task log would also be interesting. > The only thing thats helps ist to shutdown the VM and reboot the whole > Node on which the VM is runninng. Shutdown, waiting for the quemu to > stop and then restart does not help, migrating to another Node does not > help. Shutdown and migrate to another Node does not help. that sounds strange as well - a reboot of the node and a fresh start of the VM (via migration or shutdown and start) are equivalent w.r.t. what state of the old running VM is carried over.. > And this happens onyto this one VM, the other more than 70 VMs are not > affected. what does biggest mean in the context of this node? biggest disk? highest memory? ... ? thanks! From m.sauer at khm.de Mon Feb 8 12:59:29 2021 From: m.sauer at khm.de (Marc Sauer) Date: Mon, 8 Feb 2021 12:59:29 +0100 Subject: [PVE-User] Migrating from oVirt to Proxmox Message-ID: Hello, we would like to migrate our oVirt Cluster to Proxmox, because of the recent events regarding RedHat and CentOS. Also, we are getting new servers and have the one time oppurtinity to do a switch to another virtualization platform. Right now, We are using GlusterFS underneath the oVirt Cluster and would like to use it on Proxmox as well. It will be a 3 node cluster. My question to you all is: Has anyone on this list done a migration process from oVirt to Proxmox? Both systems are based on KVM, so a migration should not be that complicated. My main idea is the following: - Set up a Proxmox cluster with 3 nodes - Set up the Gluster storage on the new servers - Create VMs with the same name and copy the raw disk image from the old Gluster to the new one - Set the copied Raw disk image as bootfile for the created VM - Repeat the step for all VMs, one at a time Since we do not have thousands of VMs we can do this manually, so no need to set up a complicated migration pipeline. The main advantage of this is, that we can migrate a VM one at a time and not all at once. What are your thoughts on this? Will we get trouble with, for example, the drivers inside the VM? Thank you in advance and many greetings from Cologne, Marc Sauer -- Marc Sauer Linux Systems Administrator Kunsthochschule f?r Medien K?ln/ Academy of Media Arts Cologne Peter-Welter-Platz 2 50676 K?ln From gbr at majentis.com Mon Feb 8 13:17:15 2021 From: gbr at majentis.com (Gerald Brandt) Date: Mon, 8 Feb 2021 06:17:15 -0600 Subject: [PVE-User] Migrating from oVirt to Proxmox In-Reply-To: References: Message-ID: <29199880-8246-8125-60ef-1e71acd55b17@majentis.com> On 2021-02-08 5:59 a.m., Marc Sauer wrote: > Hello, > > we would like to migrate our oVirt Cluster to Proxmox, because of the > recent events regarding RedHat and CentOS. Also, we are getting new > servers and have the one time oppurtinity to do a switch to another > virtualization platform. > > Right now, We are using GlusterFS underneath the oVirt Cluster and > would like to use it on Proxmox as well. > > It will be a 3 node cluster. > > My question to you all is: Has anyone on this list done a migration > process from oVirt to Proxmox? Both systems are based on KVM, so a > migration should not be that complicated. > > My main idea is the following: > > - Set up a Proxmox cluster with 3 nodes > - Set up the Gluster storage on the new servers > - Create VMs with the same name and copy the raw disk image from the > old Gluster to the new one > - Set the copied Raw disk image as bootfile for the created VM > - Repeat the step for all VMs, one at a time > > Since we do not have thousands of VMs we can do this manually, so no > need to set up a complicated migration pipeline. The main advantage of > this is, that we can migrate a VM one at a time and not all at once. > > What are your thoughts on this? Will we get trouble with, for example, > the drivers inside the VM? > > Thank you in advance and many greetings from Cologne, > > Marc Sauer > > That is the exact process I used when moving from Proxmox to straight KVM and virt-manager. It's a safe way to move. If a single VM has issues, you can fall back to the original and figure out what went wrong without long downtimes. I moved thirty VM's using this process. I did have issues with Windows machines, and had to manually copy the machine's GUID to the new virtual machine in order to keep Windows activated. Gerald > _______________________________________________ > pve-user mailing list > pve-user at lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user From ronny+pve-user at aasen.cx Mon Feb 8 13:22:34 2021 From: ronny+pve-user at aasen.cx (Ronny Aasen) Date: Mon, 8 Feb 2021 13:22:34 +0100 Subject: [PVE-User] Migrating from oVirt to Proxmox In-Reply-To: References: Message-ID: On 08.02.2021 12:59, Marc Sauer wrote: > Hello, > > we would like to migrate our oVirt Cluster to Proxmox, because of the > recent events regarding RedHat and CentOS. Also, we are getting new > servers and have the one time oppurtinity to do a switch to another > virtualization platform. > > Right now, We are using GlusterFS underneath the oVirt Cluster and would > like to use it on Proxmox as well. > > It will be a 3 node cluster. > > My question to you all is: Has anyone on this list done a migration > process from oVirt to Proxmox? Both systems are based on KVM, so a > migration should not be that complicated. > > My main idea is the following: > > - Set up a Proxmox cluster with 3 nodes > - Set up the Gluster storage on the new servers > - Create VMs with the same name and copy the raw disk image from the old > Gluster to the new one > - Set the copied Raw disk image as bootfile for the created VM > - Repeat the step for all VMs, one at a time > > Since we do not have thousands of VMs we can do this manually, so no > need to set up a complicated migration pipeline. The main advantage of > this is, that we can migrate a VM one at a time and not all at once. > > What are your thoughts on this? Will we get trouble with, for example, > the drivers inside the VM? > > Thank you in advance and many greetings from Cologne, > > Marc Sauer > Not exactly i have dome something similar with vmware to proxmox. And the plan was the same + i had a convert image step that you do not need. drivers can be an issue if the OS on the VM is windows. read the xml config on ovirt and replicate accordingly on proxmox to avoid any issues. this is probably only an issue on the first VM, i assume the rest would be identical, once you get the right config. I also did one rsync of image while vm was running + another rsync while vm was stopped to have shorter downtime. depends on your network speed vs storage speed if this is sane or not. good luck Ronny From d.jaeger at proxmox.com Mon Feb 8 13:54:04 2021 From: d.jaeger at proxmox.com (Dominic =?iso-8859-1?Q?J=E4ger?=) Date: Mon, 8 Feb 2021 13:54:04 +0100 Subject: [PVE-User] Migrating from oVirt to Proxmox In-Reply-To: References: Message-ID: <20210208125404.GA130539@mala> Hello! On Mon, Feb 08, 2021 at 12:59:29PM +0100, Marc Sauer wrote: > - Set up the Gluster storage on the new servers Have you thought about using Ceph? Gluster should work, but Ceph is better integrated into Proxmox VE. > - Create VMs with the same name and copy the raw disk image from the old > Gluster to the new one > - Set the copied Raw disk image as bootfile for the created VM There are 2 commands qm importdisk and qm importovf (see man page [0]) that should help you avoid 1. using qemu-img manually and 2. having to know about Proxmox VE naming schemes If you can export your VMs as OVF, you can use qm importovf which does most work alone. If there is no such function, you have to create a VM in Proxmox VE and import each disk image individually using qm importdisk. I suggest to get an overview of the migration article in our Wiki [1]. It doesn't explicitly mention oVirt (yet), but the usage of the import commands remains the same over all examples. [0] https://pve.proxmox.com/pve-docs/qm.1.html [1] https://pve.proxmox.com/wiki/Migration_of_servers_to_Proxmox_VE#Virtual-to-Virtual From ronny+pve-user at aasen.cx Mon Feb 8 14:42:13 2021 From: ronny+pve-user at aasen.cx (Ronny Aasen) Date: Mon, 8 Feb 2021 14:42:13 +0100 Subject: [PVE-User] Migrating from oVirt to Proxmox In-Reply-To: <20210208125404.GA130539@mala> References: <20210208125404.GA130539@mala> Message-ID: On 08.02.2021 13:54, Dominic J?ger wrote: > Hello! > > On Mon, Feb 08, 2021 at 12:59:29PM +0100, Marc Sauer wrote: >> - Set up the Gluster storage on the new servers > Have you thought about using Ceph? Gluster should work, but Ceph is better > integrated into Proxmox VE. > If you can add your 3 current ovirt nodes to the cluster afterwards you would have a nice 6 node ceph cluster. 3 nodes are too little imho, there is no failure domain. but would function for the migration. Ronny From t.lamprecht at proxmox.com Tue Feb 9 12:16:02 2021 From: t.lamprecht at proxmox.com (Thomas Lamprecht) Date: Tue, 9 Feb 2021 12:16:02 +0100 Subject: [PVE-User] missing /proc/sys/net mount in LXC In-Reply-To: <1336869185.3855.1612768278982.JavaMail.zimbra@cirrus.pl> References: <1336869185.3855.1612768278982.JavaMail.zimbra@cirrus.pl> Message-ID: Hi, On 08.02.21 08:11, Micha? Szamocki wrote: > since last update from pve-no-subscription repository > # pveversion > pve-manager/6.3-3/eee5f901 (running kernel: 5.4.78-2-pve) > > My containers don't have: > # mount | grep proc/sys/net > proc on /proc/sys/net type proc (rw,nosuid,nodev,noexec,relatime) > (This mount point I found in container on Proxmox 5.4) > > How to create this mountpoint in Proxmox 6.3-3? > can you re-check with lxc-pve version 4.0.6-2 available on the pvetest repository? You could also install it manually: wget http://download.proxmox.com/debian/pve/dists/buster/pvetest/binary-amd64/lxc-pve_4.0.6-2_amd64.deb # verify checksum sha256sum lxc-pve_4.0.6-2_amd64.deb 894f07f6839bd322d74f778062732d36a36643667c33c4466796f883f7dbb291 lxc-pve_4.0.6-2_amd64.deb apt install ./lxc-pve_4.0.6-2_amd64.deb cheers, Thomas From leandro at tecnetmza.com.ar Tue Feb 9 14:46:49 2021 From: leandro at tecnetmza.com.ar (Leandro Roggerone) Date: Tue, 9 Feb 2021 10:46:49 -0300 Subject: [PVE-User] Server performance question Message-ID: Hi guys , I noticed some vms applications are running very slow. For instance , I'm running librenms on a dedicated vm , which takes too long to load its views. After checking my box I have following questions: Executing top command on pve cli , I can see: ################################### When looking to cpu load with top command , I see: top - 10:16:53 up 53 days, 13 min, 4 users, load average: 14.74, 8.60, 8.44 Tasks: 659 total, 1 running, 658 sleeping, 0 stopped, 0 zombie %Cpu0 : 1.5/12.2 14[|||||||||||||| ] %Cpu1 : 3.7/5.4 9[||||||||| ] %Cpu2 : 1.3/1.7 3[||| ] %Cpu3 : 3.0/3.0 6[|||||| ] %Cpu4 : 2.3/1.0 3[||| ] %Cpu5 : 0.7/2.6 3[|||| ] %Cpu6 : 1.0/2.7 4[|||| ] %Cpu7 : 2.3/1.3 4[||| ] %Cpu8 : 1.7/1.0 3[||| ] %Cpu9 : 1.3/7.4 9[|||||||| ] %Cpu10 : 1.3/2.0 3[||| ] %Cpu11 : 1.3/9.8 11[||||||||||| ] %Cpu12 : 1.4/1.4 3[|| ] %Cpu13 : 5.7/7.4 13[||||||||||||| ] %Cpu14 : 2.0/1.3 3[||| ] %Cpu15 : 3.3/4.7 8[|||||||| ] %Cpu16 : 1.7/1.0 3[||| ] %Cpu17 : 4.8/9.2 14[|||||||||||||| ] %Cpu18 : 1.3/0.7 2[|| ] %Cpu19 : 1.0/3.7 5[||||| ] %Cpu20 : 2.7/1.7 4[||||| ] %Cpu21 : 2.7/0.7 3[|||| ] %Cpu22 : 5.0/1.7 7[||||||| ] %Cpu23 : 2.4/2.7 5[||||| ] MiB Mem : 64404.3 total, 3516.4 free, 56097.8 used, 4790.0 buff/cache MiB Swap: 8192.0 total, 734.0 free, 7458.0 used. 5235.3 avail Mem ################################### When looking to memory usage with top command at cli: MiB Mem : 92.1/64404.3 [|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ] MiB Swap: 91.0/8192.0 [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ] ###################################When I see "summary" at pve tab it shows: CPU usage 17.08% of 24 CPU(s) Load average 9.37,10.23,9.12 RAM usage 87.79% (55.22 GiB of 62.89 GiB SWAP usage 91.04% (7.28 GiB of 8.00 GiB) ________________________________________________________________________________ Have following quesions: 1 Is my system at full capacity ?? cpu usage looks like it is around 20% and less so ... it is not at full capacity. but cpu load shows load average: 14.74, 8.60, 8.44 (top) Load average 9.37,10.23,9.12 (gui) (Both values are dynamically moving , so the are not same but refers to same parameter) It shows that the system is at very high utilization. 2 Why is swap memory almost full ? Will the Systeml use swap memory or physic memory first ? Should I increase my swap memory ? I 3 Regarding cpu resources when creating VM. I use to set 1 socket and 4 cores for applications like librenms monitoring 15 devices. but Proxmox allows me to set maximum sockets : 4 cores: 28 So , I'm wondering what happens if I use the maximum available cpu resources for all my VMs. Will it force the system to have a better distribution of cpus load ? if not ... What would be the difference to set max cpu resources to all my VMs. That's all ... sorry for the loooong email. Any feeling about this would be welcome. Leandro Libre de virus. www.avast.com <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> From jameslipski at protonmail.com Thu Feb 11 01:15:18 2021 From: jameslipski at protonmail.com (jameslipski) Date: Thu, 11 Feb 2021 00:15:18 +0000 Subject: Live Migration Question. Message-ID: Greetings, I have a couple of questions relating to live migrations, and was unable to find any answers to this. Just to preface, I have a relatively small cluster consisting of 6 nodes, somewhat homogeneous -- some nodes run Xeon E5-2660 v3 and others run Xeon E5-2690 v4 (both are Sandybridge based, clocked the same speeds, only difference is core count). Occasionally I migrate machines between nodes, and it usually does work, even between systems that are based on the CPUs mentioned above. However sometimes I run into an issue where the VM would migrate to the destination node, and dashboard reports "Internal-Error"; and from the console it looks like it's suspended. The only way I can kick it out of this state is to kill the process and start it again (or let HA start it again). So the question that I have are: 1 - Under what condition during a migration would place the VM in this "Internal-Error", suspended state and not resume? 2 - If the VM is in a "Internal-Error", suspended state, can it be resumed manually, instead of restarting/powering off. I don't really see anything in syslog events (unless they're stored somewhere else), source node shows that it is migrating, and destination node only shows that it's attempting to start it. If you need any additional information, let me know what you need and I'll post them. Thank you in advanced. From jameslipski at protonmail.com Thu Feb 11 01:12:01 2021 From: jameslipski at protonmail.com (jameslipski) Date: Thu, 11 Feb 2021 00:12:01 +0000 Subject: Live Migration Question Message-ID: Greetings, I have a couple of questions relating to live migrations, and was unable to find any answers to this. Just to preface, I have a relatively small cluster consisting of 6 nodes, somewhat homogeneous -- some nodes run Xeon E5-2660 v3 and others run Xeon E5-2690 v4 (both are Sandybridge based, clocked the same speeds, only difference is core count). Occasionally I migrate machines between nodes, and it usually does work, even between systems that are based on the CPUs mentioned above. However sometimes I run into an issue where the VM would migrate to the destination node, and dashboard reports "Internal-Error"; and from the console it looks like it's suspended. The only way I can kick it out of this state is to kill the process and start it again (or let HA start it again). So the question that I have are: 1 - Under what condition during a migration would place the VM in this "Internal-Error", suspended state and not resume? 2 - If the VM is in a "Internal-Error", suspended state, can it be resumed manually, instead of restarting/powering off. I don't really see anything in syslog events (unless they're stored somewhere else), source node shows that it is migrating, and destination node only shows that it's attempting to start it. If you need any additional information, let me know what you need and I'll post them. Thank you in advanced. From mhill at inett.de Thu Feb 11 11:19:24 2021 From: mhill at inett.de (Maximilian Hill) Date: Thu, 11 Feb 2021 11:19:24 +0100 Subject: [PVE-User] Shell without realm 'pam' Message-ID: Hello, is it, somehow, possible, to allow users in a specific group to use the shell without signing in in realm 'pam'? Specifically, we need to use it with an AD realm. Regards, Maximilian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From t.lamprecht at proxmox.com Thu Feb 11 12:58:40 2021 From: t.lamprecht at proxmox.com (Thomas Lamprecht) Date: Thu, 11 Feb 2021 12:58:40 +0100 Subject: [PVE-User] Shell without realm 'pam' In-Reply-To: References: Message-ID: Hi, On 11.02.21 11:19, Maximilian Hill wrote: > Hello, > > is it, somehow, possible, to allow users in a specific group to use the > shell without signing in in realm 'pam'? Currently not. > Specifically, we need to use it with an AD realm. To be clear, with what credentials would the login in the PVE shell? Is the LDAP also exposed in the system and valid for login or do they have credentials of an existing PAM user? cheers, Thomas From mhill at inett.de Thu Feb 11 13:25:48 2021 From: mhill at inett.de (Maximilian Hill) Date: Thu, 11 Feb 2021 13:25:48 +0100 Subject: [PVE-User] Shell without realm 'pam' In-Reply-To: References: Message-ID: Hi, On Thu, Feb 11, 2021 at 12:58:40PM +0100, Thomas Lamprecht wrote: > > Specifically, we need to use it with an AD realm. > > To be clear, with what credentials would the login in the PVE shell? > > Is the LDAP also exposed in the system and valid for login or do they > have credentials of an existing PAM user? Currently we want users of a specific group to be able to use the shell as root, whithout a dedicated PAM user. Another option we would have, would be to integrate the PVE node itself into the domain and make use of sudo. But that would come with some company-related difficulties and guidelines. For that reason we use AD for the web UI only. max -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From gaio at sv.lnf.it Thu Feb 11 13:32:09 2021 From: gaio at sv.lnf.it (Marco Gaiarin) Date: Thu, 11 Feb 2021 13:32:09 +0100 Subject: [PVE-User] Shell without realm 'pam' In-Reply-To: References: Message-ID: <20210211123209.GR3003@sv.lnf.it> Mandi! Thomas Lamprecht In chel di` si favelave... > Is the LDAP also exposed in the system and valid for login or do they > have credentials of an existing PAM user? Effectively this lead me to a side-question. If LDAP/AD users are also exposed to system (PAM/NSS) via lib(pam|nss)-ldap(d) and/or winbind, what is the difference? Could be that a simple flag can be raised 'also available to PAM stack'? -- dott. Marco Gaiarin GNUPG Key ID: 240A3D66 Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ Polo FVG - Via della Bont?, 7 - 33078 - San Vito al Tagliamento (PN) marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA) From sfrazier1111 at yahoo.com Sun Feb 14 18:03:32 2021 From: sfrazier1111 at yahoo.com (Steve Frazier) Date: Sun, 14 Feb 2021 17:03:32 +0000 (UTC) Subject: Adding an Internet NIC to existing Proxmox References: <206447580.1166532.1613322212874.ref@mail.yahoo.com> Message-ID: <206447580.1166532.1613322212874@mail.yahoo.com> I need some help.? I am new to Proxmox.? ?I have it set up now using (1) one NIC to an internal network of 192.168.20.0/23 and I would like to use another NIC I have available for the internet which has (5) available IPs. I have attached a view of my existing network. I would like to set up an additional (NIC) when would allow me to assign to a virtual server that has two virtual NICs with one being the internet and the other being my internal network. While I am at it, I have another networking 192.168.30.0/24 that I would like to assign to an available NIC so that I could assign virtual servers to that NICat a later time. Could someone please help me on how to do this Thanks very much. From sfrazier1111 at yahoo.com Sun Feb 14 18:29:14 2021 From: sfrazier1111 at yahoo.com (Steve Frazier) Date: Sun, 14 Feb 2021 17:29:14 +0000 (UTC) Subject: [PVE-User] Adding an Internet NIC to existing Proxmox In-Reply-To: References: <206447580.1166532.1613322212874.ref@mail.yahoo.com> Message-ID: <1128287218.1300602.1613323754497@mail.yahoo.com> Here is my "interfaces" file: auto loiface lo inet loopback iface enp9s0f1 inet manual auto vmbr0iface vmbr0 inet static? ? ? ? address 192.168.20.2/23? ? ? ? gateway 192.168.20.1? ? ? ? bridge_ports enp9s0f1? ? ? ? bridge_stp off? ? ? ? bridge_fd 0 iface enp11s0f0 inet manual iface enp11s0f1 inet manual iface enp9s0f0 inet manual iface enp3s0 inet manual iface enp2s0 inet manual On Sunday, February 14, 2021, 12:14:28 PM EST, Steve Frazier via pve-user wrote: I need some help.? I am new to Proxmox.? ?I have it set up now using (1) one NIC to an internal network of 192.168.20.0/23 and I would like to use another NIC I have available for the internet which has (5) available IPs. I have attached a view of my existing network. I would like to set up an additional (NIC) when would allow me to assign to a virtual server that has two virtual NICs with one being the internet and the other being my internal network. While I am at it, I have another networking 192.168.30.0/24 that I would like to assign to an available NIC so that I could assign virtual servers to that NICat a later time. Could someone please help me on how to do this Thanks very much. _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user From alain.pean at c2n.upsaclay.fr Sun Feb 14 21:31:55 2021 From: alain.pean at c2n.upsaclay.fr (=?UTF-8?Q?Alain_P=c3=a9an?=) Date: Sun, 14 Feb 2021 21:31:55 +0100 Subject: [PVE-User] Adding an Internet NIC to existing Proxmox In-Reply-To: References: <206447580.1166532.1613322212874.ref@mail.yahoo.com> Message-ID: Le 14/02/2021 ? 18:03, Steve Frazier via pve-user a ?crit?: > I need some help.? I am new to Proxmox.? ?I have it set up now using (1) one NIC to an internal network of 192.168.20.0/23 and I would like to use another NIC I have available for the internet which has (5) available IPs. > I have attached a view of my existing network. > I would like to set up an additional (NIC) when would allow me to assign to a virtual server that has two virtual NICs with one being the internet and the other being my internal network. > While I am at it, I have another networking 192.168.30.0/24 that I would like to assign to an available NIC so that I could assign virtual servers to that NICat a later time. > Could someone please help me on how to do this Hi Steve, It is easy to add a network interface to a VM. Just go in the Proxmox web interface on your VM, then Harware menu, then 'Add' button. You can there add a new NIC. It will linked by default to the bridge vmbr0, if you have only this one. This brikge is itself linked to the first NIC on your host, say eth0. If your IP ranges represent different VLANs, then you have to allow different VLAN on the same interface (eth0), and it should be a trunk allowing all these VLANs, Another possibility would be to first add a new bridge on the host (it also can be done via proxmox web interfance), which would be linked for example to the second NIC on your host, say eth1. The two NICs could be in different VLAN... See for exemple : https://pve.proxmox.com/wiki/Network_Configuration Alain -- Administrateur Syst?me/R?seau C2N Centre de Nanosciences et Nanotechnologies (UMR 9001) Boulevard Thomas Gobert (ex Avenue de La Vauve), 91120 Palaiseau Tel : 01-70-27-06-88 Bureau A255 From jameslipski at protonmail.com Tue Feb 16 14:53:17 2021 From: jameslipski at protonmail.com (jameslipski) Date: Tue, 16 Feb 2021 13:53:17 +0000 Subject: [PVE-User] Live Migration Question. In-Reply-To: References: Message-ID: <3CmgOjZXnu682Rky2tgt8mMOFscidFOi_df3cQg9b5lpMYRyq9bItZraggnGTEdY0_Y-FaM_ZU9eghc0u2W5vr_9ctbkujAQHjK0NgXVkB8=@protonmail.com> Hi, As an update, as I'm trying to understand why this is happening; at the moment, I don't see any sort of pattern. It happens at random under different guest OSs (mixture of Debian 8 and 9). As I've mentioned I'm not really seeing anything in syslog unless migrations events (and faults) are recorded elsewhere (with verbosity). Any help would be appreciated. Thank you. ??????? Original Message ??????? On Wednesday, February 10, 2021 7:15 PM, jameslipski via pve-user wrote: > Greetings, > > I have a couple of questions relating to live migrations, and was unable to find any answers to this. > > Just to preface, I have a relatively small cluster consisting of 6 nodes, somewhat homogeneous -- some nodes run Xeon E5-2660 v3 and others run Xeon E5-2690 v4 (both are Sandybridge based, clocked the same speeds, only difference is core count). > > Occasionally I migrate machines between nodes, and it usually does work, even between systems that are based on the CPUs mentioned above. However sometimes I run into an issue where the VM would migrate to the destination node, and dashboard reports "Internal-Error"; and from the console it looks like it's suspended. The only way I can kick it out of this state is to kill the process and start it again (or let HA start it again). > > So the question that I have are: > > 1 - Under what condition during a migration would place the VM in this "Internal-Error", suspended state and not resume? > 2 - If the VM is in a "Internal-Error", suspended state, can it be resumed manually, instead of restarting/powering off. > > I don't really see anything in syslog events (unless they're stored somewhere else), source node shows that it is migrating, and destination node only shows that it's attempting to start it. > > If you need any additional information, let me know what you need and I'll post them. > > Thank you in advanced._______________________________________________ > pve-user mailing list > pve-user at lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user From smr at kmi.com Tue Feb 16 19:52:51 2021 From: smr at kmi.com (Stefan M. Radman) Date: Tue, 16 Feb 2021 18:52:51 +0000 Subject: [PVE-User] Adding an Internet NIC to existing Proxmox In-Reply-To: References: <206447580.1166532.1613322212874.ref@mail.yahoo.com> Message-ID: <21687B15-6EE4-4F27-A332-6C2FE4FB86C9@kmi.com> Hi Steve Attached and below is an /etc/network/interfaces that should do what you want. Just * keep interface enp9s0f1 connected to your private subnet 192.168.20.2/23 * connect interface enp9s0f0 to your internet subnet * connect internet VMs to vmbr1 * connect interface enp11s0f1 to your private subnet 192.168.30.2/23 * connect VMs on subnet 192.168.30.2/23 to vmbr2 If you keep the comment lines (#) they'll show up in the web interface as well. Disclaimer: I did not test it but it should give you the right idea to go ahead. Stefan /etc/network/interfaces: auto lo iface lo inet loopback iface enp9s0f1 inet manual # Uplink for vmbr0 auto vmbr0 iface vmbr0 inet static address 192.168.20.2/23 gateway 192.168.20.1 bridge_ports enp9s0f1 bridge_stp off bridge_fd 0 # Host Management iface enp11s0f0 inet manual # Uplink for vmbr1 auto vmbr1 iface vmbr1 inet manual bridge_ports enp9s0f0 bridge_stp off bridge_fd 0 # Bridge for VMs on the Internet iface enp11s0f1 inet manual # Uplink for vmbr2 auto vmbr2 iface vmbr2 inet manual bridge_ports enp11s0f1 bridge_stp off bridge_fd 0 # Bridge for VMs on network 192.168.30.0/24 iface enp9s0f0 inet manual iface enp3s0 inet manual iface enp2s0 inet manual On Feb 14, 2021, at 18:29, Steve Frazier via pve-user > wrote: From: Steve Frazier > Subject: Re: [PVE-User] Adding an Internet NIC to existing Proxmox Date: February 14, 2021 at 18:29:14 GMT+1 To: "pve-user at lists.proxmox.com" > Here is my "interfaces" file: auto loiface lo inet loopback iface enp9s0f1 inet manual auto vmbr0iface vmbr0 inet static address 192.168.20.2/23 gateway 192.168.20.1 bridge_ports enp9s0f1 bridge_stp off bridge_fd 0 iface enp11s0f0 inet manual iface enp11s0f1 inet manual iface enp9s0f0 inet manual iface enp3s0 inet manual iface enp2s0 inet manual On Sunday, February 14, 2021, 12:14:28 PM EST, Steve Frazier via pve-user > wrote: I need some help. I am new to Proxmox. I have it set up now using (1) one NIC to an internal network of 192.168.20.0/23 and I would like to use another NIC I have available for the internet which has (5) available IPs. I have attached a view of my existing network. I would like to set up an additional (NIC) when would allow me to assign to a virtual server that has two virtual NICs with one being the internet and the other being my internal network. While I am at it, I have another networking 192.168.30.0/24 that I would like to assign to an available NIC so that I could assign virtual servers to that NICat a later time. Could someone please help me on how to do this Thanks very much. _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.proxmox.com%2Fcgi-bin%2Fmailman%2Flistinfo%2Fpve-user&data=04%7C01%7Csmr%40kmi.com%7Cfb3db335b1844cd9162208d8d10e1613%7Cc2283768b8d34e008f3d85b1b4f03b33%7C0%7C0%7C637489205732810378%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=bbtAAx1zpomz3hCKtKmrW3UIFF6iyprwmGcer1Q0njQ%3D&reserved=0 CONFIDENTIALITY NOTICE: This communication may contain privileged and confidential information, or may otherwise be protected from disclosure, and is intended solely for use of the intended recipient(s). If you are not the intended recipient of this communication, please notify the sender that you have received this communication in error and delete and destroy all copies in your possession. -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: interfaces.txt URL: From smr at kmi.com Tue Feb 16 19:54:53 2021 From: smr at kmi.com (Stefan M. Radman) Date: Tue, 16 Feb 2021 18:54:53 +0000 Subject: [PVE-User] Adding an Internet NIC to existing Proxmox In-Reply-To: References: <206447580.1166532.1613322212874.ref@mail.yahoo.com> Message-ID: <249BD6E3-5339-4587-A968-8453139883D1@kmi.com> Hi Steve Attached and below is an /etc/network/interfaces that should do what you want. Just * keep interface enp9s0f1 connected to your private subnet 192.168.20.2/23 * connect interface enp9s0f0 to your internet subnet * connect internet VMs to vmbr1 * connect interface enp11s0f1 to your private subnet 192.168.30.2/23 * connect VMs on subnet 192.168.30.2/23 to vmbr2 If you keep the comment lines (#) they'll show up in the web interface as well. Disclaimer: I did not test it but it should give you the right idea to go ahead. Stefan PS: please disregard previous attachment - this is the correct one /etc/network/interfaces: auto lo iface lo inet loopback iface enp9s0f1 inet manual # Uplink for vmbr0 auto vmbr0 iface vmbr0 inet static address 192.168.20.2/23 gateway 192.168.20.1 bridge_ports enp9s0f1 bridge_stp off bridge_fd 0 # Host Management iface enp11s0f0 inet manual # Uplink for vmbr1 auto vmbr1 iface vmbr1 inet manual bridge_ports enp9s0f0 bridge_stp off bridge_fd 0 # Bridge for VMs on the Internet iface enp11s0f1 inet manual # Uplink for vmbr2 auto vmbr2 iface vmbr2 inet manual bridge_ports enp11s0f1 bridge_stp off bridge_fd 0 # Bridge for VMs on network 192.168.30.0/24 iface enp9s0f0 inet manual iface enp3s0 inet manual iface enp2s0 inet manual On Feb 14, 2021, at 18:29, Steve Frazier via pve-user > wrote: From: Steve Frazier > Subject: Re: [PVE-User] Adding an Internet NIC to existing Proxmox Date: February 14, 2021 at 18:29:14 GMT+1 To: "pve-user at lists.proxmox.com" > Here is my "interfaces" file: auto loiface lo inet loopback iface enp9s0f1 inet manual auto vmbr0iface vmbr0 inet static address 192.168.20.2/23 gateway 192.168.20.1 bridge_ports enp9s0f1 bridge_stp off bridge_fd 0 iface enp11s0f0 inet manual iface enp11s0f1 inet manual iface enp9s0f0 inet manual iface enp3s0 inet manual iface enp2s0 inet manual On Sunday, February 14, 2021, 12:14:28 PM EST, Steve Frazier via pve-user > wrote: I need some help. I am new to Proxmox. I have it set up now using (1) one NIC to an internal network of 192.168.20.0/23 and I would like to use another NIC I have available for the internet which has (5) available IPs. I have attached a view of my existing network. I would like to set up an additional (NIC) when would allow me to assign to a virtual server that has two virtual NICs with one being the internet and the other being my internal network. While I am at it, I have another networking 192.168.30.0/24 that I would like to assign to an available NIC so that I could assign virtual servers to that NICat a later time. Could someone please help me on how to do this Thanks very much. _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.proxmox.com%2Fcgi-bin%2Fmailman%2Flistinfo%2Fpve-user&data=04%7C01%7Csmr%40kmi.com%7Cfb3db335b1844cd9162208d8d10e1613%7Cc2283768b8d34e008f3d85b1b4f03b33%7C0%7C0%7C637489205732810378%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=bbtAAx1zpomz3hCKtKmrW3UIFF6iyprwmGcer1Q0njQ%3D&reserved=0 CONFIDENTIALITY NOTICE: This communication may contain privileged and confidential information, or may otherwise be protected from disclosure, and is intended solely for use of the intended recipient(s). If you are not the intended recipient of this communication, please notify the sender that you have received this communication in error and delete and destroy all copies in your possession. -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: interfaces.txt URL: From sfrazier1111 at yahoo.com Tue Feb 16 20:10:54 2021 From: sfrazier1111 at yahoo.com (Steve Frazier) Date: Tue, 16 Feb 2021 19:10:54 +0000 (UTC) Subject: [PVE-User] Adding an Internet NIC to existing Proxmox In-Reply-To: <249BD6E3-5339-4587-A968-8453139883D1@kmi.com> References: <206447580.1166532.1613322212874.ref@mail.yahoo.com> <249BD6E3-5339-4587-A968-8453139883D1@kmi.com> Message-ID: <880208656.1824387.1613502654455@mail.yahoo.com> Hi Stefan! Thank you kindly! I will give it a try. Steve On Tuesday, February 16, 2021, 01:54:55 PM EST, Stefan M. Radman wrote: Hi Steve Attached and below is an /etc/network/interfaces that should do what you want. Just* keep interface enp9s0f1 connected to your private subnet?192.168.20.2/23* connect interface?enp9s0f0 to your internet subnet* connect internet VMs to vmbr1* connect interface?enp11s0f1 to your private subnet 192.168.30.2/23* connect VMs on subnet 192.168.30.2/23 to vmbr2?If you keep the comment lines (#) they'll show up in the web interface as well. Disclaimer: I did not test it but it should give you the right idea to go ahead. StefanPS: please disregard previous attachment - this is the correct one /etc/network/interfaces: auto loiface lo inet loopback iface enp9s0f1 inet manual# Uplink for vmbr0 auto vmbr0iface vmbr0 inet static? ? ? ? address 192.168.20.2/23? ? ? ? gateway 192.168.20.1? ? ? ? bridge_ports enp9s0f1? ? ? ? bridge_stp off? ? ? ? bridge_fd 0# Host Management iface enp11s0f0 inet manual# Uplink for vmbr1 auto vmbr1iface vmbr1 inet manual? ? ? ? bridge_ports enp9s0f0? ? ? ? bridge_stp off? ? ? ? bridge_fd 0# Bridge for VMs on the Internet iface enp11s0f1 inet manual# Uplink for vmbr2 auto vmbr2iface vmbr2 inet manual? ? ? ? bridge_ports enp11s0f1? ? ? ? bridge_stp off? ? ? ? bridge_fd 0# Bridge for VMs on network 192.168.30.0/24 iface enp9s0f0 inet manual iface enp3s0 inet manual iface enp2s0 inet manual On Feb 14, 2021, at 18:29, Steve Frazier via pve-user wrote: From: Steve Frazier Subject: Re: [PVE-User] Adding an Internet NIC to existing Proxmox Date: February 14, 2021 at 18:29:14 GMT+1 To: "pve-user at lists.proxmox.com" Here is my "interfaces" file: auto loiface lo inet loopback iface enp9s0f1 inet manual auto vmbr0iface vmbr0 inet static? ? ? ? address 192.168.20.2/23? ? ? ? gateway 192.168.20.1? ? ? ? bridge_ports enp9s0f1? ? ? ? bridge_stp off? ? ? ? bridge_fd 0 iface enp11s0f0 inet manual iface enp11s0f1 inet manual iface enp9s0f0 inet manual iface enp3s0 inet manual iface enp2s0 inet manual ???On Sunday, February 14, 2021, 12:14:28 PM EST, Steve Frazier via pve-user wrote: ? I need some help.? I am new to Proxmox.? ?I have it set up now using (1) one NIC to an internal network of 192.168.20.0/23 and I would like to use another NIC I have available for the internet which has (5) available IPs. I have attached a view of my existing network. I would like to set up an additional (NIC) when would allow me to assign to a virtual server that has two virtual NICs with one being the internet and the other being my internal network. While I am at it, I have another networking 192.168.30.0/24 that I would like to assign to an available NIC so that I could assign virtual servers to that NICat a later time. Could someone please help me on how to do this Thanks very much. _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.proxmox.com%2Fcgi-bin%2Fmailman%2Flistinfo%2Fpve-user&data=04%7C01%7Csmr%40kmi.com%7Cfb3db335b1844cd9162208d8d10e1613%7Cc2283768b8d34e008f3d85b1b4f03b33%7C0%7C0%7C637489205732810378%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=bbtAAx1zpomz3hCKtKmrW3UIFF6iyprwmGcer1Q0njQ%3D&reserved=0 CONFIDENTIALITY NOTICE:This communication may contain privileged and confidential information, or may otherwise be protected from disclosure, and is intended solely for use of the intended recipient(s). If you are not the intended recipient of this communication, please notify the sender that you have received this communication in error and delete and destroy all copies in your possession. From sfrazier1111 at yahoo.com Tue Feb 16 21:52:10 2021 From: sfrazier1111 at yahoo.com (Steve Frazier) Date: Tue, 16 Feb 2021 20:52:10 +0000 (UTC) Subject: [PVE-User] Adding an Internet NIC to existing Proxmox In-Reply-To: <21687B15-6EE4-4F27-A332-6C2FE4FB86C9@kmi.com> References: <206447580.1166532.1613322212874.ref@mail.yahoo.com> <21687B15-6EE4-4F27-A332-6C2FE4FB86C9@kmi.com> Message-ID: <55407184.1865958.1613508730876@mail.yahoo.com> Tried it and it worked as you explained!? Thanks Stefan.? I am taking this step by step.I haven't tried the 192.168.30.0 nic yet but will shortly. I will want to be able to trunk VLAN 20 and VLAN 30 down a NIC so that I will be able to use my VOIP phones on VLAN30 and be able to offer VLAN20 for the attached work stations, any suggestions and/or configuration file to do this? Thanks again for your help! On Tuesday, February 16, 2021, 01:52:54 PM EST, Stefan M. Radman wrote: Hi Steve Attached and below is an /etc/network/interfaces that should do what you want. Just* keep interface enp9s0f1 connected to your private subnet?192.168.20.2/23* connect interface?enp9s0f0 to your internet subnet* connect internet VMs to vmbr1* connect interface?enp11s0f1 to your private subnet 192.168.30.2/23* connect VMs on subnet 192.168.30.2/23 to vmbr2?If you keep the comment lines (#) they'll show up in the web interface as well. Disclaimer: I did not test it but it should give you the right idea to go ahead. Stefan /etc/network/interfaces: auto loiface lo inet loopback iface enp9s0f1 inet manual# Uplink for vmbr0 auto vmbr0iface vmbr0 inet static? ? ? ? address 192.168.20.2/23? ? ? ? gateway 192.168.20.1? ? ? ? bridge_ports enp9s0f1? ? ? ? bridge_stp off? ? ? ? bridge_fd 0# Host Management iface enp11s0f0 inet manual# Uplink for vmbr1 auto vmbr1iface vmbr1 inet manual? ? ? ? bridge_ports enp9s0f0? ? ? ? bridge_stp off? ? ? ? bridge_fd 0# Bridge for VMs on the Internet iface enp11s0f1 inet manual# Uplink for vmbr2 auto vmbr2iface vmbr2 inet manual? ? ? ? bridge_ports enp11s0f1? ? ? ? bridge_stp off? ? ? ? bridge_fd 0# Bridge for VMs on network 192.168.30.0/24 iface enp9s0f0 inet manual iface enp3s0 inet manual iface enp2s0 inet manual On Feb 14, 2021, at 18:29, Steve Frazier via pve-user wrote: From: Steve Frazier Subject: Re: [PVE-User] Adding an Internet NIC to existing Proxmox Date: February 14, 2021 at 18:29:14 GMT+1 To: "pve-user at lists.proxmox.com" Here is my "interfaces" file: auto loiface lo inet loopback iface enp9s0f1 inet manual auto vmbr0iface vmbr0 inet static? ? ? ? address 192.168.20.2/23? ? ? ? gateway 192.168.20.1? ? ? ? bridge_ports enp9s0f1? ? ? ? bridge_stp off? ? ? ? bridge_fd 0 iface enp11s0f0 inet manual iface enp11s0f1 inet manual iface enp9s0f0 inet manual iface enp3s0 inet manual iface enp2s0 inet manual ???On Sunday, February 14, 2021, 12:14:28 PM EST, Steve Frazier via pve-user wrote: ? I need some help.? I am new to Proxmox.? ?I have it set up now using (1) one NIC to an internal network of 192.168.20.0/23 and I would like to use another NIC I have available for the internet which has (5) available IPs. I have attached a view of my existing network. I would like to set up an additional (NIC) when would allow me to assign to a virtual server that has two virtual NICs with one being the internet and the other being my internal network. While I am at it, I have another networking 192.168.30.0/24 that I would like to assign to an available NIC so that I could assign virtual servers to that NICat a later time. Could someone please help me on how to do this Thanks very much. _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.proxmox.com%2Fcgi-bin%2Fmailman%2Flistinfo%2Fpve-user&data=04%7C01%7Csmr%40kmi.com%7Cfb3db335b1844cd9162208d8d10e1613%7Cc2283768b8d34e008f3d85b1b4f03b33%7C0%7C0%7C637489205732810378%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=bbtAAx1zpomz3hCKtKmrW3UIFF6iyprwmGcer1Q0njQ%3D&reserved=0 CONFIDENTIALITY NOTICE:This communication may contain privileged and confidential information, or may otherwise be protected from disclosure, and is intended solely for use of the intended recipient(s). If you are not the intended recipient of this communication, please notify the sender that you have received this communication in error and delete and destroy all copies in your possession. From smr at kmi.com Tue Feb 16 23:40:35 2021 From: smr at kmi.com (Stefan M. Radman) Date: Tue, 16 Feb 2021 22:40:35 +0000 Subject: [PVE-User] Adding an Internet NIC to existing Proxmox In-Reply-To: <55407184.1865958.1613508730876@mail.yahoo.com> References: <206447580.1166532.1613322212874.ref@mail.yahoo.com> <21687B15-6EE4-4F27-A332-6C2FE4FB86C9@kmi.com> <55407184.1865958.1613508730876@mail.yahoo.com> Message-ID: <0140E02A-133D-47DF-B58A-50F2CA182F1A@kmi.com> Here you go. See interface enp9s0f0 in the config below. It also shows how to run tagged (vmbr20,vmbr30) and untagged (vmbr10) traffic on the same trunk interface. I normally use the VLAN tag ID as the bridge identifier (e.g. vmbr20 and vmbr30). The host IP would typically be on vmbr0 and thus on the native (untagged) VLAN. Stefan /etc/network/interfaces: auto lo iface lo inet loopback iface enp9s0f1 inet manual # Uplink for vmbr0 auto vmbr0 iface vmbr0 inet static address 192.168.20.2/23 gateway 192.168.20.1 bridge_ports enp9s0f1 bridge_stp off bridge_fd 0 # Host Management iface enp11s0f0 inet manual # Uplink for vmbr1 auto vmbr1 iface vmbr1 inet manual bridge_ports enp9s0f0 bridge_stp off bridge_fd 0 # Bridge for VMs on the Internet iface enp11s0f1 inet manual # Uplink for vmbr2 auto vmbr2 iface vmbr2 inet manual bridge_ports enp11s0f1 bridge_stp off bridge_fd 0 # Bridge for VMs on network 192.168.30.0/24 iface enp9s0f0 inet manual # Trunk interface carrying VLAN20 and VLAN30 auto vmbr10 iface vmbr10 inet manual bridge_ports enp9s0f0 bridge_stp off bridge_fd 0 # Bridge for untagged traffic on trunk interface auto vmbr20 iface vmbr20 inet manual bridge_ports enp9s0f0.20 bridge_stp off bridge_fd 0 # Bridge for VMs on VLAN20 auto vmbr30 iface vmbr30 inet manual bridge_ports enp9s0f0.30 bridge_stp off bridge_fd 0 # Bridge for VMs on VLAN30 iface enp3s0 inet manual iface enp2s0 inet manual On Feb 16, 2021, at 21:52, Steve Frazier > wrote: Tried it and it worked as you explained! Thanks Stefan. I am taking this step by step. I haven't tried the 192.168.30.0 nic yet but will shortly. I will want to be able to trunk VLAN 20 and VLAN 30 down a NIC so that I will be able to use my VOIP phones on VLAN30 and be able to offer VLAN20 for the attached work stations, any suggestions and/or configuration file to do this? Thanks again for your help! On Tuesday, February 16, 2021, 01:52:54 PM EST, Stefan M. Radman > wrote: Hi Steve Attached and below is an /etc/network/interfaces that should do what you want. Just * keep interface enp9s0f1 connected to your private subnet 192.168.20.2/23 * connect interface enp9s0f0 to your internet subnet * connect internet VMs to vmbr1 * connect interface enp11s0f1 to your private subnet 192.168.30.2/23 * connect VMs on subnet 192.168.30.2/23 to vmbr2 If you keep the comment lines (#) they'll show up in the web interface as well. Disclaimer: I did not test it but it should give you the right idea to go ahead. Stefan /etc/network/interfaces: auto lo iface lo inet loopback iface enp9s0f1 inet manual # Uplink for vmbr0 auto vmbr0 iface vmbr0 inet static address 192.168.20.2/23 gateway 192.168.20.1 bridge_ports enp9s0f1 bridge_stp off bridge_fd 0 # Host Management iface enp11s0f0 inet manual # Uplink for vmbr1 auto vmbr1 iface vmbr1 inet manual bridge_ports enp9s0f0 bridge_stp off bridge_fd 0 # Bridge for VMs on the Internet iface enp11s0f1 inet manual # Uplink for vmbr2 auto vmbr2 iface vmbr2 inet manual bridge_ports enp11s0f1 bridge_stp off bridge_fd 0 # Bridge for VMs on network 192.168.30.0/24 iface enp9s0f0 inet manual iface enp3s0 inet manual iface enp2s0 inet manual On Feb 14, 2021, at 18:29, Steve Frazier via pve-user > wrote: From: Steve Frazier > Subject: Re: [PVE-User] Adding an Internet NIC to existing Proxmox Date: February 14, 2021 at 18:29:14 GMT+1 To: "pve-user at lists.proxmox.com" > Here is my "interfaces" file: auto loiface lo inet loopback iface enp9s0f1 inet manual auto vmbr0iface vmbr0 inet static address 192.168.20.2/23 gateway 192.168.20.1 bridge_ports enp9s0f1 bridge_stp off bridge_fd 0 iface enp11s0f0 inet manual iface enp11s0f1 inet manual iface enp9s0f0 inet manual iface enp3s0 inet manual iface enp2s0 inet manual On Sunday, February 14, 2021, 12:14:28 PM EST, Steve Frazier via pve-user > wrote: I need some help. I am new to Proxmox. I have it set up now using (1) one NIC to an internal network of 192.168.20.0/23 and I would like to use another NIC I have available for the internet which has (5) available IPs. I have attached a view of my existing network. I would like to set up an additional (NIC) when would allow me to assign to a virtual server that has two virtual NICs with one being the internet and the other being my internal network. While I am at it, I have another networking 192.168.30.0/24 that I would like to assign to an available NIC so that I could assign virtual servers to that NICat a later time. Could someone please help me on how to do this Thanks very much. _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.proxmox.com%2Fcgi-bin%2Fmailman%2Flistinfo%2Fpve-user&data=04%7C01%7Csmr%40kmi.com%7Cfb3db335b1844cd9162208d8d10e1613%7Cc2283768b8d34e008f3d85b1b4f03b33%7C0%7C0%7C637489205732810378%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=bbtAAx1zpomz3hCKtKmrW3UIFF6iyprwmGcer1Q0njQ%3D&reserved=0 CONFIDENTIALITY NOTICE: This communication may contain privileged and confidential information, or may otherwise be protected from disclosure, and is intended solely for use of the intended recipient(s). If you are not the intended recipient of this communication, please notify the sender that you have received this communication in error and delete and destroy all copies in your possession. CONFIDENTIALITY NOTICE: This communication may contain privileged and confidential information, or may otherwise be protected from disclosure, and is intended solely for use of the intended recipient(s). If you are not the intended recipient of this communication, please notify the sender that you have received this communication in error and delete and destroy all copies in your possession. From sfrazier1111 at yahoo.com Tue Feb 16 23:56:09 2021 From: sfrazier1111 at yahoo.com (Steve Frazier) Date: Tue, 16 Feb 2021 22:56:09 +0000 (UTC) Subject: [PVE-User] Adding an Internet NIC to existing Proxmox In-Reply-To: <0140E02A-133D-47DF-B58A-50F2CA182F1A@kmi.com> References: <206447580.1166532.1613322212874.ref@mail.yahoo.com> <21687B15-6EE4-4F27-A332-6C2FE4FB86C9@kmi.com> <55407184.1865958.1613508730876@mail.yahoo.com> <0140E02A-133D-47DF-B58A-50F2CA182F1A@kmi.com> Message-ID: <2026499981.1905964.1613516169524@mail.yahoo.com> Wow, thanks so much!? I will take a look and give it a try.? Thanks Stefan! Steve On Tuesday, February 16, 2021, 05:40:38 PM EST, Stefan M. Radman wrote: Here you go. See interface?enp9s0f0 in the config below.It also shows how to run tagged (vmbr20,vmbr30) and untagged (vmbr10) traffic on the same trunk interface. I normally use the VLAN tag ID as the bridge identifier (e.g. vmbr20 and vmbr30).The host IP would typically be on vmbr0 and thus on the native (untagged) VLAN. Stefan /etc/network/interfaces: auto lo iface lo inet loopback iface enp9s0f1 inet manual # Uplink for vmbr0 auto vmbr0 iface vmbr0 inet static ? ? ? ? address 192.168.20.2/23 ? ? ? ? gateway 192.168.20.1 ? ? ? ? bridge_ports enp9s0f1 ? ? ? ? bridge_stp off ? ? ? ? bridge_fd 0 # Host Management iface enp11s0f0 inet manual # Uplink for vmbr1 auto vmbr1 iface vmbr1 inet manual ? ? ? ? bridge_ports enp9s0f0 ? ? ? ? bridge_stp off ? ? ? ? bridge_fd 0 # Bridge for VMs on the Internet iface enp11s0f1 inet manual # Uplink for vmbr2 auto vmbr2 iface vmbr2 inet manual ? ? ? ? bridge_ports enp11s0f1 ? ? ? ? bridge_stp off ? ? ? ? bridge_fd 0 # Bridge for VMs on network 192.168.30.0/24 iface enp9s0f0 inet manual# Trunk interface carrying VLAN20 and VLAN30 auto vmbr10iface?vmbr10?inet manual ? ? ? ? bridge_ports?enp9s0f0 ? ? ? ? bridge_stp off ? ? ? ? bridge_fd 0 # Bridge for untagged traffic on trunk interface auto vmbr20iface vmbr20 inet manual ? ? ? ? bridge_ports?enp9s0f0.20 ? ? ? ? bridge_stp off ? ? ? ? bridge_fd 0 # Bridge for VMs on VLAN20 auto vmbr30iface vmbr30 inet manual ? ? ? ? bridge_ports?enp9s0f0.30 ? ? ? ? bridge_stp off ? ? ? ? bridge_fd 0 # Bridge for VMs on VLAN30 iface enp3s0 inet manual iface enp2s0 inet manual On Feb 16, 2021, at 21:52, Steve Frazier wrote: Tried it and it worked as you explained!? Thanks Stefan.? I am taking this step by step.I haven't tried the 192.168.30.0 nic yet but will shortly. I will want to be able to trunk VLAN 20 and VLAN 30 down a NIC so that I will be able to use my VOIP phones on VLAN30 and be able to offer VLAN20 for the attached work stations, any suggestions and/or configuration file to do this? Thanks again for your help! On Tuesday, February 16, 2021, 01:52:54 PM EST, Stefan M. Radman wrote: Hi Steve Attached and below is an /etc/network/interfaces that should do what you want. Just* keep interface enp9s0f1 connected to your private subnet?192.168.20.2/23* connect interface?enp9s0f0 to your internet subnet* connect internet VMs to vmbr1* connect interface?enp11s0f1 to your private subnet 192.168.30.2/23* connect VMs on subnet 192.168.30.2/23 to vmbr2?If you keep the comment lines (#) they'll show up in the web interface as well. Disclaimer: I did not test it but it should give you the right idea to go ahead. Stefan /etc/network/interfaces: auto loiface lo inet loopback iface enp9s0f1 inet manual# Uplink for vmbr0 auto vmbr0iface vmbr0 inet static? ? ? ? address 192.168.20.2/23? ? ? ? gateway 192.168.20.1? ? ? ? bridge_ports enp9s0f1? ? ? ? bridge_stp off? ? ? ? bridge_fd 0# Host Management iface enp11s0f0 inet manual# Uplink for vmbr1 auto vmbr1iface vmbr1 inet manual? ? ? ? bridge_ports enp9s0f0? ? ? ? bridge_stp off? ? ? ? bridge_fd 0# Bridge for VMs on the Internet iface enp11s0f1 inet manual# Uplink for vmbr2 auto vmbr2iface vmbr2 inet manual? ? ? ? bridge_ports enp11s0f1? ? ? ? bridge_stp off? ? ? ? bridge_fd 0# Bridge for VMs on network 192.168.30.0/24 iface enp9s0f0 inet manual iface enp3s0 inet manual iface enp2s0 inet manual On Feb 14, 2021, at 18:29, Steve Frazier via pve-user wrote: From:Steve Frazier Subject:Re: [PVE-User] Adding an Internet NIC to existing Proxmox Date:February 14, 2021 at 18:29:14 GMT+1 To:"pve-user at lists.proxmox.com" Here is my "interfaces" file: auto loiface lo inet loopback iface enp9s0f1 inet manual auto vmbr0iface vmbr0 inet static? ? ? ? address 192.168.20.2/23? ? ? ? gateway 192.168.20.1? ? ? ? bridge_ports enp9s0f1? ? ? ? bridge_stp off? ? ? ? bridge_fd 0 iface enp11s0f0 inet manual iface enp11s0f1 inet manual iface enp9s0f0 inet manual iface enp3s0 inet manual iface enp2s0 inet manual ???On Sunday, February 14, 2021, 12:14:28 PM EST, Steve Frazier via pve-user wrote: ? I need some help.? I am new to Proxmox.? ?I have it set up now using (1) one NIC to an internal network of 192.168.20.0/23 and I would like to use another NIC I have available for the internet which has (5) available IPs. I have attached a view of my existing network. I would like to set up an additional (NIC) when would allow me to assign to a virtual server that has two virtual NICs with one being the internet and the other being my internal network. While I am at it, I have another networking 192.168.30.0/24 that I would like to assign to an available NIC so that I could assign virtual servers to that NICat a later time. Could someone please help me on how to do this Thanks very much. _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.proxmox.com%2Fcgi-bin%2Fmailman%2Flistinfo%2Fpve-user&data=04%7C01%7Csmr%40kmi.com%7Cfb3db335b1844cd9162208d8d10e1613%7Cc2283768b8d34e008f3d85b1b4f03b33%7C0%7C0%7C637489205732810378%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=bbtAAx1zpomz3hCKtKmrW3UIFF6iyprwmGcer1Q0njQ%3D&reserved=0 CONFIDENTIALITY NOTICE: This communication may contain privileged and confidential information, or may otherwise be protected from disclosure, and is intended solely for use of the intended recipient(s). If you are not the intended recipient of this communication, please notify the sender that you have received this communication in error and delete and destroy all copies in your possession. CONFIDENTIALITY NOTICE:This communication may contain privileged and confidential information, or may otherwise be protected from disclosure, and is intended solely for use of the intended recipient(s). If you are not the intended recipient of this communication, please notify the sender that you have received this communication in error and delete and destroy all copies in your possession. From ralf.storm at konzept-is.de Thu Feb 18 09:20:10 2021 From: ralf.storm at konzept-is.de (storm) Date: Thu, 18 Feb 2021 09:20:10 +0100 Subject: [PVE-User] Backup Failing sporadically due to "PBS backups are not supported by the running QEMU version" In-Reply-To: <1612783750.os88l87hm3.astroid@nora.none> References: <1612772213.gdeednkn6n.astroid@nora.none> <1612779647.88nw1czupf.astroid@nora.none> <319abf74-8e57-666e-31df-fe1c1d8a8db8@konzept-is.de> <1612783750.os88l87hm3.astroid@nora.none> Message-ID: Hello, so the Backup was working for a few days and then the error appaers again. additionally I cannot live migrate the vm: 2021-02-18 09:15:11 ERROR: migration aborted (duration 00:00:03): VM 3091 qmp command 'query-machines' failed - unable to connect to VM 3091 qmp socket - timeout after 31 retries TASK ERROR: migration aborted and the same for console best regards From f.gruenbichler at proxmox.com Thu Feb 18 09:50:22 2021 From: f.gruenbichler at proxmox.com (=?UTF-8?Q?Fabian_Gr=C3=BCnbichler?=) Date: Thu, 18 Feb 2021 09:50:22 +0100 (CET) Subject: [PVE-User] Backup Failing sporadically due to "PBS backups are not supported by the running QEMU version" In-Reply-To: References: <1612772213.gdeednkn6n.astroid@nora.none> <1612779647.88nw1czupf.astroid@nora.none> <319abf74-8e57-666e-31df-fe1c1d8a8db8@konzept-is.de> <1612783750.os88l87hm3.astroid@nora.none> Message-ID: <1935400026.3127.1613638223121@webmail.proxmox.com> are the package versions unchanged? could you try upgrading to pve-qemu-kvm 5.1.0-8? > storm hat am 18.02.2021 09:20 geschrieben: > > > Hello, > > > so the Backup was working for a few days and then the error appaers again. > > additionally I cannot live migrate the vm: > > 2021-02-18 09:15:11 ERROR: migration aborted (duration 00:00:03): VM > 3091 qmp command 'query-machines' failed - unable to connect to VM 3091 > qmp socket - timeout after 31 retries TASK ERROR: migration aborted > > and the same for console > > > best regards From ralf.storm at konzept-is.de Thu Feb 18 11:08:04 2021 From: ralf.storm at konzept-is.de (storm) Date: Thu, 18 Feb 2021 11:08:04 +0100 Subject: [PVE-User] Backup Failing sporadically due to "PBS backups are not supported by the running QEMU version" In-Reply-To: <1935400026.3127.1613638223121@webmail.proxmox.com> References: <1612772213.gdeednkn6n.astroid@nora.none> <1612779647.88nw1czupf.astroid@nora.none> <319abf74-8e57-666e-31df-fe1c1d8a8db8@konzept-is.de> <1612783750.os88l87hm3.astroid@nora.none> <1935400026.3127.1613638223121@webmail.proxmox.com> Message-ID: <630c3278-3174-1460-ea1e-5fe9853210d0@konzept-is.de> upgrade of the nodes is in progress, moved the affected vm to a node which is upgraded to 5.1.0-8, backup is runnging right now - will keep you informed if this happens again on this version Am 18/02/2021 um 09:50 schrieb Fabian Gr?nbichler: > are the package versions unchanged? could you try upgrading to pve-qemu-kvm 5.1.0-8? > > >> storm hat am 18.02.2021 09:20 geschrieben: >> >> >> Hello, >> >> >> so the Backup was working for a few days and then the error appaers again. >> >> additionally I cannot live migrate the vm: >> >> 2021-02-18 09:15:11 ERROR: migration aborted (duration 00:00:03): VM >> 3091 qmp command 'query-machines' failed - unable to connect to VM 3091 >> qmp socket - timeout after 31 retries TASK ERROR: migration aborted >> >> and the same for console >> >> >> best regards From ralf.storm at konzept-is.de Fri Feb 19 07:20:54 2021 From: ralf.storm at konzept-is.de (storm) Date: Fri, 19 Feb 2021 07:20:54 +0100 Subject: [PVE-User] Backup Failing sporadically due to "PBS backups are not supported by the running QEMU version" In-Reply-To: <630c3278-3174-1460-ea1e-5fe9853210d0@konzept-is.de> References: <1612772213.gdeednkn6n.astroid@nora.none> <1612779647.88nw1czupf.astroid@nora.none> <319abf74-8e57-666e-31df-fe1c1d8a8db8@konzept-is.de> <1612783750.os88l87hm3.astroid@nora.none> <1935400026.3127.1613638223121@webmail.proxmox.com> <630c3278-3174-1460-ea1e-5fe9853210d0@konzept-is.de> Message-ID: <18439dd6-b2bc-de05-6701-30fe39ffdf10@konzept-is.de> so the last backup during last night failed again with the same error on the same node with pve-qemu-kvm 5.1.0-8 on which the backup was working yesterday any ideas? Am 18/02/2021 um 11:08 schrieb storm: > upgrade of the nodes is in progress, moved the affected vm to a node > which is upgraded to 5.1.0-8, backup is runnging right now - will keep > you informed if this happens again on this version > > > > Am 18/02/2021 um 09:50 schrieb Fabian Gr?nbichler: >> are the package versions unchanged? could you try upgrading to >> pve-qemu-kvm 5.1.0-8? >> >> >>> storm hat am 18.02.2021 09:20 geschrieben: >>> >>> ? Hello, >>> >>> >>> so the Backup was working for a few days and then the error appaers >>> again. >>> >>> additionally I cannot live migrate the vm: >>> >>> 2021-02-18 09:15:11 ERROR: migration aborted (duration 00:00:03): VM >>> 3091 qmp command 'query-machines' failed - unable to connect to VM 3091 >>> qmp socket - timeout after 31 retries TASK ERROR: migration aborted >>> >>> and the same for console >>> >>> >>> best regards > > _______________________________________________ > pve-user mailing list > pve-user at lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user From f.gruenbichler at proxmox.com Fri Feb 19 08:26:35 2021 From: f.gruenbichler at proxmox.com (Fabian =?iso-8859-1?q?Gr=FCnbichler?=) Date: Fri, 19 Feb 2021 08:26:35 +0100 Subject: [PVE-User] Backup Failing sporadically due to "PBS backups are not supported by the running QEMU version" In-Reply-To: <18439dd6-b2bc-de05-6701-30fe39ffdf10@konzept-is.de> References: <1612772213.gdeednkn6n.astroid@nora.none> <1612779647.88nw1czupf.astroid@nora.none> <319abf74-8e57-666e-31df-fe1c1d8a8db8@konzept-is.de> <1612783750.os88l87hm3.astroid@nora.none> <1935400026.3127.1613638223121@webmail.proxmox.com> <630c3278-3174-1460-ea1e-5fe9853210d0@konzept-is.de> <18439dd6-b2bc-de05-6701-30fe39ffdf10@konzept-is.de> Message-ID: <1613719500.vv82bs2knx.astroid@nora.none> On February 19, 2021 7:20 am, storm wrote: > so the last backup during last night failed again with the same error on > the same node with pve-qemu-kvm 5.1.0-8 on which the backup was working > yesterday > > any ideas? just to confirm, can you do the following right after a backup fails, and then again after you got the VM back into a working state? 123 should be replaced with the correct VMID.. perl -e 'use strict; use warnings; use PVE::QemuServer::Monitor; use Data::Dumper; $Data::Dumper::Sortkeys=1; print Dumper(PVE::QemuServer::Monitor::qmp_cmd(123, { "execute" => "query-proxmox-support" })), "\n";' it issues exactly the QMP command that queries the running VM for proxmox-related features. it should return right away with output like this: $VAR1 = { 'pbs-dirty-bitmap' => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' ), 'pbs-dirty-bitmap-migration' => $VAR1->{'pbs-dirty-bitmap'}, 'pbs-library-version' => '1.0.2 (0723f448fdad1fcf84ec7ae7d702baba11cb06d1)', 'query-bitmap-info' => $VAR1->{'pbs-dirty-bitmap'} }; (the commit hash will be different) or with some error message. From mhill at inett.de Fri Feb 19 15:39:49 2021 From: mhill at inett.de (Maximilian Hill) Date: Fri, 19 Feb 2021 15:39:49 +0100 Subject: [PVE-User] Keymap PVE 6.3-3 Message-ID: Hello, I have 2 pve clusters, which I need to access through Citrx. On these two clusters, something messes up the keymap pretty bad ([arrow down] ends up as [enter] in the vm via the noVNC Console. That's obviousely pretty bad, but works quite well with containers and the pve node shell via term.js Has someone experienced such problems before? Best regards, Maximilian Hil? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From elacunza at binovo.es Tue Feb 23 08:46:35 2021 From: elacunza at binovo.es (Eneko Lacunza) Date: Tue, 23 Feb 2021 08:46:35 +0100 Subject: PBS backup/restore issue Message-ID: Hi all, Yesterday I hit a problem with PBS. My sequence of actions was: 1.a Install debian 10 VM (updated to latest, only system utilities and SSH server) 1.b Add SSH access key to root 1.c Shutdown VM 1.d Backup to PBS and to NFS v?a regular backup. 1.e Start VM and install various packages - all OK. 1.f Stop VM 2.a Restore VM from PBS backup 2.b Start VM and install various packages - apt complains about no newline in linux-image-amd64 file manifest, can't install new packages. 2.c Stop VM 3.a Restore VM from NFS regular backup 3.b Start VM and install various packages - all OK. I hit the issue 3 times, and tried creating a new VM and a new Debian installation each time. Any ideas? I have the "broken" PBS backups saved. VM node: # pveversion -v proxmox-ve: 6.3-1 (running kernel: 5.4.78-2-pve) pve-manager: 6.3-3 (running version: 6.3-3/eee5f901) pve-kernel-5.4: 6.3-3 pve-kernel-helper: 6.3-3 pve-kernel-5.3: 6.1-6 pve-kernel-5.4.78-2-pve: 5.4.78-2 pve-kernel-5.4.65-1-pve: 5.4.65-1 pve-kernel-5.4.44-2-pve: 5.4.44-2 pve-kernel-5.3.18-3-pve: 5.3.18-3 ceph: 14.2.16-pve1 ceph-fuse: 14.2.16-pve1 corosync: 3.0.4-pve1 criu: 3.11-3 glusterfs-client: 5.5-3 ifupdown: residual config ifupdown2: 3.0.0-1+pve3 libjs-extjs: 6.0.1-10 libknet1: 1.16-pve1 libproxmox-acme-perl: 1.0.7 libproxmox-backup-qemu0: 1.0.2-1 libpve-access-control: 6.1-3 libpve-apiclient-perl: 3.1-3 libpve-common-perl: 6.3-2 libpve-guest-common-perl: 3.1-4 libpve-http-server-perl: 3.1-1 libpve-storage-perl: 6.3-4 libqb0: 1.0.5-1 libspice-server1: 0.14.2-4~pve6+1 lvm2: 2.03.02-pve4 lxc-pve: 4.0.3-1 lxcfs: 4.0.6-pve1 novnc-pve: 1.1.0-1 proxmox-backup-client: 1.0.6-1 proxmox-mini-journalreader: 1.1-1 proxmox-widget-toolkit: 2.4-3 pve-cluster: 6.2-1 pve-container: 3.3-2 pve-docs: 6.3-1 pve-edk2-firmware: 2.20200531-1 pve-firewall: 4.1-3 pve-firmware: 3.1-3 pve-ha-manager: 3.1-1 pve-i18n: 2.2-2 pve-qemu-kvm: 5.1.0-8 pve-xtermjs: 4.7.0-3 qemu-server: 6.3-3 smartmontools: 7.1-pve2 spiceterm: 3.1-1 vncterm: 1.6-2 zfsutils-linux: 0.8.5-pve1 PBS node: # pveversion -v proxmox-ve: 6.3-1 (running kernel: 5.4.78-2-pve) pve-manager: 6.3-3 (running version: 6.3-3/eee5f901) pve-kernel-5.4: 6.3-3 pve-kernel-helper: 6.3-3 pve-kernel-5.3: 6.1-6 pve-kernel-5.4.78-2-pve: 5.4.78-2 pve-kernel-5.4.65-1-pve: 5.4.65-1 pve-kernel-5.4.44-2-pve: 5.4.44-2 pve-kernel-5.3.18-3-pve: 5.3.18-3 ceph: residual config ceph-fuse: 12.2.13-pve1 corosync: 3.0.4-pve1 criu: 3.11-3 glusterfs-client: 5.5-3 ifupdown: 0.8.35+pve1 libjs-extjs: 6.0.1-10 libknet1: 1.16-pve1 libproxmox-acme-perl: 1.0.7 libproxmox-backup-qemu0: 1.0.2-1 libpve-access-control: 6.1-3 libpve-apiclient-perl: 3.1-3 libpve-common-perl: 6.3-2 libpve-guest-common-perl: 3.1-4 libpve-http-server-perl: 3.1-1 libpve-storage-perl: 6.3-4 libqb0: 1.0.5-1 libspice-server1: 0.14.2-4~pve6+1 lvm2: 2.03.02-pve4 lxc-pve: 4.0.3-1 lxcfs: 4.0.6-pve1 novnc-pve: 1.1.0-1 proxmox-backup-client: 1.0.6-1 proxmox-mini-journalreader: 1.1-1 proxmox-widget-toolkit: 2.4-3 pve-cluster: 6.2-1 pve-container: 3.3-2 pve-docs: 6.3-1 pve-edk2-firmware: 2.20200531-1 pve-firewall: 4.1-3 pve-firmware: 3.1-3 pve-ha-manager: 3.1-1 pve-i18n: 2.2-2 pve-qemu-kvm: 5.1.0-8 pve-xtermjs: 4.7.0-3 qemu-server: 6.3-3 smartmontools: 7.1-pve2 spiceterm: 3.1-1 vncterm: 1.6-2 zfsutils-linux: 0.8.5-pve1 -- EnekoLacunza Director T?cnico | Zuzendari teknikoa Binovo IT Human Project 943 569 206 elacunza at binovo.es binovo.es Astigarragako Bidea, 2 - 2 izda. Oficina 10-11, 20180 Oiartzun youtube linkedin From dietmar at proxmox.com Tue Feb 23 09:06:54 2021 From: dietmar at proxmox.com (Dietmar Maurer) Date: Tue, 23 Feb 2021 09:06:54 +0100 (CET) Subject: [PVE-User] PBS backup/restore issue In-Reply-To: References: Message-ID: <174161805.3587.1614067615911@webmail.proxmox.com> > I hit the issue 3 times, and tried creating a new VM and a new Debian > installation each time. > > Any ideas? I have the "broken" PBS backups saved. Already tried to verify the datastore? (Datastore Verify is OK?) From elacunza at binovo.es Tue Feb 23 10:11:46 2021 From: elacunza at binovo.es (Eneko Lacunza) Date: Tue, 23 Feb 2021 10:11:46 +0100 Subject: [PVE-User] PBS backup/restore issue In-Reply-To: <174161805.3587.1614067615911@webmail.proxmox.com> References: <174161805.3587.1614067615911@webmail.proxmox.com> Message-ID: <53e4c243-cd21-2267-094c-fdaa9e97b5f3@binovo.es> Hi Dietmar, El 23/2/21 a las 9:06, Dietmar Maurer escribi?: >> I hit the issue 3 times, and tried creating a new VM and a new Debian >> installation each time. >> >> Any ideas? I have the "broken" PBS backups saved. > Already tried to verify the datastore? (Datastore Verify is OK?) Didn't, just launched a manual verify, didn't find a way to verify only one VM's backups so it will take a while. Seems I removed 2 of 3 PBS backups, but the remaining one reproduces the issue (although shows a different error). I have the corresponding vzdump to compare. Will report back about the verify, PBS and NFS server are the same disks... :) Thanks -- EnekoLacunza Director T?cnico | Zuzendari teknikoa Binovo IT Human Project 943 569 206 elacunza at binovo.es binovo.es Astigarragako Bidea, 2 - 2 izda. Oficina 10-11, 20180 Oiartzun youtube linkedin From elacunza at binovo.es Tue Feb 23 11:17:42 2021 From: elacunza at binovo.es (Eneko Lacunza) Date: Tue, 23 Feb 2021 11:17:42 +0100 Subject: [PVE-User] PBS backup/restore issue In-Reply-To: References: <174161805.3587.1614067615911@webmail.proxmox.com> Message-ID: <801561fb-1b81-5412-b8e0-22e23d6f7892@binovo.es> Hi again, El 23/2/21 a las 10:11, Eneko Lacunza via pve-user escribi?: >>> I hit the issue 3 times, and tried creating a new VM and a new Debian >>> installation each time. >>> >>> Any ideas? I have the "broken" PBS backups saved. >> Already tried to verify the datastore? (Datastore Verify is OK?) > Didn't, just launched a manual verify, didn't find a way to verify > only one VM's backups so it will take a while. > > Seems I removed 2 of 3 PBS backups, but the remaining one reproduces > the issue (although shows a different error). I have the corresponding > vzdump to compare. > > Will report back about the verify, PBS and NFS server are the same > disks... :) Verify state is "All OK". Cheers -- EnekoLacunza Director T?cnico | Zuzendari teknikoa Binovo IT Human Project 943 569 206 elacunza at binovo.es binovo.es Astigarragako Bidea, 2 - 2 izda. Oficina 10-11, 20180 Oiartzun youtube linkedin From lists at merit.unu.edu Wed Feb 24 10:44:33 2021 From: lists at merit.unu.edu (mj) Date: Wed, 24 Feb 2021 10:44:33 +0100 Subject: [PVE-User] PBS backup/restore issue In-Reply-To: References: Message-ID: Hi Eneko, Just to say: that sounds scary! Do you have any update on this..? MJ From uwe.sauter.de at gmail.com Wed Feb 24 10:56:49 2021 From: uwe.sauter.de at gmail.com (Uwe Sauter) Date: Wed, 24 Feb 2021 10:56:49 +0100 Subject: [PVE-User] =?utf-8?q?Update_to_ZFS_2_available_=E2=80=93_breaks_?= =?utf-8?q?dependencies?= Message-ID: Hi all, since yesterday, 2021-02-23, updates for ZFS 2.0.3 are available. Currently my systems have 0.8.5 installed. When trying to update using aptitude I get a popup reading: ################### Some packages were broken and have been fixed: Keep the following packages at their current version: libnvpair2linux [Not Installed] libuutil2linux [Not Installed] libzfs3linux [Not Installed] libzpool3linux [Not Installed] zfs-initramfs [0.8.5-pve1 (now, stable)] zfs-zed [0.8.5-pve1 (now, stable)] zfsutils-linux [0.8.5-pve1 (now, stable)] ################### Could someone advice (possible from Proxmox team) advice on how to apply the updates in this particular situation? Regards, Uwe From elacunza at binovo.es Wed Feb 24 11:06:14 2021 From: elacunza at binovo.es (Eneko Lacunza) Date: Wed, 24 Feb 2021 11:06:14 +0100 Subject: [PVE-User] PBS backup/restore issue In-Reply-To: References: Message-ID: <217884a3-2097-dbc0-d259-40e95b986427@binovo.es> Hi MJ, None yet. We're testing PBS, so I have good backups with plain-old vzdump for all servers. I found this during some development tests, and affected VMs were just for testing a debian package installation. I can also add that I did the same test with Ubuntu 20.04, and I didn't reproduce problems with apt after restore (but maybe other non-apt related files are affected). Cheers El 24/2/21 a las 10:44, mj escribi?: > Hi Eneko, > > Just to say: that sounds scary! Do you have any update on this..? > > MJ > > _______________________________________________ > pve-user mailing list > pve-user at lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user > -- EnekoLacunza Director T?cnico | Zuzendari teknikoa Binovo IT Human Project 943 569 206 elacunza at binovo.es binovo.es Astigarragako Bidea, 2 - 2 izda. Oficina 10-11, 20180 Oiartzun youtube linkedin From s.ivanov at proxmox.com Wed Feb 24 11:11:40 2021 From: s.ivanov at proxmox.com (Stoiko Ivanov) Date: Wed, 24 Feb 2021 11:11:40 +0100 Subject: [PVE-User] =?utf-8?q?Update_to_ZFS_2_available_=E2=80=93_breaks_?= =?utf-8?q?dependencies?= In-Reply-To: References: Message-ID: <20210224111140.0af49108@rosa.proxmox.com> On Wed, 24 Feb 2021 10:56:49 +0100 Uwe Sauter wrote: > Hi all, > > since yesterday, 2021-02-23, updates for ZFS 2.0.3 are available. > Currently my systems have 0.8.5 installed. When trying to update using aptitude I get a popup reading: > > ################### > Some packages were broken and have been fixed: > > Keep the following packages at their current version: > libnvpair2linux [Not Installed] > libuutil2linux [Not Installed] > libzfs3linux [Not Installed] > libzpool3linux [Not Installed] > zfs-initramfs [0.8.5-pve1 (now, stable)] > zfs-zed [0.8.5-pve1 (now, stable)] > zfsutils-linux [0.8.5-pve1 (now, stable)] > ################### With ZFS 2.0.0 most shipped libraries got bumped to new major versions (and new package names) e.g. libnvpair1linux -> libnvpair2linux usually this gets resolved correctly (and worked here on many systems) by running `apt full-upgrade` (instead of `apt upgrade`) It has been a while since I worked with `aptitude` - but did you run `aptitude full-upgrade`? else - does it work if you upgrade using `apt full-upgrade`? I hope this helps! Regards, stoiko > > Could someone advice (possible from Proxmox team) advice on how to apply the updates in this > particular situation? > > > Regards, > > Uwe > > _______________________________________________ > pve-user mailing list > pve-user at lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > From mir at miras.org Wed Feb 24 11:09:42 2021 From: mir at miras.org (Michael Rasmussen) Date: Wed, 24 Feb 2021 11:09:42 +0100 Subject: [PVE-User] Update to ZFS 2 available =?UTF-8?B?4oCT?= breaks dependencies In-Reply-To: References: Message-ID: <20210224110942.4db93ebf@sleipner.datanom.net> On Wed, 24 Feb 2021 10:56:49 +0100 Uwe Sauter wrote: > Hi all, > > since yesterday, 2021-02-23, updates for ZFS 2.0.3 are available. > Currently my systems have 0.8.5 installed. When trying to update > using aptitude I get a popup reading: > > ################### > Some packages were broken and have been fixed: > > Keep the following packages at their current version: > libnvpair2linux [Not Installed] > libuutil2linux [Not Installed] > libzfs3linux [Not Installed] > libzpool3linux [Not Installed] > zfs-initramfs [0.8.5-pve1 (now, stable)] > zfs-zed [0.8.5-pve1 (now, stable)] > zfsutils-linux [0.8.5-pve1 (now, stable)] > ################### > > Could someone advice (possible from Proxmox team) advice on how to > apply the updates in this particular situation? > In pve-nosubscription: The following updates are available: zfs-zed/stable 2.0.3-pve1 amd64 [upgradable from: 0.8.5-pve1] zfsutils-linux/stable 2.0.3-pve1 amd64 [upgradable from: 0.8.5-pve1] So coming soon to a pve-enterprise near you ;-) -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc https://pgp.key-server.io/pks/lookup?search=0xD3C9A00E mir datanom net https://pgp.key-server.io/pks/lookup?search=0xE501F51C mir miras org https://pgp.key-server.io/pks/lookup?search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: He hated being thought of as one of those people that wore stupid ornamental armour. It was gilt by association. -- Terry Pratchett, "Night Watch" -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 488 bytes Desc: OpenPGP digital signature URL: From s.reiter at proxmox.com Wed Feb 24 11:34:33 2021 From: s.reiter at proxmox.com (Stefan Reiter) Date: Wed, 24 Feb 2021 11:34:33 +0100 Subject: [PVE-User] PBS backup/restore issue In-Reply-To: References: Message-ID: <748aba12-573f-0bc8-17d7-b94b9b9b8cef@proxmox.com> On 2/23/21 8:46 AM, Eneko Lacunza wrote: > Hi all, > > Yesterday I hit a problem with PBS. My sequence of actions was: > > 1.a Install debian 10 VM (updated to latest, only system utilities and > SSH server) > 1.b Add SSH access key to root > 1.c Shutdown VM > 1.d Backup to PBS and to NFS v?a regular backup. > 1.e Start VM and install various packages - all OK. > 1.f Stop VM > > 2.a Restore VM from PBS backup > 2.b Start VM and install various packages - apt complains about no > newline in linux-image-amd64 file manifest, can't install new packages. > 2.c Stop VM > > 3.a Restore VM from NFS regular backup > 3.b Start VM and install various packages - all OK. > Hi, I tried to replicate your problem by following pretty much the exact steps you mention above, but to no avail. Fresh Debian 10 VM, PBS backup + VMA to local disk, restored both and both work. I compared the disk images of the restored VMs with md5sum and the hashes matched, as expected. Tested with the latest package versions, though using QEMU 5.2.0, but the backup and restore functionality shouldn't have changed there. Could you try to see what the difference is between the restored disks from the VMA and the PBS backups (if there is any)? I.e. restore them both to seperate VMIDs and then compare hashes and/or run (v)bindiff on the disk images (best restore to a directory storage in 'raw' mode) to see where they differ - of course, this only makes sense if the backups are created when the VM was shut down. Potentially something about your NFS/PBS setup is corrupting data? You mentioned in another mail that they "are the same disks", how is that configured? Also please post the exact error messages APT is giving you, and your VM config. > I hit the issue 3 times, and tried creating a new VM and a new Debian > installation each time. > > Any ideas? I have the "broken" PBS backups saved. > > VM node: > # pveversion -v [...] > > PBS node: > # pveversion -v [...] FYI, the more useful command for PBS would be: proxmox-backup-manager versions --verbose From uwe.sauter.de at gmail.com Wed Feb 24 11:50:03 2021 From: uwe.sauter.de at gmail.com (Uwe Sauter) Date: Wed, 24 Feb 2021 11:50:03 +0100 Subject: [PVE-User] =?utf-8?q?Update_to_ZFS_2_available_=E2=80=93_breaks_?= =?utf-8?q?dependencies?= In-Reply-To: <20210224111140.0af49108@rosa.proxmox.com> References: <20210224111140.0af49108@rosa.proxmox.com> Message-ID: <806e6159-3ca1-530a-ff98-99ac4a4ced05@gmail.com> Am 24.02.21 um 11:11 schrieb Stoiko Ivanov: > On Wed, 24 Feb 2021 10:56:49 +0100 > Uwe Sauter wrote: > >> Hi all, >> >> since yesterday, 2021-02-23, updates for ZFS 2.0.3 are available. >> Currently my systems have 0.8.5 installed. When trying to update using aptitude I get a popup reading: >> >> ################### >> Some packages were broken and have been fixed: >> >> Keep the following packages at their current version: >> libnvpair2linux [Not Installed] >> libuutil2linux [Not Installed] >> libzfs3linux [Not Installed] >> libzpool3linux [Not Installed] >> zfs-initramfs [0.8.5-pve1 (now, stable)] >> zfs-zed [0.8.5-pve1 (now, stable)] >> zfsutils-linux [0.8.5-pve1 (now, stable)] >> ################### > With ZFS 2.0.0 most shipped libraries got bumped to new major versions > (and new package names) > e.g. > libnvpair1linux -> libnvpair2linux > > usually this gets resolved correctly (and worked here on many systems) > by running `apt full-upgrade` (instead of `apt upgrade`) > > It has been a while since I worked with `aptitude` - but did you > run `aptitude full-upgrade`? > > else - does it work if you upgrade using `apt full-upgrade`? ################################ # apt update Hit:1 http://ftp-stud.hs-esslingen.de/debian buster InRelease Hit:2 http://ftp-stud.hs-esslingen.de/debian buster-updates InRelease Hit:3 http://ftp-stud.hs-esslingen.de/debian-security buster/updates InRelease Hit:4 http://ftp-stud.hs-esslingen.de/debian buster-backports InRelease Hit:5 http://download.proxmox.com/debian/ceph-octopus buster InRelease Hit:6 http://download.proxmox.com/debian/pve buster InRelease Reading package lists... Done Building dependency tree Reading state information... Done 15 packages can be upgraded. Run 'apt list --upgradable' to see them. # aptitude full-upgrade The following NEW packages will be installed: libnvpair2linux{ab} libuutil2linux{ab} libzfs3linux{ab} libzpool3linux{ab} pve-headers-5.4.98-1-pve{a} pve-kernel-5.4.98-1-pve{a} The following packages will be REMOVED: pve-headers-5.4.78-2-pve{u} The following packages will be upgraded: libproxmox-backup-qemu0 libpve-common-perl libpve-guest-common-perl libpve-storage-perl pve-container pve-firmware pve-headers-5.4 pve-kernel-5.4 pve-kernel-helper pve-manager pve-qemu-kvm spl zfs-initramfs zfs-zed zfsutils-linux 15 packages upgraded, 6 newly installed, 1 to remove and 0 not upgraded. Need to get 0 B/159 MB of archives. After unpacking 296 MB will be used. The following packages have unmet dependencies: libzfs3linux : Breaks: libzfs2linux but 0.8.5-pve1 is installed libzpool3linux : Breaks: libzpool2linux but 0.8.5-pve1 is installed libnvpair2linux : Breaks: libnvpair1linux but 0.8.5-pve1 is installed libuutil2linux : Breaks: libuutil1linux but 0.8.5-pve1 is installed The following actions will resolve these dependencies: Keep the following packages at their current version: 1) libnvpair2linux [Not Installed] 2) libuutil2linux [Not Installed] 3) libzfs3linux [Not Installed] 4) libzpool3linux [Not Installed] 5) zfs-initramfs [0.8.5-pve1 (now, stable)] 6) zfs-zed [0.8.5-pve1 (now, stable)] 7) zfsutils-linux [0.8.5-pve1 (now, stable)] Accept this solution? [Y/n/q/?] q Abandoning all efforts to resolve these dependencies. Abort. ################################ Looks like I'm not getting the same updates as you do. I'll wait a few days and check back. Thanks, Uwe > I hope this helps! > Regards, > stoiko > > >> >> Could someone advice (possible from Proxmox team) advice on how to apply the updates in this >> particular situation? >> >> >> Regards, >> >> Uwe >> >> _______________________________________________ >> pve-user mailing list >> pve-user at lists.proxmox.com >> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user >> >> > > From s.ivanov at proxmox.com Wed Feb 24 12:40:53 2021 From: s.ivanov at proxmox.com (Stoiko Ivanov) Date: Wed, 24 Feb 2021 12:40:53 +0100 Subject: [PVE-User] =?utf-8?q?Update_to_ZFS_2_available_=E2=80=93_breaks_?= =?utf-8?q?dependencies?= In-Reply-To: <806e6159-3ca1-530a-ff98-99ac4a4ced05@gmail.com> References: <20210224111140.0af49108@rosa.proxmox.com> <806e6159-3ca1-530a-ff98-99ac4a4ced05@gmail.com> Message-ID: <20210224124053.6a2216b7@rosa.proxmox.com> On Wed, 24 Feb 2021 11:50:03 +0100 Uwe Sauter wrote: > Am 24.02.21 um 11:11 schrieb Stoiko Ivanov: > > On Wed, 24 Feb 2021 10:56:49 +0100 > > Uwe Sauter wrote: > > > >> Hi all, > >> > >> since yesterday, 2021-02-23, updates for ZFS 2.0.3 are available. > >> Currently my systems have 0.8.5 installed. When trying to update using aptitude I get a popup reading: > >> > >> ################### > >> Some packages were broken and have been fixed: > >> > >> Keep the following packages at their current version: > >> libnvpair2linux [Not Installed] > >> libuutil2linux [Not Installed] > >> libzfs3linux [Not Installed] > >> libzpool3linux [Not Installed] > >> zfs-initramfs [0.8.5-pve1 (now, stable)] > >> zfs-zed [0.8.5-pve1 (now, stable)] > >> zfsutils-linux [0.8.5-pve1 (now, stable)] > >> ################### > > With ZFS 2.0.0 most shipped libraries got bumped to new major versions > > (and new package names) > > e.g. > > libnvpair1linux -> libnvpair2linux > > > > usually this gets resolved correctly (and worked here on many systems) > > by running `apt full-upgrade` (instead of `apt upgrade`) > > > > It has been a while since I worked with `aptitude` - but did you > > run `aptitude full-upgrade`? > > > > else - does it work if you upgrade using `apt full-upgrade`? > > ################################ > > # apt update > Hit:1 http://ftp-stud.hs-esslingen.de/debian buster InRelease > Hit:2 http://ftp-stud.hs-esslingen.de/debian buster-updates InRelease > Hit:3 http://ftp-stud.hs-esslingen.de/debian-security buster/updates InRelease > Hit:4 http://ftp-stud.hs-esslingen.de/debian buster-backports InRelease > Hit:5 http://download.proxmox.com/debian/ceph-octopus buster InRelease > Hit:6 http://download.proxmox.com/debian/pve buster InRelease > Reading package lists... Done > Building dependency tree > Reading state information... Done > 15 packages can be upgraded. Run 'apt list --upgradable' to see them. > > # aptitude full-upgrade > The following NEW packages will be installed: > libnvpair2linux{ab} libuutil2linux{ab} libzfs3linux{ab} libzpool3linux{ab} > pve-headers-5.4.98-1-pve{a} > pve-kernel-5.4.98-1-pve{a} > The following packages will be REMOVED: > pve-headers-5.4.78-2-pve{u} > The following packages will be upgraded: > libproxmox-backup-qemu0 libpve-common-perl libpve-guest-common-perl libpve-storage-perl > pve-container pve-firmware > pve-headers-5.4 pve-kernel-5.4 pve-kernel-helper pve-manager pve-qemu-kvm spl zfs-initramfs > zfs-zed zfsutils-linux > 15 packages upgraded, 6 newly installed, 1 to remove and 0 not upgraded. > Need to get 0 B/159 MB of archives. After unpacking 296 MB will be used. > The following packages have unmet dependencies: > libzfs3linux : Breaks: libzfs2linux but 0.8.5-pve1 is installed > libzpool3linux : Breaks: libzpool2linux but 0.8.5-pve1 is installed > libnvpair2linux : Breaks: libnvpair1linux but 0.8.5-pve1 is installed > libuutil2linux : Breaks: libuutil1linux but 0.8.5-pve1 is installed > The following actions will resolve these dependencies: > > Keep the following packages at their current version: > 1) libnvpair2linux [Not Installed] > 2) libuutil2linux [Not Installed] > 3) libzfs3linux [Not Installed] > 4) libzpool3linux [Not Installed] > 5) zfs-initramfs [0.8.5-pve1 (now, stable)] > 6) zfs-zed [0.8.5-pve1 (now, stable)] > 7) zfsutils-linux [0.8.5-pve1 (now, stable)] > > > > Accept this solution? [Y/n/q/?] q > Abandoning all efforts to resolve these dependencies. > Abort. > > ################################ > > Looks like I'm not getting the same updates as you do. I'll wait a few days and check back. could you try with `apt` instead of `aptitude` I just tried - and get the same issues when running aptitude (which uses a different resolver for conflicts) - apt is working fine though :) Alternatively you should also be able to upgrade by cycling a few times through aptitudes suggestions. > > Thanks, > > Uwe > > > I hope this helps! > > Regards, > > stoiko > > > > > >> > >> Could someone advice (possible from Proxmox team) advice on how to apply the updates in this > >> particular situation? > >> > >> > >> Regards, > >> > >> Uwe > >> > >> _______________________________________________ > >> pve-user mailing list > >> pve-user at lists.proxmox.com > >> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user > >> > >> > > > > > > From elacunza at binovo.es Wed Feb 24 12:53:23 2021 From: elacunza at binovo.es (Eneko Lacunza) Date: Wed, 24 Feb 2021 12:53:23 +0100 Subject: PBS backup/restore issue In-Reply-To: <748aba12-573f-0bc8-17d7-b94b9b9b8cef@proxmox.com> References: <748aba12-573f-0bc8-17d7-b94b9b9b8cef@proxmox.com> Message-ID: <6120d3f8-67c8-873f-164c-1913c2016a54@binovo.es> Hi Stefan, El 24/2/21 a las 11:34, Stefan Reiter escribi?: > On 2/23/21 8:46 AM, Eneko Lacunza wrote: >> Hi all, >> >> Yesterday I hit a problem with PBS. My sequence of actions was: >> >> 1.a Install debian 10 VM (updated to latest, only system utilities >> and SSH server) >> 1.b Add SSH access key to root >> 1.c Shutdown VM >> 1.d Backup to PBS and to NFS v?a regular backup. >> 1.e Start VM and install various packages - all OK. >> 1.f Stop VM >> >> 2.a Restore VM from PBS backup >> 2.b Start VM and install various packages - apt complains about no >> newline in linux-image-amd64 file manifest, can't install new packages. >> 2.c Stop VM >> >> 3.a Restore VM from NFS regular backup >> 3.b Start VM and install various packages - all OK. >> > > Hi, > > I tried to replicate your problem by following pretty much the exact > steps you mention above, but to no avail. Sorry for not providing enough info to replicate. > > Fresh Debian 10 VM, PBS backup + VMA to local disk, restored both and > both work. I compared the disk images of the restored VMs with md5sum > and the hashes matched, as expected. Tested with the latest package > versions, though using QEMU 5.2.0, but the backup and restore > functionality shouldn't have changed there. > > Could you try to see what the difference is between the restored disks > from the VMA and the PBS backups (if there is any)? I.e. restore them > both to seperate VMIDs and then compare hashes and/or run (v)bindiff > on the disk images (best restore to a directory storage in 'raw' mode) > to see where they differ - of course, this only makes sense if the > backups are created when the VM was shut down. I have restored both to local storage: # ls -lhs ./901/vm-901-disk-0.raw ./900/vm-900-disk-0.raw 2,4G -rw-r----- 1 root root 10G feb 24 12:06 ./900/vm-900-disk-0.raw 2,0G -rw-r----- 1 root root 10G feb 24 12:22 ./901/vm-901-disk-0.raw 900 is from PBS, 901 from vzdump: 900: vm/133/2021-02-22T17:32:23Z (date is shown as 2121-02-22 18:32:23) 901: /mnt/pve/txoritokieta-proxmox/dump/vzdump-qemu-133-2021_02_22-18_33_03.vma.zst sha256 are the same: a3299c08a038a381a7b292a1538c975007f8637f0f3012a319d378090f434ebe ./901/vm-901-disk-0.raw a3299c08a038a381a7b292a1538c975007f8637f0f3012a319d378090f434ebe ./900/vm-900-disk-0.raw I have started vm900 and apt has run fine! "apt install debconf php php-cli rsnapshot apache2 mariadb-server php-mysql acl bzip2 php-xml libapache2-mod-php libssh2-1 mariadb-client gpg" My original restores were to a ceph storage (r2 on HDD with SSD journal/DB). So i have restored again both, to that ceph storage: # rbd export -p proxmox vm-900-disk-0 - | sha256sum Exporting image: 100% complete...done. d05b88dc0f72f9900b2f1cddac021e9f6221eadb2b29b594eb4f3d1db34342e2? - (OOPS!!) # rbd export -p proxmox vm-901-disk-0 - | sha256sum Exporting image: 100% complete...done. a3299c08a038a381a7b292a1538c975007f8637f0f3012a319d378090f434ebe? - (GOOD) > > Potentially something about your NFS/PBS setup is corrupting data? You > mentioned in another mail that they "are the same disks", how is that > configured? From my new data, PBS store seems good which should be quite relaxing for all :-) . PBS node has a mdraid. The volume is used both as NFS export and as PBS store. It seems there's a bad interaction betweeen ceph storage and PBS restore??? That ceph storage has hot data and running VMs, and vzdump restores to it never had given any problem. I restored VM 900 from PBS again but to another Ceph pool, r3 SSD backed: # rbd export -p proxmox_r3_ssd vm-900-disk-0 - | sha256sum Exporting image: 100% complete...done. cb304ca02126d741791922dfbc484e1bdc14556a3c277aa14249cf1fd4773dca? - This is wrong too (and has different SHA256 to the restore to HDD pool). > > Also please post the exact error messages APT is giving you, and your > VM config. # apt install debconf php php-cli rsnapshot apache2 mariadb-server php-mysql acl bzip2 php-xml libapache2-mod-php libssh2-1 mariadb-client gpg Leyendo lista de paquetes... Hecho Creando ?rbol de dependencias Leyendo la informaci?n de estado... Hecho bzip2 ya est? en su versi?n m?s reciente (1.0.6-9.2~deb10u1). fijado bzip2 como instalado manualmente. debconf ya est? en su versi?n m?s reciente (1.5.71). libssh2-1 ya est? en su versi?n m?s reciente (1.8.0-2.1). fijado libssh2-1 como instalado manualmente. Se instalar?n los siguientes paquetes adicionales: ? apache2-bin apache2-data apache2-utils dirmngr galera-3 gawk gnupg gnupg-l10n gnupg-utils gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm libapache2-mod-php7.3 libapr1 libaprutil1 ? libaprutil1-dbd-sqlite3 libaprutil1-ldap libassuan0 libbrotli1 libcgi-fast-perl libcgi-pm-perl libconfig-inifiles-perl libcurl4 libdbd-mysql-perl libdbi-perl libencode-locale-perl libfcgi-perl ? libhtml-parser-perl libhtml-tagset-perl libhtml-template-perl libhttp-date-perl libhttp-message-perl libio-html-perl libjansson4 libksba8 liblchown-perl liblua5.2-0 liblwp-mediatypes-perl libmariadb3 ? libmpfr6 libnpth0 libsigsegv2 libsnappy1v5 libsodium23 libterm-readkey-perl libtimedate-perl liburi-perl libxslt1.1 mariadb-client-10.3 mariadb-client-core-10.3 mariadb-common mariadb-server-10.3 ? mariadb-server-core-10.3 mysql-common php-common php7.3 php7.3-cli php7.3-common php7.3-json php7.3-mysql php7.3-opcache php7.3-readline php7.3-xml pinentry-curses psmisc rsync socat ssl-cert Paquetes sugeridos: ? apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser dbus-user-session pinentry-gnome3 tor gawk-doc parcimonie xloadimage scdaemon php-pear libclone-perl libmldbm-perl libnet-daemon-perl ? libsql-statement-perl libdata-dump-perl libipc-sharedcache-perl libwww-perl mailx mariadb-test netcat-openbsd tinyca pinentry-doc openssl-blacklist Se instalar?n los siguientes paquetes NUEVOS: ? acl apache2 apache2-bin apache2-data apache2-utils dirmngr galera-3 gawk gnupg gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm libapache2-mod-php libapache2-mod-php7.3 ? libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap libassuan0 libbrotli1 libcgi-fast-perl libcgi-pm-perl libconfig-inifiles-perl libcurl4 libdbd-mysql-perl libdbi-perl libencode-locale-perl ? libfcgi-perl libhtml-parser-perl libhtml-tagset-perl libhtml-template-perl libhttp-date-perl libhttp-message-perl libio-html-perl libjansson4 libksba8 liblchown-perl liblua5.2-0 liblwp-mediatypes-perl ? libmariadb3 libmpfr6 libnpth0 libsigsegv2 libsnappy1v5 libsodium23 libterm-readkey-perl libtimedate-perl liburi-perl libxslt1.1 mariadb-client mariadb-client-10.3 mariadb-client-core-10.3 mariadb-common ? mariadb-server mariadb-server-10.3 mariadb-server-core-10.3 mysql-common php php-cli php-common php-mysql php-xml php7.3 php7.3-cli php7.3-common php7.3-json php7.3-mysql php7.3-opcache php7.3-readline ? php7.3-xml pinentry-curses psmisc rsnapshot rsync socat ssl-cert 0 actualizados, 81 nuevos se instalar?n, 0 para eliminar y 0 no actualizados. Se necesita descargar 37,2 MB de archivos. Se utilizar?n 215 MB de espacio de disco adicional despu?s de esta operaci?n. ?Desea continuar? [S/n] Des:1 http://security.debian.org/debian-security buster/updates/main amd64 php7.3-common amd64 7.3.27-1~deb10u1 [960 kB] [... remove dowload logs ...] Des:81 http://deb.debian.org/debian buster/main amd64 ssl-cert all 1.0.39 [20,8 kB] Descargados 37,2 MB en 2s (16,8 MB/s) debconf: Perl may be unconfigured (strict.pm did not return a true value at (eval 1) line 2. BEGIN failed--compilation aborted at (eval 1) line 2. ) -- aborting Seleccionando el paquete libapr1:amd64 previamente no seleccionado. dpkg: error fatal irrecuperable, abortando: ?falta un salto de l?nea nueva al final del fichero de lista de ficheros del paquete `laptop-detect' E: Sub-process /usr/bin/dpkg returned an error code (2) > >> I hit the issue 3 times, and tried creating a new VM and a new Debian >> installation each time. >> >> Any ideas? I have the "broken" PBS backups saved. >> >> VM node: >> # pveversion -v [...] >> >> PBS node: >> # pveversion -v [...] > > FYI, the more useful command for PBS would be: > proxmox-backup-manager versions --verbose > Thanks: # proxmox-backup-manager versions --verbose proxmox-backup???????????? unknown??? running kernel: 5.4.78-2-pve proxmox-backup-server????? 1.0.6-1??? running version: 1.0.6 pve-kernel-5.4???????????? 6.3-5 pve-kernel-helper????????? 6.3-5 pve-kernel-5.3???????????? 6.1-6 pve-kernel-5.4.78-2-pve??? 5.4.78-2 pve-kernel-5.4.65-1-pve??? 5.4.65-1 pve-kernel-5.4.44-2-pve??? 5.4.44-2 pve-kernel-5.3.18-3-pve??? 5.3.18-3 ifupdown2????????????????? unknown libjs-extjs??????????????? 6.0.1-10 proxmox-backup-docs??????? 1.0.6-1 proxmox-backup-client????? 1.0.8-1 proxmox-mini-journalreader 1.1-1 proxmox-widget-toolkit???? 2.4-5 pve-xtermjs??????????????? 4.7.0-3 smartmontools????????????? 7.1-pve2 zfsutils-linux???????????? 2.0.3-pve1 Hope this info helps you to replicate the issue. Cheers -- EnekoLacunza Director T?cnico | Zuzendari teknikoa Binovo IT Human Project 943 569 206 elacunza at binovo.es binovo.es Astigarragako Bidea, 2 - 2 izda. Oficina 10-11, 20180 Oiartzun youtube linkedin From uwe.sauter.de at gmail.com Wed Feb 24 12:56:37 2021 From: uwe.sauter.de at gmail.com (Uwe Sauter) Date: Wed, 24 Feb 2021 12:56:37 +0100 Subject: [PVE-User] =?utf-8?q?Update_to_ZFS_2_available_=E2=80=93_breaks_?= =?utf-8?q?dependencies?= In-Reply-To: <20210224124053.6a2216b7@rosa.proxmox.com> References: <20210224111140.0af49108@rosa.proxmox.com> <806e6159-3ca1-530a-ff98-99ac4a4ced05@gmail.com> <20210224124053.6a2216b7@rosa.proxmox.com> Message-ID: <1407c43c-a547-7e74-4558-1a2a15d27d60@gmail.com> Am 24.02.21 um 12:40 schrieb Stoiko Ivanov: > On Wed, 24 Feb 2021 11:50:03 +0100 > Uwe Sauter wrote: > >> Am 24.02.21 um 11:11 schrieb Stoiko Ivanov: >>> On Wed, 24 Feb 2021 10:56:49 +0100 >>> Uwe Sauter wrote: >>> >>>> Hi all, >>>> >>>> since yesterday, 2021-02-23, updates for ZFS 2.0.3 are available. >>>> Currently my systems have 0.8.5 installed. When trying to update using aptitude I get a popup reading: >>>> >>>> ################### >>>> Some packages were broken and have been fixed: >>>> >>>> Keep the following packages at their current version: >>>> libnvpair2linux [Not Installed] >>>> libuutil2linux [Not Installed] >>>> libzfs3linux [Not Installed] >>>> libzpool3linux [Not Installed] >>>> zfs-initramfs [0.8.5-pve1 (now, stable)] >>>> zfs-zed [0.8.5-pve1 (now, stable)] >>>> zfsutils-linux [0.8.5-pve1 (now, stable)] >>>> ################### >>> With ZFS 2.0.0 most shipped libraries got bumped to new major versions >>> (and new package names) >>> e.g. >>> libnvpair1linux -> libnvpair2linux >>> >>> usually this gets resolved correctly (and worked here on many systems) >>> by running `apt full-upgrade` (instead of `apt upgrade`) >>> >>> It has been a while since I worked with `aptitude` - but did you >>> run `aptitude full-upgrade`? >>> >>> else - does it work if you upgrade using `apt full-upgrade`? >> >> ################################ >> >> # apt update >> Hit:1 http://ftp-stud.hs-esslingen.de/debian buster InRelease >> Hit:2 http://ftp-stud.hs-esslingen.de/debian buster-updates InRelease >> Hit:3 http://ftp-stud.hs-esslingen.de/debian-security buster/updates InRelease >> Hit:4 http://ftp-stud.hs-esslingen.de/debian buster-backports InRelease >> Hit:5 http://download.proxmox.com/debian/ceph-octopus buster InRelease >> Hit:6 http://download.proxmox.com/debian/pve buster InRelease >> Reading package lists... Done >> Building dependency tree >> Reading state information... Done >> 15 packages can be upgraded. Run 'apt list --upgradable' to see them. >> >> # aptitude full-upgrade >> The following NEW packages will be installed: >> libnvpair2linux{ab} libuutil2linux{ab} libzfs3linux{ab} libzpool3linux{ab} >> pve-headers-5.4.98-1-pve{a} >> pve-kernel-5.4.98-1-pve{a} >> The following packages will be REMOVED: >> pve-headers-5.4.78-2-pve{u} >> The following packages will be upgraded: >> libproxmox-backup-qemu0 libpve-common-perl libpve-guest-common-perl libpve-storage-perl >> pve-container pve-firmware >> pve-headers-5.4 pve-kernel-5.4 pve-kernel-helper pve-manager pve-qemu-kvm spl zfs-initramfs >> zfs-zed zfsutils-linux >> 15 packages upgraded, 6 newly installed, 1 to remove and 0 not upgraded. >> Need to get 0 B/159 MB of archives. After unpacking 296 MB will be used. >> The following packages have unmet dependencies: >> libzfs3linux : Breaks: libzfs2linux but 0.8.5-pve1 is installed >> libzpool3linux : Breaks: libzpool2linux but 0.8.5-pve1 is installed >> libnvpair2linux : Breaks: libnvpair1linux but 0.8.5-pve1 is installed >> libuutil2linux : Breaks: libuutil1linux but 0.8.5-pve1 is installed >> The following actions will resolve these dependencies: >> >> Keep the following packages at their current version: >> 1) libnvpair2linux [Not Installed] >> 2) libuutil2linux [Not Installed] >> 3) libzfs3linux [Not Installed] >> 4) libzpool3linux [Not Installed] >> 5) zfs-initramfs [0.8.5-pve1 (now, stable)] >> 6) zfs-zed [0.8.5-pve1 (now, stable)] >> 7) zfsutils-linux [0.8.5-pve1 (now, stable)] >> >> >> >> Accept this solution? [Y/n/q/?] q >> Abandoning all efforts to resolve these dependencies. >> Abort. >> >> ################################ >> >> Looks like I'm not getting the same updates as you do. I'll wait a few days and check back. > could you try with `apt` instead of `aptitude` Looks loke that made the difference. Thanks! > I just tried - and get the same issues when running aptitude (which uses a > different resolver for conflicts) - apt is working fine though :) > > Alternatively you should also be able to upgrade by cycling a few times > through aptitudes suggestions. > > > >> >> Thanks, >> >> Uwe >> >>> I hope this helps! >>> Regards, >>> stoiko >>> >>> >>>> >>>> Could someone advice (possible from Proxmox team) advice on how to apply the updates in this >>>> particular situation? >>>> >>>> >>>> Regards, >>>> >>>> Uwe >>>> >>>> _______________________________________________ >>>> pve-user mailing list >>>> pve-user at lists.proxmox.com >>>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user >>>> >>>> >>> >>> >> >> > > From pfrank at gmx.de Wed Feb 24 17:50:16 2021 From: pfrank at gmx.de (Petric Frank) Date: Wed, 24 Feb 2021 17:50:16 +0100 Subject: [PVE-User] Backup job requires email Message-ID: <3323539.L4jQFfv3LO@main> Hello, wanted to create an scheduled backup job via Datacenter -> Backup. In earlier versions on Proxmox you don't have to give a notification eMail- address. But now you have to enter one in the field "Send email to:". Also the field "Email notification" only gives the options "Always" or "On failure only", but not something like "None". Actually running Proxmox Virtual Environment 6.3-3 Is this a bug in the web-gui ? Is there a command line tool to create the job ? Ok, i can create a cron entry ... kind regards Petric From t.lamprecht at proxmox.com Wed Feb 24 20:03:46 2021 From: t.lamprecht at proxmox.com (Thomas Lamprecht) Date: Wed, 24 Feb 2021 20:03:46 +0100 Subject: [PVE-User] PBS backup/restore issue In-Reply-To: <6120d3f8-67c8-873f-164c-1913c2016a54@binovo.es> References: <748aba12-573f-0bc8-17d7-b94b9b9b8cef@proxmox.com> <6120d3f8-67c8-873f-164c-1913c2016a54@binovo.es> Message-ID: <06a2bbdd-8496-14b0-472d-ce74c59abbcb@proxmox.com> Hi! On 24.02.21 12:53, Eneko Lacunza wrote: > El 24/2/21 a las 11:34, Stefan Reiter escribi?: >> On 2/23/21 8:46 AM, Eneko Lacunza wrote: >>> Yesterday I hit a problem with PBS. My sequence of actions was: >>> >>> 1.a Install debian 10 VM (updated to latest, only system utilities and SSH server) >>> 1.b Add SSH access key to root >>> 1.c Shutdown VM >>> 1.d Backup to PBS and to NFS v?a regular backup. >>> 1.e Start VM and install various packages - all OK. >>> 1.f Stop VM >>> >>> 2.a Restore VM from PBS backup >>> 2.b Start VM and install various packages - apt complains about no newline in linux-image-amd64 file manifest, can't install new packages. >>> 2.c Stop VM >>> >>> 3.a Restore VM from NFS regular backup >>> 3.b Start VM and install various packages - all OK. >>> [snip] > My original restores were to a ceph storage (r2 on HDD with SSD journal/DB). So i have restored again both, to that ceph storage: [snip] > > Hope this info helps you to replicate the issue. FYI: Stefan could replicate it and send a patch[0] which is included now. It boiled down to caches not being flushed to disk for storage attached through user-space libraries (vs. kernel) by QEMU in the PBS restore case. In your case you probably do not have the "KRBD" checkbox ticked for the Ceph storage entry in PVE, so the user-space librbd is used for accessing that storage through QEMU, where this issue then hit on restore. If you want to try the fix out, there's a new QEMU package available on pvetest: http://download.proxmox.com/debian/pve/dists/buster/pvetest/binary-amd64/pve-qemu-kvm_5.2.0-2_amd64.deb sha256sum: 6a086e0b669447f0b8a7899f9194a26785e6b0070f3dec1be4f95067bc707c7a cheers, Thomas [0]: https://lists.proxmox.com/pipermail/pve-devel/2021-February/047131.html From mark at openvs.co.uk Wed Feb 24 20:25:20 2021 From: mark at openvs.co.uk (Mark Adams) Date: Wed, 24 Feb 2021 19:25:20 +0000 Subject: KRBD vs librbd for qemu kvm vms Message-ID: Hi All, I have done some research on this, but I can't find any resources that answer my specific questions - so I am hoping that everyone on the list can help out. In one specific use case, I have a pve 5.4-11 cluster using ceph 12.2.12. The main reason I haven't upgraded this is because of the very manual process to upgrade ceph and this cluster has quite a few osds. Also it will be replaced in the not too distant future. On this version, should I be using KRBD or librbd for my qemu vms? I have read a number of posts/articles that say it performs better as KRBD, but then also see some which say it should only be for containers or bare metal. Additionally if I were to switch (Tick the KRBD box for the specific RBD storage) what is the correct procedure to apply this? Simply select it, then reboot each node? This is an HA cluster, so, are KRBD and librbd interchangeable between hosts in the cluster? can I live migrate the VMs on to another node after I enable KRBD then simply live migrate them back after I reboot before I then do the same for the next node in the cluster? Lastly, is the performance of KRBD really that much better? Is there any downsides to using it vs librbd? I guess I still don't understand why there are 2 options if KRBD performs better? Thanks for any assistance you can all provide! Thanks, Mark From f.ebner at proxmox.com Thu Feb 25 09:01:00 2021 From: f.ebner at proxmox.com (Fabian Ebner) Date: Thu, 25 Feb 2021 09:01:00 +0100 Subject: [PVE-User] Backup job requires email In-Reply-To: <3323539.L4jQFfv3LO@main> References: <3323539.L4jQFfv3LO@main> Message-ID: Hi, yes, this was a regression and should be fixed with libpve-guest-common-perl >= 3.1-5 and libpve-common-perl >= 6.3-4 that are available in the no-subscription repository. Am 24.02.21 um 17:50 schrieb Petric Frank: > Hello, > > wanted to create an scheduled backup job via Datacenter -> Backup. > > In earlier versions on Proxmox you don't have to give a notification eMail- > address. But now you have to enter one in the field "Send email to:". Also the > field "Email notification" only gives the options "Always" or "On failure only", > but not something like "None". > > Actually running Proxmox Virtual Environment 6.3-3 > > Is this a bug in the web-gui ? > > Is there a command line tool to create the job ? > > Ok, i can create a cron entry ... > > kind regards > Petric > > > > > _______________________________________________ > pve-user mailing list > pve-user at lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > From t.lamprecht at proxmox.com Thu Feb 25 09:45:56 2021 From: t.lamprecht at proxmox.com (Thomas Lamprecht) Date: Thu, 25 Feb 2021 09:45:56 +0100 Subject: [PVE-User] KRBD vs librbd for qemu kvm vms In-Reply-To: References: Message-ID: <0d734e31-fd08-237c-4cf3-0604030fed1f@proxmox.com> Hi, On 24.02.21 20:25, Mark Adams wrote: > Hi All, > > I have done some research on this, but I can't find any resources that > answer my specific questions - so I am hoping that everyone on the list can > help out. > > In one specific use case, I have a pve 5.4-11 cluster using ceph 12.2.12. > The main reason I haven't upgraded this is because of the very manual > process to upgrade ceph and this cluster has quite a few osds. Also it will > be replaced in the not too distant future. > > On this version, should I be using KRBD or librbd for my qemu vms? I have > read a number of posts/articles that say it performs better as KRBD, but > then also see some which say it should only be for containers or bare metal. > General Tradeoff: * KRBD: a bit faster, some operations (e.g., snapshots) may see more improvements than others * librbd: matches ceph version in use, so normally its supports all current ceph RBD features, currently I'm actually not to sure if there are any not supported by the kernel, IIRC the 5.4 kernel from PVE 6.x works out fine with all features. NOTE: you talk about PVE 5.4, as it was replaced by 6.x over 1.5 years ago and is EOL since over 6 months I really do not have all version specific pitfalls in mind, so please have that in mind when reading this, I tried hard to remember any issues but at that time span my brain gets untrustworthy to be sure, so I'd highly recommend to test any change before you employ it in production. > Additionally if I were to switch (Tick the KRBD box for the specific RBD > storage) what is the correct procedure to apply this? Simply select it, > then reboot each node? It will the be used for the next VM start or disk hotplug, no reboot necessary. > > This is an HA cluster, so, are KRBD and librbd interchangeable between > hosts in the cluster? can I live migrate the VMs on to another node after I > enable KRBD then simply live migrate them back after I reboot before I then > do the same for the next node in the cluster? You should be able to do so, reboot is actually not required. Enable flag and to a merry round of migrations to get it started up with KRBD. But, this is so easy to test that it would be unwise to not do so bevorehand, setup a test VM, if you don't have on lying around already, and test with that one. > > Lastly, is the performance of KRBD really that much better? Is there any > downsides to using it vs librbd? I guess I still don't understand why there > are 2 options if KRBD performs better? Why don't you compare yourself? Start a modern version of any distro shipping fio and compare a boot with KRBD on and off. Performance is not everything ;-) User-space crashes won't bring down your whole host, for example. And as said the user-space library can adopt new features better, which can include performance relevant features but also such having other advantages. cheers, Thomas From elacunza at binovo.es Thu Feb 25 09:58:54 2021 From: elacunza at binovo.es (Eneko Lacunza) Date: Thu, 25 Feb 2021 09:58:54 +0100 Subject: PBS backup/restore issue In-Reply-To: <06a2bbdd-8496-14b0-472d-ce74c59abbcb@proxmox.com> References: <748aba12-573f-0bc8-17d7-b94b9b9b8cef@proxmox.com> <6120d3f8-67c8-873f-164c-1913c2016a54@binovo.es> <06a2bbdd-8496-14b0-472d-ce74c59abbcb@proxmox.com> Message-ID: <91bde2f6-86f7-08b6-1beb-f7db673b4257@binovo.es> Hi! El 24/2/21 a las 20:03, Thomas Lamprecht escribi?: > On 24.02.21 12:53, Eneko Lacunza wrote: >> El 24/2/21 a las 11:34, Stefan Reiter escribi?: >>> On 2/23/21 8:46 AM, Eneko Lacunza wrote: >>>> Yesterday I hit a problem with PBS. My sequence of actions was: >>>> >>>> 1.a Install debian 10 VM (updated to latest, only system utilities and SSH server) >>>> 1.b Add SSH access key to root >>>> 1.c Shutdown VM >>>> 1.d Backup to PBS and to NFS v?a regular backup. >>>> 1.e Start VM and install various packages - all OK. >>>> 1.f Stop VM >>>> >>>> 2.a Restore VM from PBS backup >>>> 2.b Start VM and install various packages - apt complains about no newline in linux-image-amd64 file manifest, can't install new packages. >>>> 2.c Stop VM >>>> >>>> 3.a Restore VM from NFS regular backup >>>> 3.b Start VM and install various packages - all OK. >>>> > [snip] > >> My original restores were to a ceph storage (r2 on HDD with SSD journal/DB). So i have restored again both, to that ceph storage: > [snip] > >> Hope this info helps you to replicate the issue. > FYI: Stefan could replicate it and send a patch[0] which is included now. > > It boiled down to caches not being flushed to disk for storage attached through > user-space libraries (vs. kernel) by QEMU in the PBS restore case. > > In your case you probably do not have the "KRBD" checkbox ticked for the Ceph > storage entry in PVE, so the user-space librbd is used for accessing that storage > through QEMU, where this issue then hit on restore. > > If you want to try the fix out, there's a new QEMU package available on pvetest: > > http://download.proxmox.com/debian/pve/dists/buster/pvetest/binary-amd64/pve-qemu-kvm_5.2.0-2_amd64.deb > sha256sum: 6a086e0b669447f0b8a7899f9194a26785e6b0070f3dec1be4f95067bc707c7a Yes I saw Stefan's patch on pve-dev (and his CC), I looked to patch it myself until I realized it was on a C source ;) Thanks for publishing to pve-test so fast. I just installed that package and its dependency: http://download.proxmox.com/debian/pve/dists/buster/pvetest/binary-amd64/libproxmox-backup-qemu0_1.0.3-1_amd64.deb on that node, and can confirm IT IS FIXED for me. Tried with both ceph storages, both report correct sha256sum now. Thanks a lot!! -- EnekoLacunza Director T?cnico | Zuzendari teknikoa Binovo IT Human Project 943 569 206 elacunza at binovo.es binovo.es Astigarragako Bidea, 2 - 2 izda. Oficina 10-11, 20180 Oiartzun youtube linkedin From hipantcii at gmail.com Thu Feb 25 13:15:06 2021 From: hipantcii at gmail.com (Ivan Dimitrov) Date: Thu, 25 Feb 2021 13:15:06 +0100 Subject: [PVE-User] Backup notes on NFS share In-Reply-To: <1714294.Zkmt1EvEu4@main> References: <1714294.Zkmt1EvEu4@main> Message-ID: Hi, I have the same configuration and I can apply notes to new backups but I can not edit the notes of the existing backups. The button is greyed out. With Best Regards, Ivan Dimitrov On Thu, 21 Jan 2021 at 15:02, Petric Frank wrote: > Hello, > > i do backups to a mounted NFS share. But when i try to apply a note to the > backup i > get the following error: > > --------------------- cut ----------------------- > volume notes are not supported for PVE::Storage::NFSPlugin at > /usr/share/perl5/PVE/ > Storage/Plugin.pm line 838. (500) > --------------------- cut ----------------------- > > Is that intentional or do i miss something. > > Running current Proxmox GUI 6.3-3 > > kind regards > Petric > > _______________________________________________ > pve-user mailing list > pve-user at lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > From hipantcii at gmail.com Thu Feb 25 13:15:06 2021 From: hipantcii at gmail.com (Ivan Dimitrov) Date: Thu, 25 Feb 2021 13:15:06 +0100 Subject: [PVE-User] Backup notes on NFS share In-Reply-To: <1714294.Zkmt1EvEu4@main> References: <1714294.Zkmt1EvEu4@main> Message-ID: Hi, I have the same configuration and I can apply notes to new backups but I can not edit the notes of the existing backups. The button is greyed out. With Best Regards, Ivan Dimitrov On Thu, 21 Jan 2021 at 15:02, Petric Frank wrote: > Hello, > > i do backups to a mounted NFS share. But when i try to apply a note to the > backup i > get the following error: > > --------------------- cut ----------------------- > volume notes are not supported for PVE::Storage::NFSPlugin at > /usr/share/perl5/PVE/ > Storage/Plugin.pm line 838. (500) > --------------------- cut ----------------------- > > Is that intentional or do i miss something. > > Running current Proxmox GUI 6.3-3 > > kind regards > Petric > > _______________________________________________ > pve-user mailing list > pve-user at lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > From piccardi at truelite.it Thu Feb 25 15:02:57 2021 From: piccardi at truelite.it (Simone Piccardi) Date: Thu, 25 Feb 2021 15:02:57 +0100 Subject: Using PBS on tapes (or external removable disks) Message-ID: <357ff565-dd26-ebb6-da00-e6ce46e2147e@truelite.it> Hi, I'm looking to PBS to do backups replacing both backuppc for files and some custom script for the VM images. But I also need to save the VM images on a removable support (tape of removable disk), there is a way to do this? Must I save the datastore and copy back it to a fresh install? There is an established procedure? Regards Simone -- Simone Piccardi Truelite Srl piccardi at truelite.it (email/jabber) Via Monferrato, 6 Tel. +39-347-1032433 50142 Firenze http://www.truelite.it Tel. +39-055-7879597 From dietmar at proxmox.com Thu Feb 25 15:22:08 2021 From: dietmar at proxmox.com (Dietmar Maurer) Date: Thu, 25 Feb 2021 15:22:08 +0100 (CET) Subject: [PVE-User] Using PBS on tapes (or external removable disks) In-Reply-To: References: Message-ID: <545824224.207.1614262929118@webmail.proxmox.com> We are working on tape support ... > On 02/25/2021 3:02 PM Simone Piccardi via pve-user wrote: > > > _______________________________________________ > pve-user mailing list > pve-user at lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user > Hi, > > I'm looking to PBS to do backups replacing both backuppc for files and > some custom script for the VM images. > > But I also need to save the VM images on a removable support (tape of > removable disk), there is a way to do this? Must I save the datastore > and copy back it to a fresh install? There is an established procedure? > > Regards > Simone > -- > Simone Piccardi Truelite Srl > piccardi at truelite.it (email/jabber) Via Monferrato, 6 > Tel. +39-347-1032433 50142 Firenze > http://www.truelite.it Tel. +39-055-7879597 From lists at merit.unu.edu Thu Feb 25 16:09:49 2021 From: lists at merit.unu.edu (mj) Date: Thu, 25 Feb 2021 16:09:49 +0100 Subject: [PVE-User] Using PBS on tapes (or external removable disks) In-Reply-To: <545824224.207.1614262929118@webmail.proxmox.com> References: <545824224.207.1614262929118@webmail.proxmox.com> Message-ID: <3fa9e744-eac6-39f5-3b36-9a376df32a08@merit.unu.edu> On 2/25/21 3:22 PM, Dietmar Maurer wrote: > We are working on tape support ... > That is GREAT news. We are looking to replace our storix LTO tape backup infra, and we would love replacing it with PBS. Is there any timeframe for tape support..? MJ From dietmar at proxmox.com Thu Feb 25 19:22:59 2021 From: dietmar at proxmox.com (Dietmar Maurer) Date: Thu, 25 Feb 2021 19:22:59 +0100 (CET) Subject: [PVE-User] Using PBS on tapes (or external removable disks) In-Reply-To: <3fa9e744-eac6-39f5-3b36-9a376df32a08@merit.unu.edu> References: <545824224.207.1614262929118@webmail.proxmox.com> <3fa9e744-eac6-39f5-3b36-9a376df32a08@merit.unu.edu> Message-ID: <1383101272.234.1614277380674@webmail.proxmox.com> > On 02/25/2021 4:09 PM mj wrote: > > > On 2/25/21 3:22 PM, Dietmar Maurer wrote: > > We are working on tape support ... > > > > That is GREAT news. We are looking to replace our storix LTO tape backup > infra, and we would love replacing it with PBS. > > Is there any timeframe for tape support..? You can follow the development on git: https://git.proxmox.com/?p=proxmox-backup.git;a=summary From lists at merit.unu.edu Thu Feb 25 20:33:34 2021 From: lists at merit.unu.edu (mj) Date: Thu, 25 Feb 2021 20:33:34 +0100 Subject: [PVE-User] Using PBS on tapes (or external removable disks) In-Reply-To: <1383101272.234.1614277380674@webmail.proxmox.com> References: <545824224.207.1614262929118@webmail.proxmox.com> <3fa9e744-eac6-39f5-3b36-9a376df32a08@merit.unu.edu> <1383101272.234.1614277380674@webmail.proxmox.com> Message-ID: <9a84293d-0708-6238-b3ac-34f1f9171d96@merit.unu.edu> On 2/25/21 7:22 PM, Dietmar Maurer wrote: > > You can follow the development on git: > > https://git.proxmox.com/?p=proxmox-backup.git;a=summary There's a lot of "tape" in there! Good news! Thanks! From aaronjwood at gmail.com Thu Feb 25 21:18:54 2021 From: aaronjwood at gmail.com (Aaron Wood) Date: Thu, 25 Feb 2021 12:18:54 -0800 Subject: [PVE-User] (no subject) Message-ID: Hi all, I just edited a few posts I made last night ( https://forum.proxmox.com/threads/latest-proxmox-update-causes-windows-10-guest-to-bsod-loop-with-kernel-security-check-failure.84867/) and now I see "You have been banned for the following reason: Spam. Please contact the administrator if this was done in error.." Does anyone know how I can contact an admin? Are there any admins watching this list? I was trying to reply to my post with some info for Stefan only to see that the system banned me... Thanks, Aaron From t.lamprecht at proxmox.com Fri Feb 26 12:17:58 2021 From: t.lamprecht at proxmox.com (Thomas Lamprecht) Date: Fri, 26 Feb 2021 12:17:58 +0100 Subject: [PVE-User] (no subject) In-Reply-To: References: Message-ID: <71510a18-b2b0-5d7a-d12b-574a48755616@proxmox.com> Hi! On 25.02.21 21:18, Aaron Wood wrote: > I just edited a few posts I made last night ( > https://forum.proxmox.com/threads/latest-proxmox-update-causes-windows-10-guest-to-bsod-loop-with-kernel-security-check-failure.84867/) > and now I see "You have been banned for the following reason: Spam. Please > contact the administrator if this was done in error.." > > Does anyone know how I can contact an admin? Are there any admins watching > this list? I was trying to reply to my post with some info for Stefan only > to see that the system banned me... Sorry about that ban, it seems the forums ban software got triggered by your post and was mistakenly confirmed as such in the moderation queue, sorry about that inconvenience! Your account should be restored again. In general, if there's any issue with such things you can write directly to office at proxmox.com Best regards, Thomas From pfrank at gmx.de Fri Feb 26 17:47:21 2021 From: pfrank at gmx.de (Petric Frank) Date: Fri, 26 Feb 2021 17:47:21 +0100 Subject: [PVE-User] Backup job requires email In-Reply-To: <3323539.L4jQFfv3LO@main> References: <3323539.L4jQFfv3LO@main> Message-ID: <2414888.kdYZ1jHi8b@main> Hello, followup - if the email parameter is not given to the vzdump command no mail is send. Also no error message about a missing parameter is issued. So in in my opinion this is a bug of the web GUI. Do you agree ? regards Petric Am Mittwoch, 24. Februar 2021, 17:50:16 CET schrieb Petric Frank: > Hello, > > wanted to create an scheduled backup job via Datacenter -> Backup. > > In earlier versions on Proxmox you don't have to give a notification eMail- > address. But now you have to enter one in the field "Send email to:". Also > the field "Email notification" only gives the options "Always" or "On > failure only", but not something like "None". > > Actually running Proxmox Virtual Environment 6.3-3 > > Is this a bug in the web-gui ? > > Is there a command line tool to create the job ? > > Ok, i can create a cron entry ... > > kind regards > Petric > > > > > _______________________________________________ > pve-user mailing list > pve-user at lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user From pfrank at gmx.de Fri Feb 26 18:09:28 2021 From: pfrank at gmx.de (Petric Frank) Date: Fri, 26 Feb 2021 18:09:28 +0100 Subject: [PVE-User] Backup job requires email In-Reply-To: References: <3323539.L4jQFfv3LO@main> Message-ID: <2485001.XQYqjrCR6X@main> Hello, confirmed. Made an upgrade of the system a few minutes ago. Bug fixed. Thanks. regards Petric Am Donnerstag, 25. Februar 2021, 09:01:00 CET schrieb Fabian Ebner: > Hi, > yes, this was a regression and should be fixed with > libpve-guest-common-perl >= 3.1-5 and libpve-common-perl >= 6.3-4 that > are available in the no-subscription repository. > > Am 24.02.21 um 17:50 schrieb Petric Frank: > > Hello, > > > > wanted to create an scheduled backup job via Datacenter -> Backup. > > > > In earlier versions on Proxmox you don't have to give a notification > > eMail- > > address. But now you have to enter one in the field "Send email to:". Also > > the field "Email notification" only gives the options "Always" or "On > > failure only", but not something like "None". > > > > Actually running Proxmox Virtual Environment 6.3-3 > > > > Is this a bug in the web-gui ? > > > > Is there a command line tool to create the job ? > > > > Ok, i can create a cron entry ... > > > > kind regards > > > > Petric > > > > _______________________________________________ > > pve-user mailing list > > pve-user at lists.proxmox.com > > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > _______________________________________________ > pve-user mailing list > pve-user at lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user From nada at verdnatura.es Sun Feb 28 12:07:17 2021 From: nada at verdnatura.es (nada) Date: Sun, 28 Feb 2021 12:07:17 +0100 Subject: [PVE-User] planning CEPH HW Message-ID: <93be9364e748b44c19a20e68c9fca57d@verdnatura.es> hi, we are beginners in CEPH and we have 3 old servers in PROXMOX cluster and just CTs are there my college wants to migrate some VMs from VMWARE cluster to PROXMOX but these VMs are really 'hungry' they need a lot of space for data... we don't want to invest more money in old SAN HP and we want to deploy CEPH so our idea is to buy 2x robust servers for CEPH with HDD 3x light servers for Ceph monitors with SSD and deploy CEPH 1st period approx 50-100 TB 2nd period 300 TB we already have prepared FC 10Gbps backbone Eth 1Gbps backbone we have studied examples of some configurations (Dell, 45Drives,...) we prefer to build up CEPH cluster separated from PROXMOX cluster and link them together to have scalable open source system and avoid vendor-locking BUT we do NOT have experience with CEPH so please, can somebody send us example of HW configuration suitable for our idea ??? Nada From dietmar at proxmox.com Sun Feb 28 13:27:44 2021 From: dietmar at proxmox.com (Dietmar Maurer) Date: Sun, 28 Feb 2021 13:27:44 +0100 (CET) Subject: [PVE-User] planning CEPH HW In-Reply-To: <93be9364e748b44c19a20e68c9fca57d@verdnatura.es> References: <93be9364e748b44c19a20e68c9fca57d@verdnatura.es> Message-ID: <192952717.100.1614515265827@webmail.proxmox.com> > BUT we do NOT have experience with CEPH > so please, > can somebody send us example of HW configuration suitable for our idea > ??? > https://www.proxmox.com/de/downloads/item/proxmox-ve-ceph-benchmark-2020-09 From nada at verdnatura.es Sun Feb 28 18:26:03 2021 From: nada at verdnatura.es (nada) Date: Sun, 28 Feb 2021 18:26:03 +0100 Subject: [PVE-User] planning CEPH HW In-Reply-To: <192952717.100.1614515265827@webmail.proxmox.com> References: <93be9364e748b44c19a20e68c9fca57d@verdnatura.es> <192952717.100.1614515265827@webmail.proxmox.com> Message-ID: <50cd68f3c4a4f2e510d4b0ff8af196da@verdnatura.es> thank you ! Dietmar i will study it Nada On 2021-02-28 13:27, Dietmar Maurer wrote: >> BUT we do NOT have experience with CEPH >> so please, >> can somebody send us example of HW configuration suitable for our idea >> ??? >> > > https://www.proxmox.com/de/downloads/item/proxmox-ve-ceph-benchmark-2020-09 From sfrazier1111 at yahoo.com Sun Feb 28 20:36:33 2021 From: sfrazier1111 at yahoo.com (Steve Frazier) Date: Sun, 28 Feb 2021 19:36:33 +0000 (UTC) Subject: [PVE-User] Adding an Internet NIC to existing Proxmox In-Reply-To: References: <206447580.1166532.1613322212874.ref@mail.yahoo.com> <21687B15-6EE4-4F27-A332-6C2FE4FB86C9@kmi.com> <55407184.1865958.1613508730876@mail.yahoo.com> <0140E02A-133D-47DF-B58A-50F2CA182F1A@kmi.com> Message-ID: <1389273689.437185.1614540993030@mail.yahoo.com> I just purchased a Netgear 48 Port GS348T that will do VLANS. Let's say I want to "trunk" VLAN 20/30 over to port 48 of the Netgear 48 Port switch, I would run a patch cord from what interface (NIC) below to port 48.? Then the (8) VOIP phones I need to do would need to trunk VLAN30 (VOIP) and VLAN20 (Computer) on say ports 1-8 and then I would just set the other ports for VLAN20 only just for VLAN20? I am sorry to get so complicated but this is what I want to do and of course try and understand what I am doing as I do it.? I really appreciate all of your help. I apologize for taking so long to get back to you, I am going to try and understand what you have shown me. This is pretty much way over my head but I am trying to understand. I have (6) Nic ports that I am using presently.? I have lost track of which NICs that I am using for what.? I come from the simple world of ETH0, ETH1, ETH3 and so on. (1) Internet(2) 192.168.20.0/23(3) 192.168.30.0/24 Since you have shown me how to have an Internet NIC and then the two local Nets, my install is geting much more complicated. I have added pfsense and wireguard into the mix throwing addition interfaces into the mix I assume, I will lay them out here: enp11s0f1: flags=4163? mtu 1500? ? ? ? ether 00:15:17:23:f8:2f? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 183025? bytes 45589221 (43.4 MiB)? ? ? ? RX errors 0? dropped 8672? overruns 0? frame 0? ? ? ? TX packets 30779? bytes 10977282 (10.4 MiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0? ? ? ? device interrupt 36? memory 0xfbee0000-fbf00000 enp9s0f0: flags=4163? mtu 1500? ? ? ? ether 00:15:17:d5:fb:8c? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 10240813? bytes 12167243782 (11.3 GiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 4609529? bytes 739192943 (704.9 MiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0? ? ? ? device interrupt 38? memory 0xfbda0000-fbdc0000 enp9s0f1: flags=4163? mtu 1500? ? ? ? ether 00:15:17:d5:fb:8d? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 313683820? bytes 439237907288 (409.0 GiB)? ? ? ? RX errors 0? dropped 36951? overruns 0? frame 0? ? ? ? TX packets 149752924? bytes 21229027987 (19.7 GiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0? ? ? ? device interrupt 40? memory 0xfbde0000-fbe00000 fwbr102i0: flags=4163? mtu 1500? ? ? ? ether 9e:b1:69:b1:fe:e5? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 902886? bytes 135941047 (129.6 MiB)? ? ? ? RX errors 0? dropped 113? overruns 0? frame 0? ? ? ? TX packets 2? bytes 108 (108.0 B)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 fwbr105i0: flags=4163? mtu 1500? ? ? ? ether 12:66:90:e0:af:bb? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 166586? bytes 11268544 (10.7 MiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 2? bytes 108 (108.0 B)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 fwbr105i1: flags=4163? mtu 1500? ? ? ? ether aa:f9:63:e1:e7:43? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 902768? bytes 135930064 (129.6 MiB)? ? ? ? RX errors 0? dropped 111? overruns 0? frame 0? ? ? ? TX packets 2? bytes 108 (108.0 B)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 fwbr113i0: flags=4163? mtu 1500? ? ? ? ether ca:d5:29:71:50:67? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 902578? bytes 135910861 (129.6 MiB)? ? ? ? RX errors 0? dropped 109? overruns 0? frame 0? ? ? ? TX packets 2? bytes 108 (108.0 B)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 fwln102i0: flags=4163? mtu 1500? ? ? ? ether 9e:b1:69:b1:fe:e5? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 1228076? bytes 210823169 (201.0 MiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 334382? bytes 47442555 (45.2 MiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 fwln105i0: flags=4163? mtu 1500? ? ? ? ether 12:66:90:e0:af:bb? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 7060111? bytes 11832514460 (11.0 GiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 4354033? bytes 678995200 (647.5 MiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 fwln105i1: flags=4163? mtu 1500? ? ? ? ether aa:f9:63:e1:e7:43? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 5494888? bytes 873377923 (832.9 MiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 9998182? bytes 11985630904 (11.1 GiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 fwln113i0: flags=4163? mtu 1500? ? ? ? ether ca:d5:29:71:50:67? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 156305554? bytes 428142574732 (398.7 GiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 137592593? bytes 9494602142 (8.8 GiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 fwpr102p0: flags=4163? mtu 1500? ? ? ? ether 9e:d6:11:e5:e9:0e? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 334382? bytes 47442555 (45.2 MiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 1228076? bytes 210823169 (201.0 MiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 fwpr105p0: flags=4163? mtu 1500? ? ? ? ether e2:eb:af:2e:36:3b? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 4354033? bytes 678995200 (647.5 MiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 7060111? bytes 11832514460 (11.0 GiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 fwpr105p1: flags=4163? mtu 1500? ? ? ? ether 32:f1:86:8e:a4:5f? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 9998182? bytes 11985630904 (11.1 GiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 5494888? bytes 873377923 (832.9 MiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 fwpr113p0: flags=4163? mtu 1500? ? ? ? ether 8a:b0:44:57:0c:98? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 137592593? bytes 9494602142 (8.8 GiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 156305554? bytes 428142574732 (398.7 GiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 lo: flags=73? mtu 65536? ? ? ? inet 127.0.0.1? netmask 255.0.0.0? ? ? ? inet6 ::1? prefixlen 128? scopeid 0x10? ? ? ? loop? txqueuelen 1000? (Local Loopback)? ? ? ? RX packets 1146862? bytes 145067314 (138.3 MiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 1146862? bytes 145067314 (138.3 MiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 tap102i0: flags=4419? mtu 1500? ? ? ? ether ee:4b:07:09:21:40? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 334381? bytes 47442501 (45.2 MiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 1228025? bytes 210819199 (201.0 MiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 tap105i0: flags=4419? mtu 1500? ? ? ? ether 76:41:53:c5:f8:17? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 4354032? bytes 678995146 (647.5 MiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 9927205? bytes 12012791415 (11.1 GiB)? ? ? ? TX errors 0? dropped 825 overruns 0? carrier 0? collisions 0 tap105i1: flags=4419? mtu 1500? ? ? ? ether 82:86:71:4b:4d:f2? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 9998181? bytes 11985630850 (11.1 GiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 5573107? bytes 877795901 (837.1 MiB)? ? ? ? TX errors 0? dropped 2604 overruns 0? carrier 0? collisions 0 tap113i0: flags=4419? mtu 1500? ? ? ? ether 42:d0:12:56:ad:37? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 137592592? bytes 9494602088 (8.8 GiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 156304086? bytes 428142336375 (398.7 GiB)? ? ? ? TX errors 0? dropped 1420 overruns 0? carrier 0? collisions 0 vmbr0: flags=4163? mtu 1500? ? ? ? inet 192.168.20.2? netmask 255.255.254.0? broadcast 192.168.21.255? ? ? ? inet6 fe80::215:17ff:fed5:fb8d? prefixlen 64? scopeid 0x20? ? ? ? ether 00:15:17:d5:fb:8d? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 2155012? bytes 1062500000 (1013.2 MiB)? ? ? ? RX errors 0? dropped 115? overruns 0? frame 0? ? ? ? TX packets 1335320? bytes 339766917 (324.0 MiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 vmbr1: flags=4163? mtu 1500? ? ? ? inet6 public? prefixlen 64? scopeid 0x0? ? ? ? inet6 fe80::215:17ff:fed5:fb8c? prefixlen 64? scopeid 0x20? ? ? ? ether 00:15:17:d5:fb:8c? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 166671? bytes 11273818 (10.7 MiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 175? bytes 15690 (15.3 KiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 vmbr2: flags=4163? mtu 1500? ? ? ? inet6 fe80::215:17ff:fe23:f82f? prefixlen 64? scopeid 0x20? ? ? ? ether 00:15:17:23:f8:2f? txqueuelen 1000? (Ethernet)? ? ? ? RX packets 118684? bytes 28211461 (26.9 MiB)? ? ? ? RX errors 0? dropped 0? overruns 0? frame 0? ? ? ? TX packets 62? bytes 4516 (4.4 KiB)? ? ? ? TX errors 0? dropped 0 overruns 0? carrier 0? collisions 0 Thanks again for your time and your help. On Tuesday, February 16, 2021, 05:57:01 PM EST, Steve Frazier via pve-user wrote: Wow, thanks so much!? I will take a look and give it a try.? Thanks Stefan! Steve ? ? On Tuesday, February 16, 2021, 05:40:38 PM EST, Stefan M. Radman wrote:? Here you go. See interface?enp9s0f0 in the config below.It also shows how to run tagged (vmbr20,vmbr30) and untagged (vmbr10) traffic on the same trunk interface. I normally use the VLAN tag ID as the bridge identifier (e.g. vmbr20 and vmbr30).The host IP would typically be on vmbr0 and thus on the native (untagged) VLAN. Stefan /etc/network/interfaces: auto lo iface lo inet loopback iface enp9s0f1 inet manual # Uplink for vmbr0 auto vmbr0 iface vmbr0 inet static ? ? ? ? address 192.168.20.2/23 ? ? ? ? gateway 192.168.20.1 ? ? ? ? bridge_ports enp9s0f1 ? ? ? ? bridge_stp off ? ? ? ? bridge_fd 0 # Host Management iface enp11s0f0 inet manual # Uplink for vmbr1 auto vmbr1 iface vmbr1 inet manual ? ? ? ? bridge_ports enp9s0f0 ? ? ? ? bridge_stp off ? ? ? ? bridge_fd 0 # Bridge for VMs on the Internet iface enp11s0f1 inet manual # Uplink for vmbr2 auto vmbr2 iface vmbr2 inet manual ? ? ? ? bridge_ports enp11s0f1 ? ? ? ? bridge_stp off ? ? ? ? bridge_fd 0 # Bridge for VMs on network 192.168.30.0/24 iface enp9s0f0 inet manual# Trunk interface carrying VLAN20 and VLAN30 auto vmbr10iface?vmbr10?inet manual ? ? ? ? bridge_ports?enp9s0f0 ? ? ? ? bridge_stp off ? ? ? ? bridge_fd 0 # Bridge for untagged traffic on trunk interface auto vmbr20iface vmbr20 inet manual ? ? ? ? bridge_ports?enp9s0f0.20 ? ? ? ? bridge_stp off ? ? ? ? bridge_fd 0 # Bridge for VMs on VLAN20 auto vmbr30iface vmbr30 inet manual ? ? ? ? bridge_ports?enp9s0f0.30 ? ? ? ? bridge_stp off ? ? ? ? bridge_fd 0 # Bridge for VMs on VLAN30 iface enp3s0 inet manual iface enp2s0 inet manual On Feb 16, 2021, at 21:52, Steve Frazier wrote: Tried it and it worked as you explained!? Thanks Stefan.? I am taking this step by step.I haven't tried the 192.168.30.0 nic yet but will shortly. I will want to be able to trunk VLAN 20 and VLAN 30 down a NIC so that I will be able to use my VOIP phones on VLAN30 and be able to offer VLAN20 for the attached work stations, any suggestions and/or configuration file to do this? Thanks again for your help! On Tuesday, February 16, 2021, 01:52:54 PM EST, Stefan M. Radman wrote: Hi Steve Attached and below is an /etc/network/interfaces that should do what you want. Just* keep interface enp9s0f1 connected to your private subnet?192.168.20.2/23* connect interface?enp9s0f0 to your internet subnet* connect internet VMs to vmbr1* connect interface?enp11s0f1 to your private subnet 192.168.30.2/23* connect VMs on subnet 192.168.30.2/23 to vmbr2?If you keep the comment lines (#) they'll show up in the web interface as well. Disclaimer: I did not test it but it should give you the right idea to go ahead. Stefan /etc/network/interfaces: auto loiface lo inet loopback iface enp9s0f1 inet manual# Uplink for vmbr0 auto vmbr0iface vmbr0 inet static? ? ? ? address 192.168.20.2/23? ? ? ? gateway 192.168.20.1? ? ? ? bridge_ports enp9s0f1? ? ? ? bridge_stp off? ? ? ? bridge_fd 0# Host Management iface enp11s0f0 inet manual# Uplink for vmbr1 auto vmbr1iface vmbr1 inet manual? ? ? ? bridge_ports enp9s0f0? ? ? ? bridge_stp off? ? ? ? bridge_fd 0# Bridge for VMs on the Internet iface enp11s0f1 inet manual# Uplink for vmbr2 auto vmbr2iface vmbr2 inet manual? ? ? ? bridge_ports enp11s0f1? ? ? ? bridge_stp off? ? ? ? bridge_fd 0# Bridge for VMs on network 192.168.30.0/24 iface enp9s0f0 inet manual iface enp3s0 inet manual iface enp2s0 inet manual On Feb 14, 2021, at 18:29, Steve Frazier via pve-user wrote: From:Steve Frazier Subject:Re: [PVE-User] Adding an Internet NIC to existing Proxmox Date:February 14, 2021 at 18:29:14 GMT+1 To:"pve-user at lists.proxmox.com" Here is my "interfaces" file: auto loiface lo inet loopback iface enp9s0f1 inet manual auto vmbr0iface vmbr0 inet static? ? ? ? address 192.168.20.2/23? ? ? ? gateway 192.168.20.1? ? ? ? bridge_ports enp9s0f1? ? ? ? bridge_stp off? ? ? ? bridge_fd 0 iface enp11s0f0 inet manual iface enp11s0f1 inet manual iface enp9s0f0 inet manual iface enp3s0 inet manual iface enp2s0 inet manual ???On Sunday, February 14, 2021, 12:14:28 PM EST, Steve Frazier via pve-user wrote: ? I need some help.? I am new to Proxmox.? ?I have it set up now using (1) one NIC to an internal network of 192.168.20.0/23 and I would like to use another NIC I have available for the internet which has (5) available IPs. I have attached a view of my existing network. I would like to set up an additional (NIC) when would allow me to assign to a virtual server that has two virtual NICs with one being the internet and the other being my internal network. While I am at it, I have another networking 192.168.30.0/24 that I would like to assign to an available NIC so that I could assign virtual servers to that NICat a later time. Could someone please help me on how to do this Thanks very much. _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.proxmox.com%2Fcgi-bin%2Fmailman%2Flistinfo%2Fpve-user&data=04%7C01%7Csmr%40kmi.com%7Cfb3db335b1844cd9162208d8d10e1613%7Cc2283768b8d34e008f3d85b1b4f03b33%7C0%7C0%7C637489205732810378%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=bbtAAx1zpomz3hCKtKmrW3UIFF6iyprwmGcer1Q0njQ%3D&reserved=0 CONFIDENTIALITY NOTICE: This communication may contain privileged and confidential information, or may otherwise be protected from disclosure, and is intended solely for use of the intended recipient(s). If you are not the intended recipient of this communication, please notify the sender that you have received this communication in error and delete and destroy all copies in your possession. CONFIDENTIALITY NOTICE:This communication may contain privileged and confidential information, or may otherwise be protected from disclosure, and is intended solely for use of the intended recipient(s). If you are not the intended recipient of this communication, please notify the sender that you have received this communication in error and delete and destroy all copies in your possession. ? _______________________________________________ pve-user mailing list pve-user at lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user