From frederic at fredericrobert.be Fri May 1 23:45:39 2020 From: frederic at fredericrobert.be (Frederic Robert) Date: Fri, 1 May 2020 21:45:39 +0000 Subject: [PVE-User] Proxmox / Debian / Resolv.conf Message-ID: <8fef9091-7e46-778e-223c-cd667e0a183f@fredericrobert.be> Hello, How are you? Is it possible to ask a question in French ? Greetings, -- Fr?d?ric ROBERT From sivakumar.saravanan.jv.ext at valeo-siemens.com Sat May 2 00:33:40 2020 From: sivakumar.saravanan.jv.ext at valeo-siemens.com (Sivakumar SARAVANAN) Date: Sat, 2 May 2020 00:33:40 +0200 Subject: [PVE-User] NAT IP configuration Message-ID: Hello, Could you please let me know the steps to configure the NAT IP for Proxmox VM's. We have configured the private IP con VM network card already. How to configure the NAT IP for the same VM ? Mit freundlichen Gr??en / Best regards / Cordialement, Sivakumar SARAVANAN Externer Dienstleister f?r / External service provider for Valeo Siemens eAutomotive Germany GmbH Research & Development R & D SWENG TE 1 INFTE Frauenauracher Stra?e 85 91056 Erlangen, Germany Tel.: +49 9131 9892 0000 Mobile: +49 176 7698 5441 sivakumar.saravanan.jv.ext at valeo-siemens.com valeo-siemens.com Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger Schwab, Michael Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * From sivakumar.saravanan.jv.ext at valeo-siemens.com Mon May 4 16:43:59 2020 From: sivakumar.saravanan.jv.ext at valeo-siemens.com (Sivakumar SARAVANAN) Date: Mon, 4 May 2020 16:43:59 +0200 Subject: [PVE-User] Not able to start the VM In-Reply-To: References: Message-ID: Hello, Kindly note that, few VM's are going on not reachable frequently. I am not sure. what could be the reason. I am trying to restart the VM. But not able to restart and getting errors always. ( TASK ERROR: start failed: hugepage allocation failed at /usr/share/perl5/PVE/QemuServer/Memory.pm line 541.). But you were saying last time that there were many VMs in the restart process. that was the reason. That's not actually the case. kindly suggest. Mit freundlichen Gr??en / Best regards / Cordialement, Sivakumar SARAVANAN Externer Dienstleister f?r / External service provider for Valeo Siemens eAutomotive Germany GmbH Research & Development R & D SWENG TE 1 INFTE Frauenauracher Stra?e 85 91056 Erlangen, Germany Tel.: +49 9131 9892 0000 Mobile: +49 176 7698 5441 sivakumar.saravanan.jv.ext at valeo-siemens.com valeo-siemens.com Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger Schwab, Michael Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 On Fri, Apr 24, 2020 at 2:32 AM Luis G. Coralle < luiscoralle at fi.uncoma.edu.ar> wrote: > I think you have too much VM started, try to stop some VMS. > > El jue., 23 de abr. de 2020 a la(s) 17:22, Sivakumar SARAVANAN ( > sivakumar.saravanan.jv.ext at valeo-siemens.com) escribi?: > > > Hello Gilberto, > > > > Thank you. > > > > Please find attached the file. > > > > > > Mit freundlichen Gr??en / Best regards / Cordialement, > > > > Sivakumar SARAVANAN > > > > Externer Dienstleister f?r / External service provider for > > Valeo Siemens eAutomotive Germany GmbH > > Research & Development > > R & D SWENG TE 1 INFTE > > Frauenauracher Stra?e 85 > > 91056 Erlangen, Germany > > Tel.: +49 9131 9892 0000 > > Mobile: +49 176 7698 5441 > > sivakumar.saravanan.jv.ext at valeo-siemens.com > > valeo-siemens.com > > > > Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger > > Schwab, Michael > > Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered > > office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 > > > > > > On Thu, Apr 23, 2020 at 9:59 PM Gilberto Nunes < > gilberto.nunes32 at gmail.com > > > > > wrote: > > > > > Using Windows, use WinSCP in order to open a ssh session to the Proxmox > > > server > > > Using Linux, use sftp line interface command, then cd > > /etc/pve/qemu-server, > > > then get VMID.conf, to save to your computer. > > > > > > > > > --- > > > Gilberto Nunes Ferreira > > > > > > > > > Em qui., 23 de abr. de 2020 ?s 16:48, Sivakumar SARAVANAN < > > > sivakumar.saravanan.jv.ext at valeo-siemens.com> escreveu: > > > > > > > Hello, > > > > > > > > Could you let me know the steps to get the file please ? > > > > > > > > > > > > Mit freundlichen Gr??en / Best regards / Cordialement, > > > > > > > > Sivakumar SARAVANAN > > > > > > > > Externer Dienstleister f?r / External service provider for > > > > Valeo Siemens eAutomotive Germany GmbH > > > > Research & Development > > > > R & D SWENG TE 1 INFTE > > > > Frauenauracher Stra?e 85 > > > > 91056 Erlangen, Germany > > > > Tel.: +49 9131 9892 0000 > > > > Mobile: +49 176 7698 5441 > > > > sivakumar.saravanan.jv.ext at valeo-siemens.com > > > > valeo-siemens.com > > > > > > > > Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger > > > > Schwab, Michael > > > > Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; > Registered > > > > office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 > > > > > > > > > > > > On Thu, Apr 23, 2020 at 9:42 PM Gilberto Nunes < > > > gilberto.nunes32 at gmail.com > > > > > > > > > wrote: > > > > > > > > > Please, give us the vm config file, placed in /etc/pve/qemu-server > > > > > > > > > > > > > > > --- > > > > > Gilberto Nunes Ferreira > > > > > > > > > > > > > > > Em qui., 23 de abr. de 2020 ?s 16:38, Sivakumar SARAVANAN < > > > > > sivakumar.saravanan.jv.ext at valeo-siemens.com> escreveu: > > > > > > > > > > > Hello, > > > > > > > > > > > > Not able to start the VM, where it is showing error as below > > > > > > > > > > > > start failed: hugepage allocation failed at > > > > > > /usr/share/perl5/PVE/QemuServer/Memory.pm line 541 > > > > > > > > > > > > > > > > > > Mit freundlichen Gr??en / Best regards / Cordialement, > > > > > > > > > > > > Sivakumar SARAVANAN > > > > > > > > > > > > Externer Dienstleister f?r / External service provider for > > > > > > Valeo Siemens eAutomotive Germany GmbH > > > > > > Research & Development > > > > > > R & D SWENG TE 1 INFTE > > > > > > Frauenauracher Stra?e 85 > > > > > > 91056 Erlangen, Germany > > > > > > Tel.: +49 9131 9892 0000 > > > > > > Mobile: +49 176 7698 5441 > > > > > > sivakumar.saravanan.jv.ext at valeo-siemens.com > > > > > > valeo-siemens.com > > > > > > > > > > > > Valeo Siemens eAutomotive Germany GmbH: Managing Directors: > Holger > > > > > > Schwab, Michael > > > > > > Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; > > > Registered > > > > > > office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 > > > > > > > > > > > > -- > > > > > > *This e-mail message is intended for the internal use of the > > intended > > > > > > recipient(s) only. > > > > > > The information contained herein is > > > > > > confidential/privileged. Its disclosure or reproduction is > strictly > > > > > > prohibited. > > > > > > If you are not the intended recipient, please inform the sender > > > > > > immediately, do not disclose it internally or to third parties > and > > > > > destroy > > > > > > it. > > > > > > > > > > > > In the course of our business relationship and for business > > purposes > > > > > > only, Valeo may need to process some of your personal data. > > > > > > For more > > > > > > information, please refer to the Valeo Data Protection Statement > > and > > > > > > Privacy notice available on Valeo.com > > > > > > * > > > > > > _______________________________________________ > > > > > > pve-user mailing list > > > > > > pve-user at pve.proxmox.com > > > > > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > > > > > > _______________________________________________ > > > > > pve-user mailing list > > > > > pve-user at pve.proxmox.com > > > > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > > > > > > > > -- > > > > *This e-mail message is intended for the internal use of the intended > > > > recipient(s) only. > > > > The information contained herein is > > > > confidential/privileged. Its disclosure or reproduction is strictly > > > > prohibited. > > > > If you are not the intended recipient, please inform the sender > > > > immediately, do not disclose it internally or to third parties and > > > destroy > > > > it. > > > > > > > > In the course of our business relationship and for business purposes > > > > only, Valeo may need to process some of your personal data. > > > > For more > > > > information, please refer to the Valeo Data Protection Statement and > > > > Privacy notice available on Valeo.com > > > > * > > > > _______________________________________________ > > > > pve-user mailing list > > > > pve-user at pve.proxmox.com > > > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > > _______________________________________________ > > > pve-user mailing list > > > pve-user at pve.proxmox.com > > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > > -- > > *This e-mail message is intended for the internal use of the intended > > recipient(s) only. > > The information contained herein is > > confidential/privileged. Its disclosure or reproduction is strictly > > prohibited. > > If you are not the intended recipient, please inform the sender > > immediately, do not disclose it internally or to third parties and > destroy > > it. > > > > In the course of our business relationship and for business purposes > > only, Valeo may need to process some of your personal data. > > For more > > information, please refer to the Valeo Data Protection Statement and > > Privacy notice available on Valeo.com > > * > > _______________________________________________ > > pve-user mailing list > > pve-user at pve.proxmox.com > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > -- > Luis G. Coralle > Secretar?a de TIC > Facultad de Inform?tica > Universidad Nacional del Comahue > (+54) 299-4490300 Int 647 > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * From mientilucci at datto.com Thu May 7 17:38:12 2020 From: mientilucci at datto.com (Marc Ientilucci) Date: Thu, 7 May 2020 11:38:12 -0400 Subject: permissions: allowing users to create their own VMs but do not let other users see the VMs Message-ID: Hello, I'd like to have individual users who can clone VMs. These cloned VMs should only show up for user who cloned it and not anyone else. Is there any way to do this without having to go in and add NoAccess rules to a new VM every time? User 1 would only see VMs created and cloned by user 1, nothing from user 2. User 2 would only see VMs created and cloned by user 2, nothing from user 1. Thanks! -- From harrim4n at harrim4n.com Thu May 7 18:15:06 2020 From: harrim4n at harrim4n.com (harrim4n) Date: Thu, 7 May 2020 18:15:06 +0200 Subject: [PVE-User] permissions: allowing users to create their own VMs but do not let other users see the VMs In-Reply-To: References: Message-ID: <0b5fe0a6-9ace-2964-44de-f94d662b5459@harrim4n.com> Hi Marc, not sure how to accomplish that on a VM level. However, you could create a pool for each user and only grant them the appropriate permission. Then, they can clone the VM into that pool, which would only be visible to them. Hope that helps! On 07.05.20 17:38, Marc Ientilucci via pve-user wrote: > Hello, > > I'd like to have individual users who can clone VMs. These cloned VMs > should only show up for user who cloned it and not anyone else. Is there > any way to do this without having to go in and add NoAccess rules to a new > VM every time? > > User 1 would only see VMs created and cloned by user 1, nothing from user > 2. > User 2 would only see VMs created and cloned by user 2, nothing from user > 1. > > Thanks! From herve.ballans at ias.u-psud.fr Thu May 7 18:28:57 2020 From: herve.ballans at ias.u-psud.fr (Herve Ballans) Date: Thu, 7 May 2020 18:28:57 +0200 Subject: [PVE-User] critical HA problem on a PVE6 cluster Message-ID: <24544520-a537-af1d-b276-6fb56c331278@ias.u-psud.fr> Hi all, *Cluster info:* * 5 nodes (version PVE 6.1-3 at the time the problem occured) * Ceph rbd storage (Nautilus) * In production since many years with no major issues * No specific network problems at the time the problem occured * Nodes are on the same date (configured with the same ntp server) *Symptoms:* Suddenly, last night (around 7 PM), all nodes of our cluster seems to have rebooted in the same time with no apparent reasons (I mean, we weren't doing antything on it) ! During the reboot, services "Corosync Cluster Engine" and "Proxmox VE replication runer" failed. After node rebooted, we are obliged to start those services manually. Once rebooted with all pve services, some nodes were in HA lrm status : old timestamp - dead? while others were in active status or in wait_for_agent_lock status ?... Nodes switch states regularly...and it loops back and forth as long as we don't change the configuration... In the same time, pve-ha-crm service got unexpected error, as for example : "Configuration file 'nodes/inf-proxmox6/qemu-server/501.conf' does not exist" even though the file exists but on an another node ! Such message is probably a consequence of the fencing between nodes due to the change of status... *What we have tried until now to stabilize the situation:* After several investigations and several operations that have failed to solve anything (in particular a complete upgrade to the latest PVE version 6.1-11), we finally removed the HA configuration of all the VM. Since, the state seems to be stabilized although, obviously, it is not nominal ! Now, all the nodes are in HA lrm status : idle and sometimes switch to old timestamp - dead? state, then come back to idle state. None of them are in "active" state. Obviously, quorum status is "no quorum" It will be noted that, as soon as we try to re-activate the HA status on the VMs, problem occurs again (nodes reboot!) :( *Question:* Have you ever experienced such a problem or do you know a way to restore a correct HA configuration in this case ? I point out that nodes are currently on version PVE 6.1-11. I can put some specific logs if useful. Thanks in advance for your help, Herv? From info at aminvakil.com Sat May 9 01:17:10 2020 From: info at aminvakil.com (Amin Vakil) Date: Sat, 9 May 2020 03:47:10 +0430 Subject: [PVE-User] Guest-agent fs-freeze command breaks the system Message-ID: <989cd1ac-0011-93ec-6db3-533aba86ed65@aminvakil.com> Tonight one of vms which I backup weekly broke when proxmox issues guest-agent 'fs-freeze' command. I could ping the server, but nothing works (almost every port were down, the remaining didn't respond to anything and I couldn't ssh to it). Although I haven't faced this issue in three years using proxmox and last three month enabling and installing qemu-guest-agent on host and vm, I waited for 20 minutes and nothing happens, then I stopped the backup, logged in to host and entered qm unlock 201 && qm reset 201 (201 is VMID of broken vm) Then when vm gets booted I edited grub command line and add this to the end (just to be sure filesystem hasn't corrupted): fsck.mode=force fsck.repair=yes When the server was booting up lots of errors were getting fixed, errors were like this: [ 170.013464] systemd-fsck[639]: Free blocks count wrong for group #2160 (23153, counted=23948). [ 170.014860] systemd-fsck[639]: Fix? yes and this [ 170.142921] systemd-fsck[639]: Directories count wrong for group #2000 (1394, counted=871). [ 170.145106] systemd-fsck[639]: Fix? yes Then the server boots up and everything was fine. I ran a manual backup again and the same problem happens, everything freezes when backup reaches guest-agent 'fs-freeze' command and I couldn't do anything expect unlock and forcefully reset the vm. Server is updated CentOS 7 (2003) with cPanel installed on it. I don't know if this helps or not, but the vm has three hard disks, only backup of the first one which OS is installed on it, is enabled. First hard disk is in "sas" directory and backup gets written to another directory "sata". Also log of first scheduled backup is here (manual backup error log is exactly the same): INFO: Starting Backup of VM 201 (qemu) INFO: Backup started at 2020-05-09 02:00:58 INFO: status = running INFO: VM Name: ircpanel1 INFO: include disk 'virtio0' 'sas:201/vm-201-disk-0.qcow2' 80G INFO: exclude disk 'virtio1' 'sata:201/vm-201-disk-0.qcow2' (backup=no) INFO: exclude disk 'virtio2' 'sata:201/vm-201-disk-1.qcow2' (backup=no) INFO: backup mode: snapshot INFO: ionice priority: 7 INFO: creating archive '/mnt/pve/sata/dump/vzdump-qemu-201-2020_05_09-02_00_58.vma.lzo' INFO: issuing guest-agent 'fs-freeze' command closing with read buffer at /usr/share/perl5/IO/Multiplex.pm line 927. ERROR: interrupted by signal INFO: issuing guest-agent 'fs-thaw' command TASK ERROR: got unexpected control message: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From dietmar at proxmox.com Sat May 9 07:50:30 2020 From: dietmar at proxmox.com (Dietmar Maurer) Date: Sat, 9 May 2020 07:50:30 +0200 (CEST) Subject: [PVE-User] permissions: allowing users to create their own VMs but do not let other users see the VMs In-Reply-To: References: Message-ID: <2076597526.0.1589003430731@webmail.proxmox.com> You need to create a pool for each such user, and give them permissions to create an use VM on that pool only. see https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_user_management section: 13.8.5. Pools > I'd like to have individual users who can clone VMs. These cloned VMs > should only show up for user who cloned it and not anyone else. Is there > any way to do this without having to go in and add NoAccess rules to a new > VM every time? > > User 1 would only see VMs created and cloned by user 1, nothing from user > 2. > User 2 would only see VMs created and cloned by user 2, nothing from user > 1. > > Thanks! > -- From herve.ballans at ias.u-psud.fr Mon May 11 10:35:19 2020 From: herve.ballans at ias.u-psud.fr (Herve Ballans) Date: Mon, 11 May 2020 10:35:19 +0200 Subject: [PVE-User] critical HA problem on a PVE6 cluster In-Reply-To: <24544520-a537-af1d-b276-6fb56c331278@ias.u-psud.fr> References: <24544520-a537-af1d-b276-6fb56c331278@ias.u-psud.fr> Message-ID: Hi everybody, I would like to take the opportunity at the beginning of this new week to ask my issue again. Has anyone had any idea why a such problem occurred, or is this problem really something new ? Thanks again, Herv? On 07/05/2020 18:28, Herve Ballans wrote: > Hi all, > > *Cluster info:* > > ?* 5 nodes (version PVE 6.1-3 at the time the problem occured) > ?* Ceph rbd storage (Nautilus) > ?* In production since many years with no major issues > ?* No specific network problems at the time the problem occured > ?* Nodes are on the same date (configured with the same ntp server) > > *Symptoms:* > > Suddenly, last night (around 7 PM), all nodes of our cluster seems to > have rebooted in the same time with no apparent reasons (I mean, we > weren't doing antything on it) ! > During the reboot, services "Corosync Cluster Engine" and "Proxmox VE > replication runer" failed. After node rebooted, we are obliged to > start those services manually. > > Once rebooted with all pve services, some nodes were in HA lrm status > : old timestamp - dead? while others were in active status or in > wait_for_agent_lock status ?... > Nodes switch states regularly...and it loops back and forth as long as > we don't change the configuration... > > In the same time, pve-ha-crm service got unexpected error, as for > example : "Configuration file > 'nodes/inf-proxmox6/qemu-server/501.conf' does not exist" even though > the file exists but on an another node ! > Such message is probably a consequence of the fencing between nodes > due to the change of status... > > *What we have tried until now to stabilize the situation:* > > After several investigations and several operations that have failed > to solve anything (in particular a complete upgrade to the latest PVE > version 6.1-11), > > we finally removed the HA configuration of all the VM. > Since, the state seems to be stabilized although, obviously, it is not > nominal ! > > Now, all the nodes are in HA lrm status : idle and sometimes switch to > old timestamp - dead? state, then come back to idle state. > None of them are in "active" state. > Obviously, quorum status is "no quorum" > > It will be noted that, as soon as we try to re-activate the HA status > on the VMs, problem occurs again (nodes reboot!) :( > > *Question:* > > Have you ever experienced such a problem or do you know a way to > restore a correct HA configuration in this case ? > I point out that nodes are currently on version PVE 6.1-11. > > I can put some specific logs if useful. > > Thanks in advance for your help, > Herv? > > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user From elacunza at binovo.es Mon May 11 10:39:10 2020 From: elacunza at binovo.es (Eneko Lacunza) Date: Mon, 11 May 2020 10:39:10 +0200 Subject: [PVE-User] critical HA problem on a PVE6 cluster In-Reply-To: References: <24544520-a537-af1d-b276-6fb56c331278@ias.u-psud.fr> Message-ID: Hi Herv?, This seems a network issue. What is the network setup in this cluster? What logs in syslog about corosync and pve-cluster? Don't enable HA until you have a stable cluster quorum. Cheers Eneko El 11/5/20 a las 10:35, Herve Ballans escribi?: > Hi everybody, > > I would like to take the opportunity at the beginning of this new week > to ask my issue again. > > Has anyone had any idea why a such problem occurred, or is this > problem really something new ? > > Thanks again, > Herv? > > On 07/05/2020 18:28, Herve Ballans wrote: >> Hi all, >> >> *Cluster info:* >> >> ?* 5 nodes (version PVE 6.1-3 at the time the problem occured) >> ?* Ceph rbd storage (Nautilus) >> ?* In production since many years with no major issues >> ?* No specific network problems at the time the problem occured >> ?* Nodes are on the same date (configured with the same ntp server) >> >> *Symptoms:* >> >> Suddenly, last night (around 7 PM), all nodes of our cluster seems to >> have rebooted in the same time with no apparent reasons (I mean, we >> weren't doing antything on it) ! >> During the reboot, services "Corosync Cluster Engine" and "Proxmox VE >> replication runer" failed. After node rebooted, we are obliged to >> start those services manually. >> >> Once rebooted with all pve services, some nodes were in HA lrm status >> : old timestamp - dead? while others were in active status or in >> wait_for_agent_lock status ?... >> Nodes switch states regularly...and it loops back and forth as long >> as we don't change the configuration... >> >> In the same time, pve-ha-crm service got unexpected error, as for >> example : "Configuration file >> 'nodes/inf-proxmox6/qemu-server/501.conf' does not exist" even though >> the file exists but on an another node ! >> Such message is probably a consequence of the fencing between nodes >> due to the change of status... >> >> *What we have tried until now to stabilize the situation:* >> >> After several investigations and several operations that have failed >> to solve anything (in particular a complete upgrade to the latest PVE >> version 6.1-11), >> >> we finally removed the HA configuration of all the VM. >> Since, the state seems to be stabilized although, obviously, it is >> not nominal ! >> >> Now, all the nodes are in HA lrm status : idle and sometimes switch >> to old timestamp - dead? state, then come back to idle state. >> None of them are in "active" state. >> Obviously, quorum status is "no quorum" >> >> It will be noted that, as soon as we try to re-activate the HA status >> on the VMs, problem occurs again (nodes reboot!) :( >> >> *Question:* >> >> Have you ever experienced such a problem or do you know a way to >> restore a correct HA configuration in this case ? >> I point out that nodes are currently on version PVE 6.1-11. >> >> I can put some specific logs if useful. >> >> Thanks in advance for your help, >> Herv? >> >> _______________________________________________ >> pve-user mailing list >> pve-user at pve.proxmox.com >> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user -- Zuzendari Teknikoa / Director T?cnico Binovo IT Human Project, S.L. Telf. 943569206 Astigarragako bidea 2, 2? izq. oficina 11; 20180 Oiartzun (Gipuzkoa) www.binovo.es From leandro at tecnetmza.com.ar Mon May 11 15:56:00 2020 From: leandro at tecnetmza.com.ar (Leandro Roggerone) Date: Mon, 11 May 2020 10:56:00 -0300 Subject: [PVE-User] pull docker container into proxmox. Message-ID: Hi guys , im new to container tech , it seems very useful. I wonder if can download an image from docker hub and install and manage from proxmox panel like container templates. Any idea ? Regards, Leandro. From herve.ballans at ias.u-psud.fr Mon May 11 17:58:21 2020 From: herve.ballans at ias.u-psud.fr (Herve Ballans) Date: Mon, 11 May 2020 17:58:21 +0200 Subject: [PVE-User] critical HA problem on a PVE6 cluster In-Reply-To: References: <24544520-a537-af1d-b276-6fb56c331278@ias.u-psud.fr> Message-ID: Hi Eneko, Thanks for your answer. I was also thinking at first a network issue but physical network equipments don't seem to be showing any specific problems...Here are more details on the cluster: 2x10Gb + 2x1Gb interface: * a 10Gb interface for ceph cluster * a 10Gb interface for main network cluster * the other 2 1Gb interfaces are used for two other VLAN for the VMs On 11/05/2020 10:39, Eneko Lacunza wrote: > Hi Herv?, > > This seems a network issue. What is the network setup in this cluster? > What logs in syslog about corosync and pve-cluster? > > Don't enable HA until you have a stable cluster quorum. > > Cheers > Eneko > > El 11/5/20 a las 10:35, Herve Ballans escribi?: >> Hi everybody, >> >> I would like to take the opportunity at the beginning of this new >> week to ask my issue again. >> >> Has anyone had any idea why a such problem occurred, or is this >> problem really something new ? >> >> Thanks again, >> Herv? >> >> On 07/05/2020 18:28, Herve Ballans wrote: >>> Hi all, >>> >>> *Cluster info:* >>> >>> ?* 5 nodes (version PVE 6.1-3 at the time the problem occured) >>> ?* Ceph rbd storage (Nautilus) >>> ?* In production since many years with no major issues >>> ?* No specific network problems at the time the problem occured >>> ?* Nodes are on the same date (configured with the same ntp server) >>> >>> *Symptoms:* >>> >>> Suddenly, last night (around 7 PM), all nodes of our cluster seems >>> to have rebooted in the same time with no apparent reasons (I mean, >>> we weren't doing antything on it) ! >>> During the reboot, services "Corosync Cluster Engine" and "Proxmox >>> VE replication runer" failed. After node rebooted, we are obliged to >>> start those services manually. >>> >>> Once rebooted with all pve services, some nodes were in HA lrm >>> status : old timestamp - dead? while others were in active status or >>> in wait_for_agent_lock status ?... >>> Nodes switch states regularly...and it loops back and forth as long >>> as we don't change the configuration... >>> >>> In the same time, pve-ha-crm service got unexpected error, as for >>> example : "Configuration file >>> 'nodes/inf-proxmox6/qemu-server/501.conf' does not exist" even >>> though the file exists but on an another node ! >>> Such message is probably a consequence of the fencing between nodes >>> due to the change of status... >>> >>> *What we have tried until now to stabilize the situation:* >>> >>> After several investigations and several operations that have failed >>> to solve anything (in particular a complete upgrade to the latest >>> PVE version 6.1-11), >>> >>> we finally removed the HA configuration of all the VM. >>> Since, the state seems to be stabilized although, obviously, it is >>> not nominal ! >>> >>> Now, all the nodes are in HA lrm status : idle and sometimes switch >>> to old timestamp - dead? state, then come back to idle state. >>> None of them are in "active" state. >>> Obviously, quorum status is "no quorum" >>> >>> It will be noted that, as soon as we try to re-activate the HA >>> status on the VMs, problem occurs again (nodes reboot!) :( >>> >>> *Question:* >>> >>> Have you ever experienced such a problem or do you know a way to >>> restore a correct HA configuration in this case ? >>> I point out that nodes are currently on version PVE 6.1-11. >>> >>> I can put some specific logs if useful. >>> >>> Thanks in advance for your help, >>> Herv? >>> >>> _______________________________________________ >>> pve-user mailing list >>> pve-user at pve.proxmox.com >>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >> _______________________________________________ >> pve-user mailing list >> pve-user at pve.proxmox.com >> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > From herve.ballans at ias.u-psud.fr Mon May 11 18:24:32 2020 From: herve.ballans at ias.u-psud.fr (Herve Ballans) Date: Mon, 11 May 2020 18:24:32 +0200 Subject: [PVE-User] critical HA problem on a PVE6 cluster In-Reply-To: References: <24544520-a537-af1d-b276-6fb56c331278@ias.u-psud.fr> Message-ID: <755b9f34-9f97-07a5-676e-3bf7d4fd9224@ias.u-psud.fr> Sorry my mail was sent too quickly. It misses some required logs. regarding syslog, some extracts here for corosync (the following log block will be repeated in a loop, but with different numbers): May? 6 18:38:02 inf-proxmox6 corosync[2674]:?? [KNET? ] link: host: 4 link: 0 is down May? 6 18:38:02 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 4 (passive) best link: 0 (pri: 1) May? 6 18:38:02 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 4 has no active links May? 6 18:38:05 inf-proxmox6 corosync[2674]:?? [KNET? ] rx: host: 4 link: 0 is up May? 6 18:38:05 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 4 (passive) best link: 0 (pri: 1) May? 6 18:38:10 inf-proxmox6 corosync[2674]:?? [KNET? ] link: host: 3 link: 0 is down May? 6 18:38:10 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 3 (passive) best link: 0 (pri: 1) May? 6 18:38:10 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 3 has no active links May? 6 18:38:12 inf-proxmox6 corosync[2674]:?? [KNET? ] rx: host: 3 link: 0 is up May? 6 18:38:12 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 3 (passive) best link: 0 (pri: 1) May? 6 18:38:12 inf-proxmox6 corosync[2674]:?? [TOTEM ] Retransmit List: 64 May? 6 18:38:18 inf-proxmox6 corosync[2674]:?? [KNET? ] link: host: 4 link: 0 is down May? 6 18:38:18 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 4 (passive) best link: 0 (pri: 1) May? 6 18:38:18 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 4 has no active links May? 6 18:38:19 inf-proxmox6 corosync[2674]:?? [KNET? ] link: host: 3 link: 0 is down May? 6 18:38:19 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 3 (passive) best link: 0 (pri: 1) May? 6 18:38:19 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 3 has no active links May? 6 18:38:20 inf-proxmox6 corosync[2674]:?? [KNET? ] rx: host: 4 link: 0 is up May? 6 18:38:20 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 4 (passive) best link: 0 (pri: 1) May? 6 18:38:21 inf-proxmox6 corosync[2674]:?? [KNET? ] rx: host: 3 link: 0 is up May? 6 18:38:21 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 3 (passive) best link: 0 (pri: 1) May? 6 18:38:29 inf-proxmox6 corosync[2674]:?? [KNET? ] link: host: 3 link: 0 is down May? 6 18:38:29 inf-proxmox6 corosync[2674]:?? [KNET? ] link: host: 4 link: 0 is down May? 6 18:38:29 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 3 (passive) best link: 0 (pri: 1) May? 6 18:38:29 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 3 has no active links May? 6 18:38:29 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 4 (passive) best link: 0 (pri: 1) May? 6 18:38:29 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 4 has no active links May? 6 18:38:31 inf-proxmox6 corosync[2674]:?? [TOTEM ] Token has not been received in 107 ms May? 6 18:38:31 inf-proxmox6 corosync[2674]:?? [KNET? ] rx: host: 3 link: 0 is up May? 6 18:38:31 inf-proxmox6 corosync[2674]:?? [KNET? ] rx: host: 4 link: 0 is up May? 6 18:38:31 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 3 (passive) best link: 0 (pri: 1) May? 6 18:38:31 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 4 (passive) best link: 0 (pri: 1) May? 6 18:38:42 inf-proxmox6 corosync[2674]:?? [TOTEM ] Retransmit List: fd May? 6 18:38:42 inf-proxmox6 corosync[2674]:?? [TOTEM ] Retransmit List: 100 May? 6 18:38:42 inf-proxmox6 corosync[2674]:?? [TOTEM ] Retransmit List: 101 May? 6 18:38:42 inf-proxmox6 corosync[2674]:?? [TOTEM ] Retransmit List: 102 May? 6 18:38:42 inf-proxmox6 corosync[2674]:?? [TOTEM ] Retransmit List: 103 May? 6 18:38:42 inf-proxmox6 corosync[2674]:?? [TOTEM ] Retransmit List: 104 May? 6 18:38:42 inf-proxmox6 corosync[2674]:?? [TOTEM ] Retransmit List: 106 May? 6 18:38:42 inf-proxmox6 corosync[2674]:?? [TOTEM ] Retransmit List: 107 May? 6 18:38:42 inf-proxmox6 corosync[2674]:?? [TOTEM ] Retransmit List: 108 May? 6 18:38:42 inf-proxmox6 corosync[2674]:?? [TOTEM ] Retransmit List: 109 May? 6 18:38:44 inf-proxmox6 corosync[2674]:?? [KNET? ] link: host: 3 link: 0 is down May? 6 18:38:44 inf-proxmox6 corosync[2674]:?? [KNET? ] link: host: 4 link: 0 is down May? 6 18:38:44 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 3 (passive) best link: 0 (pri: 1) May? 6 18:38:44 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 3 has no active links May? 6 18:38:44 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 4 (passive) best link: 0 (pri: 1) May? 6 18:38:44 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 4 has no active links May? 6 18:38:46 inf-proxmox6 corosync[2674]:?? [TOTEM ] Token has not been received in 106 ms May? 6 18:38:46 inf-proxmox6 corosync[2674]:?? [KNET? ] rx: host: 4 link: 0 is up May? 6 18:38:46 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 4 (passive) best link: 0 (pri: 1) May? 6 18:38:47 inf-proxmox6 corosync[2674]:?? [KNET? ] rx: host: 3 link: 0 is up May? 6 18:38:47 inf-proxmox6 corosync[2674]:?? [KNET? ] host: host: 3 (passive) best link: 0 (pri: 1) May? 6 18:38:51 inf-proxmox6 corosync[2674]:?? [TOTEM ] Token has not been received in 4511 ms May? 6 18:38:52 inf-proxmox6 corosync[2674]:?? [TOTEM ] A new membership (1.ea8) was formed. Members May? 6 18:38:52 inf-proxmox6 corosync[2674]:?? [CPG?? ] downlist left_list: 0 received May? 6 18:38:52 inf-proxmox6 corosync[2674]:?? [CPG?? ] downlist left_list: 0 received May? 6 18:38:52 inf-proxmox6 corosync[2674]:?? [CPG?? ] downlist left_list: 0 received May? 6 18:38:52 inf-proxmox6 corosync[2674]:?? [CPG?? ] downlist left_list: 0 received May? 6 18:38:52 inf-proxmox6 corosync[2674]:?? [QUORUM] Members[4]: 1 3 4 5 May? 6 18:38:52 inf-proxmox6 corosync[2674]:?? [MAIN? ] Completed service synchronization, ready to provide service. Nothing really relevant regarding the pve-cluster in the logs as it marks succeed ?..for instance here: May? 6 22:17:33 inf-proxmox6 systemd[1]: Stopping The Proxmox VE cluster filesystem... May? 6 22:17:33 inf-proxmox6 pmxcfs[2561]: [main] notice: teardown filesystem May? 6 22:17:33 inf-proxmox6 pvestatd[2906]: status update time (19.854 seconds) May? 6 22:17:34 inf-proxmox6 systemd[7888]: etc-pve.mount: Succeeded. May? 6 22:17:34 inf-proxmox6 systemd[1]: etc-pve.mount: Succeeded. May? 6 22:17:34 inf-proxmox6 pvestatd[2906]: rados_connect failed - Operation not supported May? 6 22:17:34 inf-proxmox6 pvestatd[2906]: rados_connect failed - Operation not supported May? 6 22:17:34 inf-proxmox6 pvestatd[2906]: rados_connect failed - Operation not supported May? 6 22:17:34 inf-proxmox6 pvestatd[2906]: rados_connect failed - Operation not supported May? 6 22:17:35 inf-proxmox6 pmxcfs[2561]: [main] notice: exit proxmox configuration filesystem (0) May? 6 22:17:35 inf-proxmox6 systemd[1]: pve-cluster.service: Succeeded. May? 6 22:17:35 inf-proxmox6 systemd[1]: Stopped The Proxmox VE cluster filesystem. May? 6 22:17:35 inf-proxmox6 systemd[1]: Starting The Proxmox VE cluster filesystem... May? 6 22:17:35 inf-proxmox6 pmxcfs[8260]: [status] notice: update cluster info (cluster name? cluster-proxmox, version = 6) May? 6 22:17:35 inf-proxmox6 corosync[8007]:?? [TOTEM ] A new membership (1.1998) was formed. Members joined: 2 3 4 5 May? 6 22:17:36 inf-proxmox6 systemd[1]: Started The Proxmox VE cluster filesystem. Here is another extract that shows also some slow ops on a Ceph osd: May? 6 18:38:59 inf-proxmox6 corosync[2674]:?? [TOTEM ] Token has not been received in 3810 ms May? 6 18:39:00 inf-proxmox6 systemd[1]: Starting Proxmox VE replication runner... May? 6 18:39:01 inf-proxmox6 ceph-mon[1119484]: 2020-05-06 18:39:01.493 7feaed4bb700 -1 mon.0 at 0(leader) e6 get_health_metrics reporting 46 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:192.168.217.8:6884/1879695,v1:192.168.217.8:6885/1879695] for 20sec e73191 v73191) May? 6 18:39:02 inf-proxmox6 corosync[2674]:?? [TOTEM ] A new membership (1.eb4) was formed. Members May? 6 18:39:02 inf-proxmox6 corosync[2674]:?? [CPG?? ] downlist left_list: 0 received May? 6 18:39:02 inf-proxmox6 corosync[2674]:?? [CPG?? ] downlist left_list: 0 received May? 6 18:39:02 inf-proxmox6 corosync[2674]:?? [CPG?? ] downlist left_list: 0 received May? 6 18:39:02 inf-proxmox6 corosync[2674]:?? [CPG?? ] downlist left_list: 0 received May? 6 18:39:02 inf-proxmox6 corosync[2674]:?? [QUORUM] Members[4]: 1 3 4 5 May? 6 18:39:02 inf-proxmox6 corosync[2674]:?? [MAIN? ] Completed service synchronization, ready to provide service. May? 6 18:39:02 inf-proxmox6 pvesr[1409653]: trying to acquire cfs lock 'file-replication_cfg' ... May? 6 18:39:03 inf-proxmox6 pvesr[1409653]: trying to acquire cfs lock 'file-replication_cfg' ... May? 6 18:39:06 inf-proxmox6 systemd[1]: pvesr.service: Succeeded. May? 6 18:39:06 inf-proxmox6 systemd[1]: Started Proxmox VE replication runner. May? 6 18:39:06 inf-proxmox6 ceph-mon[1119484]: 2020-05-06 18:39:06.493 7feaed4bb700 -1 mon.0 at 0(leader) e6 get_health_metrics reporting 46 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:192.168.217.8:6884/1879695,v1:192.168.217.8:6885/1879695] for 20sec e73191 v73191) In case of, if that makes any sense for someone, thank you again, Herv? On 11/05/2020 17:58, Herve Ballans wrote: > > Hi Eneko, > > Thanks for your answer. I was also thinking at first a network issue > but physical network equipments don't seem to be showing any specific > problems...Here are more details on the cluster: > > 2x10Gb + 2x1Gb interface: > > * a 10Gb interface for ceph cluster > * a 10Gb interface for main network cluster > * the other 2 1Gb interfaces are used for two other VLAN for the VMs > > > > On 11/05/2020 10:39, Eneko Lacunza wrote: >> Hi Herv?, >> >> This seems a network issue. What is the network setup in this >> cluster? What logs in syslog about corosync and pve-cluster? >> >> Don't enable HA until you have a stable cluster quorum. >> >> Cheers >> Eneko >> >> El 11/5/20 a las 10:35, Herve Ballans escribi?: >>> Hi everybody, >>> >>> I would like to take the opportunity at the beginning of this new >>> week to ask my issue again. >>> >>> Has anyone had any idea why a such problem occurred, or is this >>> problem really something new ? >>> >>> Thanks again, >>> Herv? >>> >>> On 07/05/2020 18:28, Herve Ballans wrote: >>>> Hi all, >>>> >>>> *Cluster info:* >>>> >>>> ?* 5 nodes (version PVE 6.1-3 at the time the problem occured) >>>> ?* Ceph rbd storage (Nautilus) >>>> ?* In production since many years with no major issues >>>> ?* No specific network problems at the time the problem occured >>>> ?* Nodes are on the same date (configured with the same ntp server) >>>> >>>> *Symptoms:* >>>> >>>> Suddenly, last night (around 7 PM), all nodes of our cluster seems >>>> to have rebooted in the same time with no apparent reasons (I mean, >>>> we weren't doing antything on it) ! >>>> During the reboot, services "Corosync Cluster Engine" and "Proxmox >>>> VE replication runer" failed. After node rebooted, we are obliged >>>> to start those services manually. >>>> >>>> Once rebooted with all pve services, some nodes were in HA lrm >>>> status : old timestamp - dead? while others were in active status >>>> or in wait_for_agent_lock status ?... >>>> Nodes switch states regularly...and it loops back and forth as long >>>> as we don't change the configuration... >>>> >>>> In the same time, pve-ha-crm service got unexpected error, as for >>>> example : "Configuration file >>>> 'nodes/inf-proxmox6/qemu-server/501.conf' does not exist" even >>>> though the file exists but on an another node ! >>>> Such message is probably a consequence of the fencing between nodes >>>> due to the change of status... >>>> >>>> *What we have tried until now to stabilize the situation:* >>>> >>>> After several investigations and several operations that have >>>> failed to solve anything (in particular a complete upgrade to the >>>> latest PVE version 6.1-11), >>>> >>>> we finally removed the HA configuration of all the VM. >>>> Since, the state seems to be stabilized although, obviously, it is >>>> not nominal ! >>>> >>>> Now, all the nodes are in HA lrm status : idle and sometimes switch >>>> to old timestamp - dead? state, then come back to idle state. >>>> None of them are in "active" state. >>>> Obviously, quorum status is "no quorum" >>>> >>>> It will be noted that, as soon as we try to re-activate the HA >>>> status on the VMs, problem occurs again (nodes reboot!) :( >>>> >>>> *Question:* >>>> >>>> Have you ever experienced such a problem or do you know a way to >>>> restore a correct HA configuration in this case ? >>>> I point out that nodes are currently on version PVE 6.1-11. >>>> >>>> I can put some specific logs if useful. >>>> >>>> Thanks in advance for your help, >>>> Herv? >>>> >>>> _______________________________________________ >>>> pve-user mailing list >>>> pve-user at pve.proxmox.com >>>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >>> _______________________________________________ >>> pve-user mailing list >>> pve-user at pve.proxmox.com >>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >> >> From herve.ballans at ias.u-psud.fr Mon May 11 19:13:15 2020 From: herve.ballans at ias.u-psud.fr (Herve Ballans) Date: Mon, 11 May 2020 19:13:15 +0200 Subject: [PVE-User] critical HA problem on a PVE6 cluster In-Reply-To: <755b9f34-9f97-07a5-676e-3bf7d4fd9224@ias.u-psud.fr> References: <24544520-a537-af1d-b276-6fb56c331278@ias.u-psud.fr> <755b9f34-9f97-07a5-676e-3bf7d4fd9224@ias.u-psud.fr> Message-ID: Hi again, (sorry for the spam!). I just found logs just before the crash of one of the nodes (time of crash : 18:36:36). It could be more useful than logs sent previously...(I deleted here normal events) First, several messages like that (first one at 11:00 am): May? 6 18:33:25 inf-proxmox7 corosync[2648]:?? [TOTEM ] Token has not been received in 2212 ms May? 6 18:33:26 inf-proxmox7 corosync[2648]:?? [TOTEM ] A processor failed, forming new configuration. Then: May? 6 18:34:14 inf-proxmox7 corosync[2648]:?? [MAIN? ] Completed service synchronization, ready to provide service. May? 6 18:34:14 inf-proxmox7 pvesr[3342642]: error with cfs lock 'file-replication_cfg': got lock request timeout May? 6 18:34:14 inf-proxmox7 systemd[1]: pvesr.service: Main process exited, code=exited, status=17/n/a May? 6 18:34:14 inf-proxmox7 systemd[1]: pvesr.service: Failed with result 'exit-code'. May? 6 18:34:14 inf-proxmox7 systemd[1]: Failed to start Proxmox VE replication runner. May? 6 18:34:14 inf-proxmox7 pmxcfs[2602]: [status] notice: cpg_send_message retry 30 May? 6 18:34:14 inf-proxmox7 pmxcfs[2602]: [status] notice: cpg_send_message retried 30 times Then again a series of processor failed messages (in totally 147 before the crash): May? 6 18:35:03 inf-proxmox7 corosync[2648]:?? [TOTEM ] Token has not been received in 2212 ms May? 6 18:35:04 inf-proxmox7 corosync[2648]:?? [TOTEM ] A processor failed, forming new configuration. Then: May? 6 18:35:40 inf-proxmox7 pmxcfs[2602]: [dcdb] notice: start cluster connection May? 6 18:35:40 inf-proxmox7 pmxcfs[2602]: [dcdb] crit: cpg_join failed: 14 May? 6 18:35:40 inf-proxmox7 pmxcfs[2602]: [dcdb] crit: can't initialize service May? 6 18:35:40 inf-proxmox7 pve-ha-lrm[5528]: lost lock 'ha_agent_inf-proxmox7_lock - cfs lock update failed - Device or resource busy May? 6 18:35:40 inf-proxmox7 pve-ha-crm[5421]: status change slave => wait_for_quorum May? 6 18:35:41 inf-proxmox7 corosync[2648]:?? [TOTEM ] A new membership (1.e60) was formed. Members joined: 1 3 4 5 Then: May? 6 18:35:41 inf-proxmox7 pmxcfs[2602]: [status] notice: node has quorum May? 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [status] notice: cpg_send_message retried 1 times May? 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [status] notice: received sync request (epoch 1/2592/00000031) May? 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [status] notice: received sync request (epoch 1/2592/00000032) May? 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [dcdb] crit: cpg_send_message failed: 9 May? 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [dcdb] crit: cpg_send_message failed: 9 May? 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [status] notice: received all states May? 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [status] notice: all data is up to date May? 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [status] notice: dfsm_deliver_queue: queue length 144 Then: May? 6 18:35:57 inf-proxmox7 corosync[2648]:?? [TOTEM ] A new membership (1.e64) was formed. Members left: 3 4 May? 6 18:35:57 inf-proxmox7 corosync[2648]:?? [TOTEM ] Failed to receive the leave message. failed: 3 4 And finally crash after this last logs: May? 6 18:36:36 inf-proxmox7 pve-ha-crm[5421]: status change wait_for_quorum => slave May? 6 18:36:36 inf-proxmox7 systemd[1]: pvesr.service: Main process exited, code=exited, status=17/n/a May? 6 18:36:36 inf-proxmox7 systemd[1]: pvesr.service: Failed with result 'exit-code'. May? 6 18:36:36 inf-proxmox7 systemd[1]: Failed to start Proxmox VE replication runner. May? 6 18:36:36 inf-proxmox7 pve-ha-crm[5421]: loop take too long (51 seconds) May? 6 18:36:36 inf-proxmox7 systemd[1]: watchdog-mux.service: Succeeded. May? 6 18:36:36 inf-proxmox7 kernel: [1292969.953131] watchdog: watchdog0: watchdog did not stop! May? 6 18:36:36 inf-proxmox7 pvestatd[2894]: status update time (5.201 seconds) ^@^@^@^@^@^@ following by a binary part... Thank you again, Herv? On 11/05/2020 10:39, Eneko Lacunza wrote: >>> Hi Herv?, >>> >>> This seems a network issue. What is the network setup in this >>> cluster? What logs in syslog about corosync and pve-cluster? >>> >>> Don't enable HA until you have a stable cluster quorum. >>> >>> Cheers >>> Eneko >>> >>> El 11/5/20 a las 10:35, Herve Ballans escribi?: >>>> Hi everybody, >>>> >>>> I would like to take the opportunity at the beginning of this new >>>> week to ask my issue again. >>>> >>>> Has anyone had any idea why a such problem occurred, or is this >>>> problem really something new ? >>>> >>>> Thanks again, >>>> Herv? >>>> >>>> On 07/05/2020 18:28, Herve Ballans wrote: >>>>> Hi all, >>>>> >>>>> *Cluster info:* >>>>> >>>>> ?* 5 nodes (version PVE 6.1-3 at the time the problem occured) >>>>> ?* Ceph rbd storage (Nautilus) >>>>> ?* In production since many years with no major issues >>>>> ?* No specific network problems at the time the problem occured >>>>> ?* Nodes are on the same date (configured with the same ntp server) >>>>> >>>>> *Symptoms:* >>>>> >>>>> Suddenly, last night (around 7 PM), all nodes of our cluster seems >>>>> to have rebooted in the same time with no apparent reasons (I >>>>> mean, we weren't doing antything on it) ! >>>>> During the reboot, services "Corosync Cluster Engine" and "Proxmox >>>>> VE replication runer" failed. After node rebooted, we are obliged >>>>> to start those services manually. >>>>> >>>>> Once rebooted with all pve services, some nodes were in HA lrm >>>>> status : old timestamp - dead? while others were in active status >>>>> or in wait_for_agent_lock status ?... >>>>> Nodes switch states regularly...and it loops back and forth as >>>>> long as we don't change the configuration... >>>>> >>>>> In the same time, pve-ha-crm service got unexpected error, as for >>>>> example : "Configuration file >>>>> 'nodes/inf-proxmox6/qemu-server/501.conf' does not exist" even >>>>> though the file exists but on an another node ! >>>>> Such message is probably a consequence of the fencing between >>>>> nodes due to the change of status... >>>>> >>>>> *What we have tried until now to stabilize the situation:* >>>>> >>>>> After several investigations and several operations that have >>>>> failed to solve anything (in particular a complete upgrade to the >>>>> latest PVE version 6.1-11), >>>>> >>>>> we finally removed the HA configuration of all the VM. >>>>> Since, the state seems to be stabilized although, obviously, it is >>>>> not nominal ! >>>>> >>>>> Now, all the nodes are in HA lrm status : idle and sometimes >>>>> switch to old timestamp - dead? state, then come back to idle state. >>>>> None of them are in "active" state. >>>>> Obviously, quorum status is "no quorum" >>>>> >>>>> It will be noted that, as soon as we try to re-activate the HA >>>>> status on the VMs, problem occurs again (nodes reboot!) :( >>>>> >>>>> *Question:* >>>>> >>>>> Have you ever experienced such a problem or do you know a way to >>>>> restore a correct HA configuration in this case ? >>>>> I point out that nodes are currently on version PVE 6.1-11. >>>>> >>>>> I can put some specific logs if useful. >>>>> >>>>> Thanks in advance for your help, >>>>> Herv? From mark at openvs.co.uk Mon May 11 19:33:31 2020 From: mark at openvs.co.uk (Mark Adams) Date: Mon, 11 May 2020 18:33:31 +0100 Subject: [PVE-User] critical HA problem on a PVE6 cluster In-Reply-To: References: <24544520-a537-af1d-b276-6fb56c331278@ias.u-psud.fr> <755b9f34-9f97-07a5-676e-3bf7d4fd9224@ias.u-psud.fr> Message-ID: As Eneko already said, this really sounds like a network problem - if your hosts lose connectivity to each other they will reboot themselves, and it sounds like this is what happened to you. You are sure there has been no changes to your network around the time this happened? Have you checked your switch config is still right (maybe it reset?) Maybe the switches have bugged out and need a reboot? check the logs on them for errors. On Mon, 11 May 2020 at 18:13, Herve Ballans wrote: > Hi again, (sorry for the spam!). > > I just found logs just before the crash of one of the nodes (time of > crash : 18:36:36). It could be more useful than logs sent > previously...(I deleted here normal events) > > First, several messages like that (first one at 11:00 am): > > May 6 18:33:25 inf-proxmox7 corosync[2648]: [TOTEM ] Token has not > been received in 2212 ms > May 6 18:33:26 inf-proxmox7 corosync[2648]: [TOTEM ] A processor > failed, forming new configuration. > > Then: > > May 6 18:34:14 inf-proxmox7 corosync[2648]: [MAIN ] Completed > service synchronization, ready to provide service. > May 6 18:34:14 inf-proxmox7 pvesr[3342642]: error with cfs lock > 'file-replication_cfg': got lock request timeout > May 6 18:34:14 inf-proxmox7 systemd[1]: pvesr.service: Main process > exited, code=exited, status=17/n/a > May 6 18:34:14 inf-proxmox7 systemd[1]: pvesr.service: Failed with > result 'exit-code'. > May 6 18:34:14 inf-proxmox7 systemd[1]: Failed to start Proxmox VE > replication runner. > May 6 18:34:14 inf-proxmox7 pmxcfs[2602]: [status] notice: > cpg_send_message retry 30 > May 6 18:34:14 inf-proxmox7 pmxcfs[2602]: [status] notice: > cpg_send_message retried 30 times > > Then again a series of processor failed messages (in totally 147 before > the crash): > > May 6 18:35:03 inf-proxmox7 corosync[2648]: [TOTEM ] Token has not > been received in 2212 ms > May 6 18:35:04 inf-proxmox7 corosync[2648]: [TOTEM ] A processor > failed, forming new configuration. > > Then: > > May 6 18:35:40 inf-proxmox7 pmxcfs[2602]: [dcdb] notice: start cluster > connection > May 6 18:35:40 inf-proxmox7 pmxcfs[2602]: [dcdb] crit: cpg_join failed: 14 > May 6 18:35:40 inf-proxmox7 pmxcfs[2602]: [dcdb] crit: can't initialize > service > May 6 18:35:40 inf-proxmox7 pve-ha-lrm[5528]: lost lock > 'ha_agent_inf-proxmox7_lock - cfs lock update failed - Device or > resource busy > May 6 18:35:40 inf-proxmox7 pve-ha-crm[5421]: status change slave => > wait_for_quorum > May 6 18:35:41 inf-proxmox7 corosync[2648]: [TOTEM ] A new membership > (1.e60) was formed. Members joined: 1 3 4 5 > > Then: > > May 6 18:35:41 inf-proxmox7 pmxcfs[2602]: [status] notice: node has quorum > May 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [status] notice: > cpg_send_message retried 1 times > May 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [status] notice: received > sync request (epoch 1/2592/00000031) > May 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [status] notice: received > sync request (epoch 1/2592/00000032) > May 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [dcdb] crit: cpg_send_message > failed: 9 > May 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [dcdb] crit: cpg_send_message > failed: 9 > May 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [status] notice: received all > states > May 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [status] notice: all data is > up to date > May 6 18:35:42 inf-proxmox7 pmxcfs[2602]: [status] notice: > dfsm_deliver_queue: queue length 144 > > Then: > > May 6 18:35:57 inf-proxmox7 corosync[2648]: [TOTEM ] A new membership > (1.e64) was formed. Members left: 3 4 > May 6 18:35:57 inf-proxmox7 corosync[2648]: [TOTEM ] Failed to > receive the leave message. failed: 3 4 > > And finally crash after this last logs: > > May 6 18:36:36 inf-proxmox7 pve-ha-crm[5421]: status change > wait_for_quorum => slave > May 6 18:36:36 inf-proxmox7 systemd[1]: pvesr.service: Main process > exited, code=exited, status=17/n/a > May 6 18:36:36 inf-proxmox7 systemd[1]: pvesr.service: Failed with > result 'exit-code'. > May 6 18:36:36 inf-proxmox7 systemd[1]: Failed to start Proxmox VE > replication runner. > May 6 18:36:36 inf-proxmox7 pve-ha-crm[5421]: loop take too long (51 > seconds) > May 6 18:36:36 inf-proxmox7 systemd[1]: watchdog-mux.service: Succeeded. > May 6 18:36:36 inf-proxmox7 kernel: [1292969.953131] watchdog: > watchdog0: watchdog did not stop! > May 6 18:36:36 inf-proxmox7 pvestatd[2894]: status update time (5.201 > seconds) > ^@^@^@^@^@^@ > > following by a binary part... > > Thank you again, > Herv? > > On 11/05/2020 10:39, Eneko Lacunza wrote: > >>> Hi Herv?, > >>> > >>> This seems a network issue. What is the network setup in this > >>> cluster? What logs in syslog about corosync and pve-cluster? > >>> > >>> Don't enable HA until you have a stable cluster quorum. > >>> > >>> Cheers > >>> Eneko > >>> > >>> El 11/5/20 a las 10:35, Herve Ballans escribi?: > >>>> Hi everybody, > >>>> > >>>> I would like to take the opportunity at the beginning of this new > >>>> week to ask my issue again. > >>>> > >>>> Has anyone had any idea why a such problem occurred, or is this > >>>> problem really something new ? > >>>> > >>>> Thanks again, > >>>> Herv? > >>>> > >>>> On 07/05/2020 18:28, Herve Ballans wrote: > >>>>> Hi all, > >>>>> > >>>>> *Cluster info:* > >>>>> > >>>>> * 5 nodes (version PVE 6.1-3 at the time the problem occured) > >>>>> * Ceph rbd storage (Nautilus) > >>>>> * In production since many years with no major issues > >>>>> * No specific network problems at the time the problem occured > >>>>> * Nodes are on the same date (configured with the same ntp server) > >>>>> > >>>>> *Symptoms:* > >>>>> > >>>>> Suddenly, last night (around 7 PM), all nodes of our cluster seems > >>>>> to have rebooted in the same time with no apparent reasons (I > >>>>> mean, we weren't doing antything on it) ! > >>>>> During the reboot, services "Corosync Cluster Engine" and "Proxmox > >>>>> VE replication runer" failed. After node rebooted, we are obliged > >>>>> to start those services manually. > >>>>> > >>>>> Once rebooted with all pve services, some nodes were in HA lrm > >>>>> status : old timestamp - dead? while others were in active status > >>>>> or in wait_for_agent_lock status ?... > >>>>> Nodes switch states regularly...and it loops back and forth as > >>>>> long as we don't change the configuration... > >>>>> > >>>>> In the same time, pve-ha-crm service got unexpected error, as for > >>>>> example : "Configuration file > >>>>> 'nodes/inf-proxmox6/qemu-server/501.conf' does not exist" even > >>>>> though the file exists but on an another node ! > >>>>> Such message is probably a consequence of the fencing between > >>>>> nodes due to the change of status... > >>>>> > >>>>> *What we have tried until now to stabilize the situation:* > >>>>> > >>>>> After several investigations and several operations that have > >>>>> failed to solve anything (in particular a complete upgrade to the > >>>>> latest PVE version 6.1-11), > >>>>> > >>>>> we finally removed the HA configuration of all the VM. > >>>>> Since, the state seems to be stabilized although, obviously, it is > >>>>> not nominal ! > >>>>> > >>>>> Now, all the nodes are in HA lrm status : idle and sometimes > >>>>> switch to old timestamp - dead? state, then come back to idle state. > >>>>> None of them are in "active" state. > >>>>> Obviously, quorum status is "no quorum" > >>>>> > >>>>> It will be noted that, as soon as we try to re-activate the HA > >>>>> status on the VMs, problem occurs again (nodes reboot!) :( > >>>>> > >>>>> *Question:* > >>>>> > >>>>> Have you ever experienced such a problem or do you know a way to > >>>>> restore a correct HA configuration in this case ? > >>>>> I point out that nodes are currently on version PVE 6.1-11. > >>>>> > >>>>> I can put some specific logs if useful. > >>>>> > >>>>> Thanks in advance for your help, > >>>>> Herv? > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user From martin at proxmox.com Tue May 12 12:16:42 2020 From: martin at proxmox.com (Martin Maurer) Date: Tue, 12 May 2020 12:16:42 +0200 Subject: [PVE-User] Proxmox VE 6.2 released! Message-ID: <1411c32b-078f-12f0-ef7c-00b4987cd068@proxmox.com> Hi all, We are proud to announce the general availability of our virtualization management platform Proxmox VE 6.2. It's built on Debian Buster 10.4 and a 5.4 longterm Linux kernel, QEMU 5.0, LXC 4.0, ZFS 0.8.3, Ceph 14.2.9 (Nautilus), and ZFS 0.8.3. This release brings a built-in validation of domains for Let's Encrypt TLS certificates via the DNS-based challenge mechanism, full support for up to eight corosync network links, support for Zstandard for Backup/Restore, and a new LDAP sync for users and groups and full support for API tokens. Countless bugfixes and smaller improvements are included as well, see the full release notes. Release notes https://pve.proxmox.com/wiki/Roadmap#Proxmox_VE_6.2 Video tutorial https://www.proxmox.com/en/training/video-tutorials/item/what-s-new-in-proxmox-ve-6-2 Download https://www.proxmox.com/en/downloads Alternate ISO download: http://download.proxmox.com/iso Documentation https://pve.proxmox.com/pve-docs Community Forum https://forum.proxmox.com Source Code https://git.proxmox.com Bugtracker https://bugzilla.proxmox.com FAQ Q: Can I dist-upgrade Proxmox VE 6.x to 6.2 with apt? A: Yes, just via GUI or via CLI with apt update && apt dist-upgrade Q: Can I install Proxmox VE 6.2 on top of Debian Buster? A: Yes, see https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Buster Q: Can I upgrade my Proxmox VE 5.4 cluster with Ceph Luminous to 6.x and higher with Ceph Nautilus? A: This is a two step process. First, you have to upgrade Proxmox VE from 5.4 to 6.2, and afterwards upgrade Ceph from Luminous to Nautilus. There are a lot of improvements and changes, please follow exactly the upgrade documentation. https://pve.proxmox.com/wiki/Upgrade_from_5.x_to_6.0 https://pve.proxmox.com/wiki/Ceph_Luminous_to_Nautilus Q: Where can I get more information about future feature updates? A: Check our roadmap, forum, mailing lists, and subscribe to our newsletter. A big THANK YOU to our active community for all your feedback, testing, bug reporting and patch submitting! -- Best Regards, Martin Maurer Proxmox VE project leader martin at proxmox.com https://www.proxmox.com From leesteken at pm.me Tue May 12 13:12:40 2020 From: leesteken at pm.me (leesteken at pm.me) Date: Tue, 12 May 2020 11:12:40 +0000 Subject: unable to hotplug cpulimit; cpu.* not in /sys? Message-ID: Hi PVE-users, Lately, on a no-subscription Proxmox (that I update regularly since release 6.0), I get the following error when changing the CPU limit of any (unpriviliged) container: Parameter verification failed. (400) cpulimit: unable to hotplug cpulimit: unable to open file '/sys/fs/cgroup/memory///lxc/115/ns/cpu.cfs_period_us' - No such file or directory /etc/pve/lxc/115.conf: arch: amd64 console: 0 cpulimit: 1.5 hostname: vnc15 memory: 2048 #mp0: ... net0: name=eth0,bridge=vmbr0,firewall=1,gw=172.17.2.1,hwaddr=52:54:56:17:02:15,ip=172.17.2.15/24,type=veth onboot: 1 ostype: debian protection: 1 rootfs: qpool-zfs:subvol-115-disk-0,size=3G swap: 128 unprivileged: 1 [pve:pending] cpulimit: 1 Indeed, there are no cpu.* files in the directories of any of my containers. Is there something wrong with my setup? Maybe someone has run into the same problem or can tell me how to fix this? kind regards, Arjen From elacunza at binovo.es Tue May 12 15:00:01 2020 From: elacunza at binovo.es (Eneko Lacunza) Date: Tue, 12 May 2020 15:00:01 +0200 Subject: [PVE-User] critical HA problem on a PVE6 cluster In-Reply-To: References: <24544520-a537-af1d-b276-6fb56c331278@ias.u-psud.fr> Message-ID: Hi Herv?, El 11/5/20 a las 17:58, Herve Ballans escribi?: > Thanks for your answer. I was also thinking at first a network issue > but physical network equipments don't seem to be showing any specific > problems...Here are more details on the cluster: > > 2x10Gb + 2x1Gb interface: > > ?* a 10Gb interface for ceph cluster > ?* a 10Gb interface for main network cluster > ?* the other 2 1Gb interfaces are used for two other VLAN for the VMs Can you post "pvecm status" to see cluster network IPs? "ip a" for a node? "cat /etc/corosync/corosync.conf "? All network interfaces go to the same switch? PVE 6.2 has been released and it supports multiple networks for cluster. I suggest you look at it and configure a second network that uses another switch. In the logs you sent, I can see that there are grave cluster problems, at 18:38:58 I can see only nodes 1,3,4,5 in quorum Also, at 18:39:01 I can see ceph-mon complaining about slow ops and failed timeout for osd.5 . I really think there is a network issue. Ceph and Proxmox clusters are completely separate, but they're both having issues. I'd try to change the networking switch; I'd try even a 1G switch just to see if that makes Proxmox cluster and ceph stable. Are 10G interfaces very loaded? Cheers Eneko > > > > On 11/05/2020 10:39, Eneko Lacunza wrote: >> Hi Herv?, >> >> This seems a network issue. What is the network setup in this >> cluster? What logs in syslog about corosync and pve-cluster? >> >> Don't enable HA until you have a stable cluster quorum. >> >> Cheers >> Eneko >> >> El 11/5/20 a las 10:35, Herve Ballans escribi?: >>> Hi everybody, >>> >>> I would like to take the opportunity at the beginning of this new >>> week to ask my issue again. >>> >>> Has anyone had any idea why a such problem occurred, or is this >>> problem really something new ? >>> >>> Thanks again, >>> Herv? >>> >>> On 07/05/2020 18:28, Herve Ballans wrote: >>>> Hi all, >>>> >>>> *Cluster info:* >>>> >>>> ?* 5 nodes (version PVE 6.1-3 at the time the problem occured) >>>> ?* Ceph rbd storage (Nautilus) >>>> ?* In production since many years with no major issues >>>> ?* No specific network problems at the time the problem occured >>>> ?* Nodes are on the same date (configured with the same ntp server) >>>> >>>> *Symptoms:* >>>> >>>> Suddenly, last night (around 7 PM), all nodes of our cluster seems >>>> to have rebooted in the same time with no apparent reasons (I mean, >>>> we weren't doing antything on it) ! >>>> During the reboot, services "Corosync Cluster Engine" and "Proxmox >>>> VE replication runer" failed. After node rebooted, we are obliged >>>> to start those services manually. >>>> >>>> Once rebooted with all pve services, some nodes were in HA lrm >>>> status : old timestamp - dead? while others were in active status >>>> or in wait_for_agent_lock status ?... >>>> Nodes switch states regularly...and it loops back and forth as long >>>> as we don't change the configuration... >>>> >>>> In the same time, pve-ha-crm service got unexpected error, as for >>>> example : "Configuration file >>>> 'nodes/inf-proxmox6/qemu-server/501.conf' does not exist" even >>>> though the file exists but on an another node ! >>>> Such message is probably a consequence of the fencing between nodes >>>> due to the change of status... >>>> >>>> *What we have tried until now to stabilize the situation:* >>>> >>>> After several investigations and several operations that have >>>> failed to solve anything (in particular a complete upgrade to the >>>> latest PVE version 6.1-11), >>>> >>>> we finally removed the HA configuration of all the VM. >>>> Since, the state seems to be stabilized although, obviously, it is >>>> not nominal ! >>>> >>>> Now, all the nodes are in HA lrm status : idle and sometimes switch >>>> to old timestamp - dead? state, then come back to idle state. >>>> None of them are in "active" state. >>>> Obviously, quorum status is "no quorum" >>>> >>>> It will be noted that, as soon as we try to re-activate the HA >>>> status on the VMs, problem occurs again (nodes reboot!) :( >>>> >>>> *Question:* >>>> >>>> Have you ever experienced such a problem or do you know a way to >>>> restore a correct HA configuration in this case ? >>>> I point out that nodes are currently on version PVE 6.1-11. >>>> >>>> I can put some specific logs if useful. >>>> >>>> Thanks in advance for your help, >>>> Herv? >>>> >>>> _______________________________________________ >>>> pve-user mailing list >>>> pve-user at pve.proxmox.com >>>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >>> _______________________________________________ >>> pve-user mailing list >>> pve-user at pve.proxmox.com >>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >> >> > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user -- Zuzendari Teknikoa / Director T?cnico Binovo IT Human Project, S.L. Telf. 943569206 Astigarragako bidea 2, 2? izq. oficina 11; 20180 Oiartzun (Gipuzkoa) www.binovo.es From sleemburg at it-functions.nl Tue May 12 20:33:58 2020 From: sleemburg at it-functions.nl (Stephan Leemburg) Date: Tue, 12 May 2020 20:33:58 +0200 Subject: [PVE-User] Proxmox VE 6.2 released! In-Reply-To: <1411c32b-078f-12f0-ef7c-00b4987cd068@proxmox.com> References: <1411c32b-078f-12f0-ef7c-00b4987cd068@proxmox.com> Message-ID: <97acec80-8afc-4dbd-586d-4cc3900a608f@it-functions.nl> Hi Martin, Thank you for this new version. One question though. What is meant by: * Improve default settings to support hundreds to thousands* of parallel running Containers per node (* thousands only with simple distributions like Alpine Linux) Is that setting feature nesting and running docker inside a lxc container or in a kvm? Could you elaborate a little more on that statement? Thank you and kind regards, Stephan On 12-05-2020 12:16, Martin Maurer wrote: > Hi all, > > We are proud to announce the general availability of our > virtualization management platform Proxmox VE 6.2. > > It's built on Debian Buster 10.4 and a 5.4 longterm Linux kernel, QEMU > 5.0, LXC 4.0, ZFS 0.8.3, Ceph 14.2.9 (Nautilus), and ZFS 0.8.3. > > This release brings a built-in validation of domains for Let's Encrypt > TLS certificates via the DNS-based challenge mechanism, full support > for up to eight corosync network links, support for Zstandard for > Backup/Restore, and a new LDAP sync for users and groups and full > support for API tokens. > > Countless bugfixes and smaller improvements are included as well, see > the full release notes. > > Release notes > https://pve.proxmox.com/wiki/Roadmap#Proxmox_VE_6.2 > > Video tutorial > https://www.proxmox.com/en/training/video-tutorials/item/what-s-new-in-proxmox-ve-6-2 > > > Download > https://www.proxmox.com/en/downloads > Alternate ISO download: > http://download.proxmox.com/iso > > Documentation > https://pve.proxmox.com/pve-docs > > Community Forum > https://forum.proxmox.com > > Source Code > https://git.proxmox.com > > Bugtracker > https://bugzilla.proxmox.com > > FAQ > Q: Can I dist-upgrade Proxmox VE 6.x to 6.2 with apt? > A: Yes, just via GUI or via CLI with apt update && apt dist-upgrade > > Q: Can I install Proxmox VE 6.2 on top of Debian Buster? > A: Yes, see > https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Buster > > Q: Can I upgrade my Proxmox VE 5.4 cluster with Ceph Luminous to 6.x > and higher with Ceph Nautilus? > A: This is a two step process. First, you have to upgrade Proxmox VE > from 5.4 to 6.2, and afterwards upgrade Ceph from Luminous to > Nautilus. There are a lot of improvements and changes, please follow > exactly the upgrade documentation. > https://pve.proxmox.com/wiki/Upgrade_from_5.x_to_6.0 > https://pve.proxmox.com/wiki/Ceph_Luminous_to_Nautilus > > Q: Where can I get more information about future feature updates? > A: Check our roadmap, forum, mailing lists, and subscribe to our > newsletter. > > A big THANK YOU to our active community for all your feedback, > testing, bug reporting and patch submitting! > From sivakumar.saravanan.jv.ext at valeo-siemens.com Wed May 13 10:14:33 2020 From: sivakumar.saravanan.jv.ext at valeo-siemens.com (Sivakumar SARAVANAN) Date: Wed, 13 May 2020 10:14:33 +0200 Subject: [PVE-User] Not able to add the new host in to cluster Message-ID: Hello, Not able to add the new Proxmox host in to cluster, where I am getting below error X509 SHA256 key fingerprint is E5:98:6E:7C:1D:BB:AC:23:71:08:E7:29:9B:B5:A8:B8:E8:37:AD:5B:9D:2A:2A:87:CC:7A:D8:4B:7A:B3:2E:A4. Are you sure you want to continue connecting (yes/no)? yes Login succeeded. Request addition of this node 500 can't lock file '/var/lock/pvecm.lock' - got timeout Kindly let me know the reason for the above error. Thanks oin advance. Mit freundlichen Gr??en / Best regards / Cordialement, Sivakumar SARAVANAN Externer Dienstleister f?r / External service provider for Valeo Siemens eAutomotive Germany GmbH Research & Development R & D SWENG TE 1 INFTE Frauenauracher Stra?e 85 91056 Erlangen, Germany Tel.: +49 9131 9892 0000 Mobile: +49 176 7698 5441 sivakumar.saravanan.jv.ext at valeo-siemens.com valeo-siemens.com Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger Schwab, Michael Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * From martin at proxmox.com Wed May 13 11:22:22 2020 From: martin at proxmox.com (Martin Maurer) Date: Wed, 13 May 2020 11:22:22 +0200 Subject: [PVE-User] Not able to add the new host in to cluster In-Reply-To: References: Message-ID: <566dd2dc-1698-8f96-ed9c-8883227e46f0@proxmox.com> Hello, On 5/13/20 10:14 AM, Sivakumar SARAVANAN wrote: > -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. Please think again about your text in the signature of your emails when you send emails to a public and distributed mailing list ... -- Best Regards, Martin Maurer martin at proxmox.com https://www.proxmox.com From t.lamprecht at proxmox.com Wed May 13 15:03:51 2020 From: t.lamprecht at proxmox.com (Thomas Lamprecht) Date: Wed, 13 May 2020 15:03:51 +0200 Subject: [PVE-User] Proxmox VE 6.2 released! In-Reply-To: <97acec80-8afc-4dbd-586d-4cc3900a608f@it-functions.nl> References: <1411c32b-078f-12f0-ef7c-00b4987cd068@proxmox.com> <97acec80-8afc-4dbd-586d-4cc3900a608f@it-functions.nl> Message-ID: <004ed5e2-b1b2-3671-da43-73f42885b862@proxmox.com> Hi, On 5/12/20 8:33 PM, Stephan Leemburg wrote: > One question though. What is meant by: > > ?* Improve default settings to support hundreds to thousands* of > ?? parallel running Containers per node (* thousands only with simple > ?? distributions like Alpine Linux) > > Is that setting feature nesting and running docker inside a lxc container or in a kvm? Could you elaborate a little more on that statement? No docker, no QEMU/KVM. Simply some sysctl default values for things like open INotify watchers, entries in the ARP neighborhood table, maximum of mmap memory area counts, maximum kernel keyring entries, ... got bumped to ensure that systems using a lot of CTs do not need to do this fine tuning themself. cheers, Thomas From info at aminvakil.com Wed May 13 15:26:24 2020 From: info at aminvakil.com (Amin Vakil) Date: Wed, 13 May 2020 17:56:24 +0430 Subject: [PVE-User] ACME Cloudflare Managed DNS Issue Message-ID: I've tested with global api key, specific api token which has full access to zone.zone and zone.dns of all zones, tested subdomain.domain.com, domain.com, let's encrypt and let's encrypt staging and none of them works. I've checked via http authentication everything goes fine, but I don't to open my 80 port globally. Is there an issue with this type of verification for now? Best Regards, Amin Vakil -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From info at aminvakil.com Wed May 13 15:29:57 2020 From: info at aminvakil.com (Amin Vakil) Date: Wed, 13 May 2020 17:59:57 +0430 Subject: [PVE-User] ACME Cloudflare Managed DNS Issue In-Reply-To: References: Message-ID: I've forgot to add errors: Using CF_Account_ID and CF_Token: [Wed May 13 17:56:58 +0430 2020] Error [Wed May 13 17:56:58 +0430 2020] Error add txt for domain:_acme-challenge.subdomain.domain.com TASK ERROR: command 'setpriv --reuid nobody --regid nogroup --clear-groups --reset-env -- /bin/bash /usr/share/proxmox-acme/proxmox-acme setup cf subdomain.domain.com' failed: exit code 1 Using CF_Email and CF_Key: [Wed May 13 17:59:03 +0430 2020] Error [Wed May 13 17:59:03 +0430 2020] Error add txt for domain:_acme-challenge.subdomain.domain.com TASK ERROR: command 'setpriv --reuid nobody --regid nogroup --clear-groups --reset-env -- /bin/bash /usr/share/proxmox-acme/proxmox-acme setup cf subdomain.domain.com' failed: exit code 1 On 5/13/20 5:56 PM, Amin Vakil wrote: > I've tested with global api key, specific api token which has full > access to zone.zone and zone.dns of all zones, tested > subdomain.domain.com, domain.com, let's encrypt and let's encrypt > staging and none of them works. > > I've checked via http authentication everything goes fine, but I don't > to open my 80 port globally. > > Is there an issue with this type of verification for now? > > Best Regards, > Amin Vakil > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From f.gruenbichler at proxmox.com Thu May 14 08:28:40 2020 From: f.gruenbichler at proxmox.com (Fabian =?iso-8859-1?q?Gr=FCnbichler?=) Date: Thu, 14 May 2020 08:28:40 +0200 Subject: [PVE-User] ACME Cloudflare Managed DNS Issue In-Reply-To: References: Message-ID: <1589437684.zyleno4vgz.astroid@nora.none> On May 13, 2020 3:29 pm, Amin Vakil wrote: > I've forgot to add errors: > > Using CF_Account_ID and CF_Token: > > [Wed May 13 17:56:58 +0430 2020] Error > [Wed May 13 17:56:58 +0430 2020] Error add txt for > domain:_acme-challenge.subdomain.domain.com > TASK ERROR: command 'setpriv --reuid nobody --regid nogroup > --clear-groups --reset-env -- /bin/bash > /usr/share/proxmox-acme/proxmox-acme setup cf subdomain.domain.com' > failed: exit code 1 > > Using CF_Email and CF_Key: > > [Wed May 13 17:59:03 +0430 2020] Error > [Wed May 13 17:59:03 +0430 2020] Error add txt for > domain:_acme-challenge.subdomain.domain.com > TASK ERROR: command 'setpriv --reuid nobody --regid nogroup > --clear-groups --reset-env -- /bin/bash > /usr/share/proxmox-acme/proxmox-acme setup cf subdomain.domain.com' > failed: exit code 1 > > On 5/13/20 5:56 PM, Amin Vakil wrote: >> I've tested with global api key, specific api token which has full >> access to zone.zone and zone.dns of all zones, tested >> subdomain.domain.com, domain.com, let's encrypt and let's encrypt >> staging and none of them works. >> >> I've checked via http authentication everything goes fine, but I don't >> to open my 80 port globally. >> >> Is there an issue with this type of verification for now? >> >> Best Regards, >> Amin Vakil see https://bugzilla.proxmox.com/show_bug.cgi?id=2733 From herve.ballans at ias.u-psud.fr Thu May 14 16:48:45 2020 From: herve.ballans at ias.u-psud.fr (Herve Ballans) Date: Thu, 14 May 2020 16:48:45 +0200 Subject: [PVE-User] critical HA problem on a PVE6 cluster In-Reply-To: References: <24544520-a537-af1d-b276-6fb56c331278@ias.u-psud.fr> Message-ID: Hi Eneko, Thanks again for trying to help me. Now, the problem is solved!? We upgraded our entire cluster in PVE 6.2 and now all is optimal, including HA status. We just upgraded each nodes, didn't change anything else (I mean in term of configuration file). Here, I'm just stating a fact, I don't say that this is the upgrade process that are solved our problems... Indeed we are trying to investigate with network engineers who manage the network equipments of our datacenter in order to see if something was happening at the moment where our cluster had crashed. I will let you know if I have the answer to that mystery... Cheers, Herv? On 12/05/2020 15:00, Eneko Lacunza wrote: > Hi Herv?, > > El 11/5/20 a las 17:58, Herve Ballans escribi?: >> Thanks for your answer. I was also thinking at first a network issue >> but physical network equipments don't seem to be showing any specific >> problems...Here are more details on the cluster: >> >> 2x10Gb + 2x1Gb interface: >> >> ?* a 10Gb interface for ceph cluster >> ?* a 10Gb interface for main network cluster >> ?* the other 2 1Gb interfaces are used for two other VLAN for the VMs > > Can you post > "pvecm status" to see cluster network IPs? > "ip a" for a node? > "cat /etc/corosync/corosync.conf "? > > All network interfaces go to the same switch? > > PVE 6.2 has been released and it supports multiple networks for > cluster. I suggest you look at it and configure a second network that > uses another switch. > > In the logs you sent, I can see that there are grave cluster problems, > at 18:38:58 I can see only nodes 1,3,4,5 in quorum > > Also, at 18:39:01 I can see ceph-mon complaining about slow ops and > failed timeout for osd.5 . > > I really think there is a network issue. Ceph and Proxmox clusters are > completely separate, but they're both having issues. > > I'd try to change the networking switch; I'd try even a 1G switch just > to see if that makes Proxmox cluster and ceph stable. Are 10G > interfaces very loaded? > > Cheers > Eneko > >> >> >> >> On 11/05/2020 10:39, Eneko Lacunza wrote: >>> Hi Herv?, >>> >>> This seems a network issue. What is the network setup in this >>> cluster? What logs in syslog about corosync and pve-cluster? >>> >>> Don't enable HA until you have a stable cluster quorum. >>> >>> Cheers >>> Eneko >>> >>> El 11/5/20 a las 10:35, Herve Ballans escribi?: >>>> Hi everybody, >>>> >>>> I would like to take the opportunity at the beginning of this new >>>> week to ask my issue again. >>>> >>>> Has anyone had any idea why a such problem occurred, or is this >>>> problem really something new ? >>>> >>>> Thanks again, >>>> Herv? >>>> >>>> On 07/05/2020 18:28, Herve Ballans wrote: >>>>> Hi all, >>>>> >>>>> *Cluster info:* >>>>> >>>>> ?* 5 nodes (version PVE 6.1-3 at the time the problem occured) >>>>> ?* Ceph rbd storage (Nautilus) >>>>> ?* In production since many years with no major issues >>>>> ?* No specific network problems at the time the problem occured >>>>> ?* Nodes are on the same date (configured with the same ntp server) >>>>> >>>>> *Symptoms:* >>>>> >>>>> Suddenly, last night (around 7 PM), all nodes of our cluster seems >>>>> to have rebooted in the same time with no apparent reasons (I >>>>> mean, we weren't doing antything on it) ! >>>>> During the reboot, services "Corosync Cluster Engine" and "Proxmox >>>>> VE replication runer" failed. After node rebooted, we are obliged >>>>> to start those services manually. >>>>> >>>>> Once rebooted with all pve services, some nodes were in HA lrm >>>>> status : old timestamp - dead? while others were in active status >>>>> or in wait_for_agent_lock status ?... >>>>> Nodes switch states regularly...and it loops back and forth as >>>>> long as we don't change the configuration... >>>>> >>>>> In the same time, pve-ha-crm service got unexpected error, as for >>>>> example : "Configuration file >>>>> 'nodes/inf-proxmox6/qemu-server/501.conf' does not exist" even >>>>> though the file exists but on an another node ! >>>>> Such message is probably a consequence of the fencing between >>>>> nodes due to the change of status... >>>>> >>>>> *What we have tried until now to stabilize the situation:* >>>>> >>>>> After several investigations and several operations that have >>>>> failed to solve anything (in particular a complete upgrade to the >>>>> latest PVE version 6.1-11), >>>>> >>>>> we finally removed the HA configuration of all the VM. >>>>> Since, the state seems to be stabilized although, obviously, it is >>>>> not nominal ! >>>>> >>>>> Now, all the nodes are in HA lrm status : idle and sometimes >>>>> switch to old timestamp - dead? state, then come back to idle state. >>>>> None of them are in "active" state. >>>>> Obviously, quorum status is "no quorum" >>>>> >>>>> It will be noted that, as soon as we try to re-activate the HA >>>>> status on the VMs, problem occurs again (nodes reboot!) :( >>>>> >>>>> *Question:* >>>>> >>>>> Have you ever experienced such a problem or do you know a way to >>>>> restore a correct HA configuration in this case ? >>>>> I point out that nodes are currently on version PVE 6.1-11. >>>>> >>>>> I can put some specific logs if useful. >>>>> >>>>> Thanks in advance for your help, >>>>> Herv? >>>>> >>>>> _______________________________________________ >>>>> pve-user mailing list >>>>> pve-user at pve.proxmox.com >>>>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >>>> _______________________________________________ >>>> pve-user mailing list >>>> pve-user at pve.proxmox.com >>>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >>> >>> >> _______________________________________________ >> pve-user mailing list >> pve-user at pve.proxmox.com >> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > From herve.ballans at ias.u-psud.fr Thu May 14 17:01:52 2020 From: herve.ballans at ias.u-psud.fr (Herve Ballans) Date: Thu, 14 May 2020 17:01:52 +0200 Subject: [PVE-User] critical HA problem on a PVE6 cluster In-Reply-To: References: <24544520-a537-af1d-b276-6fb56c331278@ias.u-psud.fr> <755b9f34-9f97-07a5-676e-3bf7d4fd9224@ias.u-psud.fr> Message-ID: <071c67b8-83dc-5305-9c39-0b6710087994@ias.u-psud.fr> Hi Mark, Thanks. Yes we are investigating with network engineers. We upgraded the entire cluster in PVE 6.2 and the cluster is fully operational now. But we think indeed that something in the network has changed and caused the problem (switch upgrades ?) Therefore, for example, does activating or disabling the IGMP protocol could have an impact on corosync or not (in PVE 6) ? Regards, Herv? On 11/05/2020 19:33, Mark Adams via pve-user wrote: > Subject: > Re: [PVE-User] critical HA problem on a PVE6 cluster > From: > Mark Adams > Date: > 11/05/2020 ? 19:33 > > To: > PVE User List > > > As Eneko already said, this really sounds like a network problem - if your > hosts lose connectivity to each other they will reboot themselves, and it > sounds like this is what happened to you. > > You are sure there has been no changes to your network around the time this > happened? Have you checked your switch config is still right (maybe it > reset?) > > Maybe the switches have bugged out and need a reboot? check the logs on > them for errors. From elacunza at binovo.es Thu May 14 17:17:40 2020 From: elacunza at binovo.es (Eneko Lacunza) Date: Thu, 14 May 2020 17:17:40 +0200 Subject: [PVE-User] critical HA problem on a PVE6 cluster In-Reply-To: References: <24544520-a537-af1d-b276-6fb56c331278@ias.u-psud.fr> Message-ID: Hi Herv?, Glad to read this :) Cheers El 14/5/20 a las 16:48, Herve Ballans escribi?: > Hi Eneko, > > Thanks again for trying to help me. > > Now, the problem is solved!? We upgraded our entire cluster in PVE 6.2 > and now all is optimal, including HA status. > We just upgraded each nodes, didn't change anything else (I mean in > term of configuration file). > > Here, I'm just stating a fact, I don't say that this is the upgrade > process that are solved our problems... > > Indeed we are trying to investigate with network engineers who manage > the network equipments of our datacenter in order to see if something > was happening at the moment where our cluster had crashed. > > I will let you know if I have the answer to that mystery... > > Cheers, > Herv? > > On 12/05/2020 15:00, Eneko Lacunza wrote: >> Hi Herv?, >> >> El 11/5/20 a las 17:58, Herve Ballans escribi?: >>> Thanks for your answer. I was also thinking at first a network issue >>> but physical network equipments don't seem to be showing any >>> specific problems...Here are more details on the cluster: >>> >>> 2x10Gb + 2x1Gb interface: >>> >>> ?* a 10Gb interface for ceph cluster >>> ?* a 10Gb interface for main network cluster >>> ?* the other 2 1Gb interfaces are used for two other VLAN for the VMs >> >> Can you post >> "pvecm status" to see cluster network IPs? >> "ip a" for a node? >> "cat /etc/corosync/corosync.conf "? >> >> All network interfaces go to the same switch? >> >> PVE 6.2 has been released and it supports multiple networks for >> cluster. I suggest you look at it and configure a second network that >> uses another switch. >> >> In the logs you sent, I can see that there are grave cluster >> problems, at 18:38:58 I can see only nodes 1,3,4,5 in quorum >> >> Also, at 18:39:01 I can see ceph-mon complaining about slow ops and >> failed timeout for osd.5 . >> >> I really think there is a network issue. Ceph and Proxmox clusters >> are completely separate, but they're both having issues. >> >> I'd try to change the networking switch; I'd try even a 1G switch >> just to see if that makes Proxmox cluster and ceph stable. Are 10G >> interfaces very loaded? >> >> Cheers >> Eneko >> >>> >>> >>> >>> On 11/05/2020 10:39, Eneko Lacunza wrote: >>>> Hi Herv?, >>>> >>>> This seems a network issue. What is the network setup in this >>>> cluster? What logs in syslog about corosync and pve-cluster? >>>> >>>> Don't enable HA until you have a stable cluster quorum. >>>> >>>> Cheers >>>> Eneko >>>> >>>> El 11/5/20 a las 10:35, Herve Ballans escribi?: >>>>> Hi everybody, >>>>> >>>>> I would like to take the opportunity at the beginning of this new >>>>> week to ask my issue again. >>>>> >>>>> Has anyone had any idea why a such problem occurred, or is this >>>>> problem really something new ? >>>>> >>>>> Thanks again, >>>>> Herv? >>>>> >>>>> On 07/05/2020 18:28, Herve Ballans wrote: >>>>>> Hi all, >>>>>> >>>>>> *Cluster info:* >>>>>> >>>>>> ?* 5 nodes (version PVE 6.1-3 at the time the problem occured) >>>>>> ?* Ceph rbd storage (Nautilus) >>>>>> ?* In production since many years with no major issues >>>>>> ?* No specific network problems at the time the problem occured >>>>>> ?* Nodes are on the same date (configured with the same ntp server) >>>>>> >>>>>> *Symptoms:* >>>>>> >>>>>> Suddenly, last night (around 7 PM), all nodes of our cluster >>>>>> seems to have rebooted in the same time with no apparent reasons >>>>>> (I mean, we weren't doing antything on it) ! >>>>>> During the reboot, services "Corosync Cluster Engine" and >>>>>> "Proxmox VE replication runer" failed. After node rebooted, we >>>>>> are obliged to start those services manually. >>>>>> >>>>>> Once rebooted with all pve services, some nodes were in HA lrm >>>>>> status : old timestamp - dead? while others were in active status >>>>>> or in wait_for_agent_lock status ?... >>>>>> Nodes switch states regularly...and it loops back and forth as >>>>>> long as we don't change the configuration... >>>>>> >>>>>> In the same time, pve-ha-crm service got unexpected error, as for >>>>>> example : "Configuration file >>>>>> 'nodes/inf-proxmox6/qemu-server/501.conf' does not exist" even >>>>>> though the file exists but on an another node ! >>>>>> Such message is probably a consequence of the fencing between >>>>>> nodes due to the change of status... >>>>>> >>>>>> *What we have tried until now to stabilize the situation:* >>>>>> >>>>>> After several investigations and several operations that have >>>>>> failed to solve anything (in particular a complete upgrade to the >>>>>> latest PVE version 6.1-11), >>>>>> >>>>>> we finally removed the HA configuration of all the VM. >>>>>> Since, the state seems to be stabilized although, obviously, it >>>>>> is not nominal ! >>>>>> >>>>>> Now, all the nodes are in HA lrm status : idle and sometimes >>>>>> switch to old timestamp - dead? state, then come back to idle state. >>>>>> None of them are in "active" state. >>>>>> Obviously, quorum status is "no quorum" >>>>>> >>>>>> It will be noted that, as soon as we try to re-activate the HA >>>>>> status on the VMs, problem occurs again (nodes reboot!) :( >>>>>> >>>>>> *Question:* >>>>>> >>>>>> Have you ever experienced such a problem or do you know a way to >>>>>> restore a correct HA configuration in this case ? >>>>>> I point out that nodes are currently on version PVE 6.1-11. >>>>>> >>>>>> I can put some specific logs if useful. >>>>>> >>>>>> Thanks in advance for your help, >>>>>> Herv? >>>>>> >>>>>> _______________________________________________ >>>>>> pve-user mailing list >>>>>> pve-user at pve.proxmox.com >>>>>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >>>>> _______________________________________________ >>>>> pve-user mailing list >>>>> pve-user at pve.proxmox.com >>>>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >>>> >>>> >>> _______________________________________________ >>> pve-user mailing list >>> pve-user at pve.proxmox.com >>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >> >> -- Zuzendari Teknikoa / Director T?cnico Binovo IT Human Project, S.L. Telf. 943569206 Astigarragako bidea 2, 2? izq. oficina 11; 20180 Oiartzun (Gipuzkoa) www.binovo.es From sivakumar.saravanan.jv.ext at valeo-siemens.com Thu May 14 17:38:48 2020 From: sivakumar.saravanan.jv.ext at valeo-siemens.com (Sivakumar SARAVANAN) Date: Thu, 14 May 2020 17:38:48 +0200 Subject: [PVE-User] Some VM's are not able to start Message-ID: Hello, We have implemented the Proxmox VE in our environment. So each server will have a maximum 6 VM. But not able to start the few VM's ON until we bring down the 1 or 2 VM's in the same Host. What could be the reason. Kindly suggest. Thanks in advance. Best Regards SK -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * From daniel at firewall-services.com Thu May 14 17:46:41 2020 From: daniel at firewall-services.com (Daniel Berteaud) Date: Thu, 14 May 2020 17:46:41 +0200 (CEST) Subject: [PVE-User] Some VM's are not able to start In-Reply-To: References: Message-ID: <1390593605.57004.1589471201696.JavaMail.zimbra@fws.fr> ----- Le 14 Mai 20, ? 17:38, Sivakumar SARAVANAN sivakumar.saravanan.jv.ext at valeo-siemens.com a ?crit : > Hello, > > We have implemented the Proxmox VE in our environment. > > So each server will have a maximum 6 VM. But not able to start the few VM's > ON until we bring down the 1 or 2 VM's in the same Host. > Please describe what you mean by "not able to start" Cheers, Daniel -- [ https://www.firewall-services.com/ ] Daniel Berteaud FIREWALL-SERVICES SAS, La s?curit? des r?seaux Soci?t? de Services en Logiciels Libres T?l : +33.5 56 64 15 32 Matrix: @dani:fws.fr [ https://www.firewall-services.com/ | https://www.firewall-services.com ] From sivakumar.saravanan.jv.ext at valeo-siemens.com Thu May 14 18:16:38 2020 From: sivakumar.saravanan.jv.ext at valeo-siemens.com (Sivakumar SARAVANAN) Date: Thu, 14 May 2020 18:16:38 +0200 Subject: [PVE-User] Some VM's are not able to start In-Reply-To: <1390593605.57004.1589471201696.JavaMail.zimbra@fws.fr> References: <1390593605.57004.1589471201696.JavaMail.zimbra@fws.fr> Message-ID: Hello Daniel, Thanks for coming back. I mean, I am unable to power ON the VM until shutdown the other VM's in the same host. There are 6 VM's running on each Host, But sometimes all 6 VM's would run without any issue. But Sometimes if I stop ( Shutdown) and Power ON ( Start) getting an error saying as below. But each will have 32 GB memory. start failed: hugepage allocation failed at /usr/share/perl5/PVE/QemuServer/Memory.pm line 541. Appreciating your suggestion. Best regards SK On Thu, May 14, 2020 at 5:46 PM Daniel Berteaud < daniel at firewall-services.com> wrote: > > > ----- Le 14 Mai 20, ? 17:38, Sivakumar SARAVANAN > sivakumar.saravanan.jv.ext at valeo-siemens.com a ?crit : > > > Hello, > > > > We have implemented the Proxmox VE in our environment. > > > > So each server will have a maximum 6 VM. But not able to start the few > VM's > > ON until we bring down the 1 or 2 VM's in the same Host. > > > > Please describe what you mean by "not able to start" > > Cheers, > Daniel > > -- > [ https://www.firewall-services.com/ ] > Daniel Berteaud > FIREWALL-SERVICES SAS, La s?curit? des r?seaux > Soci?t? de Services en Logiciels Libres > T?l : +33.5 56 64 15 32 > Matrix: @dani:fws.fr > [ https://www.firewall-services.com/ | https://www.firewall-services.com ] > > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * From mark at openvs.co.uk Thu May 14 18:19:09 2020 From: mark at openvs.co.uk (Mark Adams) Date: Thu, 14 May 2020 17:19:09 +0100 Subject: [PVE-User] Some VM's are not able to start In-Reply-To: References: <1390593605.57004.1589471201696.JavaMail.zimbra@fws.fr> Message-ID: Do you really need hugepages? if not disable it. On Thu, 14 May 2020 at 17:17, Sivakumar SARAVANAN < sivakumar.saravanan.jv.ext at valeo-siemens.com> wrote: > Hello Daniel, > > Thanks for coming back. > > I mean, I am unable to power ON the VM until shutdown the other VM's in the > same host. > > There are 6 VM's running on each Host, But sometimes all 6 VM's would run > without any issue. But Sometimes if I stop ( Shutdown) and Power ON ( > Start) getting an error saying as below. But each will have 32 GB memory. > > start failed: hugepage allocation failed at > /usr/share/perl5/PVE/QemuServer/Memory.pm line 541. > > Appreciating your suggestion. > > > > > Best regards > SK > > On Thu, May 14, 2020 at 5:46 PM Daniel Berteaud < > daniel at firewall-services.com> wrote: > > > > > > > ----- Le 14 Mai 20, ? 17:38, Sivakumar SARAVANAN > > sivakumar.saravanan.jv.ext at valeo-siemens.com a ?crit : > > > > > Hello, > > > > > > We have implemented the Proxmox VE in our environment. > > > > > > So each server will have a maximum 6 VM. But not able to start the few > > VM's > > > ON until we bring down the 1 or 2 VM's in the same Host. > > > > > > > Please describe what you mean by "not able to start" > > > > Cheers, > > Daniel > > > > -- > > [ https://www.firewall-services.com/ ] > > Daniel Berteaud > > FIREWALL-SERVICES SAS, La s?curit? des r?seaux > > Soci?t? de Services en Logiciels Libres > > T?l : +33.5 56 64 15 32 > > Matrix: @dani:fws.fr > > [ https://www.firewall-services.com/ | https://www.firewall-services.com > ] > > > > _______________________________________________ > > pve-user mailing list > > pve-user at pve.proxmox.com > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > -- > *This e-mail message is intended for the internal use of the intended > recipient(s) only. > The information contained herein is > confidential/privileged. Its disclosure or reproduction is strictly > prohibited. > If you are not the intended recipient, please inform the sender > immediately, do not disclose it internally or to third parties and destroy > it. > > In the course of our business relationship and for business purposes > only, Valeo may need to process some of your personal data. > For more > information, please refer to the Valeo Data Protection Statement and > Privacy notice available on Valeo.com > * > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user From sivakumar.saravanan.jv.ext at valeo-siemens.com Thu May 14 18:24:07 2020 From: sivakumar.saravanan.jv.ext at valeo-siemens.com (Sivakumar SARAVANAN) Date: Thu, 14 May 2020 18:24:07 +0200 Subject: [PVE-User] Some VM's are not able to start In-Reply-To: References: <1390593605.57004.1589471201696.JavaMail.zimbra@fws.fr> Message-ID: Thank you so much. What is the steps to disable the hugepage ? Best regards SK On Thu, May 14, 2020 at 6:20 PM Mark Adams via pve-user < pve-user at pve.proxmox.com> wrote: > > > > ---------- Forwarded message ---------- > From: Mark Adams > To: PVE User List > Cc: > Bcc: > Date: Thu, 14 May 2020 17:19:09 +0100 > Subject: Re: [PVE-User] Some VM's are not able to start > Do you really need hugepages? if not disable it. > > On Thu, 14 May 2020 at 17:17, Sivakumar SARAVANAN < > sivakumar.saravanan.jv.ext at valeo-siemens.com> wrote: > > > Hello Daniel, > > > > Thanks for coming back. > > > > I mean, I am unable to power ON the VM until shutdown the other VM's in > the > > same host. > > > > There are 6 VM's running on each Host, But sometimes all 6 VM's would run > > without any issue. But Sometimes if I stop ( Shutdown) and Power ON ( > > Start) getting an error saying as below. But each will have 32 GB memory. > > > > start failed: hugepage allocation failed at > > /usr/share/perl5/PVE/QemuServer/Memory.pm line 541. > > > > Appreciating your suggestion. > > > > > > > > > > Best regards > > SK > > > > On Thu, May 14, 2020 at 5:46 PM Daniel Berteaud < > > daniel at firewall-services.com> wrote: > > > > > > > > > > > ----- Le 14 Mai 20, ? 17:38, Sivakumar SARAVANAN > > > sivakumar.saravanan.jv.ext at valeo-siemens.com a ?crit : > > > > > > > Hello, > > > > > > > > We have implemented the Proxmox VE in our environment. > > > > > > > > So each server will have a maximum 6 VM. But not able to start the > few > > > VM's > > > > ON until we bring down the 1 or 2 VM's in the same Host. > > > > > > > > > > Please describe what you mean by "not able to start" > > > > > > Cheers, > > > Daniel > > > > > > -- > > > [ https://www.firewall-services.com/ ] > > > Daniel Berteaud > > > FIREWALL-SERVICES SAS, La s?curit? des r?seaux > > > Soci?t? de Services en Logiciels Libres > > > T?l : +33.5 56 64 15 32 > > > Matrix: @dani:fws.fr > > > [ https://www.firewall-services.com/ | > https://www.firewall-services.com > > ] > > > > > > _______________________________________________ > > > pve-user mailing list > > > pve-user at pve.proxmox.com > > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > > -- > > *This e-mail message is intended for the internal use of the intended > > recipient(s) only. > > The information contained herein is > > confidential/privileged. Its disclosure or reproduction is strictly > > prohibited. > > If you are not the intended recipient, please inform the sender > > immediately, do not disclose it internally or to third parties and > destroy > > it. > > > > In the course of our business relationship and for business purposes > > only, Valeo may need to process some of your personal data. > > For more > > information, please refer to the Valeo Data Protection Statement and > > Privacy notice available on Valeo.com > > * > > _______________________________________________ > > pve-user mailing list > > pve-user at pve.proxmox.com > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > ---------- Forwarded message ---------- > From: Mark Adams via pve-user > To: PVE User List > Cc: Mark Adams > Bcc: > Date: Thu, 14 May 2020 17:19:09 +0100 > Subject: Re: [PVE-User] Some VM's are not able to start > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * From gilberto.nunes32 at gmail.com Thu May 14 18:35:25 2020 From: gilberto.nunes32 at gmail.com (Gilberto Nunes) Date: Thu, 14 May 2020 13:35:25 -0300 Subject: [PVE-User] Some VM's are not able to start In-Reply-To: References: <1390593605.57004.1589471201696.JavaMail.zimbra@fws.fr> Message-ID: The preferred method to disable Transparent HugePages is to add "transparent_hugepage=never" to the kernel boot line in the "/etc/grub. conf" file. The server must be rebooted for this to take effect. --- Gilberto Nunes Ferreira Em qui., 14 de mai. de 2020 ?s 13:24, Sivakumar SARAVANAN < sivakumar.saravanan.jv.ext at valeo-siemens.com> escreveu: > Thank you so much. > > What is the steps to disable the hugepage ? > > > Best regards > SK > > On Thu, May 14, 2020 at 6:20 PM Mark Adams via pve-user < > pve-user at pve.proxmox.com> wrote: > > > > > > > > > ---------- Forwarded message ---------- > > From: Mark Adams > > To: PVE User List > > Cc: > > Bcc: > > Date: Thu, 14 May 2020 17:19:09 +0100 > > Subject: Re: [PVE-User] Some VM's are not able to start > > Do you really need hugepages? if not disable it. > > > > On Thu, 14 May 2020 at 17:17, Sivakumar SARAVANAN < > > sivakumar.saravanan.jv.ext at valeo-siemens.com> wrote: > > > > > Hello Daniel, > > > > > > Thanks for coming back. > > > > > > I mean, I am unable to power ON the VM until shutdown the other VM's in > > the > > > same host. > > > > > > There are 6 VM's running on each Host, But sometimes all 6 VM's would > run > > > without any issue. But Sometimes if I stop ( Shutdown) and Power ON ( > > > Start) getting an error saying as below. But each will have 32 GB > memory. > > > > > > start failed: hugepage allocation failed at > > > /usr/share/perl5/PVE/QemuServer/Memory.pm line 541. > > > > > > Appreciating your suggestion. > > > > > > > > > > > > > > > Best regards > > > SK > > > > > > On Thu, May 14, 2020 at 5:46 PM Daniel Berteaud < > > > daniel at firewall-services.com> wrote: > > > > > > > > > > > > > > > ----- Le 14 Mai 20, ? 17:38, Sivakumar SARAVANAN > > > > sivakumar.saravanan.jv.ext at valeo-siemens.com a ?crit : > > > > > > > > > Hello, > > > > > > > > > > We have implemented the Proxmox VE in our environment. > > > > > > > > > > So each server will have a maximum 6 VM. But not able to start the > > few > > > > VM's > > > > > ON until we bring down the 1 or 2 VM's in the same Host. > > > > > > > > > > > > > Please describe what you mean by "not able to start" > > > > > > > > Cheers, > > > > Daniel > > > > > > > > -- > > > > [ https://www.firewall-services.com/ ] > > > > Daniel Berteaud > > > > FIREWALL-SERVICES SAS, La s?curit? des r?seaux > > > > Soci?t? de Services en Logiciels Libres > > > > T?l : +33.5 56 64 15 32 > > > > Matrix: @dani:fws.fr > > > > [ https://www.firewall-services.com/ | > > https://www.firewall-services.com > > > ] > > > > > > > > _______________________________________________ > > > > pve-user mailing list > > > > pve-user at pve.proxmox.com > > > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > > > > > -- > > > *This e-mail message is intended for the internal use of the intended > > > recipient(s) only. > > > The information contained herein is > > > confidential/privileged. Its disclosure or reproduction is strictly > > > prohibited. > > > If you are not the intended recipient, please inform the sender > > > immediately, do not disclose it internally or to third parties and > > destroy > > > it. > > > > > > In the course of our business relationship and for business purposes > > > only, Valeo may need to process some of your personal data. > > > For more > > > information, please refer to the Valeo Data Protection Statement and > > > Privacy notice available on Valeo.com > > > * > > > _______________________________________________ > > > pve-user mailing list > > > pve-user at pve.proxmox.com > > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > > > ---------- Forwarded message ---------- > > From: Mark Adams via pve-user > > To: PVE User List > > Cc: Mark Adams > > Bcc: > > Date: Thu, 14 May 2020 17:19:09 +0100 > > Subject: Re: [PVE-User] Some VM's are not able to start > > _______________________________________________ > > pve-user mailing list > > pve-user at pve.proxmox.com > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > -- > *This e-mail message is intended for the internal use of the intended > recipient(s) only. > The information contained herein is > confidential/privileged. Its disclosure or reproduction is strictly > prohibited. > If you are not the intended recipient, please inform the sender > immediately, do not disclose it internally or to third parties and destroy > it. > > In the course of our business relationship and for business purposes > only, Valeo may need to process some of your personal data. > For more > information, please refer to the Valeo Data Protection Statement and > Privacy notice available on Valeo.com > * > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > From mark at openvs.co.uk Thu May 14 18:36:19 2020 From: mark at openvs.co.uk (Mark Adams) Date: Thu, 14 May 2020 17:36:19 +0100 Subject: [PVE-User] Some VM's are not able to start In-Reply-To: References: <1390593605.57004.1589471201696.JavaMail.zimbra@fws.fr> Message-ID: Remove the hugepages line from your vmid.conf (ie 100.conf) On Thu, 14 May 2020, 17:24 Sivakumar SARAVANAN, < sivakumar.saravanan.jv.ext at valeo-siemens.com> wrote: > Thank you so much. > > What is the steps to disable the hugepage ? > > > Best regards > SK > > On Thu, May 14, 2020 at 6:20 PM Mark Adams via pve-user < > pve-user at pve.proxmox.com> wrote: > > > > > > > > > ---------- Forwarded message ---------- > > From: Mark Adams > > To: PVE User List > > Cc: > > Bcc: > > Date: Thu, 14 May 2020 17:19:09 +0100 > > Subject: Re: [PVE-User] Some VM's are not able to start > > Do you really need hugepages? if not disable it. > > > > On Thu, 14 May 2020 at 17:17, Sivakumar SARAVANAN < > > sivakumar.saravanan.jv.ext at valeo-siemens.com> wrote: > > > > > Hello Daniel, > > > > > > Thanks for coming back. > > > > > > I mean, I am unable to power ON the VM until shutdown the other VM's in > > the > > > same host. > > > > > > There are 6 VM's running on each Host, But sometimes all 6 VM's would > run > > > without any issue. But Sometimes if I stop ( Shutdown) and Power ON ( > > > Start) getting an error saying as below. But each will have 32 GB > memory. > > > > > > start failed: hugepage allocation failed at > > > /usr/share/perl5/PVE/QemuServer/Memory.pm line 541. > > > > > > Appreciating your suggestion. > > > > > > > > > > > > > > > Best regards > > > SK > > > > > > On Thu, May 14, 2020 at 5:46 PM Daniel Berteaud < > > > daniel at firewall-services.com> wrote: > > > > > > > > > > > > > > > ----- Le 14 Mai 20, ? 17:38, Sivakumar SARAVANAN > > > > sivakumar.saravanan.jv.ext at valeo-siemens.com a ?crit : > > > > > > > > > Hello, > > > > > > > > > > We have implemented the Proxmox VE in our environment. > > > > > > > > > > So each server will have a maximum 6 VM. But not able to start the > > few > > > > VM's > > > > > ON until we bring down the 1 or 2 VM's in the same Host. > > > > > > > > > > > > > Please describe what you mean by "not able to start" > > > > > > > > Cheers, > > > > Daniel > > > > > > > > -- > > > > [ https://www.firewall-services.com/ ] > > > > Daniel Berteaud > > > > FIREWALL-SERVICES SAS, La s?curit? des r?seaux > > > > Soci?t? de Services en Logiciels Libres > > > > T?l : +33.5 56 64 15 32 > > > > Matrix: @dani:fws.fr > > > > [ https://www.firewall-services.com/ | > > https://www.firewall-services.com > > > ] > > > > > > > > _______________________________________________ > > > > pve-user mailing list > > > > pve-user at pve.proxmox.com > > > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > > > > > -- > > > *This e-mail message is intended for the internal use of the intended > > > recipient(s) only. > > > The information contained herein is > > > confidential/privileged. Its disclosure or reproduction is strictly > > > prohibited. > > > If you are not the intended recipient, please inform the sender > > > immediately, do not disclose it internally or to third parties and > > destroy > > > it. > > > > > > In the course of our business relationship and for business purposes > > > only, Valeo may need to process some of your personal data. > > > For more > > > information, please refer to the Valeo Data Protection Statement and > > > Privacy notice available on Valeo.com > > > * > > > _______________________________________________ > > > pve-user mailing list > > > pve-user at pve.proxmox.com > > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > > > ---------- Forwarded message ---------- > > From: Mark Adams via pve-user > > To: PVE User List > > Cc: Mark Adams > > Bcc: > > Date: Thu, 14 May 2020 17:19:09 +0100 > > Subject: Re: [PVE-User] Some VM's are not able to start > > _______________________________________________ > > pve-user mailing list > > pve-user at pve.proxmox.com > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > -- > *This e-mail message is intended for the internal use of the intended > recipient(s) only. > The information contained herein is > confidential/privileged. Its disclosure or reproduction is strictly > prohibited. > If you are not the intended recipient, please inform the sender > immediately, do not disclose it internally or to third parties and destroy > it. > > In the course of our business relationship and for business purposes > only, Valeo may need to process some of your personal data. > For more > information, please refer to the Valeo Data Protection Statement and > Privacy notice available on Valeo.com > * > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > From leesteken at pm.me Thu May 14 18:54:38 2020 From: leesteken at pm.me (leesteken at pm.me) Date: Thu, 14 May 2020 16:54:38 +0000 Subject: unable to hotplug cpulimit; cpu.* not in /sys? In-Reply-To: References: Message-ID: <5hiQtuBR2_w5q8NhKnwYAVVtm1AX6Fy2blU28hzNuKgWrRPXgJ6oBB9YvjKVE4nvcJd-RpP-vLTGq49JlS45n1-sHofL2E-BA6TCRgyBktg=@pm.me> Apologies for replying to myself, but I have a litte more information. ??????? Original Message ??????? On Tuesday, May 12, 2020 1:12 PM, wrote: > Hi PVE-users, > > Lately, on a no-subscription Proxmox (that I update regularly since release 6.0), I get the following error when changing the CPU limit of any (unpriviliged) container: > > Parameter verification failed. (400) > cpulimit: unable to hotplug cpulimit: unable to open file '/sys/fs/cgroup/memory///lxc/115/ns/cpu.cfs_period_us' > > - No such file or directory > > /etc/pve/lxc/115.conf: > arch: amd64 > console: 0 > cpulimit: 1.5 > hostname: vnc15 > memory: 2048 > #mp0: ... > net0: name=eth0,bridge=vmbr0,firewall=1,gw=172.17.2.1,hwaddr=52:54:56:17:02:15,ip=172.17.2.15/24,type=veth > onboot: 1 > ostype: debian > protection: 1 > rootfs: qpool-zfs:subvol-115-disk-0,size=3G > swap: 128 > unprivileged: 1 > > [pve:pending] > cpulimit: 1 > > Indeed, there are no cpu.* files in the directories of any of my containers. Is there something wrong with my setup? > > Maybe someone has run into the same problem or can tell me how to fix this? I just tested a fresh install of Proxmox VE 6.2 (in a VM of course) and I get the same error about not being able to hotplug cpulimit on a freshly installed (unprivileged) container (Debian 10). Why does Proxmox try to hotplug cpulimit if it is not possible? I also tested Proxmox VE 6.0 and it does not give an error. However, there are no cpu.* files in /sys/.. so maybe it just does not try to hotplug it? Maybe it never worked but only recently gives an error? Or maybe it is a regression? The work-around is to reboot the container, which is fine for me. I just want to inform you about this issue. kind regards, Arjen From sivakumar.saravanan.jv.ext at valeo-siemens.com Thu May 14 19:21:52 2020 From: sivakumar.saravanan.jv.ext at valeo-siemens.com (Sivakumar SARAVANAN) Date: Thu, 14 May 2020 19:21:52 +0200 Subject: [PVE-User] Some VM's are not able to start In-Reply-To: References: <1390593605.57004.1589471201696.JavaMail.zimbra@fws.fr> Message-ID: Hello, Thank you for your support. But I am getting below error message after removing the 'hugepages =2' line in the VM config file. kvm: -device vfio-pci,host=0000:5e:00.0,id=hostpci0,bus=pci.0,addr=0x10: vfio 0000:5e:00.0: failed to open /dev/vfio/46: Device or resource busy TASK ERROR: start failed: command '/usr/bin/kvm -id 145 -name HIL-System151 -chardev 'socket,id=qmp,path=/var/run/qemu-server/145.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/145.pid -daemonize -smbios 'type=1,uuid=750b5d38-2fa0-4a9b-8b0f-c8b2d5ef769d' -smp '4,sockets=2,cores=2,maxcpus=4' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vnc unix:/var/run/qemu-server/145.vnc,password -no-hpet -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,hv_ipi' -m 32768 -object 'memory-backend-ram,id=ram-node0,size=16384M' -numa 'node,nodeid=0,cpus=0-1,memdev=ram-node0' -object 'memory-backend-ram,id=ram-node1,size=16384M' -numa 'node,nodeid=1,cpus=2-3,memdev=ram-node1' -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'vmgenid,guid=e256bf4d-f862-47d0-a1ab-25821f586542' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'vfio-pci,host=0000:5e:00.0,id=hostpci0,bus=pci.0,addr=0x10' -device 'VGA,id=vga,bus=pci.0,addr=0x2' -chardev 'socket,path=/var/run/qemu-server/145.qga,server,nowait,id=qga0' -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:29191a75ef5e' -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -drive 'file=/dev/zvol/SSD-Storage-PRX002/vm-145-disk-0,if=none,id=drive-virtio0,cache=writethrough,format=raw,aio=threads,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -drive 'file=/dev/zvol/HDD-Storage-PRX002/vm-145-disk-1,if=none,id=drive-virtio1,cache=writethrough,format=raw,aio=threads,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio1,id=virtio1,bus=pci.0,addr=0xb' -netdev 'type=tap,id=net0,ifname=tap145i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown' -device 'e1000,mac=3E:DE:DC:87:CE:75,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -rtc 'driftfix=slew,base=localtime' -machine 'type=pc+pve1' -global 'kvm-pit.lost_tick_policy=discard'' failed: exit code 1 Best regards SK On Thu, May 14, 2020 at 6:37 PM Mark Adams via pve-user < pve-user at pve.proxmox.com> wrote: > > > > ---------- Forwarded message ---------- > From: Mark Adams > To: PVE User List > Cc: > Bcc: > Date: Thu, 14 May 2020 17:36:19 +0100 > Subject: Re: [PVE-User] Some VM's are not able to start > Remove the hugepages line from your vmid.conf (ie 100.conf) > > On Thu, 14 May 2020, 17:24 Sivakumar SARAVANAN, < > sivakumar.saravanan.jv.ext at valeo-siemens.com> wrote: > > > Thank you so much. > > > > What is the steps to disable the hugepage ? > > > > > > Best regards > > SK > > > > On Thu, May 14, 2020 at 6:20 PM Mark Adams via pve-user < > > pve-user at pve.proxmox.com> wrote: > > > > > > > > > > > > > > ---------- Forwarded message ---------- > > > From: Mark Adams > > > To: PVE User List > > > Cc: > > > Bcc: > > > Date: Thu, 14 May 2020 17:19:09 +0100 > > > Subject: Re: [PVE-User] Some VM's are not able to start > > > Do you really need hugepages? if not disable it. > > > > > > On Thu, 14 May 2020 at 17:17, Sivakumar SARAVANAN < > > > sivakumar.saravanan.jv.ext at valeo-siemens.com> wrote: > > > > > > > Hello Daniel, > > > > > > > > Thanks for coming back. > > > > > > > > I mean, I am unable to power ON the VM until shutdown the other VM's > in > > > the > > > > same host. > > > > > > > > There are 6 VM's running on each Host, But sometimes all 6 VM's would > > run > > > > without any issue. But Sometimes if I stop ( Shutdown) and Power ON ( > > > > Start) getting an error saying as below. But each will have 32 GB > > memory. > > > > > > > > start failed: hugepage allocation failed at > > > > /usr/share/perl5/PVE/QemuServer/Memory.pm line 541. > > > > > > > > Appreciating your suggestion. > > > > > > > > > > > > > > > > > > > > Best regards > > > > SK > > > > > > > > On Thu, May 14, 2020 at 5:46 PM Daniel Berteaud < > > > > daniel at firewall-services.com> wrote: > > > > > > > > > > > > > > > > > > > ----- Le 14 Mai 20, ? 17:38, Sivakumar SARAVANAN > > > > > sivakumar.saravanan.jv.ext at valeo-siemens.com a ?crit : > > > > > > > > > > > Hello, > > > > > > > > > > > > We have implemented the Proxmox VE in our environment. > > > > > > > > > > > > So each server will have a maximum 6 VM. But not able to start > the > > > few > > > > > VM's > > > > > > ON until we bring down the 1 or 2 VM's in the same Host. > > > > > > > > > > > > > > > > Please describe what you mean by "not able to start" > > > > > > > > > > Cheers, > > > > > Daniel > > > > > > > > > > -- > > > > > [ https://www.firewall-services.com/ ] > > > > > Daniel Berteaud > > > > > FIREWALL-SERVICES SAS, La s?curit? des r?seaux > > > > > Soci?t? de Services en Logiciels Libres > > > > > T?l : +33.5 56 64 15 32 > > > > > Matrix: @dani:fws.fr > > > > > [ https://www.firewall-services.com/ | > > > https://www.firewall-services.com > > > > ] > > > > > > > > > > _______________________________________________ > > > > > pve-user mailing list > > > > > pve-user at pve.proxmox.com > > > > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > > > > > > > > -- > > > > *This e-mail message is intended for the internal use of the intended > > > > recipient(s) only. > > > > The information contained herein is > > > > confidential/privileged. Its disclosure or reproduction is strictly > > > > prohibited. > > > > If you are not the intended recipient, please inform the sender > > > > immediately, do not disclose it internally or to third parties and > > > destroy > > > > it. > > > > > > > > In the course of our business relationship and for business purposes > > > > only, Valeo may need to process some of your personal data. > > > > For more > > > > information, please refer to the Valeo Data Protection Statement and > > > > Privacy notice available on Valeo.com > > > > * > > > > _______________________________________________ > > > > pve-user mailing list > > > > pve-user at pve.proxmox.com > > > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > > > > > > > ---------- Forwarded message ---------- > > > From: Mark Adams via pve-user > > > To: PVE User List > > > Cc: Mark Adams > > > Bcc: > > > Date: Thu, 14 May 2020 17:19:09 +0100 > > > Subject: Re: [PVE-User] Some VM's are not able to start > > > _______________________________________________ > > > pve-user mailing list > > > pve-user at pve.proxmox.com > > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > > -- > > *This e-mail message is intended for the internal use of the intended > > recipient(s) only. > > The information contained herein is > > confidential/privileged. Its disclosure or reproduction is strictly > > prohibited. > > If you are not the intended recipient, please inform the sender > > immediately, do not disclose it internally or to third parties and > destroy > > it. > > > > In the course of our business relationship and for business purposes > > only, Valeo may need to process some of your personal data. > > For more > > information, please refer to the Valeo Data Protection Statement and > > Privacy notice available on Valeo.com > > * > > _______________________________________________ > > pve-user mailing list > > pve-user at pve.proxmox.com > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > ---------- Forwarded message ---------- > From: Mark Adams via pve-user > To: PVE User List > Cc: Mark Adams > Bcc: > Date: Thu, 14 May 2020 17:36:19 +0100 > Subject: Re: [PVE-User] Some VM's are not able to start > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * From mark at openvs.co.uk Thu May 14 21:42:52 2020 From: mark at openvs.co.uk (Mark Adams) Date: Thu, 14 May 2020 20:42:52 +0100 Subject: [PVE-User] Some VM's are not able to start In-Reply-To: References: <1390593605.57004.1589471201696.JavaMail.zimbra@fws.fr> Message-ID: The info is in the first line of the log "kvm: -device vfio-pci,host=0000:5e:00.0,id=hostpci0,bus=pci.0,addr=0x10: vfio 0000:5e:00.0: failed to open /dev/vfio/46: Device or resource busy" This means the device is already passed through to another running VM, or it is being locked by something else. On Thu, 14 May 2020 at 18:22, Sivakumar SARAVANAN < sivakumar.saravanan.jv.ext at valeo-siemens.com> wrote: > Hello, > > Thank you for your support. > > But I am getting below error message after removing the 'hugepages =2' > line in the VM config file. > > kvm: -device vfio-pci,host=0000:5e:00.0,id=hostpci0,bus=pci.0,addr=0x10: > vfio 0000:5e:00.0: failed to open /dev/vfio/46: Device or resource busy > TASK ERROR: start failed: command '/usr/bin/kvm -id 145 -name > HIL-System151 -chardev > 'socket,id=qmp,path=/var/run/qemu-server/145.qmp,server,nowait' -mon > 'chardev=qmp,mode=control' -chardev > 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' -mon > 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/145.pid > -daemonize -smbios 'type=1,uuid=750b5d38-2fa0-4a9b-8b0f-c8b2d5ef769d' -smp > '4,sockets=2,cores=2,maxcpus=4' -nodefaults -boot > 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' > -vnc unix:/var/run/qemu-server/145.vnc,password -no-hpet -cpu > 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,hv_ipi' > -m 32768 -object 'memory-backend-ram,id=ram-node0,size=16384M' -numa > 'node,nodeid=0,cpus=0-1,memdev=ram-node0' -object > 'memory-backend-ram,id=ram-node1,size=16384M' -numa > 'node,nodeid=1,cpus=2-3,memdev=ram-node1' -device > 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device > 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device > 'vmgenid,guid=e256bf4d-f862-47d0-a1ab-25821f586542' -device > 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device > 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device > 'vfio-pci,host=0000:5e:00.0,id=hostpci0,bus=pci.0,addr=0x10' -device > 'VGA,id=vga,bus=pci.0,addr=0x2' -chardev > 'socket,path=/var/run/qemu-server/145.qga,server,nowait,id=qga0' -device > 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' -device > 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' -iscsi > 'initiator-name=iqn.1993-08.org.debian:01:29191a75ef5e' -drive > 'if=none,id=drive-ide2,media=cdrom,aio=threads' -device > 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -drive > 'file=/dev/zvol/SSD-Storage-PRX002/vm-145-disk-0,if=none,id=drive-virtio0,cache=writethrough,format=raw,aio=threads,detect-zeroes=on' > -device > 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' > -drive > 'file=/dev/zvol/HDD-Storage-PRX002/vm-145-disk-1,if=none,id=drive-virtio1,cache=writethrough,format=raw,aio=threads,detect-zeroes=on' > -device 'virtio-blk-pci,drive=drive-virtio1,id=virtio1,bus=pci.0,addr=0xb' > -netdev > 'type=tap,id=net0,ifname=tap145i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown' > -device > 'e1000,mac=3E:DE:DC:87:CE:75,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' > -rtc 'driftfix=slew,base=localtime' -machine 'type=pc+pve1' -global > 'kvm-pit.lost_tick_policy=discard'' failed: exit code 1 > > > Best regards > SK > > > On Thu, May 14, 2020 at 6:37 PM Mark Adams via pve-user < > pve-user at pve.proxmox.com> wrote: > >> >> >> >> ---------- Forwarded message ---------- >> From: Mark Adams >> To: PVE User List >> Cc: >> Bcc: >> Date: Thu, 14 May 2020 17:36:19 +0100 >> Subject: Re: [PVE-User] Some VM's are not able to start >> Remove the hugepages line from your vmid.conf (ie 100.conf) >> >> On Thu, 14 May 2020, 17:24 Sivakumar SARAVANAN, < >> sivakumar.saravanan.jv.ext at valeo-siemens.com> wrote: >> >> > Thank you so much. >> > >> > What is the steps to disable the hugepage ? >> > >> > >> > Best regards >> > SK >> > >> > On Thu, May 14, 2020 at 6:20 PM Mark Adams via pve-user < >> > pve-user at pve.proxmox.com> wrote: >> > >> > > >> > > >> > > >> > > ---------- Forwarded message ---------- >> > > From: Mark Adams >> > > To: PVE User List >> > > Cc: >> > > Bcc: >> > > Date: Thu, 14 May 2020 17:19:09 +0100 >> > > Subject: Re: [PVE-User] Some VM's are not able to start >> > > Do you really need hugepages? if not disable it. >> > > >> > > On Thu, 14 May 2020 at 17:17, Sivakumar SARAVANAN < >> > > sivakumar.saravanan.jv.ext at valeo-siemens.com> wrote: >> > > >> > > > Hello Daniel, >> > > > >> > > > Thanks for coming back. >> > > > >> > > > I mean, I am unable to power ON the VM until shutdown the other >> VM's in >> > > the >> > > > same host. >> > > > >> > > > There are 6 VM's running on each Host, But sometimes all 6 VM's >> would >> > run >> > > > without any issue. But Sometimes if I stop ( Shutdown) and Power ON >> ( >> > > > Start) getting an error saying as below. But each will have 32 GB >> > memory. >> > > > >> > > > start failed: hugepage allocation failed at >> > > > /usr/share/perl5/PVE/QemuServer/Memory.pm line 541. >> > > > >> > > > Appreciating your suggestion. >> > > > >> > > > >> > > > >> > > > >> > > > Best regards >> > > > SK >> > > > >> > > > On Thu, May 14, 2020 at 5:46 PM Daniel Berteaud < >> > > > daniel at firewall-services.com> wrote: >> > > > >> > > > > >> > > > > >> > > > > ----- Le 14 Mai 20, ? 17:38, Sivakumar SARAVANAN >> > > > > sivakumar.saravanan.jv.ext at valeo-siemens.com a ?crit : >> > > > > >> > > > > > Hello, >> > > > > > >> > > > > > We have implemented the Proxmox VE in our environment. >> > > > > > >> > > > > > So each server will have a maximum 6 VM. But not able to start >> the >> > > few >> > > > > VM's >> > > > > > ON until we bring down the 1 or 2 VM's in the same Host. >> > > > > > >> > > > > >> > > > > Please describe what you mean by "not able to start" >> > > > > >> > > > > Cheers, >> > > > > Daniel >> > > > > >> > > > > -- >> > > > > [ https://www.firewall-services.com/ ] >> > > > > Daniel Berteaud >> > > > > FIREWALL-SERVICES SAS, La s?curit? des r?seaux >> > > > > Soci?t? de Services en Logiciels Libres >> > > > > T?l : +33.5 56 64 15 32 >> > > > > Matrix: @dani:fws.fr >> > > > > [ https://www.firewall-services.com/ | >> > > https://www.firewall-services.com >> > > > ] >> > > > > >> > > > > _______________________________________________ >> > > > > pve-user mailing list >> > > > > pve-user at pve.proxmox.com >> > > > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >> > > > > >> > > > >> > > > -- >> > > > *This e-mail message is intended for the internal use of the >> intended >> > > > recipient(s) only. >> > > > The information contained herein is >> > > > confidential/privileged. Its disclosure or reproduction is strictly >> > > > prohibited. >> > > > If you are not the intended recipient, please inform the sender >> > > > immediately, do not disclose it internally or to third parties and >> > > destroy >> > > > it. >> > > > >> > > > In the course of our business relationship and for business purposes >> > > > only, Valeo may need to process some of your personal data. >> > > > For more >> > > > information, please refer to the Valeo Data Protection Statement and >> > > > Privacy notice available on Valeo.com >> > > > * >> > > > _______________________________________________ >> > > > pve-user mailing list >> > > > pve-user at pve.proxmox.com >> > > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >> > > >> > > >> > > >> > > ---------- Forwarded message ---------- >> > > From: Mark Adams via pve-user >> > > To: PVE User List >> > > Cc: Mark Adams >> > > Bcc: >> > > Date: Thu, 14 May 2020 17:19:09 +0100 >> > > Subject: Re: [PVE-User] Some VM's are not able to start >> > > _______________________________________________ >> > > pve-user mailing list >> > > pve-user at pve.proxmox.com >> > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >> > > >> > >> > -- >> > *This e-mail message is intended for the internal use of the intended >> > recipient(s) only. >> > The information contained herein is >> > confidential/privileged. Its disclosure or reproduction is strictly >> > prohibited. >> > If you are not the intended recipient, please inform the sender >> > immediately, do not disclose it internally or to third parties and >> destroy >> > it. >> > >> > In the course of our business relationship and for business purposes >> > only, Valeo may need to process some of your personal data. >> > For more >> > information, please refer to the Valeo Data Protection Statement and >> > Privacy notice available on Valeo.com >> > * >> > _______________________________________________ >> > pve-user mailing list >> > pve-user at pve.proxmox.com >> > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >> > >> >> >> >> ---------- Forwarded message ---------- >> From: Mark Adams via pve-user >> To: PVE User List >> Cc: Mark Adams >> Bcc: >> Date: Thu, 14 May 2020 17:36:19 +0100 >> Subject: Re: [PVE-User] Some VM's are not able to start >> _______________________________________________ >> pve-user mailing list >> pve-user at pve.proxmox.com >> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >> > > *This e-mail message is intended for the internal use of the intended recipient(s) only. > The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. > If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. > > In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. > For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * > > From uwe.sauter.de at gmail.com Thu May 14 22:22:01 2020 From: uwe.sauter.de at gmail.com (Uwe Sauter) Date: Thu, 14 May 2020 22:22:01 +0200 Subject: [PVE-User] Mellanox ConnectX-5 and SR-IOV Message-ID: Hi all, I had to change the hardware of one of my Proxmox installations and now have the problem that I cannot configure a Mellanox ConnectX-5 card for SR-IOV/passthrough. To be more precise I can boot the VM and it also recognizes the Infiniband device but I'm unable to assign a Node GUID and Port GUID for each of the virtual Infiniband devices. The old host had a ConnectX-3 card that "just worked". Did anyone run into this, too, and solve this issue? More details: I followed these two instructions: https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connectx-4-connectx-5-with-kvm--ethernet-x https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connect-ib-connectx-4-with-kvm--infiniband-x At about halfway through the second site I can't go on because the mentioned paths are not available in /sys: /sys/class/infiniband/mlx5_0/device/sriov/0/policy /sys/class/infiniband/mlx5_0/device/sriov/0/node /sys/class/infiniband/mlx5_0/device/sriov/0/port I also tried to install Mellanox OFED but the installation routine wants to uninstall many PVE related packages: pve-qemu-kvm ceph-mgr pve-ha-manager python-cephfs ceph-base libpve-storage-perl ceph librados2-perl libradosstriper1 python-rados librdmacm1:amd64 ceph-mon libpve-guest-common-perl rbd-nbd pve-manager glusterfs-client python-rgw libiscsi7:amd64 glusterfs-common fio libosmcomp4 python-ceph ceph-fuse pve-container spiceterm librgw2 qemu-server ceph-osd ceph-mds python-rbd proxmox-ve ceph-common ibverbs-providers:amd64 libcephfs2 librados2 librbd1 There is also this site https://docs.mellanox.com/pages/viewpage.action?pageId=12013542#SingleRootIOVirtualization(SR-IOV)-ConfiguringSR-IOVforConnectX-4/Connect-IB/ConnectX-5(InfiniBand) which mentiones a path that exists on my host: /sys/class/infiniband/mlx5_0/device/sriov_numvfs But two steps later they use a path that again doesn't exist: /sys/class/infiniband/mlx5_0/device/sriov/0/node My Proxmox installation details: # pveversion -v proxmox-ve: 6.2-1 (running kernel: 5.4.34-1-pve) pve-manager: 6.2-4 (running version: 6.2-4/9824574a) pve-kernel-5.4: 6.2-1 pve-kernel-helper: 6.2-1 pve-kernel-5.3: 6.1-6 pve-kernel-5.4.34-1-pve: 5.4.34-2 pve-kernel-5.3.18-3-pve: 5.3.18-3 ceph: 12.2.13-pve1 ceph-fuse: 12.2.13-pve1 corosync: 3.0.3-pve1 criu: 3.11-3 glusterfs-client: 5.5-3 ifupdown: 0.8.35+pve1 ksm-control-daemon: 1.3-1 libjs-extjs: 6.0.1-10 libknet1: 1.15-pve1 libproxmox-acme-perl: 1.0.3 libpve-access-control: 6.1-1 libpve-apiclient-perl: 3.0-3 libpve-common-perl: 6.1-2 libpve-guest-common-perl: 3.0-10 libpve-http-server-perl: 3.0-5 libpve-storage-perl: 6.1-7 libqb0: 1.0.5-1 libspice-server1: 0.14.2-4~pve6+1 lvm2: 2.03.02-pve4 lxc-pve: 4.0.2-1 lxcfs: 4.0.3-pve2 novnc-pve: 1.1.0-1 proxmox-mini-journalreader: 1.1-1 proxmox-widget-toolkit: 2.2-1 pve-cluster: 6.1-8 pve-container: 3.1-5 pve-docs: 6.2-4 pve-edk2-firmware: 2.20200229-1 pve-firewall: 4.1-2 pve-firmware: 3.1-1 pve-ha-manager: 3.0-9 pve-i18n: 2.1-2 pve-qemu-kvm: 5.0.0-2 pve-xtermjs: 4.3.0-1 qemu-server: 6.2-2 smartmontools: 7.1-pve2 spiceterm: 3.1-1 vncterm: 1.6-1 zfsutils-linux: 0.8.3-pve1 Regards, Uwe From chris.hofstaedtler at deduktiva.com Thu May 14 23:13:55 2020 From: chris.hofstaedtler at deduktiva.com (Chris Hofstaedtler | Deduktiva) Date: Thu, 14 May 2020 23:13:55 +0200 Subject: [PVE-User] Mellanox ConnectX-5 and SR-IOV In-Reply-To: References: Message-ID: <20200514211355.zi7clxvssiszwawq@zeha.at> * Uwe Sauter [200514 22:23]: [...] > More details: > > I followed these two instructions: > > https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connectx-4-connectx-5-with-kvm--ethernet-x > > https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connect-ib-connectx-4-with-kvm--infiniband-x > > At about halfway through the second site I can't go on because the mentioned paths are not available in /sys: > /sys/class/infiniband/mlx5_0/device/sriov/0/policy > /sys/class/infiniband/mlx5_0/device/sriov/0/node > /sys/class/infiniband/mlx5_0/device/sriov/0/port Disclaimer: I don't have the hardware you're talking about. However, I took a quick look at the mainline kernel drivers and the driver sources from mellanox.com - the mainline kernel does _NOT_ have the code for these files. I guess if you want to use that, you'll have to install the ofed driver from mellanox.com (preferably by starting with the sources for Ubuntu 20.04). PS: [1] looks like up to date documentation [1] https://docs.mellanox.com/pages/viewpage.action?pageId=25146819#SingleRootIOVirtualization(SR-IOV)-settingupsr-iov -- Chris Hofstaedtler / Deduktiva GmbH (FN 418592 b, HG Wien) www.deduktiva.com / +43 1 353 1707 From uwe.sauter.de at gmail.com Fri May 15 09:00:01 2020 From: uwe.sauter.de at gmail.com (Uwe Sauter) Date: Fri, 15 May 2020 09:00:01 +0200 Subject: [PVE-User] Mellanox ConnectX-5 and SR-IOV In-Reply-To: <20200514211355.zi7clxvssiszwawq@zeha.at> References: <20200514211355.zi7clxvssiszwawq@zeha.at> Message-ID: <73b6e696-d896-71c3-e79b-46d9a1b467b2@gmail.com> Chris, thanks for taking a look. Am 14.05.20 um 23:13 schrieb Chris Hofstaedtler | Deduktiva: > * Uwe Sauter [200514 22:23]: > [...] >> More details: >> >> I followed these two instructions: >> >> https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connectx-4-connectx-5-with-kvm--ethernet-x >> >> https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connect-ib-connectx-4-with-kvm--infiniband-x >> >> At about halfway through the second site I can't go on because the mentioned paths are not available in /sys: >> /sys/class/infiniband/mlx5_0/device/sriov/0/policy >> /sys/class/infiniband/mlx5_0/device/sriov/0/node >> /sys/class/infiniband/mlx5_0/device/sriov/0/port > > Disclaimer: I don't have the hardware you're talking about. > > However, I took a quick look at the mainline kernel drivers and the > driver sources from mellanox.com - the mainline kernel does _NOT_ > have the code for these files. > > I guess if you want to use that, you'll have to install the ofed > driver from mellanox.com (preferably by starting with the sources > for Ubuntu 20.04). As I mentioned I tried to install Mellanox OFED (for Debian) but it wants to uninstall many PVE related packages. If anyone has successfully installed MOFED on PVE 6 and can provide instructions, I'd be happy. > PS: [1] looks like up to date documentation > [1] https://docs.mellanox.com/pages/viewpage.action?pageId=25146819#SingleRootIOVirtualization(SR-IOV)-settingupsr-iov > This same document also mentions: """ Write to the sysfs file the number of Virtual Functions you need to create for the PF. You can use one of the following equivalent files: You can use one of the following equivalent files: - A standard Linux kernel generated file that is available in the new kernels. echo [num_vfs] > /sys/class/infiniband/mlx5_0/device/sriov_numvfs Note: This file will be generated only if IOMMU is set in the grub.conf file (by adding intel_iommu=on, as seen in the fourth step under ?Setting Up SR-IOV?). """ which I can see and following: """ - A file generated by the mlx5_core driver with the same functionality as the kernel generated one. echo [num_vfs] > /sys/class/infiniband/mlx5_0/device/mlx5_num_vfs Note: This file is used by old kernels that do not support the standard file. In such kernels, using sriov_numvfs results in the following error: ?bash: echo: write error: Function not implemented?. """ which I don't see. When I follow this instructions with echo 15 > /sys/class/infiniband/mlx5_0/device/sriov_numvfs I get entries in the form of /sys/class/infiniband/mlx5_0/device/virtfn{NUMBER_FROM_0_to_14} -> ../0000:5e:00.1 (and so forth) but those directories don't have files to set the necessary node and port GUIDs. Regards, Uwe From sivakumar.saravanan.jv.ext at valeo-siemens.com Fri May 15 16:02:40 2020 From: sivakumar.saravanan.jv.ext at valeo-siemens.com (Sivakumar SARAVANAN) Date: Fri, 15 May 2020 16:02:40 +0200 Subject: [PVE-User] no iommu detected please activate it.see documentation for further information Message-ID: Hello, I am unable to add the PCI device to VM's, where I am getting below error message. 'no iommu detected please activate it.see documentation for further information ' I also updated the grub as below. But not luck. / etc / defaults / grub GRUB_CMDLINE_LINUX_DEFAULT = "quiet intel_iommu = on" Appreciating your support. Best regards SK -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * From mark at openvs.co.uk Fri May 15 16:05:50 2020 From: mark at openvs.co.uk (Mark Adams) Date: Fri, 15 May 2020 15:05:50 +0100 Subject: [PVE-User] no iommu detected please activate it.see documentation for further information In-Reply-To: References: Message-ID: Have you enabled IOMMU in the BIOS? Assuming your server hardware supports it? On Fri, 15 May 2020 at 15:03, Sivakumar SARAVANAN < sivakumar.saravanan.jv.ext at valeo-siemens.com> wrote: > Hello, > > I am unable to add the PCI device to VM's, where I am getting below error > message. > > 'no iommu detected please activate it.see documentation for further > information ' > > I also updated the grub as below. But not luck. > > / etc / defaults / grub > > GRUB_CMDLINE_LINUX_DEFAULT = "quiet intel_iommu = on" > > Appreciating your support. > > Best regards > SK > > -- > *This e-mail message is intended for the internal use of the intended > recipient(s) only. > The information contained herein is > confidential/privileged. Its disclosure or reproduction is strictly > prohibited. > If you are not the intended recipient, please inform the sender > immediately, do not disclose it internally or to third parties and destroy > it. > > In the course of our business relationship and for business purposes > only, Valeo may need to process some of your personal data. > For more > information, please refer to the Valeo Data Protection Statement and > Privacy notice available on Valeo.com > * > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > From t.lamprecht at proxmox.com Fri May 15 16:29:10 2020 From: t.lamprecht at proxmox.com (Thomas Lamprecht) Date: Fri, 15 May 2020 16:29:10 +0200 Subject: [PVE-User] Mellanox ConnectX-5 and SR-IOV In-Reply-To: <73b6e696-d896-71c3-e79b-46d9a1b467b2@gmail.com> References: <20200514211355.zi7clxvssiszwawq@zeha.at> <73b6e696-d896-71c3-e79b-46d9a1b467b2@gmail.com> Message-ID: <64c67c7b-a3d2-6b2f-585d-e9c7908ed8d4@proxmox.com> On 5/15/20 9:00 AM, Uwe Sauter wrote: > Chris, > > thanks for taking a look. > > > Am 14.05.20 um 23:13 schrieb Chris Hofstaedtler | Deduktiva: >> * Uwe Sauter [200514 22:23]: >> [...] >>> More details: >>> >>> I followed these two instructions: >>> >>> https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connectx-4-connectx-5-with-kvm--ethernet-x >>> >>> https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connect-ib-connectx-4-with-kvm--infiniband-x >>> >>> At about halfway through the second site I can't go on because the mentioned paths are not available in /sys: >>> /sys/class/infiniband/mlx5_0/device/sriov/0/policy >>> /sys/class/infiniband/mlx5_0/device/sriov/0/node >>> /sys/class/infiniband/mlx5_0/device/sriov/0/port >> Disclaimer: I don't have the hardware you're talking about. >> >> However, I took a quick look at the mainline kernel drivers and the >> driver sources from mellanox.com - the mainline kernel does _NOT_ >> have the code for these files. >> >> I guess if you want to use that, you'll have to install the ofed >> driver from mellanox.com (preferably by starting with the sources >> for Ubuntu 20.04). > As I mentioned I tried to install Mellanox OFED (for Debian) but it wants to uninstall many PVE related packages. If anyone has > successfully installed MOFED on PVE 6 and can provide instructions, I'd be happy. > > I do not have this HW either but from a quick look you need to do two things: # apt install pve-headers (the script assumes that this is linux-headers-*, and thus fails here already) Open the "install.pl" file in an editor and search for the "sub uninstall" There add an early return: return 0; immediately after the opening { after that you should be able to build it. It tries to install it directly with dpkg and that may fail, if so try to install it manually with `apt install /path/to/package.deb ...other.deb` you may want to pass all debs from the DEBS sub directories to that one From sivakumar.saravanan.jv.ext at valeo-siemens.com Fri May 15 17:06:53 2020 From: sivakumar.saravanan.jv.ext at valeo-siemens.com (Sivakumar SARAVANAN) Date: Fri, 15 May 2020 17:06:53 +0200 Subject: [PVE-User] no iommu detected please activate it.see documentation for further information In-Reply-To: References: Message-ID: Hello, Yes. It is enabled already. But I am unable to add the PCI device to VM now. Appreciating your support. Regards SK On Fri, May 15, 2020 at 4:07 PM Mark Adams via pve-user < pve-user at pve.proxmox.com> wrote: > > > > ---------- Forwarded message ---------- > From: Mark Adams > To: PVE User List > Cc: > Bcc: > Date: Fri, 15 May 2020 15:05:50 +0100 > Subject: Re: [PVE-User] no iommu detected please activate it.see > documentation for further information > Have you enabled IOMMU in the BIOS? Assuming your server hardware supports > it? > > On Fri, 15 May 2020 at 15:03, Sivakumar SARAVANAN < > sivakumar.saravanan.jv.ext at valeo-siemens.com> wrote: > > > Hello, > > > > I am unable to add the PCI device to VM's, where I am getting below error > > message. > > > > 'no iommu detected please activate it.see documentation for further > > information ' > > > > I also updated the grub as below. But not luck. > > > > / etc / defaults / grub > > > > GRUB_CMDLINE_LINUX_DEFAULT = "quiet intel_iommu = on" > > > > Appreciating your support. > > > > Best regards > > SK > > > > -- > > *This e-mail message is intended for the internal use of the intended > > recipient(s) only. > > The information contained herein is > > confidential/privileged. Its disclosure or reproduction is strictly > > prohibited. > > If you are not the intended recipient, please inform the sender > > immediately, do not disclose it internally or to third parties and > destroy > > it. > > > > In the course of our business relationship and for business purposes > > only, Valeo may need to process some of your personal data. > > For more > > information, please refer to the Valeo Data Protection Statement and > > Privacy notice available on Valeo.com > > * > > _______________________________________________ > > pve-user mailing list > > pve-user at pve.proxmox.com > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > ---------- Forwarded message ---------- > From: Mark Adams via pve-user > To: PVE User List > Cc: Mark Adams > Bcc: > Date: Fri, 15 May 2020 15:05:50 +0100 > Subject: Re: [PVE-User] no iommu detected please activate it.see > documentation for further information > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * From gianni.milo22 at gmail.com Fri May 15 19:18:27 2020 From: gianni.milo22 at gmail.com (Gianni Milo) Date: Fri, 15 May 2020 18:18:27 +0100 Subject: [PVE-User] Mellanox ConnectX-5 and SR-IOV In-Reply-To: <64c67c7b-a3d2-6b2f-585d-e9c7908ed8d4@proxmox.com> References: <20200514211355.zi7clxvssiszwawq@zeha.at> <73b6e696-d896-71c3-e79b-46d9a1b467b2@gmail.com> <64c67c7b-a3d2-6b2f-585d-e9c7908ed8d4@proxmox.com> Message-ID: I don't own this hardware too, but was able to compile its kernel modules (.ko) from its source package which is provided at Mellanox web site. I used a test PVE (VM) with pve-kernel and pve-headers installed on it. Then I extracted the source package (tar.gz) on temp location and executed install.pl script which took a while to complete. That produced a couple of kernel modules within /lib/modules//updates/dkms. I guess that from that point it's just a case of loading appropriate modules with modprobe command. G. On Fri, 15 May 2020 at 15:29, Thomas Lamprecht wrote: > On 5/15/20 9:00 AM, Uwe Sauter wrote: > > Chris, > > > > thanks for taking a look. > > > > > > Am 14.05.20 um 23:13 schrieb Chris Hofstaedtler | Deduktiva: > >> * Uwe Sauter [200514 22:23]: > >> [...] > >>> More details: > >>> > >>> I followed these two instructions: > >>> > >>> > https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connectx-4-connectx-5-with-kvm--ethernet-x > >>> > >>> > https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connect-ib-connectx-4-with-kvm--infiniband-x > >>> > >>> At about halfway through the second site I can't go on because the > mentioned paths are not available in /sys: > >>> /sys/class/infiniband/mlx5_0/device/sriov/0/policy > >>> /sys/class/infiniband/mlx5_0/device/sriov/0/node > >>> /sys/class/infiniband/mlx5_0/device/sriov/0/port > >> Disclaimer: I don't have the hardware you're talking about. > >> > >> However, I took a quick look at the mainline kernel drivers and the > >> driver sources from mellanox.com - the mainline kernel does _NOT_ > >> have the code for these files. > >> > >> I guess if you want to use that, you'll have to install the ofed > >> driver from mellanox.com (preferably by starting with the sources > >> for Ubuntu 20.04). > > As I mentioned I tried to install Mellanox OFED (for Debian) but it > wants to uninstall many PVE related packages. If anyone has > > successfully installed MOFED on PVE 6 and can provide instructions, I'd > be happy. > > > > > > I do not have this HW either but from a quick look you need to do two > things: > > # apt install pve-headers > (the script assumes that this is linux-headers-*, and thus fails here > already) > > > Open the "install.pl" file in an editor and search for the "sub uninstall" > There add an early return: > > return 0; > > immediately after the opening { after that you should be able to build it. > > It tries to install it directly with dpkg and that may fail, if so try to > install it manually with `apt install /path/to/package.deb ...other.deb` > you may want to pass all debs from the DEBS sub directories to that one > > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > From uwe.sauter.de at gmail.com Fri May 15 19:43:20 2020 From: uwe.sauter.de at gmail.com (Uwe Sauter) Date: Fri, 15 May 2020 19:43:20 +0200 Subject: [PVE-User] Mellanox ConnectX-5 and SR-IOV In-Reply-To: References: <20200514211355.zi7clxvssiszwawq@zeha.at> <73b6e696-d896-71c3-e79b-46d9a1b467b2@gmail.com> <64c67c7b-a3d2-6b2f-585d-e9c7908ed8d4@proxmox.com> Message-ID: <98fe380c-4428-2384-5621-7507b25fef80@gmail.com> Gianni, Thomas, thanks for your time. I mitigated the issue by exchanging the Infiniband card with an older model that still uses the mlx4 driver. Still had some issued finding the correct module settings for this piece of hardware but finally got it working. Maybe some time, when I have a spare minute I'll try your proposals. Have a relaxing weekend, Uwe Am 15.05.20 um 19:18 schrieb Gianni Milo: > I don't own this hardware too, but was able to compile its kernel modules (.ko) from its source package which is > provided at Mellanox web site. > I used a test PVE (VM) with pve-kernel and pve-headers installed on it. Then I extracted the ? source package (tar.gz) > on temp location and executed install.pl script which took a while to complete. > That produced a couple of kernel modules within /lib/modules//updates/dkms. > ?I guess that from that point it's just a case of loading appropriate modules with modprobe command. > > G. > > > On Fri, 15 May 2020 at 15:29, Thomas Lamprecht > wrote: > > On 5/15/20 9:00 AM, Uwe Sauter wrote: > > Chris, > > > > thanks for taking a look. > > > > > > Am 14.05.20 um 23:13 schrieb Chris Hofstaedtler | Deduktiva: > >> * Uwe Sauter > [200514 22:23]: > >> [...] > >>> More details: > >>> > >>> I followed these two instructions: > >>> > >>> https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connectx-4-connectx-5-with-kvm--ethernet-x > >>> > >>> https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connect-ib-connectx-4-with-kvm--infiniband-x > >>> > >>> At about halfway through the second site I can't go on because the mentioned paths are not available in /sys: > >>> /sys/class/infiniband/mlx5_0/device/sriov/0/policy > >>> /sys/class/infiniband/mlx5_0/device/sriov/0/node > >>> /sys/class/infiniband/mlx5_0/device/sriov/0/port > >> Disclaimer: I don't have the hardware you're talking about. > >> > >> However, I took a quick look at the mainline kernel drivers and the > >> driver sources from mellanox.com - the mainline kernel does _NOT_ > >> have the code for these files. > >> > >> I guess if you want to use that, you'll have to install the ofed > >> driver from mellanox.com (preferably by starting with the sources > >> for Ubuntu 20.04). > > As I mentioned I tried to install Mellanox OFED (for Debian) but it wants to uninstall many PVE related packages. > If anyone has > > successfully installed MOFED on PVE 6 and can provide instructions, I'd be happy. > > > > > > I do not have this HW either but from a quick look you need to do two > things: > > # apt install pve-headers > (the script assumes that this is linux-headers-*, and thus fails here already) > > > Open the "install.pl " file in an editor and search for the "sub uninstall" > There add an early return: > > return 0; > > immediately after the opening { after that you should be able to build it. > > It tries to install it directly with dpkg and that may fail, if so try to > install it manually with `apt install /path/to/package.deb ...other.deb` > you may want to pass all debs from the DEBS sub directories to that one > > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > From proxmox at elchaka.de Sun May 17 10:49:52 2020 From: proxmox at elchaka.de (proxmox at elchaka.de) Date: Sun, 17 May 2020 10:49:52 +0200 Subject: [PVE-User] unable to hotplug cpulimit; cpu.* not in /sys? In-Reply-To: References: Message-ID: I am not sure if this is working with lxc... In Addition die you set hotplug enabled for CPU under options through the webgui for you container? I See this for all of my kvm... Hth Mehmet Am 12. Mai 2020 13:12:40 MESZ schrieb leesteken--- via pve-user : >_______________________________________________ >pve-user mailing list >pve-user at pve.proxmox.com >https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user From leesteken at pm.me Sun May 17 11:35:39 2020 From: leesteken at pm.me (leesteken at pm.me) Date: Sun, 17 May 2020 09:35:39 +0000 Subject: [PVE-User] unable to hotplug cpulimit; cpu.* not in /sys? In-Reply-To: References: Message-ID: <5gr6erE6EGpGiQr6lKBta5cH5iPZgsWYjm7u7CXMW-kZzOSLWxWlwCBNUa2ZVgepJVsIYPmBhbQGREdsUbaFLQ8gn1IqX1bX_eMHULo7ud8=@pm.me> On Sunday, May 17, 2020 10:49 AM, wrote: > I am not sure if this is working with lxc... > In Addition die you set hotplug enabled for CPU under options through the webgui for you container? I See this for all of my kvm... > I don't see any hotplug options for LXC, only for KVM. I get the error for LXC, not for KVM. Changing the CPU limit for KVM virtual machine while it is running does not give an error, even when CPU hotplug is not enabled. Using Proxmox 6.x, I do not see any /sys/fs/cgroup/memory/lxc/*/ns/cpu.* files, which is what changing the CPU limit (while a LXC container is running) complains about. Now that I look at the /sys/fs-path closely, I think I can explain the error: Proxmox looks for cpu.rt_period_us in the /sys/fs/cgroup/MEMORY... , while it is actually in the /sys/fs/cgroup/CPU.... I do believe this is a typo/bug in Proxmox VE code. kind regards, Arjen From leesteken at pm.me Sun May 17 11:57:47 2020 From: leesteken at pm.me (leesteken at pm.me) Date: Sun, 17 May 2020 09:57:47 +0000 Subject: [PVE-User] unable to hotplug cpulimit; cpu.* not in /sys? In-Reply-To: References: Message-ID: <6H5AG9pLIg0Xd3OfJClZo2KKQ6s_D3jIP7UKZd7NBn8GRZR2tFqLxEqLr1j4ZZYmFbL9rE03abl5hfEZcItLLf52H4IayELvfEEY8tmZvs4=@pm.me> On Sunday, May 17, 2020 11:35 AM, leesteken--- via pve-user wrote: > On Sunday, May 17, 2020 10:49 AM, proxmox at elchaka.de wrote: > > > I am not sure if this is working with lxc... > > In Addition die you set hotplug enabled for CPU under options through the webgui for you container? I See this for all of my kvm... > > I don't see any hotplug options for LXC, only for KVM. I get the error for LXC, not for KVM. > Changing the CPU limit for KVM virtual machine while it is running does not give an error, even when CPU hotplug is not enabled. > > Using Proxmox 6.x, I do not see any /sys/fs/cgroup/memory/lxc//ns/cpu. files, which is what changing the CPU limit (while a LXC container is running) complains about. > > Now that I look at the /sys/fs-path closely, I think I can explain the error: Proxmox looks for cpu.rt_period_us in the /sys/fs/cgroup/MEMORY... , while it is actually in the /sys/fs/cgroup/CPU.... > I do believe this is a typo/bug in Proxmox VE code. Proxmox VE developers have fixed it already: https://git.proxmox.com/?p=pve-container.git;a=commitdiff;h=04affe4b80c4a4d004e8db29f075270e7dde8447 @@ -449,7 +449,7 @@ sub change_cpu_quota { die "quota without period not allowed\n" if !defined($period) && defined($quota); - my ($path, $ver) = $self->get_path('memory'); + my ($path, $ver) = $self->get_path('cpu', 1); if (!defined($path)) { die "trying to change cpu quota cgroup values: container not running\n"; } elsif ($ver == 2) { Thank you, Arjen From gaio at sv.lnf.it Mon May 18 10:43:16 2020 From: gaio at sv.lnf.it (Marco Gaiarin) Date: Mon, 18 May 2020 10:43:16 +0200 Subject: [PVE-User] PVE 6, wireless and regulatory database... Message-ID: <20200518084316.GC3626@lilliput.linux.it> OK, i admit that running a wireless card on a virtualization host server is a bit strange, but... it is my home server! ;-) After upgrading to 6.2 (but the same was on 6.1, i've simply missed that...) i've seen: May 17 23:47:55 ino kernel: [ 7.839970] cfg80211: Loading compiled-in X.509 certificates for regulatory database May 17 23:47:55 ino kernel: [ 7.848525] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2 May 17 23:47:55 ino kernel: [ 7.848527] cfg80211: failed to load regulatory.db AFAI've understood by google, the current wireless-regdb buster package (2016.06.10-1) is old for the PVE kernel. Can i ask to 'backport' it as for other hardware-support package? I know is nothing more then a warning, but... thanks. PS: i've not tried to install simply from debian backport, package seems available: https://tracker.debian.org/pkg/wireless-regdb -- dott. Marco Gaiarin GNUPG Key ID: 240A3D66 Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ Polo FVG - Via della Bont?, 7 - 33078 - San Vito al Tagliamento (PN) marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA) From martin at proxmox.com Mon May 18 12:28:11 2020 From: martin at proxmox.com (Martin Maurer) Date: Mon, 18 May 2020 12:28:11 +0200 Subject: [PVE-User] PVE 6, wireless and regulatory database... In-Reply-To: <20200518084316.GC3626@lilliput.linux.it> References: <20200518084316.GC3626@lilliput.linux.it> Message-ID: Hello, use the buster-backports - https://packages.debian.org/buster-backports/wireless-regdb (see also https://backports.debian.org/) On 5/18/20 10:43 AM, Marco Gaiarin wrote: > > OK, i admit that running a wireless card on a virtualization host > server is a bit strange, but... it is my home server! ;-) > > > After upgrading to 6.2 (but the same was on 6.1, i've simply missed > that...) i've seen: > > May 17 23:47:55 ino kernel: [ 7.839970] cfg80211: Loading compiled-in X.509 certificates for regulatory database > May 17 23:47:55 ino kernel: [ 7.848525] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2 > May 17 23:47:55 ino kernel: [ 7.848527] cfg80211: failed to load regulatory.db > > AFAI've understood by google, the current wireless-regdb buster package > (2016.06.10-1) is old for the PVE kernel. > > Can i ask to 'backport' it as for other hardware-support package? > > > I know is nothing more then a warning, but... thanks. > > > PS: i've not tried to install simply from debian backport, package > seems available: > https://tracker.debian.org/pkg/wireless-regdb > -- Best Regards, Martin Maurer martin at proxmox.com https://www.proxmox.com From ralvarado at anycast.cl Tue May 19 09:06:38 2020 From: ralvarado at anycast.cl (Roberto Alvarado) Date: Tue, 19 May 2020 07:06:38 +0000 Subject: [PVE-User] freebsd12 and cloud-init - password problem Message-ID: <010001722bc176c1-709d30de-2b83-4539-aeff-76aef40b1dfd-000000@email.amazonses.com> Hi Folks! I?m trying to create a freebsd12 cloud-init image, but the network config is OK, the hostname is OK, the DNS are OK and the partition expansion is OK, BUT the password for the local user wont work, tried with the root account and the freebsd account (this account was created with cloudinit too) if I check the /etc/master.passwd , the password for the cloudinit user is empty, for this example the user was freebsd # cat /etc/master.passwd # $FreeBSD: releng/12.1/etc/master.passwd 337882 2018-08-15 23:18:34Z brd $ # root:**hidden**:0:0::0:0:Charlie &:/root:/bin/csh toor:*:0:0::0:0:Bourne-again Superuser:/root: daemon:*:1:1::0:0:Owner of many system processes:/root:/usr/sbin/nologin operator:*:2:5::0:0:System &:/:/usr/sbin/nologin bin:*:3:7::0:0:Binaries Commands and Source:/:/usr/sbin/nologin tty:*:4:65533::0:0:Tty Sandbox:/:/usr/sbin/nologin kmem:*:5:65533::0:0:KMem Sandbox:/:/usr/sbin/nologin games:*:7:13::0:0:Games pseudo-user:/:/usr/sbin/nologin news:*:8:8::0:0:News Subsystem:/:/usr/sbin/nologin man:*:9:9::0:0:Mister Man Pages:/usr/share/man:/usr/sbin/nologin sshd:*:22:22::0:0:Secure Shell Daemon:/var/empty:/usr/sbin/nologin smmsp:*:25:25::0:0:Sendmail Submission User:/var/spool/clientmqueue:/usr/sbin/nologin mailnull:*:26:26::0:0:Sendmail Default User:/var/spool/mqueue:/usr/sbin/nologin bind:*:53:53::0:0:Bind Sandbox:/:/usr/sbin/nologin unbound:*:59:59::0:0:Unbound DNS Resolver:/var/unbound:/usr/sbin/nologin proxy:*:62:62::0:0:Packet Filter pseudo-user:/nonexistent:/usr/sbin/nologin _pflogd:*:64:64::0:0:pflogd privsep user:/var/empty:/usr/sbin/nologin _dhcp:*:65:65::0:0:dhcp programs:/var/empty:/usr/sbin/nologin uucp:*:66:66::0:0:UUCP pseudo-user:/var/spool/uucppublic:/usr/local/libexec/uucp/uucico pop:*:68:6::0:0:Post Office Owner:/nonexistent:/usr/sbin/nologin auditdistd:*:78:77::0:0:Auditdistd unprivileged user:/var/empty:/usr/sbin/nologin www:*:80:80::0:0:World Wide Web Owner:/nonexistent:/usr/sbin/nologin ntpd:*:123:123::0:0:NTP Daemon:/var/db/ntp:/usr/sbin/nologin _ypldap:*:160:160::0:0:YP LDAP unprivileged user:/var/empty:/usr/sbin/nologin hast:*:845:845::0:0:HAST unprivileged user:/var/empty:/usr/sbin/nologin nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/usr/sbin/nologin _tss:*:601:601:daemon:0:0:TCG Software Stack user:/var/empty:/usr/sbin/nologin git_daemon:*:964:964::0:0:git daemon:/nonexistent:/usr/sbin/nologin freebsd:*:1001:1001::0:0:FreeBSD:/usr/home/freebsd:/bin/tcsh root at freebsd:/var/lib/cloud/data # If I try with the root account the result is the same, the account lose the password: root at test1-f12:/var/log # cat /etc/master.passwd # $FreeBSD: releng/12.1/etc/master.passwd 337882 2018-08-15 23:18:34Z brd $ # root:*:0:0::0:0:Charlie &:/root:/bin/csh toor:*:0:0::0:0:Bourne-again Superuser:/root: daemon:*:1:1::0:0:Owner of many system processes:/root:/usr/sbin/nologin operator:*:2:5::0:0:System &:/:/usr/sbin/nologin bin:*:3:7::0:0:Binaries Commands and Source:/:/usr/sbin/nologin tty:*:4:65533::0:0:Tty Sandbox:/:/usr/sbin/nologin kmem:*:5:65533::0:0:KMem Sandbox:/:/usr/sbin/nologin games:*:7:13::0:0:Games pseudo-user:/:/usr/sbin/nologin news:*:8:8::0:0:News Subsystem:/:/usr/sbin/nologin man:*:9:9::0:0:Mister Man Pages:/usr/share/man:/usr/sbin/nologin sshd:*:22:22::0:0:Secure Shell Daemon:/var/empty:/usr/sbin/nologin smmsp:*:25:25::0:0:Sendmail Submission User:/var/spool/clientmqueue:/usr/sbin/nologin mailnull:*:26:26::0:0:Sendmail Default User:/var/spool/mqueue:/usr/sbin/nologin bind:*:53:53::0:0:Bind Sandbox:/:/usr/sbin/nologin unbound:*:59:59::0:0:Unbound DNS Resolver:/var/unbound:/usr/sbin/nologin proxy:*:62:62::0:0:Packet Filter pseudo-user:/nonexistent:/usr/sbin/nologin _pflogd:*:64:64::0:0:pflogd privsep user:/var/empty:/usr/sbin/nologin _dhcp:*:65:65::0:0:dhcp programs:/var/empty:/usr/sbin/nologin uucp:*:66:66::0:0:UUCP pseudo-user:/var/spool/uucppublic:/usr/local/libexec/uucp/uucico pop:*:68:6::0:0:Post Office Owner:/nonexistent:/usr/sbin/nologin auditdistd:*:78:77::0:0:Auditdistd unprivileged user:/var/empty:/usr/sbin/nologin www:*:80:80::0:0:World Wide Web Owner:/nonexistent:/usr/sbin/nologin ntpd:*:123:123::0:0:NTP Daemon:/var/db/ntp:/usr/sbin/nologin _ypldap:*:160:160::0:0:YP LDAP unprivileged user:/var/empty:/usr/sbin/nologin hast:*:845:845::0:0:HAST unprivileged user:/var/empty:/usr/sbin/nologin nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/usr/sbin/nologin _tss:*:601:601:daemon:0:0:TCG Software Stack user:/var/empty:/usr/sbin/nologin git_daemon:*:964:964::0:0:git daemon:/nonexistent:/usr/sbin/nologin root at test1-f12:/var/log # can be a problem with the hash used by proxmox for the password? Cloud init log with a try for the root user: https://pastebin.com/1GTdXx6W Another try but now for the freebsd user: https://pastebin.com/RxX6VpQQ For the cloudinit install, I tried with the pkg version(pkg install net/cloud-init) and from ports, included the version 20.2. root at freebsd:~ # cloud-init -v /usr/local/bin/cloud-init 20.1 root at freebsd:~ # root at freebsd:~ # freebsd-version 12.1-RELEASE-p4 root at freebsd:~ # root at freebsd:~ # cat /usr/local/etc/cloud/cloud.cfg # The top level settings are used as module # and system configuration. syslog_fix_perms: root:wheel # A set of users which may be applied and/or used by various modules # when a 'default' entry is found it will reference the 'default_user' # from the distro configuration specified below users: - freebsd # If this is set, 'root' will not be able to ssh in and they # will get a message to login instead as the default $user disable_root: false # This will cause the set+update hostname module to not operate (if true) preserve_hostname: false # This should not be required, but leave it in place until the real cause of # not finding -any- datasources is resolved. datasource_list: ['NoCloud', 'ConfigDrive', 'Azure', 'OpenStack', 'Ec2'] # Example datasource config # datasource: # Ec2: # metadata_urls: [ 'blah.com' ] # timeout: 5 # (defaults to 50 seconds) # max_wait: 10 # (defaults to 120 seconds) # The modules that run in the 'init' stage cloud_init_modules: - migrator - seed_random - bootcmd - write-files - growpart - resizefs - set_hostname - update_hostname - users-groups - ssh # The modules that run in the 'config' stage cloud_config_modules: - ssh-import-id - locale - set-passwords - timezone - disable-ec2-metadata - runcmd # The modules that run in the 'final' stage cloud_final_modules: - package-update-upgrade-install - salt-minion - rightscale_userdata - scripts-vendor - scripts-per-once - scripts-per-boot - scripts-per-instance - scripts-user - ssh-authkey-fingerprints - keys-to-console - phone-home - final-message - power-state-change # System and/or distro specific settings # (not accessible to handlers/transforms) system_info: # This will affect which distro class gets used distro: freebsd # Default user name + that default users groups (if added/used) default_user: name: freebsd lock_passwd: False gecos: FreeBSD groups: [wheel] sudo: ["ALL=(ALL) NOPASSWD:ALL"] shell: /bin/tcsh root at freebsd:~ # Info from the proxmox node: root at px:~# pveversion -v proxmox-ve: 6.2-1 (running kernel: 5.3.18-2-pve) pve-manager: 6.2-4 (running version: 6.2-4/9824574a) pve-kernel-5.4: 6.2-1 pve-kernel-helper: 6.2-1 pve-kernel-5.3: 6.1-6 pve-kernel-5.0: 6.0-11 pve-kernel-5.4.34-1-pve: 5.4.34-2 pve-kernel-4.15: 5.4-8 pve-kernel-5.3.18-3-pve: 5.3.18-3 pve-kernel-5.3.18-2-pve: 5.3.18-2 pve-kernel-5.3.13-2-pve: 5.3.13-2 pve-kernel-5.0.21-5-pve: 5.0.21-10 pve-kernel-4.15.18-20-pve: 4.15.18-46 pve-kernel-4.15.18-9-pve: 4.15.18-30 ceph-fuse: 12.2.11+dfsg1-2.1+b1 corosync: 3.0.3-pve1 criu: 3.11-3 glusterfs-client: 5.5-3 ifupdown: 0.8.35+pve1 ksm-control-daemon: 1.3-1 libjs-extjs: 6.0.1-10 libknet1: 1.15-pve1 libproxmox-acme-perl: 1.0.3 libpve-access-control: 6.1-1 libpve-apiclient-perl: 3.0-3 libpve-common-perl: 6.1-2 libpve-guest-common-perl: 3.0-10 libpve-http-server-perl: 3.0-5 libpve-storage-perl: 6.1-7 libqb0: 1.0.5-1 libspice-server1: 0.14.2-4~pve6+1 lvm2: 2.03.02-pve4 lxc-pve: 4.0.2-1 lxcfs: 4.0.3-pve2 novnc-pve: 1.1.0-1 proxmox-mini-journalreader: 1.1-1 proxmox-widget-toolkit: 2.2-1 pve-cluster: 6.1-8 pve-container: 3.1-5 pve-docs: 6.2-4 pve-edk2-firmware: 2.20200229-1 pve-firewall: 4.1-2 pve-firmware: 3.1-1 pve-ha-manager: 3.0-9 pve-i18n: 2.1-2 pve-qemu-kvm: 5.0.0-2 pve-xtermjs: 4.3.0-1 qemu-server: 6.2-2 smartmontools: 7.1-pve2 spiceterm: 3.1-1 vncterm: 1.6-1 zfsutils-linux: 0.8.3-pve1 root at px:~# Thanks Regards Roberto From gaio at sv.lnf.it Tue May 19 09:40:11 2020 From: gaio at sv.lnf.it (Marco Gaiarin) Date: Tue, 19 May 2020 09:40:11 +0200 Subject: [PVE-User] PVE 6, wireless and regulatory database... In-Reply-To: References: <20200518084316.GC3626@lilliput.linux.it> Message-ID: <20200519074011.GB4975@lilliput.linux.it> Mandi! Martin Maurer In chel di` si favelave... > use the buster-backports - https://packages.debian.org/buster-backports/wireless-regdb > (see also https://backports.debian.org/) Seems is not sufficient: May 18 23:55:24 ino kernel: [ 7.751088] cfg80211: Loading compiled-in X.509 certificates for regulatory database May 18 23:55:24 ino kernel: [ 7.757944] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' May 18 23:55:24 ino kernel: [ 7.764861] cfg80211: loaded regulatory.db is malformed or signature is missing/invalid May 18 23:55:24 ino kernel: [ 7.905519] ath: EEPROM regdomain: 0x809c May 18 23:55:24 ino kernel: [ 7.905520] ath: EEPROM indicates we should expect a country code May 18 23:55:24 ino kernel: [ 7.905521] ath: doing EEPROM country->regdmn map search May 18 23:55:24 ino kernel: [ 7.905521] ath: country maps to regdmn code: 0x52 May 18 23:55:24 ino kernel: [ 7.905522] ath: Country alpha2 being used: CN May 18 23:55:24 ino kernel: [ 7.905523] ath: Regpair used: 0x52 May 18 23:55:24 ino kernel: [ 7.909904] ieee80211 phy0: Selected rate control algorithm 'minstrel_ht' May 18 23:55:24 ino kernel: [ 7.916935] ieee80211 phy0: Atheros AR9287 Rev:2 mem=0xffffb2508cdd0000, irq=16 May 18 23:55:24 ino kernel: [ 7.922095] ath9k 0000:10:00.0 wls1: renamed from wlan0 slightly different error, but still regulatory.db is not loaded. This seems a bit strange; kernel 5.4 is the focal fossa kernel, and in focal fossa the 'wireless-regdb' package is dated '2018': https://packages.ubuntu.com/search?suite=focal&searchon=names&keywords=wireless-regdb while debian backport is 2019. Package have to be recompiled against the kernel to have key match? -- dott. Marco Gaiarin GNUPG Key ID: 240A3D66 Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ Polo FVG - Via della Bont?, 7 - 33078 - San Vito al Tagliamento (PN) marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA) From jm at ginernet.com Tue May 19 10:53:55 2020 From: jm at ginernet.com (=?UTF-8?Q?Jos=c3=a9_Manuel_Giner?=) Date: Tue, 19 May 2020 10:53:55 +0200 Subject: [PVE-User] Recovering cluster after shutdown Message-ID: <17cf8c91-bbc7-6eef-8fcc-20ae91743231@ginernet.com> Hello, in the past (proxmox v4 and v5) we've used Proxmox's clustering features and found problems when the whole cluster would shut down, when we turned it back on it wouldn't synchronize. Has this problem been fixed yet? Thanks! -- Jos? Manuel Giner https://ginernet.com From a.antreich at proxmox.com Tue May 19 11:04:09 2020 From: a.antreich at proxmox.com (Alwin Antreich) Date: Tue, 19 May 2020 11:04:09 +0200 Subject: [PVE-User] PVE 6, wireless and regulatory database... In-Reply-To: <20200519074011.GB4975@lilliput.linux.it> References: <20200518084316.GC3626@lilliput.linux.it> <20200519074011.GB4975@lilliput.linux.it> Message-ID: <20200519090409.GA406757@dona.proxmox.com> On Tue, May 19, 2020 at 09:40:11AM +0200, Marco Gaiarin wrote: > Mandi! Martin Maurer > In chel di` si favelave... > > > use the buster-backports - https://packages.debian.org/buster-backports/wireless-regdb > > (see also https://backports.debian.org/) > > Seems is not sufficient: > > May 18 23:55:24 ino kernel: [ 7.751088] cfg80211: Loading compiled-in X.509 certificates for regulatory database > May 18 23:55:24 ino kernel: [ 7.757944] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' > May 18 23:55:24 ino kernel: [ 7.764861] cfg80211: loaded regulatory.db is malformed or signature is missing/invalid > May 18 23:55:24 ino kernel: [ 7.905519] ath: EEPROM regdomain: 0x809c > May 18 23:55:24 ino kernel: [ 7.905520] ath: EEPROM indicates we should expect a country code > May 18 23:55:24 ino kernel: [ 7.905521] ath: doing EEPROM country->regdmn map search > May 18 23:55:24 ino kernel: [ 7.905521] ath: country maps to regdmn code: 0x52 > May 18 23:55:24 ino kernel: [ 7.905522] ath: Country alpha2 being used: CN > May 18 23:55:24 ino kernel: [ 7.905523] ath: Regpair used: 0x52 > May 18 23:55:24 ino kernel: [ 7.909904] ieee80211 phy0: Selected rate control algorithm 'minstrel_ht' > May 18 23:55:24 ino kernel: [ 7.916935] ieee80211 phy0: Atheros AR9287 Rev:2 mem=0xffffb2508cdd0000, irq=16 > May 18 23:55:24 ino kernel: [ 7.922095] ath9k 0000:10:00.0 wls1: renamed from wlan0 > > slightly different error, but still regulatory.db is not loaded. > > > This seems a bit strange; kernel 5.4 is the focal fossa kernel, and in > focal fossa the 'wireless-regdb' package is dated '2018': > > https://packages.ubuntu.com/search?suite=focal&searchon=names&keywords=wireless-regdb > > while debian backport is 2019. > > > Package have to be recompiled against the kernel to have key match? The above message says that the signature might be missing/invalid. Debian uses a different file name for the signature file then ubuntu. See the file list for comparison. https://packages.ubuntu.com/focal/all/wireless-regdb/filelist https://packages.debian.org/buster-backports/all/wireless-regdb/filelist You can always download the latest wireless-regdb and replace the db + signature files. https://wireless.wiki.kernel.org/en/developers/regulatory/wireless-regdb -- Cheers, Alwin From mike at oeg.com.au Tue May 19 11:11:30 2020 From: mike at oeg.com.au (Mike O'Connor) Date: Tue, 19 May 2020 18:41:30 +0930 Subject: [PVE-User] Recovering cluster after shutdown In-Reply-To: <17cf8c91-bbc7-6eef-8fcc-20ae91743231@ginernet.com> References: <17cf8c91-bbc7-6eef-8fcc-20ae91743231@ginernet.com> Message-ID: <614d4e1d-fd6a-83bc-9def-76e6fe91d235@oeg.com.au> On 19/5/20 6:23 pm, Jos? Manuel Giner wrote: > Hello, in the past (proxmox v4 and v5) we've used Proxmox's clustering > features and found problems when the whole cluster would shut down, > when we turned it back on it wouldn't synchronize. Has this problem > been fixed yet? > > Thanks! > Generally its never been a problem with 4,5 or 6. There must have been something causing an issue in your particularly case. Mike From dietmar at proxmox.com Tue May 19 12:25:32 2020 From: dietmar at proxmox.com (Dietmar Maurer) Date: Tue, 19 May 2020 12:25:32 +0200 (CEST) Subject: [PVE-User] Recovering cluster after shutdown In-Reply-To: <17cf8c91-bbc7-6eef-8fcc-20ae91743231@ginernet.com> References: <17cf8c91-bbc7-6eef-8fcc-20ae91743231@ginernet.com> Message-ID: <1732134091.30.1589883932699@webmail.proxmox.com> What bug number do you talk about exactly? > Hello, in the past (proxmox v4 and v5) we've used Proxmox's clustering > features and found problems when the whole cluster would shut down, when > we turned it back on it wouldn't synchronize. Has this problem been > fixed yet? From f.cuseo at panservice.it Wed May 20 09:41:19 2020 From: f.cuseo at panservice.it (Fabrizio Cuseo) Date: Wed, 20 May 2020 09:41:19 +0200 (CEST) Subject: [PVE-User] Problem with virtio ethernet drivers PVE 6.2-4 and CentOS 5.6 Message-ID: <548633797.221279.1589960479488.JavaMail.zimbra@zimbra.panservice.it> Hello. I have updated a PVE cluster from 6.1.X to last (this night), and a VM with CentOS 5.6 and virtio ethernet drivers, after some minutes (few) the ethernet stop responding. With a VM restart, the ethernet starts working again, but again after some minutes stop working. Changing from virtio to e1000 seems working fine... Someone else with this problem ? Thanks, Fabrizio -- --- Fabrizio Cuseo - mailto:f.cuseo at panservice.it Direzione Generale - Panservice InterNetWorking Servizi Professionali per Internet ed il Networking Panservice e' associata AIIP - RIPE Local Registry Phone: +39 0773 410020 - Fax: +39 0773 470219 http://www.panservice.it mailto:info at panservice.it Numero verde nazionale: 800 901492 From gaio at sv.lnf.it Wed May 20 22:58:16 2020 From: gaio at sv.lnf.it (Marco Gaiarin) Date: Wed, 20 May 2020 22:58:16 +0200 Subject: [PVE-User] PVE 6, wireless and regulatory database... In-Reply-To: <20200519090409.GA406757@dona.proxmox.com> References: <20200518084316.GC3626@lilliput.linux.it> <20200519074011.GB4975@lilliput.linux.it> <20200519090409.GA406757@dona.proxmox.com> Message-ID: <20200520205816.GA23561@lilliput.linux.it> Mandi! Alwin Antreich In chel di` si favelave... > Debian uses a different file name for the signature file then ubuntu. A-HA! > You can always download the latest wireless-regdb and replace the db + > signature files. > https://wireless.wiki.kernel.org/en/developers/regulatory/wireless-regdb I've downloaded and installed by hand ubuntu package (wireless-regdb_2018.05.09-0ubuntu1_all.deb) and installed flawlessy. Now regulatory DB load: May 20 22:35:03 ino kernel: [ 7.795925] cfg80211: Loading compiled-in X.509 certificates for regulatory database May 20 22:35:03 ino kernel: [ 7.816343] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' May 20 22:35:03 ino kernel: [ 7.961683] ath: EEPROM regdomain: 0x809c May 20 22:35:03 ino kernel: [ 7.961684] ath: EEPROM indicates we should expect a country code May 20 22:35:03 ino kernel: [ 7.961685] ath: doing EEPROM country->regdmn map search May 20 22:35:03 ino kernel: [ 7.961686] ath: country maps to regdmn code: 0x52 May 20 22:35:03 ino kernel: [ 7.961686] ath: Country alpha2 being used: CN May 20 22:35:03 ino kernel: [ 7.961687] ath: Regpair used: 0x52 May 20 22:35:03 ino kernel: [ 7.963108] ieee80211 phy0: Selected rate control algorithm 'minstrel_ht' May 20 22:35:03 ino kernel: [ 7.964097] ieee80211 phy0: Atheros AR9287 Rev:2 mem=0xffffa51c0d300000, irq=16 May 20 22:35:03 ino kernel: [ 7.981453] ath9k 0000:10:00.0 wls1: renamed from wlan0 it make sense to fire up a bug in PVE BTS, to have this package added in repository? Thanks. -- dott. Marco Gaiarin GNUPG Key ID: 240A3D66 Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ Polo FVG - Via della Bont?, 7 - 33078 - San Vito al Tagliamento (PN) marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA) From gaio at sv.lnf.it Thu May 21 12:06:50 2020 From: gaio at sv.lnf.it (Marco Gaiarin) Date: Thu, 21 May 2020 12:06:50 +0200 Subject: [PVE-User] PVE 6, wireless and regulatory database... In-Reply-To: <20200520205816.GA23561@lilliput.linux.it> References: <20200518084316.GC3626@lilliput.linux.it> <20200519074011.GB4975@lilliput.linux.it> <20200519090409.GA406757@dona.proxmox.com> <20200520205816.GA23561@lilliput.linux.it> Message-ID: <20200521100650.GE4671@lilliput.linux.it> > it make sense to fire up a bug in PVE BTS, to have this package added > in repository? Anyway, done: https://bugzilla.proxmox.com/show_bug.cgi?id=2753 -- dott. Marco Gaiarin GNUPG Key ID: 240A3D66 Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ Polo FVG - Via della Bont?, 7 - 33078 - San Vito al Tagliamento (PN) marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA) From a.antreich at proxmox.com Fri May 22 12:04:21 2020 From: a.antreich at proxmox.com (Alwin Antreich) Date: Fri, 22 May 2020 12:04:21 +0200 Subject: [PVE-User] PVE 6, wireless and regulatory database... In-Reply-To: <20200520205816.GA23561@lilliput.linux.it> References: <20200518084316.GC3626@lilliput.linux.it> <20200519074011.GB4975@lilliput.linux.it> <20200519090409.GA406757@dona.proxmox.com> <20200520205816.GA23561@lilliput.linux.it> Message-ID: <20200522100421.GA1577721@dona.proxmox.com> On Wed, May 20, 2020 at 10:58:16PM +0200, Marco Gaiarin wrote: > Mandi! Alwin Antreich > In chel di` si favelave... > > > Debian uses a different file name for the signature file then ubuntu. > > A-HA! > > > > You can always download the latest wireless-regdb and replace the db + > > signature files. > > https://wireless.wiki.kernel.org/en/developers/regulatory/wireless-regdb > > I've downloaded and installed by hand ubuntu package > (wireless-regdb_2018.05.09-0ubuntu1_all.deb) and installed flawlessy. > > Now regulatory DB load: > > May 20 22:35:03 ino kernel: [ 7.795925] cfg80211: Loading compiled-in X.509 certificates for regulatory database > May 20 22:35:03 ino kernel: [ 7.816343] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' > May 20 22:35:03 ino kernel: [ 7.961683] ath: EEPROM regdomain: 0x809c > May 20 22:35:03 ino kernel: [ 7.961684] ath: EEPROM indicates we should expect a country code > May 20 22:35:03 ino kernel: [ 7.961685] ath: doing EEPROM country->regdmn map search > May 20 22:35:03 ino kernel: [ 7.961686] ath: country maps to regdmn code: 0x52 > May 20 22:35:03 ino kernel: [ 7.961686] ath: Country alpha2 being used: CN > May 20 22:35:03 ino kernel: [ 7.961687] ath: Regpair used: 0x52 > May 20 22:35:03 ino kernel: [ 7.963108] ieee80211 phy0: Selected rate control algorithm 'minstrel_ht' > May 20 22:35:03 ino kernel: [ 7.964097] ieee80211 phy0: Atheros AR9287 Rev:2 mem=0xffffa51c0d300000, irq=16 > May 20 22:35:03 ino kernel: [ 7.981453] ath9k 0000:10:00.0 wls1: renamed from wlan0 > > it make sense to fire up a bug in PVE BTS, to have this package added > in repository? It is not an issue with the package. I forgot about the alternatives in Debian (thanks Thomas). Once you set the alternative (tool: update-alternatives) to the regulatory.db-upstream it will be loaded without complaining. I suppose Debian can't switch to the signed database yet, since their shipped kernel version might not yet know how to handle the signed regulatory.db. -- Cheers, Alwin From f.thommen at dkfz-heidelberg.de Fri May 22 16:03:01 2020 From: f.thommen at dkfz-heidelberg.de (Frank Thommen) Date: Fri, 22 May 2020 16:03:01 +0200 Subject: [PVE-User] Some features that I miss in the PVE WebUI Message-ID: Dear all, having worked with oVirt in the past there are some features that I am really missing in PVE in my daily work: a) a tabular overview over all virtual machines. This should/might also include some performance data and the description. See the attached partial screenshot from oVirt, where this is implemented quite nicely. This is /not/ to replace proper monitoring but to provide a quick overview over the PVE-based intrastructure a1) the possibility to provide the virtual machines and containers with a short, one-line description b) the possibility to use keywords from the Notes field or the description (see a1 above) in the search box. Our hosts are all named -vm which forces us to keep a separate list for the mapping of services to hostnames It would be great to see these features in some future PVE release. Cheers, Frank From t.lamprecht at proxmox.com Fri May 22 16:10:19 2020 From: t.lamprecht at proxmox.com (Thomas Lamprecht) Date: Fri, 22 May 2020 16:10:19 +0200 Subject: [PVE-User] Some features that I miss in the PVE WebUI In-Reply-To: References: Message-ID: <7d70d084-decb-928e-dc1f-48372a3b217f@proxmox.com> Hi, On 5/22/20 4:03 PM, Frank Thommen wrote: > Dear all, > > having worked with oVirt in the past there are some features that I am really missing in PVE in my daily work: > > a) a tabular overview over all virtual machines. This should/might also include some performance data and the description.? See the attached partial screenshot from oVirt, where this is implemented quite nicely. This is /not/ to replace proper monitoring but to provide a quick overview over the PVE-based intrastructure Isn't this what Datacenter -> Search could be? Note that the list drops any attachments. > > a1) the possibility to provide the virtual machines and containers with a short, one-line description Why not use the VM/CT notes? Editable over VM/CT -> Summary panel? > > b) the possibility to use keywords from the Notes field or the description (see a1 above) in the search box.? Our hosts are all named -vm which forces us to keep a separate list for the mapping of services to hostnames Dominik has done some patches adding "tags", which then would be searchable. Some backend support is there, but we had some discussion about how to integrate them in the frontend. I think this will be picked up soonish and should provide what you seek in b) maybe also a1.. cheers, Thomas From f.thommen at dkfz-heidelberg.de Fri May 22 22:11:55 2020 From: f.thommen at dkfz-heidelberg.de (Frank Thommen) Date: Fri, 22 May 2020 22:11:55 +0200 Subject: [PVE-User] Some features that I miss in the PVE WebUI In-Reply-To: <7d70d084-decb-928e-dc1f-48372a3b217f@proxmox.com> References: <7d70d084-decb-928e-dc1f-48372a3b217f@proxmox.com> Message-ID: <9ab10312-d0bb-43ba-a374-c732140e3c01@dkfz-heidelberg.de> On 22/05/2020 16:10, Thomas Lamprecht wrote: > Hi, > > On 5/22/20 4:03 PM, Frank Thommen wrote: >> Dear all, >> >> having worked with oVirt in the past there are some features that I am really missing in PVE in my daily work: >> >> a) a tabular overview over all virtual machines. This should/might also include some performance data and the description.? See the attached partial screenshot from oVirt, where this is implemented quite nicely. This is /not/ to replace proper monitoring but to provide a quick overview over the PVE-based intrastructure > > Isn't this what Datacenter -> Search could be? Note that the list drops any attachments. Partially. It is still missing some data that I really like in the oVirt overview (e.g. comments/descriptions, IP and the thumbnailed timelines(!) for CPU, memory and network ecc.) I have uploaded the screenshot to https://pasteboard.co/J9B4cEc.png. >> a1) the possibility to provide the virtual machines and containers with a short, one-line description > > Why not use the VM/CT notes? Editable over VM/CT -> Summary panel? We do use these notes heavily, but unfortunately they don't seem to be searchable/usable for filtering. It would be nice to have a "Description" which would be a one-liner, like the description for the HA resources. >> b) the possibility to use keywords from the Notes field or the description (see a1 above) in the search box.? Our hosts are all named -vm which forces us to keep a separate list for the mapping of services to hostnames > > Dominik has done some patches adding "tags", which then would be searchable. > Some backend support is there, but we had some discussion about how to integrate > them in the frontend. I think this will be picked up soonish and should provide > what you seek in b) maybe also a1.. yes, such tags would come in very handy and could partially replace the description if they can be used for filtering. Frank > > cheers, > Thomas > > From sivakumar.saravanan.jv.ext at valeo-siemens.com Mon May 25 14:43:07 2020 From: sivakumar.saravanan.jv.ext at valeo-siemens.com (Sivakumar SARAVANAN) Date: Mon, 25 May 2020 14:43:07 +0200 Subject: [PVE-User] ZFS Storage Issue Message-ID: Hello, I created the ZFS HDD disk with mirrored instead of the SSD mirrored disk by mistake. I did not get the option to edit the HDD ZFS storage that I created by mistake. So I ran the below comment zpool destroy HDD-Storage-PRX003 But I am not getting a free pool to create SSD mirrored disk again after running the above command. Appreciating your support. Mit freundlichen Gr??en / Best regards / Cordialement, Sivakumar SARAVANAN Externer Dienstleister f?r / External service provider for Valeo Siemens eAutomotive Germany GmbH Research & Development R & D SWENG TE 1 INFTE Frauenauracher Stra?e 85 91056 Erlangen, Germany Tel.: +49 9131 9892 0000 Mobile: +49 176 7698 5441 sivakumar.saravanan.jv.ext at valeo-siemens.com valeo-siemens.com Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger Schwab, Michael Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * From lindsay.mathieson at gmail.com Mon May 25 15:48:40 2020 From: lindsay.mathieson at gmail.com (Lindsay Mathieson) Date: Mon, 25 May 2020 23:48:40 +1000 Subject: [PVE-User] ZFS Storage Issue In-Reply-To: References: Message-ID: On 25/05/2020 10:43 pm, Sivakumar SARAVANAN wrote: > So I ran the below comment > > zpool destroy HDD-Storage-PRX003 > > But I am not getting a free pool to create SSD mirrored disk again after > running the above command. I believe you need to remove the partitioning on the disks which zfs created first, there's usually two partitions on the disk. -- Lindsay From sivakumar.saravanan.jv.ext at valeo-siemens.com Mon May 25 16:13:12 2020 From: sivakumar.saravanan.jv.ext at valeo-siemens.com (Sivakumar SARAVANAN) Date: Mon, 25 May 2020 16:13:12 +0200 Subject: [PVE-User] ZFS Storage Issue In-Reply-To: References: Message-ID: Hello, I need to add 2 additional mirrored disk as below 1. SSB ZFS with 855 GB (2 disk) 2. HDD ZFS with 2 TB ( 2 disk) As I created as above already. But I wanted to remove all the mirrored disk that I created already. I used this command - zpool destroy HDD-Storage-PRX003 I want to create a new mirrored disk now. But I can see any free disk now to create a new ZFS mirrored disk. Kindly suggest. Mit freundlichen Gr??en / Best regards / Cordialement, Sivakumar SARAVANAN Externer Dienstleister f?r / External service provider for Valeo Siemens eAutomotive Germany GmbH Research & Development R & D SWENG TE 1 INFTE Frauenauracher Stra?e 85 91056 Erlangen, Germany Tel.: +49 9131 9892 0000 Mobile: +49 176 7698 5441 sivakumar.saravanan.jv.ext at valeo-siemens.com valeo-siemens.com Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger Schwab, Michael Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 On Mon, May 25, 2020 at 3:49 PM Lindsay Mathieson < lindsay.mathieson at gmail.com> wrote: > On 25/05/2020 10:43 pm, Sivakumar SARAVANAN wrote: > > So I ran the below comment > > > > zpool destroy HDD-Storage-PRX003 > > > > But I am not getting a free pool to create SSD mirrored disk again after > > running the above command. > > I believe you need to remove the partitioning on the disks which zfs > created first, there's usually two partitions on the disk. > > -- > Lindsay > > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * From gaio at sv.lnf.it Tue May 26 12:44:30 2020 From: gaio at sv.lnf.it (Marco Gaiarin) Date: Tue, 26 May 2020 12:44:30 +0200 Subject: [PVE-User] PVE 6, wireless and regulatory database... In-Reply-To: <20200522100421.GA1577721@dona.proxmox.com> References: <20200518084316.GC3626@lilliput.linux.it> <20200519074011.GB4975@lilliput.linux.it> <20200519090409.GA406757@dona.proxmox.com> <20200520205816.GA23561@lilliput.linux.it> <20200522100421.GA1577721@dona.proxmox.com> Message-ID: <20200526104430.GK3717@lilliput.linux.it> Mandi! Alwin Antreich In chel di` si favelave... > It is not an issue with the package. I forgot about the alternatives in > Debian (thanks Thomas). Once you set the alternative (tool: > update-alternatives) to the regulatory.db-upstream it will be loaded > without complaining. > I suppose Debian can't switch to the signed database yet, since their > shipped kernel version might not yet know how to handle the signed > regulatory.db. root at ino:~# dpkg -l | grep wireless-regdb ii wireless-regdb 2016.06.10-1 all wireless regulatory database root at ino:~# update-alternatives --display wireless-regdb update-alternatives: error: no alternatives for wireless-regdb But: root at ino:~# update-alternatives --display regulatory.db update-alternatives: warning: alternative /lib/firmware/regulatory.db-debian (part of link group regulatory.db) doesn't exist; removing from list of alternatives update-alternatives: warning: alternative /lib/firmware/regulatory.db-upstream (part of link group regulatory.db) doesn't exist; removing from list of alternatives regulatory.db - auto mode link best version not available link currently points to /lib/firmware/regulatory.db-debian link regulatory.db is /lib/firmware/regulatory.db slave regulatory.db.p7s is /lib/firmware/regulatory.db.p7s but anyway: root at ino:~# update-alternatives --config regulatory.db update-alternatives: error: no alternatives for regulatory.db (boh, seems also /lib/firmware/regulatory.db-debian is missing... strange...) I need to download manually the regulatory DB? -- dott. Marco Gaiarin GNUPG Key ID: 240A3D66 Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ Polo FVG - Via della Bont?, 7 - 33078 - San Vito al Tagliamento (PN) marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA) From a.antreich at proxmox.com Tue May 26 13:35:14 2020 From: a.antreich at proxmox.com (Alwin Antreich) Date: Tue, 26 May 2020 13:35:14 +0200 Subject: [PVE-User] PVE 6, wireless and regulatory database... In-Reply-To: <20200526104430.GK3717@lilliput.linux.it> References: <20200518084316.GC3626@lilliput.linux.it> <20200519074011.GB4975@lilliput.linux.it> <20200519090409.GA406757@dona.proxmox.com> <20200520205816.GA23561@lilliput.linux.it> <20200522100421.GA1577721@dona.proxmox.com> <20200526104430.GK3717@lilliput.linux.it> Message-ID: <20200526113514.GB477557@dona.proxmox.com> On Tue, May 26, 2020 at 12:44:30PM +0200, Marco Gaiarin wrote: > Mandi! Alwin Antreich > In chel di` si favelave... > > > It is not an issue with the package. I forgot about the alternatives in > > Debian (thanks Thomas). Once you set the alternative (tool: > > update-alternatives) to the regulatory.db-upstream it will be loaded > > without complaining. > > I suppose Debian can't switch to the signed database yet, since their > > shipped kernel version might not yet know how to handle the signed > > regulatory.db. > > root at ino:~# dpkg -l | grep wireless-regdb > ii wireless-regdb 2016.06.10-1 all wireless regulatory database > root at ino:~# update-alternatives --display wireless-regdb > update-alternatives: error: no alternatives for wireless-regdb > > But: > > root at ino:~# update-alternatives --display regulatory.db > update-alternatives: warning: alternative /lib/firmware/regulatory.db-debian (part of link group regulatory.db) doesn't exist; removing from list of alternatives > update-alternatives: warning: alternative /lib/firmware/regulatory.db-upstream (part of link group regulatory.db) doesn't exist; removing from list of alternatives > regulatory.db - auto mode > link best version not available > link currently points to /lib/firmware/regulatory.db-debian > link regulatory.db is /lib/firmware/regulatory.db > slave regulatory.db.p7s is /lib/firmware/regulatory.db.p7s > > but anyway: > > root at ino:~# update-alternatives --config regulatory.db > update-alternatives: error: no alternatives for regulatory.db > > (boh, seems also /lib/firmware/regulatory.db-debian is missing... > strange...) > > > I need to download manually the regulatory DB? You need the Debian package for the alternatives to work. IIRC, you finally installed the ubuntu package. ``` update-alternatives --display regulatory.db ``` -- Cheers, Alwin From gaio at sv.lnf.it Tue May 26 17:31:46 2020 From: gaio at sv.lnf.it (Marco Gaiarin) Date: Tue, 26 May 2020 17:31:46 +0200 Subject: [PVE-User] PVE 6, wireless and regulatory database... In-Reply-To: <20200526113514.GB477557@dona.proxmox.com> References: <20200518084316.GC3626@lilliput.linux.it> <20200519074011.GB4975@lilliput.linux.it> <20200519090409.GA406757@dona.proxmox.com> <20200520205816.GA23561@lilliput.linux.it> <20200522100421.GA1577721@dona.proxmox.com> <20200526104430.GK3717@lilliput.linux.it> <20200526113514.GB477557@dona.proxmox.com> Message-ID: <20200526153146.GL3717@lilliput.linux.it> Mandi! Alwin Antreich In chel di` si favelave... > > root at ino:~# dpkg -l | grep wireless-regdb > > ii wireless-regdb 2016.06.10-1 all wireless regulatory database > You need the Debian package for the alternatives to work. IIRC, you > finally installed the ubuntu package. https://packages.debian.org/buster/wireless-regdb I've installed the buster package... -- dott. Marco Gaiarin GNUPG Key ID: 240A3D66 Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ Polo FVG - Via della Bont?, 7 - 33078 - San Vito al Tagliamento (PN) marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA) From harrim4n at harrim4n.com Tue May 26 19:13:05 2020 From: harrim4n at harrim4n.com (harrim4n) Date: Tue, 26 May 2020 19:13:05 +0200 Subject: [PVE-User] ZFS Storage Issue In-Reply-To: References: Message-ID: <5ee30502-8e57-e912-202a-f72b99f93c5e@harrim4n.com> Hi, I'm not sure if I'm understanding you correctly, so I'll summarize what I got that you did: 1. Created a ZFS pool with one mirror vdev 2. Noticed that you accidentally used two HDDs instead of two SSDs that are also in the system 3. Tried to create a new pool with the SSDs, but couldn't (for some reason) 4. Destroyed ZFS HDD pool with the destroy command, but still can't create the SSD pool Is this correct or am I misunderstanding something? What happens if you try to create the ZFS pool for the SSDs manually from the commandline? I.e.: zpool create ssdpool mirror ssd_dev1 ssd_dev2 Regards, harrim4n On 25.05.20 16:13, Sivakumar SARAVANAN wrote: > Hello, > > I need to add 2 additional mirrored disk as below > > 1. SSB ZFS with 855 GB (2 disk) > 2. HDD ZFS with 2 TB ( 2 disk) > > As I created as above already. But I wanted to remove all the mirrored disk > that I created already. > > I used this command - zpool destroy HDD-Storage-PRX003 > > I want to create a new mirrored disk now. But I can see any free disk now > to create a new ZFS mirrored disk. > > Kindly suggest. > > > > Mit freundlichen Gr??en / Best regards / Cordialement, > > Sivakumar SARAVANAN > > Externer Dienstleister f?r / External service provider for > Valeo Siemens eAutomotive Germany GmbH > Research & Development > R & D SWENG TE 1 INFTE > Frauenauracher Stra?e 85 > 91056 Erlangen, Germany > Tel.: +49 9131 9892 0000 > Mobile: +49 176 7698 5441 > sivakumar.saravanan.jv.ext at valeo-siemens.com > valeo-siemens.com > > Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger > Schwab, Michael > Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered > office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 > > > On Mon, May 25, 2020 at 3:49 PM Lindsay Mathieson < > lindsay.mathieson at gmail.com> wrote: > >> On 25/05/2020 10:43 pm, Sivakumar SARAVANAN wrote: >>> So I ran the below comment >>> >>> zpool destroy HDD-Storage-PRX003 >>> >>> But I am not getting a free pool to create SSD mirrored disk again after >>> running the above command. >> I believe you need to remove the partitioning on the disks which zfs >> created first, there's usually two partitions on the disk. >> >> -- >> Lindsay >> >> _______________________________________________ >> pve-user mailing list >> pve-user at pve.proxmox.com >> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >> From sivakumar.saravanan.jv.ext at valeo-siemens.com Wed May 27 08:55:10 2020 From: sivakumar.saravanan.jv.ext at valeo-siemens.com (Sivakumar SARAVANAN) Date: Wed, 27 May 2020 08:55:10 +0200 Subject: [PVE-User] ZFS Storage Issue In-Reply-To: <5ee30502-8e57-e912-202a-f72b99f93c5e@harrim4n.com> References: <5ee30502-8e57-e912-202a-f72b99f93c5e@harrim4n.com> Message-ID: Hello, Thanks for the update. Please find the below requested details. 1. I created the ZFS pool with 2 mirrored using HDD disk by mistake (Actually I suppose to create a ZFS pool using the SSD disk) 2. Noticed that I accidentally used two HDDs instead of using two SSDs thatare also in the system. 3. Tried to create a new pool with the SSDs, but couldn't (for some reason) , Because I can't see a free disk to create it. 4. Destroyed ZFS HDD pool with the destroy command, but still can'tcreate the SSD pool. 5. I can't see any free disk to create it while creating the ZFS pool. Mit freundlichen Gr??en / Best regards / Cordialement, Sivakumar SARAVANAN Externer Dienstleister f?r / External service provider for Valeo Siemens eAutomotive Germany GmbH Research & Development R & D SWENG TE 1 INFTE Frauenauracher Stra?e 85 91056 Erlangen, Germany Tel.: +49 9131 9892 0000 Mobile: +49 176 7698 5441 sivakumar.saravanan.jv.ext at valeo-siemens.com valeo-siemens.com Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger Schwab, Michael Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 On Tue, May 26, 2020 at 7:13 PM harrim4n wrote: > Hi, > > I'm not sure if I'm understanding you correctly, so I'll summarize what > I got that you did: > 1. Created a ZFS pool with one mirror vdev > 2. Noticed that you accidentally used two HDDs instead of two SSDs that > are also in the system > 3. Tried to create a new pool with the SSDs, but couldn't (for some reason) > 4. Destroyed ZFS HDD pool with the destroy command, but still can't > create the SSD pool > > Is this correct or am I misunderstanding something? > > What happens if you try to create the ZFS pool for the SSDs manually > from the commandline? I.e.: > zpool create ssdpool mirror ssd_dev1 ssd_dev2 > > Regards, > harrim4n > > > On 25.05.20 16:13, Sivakumar SARAVANAN wrote: > > Hello, > > > > I need to add 2 additional mirrored disk as below > > > > 1. SSB ZFS with 855 GB (2 disk) > > 2. HDD ZFS with 2 TB ( 2 disk) > > > > As I created as above already. But I wanted to remove all the mirrored > disk > > that I created already. > > > > I used this command - zpool destroy HDD-Storage-PRX003 > > > > I want to create a new mirrored disk now. But I can see any free disk now > > to create a new ZFS mirrored disk. > > > > Kindly suggest. > > > > > > > > Mit freundlichen Gr??en / Best regards / Cordialement, > > > > Sivakumar SARAVANAN > > > > Externer Dienstleister f?r / External service provider for > > Valeo Siemens eAutomotive Germany GmbH > > Research & Development > > R & D SWENG TE 1 INFTE > > Frauenauracher Stra?e 85 > > 91056 Erlangen, Germany > > Tel.: +49 9131 9892 0000 > > Mobile: +49 176 7698 5441 > > sivakumar.saravanan.jv.ext at valeo-siemens.com > > valeo-siemens.com > > > > Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger > > Schwab, Michael > > Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered > > office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 > > > > > > On Mon, May 25, 2020 at 3:49 PM Lindsay Mathieson < > > lindsay.mathieson at gmail.com> wrote: > > > >> On 25/05/2020 10:43 pm, Sivakumar SARAVANAN wrote: > >>> So I ran the below comment > >>> > >>> zpool destroy HDD-Storage-PRX003 > >>> > >>> But I am not getting a free pool to create SSD mirrored disk again > after > >>> running the above command. > >> I believe you need to remove the partitioning on the disks which zfs > >> created first, there's usually two partitions on the disk. > >> > >> -- > >> Lindsay > >> > >> _______________________________________________ > >> pve-user mailing list > >> pve-user at pve.proxmox.com > >> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > >> > > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * From a.antreich at proxmox.com Wed May 27 09:01:05 2020 From: a.antreich at proxmox.com (Alwin Antreich) Date: Wed, 27 May 2020 09:01:05 +0200 Subject: [PVE-User] PVE 6, wireless and regulatory database... In-Reply-To: <20200526153146.GL3717@lilliput.linux.it> References: <20200518084316.GC3626@lilliput.linux.it> <20200519074011.GB4975@lilliput.linux.it> <20200519090409.GA406757@dona.proxmox.com> <20200520205816.GA23561@lilliput.linux.it> <20200522100421.GA1577721@dona.proxmox.com> <20200526104430.GK3717@lilliput.linux.it> <20200526113514.GB477557@dona.proxmox.com> <20200526153146.GL3717@lilliput.linux.it> Message-ID: <20200527070105.GC477557@dona.proxmox.com> On Tue, May 26, 2020 at 05:31:46PM +0200, Marco Gaiarin wrote: > Mandi! Alwin Antreich > In chel di` si favelave... > > > > root at ino:~# dpkg -l | grep wireless-regdb > > > ii wireless-regdb 2016.06.10-1 all wireless regulatory database > > You need the Debian package for the alternatives to work. IIRC, you > > finally installed the ubuntu package. > > https://packages.debian.org/buster/wireless-regdb > > I've installed the buster package... You will need the package from the backports. -- Cheers, Alwin From harrim4n at harrim4n.com Wed May 27 11:26:22 2020 From: harrim4n at harrim4n.com (harrim4n) Date: Wed, 27 May 2020 11:26:22 +0200 Subject: [PVE-User] ZFS Storage Issue In-Reply-To: References: <5ee30502-8e57-e912-202a-f72b99f93c5e@harrim4n.com> Message-ID: <011A36BD-AE07-4840-8AD1-0C90A57C43C4@harrim4n.com> Hi, how are you trying to create the pool? Are you using the WebUI under the Node view > Disks > ZFS? Can you try to create the pool from the commandline? Regards, harrim4n On May 27, 2020 8:55:10 AM GMT+02:00, Sivakumar SARAVANAN wrote: >Hello, > >Thanks for the update. > >Please find the below requested details. > >1. I created the ZFS pool with 2 mirrored using HDD disk by mistake >(Actually I suppose to create a ZFS pool using the SSD disk) >2. Noticed that I accidentally used two HDDs instead of using two SSDs >thatare also in the system. >3. Tried to create a new pool with the SSDs, but couldn't (for some >reason) , Because I can't see a free disk to create it. >4. Destroyed ZFS HDD pool with the destroy command, but still >can'tcreate >the SSD pool. >5. I can't see any free disk to create it while creating the ZFS pool. > >Mit freundlichen Gr??en / Best regards / Cordialement, > >Sivakumar SARAVANAN > >Externer Dienstleister f?r / External service provider for >Valeo Siemens eAutomotive Germany GmbH >Research & Development >R & D SWENG TE 1 INFTE >Frauenauracher Stra?e 85 >91056 Erlangen, Germany >Tel.: +49 9131 9892 0000 >Mobile: +49 176 7698 5441 >sivakumar.saravanan.jv.ext at valeo-siemens.com >valeo-siemens.com > >Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger >Schwab, Michael >Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered >office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 > > >On Tue, May 26, 2020 at 7:13 PM harrim4n wrote: > >> Hi, >> >> I'm not sure if I'm understanding you correctly, so I'll summarize >what >> I got that you did: >> 1. Created a ZFS pool with one mirror vdev >> 2. Noticed that you accidentally used two HDDs instead of two SSDs >that >> are also in the system >> 3. Tried to create a new pool with the SSDs, but couldn't (for some >reason) >> 4. Destroyed ZFS HDD pool with the destroy command, but still can't >> create the SSD pool >> >> Is this correct or am I misunderstanding something? >> >> What happens if you try to create the ZFS pool for the SSDs manually >> from the commandline? I.e.: >> zpool create ssdpool mirror ssd_dev1 ssd_dev2 >> >> Regards, >> harrim4n >> >> >> On 25.05.20 16:13, Sivakumar SARAVANAN wrote: >> > Hello, >> > >> > I need to add 2 additional mirrored disk as below >> > >> > 1. SSB ZFS with 855 GB (2 disk) >> > 2. HDD ZFS with 2 TB ( 2 disk) >> > >> > As I created as above already. But I wanted to remove all the >mirrored >> disk >> > that I created already. >> > >> > I used this command - zpool destroy HDD-Storage-PRX003 >> > >> > I want to create a new mirrored disk now. But I can see any free >disk now >> > to create a new ZFS mirrored disk. >> > >> > Kindly suggest. >> > >> > >> > >> > Mit freundlichen Gr??en / Best regards / Cordialement, >> > >> > Sivakumar SARAVANAN >> > >> > Externer Dienstleister f?r / External service provider for >> > Valeo Siemens eAutomotive Germany GmbH >> > Research & Development >> > R & D SWENG TE 1 INFTE >> > Frauenauracher Stra?e 85 >> > 91056 Erlangen, Germany >> > Tel.: +49 9131 9892 0000 >> > Mobile: +49 176 7698 5441 >> > sivakumar.saravanan.jv.ext at valeo-siemens.com >> > valeo-siemens.com >> > >> > Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger >> > Schwab, Michael >> > Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; >Registered >> > office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 >> > >> > >> > On Mon, May 25, 2020 at 3:49 PM Lindsay Mathieson < >> > lindsay.mathieson at gmail.com> wrote: >> > >> >> On 25/05/2020 10:43 pm, Sivakumar SARAVANAN wrote: >> >>> So I ran the below comment >> >>> >> >>> zpool destroy HDD-Storage-PRX003 >> >>> >> >>> But I am not getting a free pool to create SSD mirrored disk >again >> after >> >>> running the above command. >> >> I believe you need to remove the partitioning on the disks which >zfs >> >> created first, there's usually two partitions on the disk. >> >> >> >> -- >> >> Lindsay >> >> >> >> _______________________________________________ >> >> pve-user mailing list >> >> pve-user at pve.proxmox.com >> >> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >> >> >> >> _______________________________________________ >> pve-user mailing list >> pve-user at pve.proxmox.com >> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >> > >-- >*This e-mail message is intended for the internal use of the intended >recipient(s) only. >The information contained herein is >confidential/privileged. Its disclosure or reproduction is strictly >prohibited. >If you are not the intended recipient, please inform the sender >immediately, do not disclose it internally or to third parties and >destroy >it. > >In the course of our business relationship and for business purposes >only, Valeo may need to process some of your personal data. >For more >information, please refer to the Valeo Data Protection Statement and >Privacy notice available on Valeo.com >* >_______________________________________________ >pve-user mailing list >pve-user at pve.proxmox.com >https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user From sivakumar.saravanan.jv.ext at valeo-siemens.com Wed May 27 11:35:21 2020 From: sivakumar.saravanan.jv.ext at valeo-siemens.com (Sivakumar SARAVANAN) Date: Wed, 27 May 2020 11:35:21 +0200 Subject: [PVE-User] (no subject) Message-ID: Hello, I am frequently getting this error message from Proxmox Web GUI. permission denied - invalid pve ticket (401) What could be the reason ? Thanks in advance. Mit freundlichen Gr??en / Best regards / Cordialement, Sivakumar SARAVANAN Externer Dienstleister f?r / External service provider for Valeo Siemens eAutomotive Germany GmbH Research & Development R & D SWENG TE 1 INFTE Frauenauracher Stra?e 85 91056 Erlangen, Germany Tel.: +49 9131 9892 0000 Mobile: +49 176 7698 5441 sivakumar.saravanan.jv.ext at valeo-siemens.com valeo-siemens.com Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger Schwab, Michael Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * From harrim4n at harrim4n.com Wed May 27 11:37:34 2020 From: harrim4n at harrim4n.com (harrim4n) Date: Wed, 27 May 2020 11:37:34 +0200 Subject: [PVE-User] Invalid PVE Ticket (401) In-Reply-To: References: Message-ID: <08B12E3B-082E-4B70-9ED7-8D7A9F562837@harrim4n.com> Hi, are you running a cluster or a single host? I had the issue a few times if the times on the hosts are out of sync (either client to host or between cluster hosts). On May 27, 2020 11:35:21 AM GMT+02:00, Sivakumar SARAVANAN wrote: >Hello, > >I am frequently getting this error message from Proxmox Web GUI. > >permission denied - invalid pve ticket (401) > >What could be the reason ? > >Thanks in advance. > >Mit freundlichen Gr??en / Best regards / Cordialement, > >Sivakumar SARAVANAN > >Externer Dienstleister f?r / External service provider for >Valeo Siemens eAutomotive Germany GmbH >Research & Development >R & D SWENG TE 1 INFTE >Frauenauracher Stra?e 85 >91056 Erlangen, Germany >Tel.: +49 9131 9892 0000 >Mobile: +49 176 7698 5441 >sivakumar.saravanan.jv.ext at valeo-siemens.com >valeo-siemens.com > >Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger >Schwab, Michael >Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered >office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 > >-- >*This e-mail message is intended for the internal use of the intended >recipient(s) only. >The information contained herein is >confidential/privileged. Its disclosure or reproduction is strictly >prohibited. >If you are not the intended recipient, please inform the sender >immediately, do not disclose it internally or to third parties and >destroy >it. > >In the course of our business relationship and for business purposes >only, Valeo may need to process some of your personal data. >For more >information, please refer to the Valeo Data Protection Statement and >Privacy notice available on Valeo.com >* >_______________________________________________ >pve-user mailing list >pve-user at pve.proxmox.com >https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user From pve at geniorxt.de Wed May 27 11:58:55 2020 From: pve at geniorxt.de (pve at geniorxt.de) Date: Wed, 27 May 2020 11:58:55 +0200 (CEST) Subject: [PVE-User] (no subject) In-Reply-To: References: Message-ID: <718537304.370287.1590573535579@office.mailbox.org> Hello, this usually happens when you've got multiple sessions of the Proxmox VE webui to different servers in different tabs of the same browser. This is at least my exprience. Br Benjamin > Sivakumar SARAVANAN hat am 27. Mai 2020 11:35 geschrieben: > > > Hello, > > I am frequently getting this error message from Proxmox Web GUI. > > permission denied - invalid pve ticket (401) > > What could be the reason ? > > Thanks in advance. > > Mit freundlichen Gr??en / Best regards / Cordialement, > > Sivakumar SARAVANAN > > Externer Dienstleister f?r / External service provider for > Valeo Siemens eAutomotive Germany GmbH > Research & Development > R & D SWENG TE 1 INFTE > Frauenauracher Stra?e 85 > 91056 Erlangen, Germany > Tel.: +49 9131 9892 0000 > Mobile: +49 176 7698 5441 > sivakumar.saravanan.jv.ext at valeo-siemens.com > valeo-siemens.com > > Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger > Schwab, Michael > Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered > office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 > > -- > *This e-mail message is intended for the internal use of the intended > recipient(s) only. > The information contained herein is > confidential/privileged. Its disclosure or reproduction is strictly > prohibited. > If you are not the intended recipient, please inform the sender > immediately, do not disclose it internally or to third parties and destroy > it. > > In the course of our business relationship and for business purposes > only, Valeo may need to process some of your personal data. > For more > information, please refer to the Valeo Data Protection Statement and > Privacy notice available on Valeo.com > * > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user From sivakumar.saravanan.jv.ext at valeo-siemens.com Wed May 27 12:21:00 2020 From: sivakumar.saravanan.jv.ext at valeo-siemens.com (Sivakumar SARAVANAN) Date: Wed, 27 May 2020 12:21:00 +0200 Subject: [PVE-User] Invalid PVE Ticket (401) In-Reply-To: <08B12E3B-082E-4B70-9ED7-8D7A9F562837@harrim4n.com> References: <08B12E3B-082E-4B70-9ED7-8D7A9F562837@harrim4n.com> Message-ID: Hello, Thanks for the reply., Yes we are using 20 server Proxmox in the datacenter and each host will be working as a single node cluster. Mit freundlichen Gr??en / Best regards / Cordialement, Sivakumar SARAVANAN Externer Dienstleister f?r / External service provider for Valeo Siemens eAutomotive Germany GmbH Research & Development R & D SWENG TE 1 INFTE Frauenauracher Stra?e 85 91056 Erlangen, Germany Tel.: +49 9131 9892 0000 Mobile: +49 176 7698 5441 sivakumar.saravanan.jv.ext at valeo-siemens.com valeo-siemens.com Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger Schwab, Michael Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 On Wed, May 27, 2020 at 11:37 AM harrim4n wrote: > Hi, > > are you running a cluster or a single host? > I had the issue a few times if the times on the hosts are out of sync > (either client to host or between cluster hosts). > > On May 27, 2020 11:35:21 AM GMT+02:00, Sivakumar SARAVANAN < > sivakumar.saravanan.jv.ext at valeo-siemens.com> wrote: > >Hello, > > > >I am frequently getting this error message from Proxmox Web GUI. > > > >permission denied - invalid pve ticket (401) > > > >What could be the reason ? > > > >Thanks in advance. > > > >Mit freundlichen Gr??en / Best regards / Cordialement, > > > >Sivakumar SARAVANAN > > > >Externer Dienstleister f?r / External service provider for > >Valeo Siemens eAutomotive Germany GmbH > >Research & Development > >R & D SWENG TE 1 INFTE > >Frauenauracher Stra?e 85 > >91056 Erlangen, Germany > >Tel.: +49 9131 9892 0000 > >Mobile: +49 176 7698 5441 > >sivakumar.saravanan.jv.ext at valeo-siemens.com > >valeo-siemens.com > > > >Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger > >Schwab, Michael > >Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered > >office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 > > > >-- > >*This e-mail message is intended for the internal use of the intended > >recipient(s) only. > >The information contained herein is > >confidential/privileged. Its disclosure or reproduction is strictly > >prohibited. > >If you are not the intended recipient, please inform the sender > >immediately, do not disclose it internally or to third parties and > >destroy > >it. > > > >In the course of our business relationship and for business purposes > >only, Valeo may need to process some of your personal data. > >For more > >information, please refer to the Valeo Data Protection Statement and > >Privacy notice available on Valeo.com > >* > >_______________________________________________ > >pve-user mailing list > >pve-user at pve.proxmox.com > >https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * From devzero at web.de Wed May 27 16:54:37 2020 From: devzero at web.de (Roland) Date: Wed, 27 May 2020 16:54:37 +0200 Subject: [PVE-User] 2-Node Cluster - Frage zu Quorum Message-ID: <9477de6f-7659-8750-73a5-4bc8626f5703@web.de> Hallo, kann mir jemand erl?utern worin die genauen Probleme/Herausforderungen bei einem "Non-HA" Dual-Node Cluster bestehen und ob man da wirklich unbedingt ein Quorum bzw. einen Voting-Node haben sollte ? Worin bestehen die Risiken wenn man "two_node: 1" setzt ? Was f?r Nachteile hat man dadurch bzw. was kann da ganz konkret schiefgehen ? Wir w?rden gerne zun?chst 2 Nodes im Cluster betreiben, beide als "aktiv", also mit VMs die laufen - aber halt ohne Shared Storage - denn im Cluster lassen sich die VMs dann sch?n f?r Wartungszwecke hin und her migrieren... Ich kann mich gerad nicht so recht entscheiden ob wir den Qdevice auf weiterer VM bzw. auf Raspberry machen sollen (finde das irgendwie exotic, da commandline only), oder ob wir nen kleinen 3. "lightweight" Proxmox als Quorum hinzuf?gen (auf nem mini-PC) - oder ob wir via Konfiguration halt ganz auf den 3. Node verzichten... Gr?sse Roland From s.schwebel at uvensys.de Wed May 27 17:04:57 2020 From: s.schwebel at uvensys.de (Steffen Schwebel) Date: Wed, 27 May 2020 17:04:57 +0200 Subject: [PVE-User] 2-Node Cluster - Frage zu Quorum In-Reply-To: <9477de6f-7659-8750-73a5-4bc8626f5703@web.de> References: <9477de6f-7659-8750-73a5-4bc8626f5703@web.de> Message-ID: Hallo, wir haben auch einen 2 - Node Cluster im Einsatz. Solange es keinen Shared Storage hat, ist dass problemlos Der Nachteil ist, das es keinen Shared-Storage gibt ^_^ Eine Migration von einer VM von den einen Host auf den anderen ist, afaik, nur offline m?glich. Gr??e, Steffen On 5/27/20 4:54 PM, Roland wrote: > Hallo, > > kann mir jemand erl?utern worin die genauen Probleme/Herausforderungen > bei einem "Non-HA" Dual-Node Cluster bestehen und ob man da wirklich > unbedingt ein Quorum bzw. einen Voting-Node haben sollte ? Worin > bestehen die Risiken wenn man "two_node: 1" setzt ? Was f?r Nachteile > hat man dadurch bzw. was kann da ganz konkret schiefgehen ? > > Wir w?rden gerne zun?chst 2 Nodes im Cluster betreiben, beide als > "aktiv", also mit VMs die laufen - aber halt ohne Shared Storage - denn > im Cluster lassen sich die VMs dann sch?n f?r Wartungszwecke hin und her > migrieren... > > Ich kann mich gerad nicht so recht entscheiden ob wir den Qdevice auf > weiterer VM bzw. auf Raspberry machen sollen (finde das irgendwie > exotic, da commandline only), oder ob wir nen kleinen 3. "lightweight" > Proxmox als Quorum hinzuf?gen (auf nem mini-PC) - oder ob wir via > Konfiguration halt ganz auf den 3. Node verzichten... > > Gr?sse > Roland > > > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user -- Steffen Schwebel Mail: s.schwebel at uvensys.de uvensys GmbH Firmensitz und Sitz der Gesellschaft: uvensys GmbH Schorbachstra?e 11 35510 Butzbach HRB: AG Friedberg, 7780 USt-Id: DE282879294 Gesch?ftsf?hrer: Dr. Thomas Licht, t.licht at uvensys.de Volker Lieder, v.lieder at uvensys.de Mail: info at uvensys.de Internet: www.uvensys.de Durchwahl: 06033 - 18 19 225 Hotline: 06033 - 18 19 288 Zentrale: 06033 - 18 19 20 Fax: 06033 - 18 19 299 ========================================================== Jegliche Stellungnahmen und Meinungen dieser E-Mail sind alleine die des Autors und nicht notwendigerweise die der Firma. Falls erforderlich, k?nnen Sie eine gesonderte schriftliche Best?tigung anfordern. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company. If verification is required please request a hard-copy version. From devzero at web.de Wed May 27 17:24:43 2020 From: devzero at web.de (Roland) Date: Wed, 27 May 2020 17:24:43 +0200 Subject: [PVE-User] 2-Node Cluster - Frage zu Quorum In-Reply-To: References: <9477de6f-7659-8750-73a5-4bc8626f5703@web.de> Message-ID: >Eine Migration von einer VM von den einen Host auf den anderen ist, >afaik, nur offline m?glich. funktioniert hier sowohl online also auch offline. Gr?sse Roland Am 27.05.20 um 17:04 schrieb Steffen Schwebel: > Hallo, > > wir haben auch einen 2 - Node Cluster im Einsatz. > Solange es keinen Shared Storage hat, ist dass problemlos > > Der Nachteil ist, das es keinen Shared-Storage gibt ^_^ > Eine Migration von einer VM von den einen Host auf den anderen ist, > afaik, nur offline m?glich. > > Gr??e, > Steffen > > On 5/27/20 4:54 PM, Roland wrote: >> Hallo, >> >> kann mir jemand erl?utern worin die genauen Probleme/Herausforderungen >> bei einem "Non-HA" Dual-Node Cluster bestehen und ob man da wirklich >> unbedingt ein Quorum bzw. einen Voting-Node haben sollte ? Worin >> bestehen die Risiken wenn man "two_node: 1" setzt ? Was f?r Nachteile >> hat man dadurch bzw. was kann da ganz konkret schiefgehen ? >> >> Wir w?rden gerne zun?chst 2 Nodes im Cluster betreiben, beide als >> "aktiv", also mit VMs die laufen - aber halt ohne Shared Storage - denn >> im Cluster lassen sich die VMs dann sch?n f?r Wartungszwecke hin und her >> migrieren... >> >> Ich kann mich gerad nicht so recht entscheiden ob wir den Qdevice auf >> weiterer VM bzw. auf Raspberry machen sollen (finde das irgendwie >> exotic, da commandline only), oder ob wir nen kleinen 3. "lightweight" >> Proxmox als Quorum hinzuf?gen (auf nem mini-PC) - oder ob wir via >> Konfiguration halt ganz auf den 3. Node verzichten... >> >> Gr?sse >> Roland >> >> >> _______________________________________________ >> pve-user mailing list >> pve-user at pve.proxmox.com >> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user From aderumier at odiso.com Thu May 28 15:06:29 2020 From: aderumier at odiso.com (Alexandre DERUMIER) Date: Thu, 28 May 2020 15:06:29 +0200 (CEST) Subject: [PVE-User] Invalid PVE Ticket (401) In-Reply-To: References: <08B12E3B-082E-4B70-9ED7-8D7A9F562837@harrim4n.com> Message-ID: <159492303.651297.1590671189898.JavaMail.zimbra@odiso.com> Hi, Are you host clocks correctly synced ? ----- Mail original ----- De: "Sivakumar SARAVANAN" ?: "proxmoxve" Envoy?: Mercredi 27 Mai 2020 12:21:00 Objet: Re: [PVE-User] Invalid PVE Ticket (401) Hello, Thanks for the reply., Yes we are using 20 server Proxmox in the datacenter and each host will be working as a single node cluster. Mit freundlichen Gr??en / Best regards / Cordialement, Sivakumar SARAVANAN Externer Dienstleister f?r / External service provider for Valeo Siemens eAutomotive Germany GmbH Research & Development R & D SWENG TE 1 INFTE Frauenauracher Stra?e 85 91056 Erlangen, Germany Tel.: +49 9131 9892 0000 Mobile: +49 176 7698 5441 sivakumar.saravanan.jv.ext at valeo-siemens.com valeo-siemens.com Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger Schwab, Michael Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 On Wed, May 27, 2020 at 11:37 AM harrim4n wrote: > Hi, > > are you running a cluster or a single host? > I had the issue a few times if the times on the hosts are out of sync > (either client to host or between cluster hosts). > > On May 27, 2020 11:35:21 AM GMT+02:00, Sivakumar SARAVANAN < > sivakumar.saravanan.jv.ext at valeo-siemens.com> wrote: > >Hello, > > > >I am frequently getting this error message from Proxmox Web GUI. > > > >permission denied - invalid pve ticket (401) > > > >What could be the reason ? > > > >Thanks in advance. > > > >Mit freundlichen Gr??en / Best regards / Cordialement, > > > >Sivakumar SARAVANAN > > > >Externer Dienstleister f?r / External service provider for > >Valeo Siemens eAutomotive Germany GmbH > >Research & Development > >R & D SWENG TE 1 INFTE > >Frauenauracher Stra?e 85 > >91056 Erlangen, Germany > >Tel.: +49 9131 9892 0000 > >Mobile: +49 176 7698 5441 > >sivakumar.saravanan.jv.ext at valeo-siemens.com > >valeo-siemens.com > > > >Valeo Siemens eAutomotive Germany GmbH: Managing Directors: Holger > >Schwab, Michael > >Axmann; Chairman of the Supervisory Board: Hartmut Kl?tzer; Registered > >office: Erlangen, Germany; Commercial registry: F?rth, HRB 15655 > > > >-- > >*This e-mail message is intended for the internal use of the intended > >recipient(s) only. > >The information contained herein is > >confidential/privileged. Its disclosure or reproduction is strictly > >prohibited. > >If you are not the intended recipient, please inform the sender > >immediately, do not disclose it internally or to third parties and > >destroy > >it. > > > >In the course of our business relationship and for business purposes > >only, Valeo may need to process some of your personal data. > >For more > >information, please refer to the Valeo Data Protection Statement and > >Privacy notice available on Valeo.com > >* > >_______________________________________________ > >pve-user mailing list > >pve-user at pve.proxmox.com > >https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > -- *This e-mail message is intended for the internal use of the intended recipient(s) only. The information contained herein is confidential/privileged. Its disclosure or reproduction is strictly prohibited. If you are not the intended recipient, please inform the sender immediately, do not disclose it internally or to third parties and destroy it. In the course of our business relationship and for business purposes only, Valeo may need to process some of your personal data. For more information, please refer to the Valeo Data Protection Statement and Privacy notice available on Valeo.com * _______________________________________________ pve-user mailing list pve-user at pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user From jm at ginernet.com Thu May 28 18:07:10 2020 From: jm at ginernet.com (=?UTF-8?Q?Jos=c3=a9_Manuel_Giner?=) Date: Thu, 28 May 2020 18:07:10 +0200 Subject: [PVE-User] ebtables policy created to DROP but automatically changed to ACCEPT Message-ID: <6f3fa567-de19-8ae8-05c3-3449ee8d4c01@ginernet.com> Hello, when I create a ebtables chain in a HN with DROP policy, after some seconds is automatically modified to ACCEPT ebtables -N test ebtables -P test DROP some seconds later: #ebtables -L Bridge chain: test, entries: 0, policy: ACCEPT I've tried to disable "ebtables" on the datacenter, but nothing seems to affect. Any idea? Thanks! -- Jos? Manuel Giner https://ginernet.com From noc at tipgroup.net Fri May 29 11:14:21 2020 From: noc at tipgroup.net (Olivier Mascia) Date: Fri, 29 May 2020 11:14:21 +0200 Subject: [PVE-User] System not fully up to date (found 13 new packages) Message-ID: I'm seeing a discrepancy since my last round of upgrades (on a 4 nodes cluster). My pve2/pve3/pve4 nodes all report : Linux 5.4.41-1-pve #1 SMP PVE 5.4.41-1 (Fri, 15 May 2020 15:06:08 +0200) pve-manager/6.2-4/9824574a with (as writing this) no other updates available. Looks fine. Yet one node (pve1) reports: Linux 5.4.41-1-pve #1 SMP PVE 5.4.41-1 (Fri, 15 May 2020 15:06:08 +0200). [OK] pve-manager/6.1-5/9bf06119 [DISCREPANCY] Attempting an ordinary update from GUI or apt-get dist-upgrade returns this: Starting system upgrade: apt-get dist-upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following packages have been kept back: libpve-access-control libpve-common-perl libpve-guest-common-perl libpve-http-server-perl libpve-storage-perl lxc-pve pve-container pve-manager qemu-server 0 upgraded, 0 newly installed, 0 to remove and 9 not upgraded. System not fully up to date (found 13 new packages) I'm not experienced with this situation. What do you think (or better know!) I should do? Thanks a lot. ? Best Regards, Meilleures salutations, Met vriendelijke groeten, Mit freundlichen Gr??en, Olivier Mascia From lists at merit.unu.edu Fri May 29 11:27:12 2020 From: lists at merit.unu.edu (mj) Date: Fri, 29 May 2020 11:27:12 +0200 Subject: [PVE-User] System not fully up to date (found 13 new packages) In-Reply-To: References: Message-ID: Hi, Not ure about this particular situation, but you could try: apt-get --with-new-pkgs upgrade or otherwise you could try manually installing the kept-back packages with apt-get install libpve-access-control libpve-common-perl ... Perhaps that helps? MJ On 5/29/20 11:14 AM, Olivier Mascia wrote: > I'm seeing a discrepancy since my last round of upgrades (on a 4 nodes cluster). > My pve2/pve3/pve4 nodes all report : > > Linux 5.4.41-1-pve #1 SMP PVE 5.4.41-1 (Fri, 15 May 2020 15:06:08 +0200) > pve-manager/6.2-4/9824574a > > with (as writing this) no other updates available. Looks fine. > > Yet one node (pve1) reports: > > Linux 5.4.41-1-pve #1 SMP PVE 5.4.41-1 (Fri, 15 May 2020 15:06:08 +0200). [OK] > pve-manager/6.1-5/9bf06119 [DISCREPANCY] > > Attempting an ordinary update from GUI or apt-get dist-upgrade returns this: > > Starting system upgrade: apt-get dist-upgrade > Reading package lists... Done > Building dependency tree > Reading state information... Done > Calculating upgrade... Done > The following packages have been kept back: > libpve-access-control libpve-common-perl libpve-guest-common-perl > libpve-http-server-perl libpve-storage-perl lxc-pve pve-container pve-manager > qemu-server > 0 upgraded, 0 newly installed, 0 to remove and 9 not upgraded. > > System not fully up to date (found 13 new packages) > > I'm not experienced with this situation. What do you think (or better know!) I should do? > > Thanks a lot. > ? > Best Regards, Meilleures salutations, Met vriendelijke groeten, Mit freundlichen Gr??en, > Olivier Mascia > > > _______________________________________________ > pve-user mailing list > pve-user at pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > From jm at ginernet.com Fri May 29 14:24:25 2020 From: jm at ginernet.com (=?UTF-8?Q?Jos=c3=a9_Manuel_Giner?=) Date: Fri, 29 May 2020 14:24:25 +0200 Subject: [PVE-User] ebtables policy created to DROP but automatically changed to ACCEPT In-Reply-To: <6f3fa567-de19-8ae8-05c3-3449ee8d4c01@ginernet.com> References: <6f3fa567-de19-8ae8-05c3-3449ee8d4c01@ginernet.com> Message-ID: Any info? On 28/05/2020 18:07, Jos? Manuel Giner wrote: > Hello, > > when I create a ebtables chain in a HN with DROP policy, after some > seconds is automatically modified to ACCEPT > > ebtables -N test > ebtables -P test DROP > > some seconds later: > > #ebtables -L > Bridge chain: test, entries: 0, policy: ACCEPT > > I've tried to disable "ebtables" on the datacenter, but nothing seems to > affect. > > Any idea? > > Thanks! > > -- Jos? Manuel Giner https://ginernet.com From jm at ginernet.com Fri May 29 15:46:26 2020 From: jm at ginernet.com (=?UTF-8?Q?Jos=c3=a9_Manuel_Giner?=) Date: Fri, 29 May 2020 15:46:26 +0200 Subject: [PVE-User] ebtables policy created to DROP but automatically changed to ACCEPT In-Reply-To: References: <6f3fa567-de19-8ae8-05c3-3449ee8d4c01@ginernet.com> Message-ID: <6ea585b7-82de-f27b-b226-46fe51f57bef@ginernet.com> Seems it's a bug "fixed", but is still here: https://git.proxmox.com/?p=pve-firewall.git;a=commit;h=84025e9943d236414fbd869b89cb2e8e17af3208 On 29/05/2020 14:24, Jos? Manuel Giner wrote: > Any info? > > > On 28/05/2020 18:07, Jos? Manuel Giner wrote: >> Hello, >> >> when I create a ebtables chain in a HN with DROP policy, after some >> seconds is automatically modified to ACCEPT >> >> ebtables -N test >> ebtables -P test DROP >> >> some seconds later: >> >> #ebtables -L >> Bridge chain: test, entries: 0, policy: ACCEPT >> >> I've tried to disable "ebtables" on the datacenter, but nothing seems >> to affect. >> >> Any idea? >> >> Thanks! >> >> > -- Jos? Manuel Giner https://ginernet.com From s.ivanov at proxmox.com Fri May 29 16:59:18 2020 From: s.ivanov at proxmox.com (Stoiko Ivanov) Date: Fri, 29 May 2020 16:59:18 +0200 Subject: [PVE-User] ebtables policy created to DROP but automatically changed to ACCEPT In-Reply-To: <6ea585b7-82de-f27b-b226-46fe51f57bef@ginernet.com> References: <6f3fa567-de19-8ae8-05c3-3449ee8d4c01@ginernet.com> <6ea585b7-82de-f27b-b226-46fe51f57bef@ginernet.com> Message-ID: <20200529165918.02e501ae@rosa.proxmox.com> Hi, Thanks for reporting this! I managed to reproduce the issue - it seems the code currently does not account for the policy of an ebtables chain (see [0]) Please open a bug report over at https://bugzilla.proxmox.com as a mitigation until this is fixed you could add a final rule which drops all packets to your ruleset: ``` ebtables -A test -j DROP ``` kind regards, stoiko [0] https://git.proxmox.com/?p=pve-firewall.git;a=blob;f=src/PVE/Firewall.pm;h=a2105e5410590b30305bd6941ddcc8bfe40159da;hb=HEAD#l4166 On Fri, 29 May 2020 15:46:26 +0200 Jos? Manuel Giner wrote: > Seems it's a bug "fixed", but is still here: > > https://git.proxmox.com/?p=pve-firewall.git;a=commit;h=84025e9943d236414fbd869b89cb2e8e17af3208 > > > > > On 29/05/2020 14:24, Jos? Manuel Giner wrote: > > Any info? > > > > > > On 28/05/2020 18:07, Jos? Manuel Giner wrote: > >> Hello, > >> > >> when I create a ebtables chain in a HN with DROP policy, after some > >> seconds is automatically modified to ACCEPT > >> > >> ebtables -N test > >> ebtables -P test DROP > >> > >> some seconds later: > >> > >> #ebtables -L > >> Bridge chain: test, entries: 0, policy: ACCEPT > >> > >> I've tried to disable "ebtables" on the datacenter, but nothing seems > >> to affect. > >> > >> Any idea? > >> > >> Thanks! > >> > >> > > > From jm at ginernet.com Fri May 29 17:10:26 2020 From: jm at ginernet.com (=?UTF-8?Q?Jos=c3=a9_Manuel_Giner?=) Date: Fri, 29 May 2020 17:10:26 +0200 Subject: [PVE-User] ebtables policy created to DROP but automatically changed to ACCEPT In-Reply-To: <20200529165918.02e501ae@rosa.proxmox.com> References: <6f3fa567-de19-8ae8-05c3-3449ee8d4c01@ginernet.com> <6ea585b7-82de-f27b-b226-46fe51f57bef@ginernet.com> <20200529165918.02e501ae@rosa.proxmox.com> Message-ID: <4c7b1e77-3aee-a6b9-5757-e9ec1e3d601b@ginernet.com> Bug 2773 As I'm not using the pve-firewall, I've stopped and it's ok. # pve-firewall stop Thanks. On 29/05/2020 16:59, Stoiko Ivanov wrote: > Hi, > > Thanks for reporting this! > I managed to reproduce the issue - it seems the code currently does not > account for the policy of an ebtables chain (see [0]) > > Please open a bug report over at https://bugzilla.proxmox.com > > as a mitigation until this is fixed you could add a final rule which drops > all packets to your ruleset: > ``` > ebtables -A test -j DROP > ``` > > kind regards, > stoiko > > [0] > https://git.proxmox.com/?p=pve-firewall.git;a=blob;f=src/PVE/Firewall.pm;h=a2105e5410590b30305bd6941ddcc8bfe40159da;hb=HEAD#l4166 > > On Fri, 29 May 2020 15:46:26 +0200 > Jos? Manuel Giner wrote: > >> Seems it's a bug "fixed", but is still here: >> >> https://git.proxmox.com/?p=pve-firewall.git;a=commit;h=84025e9943d236414fbd869b89cb2e8e17af3208 >> >> >> >> >> On 29/05/2020 14:24, Jos? Manuel Giner wrote: >>> Any info? >>> >>> >>> On 28/05/2020 18:07, Jos? Manuel Giner wrote: >>>> Hello, >>>> >>>> when I create a ebtables chain in a HN with DROP policy, after some >>>> seconds is automatically modified to ACCEPT >>>> >>>> ebtables -N test >>>> ebtables -P test DROP >>>> >>>> some seconds later: >>>> >>>> #ebtables -L >>>> Bridge chain: test, entries: 0, policy: ACCEPT >>>> >>>> I've tried to disable "ebtables" on the datacenter, but nothing seems >>>> to affect. >>>> >>>> Any idea? >>>> >>>> Thanks! >>>> >>>> >>> >> > > -- Jos? Manuel Giner https://ginernet.com