[PVE-User] PVE 5.4 - resize a NFS disk truncated it

[Eneko Lacunza]

Thanks for the suggestions Gianni,

Partition was mounted. We have umounted and fsck'd the filesystem; 
journal is broken so we haven't tried going further before trying other 
recovery methods :)


El 28/1/20 a las 9:58, Gianni Milo escribió:
> Have you tried booting from a recovery media, then first check reported
> size within fdisk, then after try to fsck the disk (this may lead to some
> data loss by itself) ? Is the drive/partition mountable at all? If not,
> then you may have to try repairing its qcow2/raw file. There are some
> guides on the internet describing that, however there are no guarantees for
> the results.
>
> G.
>
> On Tue, 28 Jan 2020 at 08:27, Eneko Lacunza <elacunza at binovo.es> wrote:
>
>> Hi all,
>>
>> We have a PVE 5.4 cluster (details below), with a Synology DS1819+ NFS
>> server for storing file backups.
>>
>> The setup is as follows:
>>
>> - Debian 9 VM with 2 disks; system disk con Ceph RBD, file backup data
>> disk on NFS (6,5TB)
>> - NFS storage on Synology NAS.
>>
>> Backup disk was getting full, so we issued a resize disk from web GUI,
>> with a 500GB size increment.
>>
>> GUI reported timeout with 500 error. After that, disk was shown
>> incremented at VM hardware view, but storage showed the disk having
>> 500GB. VM showed the block device having 500GB (partiton was 6,5TB yet).
>>
>> We have tried incrementing the disk back to 7TB, but disk/partition is
>> corrupt (It was a bad idea, but its too late now).
>>
>> Any idea to recover lost data on NFS server? :)
>>
>> We have done this operation hundreds of times without issues, has anyone
>> had such a catastrophic experience?
>>
>> # pveversion -v
>> proxmox-ve: 5.4-1 (running kernel: 4.15.18-12-pve)
>> pve-manager: 5.4-3 (running version: 5.4-3/0a6eaa62)
>> pve-kernel-4.15: 5.3-3
>> pve-kernel-4.15.18-12-pve: 4.15.18-35
>> pve-kernel-4.15.18-7-pve: 4.15.18-27
>> pve-kernel-4.4.134-1-pve: 4.4.134-112
>> ceph: 12.2.11-pve1
>> corosync: 2.4.4-pve1
>> criu: 2.11.1-1~bpo90
>> glusterfs-client: 3.8.8-1
>> ksm-control-daemon: 1.2-2
>> libjs-extjs: 6.0.1-2
>> libpve-access-control: 5.1-8
>> libpve-apiclient-perl: 2.0-5
>> libpve-common-perl: 5.0-50
>> libpve-guest-common-perl: 2.0-20
>> libpve-http-server-perl: 2.0-13
>> libpve-storage-perl: 5.0-41
>> libqb0: 1.0.3-1~bpo9
>> lvm2: 2.02.168-pve6
>> lxc-pve: 3.1.0-3
>> lxcfs: 3.0.3-pve1
>> novnc-pve: 1.0.0-3
>> proxmox-widget-toolkit: 1.0-25
>> pve-cluster: 5.0-36
>> pve-container: 2.0-37
>> pve-docs: 5.4-2
>> pve-edk2-firmware: 1.20190312-1
>> pve-firewall: 3.0-19
>> pve-firmware: 2.0-6
>> pve-ha-manager: 2.0-9
>> pve-i18n: 1.1-4
>> pve-libspice-server1: 0.14.1-2
>> pve-qemu-kvm: 2.12.1-3
>> pve-xtermjs: 3.12.0-1
>> qemu-server: 5.0-50
>> smartmontools: 6.5+svn4324-1
>> spiceterm: 3.0-5
>> vncterm: 1.5-3
>>
>>
>> Thanks a lot
>> Eneko
>>
>> --
>> Zuzendari Teknikoa / Director Técnico
>> Binovo IT Human Project, S.L.
>> Telf. 943569206
>> Astigarragako bidea 2
>> <https://www.google.com/maps/search/Astigarragako+bidea+2?entry=gmail&source=g>,
>> 2º izq. oficina 11; 20180 Oiartzun (Gipuzkoa)
>> www.binovo.es
>>
>> _______________________________________________
>> pve-user mailing list
>> pve-user at pve.proxmox.com
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>
> _______________________________________________
> pve-user mailing list
> pve-user at pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user


-- 
Zuzendari Teknikoa / Director Técnico
Binovo IT Human Project, S.L.
Telf. 943569206
Astigarragako bidea 2, 2º izq. oficina 11; 20180 Oiartzun (Gipuzkoa)
www.binovo.es