[PVE-User] Thin LVM showing more used space than expected

[Óscar de Arriba]

Hello all,

>From ~1 week ago, one of my Proxmox nodes' data LVM is doing strange things.

 For storage, I'm using a commercial Crucial MX500 SATA SSD connected directly to the motherboard controller (no PCIe HBA for the system+data disk) and it is brand new - and S.M.A.R.T. checks are passing, only 4% of wearout. I have set up proxmox inside a cluster with LVM and making backups to a NFS external location.

Last week I tried to migrate an stopped VM of ~64 GiB from one server to another, and found out *the SSD started to underperform (~5 MB/s) after roughly 55 GiB copied *(this pattern was repeated several times). 
It was so bad that *even cancelling the migration, the SSD continued busy writting at that speeed and I need to reboot the instance, as it was completely unusable* (it is in my homelab, not running mission critical workloads, so it was okay to do that). After the reboot, I could remove the half-copied VM disk.

After that, (and several retries, even making a backup to an external storage and trying to restore the backup, just in case the bottleneck was on the migration process) I ended up creating the instance from scratch and migrating data from one VM to another - so the VM was crearted brand new and no bottleneck was hit.

The problem is that *now the pve/data logical volume is showing 377 GiB used, but the total size of stored VM disks (even if they are 100% approvisioned) is 168 GiB*. I checked and both VMs have no snapshots. 

I don't know if the reboot while writting to the disk (always having cancelled the migration first) damaged the LV in some way, but after thinking about it it does not even make sense that an SSD of this type ends up writting at 5 MB/s, even with the writting cache full. It should be writting far faster than that even without cache.

Some information about the storage:

`root at venom:~# lvs -a
  LV              VG  Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  data            pve twi-aotz-- 377.55g             96.13  1.54                           
  [data_tdata]    pve Twi-ao---- 377.55g                                                   
  [data_tmeta]    pve ewi-ao----  <3.86g                                                   
  [lvol0_pmspare] pve ewi-------  <3.86g                                                   
  root            pve -wi-ao----  60.00g                                                   
  swap            pve -wi-ao----   4.00g                                                   
  vm-150-disk-0   pve Vwi-a-tz--   4.00m data        14.06                                 
  vm-150-disk-1   pve Vwi-a-tz-- 128.00g data        100.00                                 
  vm-201-disk-0   pve Vwi-aotz--   4.00m data        14.06                                 
  vm-201-disk-1   pve Vwi-aotz--  40.00g data        71.51`

and can be also seen on this post on the forum I did a couple of days ago: https://forum.proxmox.com/threads/thin-lvm-showing-more-used-space-than-expected.120051/

Any ideas aside from doing a backup and reinstall from scratch?

Thanks in advance!