[PVE-User] Disk full errors but plenty of free space reported (KVM io-error status)
francesco.ongaro at isgroup.it
Thu Aug 27 01:09:11 CEST 2015
lately on a single node PVE server I'm experiencing what could seem
a full disk error. The issue is that the machine had ~866GB free and
plenty of unused inodes but things were definitely not okay.
Is this something that you ever experienced on your proxmox setup?
For whom who want the full details, here they are :) My mountpoint,
let's call it FOO, is a 1.8T ext3 filesystem, hosting some openvz and
KVM machines, backups, iso, etc. Proxmox version is 3.1-21 on a Debian
The issue manifests in two ways:
- Backups errors both for KVM (ERROR: vma_co_write write error - Broken
pipe) and OpenVZ (rsync says "No space left on device (28)" and lzop
"No space left on device: <stdout>").
- About two weeks later some KVM machines shown io-error as status, I
was able to reproduce it by starting a defrag on a Windows vm. All the
other vm, regardless their OS, freeze and show "io-error", when writing
My mountpoint is pretty standard:
/dev/mapper/FOO on /FOO type ext3
And the disk is half full:
# df -h
Filesystem Size Used Avail Use% Mounted on
udev 10M 0 10M 0% /dev
tmpfs 2.4G 444K 2.4G 1% /run
/dev/mapper/pve-root 95G 36G 55G 40% /
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 4.8G 9.2M 4.7G 1% /run/shm
/dev/mapper/pve-data 337G 8.2G 329G 3% /var/lib/vz
/dev/sda1 495M 81M 389M 18% /boot
/dev/fuse 30M 32K 30M 1% /etc/pve
/dev/mapper/FOO 1.8T 876G 866G 51% /FOO
I read about others who had seemingly similar issues but disk space was
correctly reported as full:
Any ideas on how to debug this issue?
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 375 bytes
Desc: not available
More information about the pve-user