[PVE-User] GlusterFS slow resync

Lindsay Mathieson lindsay.mathieson at gmail.com
Tue Jul 28 14:00:47 CEST 2015

Unfortunately yes, GlusterFS is very slow to resync. When it detects a file is out of sync it has two methods to resync.

1. Copy the entire file across – bandwidth intensive
2. Compare checksums of each block in the file and copy across blocks that don’t match. CPU intensive.

Both methods are very slow for large files

Sent from Mail for Windows 10

From: Fabrizio Cuseo
Sent: Tuesday, 28 July 2015 8:33 PM
To: pve-user at pve.proxmox.com
Subject: [PVE-User] GlusterFS slow resync

I am testing a 2 nodes cluster with glusterFS.

GlusterFS is configured with 4 (raid-5) 1Tbyte disk, so there are 2 bricks replicated (1 per server).

I have a VM with a VirtualDisk qcow2 of 2,5Tbyte, and the file is now 950Gbyte.

After rebooting one of the two nodes, glusterFS needs more than 12 hours to resync (the nodes are connected with a p2p infiniband 10Gbit card).

Someone have similar problems ? Is GlusterFS not so usable for big VM's ? 

Regards, Fabrizio 

Fabrizio Cuseo - mailto:f.cuseo at panservice.it
Direzione Generale - Panservice InterNetWorking
Servizi Professionali per Internet ed il Networking
Panservice e' associata AIIP - RIPE Local Registry
Phone: +39 0773 410020 - Fax: +39 0773 470219
http://www.panservice.it  mailto:info at panservice.it
Numero verde nazionale: 800 901492
pve-user mailing list
pve-user at pve.proxmox.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.proxmox.com/pipermail/pve-user/attachments/20150728/9dcb332f/attachment.htm>

More information about the pve-user mailing list