<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p
{mso-style-priority:99;
margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1820995559;
mso-list-type:hybrid;
mso-list-template-ids:-178254060 -1 134807577 134807579 134807567 134807577 134807579 134807567 134807577 134807579;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style></head><body lang=EN-AU link=blue vlink="#954F72"><div class=WordSection1><p>Unfortunately yes, GlusterFS is very slow to resync. When it detects a file is out of sync it has two methods to resync.</p><p><o:p> </o:p></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>1.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Copy the entire file across – bandwidth intensive</p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>2.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Compare checksums of each block in the file and copy across blocks that don’t match. CPU intensive.</p><p><o:p> </o:p></p><p>Both methods are very slow for large files</p><p><o:p> </o:p></p><p>Sent from <a href="http://go.microsoft.com/fwlink/?LinkId=550986">Mail</a> for Windows 10</p><p><o:p> </o:p></p><p><o:p> </o:p></p><div style='mso-element:para-border-div;border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p style='border:none;padding:0cm'><br><b>From: </b>Fabrizio Cuseo<br><b>Sent: </b>Tuesday, 28 July 2015 8:33 PM<br><b>To: </b>pve-user@pve.proxmox.com<br><b>Subject: </b>[PVE-User] GlusterFS slow resync</p></div><p class=MsoNormal><span style='font-family:"Times New Roman",serif'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-family:"Times New Roman",serif'><o:p> </o:p></span></p><p class=MsoNormal>Hello.</p><p class=MsoNormal>I am testing a 2 nodes cluster with glusterFS.</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>GlusterFS is configured with 4 (raid-5) 1Tbyte disk, so there are 2 bricks replicated (1 per server).</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I have a VM with a VirtualDisk qcow2 of 2,5Tbyte, and the file is now 950Gbyte.</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>After rebooting one of the two nodes, glusterFS needs more than 12 hours to resync (the nodes are connected with a p2p infiniband 10Gbit card).</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Someone have similar problems ? Is GlusterFS not so usable for big VM's ? </p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Regards, Fabrizio </p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>-- </p><p class=MsoNormal>---</p><p class=MsoNormal>Fabrizio Cuseo - mailto:f.cuseo@panservice.it</p><p class=MsoNormal>Direzione Generale - Panservice InterNetWorking</p><p class=MsoNormal>Servizi Professionali per Internet ed il Networking</p><p class=MsoNormal>Panservice e' associata AIIP - RIPE Local Registry</p><p class=MsoNormal>Phone: +39 0773 410020 - Fax: +39 0773 470219</p><p class=MsoNormal>http://www.panservice.it mailto:info@panservice.it</p><p class=MsoNormal>Numero verde nazionale: 800 901492</p><p class=MsoNormal>_______________________________________________</p><p class=MsoNormal>pve-user mailing list</p><p class=MsoNormal>pve-user@pve.proxmox.com</p><p class=MsoNormal>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p></div></body></html>