[PVE-User] 10G mesh to 40G DAC

[mj]

Hi,

I just wanted to share my experience of upgrading our cluster from 10G 
mesh networking 
(https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server) to 
arista 40G MLAG over DAC cables.

I expected DAC over switches to have a slightly increased latency over 
direct (meshed) UTP RJ45 connections. Throughput would of course be much 
higher in the new config. Ceph (on SSDs) is used mainly for VM images, RBD.

So, after having completed the migration yesterday, I can say that the 
VMs feel much quicker. Specially the VMs using databases (LAMP servers 
and a couple of management systems) are much more responsive now than 
they were before.

Over all, we're very happy now. For the record, we're using dual arista 
DCS-7050QX-32S MLAG now. Purchased refurb for around 1000 euro each, 
including three years warranty.

Just thought I'd share. :-)

MJ