[PVE-User] Cluster network via directly connected interfaces?

[Stefan M. Radman]

Hi Ronny

That's the first time I hear of a routing protocol in the corosync context.
Doesn't that add a whole lot of complexity in the setup?
Would it work with corosync multicast?

Stefan

> On Nov 23, 2018, at 12:00 PM, Ronny Aasen <ronny+pve-user at aasen.cx> wrote:
> 
> Personally if i was to try and experiment with something non-default I would try to use ospf+bfd either with bird or quagga.
> 
> -you get quick failovers due to bfd.
> -you can equal cost multipath links to utillize multiple ports between servers.
> -All links are active, so you do not have a "passive" link, as you have with STP
> -and there is no needless duplication of data, so you do not get the 50% bandwith loss of a broadcast bond.
> -you need to use corosync with targeted udp towards spesific loopback addresses.
> -traffic goes shortest path. so allways towards the correct server.
> - you can very easily expand beyond 3 nodes if you have enough ports. Or move the ospf domain onto a switch if needed. this also easily converts to a multiple switch config to maintain HA and no SPOF
> 
> Happy experimentation!
> 
> mvh
> Ronny Aasen
> 
> 
> 
> 
> 
> On 11/22/18 7:29 PM, Frank Thommen wrote:
>> Please excuse, if this is too basic, but after reading https://pve.proxmox.com/wiki/Cluster_Manager I wondered, if the cluster/corosync network could be built by directly connected network interfaces.  I.e not like this:
>>  +-------+
>>  | pve01 |----------+
>>  +-------+          |
>>                     |
>>  +-------+     +----------------+
>>  | pve02 |-----| network switch |
>>  +-------+     +----------------+
>>                     |
>>  +-------+          |
>>  | pve03 |----------+
>>  +-------+
>> but like this:
>>  +-------+
>>  | pve01 |---+
>>  +-------+   |
>>      |       |
>>  +-------+   |
>>  | pve02 |   |
>>  +-------+   |
>>      |       |
>>  +-------+   |
>>  | pve03 |---+
>>  +-------+
>> (all connections 1Gbit, there are currently not plans to extend over three nodes)
>> I can't see any drawback in that solution.  It would remove one layer of hardware dependency and potential spof (the switch).  If we don't trust the interfaces, we might be able to configure a second network with the three remaining interfaces.
>> Is such a "direct-connection" topology feasible?  Recommended? Strictly not recommended?
>> I am currently just planning and thinking and there is no cluster (or even a PROXMOX server) in place.
>> Cheers
>> frank
>> _______________________________________________
>> pve-user mailing list
>> pve-user at pve.proxmox.com
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
> 
> _______________________________________________
> pve-user mailing list
> pve-user at pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user