[PVE-User] Needing help in planning network and storage for new cluster

[Stefan Sondermann]

Hi there all,

hopefully this might be the right place to ask some questions. I've
used esxi and now XenServer until now for private gameserver for some
of my friends and me. As the community got pretty big we are now
moving to another provider with some better setup possitilities.

The setup is as follows:
- 3 server each with 1,5TB of SSD storage (1TB usable)
- public ip on eth0 for each server, 500mbit guaranteed bandwidth
- private ip on eth1 for each server, 100mbit bandwidth
- only the MAC of the physical interface allowed on each eth1 network
- 1TB iSCSI Storage accessible via private eth1 network only
- VPN access to the private network to access eth1 interfaces and
iSCSI over secured connection

Now I have some questions where I run into a deadlock.
1. I'm planning the network. Our old servers had 1 network for VM
traffic, management network and storage traffic. With the
possibilities of the second interface not accessible from internet we
have new possibilities. Should I only setup PVE cluster on that
private network and route the storage traffic over this? Should I also
route the VM interchange traffic over it (SQL Server <-> Services)? Is
there the possibility that some storage migration may block cluster
pings or can I set a priority? The cluster is manually managed and
balanced. No HA or something is configured.

2. Should I setup a routing VM (i.e. pfSense, clearos) for routing
different ports to different vm's or should I set it up with proxmox
firewall settings? Is this possible ressource pool wide?

3. Is there a easy openvSwitch controller guide or something like
this? I want to archive that the 3 nodes use the same virtual private
network but which spans over the physical hosts. As far as I know one
may need an OVS controller?

Any other hints or thoughts on the setup anybody have? Any help is
much appreciated.

Kind regards,
Stefan