[pve-devel] pve-network : new proposal for apply/reload config

[Alexandre DERUMIER]

>>I was a bit too optimist with ifquery -a -r (forget the -r for running config, 15s was to parse the file config), it's more around 2min for 2000vnets, I have debug it, it's looking about 
>>a lot of property in sysfs, but also calling command (brctl,...), and it's can slowdown things. 
>>
>>I think it should not be too difficult to implement a parser of sysfs, to get the running config (on a limited scope, related to the plugins && vnet option) 
>>I can parse whole sysfs vnet in 2-3s (with 2000vnets + 2000 vxlans interfaces), so it should be faster. 

Ok, I'll use ifquery finally, as not all attributes are available in sysfs. (like vxlan attribute, only queriable through netlink).
I have fixed slowdown of ifquery (I will send PR to ifupdown2), it's now around 15s with 2000vnets && 2000vxlan.(most time is in netlink query)
I have talked with ifupdown2 devs, it seem that ifupdown 2.0 should come for this summer, with full netlink support (no more sysfs), and with a daemon to watch/cache netlink
changes,  so ifquery/ifreload with a lot of interfaces should be really faster.




----- Mail original -----
De: "aderumier" <aderumier at odiso.com>
À: "dietmar" <dietmar at proxmox.com>
Cc: "pve-devel" <pve-devel at pve.proxmox.com>
Envoyé: Mardi 7 Mai 2019 07:59:13
Objet: Re: [pve-devel] pve-network : new proposal for apply/reload config

>>Inside pvestad, you can do a fork() to avoid blocking other things. 
ok great 


>>> for each vnet and vnet bridge-ports 
>>> compare running_config with /etc/network/interfaces.d/vnet 
>>> status: ok / error (vnet missing, or different config(vlan tag,...)) -> broadcast status 
>>> 

>>worth to try. Maybe simply write the result to a file on /etc/pve? 

I was a bit too optimist with ifquery -a -r (forget the -r for running config, 15s was to parse the file config), it's more around 2min for 2000vnets, I have debug it, it's looking about 
a lot of property in sysfs, but also calling command (brctl,...), and it's can slowdown things. 

I think it should not be too difficult to implement a parser of sysfs, to get the running config (on a limited scope, related to the plugins && vnet option) 
I can parse whole sysfs vnet in 2-3s (with 2000vnets + 2000 vxlans interfaces), so it should be faster. 

But yes, we could store status in /etc/pve too without need to broadcast. 



>>I thought we can apply pending changes when a node starts up? 
yes, I have also thinked about this, I just need to do it after /etc/pve retrieval. 
(1) network start, 2) pmxcfs is starting 3) reload network configuration if changes are detected 


----- Mail original ----- 
De: "dietmar" <dietmar at proxmox.com> 
À: "pve-devel" <pve-devel at pve.proxmox.com>, "aderumier" <aderumier at odiso.com> 
Envoyé: Mardi 7 Mai 2019 06:35:45 
Objet: Re: [pve-devel] pve-network : new proposal for apply/reload config 

comment inline ... 

> I have rethinked about reloading config, without extra daemon: 
> 
> 
> " 
> datacenter level: 
> -commit config : - mv /etc/pve/networks.cfg.new /etc/pve/networks.cfg 
> - call each (online) node reload api. 
> 
> 
> local node: 
> reload api: -> generate /etc/network/interfaces.d/vnet 
> merge pending /etc/network/interfaces.new to /etc/network/interfaces 
> ifreload -a (2minutes for 2000 vnets, but take 15s at boot ( ifup -a --perfmode) 
> ) 
> 
> 
> pvestatd: 
> 
> get running config : ifquery -a -r (15second for 2000vnets -> maybe make a cache(cron?otherdaemon?) to not block pvestatd ?) 

Inside pvestad, you can do a fork() to avoid blocking other things. 

> for each vnet and vnet bridge-ports 
> compare running_config with /etc/network/interfaces.d/vnet 
> status: ok / error (vnet missing, or different config(vlan tag,...)) -> broadcast status 
> 

worth to try. Maybe simply write the result to a file on /etc/pve? 

> (they are also a "ifquery -c <int>" to compare running and configuration, but it's interface by interface, and maybe more slower, 
> so it could be used to have an accurate detail on a specific vnet in the gui) 
> 
> Error should occur because a conflict with a local setting (already used vlan, missing uplink-id,...) 
> So user can easy see errors from pvestatd status for each vnet. (displayed in the tree ?). 
> then for fix error, user go the local network configuration, do local network fix changes, 
> and call reload api again. 
> 
> If the node was offline when we have commit network config, the vnets will also be displayed in error (different config, missing vnet,...), 

I thought we can apply pending changes when a node starts up? 

> so user can also reload it manually from local config. 
> 
> " 
> 
> 
> I think this give more control to user, and avoid to have a background daemon reloading networking. (More simpler to fix error, avoid reload loop,....) 
> 
> What do you think about this ? 

Sounds good. Now there is less magic, and the admin has more control ... 

_______________________________________________ 
pve-devel mailing list 
pve-devel at pve.proxmox.com 
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel