[pve-devel] pvefw: masquerade problems and conntrack zones

[Alexandre DERUMIER]

>>Mar 10 11:25:34 lola kernel: [259254.043987] MASQTEST: IN= OUT=vmbr1 PHYSIN=tap116i0 PHYSOUT=pm1peer SRC=10.10.10.3 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=5639 SEQ=1 >>MARK=0x1 
>>Mar 10 11:25:34 lola kernel: [259254.044020] MASQTEST: IN= OUT=pm0 SRC=10.10.10.3 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=5639 SEQ=1 
>>
>>So it seem that now the POSTROUTING chain gets called twice. The second call has the correct output interface. 

what is pm0 vs pm1peer ?


>>I am a bit scared, because the whole thing is not documented. Are there other projects using such setup? 
afaik, openstack and cloudstack managed only bridged firewall, I don't have see nat implementation.(I'll check that today)

----- Mail original ----- 

De: "Dietmar Maurer" <dietmar at proxmox.com> 
À: "Dietmar Maurer" <dietmar at proxmox.com>, "Alexandre DERUMIER" <aderumier at odiso.com> 
Cc: pve-devel at pve.proxmox.com 
Envoyé: Lundi 10 Mars 2014 11:36:33 
Objet: RE: pvefw: masquerade problems and conntrack zones 

> > post-up iptables -t raw -A PREROUTING -s '10.10.10.0/24' -i vmbr1 -j 
> > CT --zone 
> > 1 # why do we need this? 
> > post-up iptables -t raw -A PREROUTING -d '10.10.10.0/24' -i vmbr1 -j 
> > CT -- zone 1 # why do we need this? 
> > post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o pm0 -j 
> > MASQUERADE >> apply on default zone 0 
> > 
> > 
> > so, that should mean that apply -j MASQUERADE don't apply on vmbr1 
> > with zone 1 
> 
> Sure, but why is that required? Are there negative side effects? Any ideas? I 
> have not found any documentation. 

My guess is that POSTROUTING chain gets called to early (because bridge filter is enabled). Using: 

auto pm1 
iface pm1 inet static 
address 10.10.10.1 
netmask 255.255.255.0 
VETH_BRIDGETO vmbr1 
##disabled# post-up iptables -t raw -A PREROUTING -s '10.10.10.0/24' -i vmbr1 -j CT --zone 1 
##disabled# post-up iptables -t raw -A PREROUTING -d '10.10.10.0/24' -i vmbr1 -j CT --zone 1 
post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -j LOG --log-prefix "MASQTEST: " 
post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o pm0 -j MASQUERADE 
post-down iptables -t nat -F POSTROUTING 
post-down iptables -t raw -F PREROUTING 

A simple ping results in this log: 

Mar 10 11:26:17 lola kernel: [259296.464749] MASQTEST: IN= OUT=vmbr1 PHYSIN=tap116i0 PHYSOUT=pm1peer SRC=10.10.10.3 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=5645 SEQ=1 MARK=0x1 

Note: "-o pm0 -j MASQUERADE" does not trigger at all! 

If I enable zones I get: 

Mar 10 11:25:34 lola kernel: [259254.043987] MASQTEST: IN= OUT=vmbr1 PHYSIN=tap116i0 PHYSOUT=pm1peer SRC=10.10.10.3 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=5639 SEQ=1 MARK=0x1 
Mar 10 11:25:34 lola kernel: [259254.044020] MASQTEST: IN= OUT=pm0 SRC=10.10.10.3 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=5639 SEQ=1 

So it seem that now the POSTROUTING chain gets called twice. The second call has the correct output interface. 

I am a bit scared, because the whole thing is not documented. Are there other projects using such setup?