[pve-devel] pvefw: masquerade problems and conntrack zones

Mon Mar 10 14:56:55 CET 2014

just found this, not sure it's related:

http://www.spinics.net/lists/netfilter-devel/msg13554.html

    "2.6.34 introduced 'conntrack zones' to deal with cases where packets
    from multiple identical networks are handled by conntrack/NAT. Packets
    are looped through veth devices, during which they are NATed to private
    addresses, after which they can continue normally through the stack
    and possibly have NAT rules applied a second time."

    This works well, but is needlessly complicated for cases where only
    a single SNAT/DNAT mapping needs to be applied to these packets. In that
    case, all that needs to be done is to assign each network to a seperate
    zone and perform NAT as usual. However this doesn't work for packets
    destined for the machine performing NAT itself since its corrently not
    possible to configure SNAT mappings for the LOCAL_IN chain.

----- Mail original ----- 

De: "Alexandre DERUMIER" <aderumier at odiso.com> 
À: "Dietmar Maurer" <dietmar at proxmox.com> 
Cc: pve-devel at pve.proxmox.com 
Envoyé: Lundi 10 Mars 2014 14:50:50 
Objet: Re: [pve-devel] pvefw: masquerade problems and conntrack zones 

>>Oh, I have a second bridge configured as: 

oh,ok. 

so, it seem that 

>>iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -j LOG --log-prefix "MASQTEST: " 

show, that packet with source 10.10.10.0/24 in POSTROUTING, is tested against each output interface (maybe interfaces with ip adress configured)? 

>>Mar 10 11:25:34 lola kernel: [259254.043987] MASQTEST: IN= OUT=vmbr1 PHYSIN=tap116i0 PHYSOUT=pm1peer SRC=10.10.10.3 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=5639 SEQ=1 >>MARK=0x1 
>>Mar 10 11:25:34 lola kernel: [259254.044020] MASQTEST: IN= OUT=pm0 SRC=10.10.10.3 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=5639 SEQ=1 

then 

>> post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o pm0 -j MASQUERADE 

is doing his job, to nat only on pm0 

now, I don't known for the --zone 1 ..... 
(So it doesn't work on 2.6.32 without --zone 1 ?) 

----- Mail original ----- 

De: "Dietmar Maurer" <dietmar at proxmox.com> 
À: "Alexandre DERUMIER" <aderumier at odiso.com> 
Cc: pve-devel at pve.proxmox.com 
Envoyé: Lundi 10 Mars 2014 12:36:39 
Objet: RE: pvefw: masquerade problems and conntrack zones 

> >>So it seem that now the POSTROUTING chain gets called twice. The second 
> call has the correct output interface. 
> 
> what is pm0 vs pm1peer ? 

Oh, I have a second bridge configured as: 

auto vmbr0 
iface vmbr0 inet manual 
bridge_ports bond0 
bridge_stp off 
bridge_fd 0 

auto pm0 
iface pm0 inet static 
... 
VETH_BRIDGETO vmbr0 

My ifupdown.sh helper script creates a veth interface with: 

ip link add name "${IFACE}" type veth peer name "${IFACE}peer" || exit 1 

same for pm1 

> 
> 
> >>I am a bit scared, because the whole thing is not documented. Are there 
> other projects using such setup? 
> afaik, openstack and cloudstack managed only bridged firewall, I don't have 
> see nat implementation.(I'll check that today) 
> 
> ----- Mail original ----- 
> 
> De: "Dietmar Maurer" <dietmar at proxmox.com> 
> À: "Dietmar Maurer" <dietmar at proxmox.com>, "Alexandre DERUMIER" 
> <aderumier at odiso.com> 
> Cc: pve-devel at pve.proxmox.com 
> Envoyé: Lundi 10 Mars 2014 11:36:33 
> Objet: RE: pvefw: masquerade problems and conntrack zones 
> 
> > > post-up iptables -t raw -A PREROUTING -s '10.10.10.0/24' -i vmbr1 -j 
> > > CT --zone 
> > > 1 # why do we need this? 
> > > post-up iptables -t raw -A PREROUTING -d '10.10.10.0/24' -i vmbr1 -j 
> > > CT -- zone 1 # why do we need this? 
> > > post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o pm0 -j 
> > > MASQUERADE >> apply on default zone 0 
> > > 
> > > 
> > > so, that should mean that apply -j MASQUERADE don't apply on vmbr1 
> > > with zone 1 
> > 
> > Sure, but why is that required? Are there negative side effects? Any 
> > ideas? I have not found any documentation. 
> 
> My guess is that POSTROUTING chain gets called to early (because bridge 
> filter is enabled). Using: 
> 
> auto pm1 
> iface pm1 inet static 
> address 10.10.10.1 
> netmask 255.255.255.0 
> VETH_BRIDGETO vmbr1 
> ##disabled# post-up iptables -t raw -A PREROUTING -s '10.10.10.0/24' -i 
> vmbr1 -j CT --zone 1 ##disabled# post-up iptables -t raw -A PREROUTING -d 
> '10.10.10.0/24' -i vmbr1 -j CT --zone 1 post-up iptables -t nat -A 
> POSTROUTING -s '10.10.10.0/24' -j LOG --log-prefix "MASQTEST: " 
> post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o pm0 -j 
> MASQUERADE post-down iptables -t nat -F POSTROUTING post-down 
> iptables -t raw -F PREROUTING 
> 
> A simple ping results in this log: 
> 
> Mar 10 11:26:17 lola kernel: [259296.464749] MASQTEST: IN= OUT=vmbr1 
> PHYSIN=tap116i0 PHYSOUT=pm1peer SRC=10.10.10.3 DST=8.8.8.8 LEN=84 
> TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=5645 
> SEQ=1 MARK=0x1 
> 
> Note: "-o pm0 -j MASQUERADE" does not trigger at all! 
> 
> If I enable zones I get: 
> 
> Mar 10 11:25:34 lola kernel: [259254.043987] MASQTEST: IN= OUT=vmbr1 
> PHYSIN=tap116i0 PHYSOUT=pm1peer SRC=10.10.10.3 DST=8.8.8.8 LEN=84 
> TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=5639 
> SEQ=1 MARK=0x1 Mar 10 11:25:34 lola kernel: [259254.044020] MASQTEST: 
> IN= OUT=pm0 SRC=10.10.10.3 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 
> TTL=63 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=5639 SEQ=1 
> 
> So it seem that now the POSTROUTING chain gets called twice. The second 
> call has the correct output interface. 
> 
> I am a bit scared, because the whole thing is not documented. Are there 
> other projects using such setup? 
_______________________________________________ 
pve-devel mailing list 
pve-devel at pve.proxmox.com 
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 