[pve-devel] pve-firewall benchmark result

Fri Mar 21 06:27:02 CET 2014

Hi, I have done some benchmark, to see if using
 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT in begin of FORWARD, is really helping

benchmark setup:
----------------
host : old   2x xeon 5110  @ 1.60GHz

2 vms (vm123, vm110), same bridge, using iperf (tcp 5001):

server (vm110) : iperf -S
client (vm123) : iperf -c X.X.X.X

FIREWALL SETUP
--------------
I have added 100tap device in one bridge, with worse case, tap110 and tap123 at the of the bridge
only 1 tcp 5001 rule is created

-A vmbr1-FW -m physdev --physdev-is-in -j vmbr1-OUT
-A vmbr1-FW -m physdev --physdev-is-out -j vmbr1-IN
-A vmbr1-FW -m mark --mark 0x1 -j ACCEPT
-A vmbr1-FW -m physdev --physdev-is-out -j ACCEPT
-A vmbr1-FW -m comment --comment "PVESIG:fmNVk/D2Npe3kjrx6hn27VKjdMg"
-A vmbr1-IN -m physdev --physdev-out tap99 --physdev-is-bridged -j tap99-IN
-A vmbr1-IN -m physdev --physdev-out tap98 --physdev-is-bridged -j tap98-IN
-A vmbr1-IN -m physdev --physdev-out tap97 --physdev-is-bridged -j tap97-IN
-A vmbr1-IN -m physdev --physdev-out tap96 --physdev-is-bridged -j tap96-IN
-A vmbr1-IN -m physdev --physdev-out tap95 --physdev-is-bridged -j tap95-IN
-A vmbr1-IN -m physdev --physdev-out tap94 --physdev-is-bridged -j tap94-IN
-A vmbr1-IN -m physdev --physdev-out tap93 --physdev-is-bridged -j tap93-IN
-A vmbr1-IN -m physdev --physdev-out tap92 --physdev-is-bridged -j tap92-IN
-A vmbr1-IN -m physdev --physdev-out tap91 --physdev-is-bridged -j tap91-IN
-A vmbr1-IN -m physdev --physdev-out tap90 --physdev-is-bridged -j tap90-IN
-A vmbr1-IN -m physdev --physdev-out tap89 --physdev-is-bridged -j tap89-IN
-A vmbr1-IN -m physdev --physdev-out tap88 --physdev-is-bridged -j tap88-IN
-A vmbr1-IN -m physdev --physdev-out tap87 --physdev-is-bridged -j tap87-IN
-A vmbr1-IN -m physdev --physdev-out tap86 --physdev-is-bridged -j tap86-IN
-A vmbr1-IN -m physdev --physdev-out tap85 --physdev-is-bridged -j tap85-IN
-A vmbr1-IN -m physdev --physdev-out tap84 --physdev-is-bridged -j tap84-IN
-A vmbr1-IN -m physdev --physdev-out tap83 --physdev-is-bridged -j tap83-IN
-A vmbr1-IN -m physdev --physdev-out tap82 --physdev-is-bridged -j tap82-IN
-A vmbr1-IN -m physdev --physdev-out tap81 --physdev-is-bridged -j tap81-IN
-A vmbr1-IN -m physdev --physdev-out tap80 --physdev-is-bridged -j tap80-IN
-A vmbr1-IN -m physdev --physdev-out tap79 --physdev-is-bridged -j tap79-IN
-A vmbr1-IN -m physdev --physdev-out tap78 --physdev-is-bridged -j tap78-IN
-A vmbr1-IN -m physdev --physdev-out tap77 --physdev-is-bridged -j tap77-IN
-A vmbr1-IN -m physdev --physdev-out tap76 --physdev-is-bridged -j tap76-IN
-A vmbr1-IN -m physdev --physdev-out tap75 --physdev-is-bridged -j tap75-IN
-A vmbr1-IN -m physdev --physdev-out tap74 --physdev-is-bridged -j tap74-IN
-A vmbr1-IN -m physdev --physdev-out tap73 --physdev-is-bridged -j tap73-IN
-A vmbr1-IN -m physdev --physdev-out tap72 --physdev-is-bridged -j tap72-IN
-A vmbr1-IN -m physdev --physdev-out tap71 --physdev-is-bridged -j tap71-IN
-A vmbr1-IN -m physdev --physdev-out tap70 --physdev-is-bridged -j tap70-IN
-A vmbr1-IN -m physdev --physdev-out tap69 --physdev-is-bridged -j tap69-IN
-A vmbr1-IN -m physdev --physdev-out tap68 --physdev-is-bridged -j tap68-IN
-A vmbr1-IN -m physdev --physdev-out tap67 --physdev-is-bridged -j tap67-IN
-A vmbr1-IN -m physdev --physdev-out tap66 --physdev-is-bridged -j tap66-IN
-A vmbr1-IN -m physdev --physdev-out tap65 --physdev-is-bridged -j tap65-IN
-A vmbr1-IN -m physdev --physdev-out tap64 --physdev-is-bridged -j tap64-IN
-A vmbr1-IN -m physdev --physdev-out tap63 --physdev-is-bridged -j tap63-IN
-A vmbr1-IN -m physdev --physdev-out tap62 --physdev-is-bridged -j tap62-IN
-A vmbr1-IN -m physdev --physdev-out tap61 --physdev-is-bridged -j tap61-IN
-A vmbr1-IN -m physdev --physdev-out tap60 --physdev-is-bridged -j tap60-IN
-A vmbr1-IN -m physdev --physdev-out tap59 --physdev-is-bridged -j tap59-IN
-A vmbr1-IN -m physdev --physdev-out tap58 --physdev-is-bridged -j tap58-IN
-A vmbr1-IN -m physdev --physdev-out tap57 --physdev-is-bridged -j tap57-IN
-A vmbr1-IN -m physdev --physdev-out tap56 --physdev-is-bridged -j tap56-IN
-A vmbr1-IN -m physdev --physdev-out tap55 --physdev-is-bridged -j tap55-IN
-A vmbr1-IN -m physdev --physdev-out tap54 --physdev-is-bridged -j tap54-IN
-A vmbr1-IN -m physdev --physdev-out tap53 --physdev-is-bridged -j tap53-IN
-A vmbr1-IN -m physdev --physdev-out tap52 --physdev-is-bridged -j tap52-IN
-A vmbr1-IN -m physdev --physdev-out tap51 --physdev-is-bridged -j tap51-IN
-A vmbr1-IN -m physdev --physdev-out tap50 --physdev-is-bridged -j tap50-IN
-A vmbr1-IN -m physdev --physdev-out tap49 --physdev-is-bridged -j tap49-IN
-A vmbr1-IN -m physdev --physdev-out tap48 --physdev-is-bridged -j tap48-IN
-A vmbr1-IN -m physdev --physdev-out tap47 --physdev-is-bridged -j tap47-IN
-A vmbr1-IN -m physdev --physdev-out tap46 --physdev-is-bridged -j tap46-IN
-A vmbr1-IN -m physdev --physdev-out tap45 --physdev-is-bridged -j tap45-IN
-A vmbr1-IN -m physdev --physdev-out tap44 --physdev-is-bridged -j tap44-IN
-A vmbr1-IN -m physdev --physdev-out tap43 --physdev-is-bridged -j tap43-IN
-A vmbr1-IN -m physdev --physdev-out tap42 --physdev-is-bridged -j tap42-IN
-A vmbr1-IN -m physdev --physdev-out tap41 --physdev-is-bridged -j tap41-IN
-A vmbr1-IN -m physdev --physdev-out tap40 --physdev-is-bridged -j tap40-IN
-A vmbr1-IN -m physdev --physdev-out tap39 --physdev-is-bridged -j tap39-IN
-A vmbr1-IN -m physdev --physdev-out tap38 --physdev-is-bridged -j tap38-IN
-A vmbr1-IN -m physdev --physdev-out tap37 --physdev-is-bridged -j tap37-IN
-A vmbr1-IN -m physdev --physdev-out tap36 --physdev-is-bridged -j tap36-IN
-A vmbr1-IN -m physdev --physdev-out tap35 --physdev-is-bridged -j tap35-IN
-A vmbr1-IN -m physdev --physdev-out tap34 --physdev-is-bridged -j tap34-IN
-A vmbr1-IN -m physdev --physdev-out tap33 --physdev-is-bridged -j tap33-IN
-A vmbr1-IN -m physdev --physdev-out tap32 --physdev-is-bridged -j tap32-IN
-A vmbr1-IN -m physdev --physdev-out tap31 --physdev-is-bridged -j tap31-IN
-A vmbr1-IN -m physdev --physdev-out tap30 --physdev-is-bridged -j tap30-IN
-A vmbr1-IN -m physdev --physdev-out tap29 --physdev-is-bridged -j tap29-IN
-A vmbr1-IN -m physdev --physdev-out tap28 --physdev-is-bridged -j tap28-IN
-A vmbr1-IN -m physdev --physdev-out tap27 --physdev-is-bridged -j tap27-IN
-A vmbr1-IN -m physdev --physdev-out tap26 --physdev-is-bridged -j tap26-IN
-A vmbr1-IN -m physdev --physdev-out tap25 --physdev-is-bridged -j tap25-IN
-A vmbr1-IN -m physdev --physdev-out tap24 --physdev-is-bridged -j tap24-IN
-A vmbr1-IN -m physdev --physdev-out tap23 --physdev-is-bridged -j tap23-IN
-A vmbr1-IN -m physdev --physdev-out tap22 --physdev-is-bridged -j tap22-IN
-A vmbr1-IN -m physdev --physdev-out tap21 --physdev-is-bridged -j tap21-IN
-A vmbr1-IN -m physdev --physdev-out tap20 --physdev-is-bridged -j tap20-IN
-A vmbr1-IN -m physdev --physdev-out tap19 --physdev-is-bridged -j tap19-IN
-A vmbr1-IN -m physdev --physdev-out tap18 --physdev-is-bridged -j tap18-IN
-A vmbr1-IN -m physdev --physdev-out tap17 --physdev-is-bridged -j tap17-IN
-A vmbr1-IN -m physdev --physdev-out tap16 --physdev-is-bridged -j tap16-IN
-A vmbr1-IN -m physdev --physdev-out tap15 --physdev-is-bridged -j tap15-IN
-A vmbr1-IN -m physdev --physdev-out tap14 --physdev-is-bridged -j tap14-IN
-A vmbr1-IN -m physdev --physdev-out tap13 --physdev-is-bridged -j tap13-IN
-A vmbr1-IN -m physdev --physdev-out tap12 --physdev-is-bridged -j tap12-IN
-A vmbr1-IN -m physdev --physdev-out tap11 --physdev-is-bridged -j tap11-IN
-A vmbr1-IN -m physdev --physdev-out tap10 --physdev-is-bridged -j tap10-IN
-A vmbr1-IN -m physdev --physdev-out tap9 --physdev-is-bridged -j tap9-IN
-A vmbr1-IN -m physdev --physdev-out tap8 --physdev-is-bridged -j tap8-IN
-A vmbr1-IN -m physdev --physdev-out tap7 --physdev-is-bridged -j tap7-IN
-A vmbr1-IN -m physdev --physdev-out tap6 --physdev-is-bridged -j tap6-IN
-A vmbr1-IN -m physdev --physdev-out tap5 --physdev-is-bridged -j tap5-IN
-A vmbr1-IN -m physdev --physdev-out tap4 --physdev-is-bridged -j tap4-IN
-A vmbr1-IN -m physdev --physdev-out tap3 --physdev-is-bridged -j tap3-IN
-A vmbr1-IN -m physdev --physdev-out tap2 --physdev-is-bridged -j tap2-IN
-A vmbr1-IN -m physdev --physdev-out tap1 --physdev-is-bridged -j tap1-IN
-A vmbr1-IN -m physdev --physdev-out tap0 --physdev-is-bridged -j tap0-IN
-A vmbr1-IN -m physdev --physdev-out tap110i0 --physdev-is-bridged -j tap110i0-IN
-A vmbr1-IN -m physdev --physdev-out tap123i0 --physdev-is-bridged -j tap123i0-IN
-A vmbr1-IN -m comment --comment "PVESIG:pJFhZIYDpYnRQdzQNmFwb4ovZsY"
-A vmbr1-IPS -m physdev --physdev-out tap123i0 --physdev-is-bridged -j NFQUEUE --queue-num 0 --queue-bypass
-A vmbr1-IPS -m comment --comment "PVESIG:tZNdCGPCuj+IrhwxjPaK1a5vLpY"
-A vmbr1-OUT -m physdev --physdev-in tap99 -j tap99-OUT
-A vmbr1-OUT -m physdev --physdev-in tap98 -j tap98-OUT
-A vmbr1-OUT -m physdev --physdev-in tap97 -j tap97-OUT
-A vmbr1-OUT -m physdev --physdev-in tap96 -j tap96-OUT
-A vmbr1-OUT -m physdev --physdev-in tap95 -j tap95-OUT
-A vmbr1-OUT -m physdev --physdev-in tap94 -j tap94-OUT
-A vmbr1-OUT -m physdev --physdev-in tap93 -j tap93-OUT
-A vmbr1-OUT -m physdev --physdev-in tap92 -j tap92-OUT
-A vmbr1-OUT -m physdev --physdev-in tap91 -j tap91-OUT
-A vmbr1-OUT -m physdev --physdev-in tap90 -j tap90-OUT
-A vmbr1-OUT -m physdev --physdev-in tap89 -j tap89-OUT
-A vmbr1-OUT -m physdev --physdev-in tap88 -j tap88-OUT
-A vmbr1-OUT -m physdev --physdev-in tap87 -j tap87-OUT
-A vmbr1-OUT -m physdev --physdev-in tap86 -j tap86-OUT
-A vmbr1-OUT -m physdev --physdev-in tap85 -j tap85-OUT
-A vmbr1-OUT -m physdev --physdev-in tap84 -j tap84-OUT
-A vmbr1-OUT -m physdev --physdev-in tap83 -j tap83-OUT
-A vmbr1-OUT -m physdev --physdev-in tap82 -j tap82-OUT
-A vmbr1-OUT -m physdev --physdev-in tap81 -j tap81-OUT
-A vmbr1-OUT -m physdev --physdev-in tap80 -j tap80-OUT
-A vmbr1-OUT -m physdev --physdev-in tap79 -j tap79-OUT
-A vmbr1-OUT -m physdev --physdev-in tap78 -j tap78-OUT
-A vmbr1-OUT -m physdev --physdev-in tap77 -j tap77-OUT
-A vmbr1-OUT -m physdev --physdev-in tap76 -j tap76-OUT
-A vmbr1-OUT -m physdev --physdev-in tap75 -j tap75-OUT
-A vmbr1-OUT -m physdev --physdev-in tap74 -j tap74-OUT
-A vmbr1-OUT -m physdev --physdev-in tap73 -j tap73-OUT
-A vmbr1-OUT -m physdev --physdev-in tap72 -j tap72-OUT
-A vmbr1-OUT -m physdev --physdev-in tap71 -j tap71-OUT
-A vmbr1-OUT -m physdev --physdev-in tap70 -j tap70-OUT
-A vmbr1-OUT -m physdev --physdev-in tap69 -j tap69-OUT
-A vmbr1-OUT -m physdev --physdev-in tap68 -j tap68-OUT
-A vmbr1-OUT -m physdev --physdev-in tap67 -j tap67-OUT
-A vmbr1-OUT -m physdev --physdev-in tap66 -j tap66-OUT
-A vmbr1-OUT -m physdev --physdev-in tap65 -j tap65-OUT
-A vmbr1-OUT -m physdev --physdev-in tap64 -j tap64-OUT
-A vmbr1-OUT -m physdev --physdev-in tap63 -j tap63-OUT
-A vmbr1-OUT -m physdev --physdev-in tap62 -j tap62-OUT
-A vmbr1-OUT -m physdev --physdev-in tap61 -j tap61-OUT
-A vmbr1-OUT -m physdev --physdev-in tap60 -j tap60-OUT
-A vmbr1-OUT -m physdev --physdev-in tap59 -j tap59-OUT
-A vmbr1-OUT -m physdev --physdev-in tap58 -j tap58-OUT
-A vmbr1-OUT -m physdev --physdev-in tap57 -j tap57-OUT
-A vmbr1-OUT -m physdev --physdev-in tap56 -j tap56-OUT
-A vmbr1-OUT -m physdev --physdev-in tap55 -j tap55-OUT
-A vmbr1-OUT -m physdev --physdev-in tap54 -j tap54-OUT
-A vmbr1-OUT -m physdev --physdev-in tap53 -j tap53-OUT
-A vmbr1-OUT -m physdev --physdev-in tap52 -j tap52-OUT
-A vmbr1-OUT -m physdev --physdev-in tap51 -j tap51-OUT
-A vmbr1-OUT -m physdev --physdev-in tap50 -j tap50-OUT
-A vmbr1-OUT -m physdev --physdev-in tap49 -j tap49-OUT
-A vmbr1-OUT -m physdev --physdev-in tap48 -j tap48-OUT
-A vmbr1-OUT -m physdev --physdev-in tap47 -j tap47-OUT
-A vmbr1-OUT -m physdev --physdev-in tap46 -j tap46-OUT
-A vmbr1-OUT -m physdev --physdev-in tap45 -j tap45-OUT
-A vmbr1-OUT -m physdev --physdev-in tap44 -j tap44-OUT
-A vmbr1-OUT -m physdev --physdev-in tap43 -j tap43-OUT
-A vmbr1-OUT -m physdev --physdev-in tap42 -j tap42-OUT
-A vmbr1-OUT -m physdev --physdev-in tap41 -j tap41-OUT
-A vmbr1-OUT -m physdev --physdev-in tap40 -j tap40-OUT
-A vmbr1-OUT -m physdev --physdev-in tap39 -j tap39-OUT
-A vmbr1-OUT -m physdev --physdev-in tap38 -j tap38-OUT
-A vmbr1-OUT -m physdev --physdev-in tap37 -j tap37-OUT
-A vmbr1-OUT -m physdev --physdev-in tap36 -j tap36-OUT
-A vmbr1-OUT -m physdev --physdev-in tap35 -j tap35-OUT
-A vmbr1-OUT -m physdev --physdev-in tap34 -j tap34-OUT
-A vmbr1-OUT -m physdev --physdev-in tap33 -j tap33-OUT
-A vmbr1-OUT -m physdev --physdev-in tap32 -j tap32-OUT
-A vmbr1-OUT -m physdev --physdev-in tap31 -j tap31-OUT
-A vmbr1-OUT -m physdev --physdev-in tap30 -j tap30-OUT
-A vmbr1-OUT -m physdev --physdev-in tap29 -j tap29-OUT
-A vmbr1-OUT -m physdev --physdev-in tap28 -j tap28-OUT
-A vmbr1-OUT -m physdev --physdev-in tap27 -j tap27-OUT
-A vmbr1-OUT -m physdev --physdev-in tap26 -j tap26-OUT
-A vmbr1-OUT -m physdev --physdev-in tap25 -j tap25-OUT
-A vmbr1-OUT -m physdev --physdev-in tap24 -j tap24-OUT
-A vmbr1-OUT -m physdev --physdev-in tap23 -j tap23-OUT
-A vmbr1-OUT -m physdev --physdev-in tap22 -j tap22-OUT
-A vmbr1-OUT -m physdev --physdev-in tap21 -j tap21-OUT
-A vmbr1-OUT -m physdev --physdev-in tap20 -j tap20-OUT
-A vmbr1-OUT -m physdev --physdev-in tap19 -j tap19-OUT
-A vmbr1-OUT -m physdev --physdev-in tap18 -j tap18-OUT
-A vmbr1-OUT -m physdev --physdev-in tap17 -j tap17-OUT
-A vmbr1-OUT -m physdev --physdev-in tap16 -j tap16-OUT
-A vmbr1-OUT -m physdev --physdev-in tap15 -j tap15-OUT
-A vmbr1-OUT -m physdev --physdev-in tap14 -j tap14-OUT
-A vmbr1-OUT -m physdev --physdev-in tap13 -j tap13-OUT
-A vmbr1-OUT -m physdev --physdev-in tap12 -j tap12-OUT
-A vmbr1-OUT -m physdev --physdev-in tap11 -j tap11-OUT
-A vmbr1-OUT -m physdev --physdev-in tap10 -j tap10-OUT
-A vmbr1-OUT -m physdev --physdev-in tap9 -j tap9-OUT
-A vmbr1-OUT -m physdev --physdev-in tap8 -j tap8-OUT
-A vmbr1-OUT -m physdev --physdev-in tap7 -j tap7-OUT
-A vmbr1-OUT -m physdev --physdev-in tap6 -j tap6-OUT
-A vmbr1-OUT -m physdev --physdev-in tap5 -j tap5-OUT
-A vmbr1-OUT -m physdev --physdev-in tap4 -j tap4-OUT
-A vmbr1-OUT -m physdev --physdev-in tap3 -j tap3-OUT
-A vmbr1-OUT -m physdev --physdev-in tap2 -j tap2-OUT
-A vmbr1-OUT -m physdev --physdev-in tap1 -j tap1-OUT
-A vmbr1-OUT -m physdev --physdev-in tap0 -j tap0-OUT
-A vmbr1-OUT -m physdev --physdev-in tap110i0 -j tap110i0-OUT
-A vmbr1-OUT -m physdev --physdev-in tap123i0 -j tap123i0-OUT
-A vmbr1-OUT -m comment --comment "PVESIG:NUbmEvobWWY2FG3WIDlqPMw+WWg"
-N tap110i0-IN
-N tap110i0-OUT
-A tap110i0-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A tap110i0-IN -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A tap110i0-IN -p tcp -j PVEFW-tcpflags
-A tap110i0-IN -m conntrack --ctstate INVALID -j DROP
-A tap110i0-IN -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A tap110i0-IN -j PVEFW-Drop
-A tap110i0-IN -j NFLOG --nflog-prefix  ":110:6:tap110i0-IN: policy DROP: "
-A tap110i0-IN -j DROP
-A tap110i0-IN -m comment --comment "PVESIG:JZF+d2tA8kTDJzVnH9c2+v3a18o"
-A tap110i0-OUT -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A tap110i0-OUT -p udp -m udp --sport 68 --dport 67 -g PVEFW-SET-ACCEPT-MARK
-A tap110i0-OUT -p tcp -j PVEFW-tcpflags
-A tap110i0-OUT -m conntrack --ctstate INVALID -j DROP
-A tap110i0-OUT -m conntrack --ctstate RELATED,ESTABLISHED -g PVEFW-SET-ACCEPT-MARK
-A tap110i0-OUT -j MARK --set-xmark 0x0/0xffffffff
-A tap110i0-OUT -p tcp -m tcp --dport 5001 -g PVEFW-SET-ACCEPT-MARK
-A tap110i0-OUT -j PVEFW-Drop
-A tap110i0-OUT -j NFLOG --nflog-prefix  ":110:6:tap110i0-OUT: policy DROP: "
-A tap110i0-OUT -j DROP
-A tap110i0-OUT -m comment --comment "PVESIG:ItuHycGJvhs7KqRI9ZNOYRXMshE"
-N tap123i0-IN
-N tap123i0-OUT
-A tap123i0-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A tap123i0-IN -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A tap123i0-IN -p tcp -j PVEFW-tcpflags
-A tap123i0-IN -m conntrack --ctstate INVALID -j DROP
-A tap123i0-IN -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A tap123i0-IN -p tcp -m tcp --dport 5001 -j ACCEPT
-A tap123i0-IN -j PVEFW-Drop
-A tap123i0-IN -j NFLOG --nflog-prefix  ":123:6:tap123i0-IN: policy DROP: "
-A tap123i0-IN -j DROP
-A tap123i0-IN -m comment --comment "PVESIG:J0ZQDWY79tE8N5pUOfQD9MkMW/g"
-A tap123i0-OUT -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A tap123i0-OUT -p udp -m udp --sport 68 --dport 67 -g PVEFW-SET-ACCEPT-MARK
-A tap123i0-OUT -p tcp -j PVEFW-tcpflags
-A tap123i0-OUT -m conntrack --ctstate INVALID -j DROP
-A tap123i0-OUT -m conntrack --ctstate RELATED,ESTABLISHED -g PVEFW-SET-ACCEPT-MARK
-A tap123i0-OUT -j MARK --set-xmark 0x0/0xffffffff
-A tap123i0-OUT -p tcp -m tcp --dport 5001 -g PVEFW-SET-ACCEPT-MARK
-A tap123i0-OUT -j PVEFW-Drop
-A tap123i0-OUT -j NFLOG --nflog-prefix  ":123:6:tap123i0-OUT: policy DROP: "
-A tap123i0-OUT -j DROP
-A tap123i0-OUT -m comment --comment "PVESIG:9+NiJESC3XWTiHNL1jGSgdWrfTM"

RESULTS
-------
firewall disabled: 
------------------
bandwidth : 3,8 Gbits/s . host CPU satured (vhost-net and kvm process)

firewall enabled:
-----------------

(-m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT in tap-in chain):

3,4gbit/s

so around 10% loss 

-A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT 

3,8gbit/s

So, It's really helping to do as soon as possible the ACCEPT for ESTABLISHED connections.
(Of course, my example is a little crazy, with 100taps on same vmbr1)