[pve-devel] [PATCH http-server] fix #6230: increase allowed post size

[Dominik Csapak]

In some situations, e.g. having a large resource mapping, the UI can
generate a request that is bigger than the current limit of 64KiB.

Our files in pmxcfs can grow up to 1 MiB, so theoretically, a single
mapping can grow to that size. In practice, a single entry will have
much less. In #6230, a user has a mapping with about ~130KiB.

Increase the limit to 512KiB so we have a bit of headroom left.

We have to also increase the 'rbuf_max' size here, otherwise the request
will fail (since the buffer is too small for the request). Since the
post limit and the rbuf_max are tightly coupled, let it reflect that in
the code. To do that sum the post size + max header size there.

A short benchmark shows that it only slightly impacts performance for
the same amount of data (but that could be runtime variance too):

I used a 4 node virtualized cluster, benchmarked with oha[0] with these
options:

oha --insecure -H $COOKIE -H $CSRFTOKEN -D bodyfile -m "PUT" -T
"application/x-www-form-urlencoded" -n 3000 -c 50 --disable-keepalive
--latency-correction https://<IP>:8006/api2/json/cluster/mapping/pci/test

So 3000 requests with 50 parallel. I also restarted pveproxy and daemon
in between runs, and took the rss values around the 50% runtime of the
benchmark.

                    average time  requests/s  pvedaemon rss   pveproxy rss
old with 60k body   3.0067s       16.3487     140M-155M       141M-170M
new with 60k body   3.0865s       15.7623     140M-155M       141M-171M
new with 180k body  8.3834s       5.8934      140M-158M       141M-181M

Using a bigger body size had a large impact on the time, but that's IMHO
expected. Also, RSS is not that much impacted, only when using many
requests with larger request size, but this should also be expected.

0: https://github.com/hatoo/oha

Signed-off-by: Dominik Csapak <d.csapak at proxmox.com>
---
changes from rfc:
* s/buffer/headroom
* added benchmark data to the commit message
* corrected rbuf_max calculation

 src/PVE/APIServer/AnyEvent.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/PVE/APIServer/AnyEvent.pm b/src/PVE/APIServer/AnyEvent.pm
index 8a52836..7499474 100644
--- a/src/PVE/APIServer/AnyEvent.pm
+++ b/src/PVE/APIServer/AnyEvent.pm
@@ -52,7 +52,7 @@ use PVE::APIServer::Utils;
 
 my $limit_max_headers = 64;
 my $limit_max_header_size = 8*1024;
-my $limit_max_post = 64*1024;
+my $limit_max_post = 512*1024;
 
 my $known_methods = {
     GET => 1,
@@ -1891,7 +1891,7 @@ sub accept_connections {
 	    $self->{conn_count}++;
 	    $reqstate->{hdl} = AnyEvent::Handle->new(
 		fh => $clientfh,
-		rbuf_max => 64*1024,
+		rbuf_max => $limit_max_post + $limit_max_header_size,
 		timeout => $self->{timeout},
 		linger => 0, # avoid problems with ssh - really needed ?
 		on_eof   => sub {
-- 
2.39.5