[pbs-devel] [PATCH proxmox/proxmox-backup] Rate Limiter Implementation
Dietmar Maurer
dietmar at proxmox.com
Tue Nov 9 07:52:37 CET 2021
This implements a simple Token Bucket based rate limiter. That filter
can be used to:
- limit http client connection
- limit speed at server side (proxmox-backup-proxy)
There is also a new configuration file to configure the server side
rate limits: /etc/proxmox-backup/traffic-control.cfg
The server side filter dynamically updates when the user changes
configuration (even for existing connections).
Dietmar Maurer (proxmox/7):
Implement a rate limiting stream (AsyncRead, AsyncWrite)
RateLimitedStream: implement poll_write_vectored
HttpsConnector: use RateLimitedStream
RateLimitedStream: allow periodic limiter updates
RateLimiter: avoid panic in time computations
RateLimitedStream: implement peer_addr
RateLimiter: add update_rate method
proxmox-http/src/client/connector.rs | 51 +++-
proxmox-http/src/client/mod.rs | 6 +
.../src/client/rate_limited_stream.rs | 233 ++++++++++++++++++
proxmox-http/src/client/rate_limiter.rs | 84 +++++++
4 files changed, 366 insertions(+), 8 deletions(-)
create mode 100644 proxmox-http/src/client/rate_limited_stream.rs
create mode 100644 proxmox-http/src/client/rate_limiter.rs
Dietmar Maurer (proxmox-backup/9):
pbs-client: add option to use the new RateLimiter
proxmox-backup-client: add rate/burst parameter to backup CLI
implement Servive for RateLimitedStream
New DailyDuration type with nom parser
DailyDuration: implement time_match()
Add traffic control configuration config with API
traffic_control: use Memcom to track. config versions
implement a traffic control cache for fast rate control limiter
lockups
proxmox-backup-proxy: implement traffic control
Cargo.toml | 1 +
pbs-api-types/src/lib.rs | 7 +
pbs-api-types/src/traffic_control.rs | 81 +++++
pbs-client/src/http_client.rs | 24 +-
pbs-client/src/tools/mod.rs | 23 +-
pbs-config/src/lib.rs | 3 +-
pbs-config/src/memcom.rs | 14 +
pbs-config/src/traffic_control.rs | 98 ++++++
proxmox-backup-client/src/main.rs | 19 +-
proxmox-rest-server/Cargo.toml | 1 +
proxmox-rest-server/src/rest.rs | 28 ++
proxmox-systemd/src/daily_duration.rs | 152 ++++++++++
proxmox-systemd/src/lib.rs | 1 +
proxmox-systemd/src/parse_time.rs | 56 ++++
proxmox-systemd/src/time.rs | 2 +-
src/api2/config/mod.rs | 2 +
src/api2/config/traffic_control.rs | 283 ++++++++++++++++++
src/bin/proxmox-backup-manager.rs | 1 +
src/bin/proxmox-backup-proxy.rs | 25 +-
src/bin/proxmox_backup_manager/mod.rs | 2 +
.../proxmox_backup_manager/traffic_control.rs | 105 +++++++
src/cached_traffic_control.rs | 240 +++++++++++++++
src/lib.rs | 3 +
23 files changed, 1161 insertions(+), 10 deletions(-)
create mode 100644 pbs-api-types/src/traffic_control.rs
create mode 100644 pbs-config/src/traffic_control.rs
create mode 100644 proxmox-systemd/src/daily_duration.rs
create mode 100644 src/api2/config/traffic_control.rs
create mode 100644 src/bin/proxmox_backup_manager/traffic_control.rs
create mode 100644 src/cached_traffic_control.rs
--
2.30.2
More information about the pbs-devel
mailing list