[pbs-devel] [PATCH proxmox-backup 00/15] RRD database improvements

[Dietmar Maurer]

- use a journal. This way we can reduce the overall number of bytes
  written by increasing the flush/commit interval (30 minutes).

- the new CBOR base format is more flexible, and we store much more
  data points now.

We previously wrote about 7MB/h. With the new format and journal, we now write about 3MB/h.

Dietmar Maurer (15):
  proxmox-rrd: use a journal to reduce amount of bytes written
  RRD_CACHE: use a OnceCell instead of lazy_static
  proxmox-backup-proxy: use tokio::task::spawn_blocking instead of
    block_in_place
  proxmox-rrd: implement new CBOR based format
  proxmox-rrd: remove dependency to proxmox-rrd-api-types
  proxmox-rrd: extract_data: include values from current slot
  remove proxmox-rrd-api-types crate,
    s/RRDTimeFrameResolution/RRDTimeFrame/
  proxmox-rrd: support CF::Last
  proxmox-rrd: split out load_rrd (cleanup)
  proxmox-rrd: add binary to create/manage rrd files
  proxmox-rrd: avoid % inside loop
  proxmox-rrd: new helper methods - slot() and slot_end_time()
  proxmox-rrd: protect against negative update time
  proxmox-rrd: rename last_counter to last_value
  proxmox-rrd: add more commands to the rrd cli tool

 Cargo.toml                       |   2 -
 Makefile                         |   1 -
 pbs-api-types/Cargo.toml         |   1 -
 pbs-api-types/src/lib.rs         |  30 +-
 proxmox-rrd-api-types/Cargo.toml |  11 -
 proxmox-rrd-api-types/src/lib.rs |  32 --
 proxmox-rrd/Cargo.toml           |   8 +-
 proxmox-rrd/src/bin/rrd.rs       | 412 +++++++++++++++++++++++
 proxmox-rrd/src/cache.rs         | 283 +++++++++++++---
 proxmox-rrd/src/lib.rs           |  17 +-
 proxmox-rrd/src/rrd.rs           | 539 ++++++++++++++++---------------
 proxmox-rrd/src/rrd_v1.rs        | 296 +++++++++++++++++
 src/api2/admin/datastore.rs      |   6 +-
 src/api2/node/rrd.rs             |  52 +--
 src/api2/status.rs               |  14 +-
 src/bin/proxmox-backup-api.rs    |   4 -
 src/bin/proxmox-backup-proxy.rs  | 166 +++++-----
 src/lib.rs                       |  86 +++--
 18 files changed, 1465 insertions(+), 495 deletions(-)
 delete mode 100644 proxmox-rrd-api-types/Cargo.toml
 delete mode 100644 proxmox-rrd-api-types/src/lib.rs
 create mode 100644 proxmox-rrd/src/bin/rrd.rs
 create mode 100644 proxmox-rrd/src/rrd_v1.rs

-- 
2.30.2