[pdm-devel] [PATCH proxmox-datacenter-manager v6 00/23] metric collection improvements (concurrency, API, CLI)

[Thomas Lamprecht]

On 21/08/2025 14:54, Dominik Csapak wrote:
> just a first high level question while i'm testing/reviewing this:
> 
> when using this patch, it seems the collection interval is much reduced?
> 
> e.g. in the gui I'm missing up to the last 10 minutes now?
> (at 14:49 the last point i have for the rrd is from 14:40)
> 
> is this by design? i get that we don't want to pull too often,
> but showing up to 10minutes out of date graphs is also not practical?

Yes this was by design, or well indeed mine and also Dietmars request.
But surely something that might warrant a central knob with a per
remote override in the future (or smarter heuristics depending on
remote node count and latency, i.e. how long the last few metric
queries took on average).

But the idea was also to fetch missing data on demand if required, e.g.
when a user visits a specific resource with a time frame length where
this is noticeable, like you mentioned that it mostly sticks out for
the last-hour time frame.

> my naive solution would be to proxy the rrd requests to the
> pve nodes directly, but then why would we need the metric
> collection in the first place?

Those two are a bit orthogonal to each other, we want RRD collection
to not depend on every remote having to be online all the time (to see
when it went offline, to avoid high latency to add many delays, ...), and
being able to do quick metrics and stats computations on all resources
(which will never be 100% real time anyway). The other is seeing most
recent data, e.g. when investigating why something is slow or what not.