[pdm-devel] [PATCH proxmox-datacenter-manager v6 00/23] metric collection improvements (concurrency, API, CLI)

[Lukas Wagner]

On Fri Aug 22, 2025 at 1:51 PM CEST, Dominik Csapak wrote:
> aside from the issue with the last 10 minute gap of data in the hourly
> view, code LGTM and tested fine, did not notice anything else off
>
> Some patches (especially the last 3) could possibly be squashed in to
> earlier patches (no biggie though)

Will do, if I can do it without large conflicts while rebasing.

>
> Since this is a rather large series, I'd like to see a reviewed-by from
> someone else before applying, since the reviewed-by from maximiliano
> came in at v2 i think, and there were some noticeable changes since
> then.
>
> With that said, consider this round
>
> Reviewed-by: Dominik Csapak <d.csapak at proxmox.com>
> Tested-by: Dominik Csapak <d.csapak at proxmox.com>


[copied from your other reponse]:
On Fri Aug 22, 2025 at 2:49 PM CEST, Dominik Csapak wrote:
> ah one thing i forgot to mention:
>
> i'd probably favor doing it the way you described here:
>
>
>  > - Should `GET /remotes/<remote>/metric-collection-rrddata` be
>  >   just `rrddata`?
>  >   not sure if we are going to add any other PDM-native per-remote
>  >   metrics and whether we want to return that from the same API call
>  >   as this...
>
> i think it's fine to mix it here

Alright, I'll rename it to just `rrddata` then.

>
> also the  /metric-collection/rrddata could be merged
> with a (not yet implemented) rrddata endpoint in the /nodes/localhost path

Will do!

>
> i guess we want to expose some pdm host rrddata sooner or later anyway
> there we can include this too?

[copied from your other response]:
On Fri Aug 22, 2025 at 1:27 PM CEST, Dominik Csapak wrote:
>> We also could trigger an out-of-schedule metric collection for a remote
>> when the RRD graph calls the rrddata endpoint (the functions for
>> triggering the collection of a single remote are already there, albeit
>> non-blocking, so this would need some changes). Fetching the missing
>> data for a single remote should be fast enough any way. The rrddata
>> endpoint could have some timeout for waiting for the results of
>> collecting that single remote; if that one is exceeded we don't wait
>> until the collection results are done but simply return the existing
>> data.
>> 
>
> The problem is actually just the 'hourly' graphs, since there the 
> interval is so small one noticed the 10 minutes quite often.
>
> All others (daily,monthly, etc.) don't exhibit the same problem
>
> so i'd say if we can fetch missing data on-the-fly in the api call
> for hourly calls, it would be good enough

Alright, will try to implement something akin to what I've described for
v7.