[pve-devel] [RFC many 0/3] combine and simplify RRD handling

[Aaron Lauterer]

On  2025-09-05  10:12, Thomas Lamprecht wrote:
> Am 05.09.25 um 10:03 schrieb Aaron Lauterer:
>>
>>
>> On  2025-09-04  20:20, Thomas Lamprecht wrote:
>>> Am 04.09.25 um 16:09 schrieb Aaron Lauterer:
>>>> this series does two things:
>>>> * switch the RRD API backend to use the old RRD files is available
>>>> * always use RRD files with the new 9.0 schema
>>>>
>>>> The motivation for the combination of old and new RRD files is that during the
>>>> migration of old RRD files to the new ones, spikes get flattened. By combining
>>>> the old and new RRD files, we can keep the old coarse data with all it spikes
>>>> and only show the new much mode finer stepped data where available.
>>>>
>>>> This will also enable us to just write any new data into an RRD file that is
>>>> created with the new 9.0 schema. As a result, no migration step is needed
>>>> anymore and we can simplify the logic around the whole "does the only the old
>>>> file exist or is the new one also present" alot in quite a few places.
>>>>
>>>> Some examples which we can simplify or revert:
>>>> pve-manager: API2Tools::get_rrd_key can most likely be dropped
>>>> qemu-server, pve-storage, pve-container,…: rrd API endpoints which check which files exist
>>>> drop the whole migration step on upgrade
>>>>
>>>> other TODOs:
>>>>
>>>> RRD::create_rrd_graph -> see if we can combine both files into one graph. But we
>>>> are relying on RRD itself for this. So that might be interesting. On the other
>>>> hand, do we want to keep that API endpoint around? People are using it though.
>>>> So we need to weigh that decision accoringly.
>>>
>>> Keeping it around for the time being is definitively required now that
>>> PVE 9.0 is out, we might sunset it for a future major release.
>>>
>>> Another option might be to generate both images and merge them manually,
>>> i.e. literally make the right part the old and the left part the new
>>> image with some imagemagick command. Or does rrd indeed supports merging
>>> two files together?
>>
>> One thing that should be possible is to give it two datasets from different files. That would be a middle ground as in, all data is there, but it would show up as separata data lines.
>>
>> I am looking into it to see if we could get it nicer. Manually stitching images together will most likely not work because I don't expected the scales to match up perfectly.
> 
> True w.r.t. not matching scales. But as one can override the line colors we could probably just do that and use the same ones for both data series?

I did some prototyping and we can get both lines looking the same in the 
final graph directly from rrdtool graph.

I am now preparing a larger patch series, that also will include 
simplified/reverted changes now that we can count on the new 
pve-TYPE-9.0 rrd file to be present.