[pve-devel] [PATCH manager] api2 tools: rrd data: prefer new-format keys over old-format keys

[Friedrich Weber]

Ping, as Hannes Dürr also ran into this issue just now.

On 21/10/2025 17:50, Friedrich Weber wrote:
> This fixes an issue where on a fresh Proxmox VE 9 cluster, the status
> of the first VM/container would not be updated in the resource tree
> for the first 5 minutes after creation. For example, starting the
> newly created guest after the first pvestatd status update would not
> update the state to "running" for 5 minutes.
> 
> The reason is that pvestatd, when broadcasting RRD data, checks for
> existence of /var/lib/rrdcached/db/pve-vm-9.0/ to decide whether to
> send RRD data with new-format (pve-vm-9.0/) or old-format (pve2.3-vm/)
> keys. The directory does not exist on nodes of a fresh cluster without
> guests, so pvestatd falls back to sending an old-format key. pmxcfs,
> upon receiving the key, creates the directory, so on the next pvestatd
> status update, pvestatd will send out new-format keys. But until the
> entry with the old-format key expires (after 5 minutes), both the
> new-format and the old-format key exist in the RRD data. Currently, if
> both keys are present, get_rrd_key will return the old-format key,
> meaning that any status updates with the new-format key are not
> registered. In order to fix this, prefer the new-format key if it is
> present.
> 
> Signed-off-by: Friedrich Weber <f.weber at proxmox.com>
> ---
> 
> Notes:
>     Encountered during a training, where one is more likely to closely
>     watch the state of the first created guest.
>     
>     I'm unsure if this is a good fix.
>     
>     An alternative fix for this particular case would be to instead have
>     pvestatd always send out new-style keys on a fresh cluster.
>     
>     However, another situation where there could be both old-format and
>     new-format keys is in a mixed 8+9 cluster (e.g. during upgrade).
>     
>     One issue is that get_rrd_key on 8 would still prefer old-style keys
>     (unless we backport this patch to 8), so behavior on 8 and 9 would be
>     different.
>     
>     Another issue is that having both an old-format and new-format key
>     could happen after migrating a VM from a PVE 8 node (sending
>     old-format keys) to a PVE 9 node (sending new-format keys). In this
>     direction, I guess we also want to prefer new-format keys over the
>     old-format keys. But when migrating a VM from a PVE 9 (sending
>     new-format keys) to a PVE 8 node (sending old-format keys) -- which is
>     admittedly not supported -- it would make more sense to prefer
>     old-format keys.
>     
>     Perhaps a better solution is, if both old-format and new-format keys
>     are present, to prefer the key whose entry has the more recent
>     timestamp (ctime)?

Honestly, I'm not sure if this extra complexity just to make it work in
an unsupported usecase (PVE9->8 migration) is worth it. Maybe this patch
(always prefer new format over old format) is OK after all (and could be
backported to PVE 8).