[pbs-devel] partially applied: [PATCH proxmox-backup v3 00/23] fix chunk upload/insert, rename corrupt chunks and GC race conditions for s3 backend

[Fabian Grünbichler]

On Wed, 05 Nov 2025 13:22:10 +0100, Christian Ebner wrote:
> These patches fix possible race conditions on datastores with s3 backend for
> chunk insert, renaming of corrupt chunks during verification and cleanup during
> garbage collection. Further, the patches assure consistency between the chunk
> marker file of the local datastore cache, the s3 object store and the in-memory
> LRU cache during state changes occurring by one of the above mentioned operations.
> 
> Consistency is achieved by using a per-chunk file locking mechanism. File locks
> are stored on the predefined location for datastore file locks, using the same
> `.chunks/prefix/digest` folder layout for consistency and to keep readdir and
> other fs operations performant.
> 
> [...]

Applied the precursor patches, except for #7

[01/23] sync: pull: instantiate backend only once per sync job
        commit: c03e428d5a4ef138df9e46d6bbc427d52d2456de
[02/23] api/datastore: move group notes setting to the datastore
        commit: 09e740001ff02dfb10d0e8c43cc3430eac25ed09
[03/23] api/datastore: move snapshot deletion into dedicated datastore helper
        commit: 800e960c59af828bf4091399bbb904197a3c8975
[04/23] api/datastore: move backup log upload by implementing datastore helper
        commit: 76b0787cd6f8cacc3726bac6984add75601d1307
[05/23] api: backup: use datastore add_blob helper for backup session
        commit: f00261fec3be3e13c0c59a5858c4d4140df1fd1a
[06/23] api/datastore: add dedicated datastore helper to set snapshot notes
        commit: 5c053536fbf5c212239736a43814993e37b89a2e
[08/23] datastore: refactor chunk insert based on backend
        commit: 72dacfcf347fa794ea80eb0c3aa4741dd2d7a1b5
[09/23] verify: rename corrupted to corrupt in log output and function names
        commit: a77332b95f69ef54f940748a26229be14f7d1717
[10/23] verify/datastore: make rename corrupt chunk a datastore helper method
        commit: 36e0bb6111cea60a3e0262e29e7112c29f02e278
[11/23] datastore: refactor rename_corrupt_chunk error handling
        commit: 636ec15ab56703a40e561d48de74745a34fc5bff

and added a small follow-up for patch #1.

Best regards,
-- 
Fabian Grünbichler <f.gruenbichler at proxmox.com>