[pve-devel] S3 Support

[Fabian Grünbichler]

> Tiziano Bacocco via pve-devel <pve-devel at lists.proxmox.com> hat am 16.07.2024 00:20 CEST geschrieben:
> Hi, to everyone, i'm interested in native S3 support on proxmox, is anyone
> working on it ( to avoid double work )?

None of us is, but there is an external developer that is trying to implement something like this. The whole topic might be a better fit for pbs-devel though ;)

> Second question, my idea is to reuse existing pbs code but upload chunks
> and index to s3, sounds good? The thing puzzling me is that a lot of code
> will end up duplicated , when actually only the part uploading to S3
> instead of pbs would be different, chunking, fleeching, restoring, backup,
> qemu interface would be almost identical

https://bugzilla.proxmox.com/show_bug.cgi?id=2943

backing up directly to S3 will be hard for a multitude of reasons:
- you don't want to write individual chunks as objects (too expensive)
- you don't want to slow down the guest because of the speed of the object storage
- ..

instead, a similar approach to the tape backup feature would be needed:
- define retention and bucket allocation policies
- combine metadata (snapshots/..) and chunks of a local datastore in a meaningful way into buckets
- have some sort of index/catalog that tells you which buckets you need to restore which snapshots into a datastore
- ..

hope this makes sense!