[pbs-devel] [RFC v2 proxmox/bookworm-stable proxmox-backup 00/42] S3 storage backend for datastores

[Lukas Wagner]

On Wed Jul 23, 2025 at 4:11 PM CEST, Thomas Lamprecht wrote:
> Am 06.06.25 um 13:12 schrieb Lukas Wagner:
>> Furthermore, I suggested that maybe the 'bucket' should be a property of the
>> datastore config, not of the s3 config. That way, the s3 config contains only the
>> connection info and credentials, which make it easy to use the same s3 config for
>> multiple datastores which use different buckets as as a backing storage.
>
> But that promotes re-using a central access secret/token multiple times
> vs. having a dedicated token per bucket. Especially as we use a top-level
> directory with the datastore name inside the bucket anyway, i.e., one can
> use the same bucket easily for multiple datastores already, I do not
> think this is great for UX in general besides maybe dev-focused testing.

Fair, that's a very valid point, I didn't really view it from that
angle.

FWIW, I think in v2 this series still created the .chunks 'directory'
at the top-level of the bucket; it did not use the datastore name as a
prefix, meaning there was a 1:1 mapping of datastore to bucket and
no way to use the same bucket for multiple datastores.

>
> As either I want to have different buckets, where one should always
> prefer using dedicated access control for each and now needs to add the
> same endpoint twice anyway, or one want's to re-use the same bucket
> anyway, which still works if its defined in the endpoint.
>
> Anyway, if we can sensibly query the list of buckets for the datastore
> create, some UX pain will be gone with the newer variant and it can
> probably stay, as redoing this now is rather a bit of churn and we are
> not locked in here. A trade-off for the future might be to allow
> configuring a default bucket in the endpoint, that is auto-filled when
> selecting the endpoint in the datastore create window.
>
> But mostly UX polishing and definitively not pressing for now.