[pbs-devel] [PATCH proxmox{, -backup} 00/10] s3: implement list buckets and use bucket selector for datastore creation

[Christian Ebner]

On 7/30/25 3:46 PM, Lukas Wagner wrote:
> On Mon Jul 28, 2025 at 8:46 AM CEST, Christian Ebner wrote:
>> This patches aim to increase usability for the user by allowing to list
>> accessible buckets during datastore creation.
>>
>> For this, the proxmox s3 client is extended by the list buckets method, fetching
>> accessible buckets from the s3 api. This requires the client instantiation to be
>> possible without the bucket name, making it therefore optional. Further, the
>> regex for the config endpoint url must be restricted to have the bucket name
>> template pattern always at the start of the template. With these changes, it is
>> then possible to fetch the buckets given the required permissions to do so.
>>
>> On the proxmox backup server side, the api endpoints to fetch the bucket list
>> are implemented as well as the corresponding cli command. Finally, the bucket
>> name field in the datastore edit window is replaced by an s3 bucket selector,
>> which loads the accessible bucket names via the api.
>>
> 
> Already reported to Chris off-list, but there seems to be something
> wrong in the response parser when multiple buckets are returned:
> 
> $ proxmox-backup-manager s3 endpoint list-buckets minio
> Error: failed to list buckets
> 
> Caused by:
>      0: failed to parse response body
>      1: custom: duplicate field `Bucket`
> 
> If I remove all but one buckets from the MinIO server, it works.

Thanks for catching this!

Will be fixed in a version 2 of the patches. I will also work on some 
regression tests for the whole of the s3 client response parsing. This I 
will however send as followup patches.