[pbs-devel] [PATCH proxmox-backup 6/6] verify: distinguish s3 object fetching and chunk loading error

[Fabian Grünbichler]

On October 16, 2025 3:18 pm, Christian Ebner wrote:
> Errors while loading chunks from the object store might be cause by
> transient issues, and must therefore handled so they do not
> incorrectly mark chunks as corrupt.
> 
> On creating the chunk from the response data, which includes the
> chunk header and validity checks, errors must however lead to the
> chunk being flagged as bad. Adapt the code so these errors are
> correctly distinguished.
> 
> Signed-off-by: Christian Ebner <c.ebner at proxmox.com>
> ---

this patch is also independent from the rest and fixes a known bug,
right? if so, please order it up front and/or say so, then it can be
applied independently from the bigger refactoring..

>  src/backup/verify.rs | 31 ++++++++++++++++++-------------
>  1 file changed, 18 insertions(+), 13 deletions(-)
> 
> diff --git a/src/backup/verify.rs b/src/backup/verify.rs
> index b1066f6f5..4cfd81350 100644
> --- a/src/backup/verify.rs
> +++ b/src/backup/verify.rs
> @@ -225,19 +225,24 @@ impl VerifyWorker {
>                  let object_key = pbs_datastore::s3::object_key_from_digest(&info.digest)?;
>                  match proxmox_async::runtime::block_on(s3_client.get_object(object_key)) {
>                      Ok(Some(response)) => {
> -                        let chunk_result = proxmox_lang::try_block!({
> -                            let bytes =
> -                                proxmox_async::runtime::block_on(response.content.collect())?
> -                                    .to_bytes();
> -                            DataBlob::from_raw(bytes.to_vec())
> -                        });
> -
> -                        match chunk_result {
> -                            Ok(chunk) => {
> -                                let size = info.size();
> -                                *read_bytes += chunk.raw_size();
> -                                decoder_pool.send((chunk, info.digest, size))?;
> -                                *decoded_bytes += size;
> +                        match proxmox_async::runtime::block_on(response.content.collect()) {
> +                            Ok(raw_chunk) => {
> +                                match DataBlob::from_raw(raw_chunk.to_bytes().to_vec()) {
> +                                    Ok(chunk) => {
> +                                        let size = info.size();
> +                                        *read_bytes += chunk.raw_size();
> +                                        decoder_pool.send((chunk, info.digest, size))?;
> +                                        *decoded_bytes += size;
> +                                    }
> +                                    Err(err) => self.add_corrupt_chunk(
> +                                        info.digest,
> +                                        errors,
> +                                        &format!(
> +                                            "can't verify chunk with digest {} - {err}",
> +                                            hex::encode(info.digest)
> +                                        ),
> +                                    ),
> +                                }
>                              }
>                              Err(err) => {
>                                  errors.fetch_add(1, Ordering::SeqCst);
> -- 
> 2.47.3
> 
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel at lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> 
> 
>