[pbs-devel] [PATCH v3 proxmox-backup 45/58] client: pxar: add method for metadata comparison

[Fabian Grünbichler]

Quoting Christian Ebner (2024-03-28 13:36:54)
> Adds a method to compare the metadata of the current file entry
> against the metadata of the entry looked up in the previous backup
> snapshot.
> 
> If the metadata matched, the start offset for the payload stream is
> returned.
> 
> This is in preparation for reusing payload chunks for unchanged files.
> 
> Signed-off-by: Christian Ebner <c.ebner at proxmox.com>
> ---
> changes since version 2:
> - refactored to new padding based threshold
> 
>  pbs-client/src/pxar/create.rs | 31 ++++++++++++++++++++++++++++++-
>  1 file changed, 30 insertions(+), 1 deletion(-)
> 
> diff --git a/pbs-client/src/pxar/create.rs b/pbs-client/src/pxar/create.rs
> index 79925bba2..c64084a74 100644
> --- a/pbs-client/src/pxar/create.rs
> +++ b/pbs-client/src/pxar/create.rs
> @@ -21,7 +21,7 @@ use pbs_datastore::index::IndexFile;
>  use proxmox_sys::error::SysError;
>  use pxar::accessor::aio::{Accessor, Directory};
>  use pxar::encoder::{LinkOffset, PayloadOffset, SeqWrite};
> -use pxar::Metadata;
> +use pxar::{EntryKind, Metadata};
>  
>  use proxmox_io::vec;
>  use proxmox_lang::c_str;
> @@ -466,6 +466,35 @@ impl Archiver {
>          .boxed()
>      }
>  
> +    async fn is_reusable_entry(
> +        &mut self,
> +        previous_metadata_accessor: &mut Directory<LocalDynamicReadAt<RemoteChunkReader>>,
> +        file_name: &Path,
> +        stat: &FileStat,
> +        metadata: &Metadata,
> +    ) -> Result<Option<u64>, Error> {
> +        if stat.st_nlink > 1 {
> +            log::debug!("re-encode: {file_name:?} has hardlinks.");
> +            return Ok(None);
> +        }

it would be nice if we had a way to handle those as well.. what's the current
blocker? shouldn't we be able to use the same scheme as for regular archives?

first encounter adds (possibly re-uses) the payload and remembers the offset,
subsequent ones just add another reference/meta entry?

> +
> +        if let Some(file_entry) = previous_metadata_accessor.lookup(file_name).await? {
> +            if metadata == file_entry.metadata() {
> +                if let EntryKind::File { payload_offset, .. } = file_entry.entry().kind() {
> +                    log::debug!("possible re-use: {file_name:?} at offset {payload_offset:?} has unchanged metadata.");
> +                    return Ok(*payload_offset);
> +                }
> +                log::debug!("re-encode: {file_name:?} not a regular file.");
> +                return Ok(None);
> +            }
> +            log::debug!("re-encode: {file_name:?} metadata did not match.");
> +            return Ok(None);
> +        }
> +
> +        log::debug!("re-encode: {file_name:?} not found in previous archive.");
> +        Ok(None)
> +    }
> +
>      /// openat() wrapper which allows but logs `EACCES` and turns `ENOENT` into `None`.
>      ///
>      /// The `existed` flag is set when iterating through a directory to note that we know the file
> -- 
> 2.39.2
> 
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel at lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> 
>