[pbs-devel] [PATCH v3 proxmox-backup 45/58] client: pxar: add method for metadata comparison

[Christian Ebner]

On 4/5/24 10:08, Fabian Grünbichler wrote:
> Quoting Christian Ebner (2024-03-28 13:36:54)
>> Adds a method to compare the metadata of the current file entry
>> against the metadata of the entry looked up in the previous backup
>> snapshot.
>>
>> If the metadata matched, the start offset for the payload stream is
>> returned.
>>
>> This is in preparation for reusing payload chunks for unchanged files.
>>
>> Signed-off-by: Christian Ebner <c.ebner at proxmox.com>
>> ---
>> changes since version 2:
>> - refactored to new padding based threshold
>>
>>   pbs-client/src/pxar/create.rs | 31 ++++++++++++++++++++++++++++++-
>>   1 file changed, 30 insertions(+), 1 deletion(-)
>>
>> diff --git a/pbs-client/src/pxar/create.rs b/pbs-client/src/pxar/create.rs
>> index 79925bba2..c64084a74 100644
>> --- a/pbs-client/src/pxar/create.rs
>> +++ b/pbs-client/src/pxar/create.rs
>> @@ -21,7 +21,7 @@ use pbs_datastore::index::IndexFile;
>>   use proxmox_sys::error::SysError;
>>   use pxar::accessor::aio::{Accessor, Directory};
>>   use pxar::encoder::{LinkOffset, PayloadOffset, SeqWrite};
>> -use pxar::Metadata;
>> +use pxar::{EntryKind, Metadata};
>>   
>>   use proxmox_io::vec;
>>   use proxmox_lang::c_str;
>> @@ -466,6 +466,35 @@ impl Archiver {
>>           .boxed()
>>       }
>>   
>> +    async fn is_reusable_entry(
>> +        &mut self,
>> +        previous_metadata_accessor: &mut Directory<LocalDynamicReadAt<RemoteChunkReader>>,
>> +        file_name: &Path,
>> +        stat: &FileStat,
>> +        metadata: &Metadata,
>> +    ) -> Result<Option<u64>, Error> {
>> +        if stat.st_nlink > 1 {
>> +            log::debug!("re-encode: {file_name:?} has hardlinks.");
>> +            return Ok(None);
>> +        }
> 
> it would be nice if we had a way to handle those as well.. what's the current
> blocker? shouldn't we be able to use the same scheme as for regular archives?
> 
> first encounter adds (possibly re-uses) the payload and remembers the offset,
> subsequent ones just add another reference/meta entry?

True, this is a leftover from the initial approach with the appendix 
section instead of the split archive where it caused issues.