[pbs-devel] [PATCH v3 proxmox-backup 54/58] client: pxar: add flow chart for metadata change detection

[Christian Ebner]

On 4/5/24 12:16, Fabian Grünbichler wrote:
> Quoting Christian Ebner (2024-03-28 13:37:03)
>> A high level flow chart describing the logic used for the metadata
>> based file change detection mode.
>>
>> Signed-off-by: Christian Ebner <c.ebner at proxmox.com>
>> ---
>> changes since version 2:
>> - not present in previous version
>>
>>   ...ow-chart-metadata-based-file-change-detection.svg |  1 +
>>   ...ow-chart-metadata-based-file-change-detection.txt | 12 ++++++++++++
>>   2 files changed, 13 insertions(+)
>>   create mode 100644 pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.svg
>>   create mode 100644 pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.txt
>>
>> diff --git a/pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.svg b/pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.svg
>> new file mode 100644
>> index 000000000..5e6df4815
>> --- /dev/null
>> +++ b/pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.svg
>> @@ -0,0 +1 @@
> 
> [snip]
> 
> something here got broken (I guess mail related somewhere along the way?). it
> does work if the contents are manually merged backed into a single line before
> applying the patch though. in any case, it probably would be nice to have this
> autogenerated and moved to some part of the docs :)
> 
>> diff --git a/pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.txt b/pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.txt
>> new file mode 100644
>> index 000000000..5eace70be
>> --- /dev/null
>> +++ b/pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.txt
>> @@ -0,0 +1,12 @@
>> +flowchart TD
>> +    A[Archiver] -->|lookup metadata| B[Accessor]
>> +    B -->|is reusable entry| C[Lookahead Cache]
>> +    C -->|lookup reusable chunks| D[Dynamic Index]
>> +    D -->|insert and deduplicate dynamic entries| E[Reused Chunks]
>> +    B -->|is not reusable entry| F(re-encode cached entries and current entry)
>> +    F -->|caching disabled| A
>> +    E -->|padding above threshold, non-continuous chunks, caching disabled| F
>> +    E -->|padding above threshold, chunks continuous, caching enabled| A
>> +    E -->|padding below threshold| G(force boundary, inject chunks, keepback last chunk for potential followup)
>> +    G -->|caching enabled| A
> 
> the caching enabled/disabled parts here are probably confusing (do those edges
> mean caching is enabled/disabled at that point? or are they taken if it is
> enabled/disabled?)
> 
> but probably it makes sense to re-visit this in detail once the dust has settled :)
> 

Yes agreed, will drop this patch for now and rethink on how to include 
this better, or even move some of this into the documentation in text 
form (although in a more user oriented fashion rather than dev oriented).