[pbs-devel] applied: [PATCH proxmox-backup] chunk store: handle insertion edge cases

[Thomas Lamprecht]

Am 31/03/2023 um 10:43 schrieb Fabian Grünbichler:
> these were previously called out in a comment, but should now be handled (as
> much as they can be).
> 
> the performance impact shouldn't be too bad, since we only look at the magic 8
> bytes at the start of the existing chunk (we already did a stat on it, so that
> might even be prefetched already by storage), and only if there is a size
> mismatch and encryption is enabled.
> 
> Signed-off-by: Fabian Grünbichler <f.gruenbichler at proxmox.com>
> ---
> 
> Notes:
>     we could verify the CRC when deciding between existing and incoming encrypted
>     chunk, but that would require reading the full chunk, and that would be quite
>     the load if we ever upgrade ZSTD or change its parameters and the new version
>     is substantially better or worse at compressing.. the CRC is verified on
>     every regular load (such as verify, sync, restore) anyway.
>     
>     we cannot decide which of the two potentially invalid encrypted chunks to keep
>     based on the size and compression status: uncompressed chunks should always
>     be bigger than compressed ones, but both the size and the compression bit is
>     100% under a potential attacker's control anyhow, so we cannot prevent them
>     from sending us rather small chunks that we still need to discard out of
>     caution, even if they are smaller than the existing one.
> 
>  pbs-datastore/src/chunk_store.rs | 36 ++++++++++++++++++++++++++------
>  pbs-datastore/src/data_blob.rs   |  6 ++++++
>  2 files changed, 36 insertions(+), 6 deletions(-)
> 
>

applied, with touch_chunk calls amended like discussed off-list, thanks!