[pbs-devel] [RFC v2 proxmox-backup 27/36] client: implement prepare reference method

[Fabian Grünbichler]

> Christian Ebner <c.ebner at proxmox.com> hat am 19.03.2024 12:51 CET geschrieben:
> On 3/12/24 11:07, Fabian Grünbichler wrote:
> > for these two, it might make sense to differentiate between:
> > - previous manifest doesn't have that index -> no need to try download,
> >    we can just skip
> > - previous manifest has that index -> we try to download -> we need to
> >    handle the error (and tell the user about the error message - it might
> >    indicate a problem after all!)
> 
> I think opting for the fallback to regular encoding should be fine here, 
> as I could not find a straight forward way to differentiate between the 
> two cases. Or did I miss something?
> 
> Not sure the current backup should fail because the previous one is not 
> fine.

you misunderstood what I meant. instead of attempting to download the previous index, we could first check whether it even exists. if it doesn't exist, than we can skip the download, just log an info value, and continue without a reference. if it does (should) exist, and we attempt to download it, but the download fails - then we at least want to print a different message, or potentially fail the backup.

e.g., this is how the regular incremental backup part of the code handles this:

Previous manifest does not contain an archive called 'mail2.pxar.meta.didx', skipping download..

we used to always try to download and always print an error there if that download failed, which was confusing for host backups when the user added an archive or renamed one (scary message for no reason). there the only downside when a previous index cannot be downloaded is that the chunks have to be re-uploaded, there is no difference in the resulting archives. 

in the new code here we have a different variant - we always try to download (unnecessary!), but always treat a failure as benign (not sure about that part - it actually makes a difference content-wise, but also possibly speed/load wise ;)).

also we now download the meta index twice I guess (if it exists).

last, but not least - there is no actual "fallback to regular encoding" (or at least, I think this would confuse users, for whom "regular encoding" might mean v1 archive, not "v2 archive but no re-use via metadata caching").