[pbs-devel] [PATCH proxmox-backup 5/6] datastore: verify: evict corrupt chunks from in-memory LRU cache

[Christian Ebner]

Chunks detected as corrupt have been renamed on both, the S3 backend
and the local datastore cache, but not evicted from the in-memory
cache containing the LRU chunk digests. This can lead to the chunks
being considered as already present if their digest is still cached,
and therefore not being re-inserted in the local store cache and S3
backend on backup upload.

Fix this by not only renaming the local datastore's chunk marker
file, but also removing it from the in-memory cache while holding the
chunk store mutex lock to exclude interference from concurrent chunk
inserts.

Reported-by: Fabian Grünbichler <f.gruenbichler at proxmox.com>
Signed-off-by: Christian Ebner <c.ebner at proxmox.com>
---
 pbs-datastore/src/datastore.rs | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs
index a7ea8fd96..c551679e3 100644
--- a/pbs-datastore/src/datastore.rs
+++ b/pbs-datastore/src/datastore.rs
@@ -2467,10 +2467,18 @@ impl DataStore {
 
         let _lock = self.inner.chunk_store.mutex().lock().unwrap();
 
-        match std::fs::rename(&path, &new_path) {
+        let result = match std::fs::rename(&path, &new_path) {
             Ok(_) => Ok(Some(format!("corrupted chunk renamed to {new_path:?}"))),
             Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(None),
             Err(err) => bail!("could not rename corrupted chunk {path:?} - {err}"),
+        };
+
+        if let Some(cache) = self.cache() {
+            // Requiremets for calling the unsafe method are met, since snapshots referencing the
+            // corrupt chunk are to be considered corrupt. Ignore the error due to the missing file.
+            let _ = unsafe { cache.remove(digest) };
         }
+
+        result
     }
 }
-- 
2.47.3