[pbs-devel] [PATCH proxmox-backup] tape: skip setting encryption if we can't and don't want to

[Dominik Csapak]

Some settings on changers prevents changing the encryption parameters
via the application, e.g. some libraries have a 'encryption disabled' or
'encryption is library managed' option. While the former situation can
be fixed by setting the library to 'application managed', the latter is
sometimes necessary for FIPS compliance (to ensure the tape data is
encrypted).

When libraries are configured this way, the code currently fails with
'drive does not support AES-GCM encryption'. Instead of failing, check
on first call to set_encryption if we could set it, and save that
result.

Only fail when encryption is to be enabled but it is not allowed, but
ignore the error when the backup should be done unencrypted.

With this change, it should be possible to use such configured libraries
when there is no encryption configured on the PBS side. (We currently
don't have a library with such capabilities to test.)

Note that in contrast to normal operation, the tape label will also be
encrypted then and will not be readable in case the encryption key is
lost or changed.

Additionally, return an error for 'drive_set_encryption' in case the
drive reports that it does not support hardware encryption, because this
is now already caught one level above in 'set_encryption'.

This was reported in the community forum:

https://forum.proxmox.com/threads/107383/
https://forum.proxmox.com/threads/164941/

Signed-off-by: Dominik Csapak <d.csapak at proxmox.com>
---
 pbs-tape/src/sg_tape.rs            | 24 +++++++++++++++++++++++-
 pbs-tape/src/sg_tape/encryption.rs | 10 +---------
 2 files changed, 24 insertions(+), 10 deletions(-)

diff --git a/pbs-tape/src/sg_tape.rs b/pbs-tape/src/sg_tape.rs
index 39e6a2f94..c49714f6e 100644
--- a/pbs-tape/src/sg_tape.rs
+++ b/pbs-tape/src/sg_tape.rs
@@ -136,6 +136,8 @@ pub struct SgTape {
     file: File,
     locate_offset: Option<i64>,
     info: InquiryInfo,
+    // auto-detect if we can set the encryption mode
+    encryption_possible: Option<bool>,
 }
 
 impl SgTape {
@@ -158,6 +160,7 @@ impl SgTape {
             file,
             info,
             locate_offset: None,
+            encryption_possible: None,
         })
     }
 
@@ -690,6 +693,14 @@ impl SgTape {
     }
 
     pub fn set_encryption(&mut self, key_data: Option<([u8; 32], Uuid)>) -> Result<(), Error> {
+        if self.encryption_possible == Some(false) {
+            if key_data.is_some() {
+                bail!("Drive does not support setting encryption mode");
+            } else {
+                // skip trying to set encryption if not supported and don't wanted
+                return Ok(());
+            }
+        }
         let key = if let Some((ref key, ref uuid)) = key_data {
             // derive specialized key for each media-set
 
@@ -710,7 +721,18 @@ impl SgTape {
             None
         };
 
-        drive_set_encryption(&mut self.file, key)
+        match drive_set_encryption(&mut self.file, key) {
+            Ok(()) => self.encryption_possible = Some(true),
+            Err(err) => {
+                self.encryption_possible = Some(false);
+                if key.is_some() {
+                    bail!("could not set encryption mode on drive: {err}");
+                } else {
+                    log::info!("could not set encryption mode on drive: {err}, ignoring.");
+                }
+            }
+        }
+        Ok(())
     }
 
     // Note: use alloc_page_aligned_buffer to alloc data transfer buffer
diff --git a/pbs-tape/src/sg_tape/encryption.rs b/pbs-tape/src/sg_tape/encryption.rs
index 7247d257f..c87986dca 100644
--- a/pbs-tape/src/sg_tape/encryption.rs
+++ b/pbs-tape/src/sg_tape/encryption.rs
@@ -12,15 +12,7 @@ use crate::sgutils2::{alloc_page_aligned_buffer, SgRaw};
 ///
 /// We always use mixed mode,
 pub fn drive_set_encryption<F: AsRawFd>(file: &mut F, key: Option<[u8; 32]>) -> Result<(), Error> {
-    let data = match sg_spin_data_encryption_caps(file) {
-        Ok(data) => data,
-        Err(_) if key.is_none() => {
-            // Assume device does not support HW encryption
-            // We can simply ignore the clear key request
-            return Ok(());
-        }
-        Err(err) => return Err(err),
-    };
+    let data = sg_spin_data_encryption_caps(file)?;
 
     let algorithm_index = decode_spin_data_encryption_caps(&data)?;
 
-- 
2.39.5