[pbs-devel] [PATCH proxmox-backup v5 00/22] add removable datastores

[Hannes Laimer]

These patches add support for removable datastores. All removable
datastores have a backing-device(a UUID) associated with them. Removable
datastores work like normal ones, just that they can be unplugged. It is
possible to create a removable datastore, sync backups onto it, unplug
it and use it on a different PBS.

The datastore path is also the mountpoint for the removable datastore.
By default when creating one through the web UI it will be
`/mnt/removable-datastores/<UUID>`, using the CLI it is possible to
specify something else. Since a removable datastore is associated with
the UUID of the fs on a partition, it is technically possible to have two
removable datastores on the same device, but I don't think there is a
use-case that couldn't also be done using namespaces.

When a removable datastore is deleted and 'destroy-data' is set, the
device has to be plugged in. If 'destroy-data' is not set the datastore
can be deleted even if the device is not present. Removable datastores
are automatically mounted when plugged in. At the API service start all
removable datastores are marked as 'unplugged', unless they are already
mounted.

Patches 09/22 and 20/22 are not strictly needed, but they made sense in
this context, so I kept them in this series. 

Tested-by: Christian Ebner <c.ebner at proxmox.com>

(for Rust)
Reviewed-by: Christian Ebner <c.ebner at proxmox.com>
 
v5: thanks @Dietmar and @Christian
 * drop --force for unmount since it'll always fail if tasks are still running, and if
    there are not normal unount will work
 * improve several commit messages
 * improve error message wording
 * add removable datastore section to docs
 * add documentation for is_datastore_available

v4: thanks a lot @Dietmar and @Christian
 * make check if mounted wayyy faster
 * don't keep track of mounting state
 * drop Unplugged maintenance mode
 * use UUID_FORMAT for uuid field
 * a lot of small things, like use of bail!, inline format!, ...
 * include improvement to cache handling

v3:
 * remove lazy unmounting (since 9cba51ac782d04085c0af55128f32178e5132358 is applied)
 * fix CLI (un)mount command, thanks @Gabriel
 * add removable datastore CLI autocomplete helper
 * rebase onto master
 * move ui patches to the end

thanks @Lukas and @Thomas for the feedback
v2:
 * fix datastore 'add' button in the UI
 * some format!("{}", a) -> format!("{a}")
 * replace `const` with `let` in js code
 * change icon `fa-usb` -> `fa-plug`
 * add some docs
 * add JDoc for parseMaintenanceMode
 * proxmox-schema dep bump

Hannes Laimer (22):
  tools: add disks utility functions
  datastore: add cache entry flag for droping it after last task
  pbs-api-types: add backing-device to DataStoreConfig
  disks: add UUID to partition info
  add helper for checking if a removable datastore is available
  api2: admin: add (un)mount endpoint for removable datastores
  api2: removable datastore creation
  api2: disks list: add only-unused flag
  pbs-api-types: use SchemaDeserializer for maintenance mode
  pbs-api-types: add removable/is-available flag to DataStoreListItem
  bin: manager: add (un)mount command
  add auto-mounting for removable datastores
  datastore: handle deletion of removable datastore properly
  docs: add removable datastores section
  ui: add partition selector form
  ui: add removable datastore creation support
  ui: add (un)mount button to summary
  ui: display removable datastores in list
  ui: utils: render unplugged datastores correctly
  ui: utils: make parseMaintenanceMode more robust
  ui: add datastore status mask for unplugged removable datastores
  ui: maintenance: fix disable msg field if no type is selected

 debian/proxmox-backup-server.install        |   1 +
 debian/proxmox-backup-server.udev           |   3 +
 docs/storage.rst                            |  23 +++
 etc/Makefile                                |   3 +-
 etc/removable-device-attach at .service.in     |   8 +
 pbs-api-types/src/datastore.rs              |  33 +++-
 pbs-api-types/src/maintenance.rs            |   4 +
 pbs-config/src/datastore.rs                 |  14 ++
 pbs-datastore/src/datastore.rs              | 123 +++++++++----
 pbs-datastore/src/lib.rs                    |   2 +-
 src/api2/admin/datastore.rs                 | 184 ++++++++++++++++++--
 src/api2/config/datastore.rs                |  75 +++++++-
 src/api2/node/disks/mod.rs                  |   8 +
 src/api2/status.rs                          |  18 +-
 src/bin/proxmox_backup_manager/datastore.rs | 130 +++++++++++++-
 src/tools/disks/mod.rs                      |  90 ++++++++--
 www/Makefile                                |   1 +
 www/NavigationTree.js                       |  15 +-
 www/Utils.js                                |  31 +++-
 www/css/ext6-pbs.css                        |  20 +++
 www/datastore/DataStoreListSummary.js       |   1 +
 www/datastore/Summary.js                    | 103 +++++++++--
 www/form/PartitionSelector.js               |  74 ++++++++
 www/window/DataStoreEdit.js                 |  51 ++++++
 www/window/MaintenanceOptions.js            |   7 +-
 25 files changed, 919 insertions(+), 103 deletions(-)
 create mode 100644 etc/removable-device-attach at .service.in
 create mode 100644 www/form/PartitionSelector.js

-- 
2.39.2