[pbs-devel] [PATCH proxmox-backup v6 0/6] closes #3071: maintenance mode for datastore

[Dominik Csapak]

hi,
it seems the tape patches are squashed into the wrong patch 3/6 instead
of 2/6, but no big issue imho

i found some issues that were already there last time around (sorry)
but i think those are small enough that you can send a follow up..

aside from the issues in 3/6 and 6/6 consider it
(note, i tested it with my suggestion in 3/6)

Reviewed-by: Dominik Csapak <d.csapak at proxmox.com>
Tested-by: Dominik Csapak <d.csapak at proxmox.com>

On 2/2/22 16:49, Hannes Laimer wrote:
> Adds maintenance mode and tracking of active reading/writing operations.
> The maintenance mode prevents the start of new operations if the type of
> operation they would perform on the datastore would conflict with the
> maintenance type that is currently set. This check is performed when
> lookup_datastore is called. Tasks only call this function once at the
> beginning, therefore updating the maintenance type cannot interfere with
> already running tasks.
> 
> active operations tracking:
> Changed file layout to now also keep track of the pid+starttime and the
> counts of operations that that pid started, like this it is possible to
> not count operations that were started by a dead process, since they
> are also not active anymore. Whenever the file is updated, also entries
> of dead processes are removed. When the file is read, only entries of
> active processes are counted.
> 
> The UI shows a spinner with the count of conflictintg tasks (the tasks
> that were started before the maintenance type was updated) next to it.
> As soon as all conflicting tasks are finished a checkmark appears.
> 
> v6:
>   - also use process start time in order to avoid pid clashes(as suggested
>   by Thomas and somehow missed by me in the last version)
>   - now a single call of get_active_operations return reads and writes
>   - improved code structure
>   - don't lock when reading
> 
> v5:
>   - use simple struct and serde instead of manual parsing for file
>   - move tracking related stuff into new file (task_tracking.rs)
> 
> v4:
>   - clones are not also tracked
>   - use lockfile, instead of locking the file
>   - track pid of the process which started smth
>   - updating maintenance mode is now always possible
>   - add get_active_operations endpoint for datastore
>   - ui: show count of conflicting tasks (or checkmark if no conflicting
>       operations are active)
> 
>   v3, based on Dominik Csapak <d.csapak at proxmox.com>'s feedback:
>   - added Operation enum(r/w), as suggested by
>   - added active operation tracking
>   - combine type and message into on field
> 
> v2:
>   - check for maintenance now directly in lookup_datastore
>   - parameter for checking is now the last acceptable maintenance type,
>     description in commit msg of 2nd patch
>   - ui cleanup
> 
> Hannes Laimer (6):
>    api-types: add maintenance type
>    datastore: add check for maintenance in lookup
>    pbs-datastore: add active operations tracking
>    api: make maintenance_type updatable
>    api: add get_active_operations endpoint
>    ui: add option to change the maintenance type
> 
>   pbs-api-types/src/datastore.rs       |   8 +-
>   pbs-api-types/src/lib.rs             |   3 +
>   pbs-api-types/src/maintenance.rs     |  83 +++++++++++++++++++
>   pbs-datastore/Cargo.toml             |   1 +
>   pbs-datastore/src/datastore.rs       | 119 +++++++++++++++++++--------
>   pbs-datastore/src/lib.rs             |   4 +
>   pbs-datastore/src/snapshot_reader.rs |   6 +-
>   pbs-datastore/src/task_tracking.rs   | 110 +++++++++++++++++++++++++
>   src/api2/admin/datastore.rs          |  81 ++++++++++++------
>   src/api2/backup/mod.rs               |   4 +-
>   src/api2/config/datastore.rs         |   5 ++
>   src/api2/reader/mod.rs               |   6 +-
>   src/api2/status.rs                   |   4 +-
>   src/api2/tape/backup.rs              |   6 +-
>   src/api2/tape/restore.rs             |   6 +-
>   src/bin/proxmox-backup-api.rs        |   1 +
>   src/bin/proxmox-backup-proxy.rs      |   6 +-
>   src/server/mod.rs                    |  16 +++-
>   src/server/prune_job.rs              |   4 +-
>   src/server/pull.rs                   |   4 +-
>   src/server/verify_job.rs             |   4 +-
>   www/Makefile                         |   1 +
>   www/Utils.js                         |  23 ++++++
>   www/datastore/OptionView.js          |  30 +++++++
>   www/window/MaintenanceOptions.js     |  72 ++++++++++++++++
>   25 files changed, 522 insertions(+), 85 deletions(-)
>   create mode 100644 pbs-api-types/src/maintenance.rs
>   create mode 100644 pbs-datastore/src/task_tracking.rs
>   create mode 100644 www/window/MaintenanceOptions.js
>