[pve-devel] [PATCH storage v5 0/7] support moving volumes between storages

[Filip Schauer]

Add the ability to move a backup, ISO, container template, snippet, or
OVA/OVF between storages and nodes via an API method. Moving a VMA
backup to a Proxmox Backup Server requires the proxmox-vma-to-pbs
package to be installed. Currently only VMA backups can be moved to a
Proxmox Backup Server and moving backups from a Proxmox Backup Server is
currently not supported.

The method can be called from the PVE shell with `pvesm move-volume`:

```
pvesm move-volume <source volume> <target storage> [--target-node <node>] [--delete]
```

For example to move a VMA backup to a Proxmox Backup Server:

```
pvesm move-volume \
    local:backup/vzdump-qemu-100-2024_06_25-13_08_56.vma.zst pbs
```

Or move a container template to another node and delete the source:

```
pvesm move-volume \
    local:vztmpl/devuan-4.0-standard_4.0_amd64.tar.gz local \
    --target-node pvenode2 --delete
```

Or use curl to call the API method:

```
curl https://$APINODE:8006/api2/json/nodes/$SOURCENODE/storage/$SOURCESTORAGE/content/$SOURCEVOLUME \
    --insecure --cookie "$(<cookie)" -H "$(<csrftoken)" -X POST \
    --data-raw "target-storage=$TARGETSTORAGE&target-node=$TARGETNODE"
```

Changes since v4:
* Remove the volume_move subroutine, instead use storage_migrate for
  moving volumes between storages on the same node
* Avoid ssh when moving a volume between storages on the same node
* Add command completion to move-volume parameters
* Make the success message of move-volume less verbose for moves within
  the same node
* utf8 encode/decode backup notes during export/import
* Support the new "import" volume type
* Code cleanup
* Add descriptions to single line commit messages

Changes since v3:
* Split changes into multiple commits
* Remove superfluous parentheses from post-ifs
* Move vma_to_pbs branch from move_volume into its own helper inside
  PBSPlugin
* Use $! instead of $@ to retrieve unlink error in move_volume
* Also support content type 'rootdir'
* Rework permission checks on the move API method
* Fix permissions description on move API method
* Add error for unimplemented content types

Changes since v2:
* Specify permissions for move method
* Add success message to move method
* Limit the move method to non-vdisk volumes
* Check that source and target are not the same in the move method
* Remove the unnecessary movevolume method from pvesm and make the
  move-volume command call the move API method directly
* Fail when trying to move a protected volume with the delete option
  enabled, instead of ignoring the protection
* Change "not yet supported" to "not supported" in messages indicating
  unimplemented features
* Process auxiliary files first when moving a volume locally on a node
* Move a volume instead of copying it when trying to move a volume
  locally on a node with the delete option enabled.
* Use the more general `path` function instead of `filesystem_path` to
  get the path of a volume
* Loosen the required privileges to move an ISO or a container template,
  or when the delete option is not set.
* Move the volume_move sub from PVE::Storage to
  PVE::API2::Storage::Content since it is only used there.
* Explicitly check that storages are path-based in volume_move,
  except when moving a vma to a Proxmox Backup Server

Changes since v1:
* Rename pvesm command to move-volume
* Add a delete option to control whether the source volume should be
  kept
* Move the API method to the POST endpoint of
  /nodes/{node}/storage/{storage}/content/{volume}, replacing the
  experimental copy method that has not been used since its introduction
  in October 2011 883eeea6.
* Implement migrating volumes between nodes

Filip Schauer (7):
  plugin: allow volume import of iso, snippets, vztmpl and import
  api: content: implement moving a volume between storages
  api: content: support moving backups between path based storages
  storage: introduce decompress_archive_into_pipe helper
  support moving VMA backups to PBS
  pvesm: add a move-volume command
  storage migrate: avoid ssh when moving a volume locally

 debian/control                  |   1 +
 src/PVE/API2/Storage/Content.pm | 131 +++++++++++++++++++++++---------
 src/PVE/CLI/pvesm.pm            |   2 +
 src/PVE/Storage.pm              |  80 ++++++++++++-------
 src/PVE/Storage/PBSPlugin.pm    |  65 ++++++++++++++++
 src/PVE/Storage/Plugin.pm       | 112 ++++++++++++++++++++-------
 6 files changed, 299 insertions(+), 92 deletions(-)

-- 
2.39.5