[pbs-devel] [PATCH proxmox-backup v14 00/26] add removable datastores

[Lukas Wagner]

On  2024-11-22 15:46, Hannes Laimer wrote:
> These patches add support for removable datastores. All removable
> datastores have a backing-device(a UUID) associated with them. Removable
> datastores work like normal ones, just that they can be unplugged. It is
> possible to create a removable datastore, sync backups onto it, unplug
> it and use it on a different PBS.
> 
> The datastore path specified is relative to the root of the used device.
> Removable datastores are bind mounted to /mnt/datastore/<NAME>.
> Multiple datastores can be created on a single device, but only device with 
> a single datastore on them will be auto-mounted.
> 
> When a removable datastore is deleted and 'destroy-data' is set, the
> device has to be mounted. If 'destroy-data' is not set the datastore
> can be deleted even if the device is not present. Removable datastores
> are automatically mounted when plugged in. 
> 

Tested these patches against the latest state on master. Works fine generally,
but there were a couple of rough edges that were at least unexpected (at least to me, not being
very familiar with this new feature)

In no particular order, they were:

- When creating a removable datastore, it struck me as odd that the "On device path" has to be absolute
  (start with a /), considering that the path you enter is relative to the root directory of the
  partition.

- a removable datastore cannot be ZFS. I guess there are technical reasons for this, but it was a
  bit unexpected, especially considering that you can create a non-removable datastore on ZFS without
  any issues. Maybe the docs could give some background on this, or at least mention that
  removable datastores cannot be created on ZFS formatted disks

- ran into an issue with the 'unmounting' maintenance mode. I pressed 'unmount' in the UI.
  One of my bash session still had its working directory in the mounted datastore, leading to umount failing
  with a 'device busy' message. After this, the datastore was in the 'unmounting' maintenance mode
  which I could not clear anymore, neither through the UI, nor through the proxmox-backup-manager
  CLI tool. Eventually I had to remove the `maintenance-mode` line from `datastores.cfg` by hand.

- When a datastore is unmounted, accessing the GC job view or content view, the system logs are spammed by HTTP 400 error messages, e.g.

    GET /api2/json/admin/datastore/disk2/status: 400 Bad Request: [client [::xxxxxxxxx]:34382] datastore 'disk2' is not mounted
  
  Might be annoying and/or confusing for system admins looking through the logs when they debug some other issue.

- Similarly, if a removeable-datastore is added as a storage in PVE, pvestatd logs errors every 10 seconds if the
  datastore is not mounted. Not sure if this is possible, but maybe we could handle this more graceful to not
  spam the logs?

- Something that also was a bit confusing was the following:
  - Attached new disk to my PBS test VM
  - Formatted the disk as ext4 using the webui (name: disk1, device: sdb1), opting to NOT create a datastore as part of the formatting process
  - Created a removable datastore on the new partition (name: disk1-store, device: sdb1)
  -> This led to the partition being mounted twice:

/dev/sdb1 on /mnt/datastore/disk1 type ext4 (rw,relatime)
/dev/sdb1 on /mnt/datastore/disk1-store type ext4 (rw,relatime)

  -> if 'unmount' is pressed for the datastore, only the second mount is unmounted. This could be confusing
    to users who expect to be able to safely remove the disk after unmounting the datastore.

  - For contrast, if while creating the partition one opts to create the removable datastore immediately,
    the partition is only mounted once.


- Also, when trying to mount a datastore without the disk being available, the error message in the task
  log might not be super clear to some users:

TASK ERROR: mounting to tmp path failed: command "mount" "UUID=a264c664-9e0e-47ad-abf5-960e0aabfe0b" "/run/proxmox-backup/mount/5292c7d2-a54b-42f1-be87-88810b2e90cd" failed - status code: 1 - mount: /run/proxmox-backup/mount/5292c7d2-a54b-42f1-be87-88810b2e90cd: can't find UUID=a264c664-9e0e-47ad-abf5-960e0aabfe0b.)

  Maybe we could add clearer error messages for the more common error cases, like disks being not available?


I don't think any of these are blockers, but nevertheless I thought I'd bring these up.

-- 
- Lukas