[pbs-devel] [PATCH proxmox-backup v12 12/26] add auto-mounting for removable datastores
Fabian Grünbichler
f.gruenbichler at proxmox.com
Wed Oct 30 11:21:32 CET 2024
Quoting Hannes Laimer (2024-10-29 16:52:41)
> On Mon Oct 14, 2024 at 3:42 PM CEST, Fabian Grünbichler wrote:
> > On September 4, 2024 4:11 pm, Hannes Laimer wrote:
> > > Signed-off-by: Hannes Laimer <h.laimer at proxmox.com>
> > > ---
> > > debian/proxmox-backup-server.install | 1 +
> > > debian/proxmox-backup-server.udev | 3 ++
> > > etc/Makefile | 3 +-
> > > etc/removable-device-attach at .service.in | 8 +++
> > > src/bin/proxmox_backup_manager/datastore.rs | 55 ++++++++++++++++++++-
> > > 5 files changed, 68 insertions(+), 2 deletions(-)
> > > create mode 100644 etc/removable-device-attach at .service.in
> > >
> > > diff --git a/debian/proxmox-backup-server.install b/debian/proxmox-backup-server.install
> > > index ef1e9ba1..34bfc454 100644
> > > --- a/debian/proxmox-backup-server.install
> > > +++ b/debian/proxmox-backup-server.install
> > > @@ -4,6 +4,7 @@ etc/proxmox-backup-daily-update.service /lib/systemd/system/
> > > etc/proxmox-backup-daily-update.timer /lib/systemd/system/
> > > etc/proxmox-backup-proxy.service /lib/systemd/system/
> > > etc/proxmox-backup.service /lib/systemd/system/
> > > +etc/removable-device-attach at .service /lib/systemd/system/
> > > usr/bin/pmt
> > > usr/bin/pmtx
> > > usr/bin/proxmox-tape
> > > diff --git a/debian/proxmox-backup-server.udev b/debian/proxmox-backup-server.udev
> > > index afdfb2bc..e21b8bc7 100644
> > > --- a/debian/proxmox-backup-server.udev
> > > +++ b/debian/proxmox-backup-server.udev
> > > @@ -16,3 +16,6 @@ SUBSYSTEM=="scsi_generic", SUBSYSTEMS=="scsi", ATTRS{type}=="1", ENV{ID_SCSI_SER
> > > SYMLINK+="tape/by-id/scsi-$env{ID_SCSI_SERIAL}-sg"
> > >
> > > LABEL="persistent_storage_tape_end"
> > > +
> > > +# triggers the mounting of a removable device
> > > +ACTION=="add", SUBSYSTEM=="block", ENV{ID_FS_UUID}!="", TAG+="systemd", ENV{SYSTEMD_WANTS}="removable-device-attach@$env{ID_FS_UUID}"
> >
> > so this adds a Wants on our new unit for all block devices with an FS
> > UUID set?
> >
>
> yes, there is not really another way to do this. At least I couldn't
> come up with one, I'm open to suggestions tough
could execute a tiny helper binary that maps the UUID to a datastore name..
there's plenty of precedent for this (e.g., it's how zvols and rbd volumes get
their readable names/links).
could even create a symlink in /dev/disk/by-id then ;)
pbs-datastore-name-$NAME
> > > \ No newline at end of file
> > > diff --git a/etc/Makefile b/etc/Makefile
> > > index 42f639f6..730de4f8 100644
> > > --- a/etc/Makefile
> > > +++ b/etc/Makefile
> > > @@ -7,7 +7,8 @@ DYNAMIC_UNITS := \
> > > proxmox-backup-banner.service \
> > > proxmox-backup-daily-update.service \
> > > proxmox-backup.service \
> > > - proxmox-backup-proxy.service
> > > + proxmox-backup-proxy.service \
> > > + removable-device-attach at .service
> > >
> > > all: $(UNITS) $(DYNAMIC_UNITS) pbs-enterprise.list
> > >
> > > diff --git a/etc/removable-device-attach at .service.in b/etc/removable-device-attach at .service.in
> > > new file mode 100644
> > > index 00000000..e10d1ea3
> > > --- /dev/null
> > > +++ b/etc/removable-device-attach at .service.in
> >
> > why is this `.in` / a dynamic unit?
> >
>
> This is mostly because I wasn't sure where else to put it, where should
> it be/would you put it?
just ship the file as it is, without making it a "dynamic unit" that doesn't
contain anything that is replaced and is 100% static? ;)
>
> > > @@ -0,0 +1,8 @@
> > > +[Unit]
> > > +Description=Try to mount the removable device of a datastore with uuid '%i'.
> > > +After=proxmox-backup-proxy.service
> > > +Requires=proxmox-backup-proxy.service
> > > +
> > > +[Service]
> > > +Type=simple
> > > +ExecStart=/usr/sbin/proxmox-backup-manager datastore uuid-mount %i
> >
> > does this really need to be a custom command?
> >
>
> Is there a good alternative? It does use the normal mount endpoint after
> finding tge datastore(s), the UUID belongs to.
>
> Actually, it only mounts the first, I feel like this is not what we
> want. Either none or all? I'm actually not sure what we want here.
if the unit is templated via the datastore name, then this issue would disappear as well..
>
> > > diff --git a/src/bin/proxmox_backup_manager/datastore.rs b/src/bin/proxmox_backup_manager/datastore.rs
> > > index f2795b39..e74c8537 100644
> > > --- a/src/bin/proxmox_backup_manager/datastore.rs
> > > +++ b/src/bin/proxmox_backup_manager/datastore.rs
> > > @@ -1,4 +1,4 @@
> > > -use anyhow::{format_err, Error};
> > > +use anyhow::{bail, format_err, Error};
> > > use serde_json::Value;
> > >
> > > use proxmox_router::{cli::*, ApiHandler, RpcEnvironment};
> > > @@ -195,6 +195,55 @@ async fn delete_datastore(mut param: Value, rpcenv: &mut dyn RpcEnvironment) ->
> > > Ok(())
> > > }
> > >
> > > +#[api(
> > > + protected: true,
> > > + input: {
> > > + properties: {
> > > + uuid: {
> > > + type: String,
> > > + description: "The UUID of the device that should be mounted",
> > > + },
> > > + "output-format": {
> > > + schema: OUTPUT_FORMAT,
> > > + optional: true,
> > > + },
> > > + },
> > > + },
> > > +)]
> > > +/// Try mounting a removable datastore given the UUID.
> > > +async fn uuid_mount(param: Value, rpcenv: &mut dyn RpcEnvironment) -> Result<Value, Error> {
> > > + let uuid = param["uuid"]
> > > + .as_str()
> > > + .ok_or_else(|| format_err!("uuid has to be specified"))?;
> > > +
> > > + let info = &api2::config::datastore::API_METHOD_LIST_DATASTORES;
> > > + let data: Value = match info.handler {
> > > + ApiHandler::Sync(handler) => (handler)(serde_json::json!({}), info, rpcenv)?,
> > > + _ => unreachable!(),
> > > + };
> > > +
> > > + let store_name = data.as_array().and_then(|list| {
> > > + list.iter()
> > > + .filter_map(Value::as_object)
> > > + .find(|store| store.get("backing-device").map_or(false, |d| d.eq(&uuid)))
> > > + .and_then(|s| s.get("name").and_then(Value::as_str))
> > > + });
> >
> > wouldn't it be easier to just call
> >
> > let config = pbs_config::datastore::config()?;
> > if let Some(store) = config.convert_to_typed_array("datastore")?.into_iter().find(...) {
> > ... store.name
>
> well, yes :)
>
> > }
> >
> > if we stay with this approach?
> >
> > > +
> > > + if let Some(store_name) = store_name {
> > > + let info = &api2::admin::datastore::API_METHOD_MOUNT;
> > > + let mount_param = serde_json::json!({
> > > + "store": store_name,
> > > + });
> > > + let result = match info.handler {
> > > + ApiHandler::Sync(handler) => (handler)(mount_param, info, rpcenv)?,
> > > + _ => unreachable!(),
> > > + };
> > > + crate::wait_for_local_worker(result.as_str().unwrap()).await?;
> > > + return Ok(Value::Null);
> > > + }
> > > + bail!("'{uuid}' is not associated with any datastore")
> >
> > if this triggers for all block devices, isn't it kinda noisy? or am I
> > missing some point where additional filtering takes place..
> >
>
> it is, relatively, depending on how often things are plugged in. It might
> make sense to drop this logging (before having found a matching ds) all together?
see above
>
> > > +}
> > > +
> > > pub fn datastore_commands() -> CommandLineInterface {
> > > let cmd_def = CliCommandMap::new()
> > > .insert("list", CliCommand::new(&API_METHOD_LIST_DATASTORES))
> > > @@ -240,6 +289,10 @@ pub fn datastore_commands() -> CommandLineInterface {
> > > pbs_config::datastore::complete_calendar_event,
> > > ),
> > > )
> > > + .insert(
> > > + "uuid-mount",
> > > + CliCommand::new(&API_METHOD_UUID_MOUNT).arg_param(&["uuid"]),
> > > + )
> > > .insert(
> > > "remove",
> > > CliCommand::new(&API_METHOD_DELETE_DATASTORE)
> > > --
> > > 2.39.2
> > >
> > >
> > >
> > > _______________________________________________
> > > pbs-devel mailing list
> > > pbs-devel at lists.proxmox.com
> > > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> > >
> > >
> > >
> >
> >
> > _______________________________________________
> > pbs-devel mailing list
> > pbs-devel at lists.proxmox.com
> > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
>
>
>
> _______________________________________________
> pbs-devel mailing list
> pbs-devel at lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
More information about the pbs-devel
mailing list