[pbs-devel] [PATCH many v5 00/44] integrate notification system
Lukas Wagner
l.wagner at proxmox.com
Tue Apr 23 13:51:46 CEST 2024
These patches integrate the notification system which was introduced in Proxmox VE
in 8.1 into Proxmox Backup Server.
Some highlights/noteworthy details from the series:
- notification template files are installed in /usr/share/proxmox-backup/templates/default
and are rendered when a notification is sent
- since sending notifications needs to be done from a privileged context
(to read protected passwords/tokens from the priv config file),
we queue notifications to be sent in /var/lib/proxmox-backup/notifications
and periodically send any queued notifications via a worker in the
privileged process
- The API endpoint paths are prefixed with /config/notifications
- API endpoints which read/modify notification system config require
Sys.Audit (read) or Sys.Modify (modify/test endpoint) permissions
on /system/notifications
- tape-{backup,restore} settings and datastore options now have a
'notification-mode' parameter, which allows to choose between the
'legacy' behavior (sendmail to a selected user's email address) and
the event-based notification system. If the parameter is not set, we
default to the legacy behavior in order to keep existing behavior
as is for now. For new datastores/tape backup jobs created
interactively from the UI, we set the parameter to 'notification-system'
and opt in into the new system by default. For the CLI/API we don't
so that we don't cause problems with any automations/scripts -
there I'd change the default to the new system with the next major release
Rough edges:
- Datastore option view in UI could be improved. When 'notification-mode' is
set to 'notification-system', we should indicate that 'mailto-user' and the
other notification settings have no effect. Already did that in the edit
window, but in the grid panel I did not find a quick way to do that.
Prerequisites:
- This patch series requires the patches for 'proxmox' from
https://lists.proxmox.com/pipermail/pve-devel/2024-April/063493.html
to be applied first
- The dependency for proxmox-widget-toolkit should be bumped to at least
4.1.4 (we need: "utils: add mechanism to add and override translatable notification event
descriptions in the product specific UIs")
Changes since v4:
- add __default__ for notification mode combo boxes to avoid issues with
existing jobs/datastores
- allow users with Datastore.Backup to see datastores in matcher value suggestions
- fix copy/paste error in docs
- cargo fmt
Changes since v3:
- Pull in UI changes from 'notification metadata matching improvments' as
component overrides - this can and should be removed once the
widget toolkit patches are applied.
- no other changes apart from that
Changes since v2:
- consolidate verify/prune/sync/tape-job to job-id
- add a note to docs that package update notifications
are only sent by nodes with an active subscription
Changes since v1:
- proxmox_notify: Change default-matcher to exclude 'prune success' notifications
- proxmox_notify: Move sendmail/forward functions from proxmox_sys
- proxmox_notify: Improve docs for API types
- proxmox-widget-toolkit: allow to override default mail author
- Add documentation, man pages, etc. (mostly copied from PVE, some
stylistic touchups, some rephrasing)
- Add additional matchable metadata fields, allowing to match on
individual jobs:
- verification-job
- prune-job
- sync-job
- tape-job
- Sort queued notifications by timestamp before sending
- Fix missing hostname in package-update notification
- Change default mail author in the UI from Proxmox VE to Proxmox Backup Server ($hostname)
proxmox:
Lukas Wagner (5):
notify: expose `config` module
notify: use std::sync::OnceCell instead of lazy_static!
notify: pbs-context: exclude successful prunes in default matcher
notify: endpoints: matcher: improve descriptions for API types
notify: add getter for notification timestamp
proxmox-notify/Cargo.toml | 1 -
proxmox-notify/src/config.rs | 23 ++++++++++++-------
proxmox-notify/src/context/pbs.rs | 5 ++++-
proxmox-notify/src/endpoints/gotify.rs | 6 ++---
proxmox-notify/src/endpoints/sendmail.rs | 15 ++++++++-----
proxmox-notify/src/endpoints/smtp.rs | 28 +++++++++++++++---------
proxmox-notify/src/lib.rs | 11 ++++++----
proxmox-notify/src/matcher.rs | 18 +++++++--------
8 files changed, 66 insertions(+), 41 deletions(-)
proxmox-widget-toolkit:
Lukas Wagner (1):
sendmail: smtp: allow to overide default mail author
src/Schema.js | 9 +++++++++
src/panel/SendmailEditPanel.js | 2 +-
src/panel/SmtpEditPanel.js | 2 +-
src/window/EndpointEditBase.js | 1 +
4 files changed, 12 insertions(+), 2 deletions(-)
proxmox-backup:
Lukas Wagner (38):
pbs-config: add module for loading notification config
server: rename email_notifications module to notifications
notifications: allow sending notifications via proxmox_notify
buildsys: install templates for test notifications
pbs-config: acl: add /system/notifications as known ACL path
api: add endpoints for querying/testing notification targets
api: add endpoints for notification matchers
api: add endpoints for sendmail targets
api: add endpoints for smtp targets
api: add endpoints for gotify targets
api: add endpoints for querying known notification values/fields
api-types: api: datatore: add notification-mode parameter
api-types: api: tape: add notification-mode parameter
server: notifications: send GC notifications via notification system
server: notifications: send prune notifications via notification
system
server: notifications: send verify notifications via notification
system
server: notifications: send sync notifications via notification system
server: notifications: send update notifications via notification
system
server: notifications: send acme notifications via notification system
server: notifications: send tape notifications via notification system
ui: add notification config panel
ui: tape backup job: add selector for notification-mode
ui: tape backup: add selector for 'notification-mode'
ui: tape restore: add 'notification-mode' parameter
ui: datastore options: add 'notification-mode' parameter
ui: utils: add overrides for known notification metadata fields/values
ui: datastore edit: make new stores use notification system by default
ui: permissions paths: add /system/notifications to combobox
proxmox-backup-manager: add CLI for notification targets
proxmox-backup-manager: add CLI for notification matchers
proxmox-backup-manager: add CLI for gotify endpoints
proxmox-backup-manager: add CLI for sendmail endpoints
proxmox-backup-manager: add CLI for SMTP endpoints
docgen: generate synopsis for notifications{-priv,}.cfg
docs: add documentation for notification system
ui: util: override default mail author for sendmail/smtp targets
ui: notifications: pull in UX improvements for match rules creation
api: notification: also list datastores if user has only Backup privs
Cargo.toml | 3 +
Makefile | 5 +-
debian/proxmox-backup-server.install | 31 +
docs/Makefile | 6 +-
docs/conf.py | 2 +
docs/config/notifications-priv/format.rst | 1 +
docs/config/notifications-priv/man5.rst | 24 +
docs/config/notifications/format.rst | 2 +
docs/config/notifications/man5.rst | 24 +
docs/configuration-files.rst | 30 +
docs/index.rst | 1 +
docs/notifications.rst | 213 ++++
pbs-api-types/src/datastore.rs | 22 +
pbs-api-types/src/jobs.rs | 8 +-
pbs-config/Cargo.toml | 1 +
pbs-config/src/acl.rs | 3 +-
pbs-config/src/lib.rs | 1 +
pbs-config/src/notifications.rs | 41 +
src/api2/config/datastore.rs | 9 +
src/api2/config/mod.rs | 2 +
src/api2/config/notifications/gotify.rs | 190 +++
src/api2/config/notifications/matchers.rs | 170 +++
src/api2/config/notifications/mod.rs | 205 +++
src/api2/config/notifications/sendmail.rs | 178 +++
src/api2/config/notifications/smtp.rs | 191 +++
src/api2/config/notifications/targets.rs | 63 +
src/api2/config/tape_backup_job.rs | 8 +
src/api2/pull.rs | 10 +-
src/api2/tape/backup.rs | 62 +-
src/api2/tape/restore.rs | 46 +-
src/bin/docgen.rs | 4 +
src/bin/proxmox-backup-api.rs | 11 +
src/bin/proxmox-backup-manager.rs | 2 +
src/bin/proxmox-backup-proxy.rs | 1 +
src/bin/proxmox_backup_manager/mod.rs | 2 +
.../notifications/gotify.rs | 93 ++
.../notifications/matchers.rs | 93 ++
.../notifications/mod.rs | 21 +
.../notifications/sendmail.rs | 94 ++
.../notifications/smtp.rs | 96 ++
.../notifications/targets.rs | 51 +
src/server/email_notifications.rs | 763 ------------
src/server/gc_job.rs | 10 +-
src/server/mod.rs | 4 +-
src/server/notifications.rs | 566 +++++++++
src/server/verify_job.rs | 10 +-
src/tape/drive/mod.rs | 22 +-
src/tape/mod.rs | 27 +
src/tape/pool_writer/mod.rs | 11 +-
templates/Makefile | 41 +
templates/default/acme-err-body.txt.hbs | 7 +
templates/default/acme-err-subject.txt.hbs | 1 +
templates/default/gc-err-body.txt.hbs | 8 +
templates/default/gc-err-subject.txt.hbs | 1 +
templates/default/gc-ok-body.txt.hbs | 23 +
templates/default/gc-ok-subject.txt.hbs | 1 +
.../default/package-updates-body.txt.hbs | 8 +
.../default/package-updates-subject.txt.hbs | 1 +
templates/default/prune-err-body.txt.hbs | 10 +
templates/default/prune-err-subject.txt.hbs | 1 +
templates/default/prune-ok-body.txt.hbs | 10 +
templates/default/prune-ok-subject.txt.hbs | 1 +
templates/default/sync-err-body.txt.hbs | 14 +
templates/default/sync-err-subject.txt.hbs | 5 +
templates/default/sync-ok-body.txt.hbs | 14 +
templates/default/sync-ok-subject.txt.hbs | 5 +
.../default/tape-backup-err-body.txt.hbs | 26 +
.../default/tape-backup-err-subject.txt.hbs | 5 +
templates/default/tape-backup-ok-body.txt.hbs | 27 +
.../default/tape-backup-ok-subject.txt.hbs | 5 +
templates/default/tape-load-body.txt.hbs | 15 +
templates/default/tape-load-subject.txt.hbs | 1 +
templates/default/test-body.html.hbs | 1 +
templates/default/test-body.txt.hbs | 1 +
templates/default/test-subject.txt.hbs | 1 +
templates/default/verify-err-body.txt.hbs | 14 +
templates/default/verify-err-subject.txt.hbs | 1 +
templates/default/verify-ok-body.txt.hbs | 10 +
templates/default/verify-ok-subject.txt.hbs | 1 +
www/Makefile | 2 +
www/NavigationTree.js | 6 +
www/OnlineHelpInfo.js | 24 +
www/Utils.js | 39 +
www/config/NotificationConfigView.js | 11 +
www/datastore/OptionView.js | 15 +
www/form/PermissionPathSelector.js | 1 +
www/tape/window/TapeBackup.js | 25 +
www/tape/window/TapeBackupJob.js | 33 +
www/tape/window/TapeRestore.js | 17 +
www/window/DataStoreEdit.js | 13 +
www/window/NotificationMatcherOverride.js | 1105 +++++++++++++++++
www/window/NotifyOptions.js | 44 +
92 files changed, 4158 insertions(+), 869 deletions(-)
create mode 100644 docs/config/notifications-priv/format.rst
create mode 100644 docs/config/notifications-priv/man5.rst
create mode 100644 docs/config/notifications/format.rst
create mode 100644 docs/config/notifications/man5.rst
create mode 100644 docs/notifications.rst
create mode 100644 pbs-config/src/notifications.rs
create mode 100644 src/api2/config/notifications/gotify.rs
create mode 100644 src/api2/config/notifications/matchers.rs
create mode 100644 src/api2/config/notifications/mod.rs
create mode 100644 src/api2/config/notifications/sendmail.rs
create mode 100644 src/api2/config/notifications/smtp.rs
create mode 100644 src/api2/config/notifications/targets.rs
create mode 100644 src/bin/proxmox_backup_manager/notifications/gotify.rs
create mode 100644 src/bin/proxmox_backup_manager/notifications/matchers.rs
create mode 100644 src/bin/proxmox_backup_manager/notifications/mod.rs
create mode 100644 src/bin/proxmox_backup_manager/notifications/sendmail.rs
create mode 100644 src/bin/proxmox_backup_manager/notifications/smtp.rs
create mode 100644 src/bin/proxmox_backup_manager/notifications/targets.rs
delete mode 100644 src/server/email_notifications.rs
create mode 100644 src/server/notifications.rs
create mode 100644 templates/Makefile
create mode 100644 templates/default/acme-err-body.txt.hbs
create mode 100644 templates/default/acme-err-subject.txt.hbs
create mode 100644 templates/default/gc-err-body.txt.hbs
create mode 100644 templates/default/gc-err-subject.txt.hbs
create mode 100644 templates/default/gc-ok-body.txt.hbs
create mode 100644 templates/default/gc-ok-subject.txt.hbs
create mode 100644 templates/default/package-updates-body.txt.hbs
create mode 100644 templates/default/package-updates-subject.txt.hbs
create mode 100644 templates/default/prune-err-body.txt.hbs
create mode 100644 templates/default/prune-err-subject.txt.hbs
create mode 100644 templates/default/prune-ok-body.txt.hbs
create mode 100644 templates/default/prune-ok-subject.txt.hbs
create mode 100644 templates/default/sync-err-body.txt.hbs
create mode 100644 templates/default/sync-err-subject.txt.hbs
create mode 100644 templates/default/sync-ok-body.txt.hbs
create mode 100644 templates/default/sync-ok-subject.txt.hbs
create mode 100644 templates/default/tape-backup-err-body.txt.hbs
create mode 100644 templates/default/tape-backup-err-subject.txt.hbs
create mode 100644 templates/default/tape-backup-ok-body.txt.hbs
create mode 100644 templates/default/tape-backup-ok-subject.txt.hbs
create mode 100644 templates/default/tape-load-body.txt.hbs
create mode 100644 templates/default/tape-load-subject.txt.hbs
create mode 100644 templates/default/test-body.html.hbs
create mode 100644 templates/default/test-body.txt.hbs
create mode 100644 templates/default/test-subject.txt.hbs
create mode 100644 templates/default/verify-err-body.txt.hbs
create mode 100644 templates/default/verify-err-subject.txt.hbs
create mode 100644 templates/default/verify-ok-body.txt.hbs
create mode 100644 templates/default/verify-ok-subject.txt.hbs
create mode 100644 www/config/NotificationConfigView.js
create mode 100644 www/window/NotificationMatcherOverride.js
Summary over all repositories:
104 files changed, 4236 insertions(+), 912 deletions(-)
--
Generated by git-murpp 0.7.1
More information about the pbs-devel
mailing list