[pve-devel] [PATCH v2 pve-manager 00/22] storage replication - second try v2

[Dietmar Maurer]

The first series from Wolfgang introduced circular package dependency. So
I moved most of the code to package pve-manager.

Notable changes:

- use a separate replication config file: /etc/pve/replication.cfg
- allow more than one job per guest (as long as targets are different).
- do not add ssh calls to the storage API
- add regression tests
- use systemd like calendar events ('schedule') instead of a simlple integer 'interval'
- more flexible disk handling (add disks is now possible).
- reduce number of ssh invocation on guests with more than one disk
- improved command line tool 'pvesr' can now edit/change configuration
- pvesr status: show additional values like 'NextSync' and 'Duration' of last sync
- save pid in state, so that we can check if replication is still active/running

Please note that actual volume replication is not implemented, but you can run
some interesting regression tests (replication-test5.pl). I just wanted to send
what I have so far because the series already contain 18 patches ...

Changes in v2:
  add cleanups requested by fabian
  implement replicate_volume()
  aquire guest_migration_lock during replication
  PVE::API2Tools::resolve_proxyto - new helper
  PVE::API2::ReplicationConfig - implement proxyto_callback for delete


Dietmar Maurer (22):
  pvesr: add pve storage replication tool
  add regression test environment for replication
  replication_test2.pl: test replication scheduler
  PVE/Replication.pm: save pid/ptime to running job state
  PVE::Replication - use new calendar events instead of interval
  replication_test3.pl: regression test for replication to same node
  ReplicationTestEnv.pm: add job tracking log
  replication_test4.pl: Test replication job failure
  pvesr run: add --verbose flag
  pvesr prepare-local-job: new helper
  ReplicationTestEnv.pm: mock storage content methods
  ReplicationTestEnv.pm: mock get_ssh_info and ssh_info_to_command
  pvesr finalize-local-job: add helper to cleanup job
  PVE::Replication::remote_prepare_local_job - new helper
  PVE::Replication::remote_finalize_local_job - new helper
  PVE::Replication::replicate - implement replicate
  bin/test/Makefile: run all replication tests
  ReplicationTestEnv.pm: avoid warning about undefined value
  PVE::Replication - implement replicate_volume()
  PVE::Replication - aquire guest_migration_lock during replication
  PVE::API2Tools::resolve_proxyto - new helper
  PVE::API2::ReplicationConfig - implement proxyto_callback for delete

 Makefile                       |   1 +
 PVE/API2/Cluster.pm            |   7 +
 PVE/API2/Makefile              |   2 +
 PVE/API2/Nodes.pm              |   7 +
 PVE/API2/Replication.pm        |  99 +++++++++
 PVE/API2/ReplicationConfig.pm  | 246 +++++++++++++++++++++
 PVE/API2Tools.pm               |  18 ++
 PVE/CLI/Makefile               |   2 +-
 PVE/CLI/pvesr.pm               | 343 +++++++++++++++++++++++++++++
 PVE/HTTPServer.pm              |   9 +-
 PVE/Makefile                   |   1 +
 PVE/Replication.pm             | 483 +++++++++++++++++++++++++++++++++++++++++
 bin/Makefile                   |   2 +-
 bin/pvesh                      |   8 +-
 bin/pvesr                      |   8 +
 bin/test/Makefile              |   6 +
 bin/test/ReplicationTestEnv.pm | 328 ++++++++++++++++++++++++++++
 bin/test/replication_test1.pl  |  47 ++++
 bin/test/replication_test2.pl  | 102 +++++++++
 bin/test/replication_test3.pl  |  51 +++++
 bin/test/replication_test4.log |  13 ++
 bin/test/replication_test4.pl  |  68 ++++++
 bin/test/replication_test5.log |  50 +++++
 bin/test/replication_test5.pl  | 128 +++++++++++
 24 files changed, 2018 insertions(+), 11 deletions(-)
 create mode 100644 PVE/API2/Replication.pm
 create mode 100644 PVE/API2/ReplicationConfig.pm
 create mode 100644 PVE/CLI/pvesr.pm
 create mode 100644 PVE/Replication.pm
 create mode 100644 bin/pvesr
 create mode 100755 bin/test/ReplicationTestEnv.pm
 create mode 100755 bin/test/replication_test1.pl
 create mode 100755 bin/test/replication_test2.pl
 create mode 100755 bin/test/replication_test3.pl
 create mode 100644 bin/test/replication_test4.log
 create mode 100755 bin/test/replication_test4.pl
 create mode 100644 bin/test/replication_test5.log
 create mode 100755 bin/test/replication_test5.pl

-- 
2.11.0