[pve-devel] applied: [PATCH v3 pve-manager 00/23] storage replication - second try v3

[Dietmar Maurer]

applied, so that we can start work on the GUI

> On May 30, 2017 at 3:19 PM Dietmar Maurer <dietmar at proxmox.com> wrote:
> 
> 
> The first series from Wolfgang introduced circular package dependency. So
> I moved most of the code to package pve-manager.
> 
> Notable changes:
> 
> - use a separate replication config file: /etc/pve/replication.cfg
> - allow more than one job per guest (as long as targets are different).
> - do not add ssh calls to the storage API
> - add regression tests
> - use systemd like calendar events ('schedule') instead of a simlple integer
> 'interval'
> - more flexible disk handling (add disks is now possible).
> - reduce number of ssh invocation on guests with more than one disk
> - improved command line tool 'pvesr' can now edit/change configuration
> - pvesr status: show additional values like 'NextSync' and 'Duration' of last
> sync
> - save pid in state, so that we can check if replication is still
> active/running
> 
> Please note that actual volume replication is not implemented, but you can run
> some interesting regression tests (replication-test5.pl). I just wanted to
> send
> what I have so far because the series already contain 18 patches ...
> 
> Changes in v2:
> -  add cleanups requested by fabian
> -  implement replicate_volume()
> -  aquire guest_migration_lock during replication
> -  PVE::API2Tools::resolve_proxyto - new helper
> -  PVE::API2::ReplicationConfig - implement proxyto_callback for delete
> 
> Changes in v3:
> - do not use proxyto_callback for delete. Instead, use the new remove_job
> property to mark a job for removal.
> - implement delete
> - new regression tests for delete
> - use new GuestHelpers class
> - minor cleanups
> 
> 
> Dietmar Maurer (23):
>   pvesr: add pve storage replication tool
>   add regression test environment for replication
>   replication_test2.pl: test replication scheduler
>   PVE/Replication.pm: save pid/ptime to running job state
>   PVE::Replication - use new calendar events instead of interval
>   replication_test3.pl: regression test for replication to same node
>   ReplicationTestEnv.pm: add job tracking log
>   replication_test4.pl: Test replication job failure
>   pvesr run: add --verbose flag
>   pvesr prepare-local-job: new helper
>   ReplicationTestEnv.pm: mock storage content methods
>   ReplicationTestEnv.pm: mock get_ssh_info and ssh_info_to_command
>   pvesr finalize-local-job: add helper to cleanup job
>   PVE::Replication::remote_prepare_local_job - new helper
>   PVE::Replication::remote_finalize_local_job - new helper
>   PVE::Replication::replicate - implement replicate
>   bin/test/Makefile: run all replication tests
>   ReplicationTestEnv.pm: avoid warning about undefined value
>   PVE::Replication - implement replicate_volume()
>   PVE::Replication - aquire guest_migration_lock during replication
>   PVE::API2Tools::resolve_proxyto - new helper
>   PVE::API2::ReplicationConfig - implement delete
>   replication_test5.pl: add test for job removal
> 
>  Makefile                       |   1 +
>  PVE/API2/Cluster.pm            |   7 +
>  PVE/API2/Makefile              |   2 +
>  PVE/API2/Nodes.pm              |   7 +
>  PVE/API2/Replication.pm        |  99 ++++++++
>  PVE/API2/ReplicationConfig.pm  | 217 +++++++++++++++++
>  PVE/API2Tools.pm               |  18 ++
>  PVE/CLI/Makefile               |   2 +-
>  PVE/CLI/pvesr.pm               | 350 +++++++++++++++++++++++++++
>  PVE/HTTPServer.pm              |   9 +-
>  PVE/Makefile                   |   1 +
>  PVE/Replication.pm             | 526
> +++++++++++++++++++++++++++++++++++++++++
>  bin/Makefile                   |   2 +-
>  bin/pvesh                      |   8 +-
>  bin/pvesr                      |   8 +
>  bin/test/Makefile              |   6 +
>  bin/test/ReplicationTestEnv.pm | 328 +++++++++++++++++++++++++
>  bin/test/replication_test1.pl  |  47 ++++
>  bin/test/replication_test2.pl  | 102 ++++++++
>  bin/test/replication_test3.pl  |  51 ++++
>  bin/test/replication_test4.log |  13 +
>  bin/test/replication_test4.pl  |  68 ++++++
>  bin/test/replication_test5.log |  59 +++++
>  bin/test/replication_test5.pl  | 149 ++++++++++++
>  24 files changed, 2069 insertions(+), 11 deletions(-)
>  create mode 100644 PVE/API2/Replication.pm
>  create mode 100644 PVE/API2/ReplicationConfig.pm
>  create mode 100644 PVE/CLI/pvesr.pm
>  create mode 100644 PVE/Replication.pm
>  create mode 100644 bin/pvesr
>  create mode 100755 bin/test/ReplicationTestEnv.pm
>  create mode 100755 bin/test/replication_test1.pl
>  create mode 100755 bin/test/replication_test2.pl
>  create mode 100755 bin/test/replication_test3.pl
>  create mode 100644 bin/test/replication_test4.log
>  create mode 100755 bin/test/replication_test4.pl
>  create mode 100644 bin/test/replication_test5.log
>  create mode 100755 bin/test/replication_test5.pl
> 
> -- 
> 2.11.0
> 
>