[pve-devel] [PATCH storage/qemu-server/pve-manager] implement ova/ovf import for directory type storages

[Dominik Csapak]

This series enables importing ova/ovf from directory based storages,
inclusive upload/download via the webui (ova only).

It also improves the ovf importer by parsing the ostype, nics, bootorder
(and firmware from vmware exported files).

I currently opted to move the OVF.pm to pve-storage, since there is no
real other place where we could put it. Building a seperate package
from qemu-servers git repo would also not be ideal, since we still
have a cyclic dev dependency then
(If someone has a better idea how to handle that, please do tell, and
i can do that in a v2)

There are surely some wrinkles left i did not think of, but all in all,
it should be pretty usable. E.g. i downloaded some ovas, uploaded them
on my cephfs in my virtual cluster, and successfully imported that with
live-import.

The biggest caveat when importing from ovas is that we have to
temporarily extract the disk images. I opted for doing that into the
import storage, but if we have a better idea where to put that, i can
implement it in a v2 (or as a follow up). For example, we could add a
new 'tmpdir' parameter to the create call and use that for extractig.

pve-storage:

Dominik Csapak (9):
  copy OVF.pm from qemu-server
  plugin: dir: implement import content type
  plugin: dir: handle ova files for import
  ovf: implement parsing the ostype
  ovf: implement parsing out firmware type
  ovf: implement rudimentary boot order
  ovf: implement parsing nics
  api: allow ova upload/download
  plugin: enable import for nfs/btfs/cifs/cephfs

 src/PVE/API2/Storage/Status.pm                |  15 +-
 src/PVE/Storage.pm                            |  78 +++-
 src/PVE/Storage/BTRFSPlugin.pm                |   5 +
 src/PVE/Storage/CIFSPlugin.pm                 |   6 +-
 src/PVE/Storage/CephFSPlugin.pm               |   6 +-
 src/PVE/Storage/DirPlugin.pm                  |  53 ++-
 src/PVE/Storage/Makefile                      |   1 +
 src/PVE/Storage/NFSPlugin.pm                  |   6 +-
 src/PVE/Storage/OVF.pm                        | 381 ++++++++++++++++++
 src/PVE/Storage/Plugin.pm                     |  23 +-
 src/test/Makefile                             |   5 +-
 src/test/ovf_manifests/Win10-Liz-disk1.vmdk   | Bin 0 -> 65536 bytes
 src/test/ovf_manifests/Win10-Liz.ovf          | 142 +++++++
 .../ovf_manifests/Win10-Liz_no_default_ns.ovf | 143 +++++++
 .../ovf_manifests/Win_2008_R2_two-disks.ovf   | 145 +++++++
 src/test/ovf_manifests/disk1.vmdk             | Bin 0 -> 65536 bytes
 src/test/ovf_manifests/disk2.vmdk             | Bin 0 -> 65536 bytes
 src/test/parse_volname_test.pm                |  13 +
 src/test/path_to_volume_id_test.pm            |  16 +
 src/test/run_ovf_tests.pl                     |  83 ++++
 20 files changed, 1112 insertions(+), 9 deletions(-)
 create mode 100644 src/PVE/Storage/OVF.pm
 create mode 100644 src/test/ovf_manifests/Win10-Liz-disk1.vmdk
 create mode 100755 src/test/ovf_manifests/Win10-Liz.ovf
 create mode 100755 src/test/ovf_manifests/Win10-Liz_no_default_ns.ovf
 create mode 100755 src/test/ovf_manifests/Win_2008_R2_two-disks.ovf
 create mode 100644 src/test/ovf_manifests/disk1.vmdk
 create mode 100644 src/test/ovf_manifests/disk2.vmdk
 create mode 100755 src/test/run_ovf_tests.pl

qemu-server:

Dominik Csapak (3):
  api: delete unused OVF.pm
  use OVF from Storage
  api: create: implement extracting disks when needed for import-from

 PVE/API2/Qemu.pm                              |  26 +-
 PVE/API2/Qemu/Makefile                        |   2 +-
 PVE/API2/Qemu/OVF.pm                          |  53 ----
 PVE/CLI/qm.pm                                 |   4 +-
 PVE/QemuServer.pm                             |   5 +-
 PVE/QemuServer/Helpers.pm                     |   9 +
 PVE/QemuServer/Makefile                       |   1 -
 PVE/QemuServer/OVF.pm                         | 242 ------------------
 test/Makefile                                 |   5 +-
 test/ovf_manifests/Win10-Liz-disk1.vmdk       | Bin 65536 -> 0 bytes
 test/ovf_manifests/Win10-Liz.ovf              | 142 ----------
 .../ovf_manifests/Win10-Liz_no_default_ns.ovf | 142 ----------
 test/ovf_manifests/Win_2008_R2_two-disks.ovf  | 145 -----------
 test/ovf_manifests/disk1.vmdk                 | Bin 65536 -> 0 bytes
 test/ovf_manifests/disk2.vmdk                 | Bin 65536 -> 0 bytes
 test/run_ovf_tests.pl                         |  71 -----
 16 files changed, 37 insertions(+), 810 deletions(-)
 delete mode 100644 PVE/API2/Qemu/OVF.pm
 delete mode 100644 PVE/QemuServer/OVF.pm
 delete mode 100644 test/ovf_manifests/Win10-Liz-disk1.vmdk
 delete mode 100755 test/ovf_manifests/Win10-Liz.ovf
 delete mode 100755 test/ovf_manifests/Win10-Liz_no_default_ns.ovf
 delete mode 100755 test/ovf_manifests/Win_2008_R2_two-disks.ovf
 delete mode 100644 test/ovf_manifests/disk1.vmdk
 delete mode 100644 test/ovf_manifests/disk2.vmdk
 delete mode 100755 test/run_ovf_tests.pl

pve-manager:

Dominik Csapak (4):
  ui: fix special 'import' icon for non-esxi storages
  ui: guest import: add ova-needs-extracting warning text
  ui: enable import content type for relevant storages
  ui: enable upload/download buttons for 'import' type storages

 www/manager6/Utils.js                    | 3 ++-
 www/manager6/form/ContentTypeSelector.js | 2 +-
 www/manager6/storage/Browser.js          | 7 ++++++-
 www/manager6/storage/CephFSEdit.js       | 2 +-
 www/manager6/window/GuestImport.js       | 1 +
 www/manager6/window/UploadToStorage.js   | 1 +
 6 files changed, 12 insertions(+), 4 deletions(-)

-- 
2.39.2