[pve-devel] [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages
Filip Schauer
f.schauer at proxmox.com
Mon Nov 18 18:44:29 CET 2024
I tried importing the GNS3 OVA files found here:
https://www.gns3.com/software/download-vm
I got the same XPath error with the VirtualBox file.
The ESXi and Workstation & Fusion OVA files were imported successfully
and they boot aswell.
On 18/11/2024 18:14, Filip Schauer wrote:
> Hello,
>
> I tried to quickly test the OVA import with a CentOS 7 OVA file that I
> downloaded from:
> https://dlconusc1.linuxvmimages.com/046389e06777452db2ccf9a32efa3760:dldatac/VirtualBox/C/7/CentOS_7.9.2009_VBM.zip
>
>
> 6785cb04dc5e7ab5a28ec139f631dfc8aeb8a6278c5fc66e1e7030d3f5a83b18
> /var/lib/vz/import/CentOS_7.9.2009_VBM_LinuxVMImages.COM.ova
>
> After enabling the "import" content type on my local directory storage,
> the ova file shows up in the import section of the storage.
>
> Upon clicking "Import" in the web UI the "Import Guest" dialog opens
> along with an error message stating:
> XPath error : Undefined namespace prefix at
> /usr/share/perl5/PVE/GuestImport/OVF.pm line 239. (500)
>
> Closing the error message also closes the import dialog, preventing me
> from importing.
>
> The same thing happens with this ArcoLinux OVA file:
> https://master.dl.sourceforge.net/project/linuxovafiles/GUI/ArcoLinux-vm.ova?viasf=1
> to /var/lib/vz/import/ArcoLinux-vm.ova
>
> 2b353ac59598bac24d4cf0721e367935d00178a31d84e09c94425bdee3dab6a1
> /var/lib/vz/import/ArcoLinux-vm.ova
>
> Tested-by: Filip Schauer <f.schauer at proxmox.com>
>
> On 18/11/2024 16:29, Dominik Csapak wrote:
>> This series enables importing ova/ovf from directory based storages,
>> inclusive upload/download via the webui (ova only).
>>
>> It also improves the ovf importer by parsing the ostype, nics, bootorder
>> (and firmware from vmware exported files).
>>
>> We now extract the images into either a given target storage or in the
>> import storage in the 'images' dir so accidentally left over images
>> are discoverable by the ui/cli.
>>
>> This version includes a modified qemu-server patch of fabians
>> hardening series:
>> https://lore.proxmox.com/pve-devel/20241104104221.228730-1-f.gruenbichler@proxmox.com/
>>
>>
>> I sent the qemu-server patch from fabian again but omitted some
>> problematic checks. I add them later with a check
>> against the import vtype again (last patch in qemu-server)
>>
>> changes from v6:
>> * rebase on master (omit already applied patches)
>> * style/indentation fixes
>> * omit explicit check for .ova in upload path
>> * use assert_file_validity check for download_file_from_url
>> * only warn on whitespace in parent image (file_size_info) to
>> preserve backward compatibility
>> for possible existing cases
>> * fix cleanup on live-import (added wrong variable to cleanup list)
>> * renamed 'import-extraction-storage' to 'import-working-storage'
>> * mad priv check for extraction storage simpler (combine branches)
>> * reworded some gettexts
>>
>> changes from v5:
>> * removed leftover hunks in makefile
>> * moved ova checks to correct patch
>> * split up error messages for unexpected format
>> * remove unnecessary untaint
>> * reword error message
>> * reintroduce symlink check in ova/ovf check
>> * added sanity check for ovas after uploading/downloading
>> * added new patch for checking import vtypes
>> * fixed issue with files with absolute path
>>
>> changes from v4:
>> * rebased on master/fabians series
>> * added the file_size_info check for untrusted images after extracting
>>
>> changes from v3:
>> * fixed dependencies in control file
>> * removed unnecessary use statements
>> * removed unnecessary remove helper
>> * moved 'needs_extract' helper to qemu-server
>> * removed import storage param from PUT call
>> * check down/uploaded ova filename more strictly (same as listing)
>> * improved filepath checking in ovf
>> * forbid importing when extracted image references a base/backing file
>> * instead of trying to manually create a proper filename, use 'alloc' to
>> create a small (1M) file with the same format and overwrite it with
>> renaming. this also solves the cluster locking issue
>> * prefer using PVE::Storage functions instead of plugin methods in
>> ova extraction code
>> * use $vollist for cleaning up extracted images in qemu-server and
>> add manual cleanup for the success case
>>
>> changes from v2:
>> * use better 'format' values for embedded images (e.g. ova+vmdk)
>> * use this format to decide if images should be extracted
>> * consistent use of the 'safe character' classes when listing
>> and parsing
>> * also list vmdk/qcow2/raw images in content listing
>> (this will be useful when we have a gui for the 'import-from'
>> in the wizard/disk edit for vms)
>> * a few gui adaptions
>>
>>
>> changes from v1:
>> * move ovf code to GuestImport
>> * move extract/checking code to GuestImport
>> * don't return 'image' types from import volumes
>> * use allow 'safe' characters for filenames of ova/ovfs and inside
>> * check for non-regular files (e.g. symlinks) after extraction
>> * add new 'import-extraction-storage' for import
>> * rename panel in gui for directory storages
>> * typo fixes
>> * and probably more, see the individual patches for details
>>
>> pve-storage:
>>
>> Dominik Csapak (11):
>> plugin: dir: implement import content type
>> plugin: dir: handle ova files for import
>> ovf: improve and simplify path checking code
>> ovf: implement parsing the ostype
>> ovf: implement parsing out firmware type
>> ovf: implement rudimentary boot order
>> ovf: implement parsing nics
>> api: allow ova upload/download
>> plugin: enable import for nfs/btrfs/cifs/cephfs/glusterfs
>> add 'import' content type to 'check_volume_access'
>> plugin: file_size_info: warn on parent images with unusual path
>>
>> src/PVE/API2/Storage/Status.pm | 56 +++++-
>> src/PVE/GuestImport.pm | 79 ++++++++
>> src/PVE/GuestImport/OVF.pm | 187 ++++++++++++++++--
>> src/PVE/Makefile | 1 +
>> src/PVE/Storage.pm | 23 ++-
>> src/PVE/Storage/BTRFSPlugin.pm | 5 +
>> src/PVE/Storage/CIFSPlugin.pm | 6 +-
>> src/PVE/Storage/CephFSPlugin.pm | 6 +-
>> src/PVE/Storage/DirPlugin.pm | 52 ++++-
>> src/PVE/Storage/GlusterfsPlugin.pm | 6 +-
>> src/PVE/Storage/NFSPlugin.pm | 6 +-
>> src/PVE/Storage/Plugin.pm | 16 +-
>> .../ovf_manifests/Win10-Liz_no_default_ns.ovf | 1 +
>> src/test/parse_volname_test.pm | 33 ++++
>> src/test/path_to_volume_id_test.pm | 21 ++
>> src/test/run_ovf_tests.pl | 14 ++
>> 16 files changed, 480 insertions(+), 32 deletions(-)
>> create mode 100644 src/PVE/GuestImport.pm
>>
>> qemu-server:
>>
>> Dominik Csapak (4):
>> use OVF from Storage
>> api: create: implement extracting disks when needed for import-from
>> api: create: add 'import-working-storage' parameter
>> api: check untrusted image files for import content type
>>
>> Fabian Grünbichler (1):
>> disk import: add additional safeguards for imported image files
>>
>> PVE/API2/Qemu.pm | 108 ++++++--
>> PVE/CLI/qm.pm | 4 +-
>> PVE/QemuServer.pm | 12 +
>> PVE/QemuServer/Helpers.pm | 5 +
>> PVE/QemuServer/Makefile | 1 -
>> PVE/QemuServer/OVF.pm | 242 ------------------
>> debian/control | 2 -
>> test/Makefile | 5 +-
>> test/ovf_manifests/Win10-Liz-disk1.vmdk | Bin 65536 -> 0 bytes
>> test/ovf_manifests/Win10-Liz.ovf | 142 ----------
>> .../ovf_manifests/Win10-Liz_no_default_ns.ovf | 142 ----------
>> test/ovf_manifests/Win_2008_R2_two-disks.ovf | 145 -----------
>> test/ovf_manifests/disk1.vmdk | Bin 65536 -> 0 bytes
>> test/ovf_manifests/disk2.vmdk | Bin 65536 -> 0 bytes
>> test/run_ovf_tests.pl | 71 -----
>> 15 files changed, 112 insertions(+), 767 deletions(-)
>> delete mode 100644 PVE/QemuServer/OVF.pm
>> delete mode 100644 test/ovf_manifests/Win10-Liz-disk1.vmdk
>> delete mode 100755 test/ovf_manifests/Win10-Liz.ovf
>> delete mode 100755 test/ovf_manifests/Win10-Liz_no_default_ns.ovf
>> delete mode 100755 test/ovf_manifests/Win_2008_R2_two-disks.ovf
>> delete mode 100644 test/ovf_manifests/disk1.vmdk
>> delete mode 100644 test/ovf_manifests/disk2.vmdk
>> delete mode 100755 test/run_ovf_tests.pl
>>
>> pve-manager:
>>
>> Dominik Csapak (9):
>> ui: guest import: add ova-needs-extracting warning text
>> ui: enable import content type for relevant storages
>> ui: enable upload/download/remove buttons for 'import' type storages
>> ui: disable 'import' button for non importable formats
>> ui: import: improve rendering of volume names
>> ui: guest import: add storage selector for ova extraction storage
>> ui: guest import: change icon/text for non-esxi import storage
>> ui: import: show size for dir-based storages
>> ui: import: adapt live import help text to ova
>>
>> www/manager6/Utils.js | 9 +++++++-
>> www/manager6/form/ContentTypeSelector.js | 2 +-
>> www/manager6/storage/Browser.js | 25 ++++++++++++++++-----
>> www/manager6/storage/CephFSEdit.js | 2 +-
>> www/manager6/storage/GlusterFsEdit.js | 2 +-
>> www/manager6/window/GuestImport.js | 28 +++++++++++++++++++++++-
>> www/manager6/window/UploadToStorage.js | 1 +
>> 7 files changed, 58 insertions(+), 11 deletions(-)
>>
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel at lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
More information about the pve-devel
mailing list