[pve-devel] [PATCH v3 00/30] add automated/unattended installation
Aaron Lauterer
a.lauterer at proxmox.com
Thu Mar 28 14:49:58 CET 2024
This patch series adds the possibility to do an automated / unattended
installation of Proxmox VE.
The overall idea is that we will have a dedicated ISO for the unattended
installation. It should be configured in such a way that it will start
the installation without any user interaction.
Though the integration in the installation environmend isn't part of
this patch series.
The information for the installer that is usually gathered interactively
from the user is provided via an `answer.toml` file.
The answer file allows to select disks and the network card via filters.
The installer also allows to run custom commands pre and post
installation. This should give users plenty of possibilities to either
further customize/prepare the installation or integrate it into a larger
automated installation setup.
For example, one could issue HTTP requests to signal the status and
progress of the installation.
When the installer is called with 'proxauto' in the kernel cmdline, the
'proxmox-fetch-answer' binary is called. It tries to find the answer
file and once found, will start the 'proxmox-auto-installer' binary and
pass the contents to it via stdin.
The auto-installer then parses the answer file and determines what
parameters need to be passed to the low-level installer. For example,
which disks and NIC to use, network IP settings and so forth.
The current status reporting of the actual installation is kept rather
simple.
Both binaries log into the tmp directory.
There is a third binary, the 'proxmox-autoinst-helper'. It provides a
few subcommands, from the help:
answer Validate if an answer file is formatted correctly
device-match Test which devices the given filter matches against
device-info Show device information that can be used for filters
identifiers Show identifiers for the current machine. This information is part of the POST request to fetch an answer file
The fetch-answer binary is trying to get an answer file. It does so by
first searching for a partition/FS labeled `proxmoxinst`, or all upper
case, and an `answer.toml` in there. This could be provided by another
USB flash drive.
If that is not successful, the next step is to send an HTTP POST request
to a URL to get the TOML contents in return. A POST request was chosen
because we also send information to identify the host in JSON format.
The question then is, where to get that URL from. Right now, there are
two options implemented. The first is looking for a custom DHCP option
and the second is querying for a TXT record in the `proxmoxinst`
subdomain of the search domain.
It is possible to provide a SHA256 fingerprint of the SSL cert used by
the answer server. The safest option is to place a
`cert_fingerprint.txt` file in the same `proxmoxinst` partition as where
you alternatively would place the `answer.toml`.
If that is not found, then it can be provided by a second custom DHCP
option or placed as TXT record in the subdomain `proxmoxinst-fp`.
This patch series now also separates the 3 binaries into their own
crate. The 'proxmox-fetch-answer' to keep the OpenSSL dependency as
localized as possible, and the 'proxmox-autoinst-helper' to make it easy
to compile just that binary.
The new `proxmox-chroot` utility helps to prepare everything to chroot
into a fresh installation and clean it up once done.
This will be useful in the post commands when further customizing the
installation.
Other plans / ideas for the future:
* add option to define remote SSH access (password and,or public key).
This could make remote debugging in case of problems easier
Regarding the patch series itself:
01-03 are needed to move some code into the common crate and
make structs/functions already in the common crate accessible.
I did split up the individual parts of the auto installer into their own
patches as much as possible, and (hopefully) in the order they depend on
each other.
Patches after the `unconfigured` one (16), switch the pattern matching
to the glob crate, add the helper tool and the fetching via HTTP.
Patch 25 factors our the binaries into their own crates.
Patches 26-30 are for the 'proxmox-chroot' utility and preparations for
it to work.
Areas that can be improved/extended:
* Testing possibility integrated in the Makefile
I did test it with all 3 installers, PVE, PMG and PBS and it worked.
WIP: writing documentation for it. Currently intended for the Wiki
as it can be used for all 3 products.
Changes since v2:
* don't use 'dmidecode' but check in the source locations directly for
identifiers
* fixed makefile to get builds working every time
* reworked filesystem and raid level layout in the answer file
definition
* factor out binaries into their own crates
* add 'proxmox-chroot' helper tool
Changes since v1:
* incorporated feedback (thx @cheiss)
* the proxmox-installer-filter tool got renamed and extended to the
proxmox-autoinst-helper
* added HTTP fetch functionality
* moved the fetch_plugins/utils.rs into its own subdirectory as it grew
quite a bit
* added builddeps
[0] https://lists.proxmox.com/pipermail/pve-devel/2023-September/059020.html
Aaron Lauterer (30):
tui: common: move InstallConfig struct to common crate
common: make InstallZfsOption members public
common: tui: use BTreeMap for predictable ordering
common: utils: add deserializer for CidrAddress
common: options: add Deserialize trait
low-level: add dump-udev command
add auto-installer crate
auto-installer: add dependencies
auto-installer: add answer file definition
auto-installer: add struct to hold udev info
auto-installer: add utils
auto-installer: add simple logging
auto-installer: add tests for answer file parsing
auto-installer: add auto-installer binary
auto-installer: add fetch answer binary
unconfigured: add proxauto as option to start auto installer
auto-installer: use glob crate for pattern matching
auto-installer: utils: make get_udev_index functions public
auto-installer: add proxmox-autoinst-helper tool
auto-installer: fetch: add gathering of system identifiers and
restructure code
auto-installer: helper: add subcommand to view indentifiers
auto-installer: fetch: add http post utility module
auto-installer: fetch: add http plugin to fetch answer
control: update build depends for auto installer
auto installer: factor out fetch-answer and autoinst-helper
low-level: write low level config to /tmp
common: add deserializer for FsType
common: skip target_hd when deserializing InstallConfig
common: add Display trait to ProxmoxProduct
add proxmox-chroot utility
Cargo.toml | 4 +
Makefile | 19 +-
Proxmox/Makefile | 1 +
Proxmox/Sys/Udev.pm | 54 +++
debian/control | 10 +
proxmox-auto-installer/Cargo.toml | 20 +
proxmox-auto-installer/src/answer.rs | 257 +++++++++++
.../src/bin/proxmox-auto-installer.rs | 193 ++++++++
proxmox-auto-installer/src/lib.rs | 5 +
proxmox-auto-installer/src/log.rs | 38 ++
proxmox-auto-installer/src/sysinfo.rs | 81 ++++
proxmox-auto-installer/src/udevinfo.rs | 9 +
proxmox-auto-installer/src/utils.rs | 430 ++++++++++++++++++
proxmox-auto-installer/tests/parse-answer.rs | 102 +++++
.../tests/resources/iso-info.json | 1 +
.../tests/resources/locales.json | 1 +
.../resources/parse_answer/disk_match.json | 29 ++
.../resources/parse_answer/disk_match.toml | 17 +
.../parse_answer/disk_match_all.json | 26 ++
.../parse_answer/disk_match_all.toml | 17 +
.../parse_answer/disk_match_any.json | 33 ++
.../parse_answer/disk_match_any.toml | 17 +
.../tests/resources/parse_answer/minimal.json | 17 +
.../tests/resources/parse_answer/minimal.toml | 14 +
.../resources/parse_answer/nic_matching.json | 17 +
.../resources/parse_answer/nic_matching.toml | 19 +
.../tests/resources/parse_answer/readme | 4 +
.../resources/parse_answer/specific_nic.json | 17 +
.../resources/parse_answer/specific_nic.toml | 19 +
.../tests/resources/parse_answer/zfs.json | 27 ++
.../tests/resources/parse_answer/zfs.toml | 20 +
.../tests/resources/run-env-info.json | 1 +
.../tests/resources/run-env-udev.json | 1 +
proxmox-autoinst-helper/Cargo.toml | 21 +
proxmox-autoinst-helper/src/main.rs | 337 ++++++++++++++
proxmox-chroot/Cargo.toml | 16 +
proxmox-chroot/src/main.rs | 353 ++++++++++++++
proxmox-fetch-answer/Cargo.toml | 22 +
.../src/fetch_plugins/http.rs | 191 ++++++++
proxmox-fetch-answer/src/fetch_plugins/mod.rs | 3 +
.../src/fetch_plugins/partition.rs | 32 ++
.../src/fetch_plugins/utils/mod.rs | 86 ++++
.../src/fetch_plugins/utils/post.rs | 94 ++++
proxmox-fetch-answer/src/main.rs | 76 ++++
proxmox-installer-common/Cargo.toml | 1 +
proxmox-installer-common/src/options.rs | 21 +-
proxmox-installer-common/src/setup.rs | 154 ++++++-
proxmox-installer-common/src/utils.rs | 11 +
proxmox-low-level-installer | 14 +
proxmox-tui-installer/src/options.rs | 4 +-
proxmox-tui-installer/src/setup.rs | 100 +---
.../src/views/install_progress.rs | 4 +-
unconfigured.sh | 17 +
53 files changed, 2959 insertions(+), 118 deletions(-)
create mode 100644 Proxmox/Sys/Udev.pm
create mode 100644 proxmox-auto-installer/Cargo.toml
create mode 100644 proxmox-auto-installer/src/answer.rs
create mode 100644 proxmox-auto-installer/src/bin/proxmox-auto-installer.rs
create mode 100644 proxmox-auto-installer/src/lib.rs
create mode 100644 proxmox-auto-installer/src/log.rs
create mode 100644 proxmox-auto-installer/src/sysinfo.rs
create mode 100644 proxmox-auto-installer/src/udevinfo.rs
create mode 100644 proxmox-auto-installer/src/utils.rs
create mode 100644 proxmox-auto-installer/tests/parse-answer.rs
create mode 100644 proxmox-auto-installer/tests/resources/iso-info.json
create mode 100644 proxmox-auto-installer/tests/resources/locales.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_all.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/disk_match_any.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/minimal.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/nic_matching.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/readme
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/specific_nic.toml
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.json
create mode 100644 proxmox-auto-installer/tests/resources/parse_answer/zfs.toml
create mode 100644 proxmox-auto-installer/tests/resources/run-env-info.json
create mode 100644 proxmox-auto-installer/tests/resources/run-env-udev.json
create mode 100644 proxmox-autoinst-helper/Cargo.toml
create mode 100644 proxmox-autoinst-helper/src/main.rs
create mode 100644 proxmox-chroot/Cargo.toml
create mode 100644 proxmox-chroot/src/main.rs
create mode 100644 proxmox-fetch-answer/Cargo.toml
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/http.rs
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/mod.rs
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/partition.rs
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/utils/mod.rs
create mode 100644 proxmox-fetch-answer/src/fetch_plugins/utils/post.rs
create mode 100644 proxmox-fetch-answer/src/main.rs
--
2.39.2
More information about the pve-devel
mailing list