[PVE-User] zfs raidz2 expansion

[Randy Bush]

> There is no option to replicate a full ZFS pool to another.

not exactly what i want to do.  my bad in saying "full replication."
what i meant was all vms are replicated. on other nodes.

i was thinking that each node could have one pool for primary vm images
and a second to receive replication from other nodes.

> So, you have a current pool with one raidz2 VDEV made up of 4x 2TB
> disks.

yup


> Because if you have another set of 4x 2TB disks, you can just expand
> the pool with another raidz2 VDEV, without expanding the current VDEV
> you have.

yup.  what are the performance implications?

> If you add another VDEV, the pool could have the following layout:
> 
> pool
>   raidz2-0
>     disk0
>     disk1
>     disk2
>     disk3
>   raidz2-1
>     disk4
>     disk5
>     disk6
>     disk7

yup

> If you want to create a new pool, then things will be a bit more
> complicated, as you would need to create a new storage config for it
> as well, Move-Disk all the disks over to it. If you have a cluster and
> use the VM replication feature, that new pool must be present on the
> other nodes as well and you will have to remove the replication jobs
> before you move the disks to the new pool and then re-create them once
> all VM disks are on the new pool.

we would keep the nodes all symmetric, so that would not be an issue.
and it's just a few hours of ops pain to de-repl and re-repl.  but what
i do not see is how to tell `/etc/pve/storage.cfg` that pool0 is for
images and pool1 is for incoming replication.  maybe i am just trying to
do something too weird.

randy