[pve-devel] ZFS over LIO iscsi

[Gerhard W. Recher]

Mark,

I have a all NVMe 4 nodes cluster
rdma is still not working as expected, but tcp performance is great even
on bigger Block size(s)

rados bench -p rbd 60 write --no-cleanup -t 56 -b 256K -o 1M

Total time run:         60.008792
Total writes made:      633524
Write size:             262144
Object size:            1048576
Bandwidth (MB/sec):     2639.3
Stddev Bandwidth:       139.635
Max bandwidth (MB/sec): 2850.75
Min bandwidth (MB/sec): 2300.5
Average IOPS:           10557
Stddev IOPS:            558
Max IOPS:               11403
Min IOPS:               9202
Average Latency(s):     0.00530374
Stddev Latency(s):      0.00560477
Max latency(s):         0.113396
Min latency(s):         0.000747469

and 1M blocksize:
Total time run:         60.025703
Total writes made:      198400
Write size:             1048576
Object size:            1048576
Bandwidth (MB/sec):     3305.25
Stddev Bandwidth:       35.2218
Max bandwidth (MB/sec): 3379
Min bandwidth (MB/sec): 3229
Average IOPS:           3305
Stddev IOPS:            35
Max IOPS:               3379
Min IOPS:               3229
Average Latency(s):     0.0169412
Stddev Latency(s):      0.0272042
Max latency(s):         0.209305
Min latency(s):         0.00253389



Gerhard W. Recher

net4sec UG (haftungsbeschränkt)
Leitenweg 6
86929 Penzing

+49 171 4802507
Am 24.10.2017 um 16:31 schrieb Mark Adams:
> The last time I used ceph (admittedly a few years ago) I couldn't get it to
> go faster than about 30MB/s ... what sort of performance are you seeing
> with it now? I know this will be dependant on setup and workload, but if
> "all the best" hardware is used how fast can it go?
>
> Regards,
> Mark
>
> On 24 October 2017 at 15:28, Dietmar Maurer <dietmar at proxmox.com> wrote:
>
>>> The end goal for me here is a ZFS over iscsi setup with HA ZFS storage.
>> Do
>>> you have any other existing solution for this?
>> No - we always use ceph for such requirements ...
>>
> _______________________________________________
> pve-devel mailing list
> pve-devel at pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel