[PVE-User] NFS-Performance

[Emmanuel Kasper]

On 07/19/2015 06:17 PM, Holger Hampel | RA Consulting wrote:
> Hello,
> 
> Last weekend, the backup stalled (also the backups before, but nobody was there to care for it):
> 
> 657: Jul 12 10:37:13 INFO: status: 77% (57894961152/75161927680), sparse 3% (2493112320), duration 14059, 15/15 MB/s 657: Jul 12 10:45:29 INFO: status: 78% (58630930432/75161927680), sparse 3% (2497314816), duration 14555, 1/1 MB/s 657: Jul 12 10:50:34 INFO: status: 79% (59411005440/75161927680), sparse 3% (2497314816), duration 14860, 2/2 MB/s 657: Jul 12 10:54:53 INFO: status: 80% (60162506752/75161927680), sparse 3% (2509971456), duration 15119, 2/2 MB/s 657: Jul 12 10:55:15 INFO: status: 81% (60890480640/75161927680), sparse 3% (2518409216), duration 15141, 33/32 MB/s 657: Jul 12 11:03:05 INFO: status: 82% (61646045184/75161927680), sparse 3% (2518618112), duration 15611, 1/1 MB/s 657: Jul 12 11:06:07 INFO: status: 83% (62387257344/75161927680), sparse 3% (2518822912), duration 15793, 4/4 MB/s 657: Jul 12 11:09:22 INFO: status: 84% (63160647680/75161927680), sparse 3% (2519183360), duration 15988, 3/3 MB/s 657: Jul 12 11:13:33 INFO: status: 85% (63918243840/75161927680), sparse
  

3% (2519449600), duration 16239, 3/3 MB/s 657: Jul 12 11:19:26 INFO:
status: 86% (64648904704/75161927680), sparse 3% (2524684288), duration
16592, 2/2 MB/s 657: Jul 12 11:23:00 INFO: status: 87%
(65413185536/75161927680), sparse 3% (2542366720), duration 16806, 3/3
MB/s 657: Jul 12 11:25:41 INFO: status: 88% (66164162560/75161927680),
sparse 3% (2704044032), duration 16967, 4/3 MB/s 657: Jul 12 11:28:49
INFO: status: 89% (66912124928/75161927680), sparse 3% (2730418176),
duration 17155, 3/3 MB/s 657: Jul 12 11:32:25 INFO: status: 90%
(67659366400/75161927680), sparse 3% (2752782336), duration 17371, 3/3
MB/s 657: Jul 12 11:34:40 INFO: status: 91% (68420632576/75161927680),
sparse 3% (2766213120), duration 17506, 5/5 MB/s 657: Jul 12 11:39:33
INFO: status: 92% (69151490048/75161927680), sparse 3% (2789863424),
duration 17799, 2/2 MB/s 657: Jul 12 11:42:29 INFO: status: 93%
(69910659072/75161927680), sparse 3% (2809716736), duration 17975, 4/4
MB/s 657: Jul 12 11:52:13 INFO: status: 94
% (70690799616/75161927680), sparse 3% (2822275072), duration 18559, 1/1
MB/s
> 
> After backup completion I tested the speed of NFS access (also using different blocksizes without siginificant changes):
> 
> root at gaia:~# dd if=/dev/sda of=/mnt/pve/Backup-Chaos/test bs=8192 count=10000
> 10000+0 Datensätze ein
> 10000+0 Datensätze aus
> 81920000 Bytes (82 MB) kopiert, 16,086 s, 3,1 MB/s
> 
> On another node (same hardware and config) I'm testing the 3.10 kernel:
> 
> root at tartaros:~# dd if=/dev/sda of=/mnt/pve/Backup-Chaos/test bs=8192 count=10000
> 10000+0 Datensätze ein
> 10000+0 Datensätze aus
> 81920000 Bytes (82 MB) kopiert, 0,950729 s, 86,2 MB/s
> 
> So I upgraded the kernel on the first node and:
> 
> root at nyx:~# dd if=/dev/sda of=/mnt/pve/Backup-Chaos/test bs=8192 count=10000
> 10000+0 Datensätze ein
> 10000+0 Datensätze aus
> 81920000 Bytes (82 MB) kopiert, 0,912615 s, 89,8 MB/s
> 
> 
> It seems independend of using IPv4/v6 and another machine (on pve-test, other hardware) without bonding has no problems accessing the same NFS-Server.
> 


Hi Holger
I just tested the speed of nfs writes from two nodes, one running with
2.6.32 and the other one with 3.10, and I don't see any differences with
bonnie++ here.

>From both I get a speed of 70 MB/s ( 72701 in output under) in
sequential writes.


2.6.32 nfs client:

root at pve3:/mnt/pve/nas-write# bonnie++ -u manu -d .
Version  1.96       ------Sequential Output------ --Sequential Input-
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
pve3             2G  2125  96 72701   2 42946   2 +++++ +++ 97566   4
2059  20

Emmanuel