[PVE-User] NFS-Performance

Tue Jul 21 09:28:16 CEST 2015

Hello Emmanuel,

Thank you for your check. Yes, I also have no trouble on two other systems. So I wanted to hear of similar problems to get an idea, what driver or config parameter is responsible. So it is hard to find (but we have no presure, as we will not start using openVZ on the new machines). I verified the problem also on the third node: same result.
If nobody else has this trooble: fine. But as long as I cannot understand it, there is a fear, it might return in some future kernel.

Regards
Holger Hampel

-----Ursprüngliche Nachricht-----
Von: pve-user [mailto:pve-user-bounces at pve.proxmox.com] Im Auftrag von Emmanuel Kasper
Gesendet: Montag, 20. Juli 2015 10:22
An: pve-user at pve.proxmox.com
Betreff: Re: [PVE-User] NFS-Performance

On 07/19/2015 06:17 PM, Holger Hampel | RA Consulting wrote:
> Hello,
> 
> Last weekend, the backup stalled (also the backups before, but nobody was there to care for it):
> 
> 657: Jul 12 10:37:13 INFO: status: 77% (57894961152/75161927680), sparse 3% (2493112320), duration 14059, 15/15 MB/s 657: Jul 12 10:45:29 INFO: status: 78% (58630930432/75161927680), sparse 3% (2497314816), duration 14555, 1/1 MB/s 657: Jul 12 10:50:34 INFO: status: 79% (59411005440/75161927680), sparse 3% (2497314816), duration 14860, 2/2 MB/s 657: Jul 12 10:54:53 INFO: status: 80% (60162506752/75161927680), sparse 3% (2509971456), duration 15119, 2/2 MB/s 657: Jul 12 10:55:15 INFO: status: 81% (60890480640/75161927680), sparse 3% (2518409216), duration 15141, 33/32 MB/s 657: Jul 12 11:03:05 INFO: status: 82% (61646045184/75161927680), sparse 3% (2518618112), duration 15611, 1/1 MB/s 657: Jul 12 11:06:07 INFO: status: 83% (62387257344/75161927680), sparse 3% (2518822912), duration 15793, 4/4 MB/s 657: Jul 12 11:09:22 INFO: status: 84% (63160647680/75161927680), sparse 3% (2519183360), duration 15988, 3/3 MB/s 657: Jul 12 11:13:33 INFO: status: 85% (63918243840/75161927680), sparse

3% (2519449600), duration 16239, 3/3 MB/s 657: Jul 12 11:19:26 INFO:
status: 86% (64648904704/75161927680), sparse 3% (2524684288), duration
16592, 2/2 MB/s 657: Jul 12 11:23:00 INFO: status: 87%
(65413185536/75161927680), sparse 3% (2542366720), duration 16806, 3/3
MB/s 657: Jul 12 11:25:41 INFO: status: 88% (66164162560/75161927680),
sparse 3% (2704044032), duration 16967, 4/3 MB/s 657: Jul 12 11:28:49
INFO: status: 89% (66912124928/75161927680), sparse 3% (2730418176),
duration 17155, 3/3 MB/s 657: Jul 12 11:32:25 INFO: status: 90%
(67659366400/75161927680), sparse 3% (2752782336), duration 17371, 3/3
MB/s 657: Jul 12 11:34:40 INFO: status: 91% (68420632576/75161927680),
sparse 3% (2766213120), duration 17506, 5/5 MB/s 657: Jul 12 11:39:33
INFO: status: 92% (69151490048/75161927680), sparse 3% (2789863424),
duration 17799, 2/2 MB/s 657: Jul 12 11:42:29 INFO: status: 93%
(69910659072/75161927680), sparse 3% (2809716736), duration 17975, 4/4
MB/s 657: Jul 12 11:52:13 INFO: status: 94
% (70690799616/75161927680), sparse 3% (2822275072), duration 18559, 1/1
MB/s
> 
> After backup completion I tested the speed of NFS access (also using different blocksizes without siginificant changes):
> 
> root at gaia:~# dd if=/dev/sda of=/mnt/pve/Backup-Chaos/test bs=8192 count=10000
> 10000+0 Datensätze ein
> 10000+0 Datensätze aus
> 81920000 Bytes (82 MB) kopiert, 16,086 s, 3,1 MB/s
> 
> On another node (same hardware and config) I'm testing the 3.10 kernel:
> 
> root at tartaros:~# dd if=/dev/sda of=/mnt/pve/Backup-Chaos/test bs=8192 count=10000
> 10000+0 Datensätze ein
> 10000+0 Datensätze aus
> 81920000 Bytes (82 MB) kopiert, 0,950729 s, 86,2 MB/s
> 
> So I upgraded the kernel on the first node and:
> 
> root at nyx:~# dd if=/dev/sda of=/mnt/pve/Backup-Chaos/test bs=8192 count=10000
> 10000+0 Datensätze ein
> 10000+0 Datensätze aus
> 81920000 Bytes (82 MB) kopiert, 0,912615 s, 89,8 MB/s
> 
> 
> It seems independend of using IPv4/v6 and another machine (on pve-test, other hardware) without bonding has no problems accessing the same NFS-Server.
> 

Hi Holger
I just tested the speed of nfs writes from two nodes, one running with
2.6.32 and the other one with 3.10, and I don't see any differences with
bonnie++ here.

>From both I get a speed of 70 MB/s ( 72701 in output under) in
sequential writes.

2.6.32 nfs client:

root at pve3:/mnt/pve/nas-write# bonnie++ -u manu -d .
Version  1.96       ------Sequential Output------ --Sequential Input-
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
pve3             2G  2125  96 72701   2 42946   2 +++++ +++ 97566   4
2059  20

Emmanuel

_______________________________________________
pve-user mailing list
pve-user at pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user