[pve-devel] pve-manager and disk IO monitoring

Michael Rasmussen mir at datanom.net
Fri Sep 30 08:36:28 CEST 2016 

On Fri, 30 Sep 2016 08:17:58 +0200
Michael Rasmussen <mir at datanom.net> wrote:

> 
> I will run another test now.
> 
New test run. Here scsi-generic loose but again I cannot run a clinical
test. My best guess is that if you run a number of tests on equal
hardware and under similar conditions and make an average calculation
the two would show more or less identical performance and therefore
either one could be a candidate except for one crucial thing. With
scsi-generic you have no disk IO stats which in my book disqualify
scsi-generic.

After this discovery I will change the code locally so that I use
scsi-block since disk IO stats for vm's is important in my book.

scsi-generic
iometer: (g=0): rw=randrw, bs=512-64K/512-64K/512-64K, ioengine=libaio, iodepth=64
fio-2.1.11
Starting 1 process
iometer: Laying out IO file(s) (1 file(s) / 3072MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [25688KB/6534KB/0KB /s] [6096/1549/0 iops] [eta 00m:00s] 
iometer: (groupid=0, jobs=1): err= 0: pid=692: Fri Sep 30 08:29:45 2016
  Description  : [Emulation of Intel IOmeter File Server Access Pattern]
  read : io=2454.9MB, bw=61632KB/s, iops=10092, runt= 40786msec
    slat (usec): min=5, max=4132, avg=11.21, stdev=12.77
    clat (usec): min=20, max=14294K, avg=3429.47, stdev=66357.97
     lat (usec): min=181, max=14295K, avg=3441.05, stdev=66357.98
    clat percentiles (usec):
     |  1.00th=[  237],  5.00th=[  290], 10.00th=[  334], 20.00th=[  430],
     | 30.00th=[  900], 40.00th=[ 2544], 50.00th=[ 2832], 60.00th=[ 3088],
     | 70.00th=[ 3312], 80.00th=[ 3472], 90.00th=[ 3824], 95.00th=[ 8896],
     | 99.00th=[20608], 99.50th=[23680], 99.90th=[36608], 99.95th=[42240],
     | 99.99th=[905216]
    bw (KB  /s): min= 8261, max=159707, per=100.00%, avg=62673.16, stdev=38084.27
  write: io=631998KB, bw=15495KB/s, iops=2529, runt= 40786msec
    slat (usec): min=6, max=22420, avg=13.61, stdev=74.39
    clat (usec): min=550, max=10671K, avg=11538.39, stdev=51649.55
     lat (msec): min=1, max=10671, avg=11.55, stdev=51.65
    clat percentiles (msec):
     |  1.00th=[    3],  5.00th=[    4], 10.00th=[    4], 20.00th=[    4],
     | 30.00th=[    4], 40.00th=[    4], 50.00th=[    5], 60.00th=[   10],
     | 70.00th=[   16], 80.00th=[   21], 90.00th=[   24], 95.00th=[   27],
     | 99.00th=[   42], 99.50th=[   47], 99.90th=[   61], 99.95th=[   80],
     | 99.99th=[  922]
    bw (KB  /s): min= 2100, max=43759, per=100.00%, avg=15752.12, stdev=9676.98
    lat (usec) : 50=0.01%, 100=0.01%, 250=1.30%, 500=18.10%, 750=3.84%
    lat (usec) : 1000=1.12%
    lat (msec) : 2=3.06%, 4=53.90%, 10=7.08%, 20=6.08%, 50=5.44%
    lat (msec) : 100=0.05%, 500=0.01%, 750=0.01%, 1000=0.01%, 2000=0.01%
    lat (msec) : >=2000=0.01%
  cpu          : usr=6.63%, sys=20.68%, ctx=449891, majf=0, minf=8
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=411627/w=103162/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=2454.9MB, aggrb=61632KB/s, minb=61632KB/s, maxb=61632KB/s, mint=40786msec, maxt=40786msec
  WRITE: io=631997KB, aggrb=15495KB/s, minb=15495KB/s, maxb=15495KB/s, mint=40786msec, maxt=40786msec

Disk stats (read/write):
  sda: ios=411628/103171, merge=0/37, ticks=1378732/1182232, in_queue=2727776, util=99.81%

Disk stats (read/write):
  sda: ios=412899/103503, merge=0/13, ticks=3375120/854608, in_queue=4309832, util=99.92%

scsi-block
iometer: (g=0): rw=randrw, bs=512-64K/512-64K/512-64K, ioengine=libaio, iodepth=64
fio-2.1.11
Starting 1 process
iometer: Laying out IO file(s) (1 file(s) / 3072MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [64417KB/17015KB/0KB /s] [14.1K/3777/0 iops] [eta 00m:00s]
iometer: (groupid=0, jobs=1): err= 0: pid=693: Fri Sep 30 08:26:45 2016
  Description  : [Emulation of Intel IOmeter File Server Access Pattern]
  read : io=2454.9MB, bw=90185KB/s, iops=14767, runt= 27873msec
    slat (usec): min=5, max=2673, avg=10.15, stdev=11.94
    clat (usec): min=205, max=2095.6K, avg=3410.58, stdev=12296.52
     lat (usec): min=220, max=2095.6K, avg=3421.09, stdev=12296.53
    clat percentiles (usec):
     |  1.00th=[ 1864],  5.00th=[ 2480], 10.00th=[ 2736], 20.00th=[ 2896],
     | 30.00th=[ 2992], 40.00th=[ 3056], 50.00th=[ 3120], 60.00th=[ 3184],
     | 70.00th=[ 3248], 80.00th=[ 3344], 90.00th=[ 3664], 95.00th=[ 4192],
     | 99.00th=[ 7072], 99.50th=[ 9536], 99.90th=[34048], 99.95th=[51968],
     | 99.99th=[415744]
    bw (KB  /s): min=56041, max=146699, per=100.00%, avg=90641.29, stdev=22841.18
  write: io=631998KB, bw=22674KB/s, iops=3701, runt= 27873msec
    slat (usec): min=6, max=4419, avg=12.22, stdev=21.60
    clat (usec): min=214, max=1518.1K, avg=3618.09, stdev=9459.93
     lat (usec): min=225, max=1518.1K, avg=3630.69, stdev=9459.99
    clat percentiles (msec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    4], 40.00th=[    4], 50.00th=[    4], 60.00th=[    4],
     | 70.00th=[    4], 80.00th=[    4], 90.00th=[    5], 95.00th=[    6],
     | 99.00th=[   10], 99.50th=[   13], 99.90th=[   32], 99.95th=[   50],
     | 99.99th=[  416]
    bw (KB  /s): min=13984, max=36269, per=100.00%, avg=22789.24, stdev=5880.73
    lat (usec) : 250=0.01%, 500=0.01%, 750=0.02%, 1000=0.03%
    lat (msec) : 2=1.15%, 4=91.52%, 10=6.75%, 20=0.35%, 50=0.12%
    lat (msec) : 100=0.03%, 250=0.01%, 500=0.01%, 2000=0.01%, >=2000=0.01%
  cpu          : usr=9.06%, sys=28.10%, ctx=469516, majf=0, minf=8
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=411627/w=103162/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=2454.9MB, aggrb=90185KB/s, minb=90185KB/s, maxb=90185KB/s, mint=27873msec, maxt=27873msec
  WRITE: io=631997KB, aggrb=22674KB/s, minb=22674KB/s, maxb=22674KB/s, mint=27873msec, maxt=27873msec

Disk stats (read/write):
  sda: ios=409890/102722, merge=0/38, ticks=1394324/378876, in_queue=1822544, util=99.70%


-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
Use statement labels that mean something.
            - The Elements of Programming Style (Kernighan & Plaugher)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.proxmox.com/pipermail/pve-devel/attachments/20160930/e1f6f1fd/attachment.sig>

More information about the pve-devel mailing list