[PVE-User] Incredible low fsync with HW raid: Intel RS2BL040

[Pongrácz István]

Hi All,

I have a system with PVE 1.8 with HW raid card (Intel RS2BL040), without BBU, 4 x 1.5Tbyte hdd in RAID10

I found that, the pveperf shows incredibeli low FSYNC, even 0.89.

Guys, seriously, what the hack could happen to cause this incredibely low measurement? Here are some test data and details about the system and settings.

Does anybody meet this kind of situation?

Any hints, documentation on the web are welcome.

Thank you,

István

 

Details

 

hdparm -tT /dev/pve/data 
/dev/pve/data:
Timing cached reads: 15292 MB in 2.00 seconds = 7654.83 MB/sec
Timing buffered disk reads: 522 MB in 3.00 seconds = 173.97 MB/sec
hn3:~# hdparm -tT /dev/pve/root 
/dev/pve/root:
Timing cached reads: 15002 MB in 2.00 seconds = 7509.61 MB/sec
Timing buffered disk reads: 532 MB in 3.29 seconds = 161.89 MB/sec

-----------

# pveperf 
CPU BOGOMIPS: 40530.64
REGEX/SECOND: 820137
HD SIZE: 94.49 GB (/dev/mapper/pve-root)
BUFFERED READS: 114.54 MB/sec
AVERAGE SEEK TIME: 12.01 ms
FSYNCS/SECOND: 377.33
DNS EXT: 46.11 ms
DNS INT: 53.96 ms

pveperf /var/lib/vz
CPU BOGOMIPS: 40530.64
REGEX/SECOND: 841456
HD SIZE: 1663.48 GB (/dev/mapper/pve-data)
BUFFERED READS: 153.84 MB/sec
AVERAGE SEEK TIME: 26.99 ms
FSYNCS/SECOND: 280.32
DNS EXT: 49.93 ms
DNS INT: 44.12 ms

----------- RAID settings -----------

Settings
================
Current Time : 12:9:50 6/2, 2012
Predictive Fail Poll Interval : 300sec
Interrupt Throttle Active Count : 16
Interrupt Throttle Completion : 50us
Rebuild Rate : 30%
PR Rate : 30%
BGI Rate : 30%
Check Consistency Rate : 30%
Reconstruction Rate : 30%
Cache Flush Interval : 4s
Max Drives to Spinup at One Time : 4
Delay Among Spinup Groups : 2s
Physical Drive Coercion Mode : 1GB
Cluster Mode : Disabled
Alarm : Enabled
Auto Rebuild : Enabled
Battery Warning : Enabled
Ecc Bucket Size : 15
Ecc Bucket Leak Rate : 1440 Minutes
Restore HotSpare on Insertion : Enabled
Expose Enclosure Devices : Disabled
Maintain PD Fail History : Enabled
Host Request Reordering : Enabled
Auto Detect BackPlane Enabled : SGPIO/i2c SEP
Load Balance Mode : Auto
Use FDE Only : No
Security Key Assigned : No
Security Key Failed : No
Security Key Not Backedup : No
Default LD PowerSave Policy : Controller Defined
Maximum number of direct attached drives to spin up in 1 min : 0 
Any Offline VD Cache Preserved : No
Allow Boot with Preserved Cache : No
Disable Online Controller Reset : No
PFK in NVRAM : No
Use disk activity for locate : No

 

Default Settings
================
Phy Polarity : 0 
Phy PolaritySplit : 0 
Background Rate : 30 
Strip Size : 256kB
Flush Time : 4 seconds
Write Policy : WB
Read Policy : Adaptive
Cache When BBU Bad : Disabled
Cached IO : No
SMART Mode : Mode 6
Alarm Disable : Yes
Coercion Mode : 1GB
ZCR Config : Unknown
Dirty LED Shows Drive Activity : No
BIOS Continue on Error : No
Spin Down Mode : None
Allowed Device Type : SAS/SATA Mix
Allow Mix in Enclosure : Yes
Allow HDD SAS/SATA Mix in VD : Yes
Allow SSD SAS/SATA Mix in VD : No
Allow HDD/SSD Mix in VD : No
Allow SATA in Cluster : No
Max Chained Enclosures : 16 
Disable Ctrl-R : Yes
Enable Web BIOS : Yes
Direct PD Mapping : No
BIOS Enumerate VDs : Yes
Restore Hot Spare on Insertion : Yes
Expose Enclosure Devices : No
Maintain PD Fail History : Yes
Disable Puncturing : No
Zero Based Enclosure Enumeration : No
PreBoot CLI Enabled : Yes
LED Show Drive Activity : Yes
Cluster Disable : Yes
SAS Disable : No
Auto Detect BackPlane Enable : SGPIO/i2c SEP
Use FDE Only : No
Enable Led Header : Yes
Delay during POST : 0 
EnableCrashDump : No
Disable Online Controller Reset : No
EnableLDBBM : No
Un-Certified Hard Disk Drives : Allow
Treat Single span R1E as R10 : No
Max LD per array : 16
Power Saving option : All power saving options are enabled
Default spin down time in minutes: 30 
Enable JBOD : No
TTY Log In Flash : No
Auto Enhanced Import : No
BreakMirror RAID Support : No
Disable Join Mirror : No
Enable Shield State : No
Time taken to detect CME : 60s
 

 

pve-manager: 1.8-18 (pve-manager/1.8/6070)
running kernel: 2.6.32-4-pve
proxmox-ve-2.6.32: 1.8-33
pve-kernel-2.6.32-4-pve: 2.6.32-33
qemu-server: 1.1-30
pve-firmware: 1.0-11
libpve-storage-perl: 1.0-17
vncterm: 0.9-2
vzctl: 3.0.28-1pve1
vzdump: 1.2-14
vzprocps: 2.0.11-2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.14.1-1
ksm-control-daemon: 1.0-6 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.proxmox.com/pipermail/pve-user/attachments/20120602/9999046f/attachment.htm>