[PVE-User] Patch to Storage.pm : storage listing 3x faster

[Gilles Mocellin]

Hello,

I have troubles since some times as I have 35 Volume Groups in my
cluster.

You can see the problem here :
http://forum.proxmox.com/threads/4397-iSCSI-Hardware.htm-Error-in-Perl-code-500-read-timeout

It is barrely impossible to open the Hardware Tab of a VM.

The first thing I do was setting a higher timeout.
But now, it takes 35 to 50s, and it is a real pain.

I know, I should have made less VGs.
I'm in a process to reorganise that as it is also better for backups in
snapshot mode.

But I need a woraround for now.

And I found one.

I have profiled the "pvesm list" :
# dprofpp -rI pve-tmon.out 
Total Elapsed Time = 20.77974 Seconds
         Real Time = 20.77974 Seconds
Inclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c  Name
 99.5   0.010 20.682      1   0.0099 20.682  PVE::Storage::storage_info
 99.3   0.033 20.654    105   0.0003 0.1967  PVE::Storage::run_command
 98.0   20.35 20.379    210   0.0969 0.0970  IO::Select::can_read
 81.1   0.010 16.860     70   0.0001 0.2409  PVE::Storage::lvm_vgs
 59.0       - 12.262     40        - 0.3066  PVE::Storage::__activate_storage_f
                                             ull
 59.0       - 12.262      1        - 12.262  PVE::Storage::activate_storage_lis
                                          2  t
 1.03   0.027  0.214    105   0.0003 0.0020  IPC::Open3::_open3
 1.03       -  0.214    105        - 0.0020  IPC::Open3::open3
 0.91   0.190  0.190    105   0.0018 0.0018  IPC::Open3::xfork
 0.47   0.040  0.098      5   0.0080 0.0197  main::BEGIN
 0.24       -  0.049     17        - 0.0029  PVE::Storage::BEGIN
 0.10   0.020  0.020    210   0.0001 0.0001  IO::Select::handles
 0.10       -  0.020     10        - 0.0020  IO::Dir::BEGIN
 0.05   0.010  0.010      1   0.0100 0.0100  POSIX::bootstrap
 0.05   0.010  0.010      8   0.0012 0.0012  File::Path::_mkpath

As you can see, the time is spent mainly in PVE::Storage::lvm_vgs, launching 105 times 'vgs --noheading ...', 3 time by VG.

My proposed patch create a cache for the result of the vgs command, valid for an entire PVE::Storage::storage_info run.

# dprofpp -rI gmo-tmon.out 
Total Elapsed Time = 7.479416 Seconds
         Real Time = 7.479416 Seconds
Inclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c  Name
 98.1   0.010  7.342     40   0.0002 0.1835  PVE::Storage::__activate_storage_f
                                             ull
 98.1       -  7.342      1        - 7.3419  PVE::Storage::activate_storage_lis
                                             t
 98.1       -  7.342      1        - 7.3417  PVE::Storage::storage_info
 98.0   0.039  7.334     36   0.0011 0.2037  PVE::Storage::run_command
 96.5   7.220  7.220     72   0.1003 0.1003  IO::Select::can_read
 5.35       -  0.400     70        - 0.0057  PVE::Storage::lvm_vgs
 1.44   0.030  0.108      5   0.0060 0.0217  main::BEGIN
 0.92   0.010  0.069     17   0.0006 0.0041  PVE::Storage::BEGIN
 0.91   0.019  0.068     36   0.0005 0.0019  IPC::Open3::_open3
 0.91       -  0.068     36        - 0.0019  IPC::Open3::open3
 0.67   0.050  0.050     36   0.0014 0.0014  IPC::Open3::xfork
 0.40   0.020  0.030     45   0.0004 0.0007  Exporter::import
 0.27       -  0.020     10        - 0.0020  IO::Dir::BEGIN
 0.13   0.010  0.010      6   0.0017 0.0017  Exporter::export
 0.13       -  0.010      1        - 0.0100  POSIX::import

Sufficient for me for the moment !

I'm sure my hack is not the best way to handle that.
If the Proxmox team can include such an improvement, I would be pleased
:-)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cache-lvm_vgs.diff
Type: text/x-diff
Size: 966 bytes
Desc: not available
URL: <http://lists.proxmox.com/pipermail/pve-user/attachments/20100911/dfe9b087/attachment.diff>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.proxmox.com/pipermail/pve-user/attachments/20100911/dfe9b087/attachment.sig>