[pve-devel] iscsi, some hang : pvestatd always do iscsi_session_rescan

[Alexandre DERUMIER]

>> Well, I think the problem is that you iscsi server is slow. 
Yes indeed, the san was overload yesterday. 
Also I have 2 scsi controllers (active/passive), failover can take 2min. (vm can handle this)


But the problem it's that proxmox become unresponsive when calling the iscsiadm during this time
also pvestatd hang, so rrds for vm stats are not updated during this time.
And with ha cluster, maybe fencing can be call ? (don't have tested it)


>>Forcing a timeout is easy (we have a timeout parameter for run_command()). 

oh great!

>>The question is what timeout do you want? 

I think 5sec timeout must be enough for session rescan.

root at kvm6:~# time /usr/bin/iscsiadm --mode session -r 1 -R 

real    0m0.616s
user    0m0.003s
sys     0m0.084s
root at kvm6:~# time /usr/bin/iscsiadm --mode session -r 1 -R 
Rescanning session [sid: 1, target: iqn.1986-03.com.sun:02:316dd6a9-76bc-62ea-93fa-d0140e876a4b, portal: 10.6.0.18,3260]

real    0m0.764s
user    0m0.006s
sys     0m0.114s
root at kvm6:~# time /usr/bin/iscsiadm --mode session -r 1 -R 
Rescanning session [sid: 1, target: iqn.1986-03.com.sun:02:316dd6a9-76bc-62ea-93fa-d0140e876a4b, portal: 10.6.0.18,3260]

real    0m0.753s
user    0m0.005s
sys     0m0.077s
root at kvm6:~# time /usr/bin/iscsiadm --mode session -r 1 -R 
Rescanning session [sid: 1, target: iqn.1986-03.com.sun:02:316dd6a9-76bc-62ea-93fa-d0140e876a4b, portal: 10.6.0.18,3260]

real    0m0.734s
user    0m0.005s
sys     0m0.094s


Thanks !
Alexandre



----- Mail original ----- 

De: "Dietmar Maurer" <dietmar at proxmox.com> 
À: "Alexandre DERUMIER" <aderumier at odiso.com>, pve-devel at pve.proxmox.com 
Envoyé: Vendredi 27 Avril 2012 10:12:46 
Objet: RE: [pve-devel] iscsi, some hang : pvestatd always do iscsi_session_rescan 

> I check more deeply, 
> the main problem seem to be that iscsiadm have a long timeout. 

Well, I think the problem is that you iscsi server is slow. 

> (In fact the iscsi timeout, so it can be huge in some config with iscsi failover, 
> or if a path fail). 
> 
> Maybe can we implemented some kind of "kill process iscsiadm" if it take too 
> much time ? 
> 
> 
> I see that "timelimit" package exist in debian, "timelimit /usr/bin/iscsiadm/ 
> ...." 
> 
> maybe this can add some protections to iscsiadm command? 

Forcing a timeout is easy (we have a timeout parameter for run_command()). 

The question is what timeout do you want? 

- Dietmar 



-- 

-- 




	Alexandre D erumier 
Ingénieur Système 
Fixe : 03 20 68 88 90 
Fax : 03 20 68 90 81 
45 Bvd du Général Leclerc 59100 Roubaix - France 
12 rue Marivaux 75002 Paris - France