[pve-devel] Fwd: Re: recommanded cache setting for rbd image

[Alexandre DERUMIER]

>>Data is only pushed to RAID cache if you do an sync - else it can be in RAM
>>(no matter if you have BBU or not).
>>
>>BBU only make sure that sync really makes data persistent.
>>
>>Or do I still miss something?

Oh yes, you are right, I'm a bit tired,sorry ;). Host keep writes in ram until the sync occur.
(I think this is the same in kvm guest).

so with cache=none, you have writes datas in guest ram
with cache=writeback, you have writes datas in guest ram + host ram

>>The user can choose on the GUI, but we need a reasonable default. 
I think cache=none is a good default 
But user really need support for barrier with kernel < 2.6.37, or it can really break things,and it's really depend of his guest setup.
http://monolight.cc/2011/06/barriers-caches-filesystems/ 

Barrier support 	                                   Kernel version 	
I/O barrier support 	                                        2.6.9 	
ext3 	                                                        2.6.9 	
reiserfs 	                                                2.6.9 	
SATA 	                                                        2.6.12 	
XFS – barriers enabled by default 	                        2.6.16 
ext4 – barriers enabled by default 	                        2.6.26 	
DM – simple devices (i.e. a single underlying device) 	        2.6.28 	
loop 	                                                        2.6.30 
DM – rewrite of the barrier code 	                        2.6.30 	
DM – crypt 	                                                2.6.31 	
DM – linear (i.e. standard LVM concatenated volumes) 	        2.6.31 	
DM – mpath 	                                                2.6.31 	
virtio-blk (only really safe with O_DIRECT backing devices) 	2.6.32 	
DM – dm-raid1 	                                                2.6.33 
DM – request based devices 	                                2.6.33 
MD barrier support on all personalities * 	                2.6.33 
barriers removed and replaced with FUA / explicit flushes 	2.6.37 	

----- Mail original ----- 

De: "Dietmar Maurer" <dietmar at proxmox.com> 
À: "Alexandre DERUMIER" <aderumier at odiso.com> 
Cc: pve-devel at pve.proxmox.com, "Stefan Priebe - Profihost AG" <s.priebe at profihost.ag> 
Envoyé: Mardi 6 Novembre 2012 18:34:55 
Objet: RE: [pve-devel] Fwd: Re: recommanded cache setting for rbd image 

> >>But this is standard behavior, also on physical hosts? 
> Yes, indeed, this is the same with physical hosts. (mainly disk cache). 
> Generaly, if you have a hardware raid controller with battery, the raid 
> controller disable the cache of the disk and use his own cache (so you don't 
> loose data). 
> But if you have single disks with special raid card, you can really loose datas 
> (little, disks have generally some MB of cache) 

Data is only pushed to RAID cache if you do an sync - else it can be in RAM 
(no matter if you have BBU or not). 

BBU only make sure that sync really makes data persistent. 

Or do I still miss something? 

> >>I guess we loose much performance with those syncs. 
> yes, we can have a big loose a performance. (or you need a really fast journal 
> (ssd,nvram) on your storage which can handle these syncs) 
> 
> >>Why do we want to be safer than a physical host? 
> I think that user should choose, depend of his applications needs 

The user can choose on the GUI, but we need a reasonable default. 

Note: this is a really complex issue (most users simply do not understand it, 
and are thus unable to select something reasonable).