[pve-devel] question about pve-qemu && librbd

[Alexandre DERUMIER]

>>but for other use cases, we don't (yet?) have that limitation. e.g., 
>>running a radosgw (which is a client from Ceph's perspective) or just 
>>rbd access to an external cluster is supported using Jewel as well. 

mmm, I don't have thinked about radosgw.


>>if we move librbd1, we'd have to move ceph-common, which means we'd have 
>>to move ceph-base, ceph-osd, ceph-mon and ceph, as well as radosgw and 
>>end up with a horrible mix of package versions which is most likely not 
>>tested by anyone else. 


>>so these are the options: 
>>- merge ceph and regular PVE repos (not possible to support multiple 
>>Ceph versions in the future!) 
>>- keep everything as is, losing a bit of performance 
- patch Qemu to do runtime loading and feature detection for librbd 

I think this should be the good way


qemu block/rbd.c have

/* The LIBRBD_SUPPORTS_IOVEC is defined in librbd.h */
#ifdef LIBRBD_SUPPORTS_IOVEC
#define LIBRBD_USE_IOVEC 1
#else
#define LIBRBD_USE_IOVEC 0
#endif



I thinked it was dynamic, but looking at qemu-binary if builded with lastrbd,
I only see rbd_aio_writev symbol.

Maybe it is possible to detect this at runtime ?



>>- build a second qemu package linked with librbd1 from Ceph repo and put 
>>it in Ceph repo (either with higher version, or with different package 
>>name and provides+replaces) 

Good idea too. (don't known how much work is it for proxmox team to mainain 2 version)


----- Mail original -----
De: "Fabian Grünbichler" <f.gruenbichler at proxmox.com>
À: "pve-devel" <pve-devel at pve.proxmox.com>
Envoyé: Lundi 25 Septembre 2017 15:29:20
Objet: Re: [pve-devel] question about pve-qemu && librbd

On Mon, Sep 25, 2017 at 03:10:11PM +0200, Alexandre DERUMIER wrote: 
> >>there might be some yet undiscovered incompatibility when mixing versions like 
> >>that (there are other ceph libraries with reverse-dependencies to 
> >>consider as well) 
> 
> But proxmox 5.0 ceph-server only support luminous, right ? 
> so, I think it shouldn't be a problem ? 

yes, as hyper-converged setup using pveceph we only support Luminous. 

but for other use cases, we don't (yet?) have that limitation. e.g., 
running a radosgw (which is a client from Ceph's perspective) or just 
rbd access to an external cluster is supported using Jewel as well. 

> and from client side, librbd is backward compatible to connect on older cluster (I have tested lirbd luminous to jewel && hammer cluster without any problem) 
> 

if we move librbd1, we'd have to move ceph-common, which means we'd have 
to move ceph-base, ceph-osd, ceph-mon and ceph, as well as radosgw and 
end up with a horrible mix of package versions which is most likely not 
tested by anyone else. 

so these are the options: 
- merge ceph and regular PVE repos (not possible to support multiple 
Ceph versions in the future!) 
- keep everything as is, losing a bit of performance 
- patch Qemu to do runtime loading and feature detection for librbd 
- build a second qemu package linked with librbd1 from Ceph repo and put 
it in Ceph repo (either with higher version, or with different package 
name and provides+replaces) 

_______________________________________________ 
pve-devel mailing list 
pve-devel at pve.proxmox.com 
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel