[PVE-User] Ceph RBD image striping

[Alwin Antreich]

Hi Holger,
hi Markus,

On 07/06/2016 02:47 PM, Markus Dellermann wrote:
> Am Mittwoch, 6. Juli 2016, 14:31:48 CEST schrieb Alwin Antreich:
>> Hi Markus,
>>
>> On 07/06/2016 02:06 PM, Markus Dellermann wrote:
>>> Hi..
>>>
>>> Am Mittwoch, 6. Juli 2016, 13:56:21 CEST schrieb Alwin Antreich:
>>>> Hi,
>>>>
>>>> On 07/06/2016 09:33 AM, Eneko Lacunza wrote:
>>>>> Hi,
>>>>>
>>>>> El 06/07/16 a las 09:26, Alwin Antreich escribió:
>>>>>> On 07/06/2016 08:27 AM, Eneko Lacunza wrote:
>>>>>>> Hi Alwin,
>>>>>>>
>>>>>>> In Proxmox Ceph client integration is done using librbd, not krbd.
>>>>>>> Stripe parameters can't be defined from Proxmox GUI.>>
>>>>>>
>>>>>> If I understood correctly, this depends, if you add a ceph pool to
>>>>>> proxmox with the KRBD option enabled or not. I know that these settings
>>>>>> aren't done through the GUI, as you need to create the image with ceph
>>>>>> tools manually.>
>>>>>
>>>>> Sorry, I didn't recall the "new" krbd checkbox in storage config, never
>>>>> used that myself. Why are you using it?
>>>>
>>>> For LXC it's required, as this maps the rbd image, so it can be mounted
>>>> by
>>>> lxc.
>>>>
>>>>>>> Why are you changing striping parameters?
>>>>>>
>>>>>> We have some write intense tasks to perform and stripping can boost
>>>>>> write
>>>>>> performance. To test this I would like to add such a disk to a VM.
>>>>>
>>>>> I would do as follows:
>>>>> - Create the VM rbd disk from GUI
>>>>> - Then from CLI, remove rbd volume and recreate with the same name and
>>>>> the
>>>>> striping parameters you want.
>>>>
>>>> That's the way I went, but the disk isn't picked up and the only thing I
>>>> see is the timeout.
>>>>
>>>>> Are you using SSDs for journals?
>>>>>
>>>>>
>>>>> Cheers
>>>>
>>>> It is working now, but in the end, I found two different issues. The
>>>> first
>>>> one, had to do with the VM I was testing, once I tried the same setup
>>>> with
>>>> a different VM it gave me the second issue. Apparently there is a
>>>> difference between mon_host address list formatting with comma or space
>>>> between IPs.
>>>>
>>>> ==with KRBD==
>>>> monhost 192.168.4.50,192.168.4.51,192.168.4.52
>>>> ==w/o KRBD==
>>>> monhost 192.168.4.50 192.168.4.51 192.168.4.52
>>>
>>> You should use semicolon
>>
>> But this gives me the error message, when I try to mount the image. I
>> created a separate email on this to follow up.
>>
> 
> You have tried this:
> monhost 192.168.4.50;192.168.4.51;192.168.4.52

Thanks for clarification.

> 
> ?
> 
>> Thanks.
>>
>>>> ==error message==
>>>> kvm: -drive
>>>> file=rbd:rbd2/vm-101-disk-1:mon_host=192.168.4.50,192.168.4.51,192.168.4.
>>>> 52:
>>>> id=admin:auth_supported=cephx:keyring=/etc/pve/priv/ceph/rbd2.keyring,if
>>>> =non
>>>> e,id=drive-virtio0,cache=writeback,discard=on,format=raw,aio=threads,det
>>>> ect- zeroes=unmap: Block format 'raw' used by device 'drive-virtio0'
>>>> doesn't support the option '192.168.4.51'
>>>>
>>>> Now, is this a ceph issue or proxmox? I will send a new email to follow
>>>> up
>>>> on this separately.

Still, the GUI allows all of these notations and adds the storage (incl. content browsing). So then the Proxmox GUI
should catch the wrong notation, shouldn't it?

>>>>
>>>> Cheers,
>>>> Alwin
>>>
>>> Markus
>>>
>>>> _______________________________________________
>>>> pve-user mailing list
>>>> pve-user at pve.proxmox.com
>>>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>>
>>> _______________________________________________
>>> pve-user mailing list
>>> pve-user at pve.proxmox.com
>>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>
>> Cheers,
>> Alwin
>> _______________________________________________
>> pve-user mailing list
>> pve-user at pve.proxmox.com
>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
> 
> 
> _______________________________________________
> pve-user mailing list
> pve-user at pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
> 

Cheers,
Alwin