[PVE-User] Trooble with ceph on the last version of proxmox

[Jean-mathieu CHANTREIN]

> De: "Jean-mathieu CHANTREIN" <jean-mathieu.chantrein at univ-angers.fr>
> À: "pve-user" <pve-user at pve.proxmox.com>
> Envoyé: Lundi 11 Septembre 2017 15:24:12
> Objet: Re: [PVE-User] Trooble with ceph on the last version of proxmox

>> De: "Dominik Csapak" <d.csapak at proxmox.com>
>> À: "pve-user" <pve-user at pve.proxmox.com>
>> Envoyé: Lundi 11 Septembre 2017 14:04:51
>> Objet: Re: [PVE-User] Trooble with ceph on the last version of proxmox
>> On 09/11/2017 11:47 AM, Jean-mathieu CHANTREIN wrote:
>>> _ When I add my ceph-vm pool to my proxmox storage, I can not use it and my
>>> local storage is no longer visible when I try to create VMs, the ceph pools are
>>> no longer visible in GUI but clearly visible in CLI.
>> are you sure you have copied the keyring to the correct place?
>> /etc/pve/priv/ceph/<STORAGE-ID>.keyring
>> ?
> Yes.

> ls /etc/pve/priv/ceph/
> ceph-lxc.keyring ceph-vm.keyring

> I was created pool with this cmd:
> pveceph createpool ceph-vm -min_size 2 -size 3 -pg_num 256
> pveceph createpool ceph-lxc -min_size 2 -size 3 -pg_num 256

> and add to the storage with this cmd:
> pvesm add rbd ceph-vm -content images -monhost "IP_PROXMOX_NODE[1-4]" -username
> admin
> pvesm add rbd ceph-lxc -krbd 1 -content rootdir -monhost "IP_PROXMOX_NODE[1-4]"
> -username admin

> I just see I have use the proxmox subnetwork IP and not the ceph subnetwork IP
> :-\ . I will recreate with the ceph network IP. Maybe is the trick.

It's effectively better with the good subnet... Additionnaly, I see my ceph pool create with the CLI does not work. When I try to create a VM: 
"Ensure that the default pool has been created or specify an alternate pool name." 

So I'll try rbd list: 
"rbd: error opening default pool 'rbd' 
Ensure that the default pool has been created or specify an alternate pool name." 

In documentation: http://ceph.com/releases/v12-1-1-luminous-rc-released/ : 
"The default ‘rbd’ pool is no longer created automatically during cluster creation. Additionally, the name of the default pool used by the rbd CLI when no pool is specified can be overridden via a new rbd default pool = <pool name> configuration option." 

I was think PVE try to connect to the default rbd pool (like old pve version ?) 

So I create one with GUI (ceph-test) to see any difference between configuration file. ceph-test working fine. Here is a part of output of /etc/pve/storage.cfg: 

rbd: ceph-vm 
monhost IP_MON[1-4] 
content images 
username admin 

rbd: ceph-test 
monhost IP_MON[1-4] 
content images 
krbd 0 
pool ceph-test 
username admin 

We can see a line "pool ceph-test" append by GUI PVE Add storage. So I have to add -pool options on CLI: 

pvesm add rbd ceph-vm -content images -monhost "IP_PROXMOX_NODE[1-4]" -username 
admin -pool ceph-vm 
pvesm add rbd ceph-lxc -krbd 1 -content rootdir -monhost "IP_PROXMOX_NODE[1-4]" 
-username admin -pool ceph-lxc 

And everything looks good :-) 

Thanks. 

Jean-Mathieu