[PVE-User] Orphaned entries in Ceph crush map

[Karsten Becker]

Oh, forget it... I just read your mail completely...

How did you remove the OSDs?

I use:

> export CEPH_OSD_ID=X; \
> systemctl stop ceph-osd@${CEPH_OSD_ID}.service; \
> ceph osd down ${CEPH_OSD_ID}; \
> ceph osd out ${CEPH_OSD_ID}; \
> ceph osd crush remove osd.${CEPH_OSD_ID}; \
> ceph auth del osd.${CEPH_OSD_ID}; \
> ceph osd rm ${CEPH_OSD_ID};

Best
Karsten



On 19.02.2018 13:01, Karsten Becker wrote:
> Hi.
> 
> What is "ceph osd tree" displaying? Are they away there?
> 
> Have you tried to reload the WebGui completely - in Firefox by pressing
> SHIFT and the RELOAD of the browser?
> 
> Best
> Karsten
> 
> 
> 
> On 19.02.2018 12:37, Alan Urquhart wrote:
>> I have a similar issue.  
>>
>> Ceph osd tree shows 7 osd's as down, and lists them in the GUI (summary, but not in the OSD tab) also, however these OSD's have been previously removed from ceph / proxmox and DON'T appear in the crush map...
>>
>> Any ideas?
>>
>> Alan Urquhart
>> ASU Web Services Ltd
>> www.asuweb.co.uk
>> -----Original Message-----
>> From: pve-user [mailto:pve-user-bounces at pve.proxmox.com] On Behalf Of Karsten Becker
>> Sent: 19 February 2018 10:18
>> To: pve-user at pve.proxmox.com
>> Subject: Re: [PVE-User] Orphaned entries in Ceph crush map
>>
>> Ok. No, they are just visible in the decompiled map.
>>
>> Best
>> Karsten
>>
>>
>> On 19.02.2018 11:09, proxmox at elchaka.de wrote:
>>> Hi Karsten,
>>>
>>> IIRC this is normal and only a placeholder.
>>>
>>> Do you See the OSDs also with ceph osd tree?
>>>
>>> If "no" then Everything is ok. 
>>>
>>> - Mehmet
>>>
>>> Am 16. Februar 2018 21:55:27 MEZ schrieb Karsten Becker <karsten.becker at ecologic.eu>:
>>>> Hi.
>>>>
>>>> during the reorgainzation of my Cephb cluster I removed some OSDs.
>>>> Obviously something went wrong for 2 of them, osd.19 and osd.20.
>>>>
>>>> If I get my current Crush map, decompile and edit them, I see 2 
>>>> orphaned/stale entries for the former OSDs:
>>>>
>>>>> device 16 osd.16 class hdd
>>>>> device 17 osd.17 class hdd
>>>>> device 18 osd.18 class hdd
>>>>> device 19 device19
>>>>> device 20 device20
>>>>> device 21 osd.21 class hdd
>>>>> device 22 osd.22 class hdd
>>>>> device 23 osd.23 class hdd
>>>> If I delete them from the Crush map (file), recompile it and set it 
>>>> productive - they appear again... if I get the current map again and 
>>>> decompile them, they are in again.
>>>>
>>>> So how to get rid of these entries?
>>>>
>>>> Best from Berlin/Germany
>>>> Karsten
>>>>
>>>> Ecologic Institut gemeinnuetzige GmbH Pfalzburger Str. 43/44, D-10717 
>>>> Berlin Geschaeftsfuehrerin / Director: Dr. Camilla Bausch Sitz der 
>>>> Gesellschaft / Registered Office: Berlin (Germany) Registergericht / 
>>>> Court of Registration: Amtsgericht Berlin (Charlottenburg), HRB 57947 
>>>> _______________________________________________
>>>> pve-user mailing list
>>>> pve-user at pve.proxmox.com
>>>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>> _______________________________________________
>>> pve-user mailing list
>>> pve-user at pve.proxmox.com
>>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>>
>>
>>
>> Ecologic Institut gemeinnuetzige GmbH
>> Pfalzburger Str. 43/44, D-10717 Berlin
>> Geschaeftsfuehrerin / Director: Dr. Camilla Bausch Sitz der Gesellschaft / Registered Office: Berlin (Germany) Registergericht / Court of Registration: Amtsgericht Berlin (Charlottenburg), HRB 57947 _______________________________________________
>> pve-user mailing list
>> pve-user at pve.proxmox.com
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>
>> _______________________________________________
>> pve-user mailing list
>> pve-user at pve.proxmox.com
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>
> 
> 
> Ecologic Institut gemeinnuetzige GmbH
> Pfalzburger Str. 43/44, D-10717 Berlin
> Geschaeftsfuehrerin / Director: Dr. Camilla Bausch
> Sitz der Gesellschaft / Registered Office: Berlin (Germany)
> Registergericht / Court of Registration: Amtsgericht Berlin (Charlottenburg), HRB 57947
> _______________________________________________
> pve-user mailing list
> pve-user at pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
> 


Ecologic Institut gemeinnuetzige GmbH
Pfalzburger Str. 43/44, D-10717 Berlin
Geschaeftsfuehrerin / Director: Dr. Camilla Bausch
Sitz der Gesellschaft / Registered Office: Berlin (Germany)
Registergericht / Court of Registration: Amtsgericht Berlin (Charlottenburg), HRB 57947