[PVE-User] Ceph - Mixed harddrives

[Karsten Becker]

Hi.

Now it's the time that I need your help.

I bought some fast SSDs and added them to 2 (kong & godzilla) of my 3
Proxmox servers. The servers are now populated with mixed drives - both
SSDs and spinning.

My plan was to add a new ruleset to the crush map to have both the new,
fast SSDs as a Ceph pool and to keep the old spinning Ceph pool.

So my crush map looks like that:

> host kong {
> 	id -5		# do not change unnecessarily
> 	# weight 12.710
> 	alg straw2
> 	hash 0	# rjenkins1
> 	item osd.8 weight 3.630
> 	item osd.9 weight 3.630
> 	item osd.10 weight 3.630
> 	item osd.11 weight 1.820
> }
> rack lan3 {
> 	id -11		# do not change unnecessarily
> 	# weight 12.710
> 	alg straw2
> 	hash 0	# rjenkins1
> 	item kong weight 12.710
> }
> room dc3 {
> 	id -12		# do not change unnecessarily
> 	# weight 12.710
> 	alg straw2
> 	hash 0	# rjenkins1
> 	item lan3 weight 12.710
> }
> host king {
> 	id -6		# do not change unnecessarily
> 	# weight 16.350
> 	alg straw2
> 	hash 0	# rjenkins1
> 	item osd.14 weight 3.630
> 	item osd.15 weight 3.630
> 	item osd.16 weight 3.630
> 	item osd.17 weight 1.820
> 	item osd.18 weight 1.820
> 	item osd.19 weight 1.820
> }
> rack lan6 {
> 	id -8		# do not change unnecessarily
> 	# weight 16.350
> 	alg straw2
> 	hash 0	# rjenkins1
> 	item king weight 16.350
> }
> room dc6 {
> 	id -10		# do not change unnecessarily
> 	# weight 16.350
> 	alg straw2
> 	hash 0	# rjenkins1
> 	item lan6 weight 16.350
> }
> root ecologic {
> 	id -1		# do not change unnecessarily
> 	# weight 29.060
> 	alg straw2
> 	hash 0	# rjenkins1
> 	item dc3 weight 12.710
> 	item dc6 weight 16.350
> }
> rule replicated_ruleset {
> 	ruleset 0
> 	type replicated
> 	min_size 1
> 	max_size 10
> 	step take ecologic
> 	step chooseleaf firstn 0 type room
> 	step emit
> }
> 
> 
> ###
> ###   SSD POOL
> ###
> 
> 
> host kong_ssd {
> 	id -13		# do not change unnecessarily
> 	# weight 2.400
> 	alg straw2
> 	hash 0	# rjenkins1
> 	item osd.3 weight 0.800
> 	item osd.4 weight 0.800
> 	item osd.5 weight 0.800
> }
> rack lan3_ssd {
> 	id -14		# do not change unnecessarily
> 	# weight 2.400
> 	alg straw2
> 	hash 0	# rjenkins1
> 	item kong_ssd weight 2.400
> }
> room dc3_ssd {
> 	id -15		# do not change unnecessarily
> 	# weight 2.400
> 	alg straw2
> 	hash 0	# rjenkins1
> 	item lan3_ssd weight 2.400
> }
> host godzilla_ssd {
> 	id -16		# do not change unnecessarily
> 	# weight 2.400
> 	alg straw2
> 	hash 0	# rjenkins1
> 	item osd.0 weight 0.800
> 	item osd.1 weight 0.800
> 	item osd.2 weight 0.800
> }
> rack lan6_ssd {
> 	id -17		# do not change unnecessarily
> 	# weight 2.400
> 	alg straw2
> 	hash 0	# rjenkins1
> 	item godzilla_ssd weight 2.400
> }
> room dc6_ssd {
> 	id -18		# do not change unnecessarily
> 	# weight 2.400
> 	alg straw2
> 	hash 0	# rjenkins1
> 	item lan6_ssd weight 2.400
> }
> root ecologic_ssd {
> 	id -19		# do not change unnecessarily
> 	# weight 4.800
> 	alg straw2
> 	hash 0	# rjenkins1
> 	item dc3_ssd weight 2.400
> 	item dc6_ssd weight 2.400
> }
> rule replicated_ruleset_ssd {
> 	ruleset 1
> 	type replicated
> 	min_size 1
> 	max_size 10
> 	step take ecologic_ssd
> 	step chooseleaf firstn 0 type room
> 	step emit
> }
> 
> 
> # end crush map

But unfortunately this doesn't seem to work combined with Proxmox.
First, Proxmox did not know the hostnames when I was trying to start the
new OSDs. So I added the <hostname>_ssd to the /etc/hosts.

After that I get a "Connection error 596: ssl3_get_server_certificate:
certificate verify failed" when trying to start the OSDs in the GUI.

How can I archive the mixed harddrive/pool setup with Proxmox?

Best regards
Karsten

Ecologic Institut gemeinnuetzige GmbH
Pfalzburger Str. 43/44, D-10717 Berlin
Geschaeftsfuehrerin / Director: Dr. Camilla Bausch
Sitz der Gesellschaft / Registered Office: Berlin (Germany)
Registergericht / Court of Registration: Amtsgericht Berlin (Charlottenburg), HRB 57947