[PVE-User] pveceph : Unable to add any OSD

[Alwin Antreich]

On Sun, Sep 17, 2017 at 11:18:51AM +0200, Phil Schwarz wrote:
> Hi,
> going on on the same problem (links [1] & [2] )
>
> [1] : https://pve.proxmox.com/pipermail/pve-user/2017-July/168578.html
> [2] : https://pve.proxmox.com/pipermail/pve-user/2017-September/168775.html
>
> -Added a brand new node, updated to last ceph version (the proxmox team
> recompiled one)
Can you please post a 'ceph versions' and a 'ceph osd tree' to get some
overview on your setup?

>
> -plugged a new disk
>
> -used GUI (same result with pveceph createosd from cli) to create a new osd
> (with bluestore feature).
PVE GUI and CLI use the same API for managing ceph.

>
> 1. The OSD doesn't appear on gui
> 2. The OSD is seen as down and out of any node
> 3. the /var/log/ceph/ceph-osd.admin.log logfile seems to figure a
> mismatch between filestore and bluestore:
Do you see any errors in the mon logs or ceph.log itself?

>
> 2017-09-16 19:12:00.468481 7f6469cdde00  0 ceph version 12.2.0
> (36f6c5ea099d43087ff0276121fd34e71668ae0e) luminous (rc), process (unknown),
> pid 5624
> 2017-09-16 19:12:00.470154 7f6469cdde00 -1 bluestore(/dev/sdb2)
> _read_bdev_label unable to decode label at offset 102:
> buffer::malformed_input: void
> bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode past
> end of struct encoding
> 2017-09-16 19:12:00.471408 7f6469cdde00  1 journal _open /dev/sdb2 fd 4:
> 750050447360 bytes, block size 4096 bytes, directio = 0, aio = 0
> 2017-09-16 19:12:00.471727 7f6469cdde00  1 journal close /dev/sdb2
> 2017-09-16 19:12:00.471994 7f6469cdde00  0 probe_block_device_fsid /dev/sdb2
> is filestore, 00000000-0000-0000-0000-000000000000
> 2017-09-16 19:12:05.042622 7f000b944e00  0 ceph version 12.2.0
> (36f6c5ea099d43087ff0276121fd34e71668ae0e) luminous (rc), process (unknown),
> pid 5702
> 2017-09-16 19:12:05.066343 7f000b944e00 -1 bluestore(/dev/sdb2)
> _read_bdev_label unable to decode label at offset 102:
> buffer::malformed_input: void
> bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode past
> end of struct encoding
> 2017-09-16 19:12:05.066549 7f000b944e00  1 journal _open /dev/sdb2 fd 4:
> 750050447360 bytes, block size 4096 bytes, directio = 0, aio = 0
> 2017-09-16 19:12:05.066717 7f000b944e00  1 journal close /dev/sdb2
> 2017-09-16 19:12:05.066843 7f000b944e00  0 probe_block_device_fsid /dev/sdb2
> is filestore, 00000000-0000-0000-0000-000000000000
> 2017-09-16 19:12:08.198548 7f5740748e00  0 ceph version 12.2.0
> (36f6c5ea099d43087ff0276121fd34e71668ae0e) luminous (rc), process (unknown),
> pid 5767
> 2017-09-16 19:12:08.223674 7f5740748e00 -1 bluestore(/dev/sdb2)
> _read_bdev_label unable to decode label at offset 102:
> buffer::malformed_input: void
> bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode past
> end of struct encoding
> 2017-09-16 19:12:08.223831 7f5740748e00  1 journal _open /dev/sdb2 fd 4:
> 750050447360 bytes, block size 4096 bytes, directio = 0, aio = 0
> 2017-09-16 19:12:08.224213 7f5740748e00  1 journal close /dev/sdb2
> 2017-09-16 19:12:08.224342 7f5740748e00  0 probe_block_device_fsid /dev/sdb2
> is filestore, 00000000-0000-0000-0000-000000000000
> 2017-09-16 19:12:09.149622 7f7b06058e00  0 ceph version 12.2.0
> (36f6c5ea099d43087ff0276121fd34e71668ae0e) luminous (rc), process (unknown),
> pid 5800
> 2017-09-16 19:12:09.173319 7f7b06058e00 -1 bluestore(/dev/sdb2)
> _read_bdev_label unable to decode label at offset 102:
> buffer::malformed_input: void
> bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode past
> end of struct encoding
> 2017-09-16 19:12:09.173402 7f7b06058e00  1 journal _open /dev/sdb2 fd 4:
> 750050447360 bytes, block size 4096 bytes, directio = 0, aio = 0
> 2017-09-16 19:12:09.173485 7f7b06058e00  1 journal close /dev/sdb2
> 2017-09-16 19:12:09.173511 7f7b06058e00  0 probe_block_device_fsid /dev/sdb2
> is filestore, 00000000-0000-0000-0000-000000000000
> 2017-09-16 19:12:10.197944 7f7561d50e00  0 ceph version 12.2.0
> (36f6c5ea099d43087ff0276121fd34e71668ae0e) luminous (rc), process (unknown),
> pid 5828
> 2017-09-16 19:12:10.222504 7f7561d50e00 -1 bluestore(/dev/sdb2)
> _read_bdev_label unable to decode label at offset 102:
> buffer::malformed_input: void
> bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode past
> end of struct encoding
> 2017-09-16 19:12:10.222723 7f7561d50e00  1 journal _open /dev/sdb2 fd 4:
> 750050447360 bytes, block size 4096 bytes, directio = 0, aio = 0
> 2017-09-16 19:12:10.222753 7f7561d50e00  1 journal close /dev/sdb2
> 2017-09-16 19:12:10.222785 7f7561d50e00  0 probe_block_device_fsid /dev/sdb2
> is filestore, 00000000-0000-0000-0000-000000000000
> 2017-09-16 19:12:14.370797 7f9fecb7fe00  0 ceph version 12.2.0
> (36f6c5ea099d43087ff0276121fd34e71668ae0e) luminous (rc), process (unknown),
> pid 5964
> 2017-09-16 19:12:14.371221 7f9fecb7fe00 -1 bluestore(/dev/sdb2)
> _read_bdev_label unable to decode label at offset 102:
> buffer::malformed_input: void
> bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode past
> end of struct encoding
> 2017-09-16 19:12:14.371350 7f9fecb7fe00  1 journal _open /dev/sdb2 fd 4:
> 750050447360 bytes, block size 4096 bytes, directio = 0, aio = 0
> 2017-09-16 19:12:14.371616 7f9fecb7fe00  1 journal close /dev/sdb2
> 2017-09-16 19:12:14.371745 7f9fecb7fe00  0 probe_block_device_fsid /dev/sdb2
> is filestore, 00000000-0000-0000-0000-000000000000
> 2017-09-16 19:12:21.171036 7f5d7579be00  0 ceph version 12.2.0
> (36f6c5ea099d43087ff0276121fd34e71668ae0e) luminous (rc), process (unknown),
> pid 6130
> 2017-09-16 19:12:21.209441 7f5d7579be00  0 probe_block_device_fsid /dev/sdb2
> is bluestore, 92a4a9eb-0a6a-405d-be83-11e4af42fa30
>
>
>
> Any hint ?
>
> Thanks by advance
> Best regards
>
--
Cheers,
Alwin