[PVE-User] [ceph-users] OSD won't start, even created ??

Phil Schwarz infolist at schwarz-fr.net
Sat Sep 9 16:08:33 CEST 2017


Did a few more tests :

Older Ceph server with a pveceph create osd command (

(pveceph create osd /dev/sdb

equivalent to

ceph-disk prepare --zap-disk --fs-type xfs --cluster ceph --cluster-uuid 
a5c0cfed-...4bf939ed70 /dev/sdb )

sgdisk --print /dev/sdd

Disk /dev/sdd: 2930277168 sectors, 1.4 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 638646CF-......-62296C871132
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 2930277134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
    1        10487808      2930277134   1.4 TiB     F800  ceph data
    2            2048        10487807   5.0 GiB     F802  ceph journal


On a newer ceph server ( dpkg -l : 12.2.0-pve1 version)

sgdisk --print /dev/sdb

Disk /dev/sdb: 1465149168 sectors, 698.6 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): D63886B6-0.....26-BCBCD6FFCA3C
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1465149134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
    1            2048          206847   100.0 MiB   F800  ceph data
    2          206848      1465149134   698.5 GiB   FFFF  ceph block


Related to the cep-osd.admin log , i think i used a osd creation process 
leading to a bluestore osd (instead of a filestore one).
And seems that afterward the ceph server is unable to use the new 
bluestore :

( bluestore(/dev/sdb2) _read_bdev_label unable to decode label at offset 
102: buffer::malformed_input: void 
bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode 
past end of struct encoding
)
just before trying to use it as a filestore one :

( probe_block_device_fsid /dev/sdb2 is filestore )


Tried to use the --bluestore 0 flag when creating the osd, but the flag 
is unknown.

thanks by advance for any hint.
Being ready to do a few more tests.
Best regards.

Le 08/09/2017 à 17:27, Phil Schwarz a écrit :
> Hi,
> any help would be really useful.
> Does anyone got a clue with my issue ?
>
> Thanks by advance.
> Best regards;
>
>
> Le 05/09/2017 à 20:25, Phil Schwarz a écrit :
>> Hi,
>> I come back with same issue as seen in previous thread ( link given)
>>
>> trying to a 2TB SATA as OSD:
>> Using proxmox GUI or CLI (command given) give the same (bad) result.
>>
>> Didn't want to use a direct 'ceph osd create', thus bypassing pxmfs
>> redundant filesystem.
>>
>> I tried to build an OSD woth same disk on another machine (stronger one
>> with Opteron QuadCore), failing at the same time.
>>
>>
>> Sorry for crossposting, but i think, i fail against the pveceph wrapper.
>>
>>
>> Any help or clue would be really useful..
>>
>> Thanks
>> Best regards.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> -- Link to previous thread (but same problem):
>> https://www.mail-archive.com/ceph-users@lists.ceph.com/msg38897.html
>>
>>
>> -- commands :
>> fdisk /dev/sdc ( mklabel msdos, w, q)
>> ceph-disk zap /dev/sdc
>> pveceph createosd /dev/sdc
>>
>> -- dpkg -l
>>
>>  dpkg -l |grep ceph
>> ii  ceph                                 12.1.2-pve1 amd64
>> distributed storage and file system
>> ii  ceph-base                            12.1.2-pve1 amd64        common
>> ceph daemon libraries and management tools
>> ii  ceph-common                          12.1.2-pve1 amd64        common
>> utilities to mount and interact with a ceph storage cluster
>> ii  ceph-mgr                             12.1.2-pve1 amd64
>> manager for the ceph distributed storage system
>> ii  ceph-mon                             12.1.2-pve1 amd64
>> monitor server for the ceph storage system
>> ii  ceph-osd                             12.1.2-pve1 amd64        OSD
>> server for the ceph storage system
>> ii  libcephfs1                           10.2.5-7.2 amd64        Ceph
>> distributed file system client library
>> ii  libcephfs2                           12.1.2-pve1 amd64        Ceph
>> distributed file system client library
>> ii  python-cephfs                        12.1.2-pve1 amd64        Python
>> 2 libraries for the Ceph libcephfs library
>>
>> -- tail -f /var/log/ceph/ceph-osd.admin.log
>>
>> 2017-09-03 18:28:20.856641 7fad97e45e00  0 ceph version 12.1.2
>> (cd7bc3b11cdbe6fa94324b7322fb2a4716a052a7) luminous (rc), process
>> (unknown), pid 5493
>> 2017-09-03 18:28:20.857104 7fad97e45e00 -1 bluestore(/dev/sdc2)
>> _read_bdev_label unable to decode label at offset 102:
>> buffer::malformed_input: void
>> bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode
>> past end of struct encoding
>> 2017-09-03 18:28:20.857200 7fad97e45e00  1 journal _open /dev/sdc2 fd 4:
>> 2000293007360 bytes, block size 4096 bytes, directio = 0, aio = 0
>> 2017-09-03 18:28:20.857366 7fad97e45e00  1 journal close /dev/sdc2
>> 2017-09-03 18:28:20.857431 7fad97e45e00  0 probe_block_device_fsid
>> /dev/sdc2 is filestore, 00000000-0000-0000-0000-000000000000
>> 2017-09-03 18:28:21.937285 7fa5766a5e00  0 ceph version 12.1.2
>> (cd7bc3b11cdbe6fa94324b7322fb2a4716a052a7) luminous (rc), process
>> (unknown), pid 5590
>> 2017-09-03 18:28:21.944189 7fa5766a5e00 -1 bluestore(/dev/sdc2)
>> _read_bdev_label unable to decode label at offset 102:
>> buffer::malformed_input: void
>> bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode
>> past end of struct encoding
>> 2017-09-03 18:28:21.944305 7fa5766a5e00  1 journal _open /dev/sdc2 fd 4:
>> 2000293007360 bytes, block size 4096 bytes, directio = 0, aio = 0
>> 2017-09-03 18:28:21.944527 7fa5766a5e00  1 journal close /dev/sdc2
>> 2017-09-03 18:28:21.944588 7fa5766a5e00  0 probe_block_device_fsid
>> /dev/sdc2 is filestore, 00000000-0000-0000-0000-000000000000
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users at lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
> _______________________________________________
> pve-user mailing list
> pve-user at pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>



More information about the pve-user mailing list