Re: FW: Reg:.[CBT] Unable to do a CBT Clean Run.

Re: FW: Reg:.[CBT] Unable to do a CBT Clean Run.

[Mark Nelson]

Hi Vish,

Thanks for the report!

I seem to remember seeing something very similar to this a while back 
and it turned out to be an issue on the monitor where crushtool wasn't 
in the path.

The way I had to solve it was to set the "crushtool" path via ceph.conf:

https://github.com/ceph/ceph/blob/master/src/common/config_opts.h#L29

Please let me know if that helps.

Thanks,
Mark

On 02/05/2016 10:26 AM, Vish (Vishwanath) Maram-SSI wrote:
> Hi,
>
> Sorry I am forwarding again as the mails are not going through the list.
>
> Thanks,
>
> -Vish
>
> *From:*Vish (Vishwanath) Maram-SSI
> *Sent:* Thursday, February 04, 2016 3:45 PM
> *To:* 'ceph-devel-owner@vger.kernel.org'; 'ceph-devel@vger.kernel.org'
> *Cc:* 'Brent Compton'; 'Kyle Bader'
> *Subject:* Reg:.[CBT] Unable to do a CBT Clean Run.
>
> Hi All,
>
> We are trying to do a clean run with CBT and please find below all the
> details:
>
> 1.OS – CentOS7.2
>
> 2.CBT – Commit ID “1203c2b9d25344d1e15b28236ea2f19aa3103e0b” from “git log”
>
> 3.CEPH Code – Hammer Version – 0.94.5
>
> 4.Test.yaml – Please find below
>
> 5.Ceph.conf – Please find below
>
> *__*
>
> *_Issue:_*
>
> Command - ./cbt.py --archive="Archive" --conf=./ceph.conf ./test.yaml >
> cmd_log.txt
>
> After the above command is executed we were able to check the Cluster is
> being created on the Server and able to see the status/OSD Tree as well.
> All those show correct, but we didn’t see any IO going on the disk
> (iostat –t 5). So we started debugging the CBT code ceph.py under
> cluster directory. When we debugged and we observed that pool creation
> is not happening and we inserted break in ceph.py just before pool
> creation and executed the pool creation command on Server and get the
> below error:
>
> *Error EINVAL: error running crushmap through crushtool: (125) Operation
> canceled*
>
> And because of the above issue we weren’t able to see any FIO run.
>
> Appreciate any inputs.
>
> Thanks,
>
> -Vish
>
> Test.yaml –
>
>    1  cluster:
>
>    2   user: 'root'
>
>    3   head: "Server"
>
>    4   clients: ["client"]
>
>    5   osds: ["Server"]
>
>    6   mons:
>
>    7     Mon1:
>
>    8       a: "10.10.10.150:6789"
>
>    9   osds_per_node: 1
>
> 10   fs: 'xfs'
>
> 11   mkfs_opts: '-f -i size=2048 -n size=64k'
>
> 12   mount_opts: '-o inode64,noatime,logbsize=256k'
>
> 13   conf_file: '/usr/local/ceph-cbt/ceph.conf.1osd'
>
> 14   iterations: 1
>
> 15   use_existing: False
>
> 16 #  clusterid: "8eda02e2-04b7-4eed-a85a-8471ea51528c"
>
> 17   clusterid: "cbttest"
>
> 18   tmp_dir: "/tmp/cbt"
>
> 19   pool_profiles:
>
> 20     rbd:
>
> 21       pg_size: 256
>
> 22       pgp_size: 256
>
> 23       replication: 1
>
> 24 benchmarks:
>
> 25   librbdfio:
>
> 26     time: 300
>
> 27     vol_size: 16384
>
> 28     mode: [write]
>
> 29     op_size: [1048576]
>
> 30     concurrent_procs: [1]
>
> 31     iodepth: [64]
>
> 32     osd_ra: [4096]
>
> 33     cmd_path: '/usr/local/bin/fio'
>
> 34     pool_profile: 'rbd'
>
> Ceph.conf –
>
> [global]
>
>          osd pool default size = 1
>
>          auth cluster required = none
>
>          auth service required = none
>
>          auth client required = none
>
>          keyring = /tmp/cbt/ceph/keyring
>
>          osd pg bits = 8
>
>          osd pgp bits = 8
>
>          log to syslog = false
>
>          log file = /tmp/cbt/ceph/log/$name.log
>
>          public network = 10.10.10.0/24
>
>          cluster network = 10.10.10.0/24
>
>          rbd cache = true
>
>          osd scrub load threshold = 0.01
>
>          osd scrub min interval = 137438953472
>
>          osd scrub max interval = 137438953472
>
>          osd deep scrub interval = 137438953472
>
>          osd max scrubs = 16
>
>          filestore merge threshold = 40
>
>          filestore split multiple = 8
>
>          osd op threads = 8
>
>          mon pg warn max object skew = 100000
>
>          mon pg warn min per osd = 0
>
>          mon pg warn max per osd = 32768
>
> [mon]
>
>          mon data = /tmp/cbt/ceph/mon.$id
>
> [mon.a]
>
>          host = Mon1
>
>          mon addr = 10.10.10.150:6789
>
> [osd.0]
>
>          host = Server
>
>          osd data = /tmp/cbt/mnt/osd-device-0-data
>
>          osd journal = /dev/disk/by-partlabel/osd-device-0-journal
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html