diff for duplicates of <20120503194924.GC7313@localhost.localdomain> diff --git a/a/1.txt b/N1/1.txt index 4326686..01fd86e 100644 --- a/a/1.txt +++ b/N1/1.txt @@ -4,44 +4,61 @@ On Thu, May 03, 2012 at 09:38:27AM -0700, Josh Durgin wrote: > > On Thu, May 03, 2012 at 08:17:43AM -0700, Josh Durgin wrote: > >> On Thu, 3 May 2012 10:13:55 -0400, Josef Bacik <josef@redhat.com> > >> wrote: -> >> > On Fri, Apr 27, 2012 at 01:02:08PM +0200, Christian Brunner wrote: +> >> > On Fri, Apr 27, 2012 at 01:02:08PM +0200, Christian Brunner wrot= +e: > >> >> Am 24. April 2012 18:26 schrieb Sage Weil <sage@newdream.net>: > >> >> > On Tue, 24 Apr 2012, Josef Bacik wrote: -> >> >> >> On Fri, Apr 20, 2012 at 05:09:34PM +0200, Christian Brunner wrote: -> >> >> >> > After running ceph on XFS for some time, I decided to try btrfs again. -> >> >> >> > Performance with the current "for-linux-min" branch and big metadata -> >> >> >> > is much better. The only problem (?) I'm still seeing is a warning +> >> >> >> On Fri, Apr 20, 2012 at 05:09:34PM +0200, Christian Brunner = +wrote: +> >> >> >> > After running ceph on XFS for some time, I decided to try = +btrfs again. +> >> >> >> > Performance with the current "for-linux-min" branch and bi= +g metadata +> >> >> >> > is much better. The only problem (?) I'm still seeing is a= + warning > >> >> >> > that seems to occur from time to time: > >> >> > > >> >> > Actually, before you do that... we have a new tool, -> >> >> > test_filestore_workloadgen, that generates a ceph-osd-like workload on the -> >> >> > local file system. It's a subset of what a full OSD might do, but if -> >> >> > we're lucky it will be sufficient to reproduce this issue. Something like +> >> >> > test_filestore_workloadgen, that generates a ceph-osd-like wo= +rkload on the +> >> >> > local file system. =A0It's a subset of what a full OSD might = +do, but if +> >> >> > we're lucky it will be sufficient to reproduce this issue. =A0= +Something like > >> >> > -> >> >> > test_filestore_workloadgen --osd-data /foo --osd-journal /bar +> >> >> > =A0test_filestore_workloadgen --osd-data /foo --osd-journal /= +bar > >> >> > > >> >> > will hopefully do the trick. > >> >> > -> >> >> > Christian, maybe you can see if that is able to trigger this warning? -> >> >> > You'll need to pull it from the current master branch; it wasn't in the +> >> >> > Christian, maybe you can see if that is able to trigger this = +warning? +> >> >> > You'll need to pull it from the current master branch; it was= +n't in the > >> >> > last release. > >> >> -> >> >> Trying to reproduce with test_filestore_workloadgen didn't work for -> >> >> me. So here are some instructions on how to reproduce with a minimal +> >> >> Trying to reproduce with test_filestore_workloadgen didn't work= + for +> >> >> me. So here are some instructions on how to reproduce with a mi= +nimal > >> >> ceph setup. > >> >> -> >> >> You will need a single system with two disks and a bit of memory. +> >> >> You will need a single system with two disks and a bit of memor= +y. > >> >> > >> >> - Compile and install ceph (detailed instructions: > >> >> http://ceph.newdream.net/docs/master/ops/install/mkcephfs/) > >> >> -> >> >> - For the test setup I've used two tmpfs files as journal devices. To +> >> >> - For the test setup I've used two tmpfs files as journal devic= +es. To > >> >> create these, do the following: > >> >> > >> >> # mkdir -p /ceph/temp > >> >> # mount -t tmpfs tmpfs /ceph/temp -> >> >> # dd if=/dev/zero of=/ceph/temp/journal0 count=500 bs=1024k -> >> >> # dd if=/dev/zero of=/ceph/temp/journal1 count=500 bs=1024k +> >> >> # dd if=3D/dev/zero of=3D/ceph/temp/journal0 count=3D500 bs=3D1= +024k +> >> >> # dd if=3D/dev/zero of=3D/ceph/temp/journal1 count=3D500 bs=3D1= +024k > >> >> > >> >> - Now you should create and mount btrfs. Here is what I did: > >> >> @@ -49,10 +66,13 @@ On Thu, May 03, 2012 at 09:38:27AM -0700, Josh Durgin wrote: > >> >> # mkfs.btrfs -l 64k -n 64k /dev/sdb > >> >> # mkdir /ceph/osd.000 > >> >> # mkdir /ceph/osd.001 -> >> >> # mount -o noatime,space_cache,inode_cache,autodefrag /dev/sda /ceph/osd.000 -> >> >> # mount -o noatime,space_cache,inode_cache,autodefrag /dev/sdb /ceph/osd.001 +> >> >> # mount -o noatime,space_cache,inode_cache,autodefrag /dev/sda = +/ceph/osd.000 +> >> >> # mount -o noatime,space_cache,inode_cache,autodefrag /dev/sdb = +/ceph/osd.001 > >> >> -> >> >> - Create /etc/ceph/ceph.conf similar to the attached ceph.conf. You +> >> >> - Create /etc/ceph/ceph.conf similar to the attached ceph.conf.= + You > >> >> will probably have to change the btrfs devices and the hostname > >> >> (os39). > >> >> @@ -63,29 +83,35 @@ On Thu, May 03, 2012 at 09:38:27AM -0700, Josh Durgin wrote: > >> >> > >> >> - Start ceph (e.g. "service ceph start") > >> >> -> >> >> - Now you should be able to use ceph - "ceph -s" will tell you about +> >> >> - Now you should be able to use ceph - "ceph -s" will tell you = +about > >> >> the state of the ceph cluster. > >> >> -> >> >> - "rbd create -size 100 testimg" will create an rbd image on the ceph cluster. +> >> >> - "rbd create -size 100 testimg" will create an rbd image on th= +e ceph cluster. > >> >> > >> > > >> > It's failing here > >> > > >> > http://fpaste.org/e3BG/ > >> -> >> 2012-05-03 10:11:28.818308 7fcb5a0ee700 -- 127.0.0.1:0/1003269 <== -> >> osd.1 127.0.0.1:6803/2379 3 ==== osd_op_reply(3 rbd_info [call] = -5 -> >> (Input/output error)) v4 ==== 107+0+0 (3948821281 0 0) 0x7fcb380009a0 +> >> 2012-05-03 10:11:28.818308 7fcb5a0ee700 -- 127.0.0.1:0/1003269 <=3D= +=3D +> >> osd.1 127.0.0.1:6803/2379 3 =3D=3D=3D=3D osd_op_reply(3 rbd_info [= +call] =3D -5 +> >> (Input/output error)) v4 =3D=3D=3D=3D 107+0+0 (3948821281 0 0) 0x7= +fcb380009a0 > >> con 0x1cad3e0 > >> > >> This is probably because the osd isn't finding the rbd class. > >> Do you have 'rbd_cls.so' in /usr/lib64/rados-classes? Wherever > >> rbd_cls.so is, -> >> try adding 'osd class dir = /path/to/rados-classes' to the [osd] +> >> try adding 'osd class dir =3D /path/to/rados-classes' to the [osd] > >> section > >> in your ceph.conf, and restarting the osds. > >> -> >> If you set 'debug osd = 10' you should see '_load_class rbd' in the osd +> >> If you set 'debug osd =3D 10' you should see '_load_class rbd' in = +the osd > >> log > >> when you try to create an rbd image. > >> @@ -93,28 +119,32 @@ On Thu, May 03, 2012 at 09:38:27AM -0700, Josh Durgin wrote: > >> you're > >> running the osds in a chroot or something the path would be wrong. > >> -> > +> >=20 > > Yeah all that was in the right place, I rebooted and I magically > > stopped getting > > that error, but now I'm getting this -> > +> >=20 > > http://fpaste.org/OE92/ -> > +> >=20 > > with that ping thing repeating over and over. Thanks, -> +>=20 > That just looks like the osd isn't running. If you restart the -> osd with 'debug osd = 20' the osd log should tell us what's going on. +> osd with 'debug osd =3D 20' the osd log should tell us what's going o= +n. -Ok that part was my fault, Duh I need to redo the tmpfs and mkcephfs stuff after +Ok that part was my fault, Duh I need to redo the tmpfs and mkcephfs st= +uff after reboot. But now I'm back to my original problem http://fpaste.org/PfwO/ -I have the osd class dir = /usr/lib64/rados-classes thing set and libcls_rbd is +I have the osd class dir =3D /usr/lib64/rados-classes thing set and lib= +cls_rbd is in there, so I'm not sure what is wrong. Thanks, Josef -- -To unsubscribe from this list: send the line "unsubscribe ceph-devel" in +To unsubscribe from this list: send the line "unsubscribe ceph-devel" i= +n the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/a/content_digest b/N1/content_digest index 9d0e8e0..972a690 100644 --- a/a/content_digest +++ b/N1/content_digest @@ -23,44 +23,61 @@ "> > On Thu, May 03, 2012 at 08:17:43AM -0700, Josh Durgin wrote:\n" "> >> On Thu, 3 May 2012 10:13:55 -0400, Josef Bacik <josef@redhat.com>\n" "> >> wrote:\n" - "> >> > On Fri, Apr 27, 2012 at 01:02:08PM +0200, Christian Brunner wrote:\n" + "> >> > On Fri, Apr 27, 2012 at 01:02:08PM +0200, Christian Brunner wrot=\n" + "e:\n" "> >> >> Am 24. April 2012 18:26 schrieb Sage Weil <sage@newdream.net>:\n" "> >> >> > On Tue, 24 Apr 2012, Josef Bacik wrote:\n" - "> >> >> >> On Fri, Apr 20, 2012 at 05:09:34PM +0200, Christian Brunner wrote:\n" - "> >> >> >> > After running ceph on XFS for some time, I decided to try btrfs again.\n" - "> >> >> >> > Performance with the current \"for-linux-min\" branch and big metadata\n" - "> >> >> >> > is much better. The only problem (?) I'm still seeing is a warning\n" + "> >> >> >> On Fri, Apr 20, 2012 at 05:09:34PM +0200, Christian Brunner =\n" + "wrote:\n" + "> >> >> >> > After running ceph on XFS for some time, I decided to try =\n" + "btrfs again.\n" + "> >> >> >> > Performance with the current \"for-linux-min\" branch and bi=\n" + "g metadata\n" + "> >> >> >> > is much better. The only problem (?) I'm still seeing is a=\n" + " warning\n" "> >> >> >> > that seems to occur from time to time:\n" "> >> >> >\n" "> >> >> > Actually, before you do that... we have a new tool,\n" - "> >> >> > test_filestore_workloadgen, that generates a ceph-osd-like workload on the\n" - "> >> >> > local file system. \302\240It's a subset of what a full OSD might do, but if\n" - "> >> >> > we're lucky it will be sufficient to reproduce this issue. \302\240Something like\n" + "> >> >> > test_filestore_workloadgen, that generates a ceph-osd-like wo=\n" + "rkload on the\n" + "> >> >> > local file system. =A0It's a subset of what a full OSD might =\n" + "do, but if\n" + "> >> >> > we're lucky it will be sufficient to reproduce this issue. =A0=\n" + "Something like\n" "> >> >> >\n" - "> >> >> > \302\240test_filestore_workloadgen --osd-data /foo --osd-journal /bar\n" + "> >> >> > =A0test_filestore_workloadgen --osd-data /foo --osd-journal /=\n" + "bar\n" "> >> >> >\n" "> >> >> > will hopefully do the trick.\n" "> >> >> >\n" - "> >> >> > Christian, maybe you can see if that is able to trigger this warning?\n" - "> >> >> > You'll need to pull it from the current master branch; it wasn't in the\n" + "> >> >> > Christian, maybe you can see if that is able to trigger this =\n" + "warning?\n" + "> >> >> > You'll need to pull it from the current master branch; it was=\n" + "n't in the\n" "> >> >> > last release.\n" "> >> >>\n" - "> >> >> Trying to reproduce with test_filestore_workloadgen didn't work for\n" - "> >> >> me. So here are some instructions on how to reproduce with a minimal\n" + "> >> >> Trying to reproduce with test_filestore_workloadgen didn't work=\n" + " for\n" + "> >> >> me. So here are some instructions on how to reproduce with a mi=\n" + "nimal\n" "> >> >> ceph setup.\n" "> >> >>\n" - "> >> >> You will need a single system with two disks and a bit of memory.\n" + "> >> >> You will need a single system with two disks and a bit of memor=\n" + "y.\n" "> >> >>\n" "> >> >> - Compile and install ceph (detailed instructions:\n" "> >> >> http://ceph.newdream.net/docs/master/ops/install/mkcephfs/)\n" "> >> >>\n" - "> >> >> - For the test setup I've used two tmpfs files as journal devices. To\n" + "> >> >> - For the test setup I've used two tmpfs files as journal devic=\n" + "es. To\n" "> >> >> create these, do the following:\n" "> >> >>\n" "> >> >> # mkdir -p /ceph/temp\n" "> >> >> # mount -t tmpfs tmpfs /ceph/temp\n" - "> >> >> # dd if=/dev/zero of=/ceph/temp/journal0 count=500 bs=1024k\n" - "> >> >> # dd if=/dev/zero of=/ceph/temp/journal1 count=500 bs=1024k\n" + "> >> >> # dd if=3D/dev/zero of=3D/ceph/temp/journal0 count=3D500 bs=3D1=\n" + "024k\n" + "> >> >> # dd if=3D/dev/zero of=3D/ceph/temp/journal1 count=3D500 bs=3D1=\n" + "024k\n" "> >> >>\n" "> >> >> - Now you should create and mount btrfs. Here is what I did:\n" "> >> >>\n" @@ -68,10 +85,13 @@ "> >> >> # mkfs.btrfs -l 64k -n 64k /dev/sdb\n" "> >> >> # mkdir /ceph/osd.000\n" "> >> >> # mkdir /ceph/osd.001\n" - "> >> >> # mount -o noatime,space_cache,inode_cache,autodefrag /dev/sda /ceph/osd.000\n" - "> >> >> # mount -o noatime,space_cache,inode_cache,autodefrag /dev/sdb /ceph/osd.001\n" + "> >> >> # mount -o noatime,space_cache,inode_cache,autodefrag /dev/sda =\n" + "/ceph/osd.000\n" + "> >> >> # mount -o noatime,space_cache,inode_cache,autodefrag /dev/sdb =\n" + "/ceph/osd.001\n" "> >> >>\n" - "> >> >> - Create /etc/ceph/ceph.conf similar to the attached ceph.conf. You\n" + "> >> >> - Create /etc/ceph/ceph.conf similar to the attached ceph.conf.=\n" + " You\n" "> >> >> will probably have to change the btrfs devices and the hostname\n" "> >> >> (os39).\n" "> >> >>\n" @@ -82,29 +102,35 @@ "> >> >>\n" "> >> >> - Start ceph (e.g. \"service ceph start\")\n" "> >> >>\n" - "> >> >> - Now you should be able to use ceph - \"ceph -s\" will tell you about\n" + "> >> >> - Now you should be able to use ceph - \"ceph -s\" will tell you =\n" + "about\n" "> >> >> the state of the ceph cluster.\n" "> >> >>\n" - "> >> >> - \"rbd create -size 100 testimg\" will create an rbd image on the ceph cluster.\n" + "> >> >> - \"rbd create -size 100 testimg\" will create an rbd image on th=\n" + "e ceph cluster.\n" "> >> >>\n" "> >> >\n" "> >> > It's failing here\n" "> >> >\n" "> >> > http://fpaste.org/e3BG/\n" "> >>\n" - "> >> 2012-05-03 10:11:28.818308 7fcb5a0ee700 -- 127.0.0.1:0/1003269 <==\n" - "> >> osd.1 127.0.0.1:6803/2379 3 ==== osd_op_reply(3 rbd_info [call] = -5\n" - "> >> (Input/output error)) v4 ==== 107+0+0 (3948821281 0 0) 0x7fcb380009a0\n" + "> >> 2012-05-03 10:11:28.818308 7fcb5a0ee700 -- 127.0.0.1:0/1003269 <=3D=\n" + "=3D\n" + "> >> osd.1 127.0.0.1:6803/2379 3 =3D=3D=3D=3D osd_op_reply(3 rbd_info [=\n" + "call] =3D -5\n" + "> >> (Input/output error)) v4 =3D=3D=3D=3D 107+0+0 (3948821281 0 0) 0x7=\n" + "fcb380009a0\n" "> >> con 0x1cad3e0\n" "> >>\n" "> >> This is probably because the osd isn't finding the rbd class.\n" "> >> Do you have 'rbd_cls.so' in /usr/lib64/rados-classes? Wherever\n" "> >> rbd_cls.so is,\n" - "> >> try adding 'osd class dir = /path/to/rados-classes' to the [osd]\n" + "> >> try adding 'osd class dir =3D /path/to/rados-classes' to the [osd]\n" "> >> section\n" "> >> in your ceph.conf, and restarting the osds.\n" "> >>\n" - "> >> If you set 'debug osd = 10' you should see '_load_class rbd' in the osd\n" + "> >> If you set 'debug osd =3D 10' you should see '_load_class rbd' in =\n" + "the osd\n" "> >> log\n" "> >> when you try to create an rbd image.\n" "> >>\n" @@ -112,30 +138,34 @@ "> >> you're\n" "> >> running the osds in a chroot or something the path would be wrong.\n" "> >>\n" - "> > \n" + "> >=20\n" "> > Yeah all that was in the right place, I rebooted and I magically\n" "> > stopped getting\n" "> > that error, but now I'm getting this\n" - "> > \n" + "> >=20\n" "> > http://fpaste.org/OE92/\n" - "> > \n" + "> >=20\n" "> > with that ping thing repeating over and over. Thanks,\n" - "> \n" + ">=20\n" "> That just looks like the osd isn't running. If you restart the\n" - "> osd with 'debug osd = 20' the osd log should tell us what's going on.\n" + "> osd with 'debug osd =3D 20' the osd log should tell us what's going o=\n" + "n.\n" "\n" - "Ok that part was my fault, Duh I need to redo the tmpfs and mkcephfs stuff after\n" + "Ok that part was my fault, Duh I need to redo the tmpfs and mkcephfs st=\n" + "uff after\n" "reboot. But now I'm back to my original problem\n" "\n" "http://fpaste.org/PfwO/\n" "\n" - "I have the osd class dir = /usr/lib64/rados-classes thing set and libcls_rbd is\n" + "I have the osd class dir =3D /usr/lib64/rados-classes thing set and lib=\n" + "cls_rbd is\n" "in there, so I'm not sure what is wrong. Thanks,\n" "\n" "Josef\n" "--\n" - "To unsubscribe from this list: send the line \"unsubscribe ceph-devel\" in\n" + "To unsubscribe from this list: send the line \"unsubscribe ceph-devel\" i=\n" + "n\n" "the body of a message to majordomo@vger.kernel.org\n" More majordomo info at http://vger.kernel.org/majordomo-info.html -8d7b27074571c604a96e2a72b47693737432f2ae89916eb725895b3f23f9d52b +ada076109e609de90fafa51b983c16036dc73a235a1ca657b670b9cbbd9cb93e
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.