* librados AIO problem diagnostic @ 2014-10-01 7:01 Loic Dachary 2014-10-01 7:28 ` Sebastien Ponce 0 siblings, 1 reply; 15+ messages in thread From: Loic Dachary @ 2014-10-01 7:01 UTC (permalink / raw) To: Sebastien Ponce; +Cc: Ceph Development [-- Attachment #1: Type: text/plain, Size: 377 bytes --] Hi Sebastien, In the context of http://tracker.ceph.com/issues/9356 I'm trying to collect as much information as I can in an environment that can be reconstructed to avoid interferences. Could you please let me know which ./configure line you are using ? Or which do_autogen.sh command you are using. Thanks in advance :-) -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 263 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: librados AIO problem diagnostic 2014-10-01 7:01 librados AIO problem diagnostic Loic Dachary @ 2014-10-01 7:28 ` Sebastien Ponce 2014-10-01 8:31 ` Loic Dachary 2014-10-01 8:33 ` Loic Dachary 0 siblings, 2 replies; 15+ messages in thread From: Sebastien Ponce @ 2014-10-01 7:28 UTC (permalink / raw) To: Loic Dachary; +Cc: Ceph Development Thanks for this effort ! I'll look at all that this afternoon (it's mid morning here). Concerning configure and autogen, I do the standard (naive ?) "./autogen.sh; ./configure; make" given by the readme. Sebastien On Wed, 2014-10-01 at 09:01 +0200, Loic Dachary wrote: > Hi Sebastien, > > In the context of http://tracker.ceph.com/issues/9356 I'm trying to collect as much information as I can in an environment that can be reconstructed to avoid interferences. Could you please let me know which ./configure line you are using ? Or which do_autogen.sh command you are using. > > Thanks in advance :-) > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: librados AIO problem diagnostic 2014-10-01 7:28 ` Sebastien Ponce @ 2014-10-01 8:31 ` Loic Dachary 2014-10-01 11:37 ` Sebastien Ponce 2014-10-01 8:33 ` Loic Dachary 1 sibling, 1 reply; 15+ messages in thread From: Loic Dachary @ 2014-10-01 8:31 UTC (permalink / raw) To: Sebastien Ponce; +Cc: Ceph Development [-- Attachment #1: Type: text/plain, Size: 1482 bytes --] Hi, On 01/10/2014 09:28, Sebastien Ponce wrote: > Thanks for this effort ! > I'll look at all that this afternoon (it's mid morning here). Same here : I'm in Paris ;-) > Concerning configure and autogen, I do the standard (naive ?) > "./autogen.sh; ./configure; make" given by the readme. Could you try with ./autogen.sh ; ./configure --disable-static --with-debug CC='ccache gcc' CXX='ccache g++' CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" ; make and let me know if you see a problem when running cd src rm -fr dev out ; mkdir -p dev ; CEPH_NUM_MON=1 CEPH_NUM_OSD=3 ./vstart.sh -d -n -X -l mon osd ./ceph_test_rados_striper_api_aio I'm trying with your "./configure" on my end to check if this changes something. It's going to be a fun one ;-) Cheers > > Sebastien > > > On Wed, 2014-10-01 at 09:01 +0200, Loic Dachary wrote: >> Hi Sebastien, >> >> In the context of http://tracker.ceph.com/issues/9356 I'm trying to collect as much information as I can in an environment that can be reconstructed to avoid interferences. Could you please let me know which ./configure line you are using ? Or which do_autogen.sh command you are using. >> >> Thanks in advance :-) >> > > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 263 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: librados AIO problem diagnostic 2014-10-01 8:31 ` Loic Dachary @ 2014-10-01 11:37 ` Sebastien Ponce 2014-10-01 13:08 ` Loic Dachary 0 siblings, 1 reply; 15+ messages in thread From: Sebastien Ponce @ 2014-10-01 11:37 UTC (permalink / raw) To: Loic Dachary; +Cc: Ceph Development I've tried it on the RHEL6 like machine, but no luck. Or should I say I was lucky ? Anyway no crash. Discussing with Dan Van Der Ster, he told me that he had once the case where the crash was there or not depending on the libc version. Just in case, what are you using ? On the machine I've used this morning, it's glibc-2.12-1.107.el6.x86_64 Sebastien On Wed, 2014-10-01 at 10:31 +0200, Loic Dachary wrote: > Hi, > > On 01/10/2014 09:28, Sebastien Ponce wrote: > > Thanks for this effort ! > > I'll look at all that this afternoon (it's mid morning here). > > Same here : I'm in Paris ;-) > > > Concerning configure and autogen, I do the standard (naive ?) > > "./autogen.sh; ./configure; make" given by the readme. > > Could you try with > > ./autogen.sh ; ./configure --disable-static --with-debug CC='ccache gcc' CXX='ccache g++' CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" ; make > > and let me know if you see a problem when running > > cd src > rm -fr dev out ; mkdir -p dev ; CEPH_NUM_MON=1 CEPH_NUM_OSD=3 ./vstart.sh -d -n -X -l mon osd > ./ceph_test_rados_striper_api_aio > > I'm trying with your "./configure" on my end to check if this changes something. > > It's going to be a fun one ;-) > > Cheers > > > > > Sebastien > > > > > > On Wed, 2014-10-01 at 09:01 +0200, Loic Dachary wrote: > >> Hi Sebastien, > >> > >> In the context of http://tracker.ceph.com/issues/9356 I'm trying to collect as much information as I can in an environment that can be reconstructed to avoid interferences. Could you please let me know which ./configure line you are using ? Or which do_autogen.sh command you are using. > >> > >> Thanks in advance :-) > >> > > > > > > -- > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: librados AIO problem diagnostic 2014-10-01 11:37 ` Sebastien Ponce @ 2014-10-01 13:08 ` Loic Dachary 2014-10-01 13:09 ` Sebastien Ponce 0 siblings, 1 reply; 15+ messages in thread From: Loic Dachary @ 2014-10-01 13:08 UTC (permalink / raw) To: Sebastien Ponce; +Cc: Ceph Development [-- Attachment #1: Type: text/plain, Size: 2845 bytes --] On 01/10/2014 13:37, Sebastien Ponce wrote: > I've tried it on the RHEL6 like machine, but no luck. Or should I say I > was lucky ? Anyway no crash. > > Discussing with Dan Van Der Ster, he told me that he had once the case > where the crash was there or not depending on the libc version. Just in > case, what are you using ? On the machine I've used this morning, it's > glibc-2.12-1.107.el6.x86_64 I'm using ii libc-bin 2.19-0ubuntu3 and I do not have glibc installed. I'm quite sure you can reproduce the problem on a fresh Ubuntu 14.04 with the following ./autogen.sh ; ./configure --disable-static --with-debug CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" it fails consistently in my container. When I try with ./configure it does not fail. Something is going on when there is -g instead of the default flags. I checked that --disable-static and --with-debug make no difference. Cheers > > Sebastien > > On Wed, 2014-10-01 at 10:31 +0200, Loic Dachary wrote: >> Hi, >> >> On 01/10/2014 09:28, Sebastien Ponce wrote: >>> Thanks for this effort ! >>> I'll look at all that this afternoon (it's mid morning here). >> >> Same here : I'm in Paris ;-) >> >>> Concerning configure and autogen, I do the standard (naive ?) >>> "./autogen.sh; ./configure; make" given by the readme. >> >> Could you try with >> >> ./autogen.sh ; ./configure --disable-static --with-debug CC='ccache gcc' CXX='ccache g++' CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" ; make >> >> and let me know if you see a problem when running >> >> cd src >> rm -fr dev out ; mkdir -p dev ; CEPH_NUM_MON=1 CEPH_NUM_OSD=3 ./vstart.sh -d -n -X -l mon osd >> ./ceph_test_rados_striper_api_aio >> >> I'm trying with your "./configure" on my end to check if this changes something. >> >> It's going to be a fun one ;-) >> >> Cheers >> >>> >>> Sebastien >>> >>> >>> On Wed, 2014-10-01 at 09:01 +0200, Loic Dachary wrote: >>>> Hi Sebastien, >>>> >>>> In the context of http://tracker.ceph.com/issues/9356 I'm trying to collect as much information as I can in an environment that can be reconstructed to avoid interferences. Could you please let me know which ./configure line you are using ? Or which do_autogen.sh command you are using. >>>> >>>> Thanks in advance :-) >>>> >>> >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >> > > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 263 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: librados AIO problem diagnostic 2014-10-01 13:08 ` Loic Dachary @ 2014-10-01 13:09 ` Sebastien Ponce 2014-10-02 9:49 ` Sebastien Ponce 0 siblings, 1 reply; 15+ messages in thread From: Sebastien Ponce @ 2014-10-01 13:09 UTC (permalink / raw) To: Loic Dachary; +Cc: Ceph Development Ok, we are getting closer then. I'll try to reinstall the ubuntu machine and run with -g. Sebastien On Wed, 2014-10-01 at 15:08 +0200, Loic Dachary wrote: > > On 01/10/2014 13:37, Sebastien Ponce wrote: > > I've tried it on the RHEL6 like machine, but no luck. Or should I say I > > was lucky ? Anyway no crash. > > > > Discussing with Dan Van Der Ster, he told me that he had once the case > > where the crash was there or not depending on the libc version. Just in > > case, what are you using ? On the machine I've used this morning, it's > > glibc-2.12-1.107.el6.x86_64 > > I'm using > > ii libc-bin 2.19-0ubuntu3 > > and I do not have glibc installed. I'm quite sure you can reproduce the problem on a fresh Ubuntu 14.04 with the following > > ./autogen.sh ; ./configure --disable-static --with-debug CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" > > it fails consistently in my container. When I try with ./configure it does not fail. Something is going on when there is -g instead of the default flags. I checked that --disable-static and --with-debug make no difference. > > Cheers > > > > > Sebastien > > > > On Wed, 2014-10-01 at 10:31 +0200, Loic Dachary wrote: > >> Hi, > >> > >> On 01/10/2014 09:28, Sebastien Ponce wrote: > >>> Thanks for this effort ! > >>> I'll look at all that this afternoon (it's mid morning here). > >> > >> Same here : I'm in Paris ;-) > >> > >>> Concerning configure and autogen, I do the standard (naive ?) > >>> "./autogen.sh; ./configure; make" given by the readme. > >> > >> Could you try with > >> > >> ./autogen.sh ; ./configure --disable-static --with-debug CC='ccache gcc' CXX='ccache g++' CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" ; make > >> > >> and let me know if you see a problem when running > >> > >> cd src > >> rm -fr dev out ; mkdir -p dev ; CEPH_NUM_MON=1 CEPH_NUM_OSD=3 ./vstart.sh -d -n -X -l mon osd > >> ./ceph_test_rados_striper_api_aio > >> > >> I'm trying with your "./configure" on my end to check if this changes something. > >> > >> It's going to be a fun one ;-) > >> > >> Cheers > >> > >>> > >>> Sebastien > >>> > >>> > >>> On Wed, 2014-10-01 at 09:01 +0200, Loic Dachary wrote: > >>>> Hi Sebastien, > >>>> > >>>> In the context of http://tracker.ceph.com/issues/9356 I'm trying to collect as much information as I can in an environment that can be reconstructed to avoid interferences. Could you please let me know which ./configure line you are using ? Or which do_autogen.sh command you are using. > >>>> > >>>> Thanks in advance :-) > >>>> > >>> > >>> > >>> -- > >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > >>> the body of a message to majordomo@vger.kernel.org > >>> More majordomo info at http://vger.kernel.org/majordomo-info.html > >>> > >> > > > > > > -- > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: librados AIO problem diagnostic 2014-10-01 13:09 ` Sebastien Ponce @ 2014-10-02 9:49 ` Sebastien Ponce 2014-10-02 14:31 ` Sebastien Ponce 0 siblings, 1 reply; 15+ messages in thread From: Sebastien Ponce @ 2014-10-02 9:49 UTC (permalink / raw) To: Loic Dachary; +Cc: Ceph Development Hurray, I manage to seg fault ! (sounds odd this sentence...) So indeed, fresh ubuntu, debug mode does it. What I've also changed in the meantime is the type of machine. It's still virtual, but more powerful (4 cores). Let's debug it now... Sebastien On Wed, 2014-10-01 at 15:09 +0200, Sebastien Ponce wrote: > Ok, we are getting closer then. > I'll try to reinstall the ubuntu machine and run with -g. > > Sebastien > > On Wed, 2014-10-01 at 15:08 +0200, Loic Dachary wrote: > > > > On 01/10/2014 13:37, Sebastien Ponce wrote: > > > I've tried it on the RHEL6 like machine, but no luck. Or should I say I > > > was lucky ? Anyway no crash. > > > > > > Discussing with Dan Van Der Ster, he told me that he had once the case > > > where the crash was there or not depending on the libc version. Just in > > > case, what are you using ? On the machine I've used this morning, it's > > > glibc-2.12-1.107.el6.x86_64 > > > > I'm using > > > > ii libc-bin 2.19-0ubuntu3 > > > > and I do not have glibc installed. I'm quite sure you can reproduce the problem on a fresh Ubuntu 14.04 with the following > > > > ./autogen.sh ; ./configure --disable-static --with-debug CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" > > > > it fails consistently in my container. When I try with ./configure it does not fail. Something is going on when there is -g instead of the default flags. I checked that --disable-static and --with-debug make no difference. > > > > Cheers > > > > > > > > Sebastien > > > > > > On Wed, 2014-10-01 at 10:31 +0200, Loic Dachary wrote: > > >> Hi, > > >> > > >> On 01/10/2014 09:28, Sebastien Ponce wrote: > > >>> Thanks for this effort ! > > >>> I'll look at all that this afternoon (it's mid morning here). > > >> > > >> Same here : I'm in Paris ;-) > > >> > > >>> Concerning configure and autogen, I do the standard (naive ?) > > >>> "./autogen.sh; ./configure; make" given by the readme. > > >> > > >> Could you try with > > >> > > >> ./autogen.sh ; ./configure --disable-static --with-debug CC='ccache gcc' CXX='ccache g++' CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" ; make > > >> > > >> and let me know if you see a problem when running > > >> > > >> cd src > > >> rm -fr dev out ; mkdir -p dev ; CEPH_NUM_MON=1 CEPH_NUM_OSD=3 ./vstart.sh -d -n -X -l mon osd > > >> ./ceph_test_rados_striper_api_aio > > >> > > >> I'm trying with your "./configure" on my end to check if this changes something. > > >> > > >> It's going to be a fun one ;-) > > >> > > >> Cheers > > >> > > >>> > > >>> Sebastien > > >>> > > >>> > > >>> On Wed, 2014-10-01 at 09:01 +0200, Loic Dachary wrote: > > >>>> Hi Sebastien, > > >>>> > > >>>> In the context of http://tracker.ceph.com/issues/9356 I'm trying to collect as much information as I can in an environment that can be reconstructed to avoid interferences. Could you please let me know which ./configure line you are using ? Or which do_autogen.sh command you are using. > > >>>> > > >>>> Thanks in advance :-) > > >>>> > > >>> > > >>> > > >>> -- > > >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > >>> the body of a message to majordomo@vger.kernel.org > > >>> More majordomo info at http://vger.kernel.org/majordomo-info.html > > >>> > > >> > > > > > > > > > -- > > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > > the body of a message to majordomo@vger.kernel.org > > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > > > > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: librados AIO problem diagnostic 2014-10-02 9:49 ` Sebastien Ponce @ 2014-10-02 14:31 ` Sebastien Ponce 2014-10-02 14:58 ` Loic Dachary 0 siblings, 1 reply; 15+ messages in thread From: Sebastien Ponce @ 2014-10-02 14:31 UTC (permalink / raw) To: Loic Dachary; +Cc: Ceph Development As mentioned in the bug, I was a bit fast in concluding I could reproduce the problem. I can reproduce the original one, but not the remaining issue once the fix was applied. Loic, can you confirm that you are using the wip-9356 branch of repo git@github.com:sponce/ceph.git ? Sebastien On Thu, 2014-10-02 at 11:49 +0200, Sebastien Ponce wrote: > Hurray, I manage to seg fault ! (sounds odd this sentence...) > So indeed, fresh ubuntu, debug mode does it. What I've also changed in > the meantime is the type of machine. It's still virtual, but more > powerful (4 cores). > > Let's debug it now... > > Sebastien > > > On Wed, 2014-10-01 at 15:09 +0200, Sebastien Ponce wrote: > > Ok, we are getting closer then. > > I'll try to reinstall the ubuntu machine and run with -g. > > > > Sebastien > > > > On Wed, 2014-10-01 at 15:08 +0200, Loic Dachary wrote: > > > > > > On 01/10/2014 13:37, Sebastien Ponce wrote: > > > > I've tried it on the RHEL6 like machine, but no luck. Or should I say I > > > > was lucky ? Anyway no crash. > > > > > > > > Discussing with Dan Van Der Ster, he told me that he had once the case > > > > where the crash was there or not depending on the libc version. Just in > > > > case, what are you using ? On the machine I've used this morning, it's > > > > glibc-2.12-1.107.el6.x86_64 > > > > > > I'm using > > > > > > ii libc-bin 2.19-0ubuntu3 > > > > > > and I do not have glibc installed. I'm quite sure you can reproduce the problem on a fresh Ubuntu 14.04 with the following > > > > > > ./autogen.sh ; ./configure --disable-static --with-debug CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" > > > > > > it fails consistently in my container. When I try with ./configure it does not fail. Something is going on when there is -g instead of the default flags. I checked that --disable-static and --with-debug make no difference. > > > > > > Cheers > > > > > > > > > > > Sebastien > > > > > > > > On Wed, 2014-10-01 at 10:31 +0200, Loic Dachary wrote: > > > >> Hi, > > > >> > > > >> On 01/10/2014 09:28, Sebastien Ponce wrote: > > > >>> Thanks for this effort ! > > > >>> I'll look at all that this afternoon (it's mid morning here). > > > >> > > > >> Same here : I'm in Paris ;-) > > > >> > > > >>> Concerning configure and autogen, I do the standard (naive ?) > > > >>> "./autogen.sh; ./configure; make" given by the readme. > > > >> > > > >> Could you try with > > > >> > > > >> ./autogen.sh ; ./configure --disable-static --with-debug CC='ccache gcc' CXX='ccache g++' CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" ; make > > > >> > > > >> and let me know if you see a problem when running > > > >> > > > >> cd src > > > >> rm -fr dev out ; mkdir -p dev ; CEPH_NUM_MON=1 CEPH_NUM_OSD=3 ./vstart.sh -d -n -X -l mon osd > > > >> ./ceph_test_rados_striper_api_aio > > > >> > > > >> I'm trying with your "./configure" on my end to check if this changes something. > > > >> > > > >> It's going to be a fun one ;-) > > > >> > > > >> Cheers > > > >> > > > >>> > > > >>> Sebastien > > > >>> > > > >>> > > > >>> On Wed, 2014-10-01 at 09:01 +0200, Loic Dachary wrote: > > > >>>> Hi Sebastien, > > > >>>> > > > >>>> In the context of http://tracker.ceph.com/issues/9356 I'm trying to collect as much information as I can in an environment that can be reconstructed to avoid interferences. Could you please let me know which ./configure line you are using ? Or which do_autogen.sh command you are using. > > > >>>> > > > >>>> Thanks in advance :-) > > > >>>> > > > >>> > > > >>> > > > >>> -- > > > >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > > >>> the body of a message to majordomo@vger.kernel.org > > > >>> More majordomo info at http://vger.kernel.org/majordomo-info.html > > > >>> > > > >> > > > > > > > > > > > > -- > > > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > > > the body of a message to majordomo@vger.kernel.org > > > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > > > > > > > > > > -- > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: librados AIO problem diagnostic 2014-10-02 14:31 ` Sebastien Ponce @ 2014-10-02 14:58 ` Loic Dachary 2014-10-08 7:14 ` Loic Dachary 0 siblings, 1 reply; 15+ messages in thread From: Loic Dachary @ 2014-10-02 14:58 UTC (permalink / raw) To: Sebastien Ponce; +Cc: Ceph Development [-- Attachment #1: Type: text/plain, Size: 4975 bytes --] On 02/10/2014 16:31, Sebastien Ponce wrote: > As mentioned in the bug, I was a bit fast in concluding I could > reproduce the problem. I can reproduce the original one, but not the > remaining issue once the fix was applied. > > Loic, can you confirm that you are using the wip-9356 branch of repo > git@github.com:sponce/ceph.git ? Yes. I'm able to reproduce the problem with 0daddfbf1164d6ba3f38eee29d2f11acfa62f2b6 from your tree https://github.com/sponce/ceph.git which is the head of wip-9356 Cheers > > Sebastien > > > On Thu, 2014-10-02 at 11:49 +0200, Sebastien Ponce wrote: >> Hurray, I manage to seg fault ! (sounds odd this sentence...) >> So indeed, fresh ubuntu, debug mode does it. What I've also changed in >> the meantime is the type of machine. It's still virtual, but more >> powerful (4 cores). >> >> Let's debug it now... >> >> Sebastien >> >> >> On Wed, 2014-10-01 at 15:09 +0200, Sebastien Ponce wrote: >>> Ok, we are getting closer then. >>> I'll try to reinstall the ubuntu machine and run with -g. >>> >>> Sebastien >>> >>> On Wed, 2014-10-01 at 15:08 +0200, Loic Dachary wrote: >>>> >>>> On 01/10/2014 13:37, Sebastien Ponce wrote: >>>>> I've tried it on the RHEL6 like machine, but no luck. Or should I say I >>>>> was lucky ? Anyway no crash. >>>>> >>>>> Discussing with Dan Van Der Ster, he told me that he had once the case >>>>> where the crash was there or not depending on the libc version. Just in >>>>> case, what are you using ? On the machine I've used this morning, it's >>>>> glibc-2.12-1.107.el6.x86_64 >>>> >>>> I'm using >>>> >>>> ii libc-bin 2.19-0ubuntu3 >>>> >>>> and I do not have glibc installed. I'm quite sure you can reproduce the problem on a fresh Ubuntu 14.04 with the following >>>> >>>> ./autogen.sh ; ./configure --disable-static --with-debug CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" >>>> >>>> it fails consistently in my container. When I try with ./configure it does not fail. Something is going on when there is -g instead of the default flags. I checked that --disable-static and --with-debug make no difference. >>>> >>>> Cheers >>>> >>>>> >>>>> Sebastien >>>>> >>>>> On Wed, 2014-10-01 at 10:31 +0200, Loic Dachary wrote: >>>>>> Hi, >>>>>> >>>>>> On 01/10/2014 09:28, Sebastien Ponce wrote: >>>>>>> Thanks for this effort ! >>>>>>> I'll look at all that this afternoon (it's mid morning here). >>>>>> >>>>>> Same here : I'm in Paris ;-) >>>>>> >>>>>>> Concerning configure and autogen, I do the standard (naive ?) >>>>>>> "./autogen.sh; ./configure; make" given by the readme. >>>>>> >>>>>> Could you try with >>>>>> >>>>>> ./autogen.sh ; ./configure --disable-static --with-debug CC='ccache gcc' CXX='ccache g++' CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" ; make >>>>>> >>>>>> and let me know if you see a problem when running >>>>>> >>>>>> cd src >>>>>> rm -fr dev out ; mkdir -p dev ; CEPH_NUM_MON=1 CEPH_NUM_OSD=3 ./vstart.sh -d -n -X -l mon osd >>>>>> ./ceph_test_rados_striper_api_aio >>>>>> >>>>>> I'm trying with your "./configure" on my end to check if this changes something. >>>>>> >>>>>> It's going to be a fun one ;-) >>>>>> >>>>>> Cheers >>>>>> >>>>>>> >>>>>>> Sebastien >>>>>>> >>>>>>> >>>>>>> On Wed, 2014-10-01 at 09:01 +0200, Loic Dachary wrote: >>>>>>>> Hi Sebastien, >>>>>>>> >>>>>>>> In the context of http://tracker.ceph.com/issues/9356 I'm trying to collect as much information as I can in an environment that can be reconstructed to avoid interferences. Could you please let me know which ./configure line you are using ? Or which do_autogen.sh command you are using. >>>>>>>> >>>>>>>> Thanks in advance :-) >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>>>> the body of a message to majordomo@vger.kernel.org >>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>> the body of a message to majordomo@vger.kernel.org >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>> >>>> >>> >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 263 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: librados AIO problem diagnostic 2014-10-02 14:58 ` Loic Dachary @ 2014-10-08 7:14 ` Loic Dachary 2014-10-08 8:00 ` Sebastien Ponce 0 siblings, 1 reply; 15+ messages in thread From: Loic Dachary @ 2014-10-08 7:14 UTC (permalink / raw) To: Sebastien Ponce; +Cc: Ceph Development [-- Attachment #1: Type: text/plain, Size: 5272 bytes --] Hi Sebastien, Have you been able to reproduce it or did it vanish again ? This is a tricky one ;-) Cheers On 02/10/2014 16:58, Loic Dachary wrote: > > > On 02/10/2014 16:31, Sebastien Ponce wrote: >> As mentioned in the bug, I was a bit fast in concluding I could >> reproduce the problem. I can reproduce the original one, but not the >> remaining issue once the fix was applied. >> >> Loic, can you confirm that you are using the wip-9356 branch of repo >> git@github.com:sponce/ceph.git ? > > Yes. I'm able to reproduce the problem with 0daddfbf1164d6ba3f38eee29d2f11acfa62f2b6 from your tree https://github.com/sponce/ceph.git which is the head of wip-9356 > > Cheers > >> >> Sebastien >> >> >> On Thu, 2014-10-02 at 11:49 +0200, Sebastien Ponce wrote: >>> Hurray, I manage to seg fault ! (sounds odd this sentence...) >>> So indeed, fresh ubuntu, debug mode does it. What I've also changed in >>> the meantime is the type of machine. It's still virtual, but more >>> powerful (4 cores). >>> >>> Let's debug it now... >>> >>> Sebastien >>> >>> >>> On Wed, 2014-10-01 at 15:09 +0200, Sebastien Ponce wrote: >>>> Ok, we are getting closer then. >>>> I'll try to reinstall the ubuntu machine and run with -g. >>>> >>>> Sebastien >>>> >>>> On Wed, 2014-10-01 at 15:08 +0200, Loic Dachary wrote: >>>>> >>>>> On 01/10/2014 13:37, Sebastien Ponce wrote: >>>>>> I've tried it on the RHEL6 like machine, but no luck. Or should I say I >>>>>> was lucky ? Anyway no crash. >>>>>> >>>>>> Discussing with Dan Van Der Ster, he told me that he had once the case >>>>>> where the crash was there or not depending on the libc version. Just in >>>>>> case, what are you using ? On the machine I've used this morning, it's >>>>>> glibc-2.12-1.107.el6.x86_64 >>>>> >>>>> I'm using >>>>> >>>>> ii libc-bin 2.19-0ubuntu3 >>>>> >>>>> and I do not have glibc installed. I'm quite sure you can reproduce the problem on a fresh Ubuntu 14.04 with the following >>>>> >>>>> ./autogen.sh ; ./configure --disable-static --with-debug CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" >>>>> >>>>> it fails consistently in my container. When I try with ./configure it does not fail. Something is going on when there is -g instead of the default flags. I checked that --disable-static and --with-debug make no difference. >>>>> >>>>> Cheers >>>>> >>>>>> >>>>>> Sebastien >>>>>> >>>>>> On Wed, 2014-10-01 at 10:31 +0200, Loic Dachary wrote: >>>>>>> Hi, >>>>>>> >>>>>>> On 01/10/2014 09:28, Sebastien Ponce wrote: >>>>>>>> Thanks for this effort ! >>>>>>>> I'll look at all that this afternoon (it's mid morning here). >>>>>>> >>>>>>> Same here : I'm in Paris ;-) >>>>>>> >>>>>>>> Concerning configure and autogen, I do the standard (naive ?) >>>>>>>> "./autogen.sh; ./configure; make" given by the readme. >>>>>>> >>>>>>> Could you try with >>>>>>> >>>>>>> ./autogen.sh ; ./configure --disable-static --with-debug CC='ccache gcc' CXX='ccache g++' CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" ; make >>>>>>> >>>>>>> and let me know if you see a problem when running >>>>>>> >>>>>>> cd src >>>>>>> rm -fr dev out ; mkdir -p dev ; CEPH_NUM_MON=1 CEPH_NUM_OSD=3 ./vstart.sh -d -n -X -l mon osd >>>>>>> ./ceph_test_rados_striper_api_aio >>>>>>> >>>>>>> I'm trying with your "./configure" on my end to check if this changes something. >>>>>>> >>>>>>> It's going to be a fun one ;-) >>>>>>> >>>>>>> Cheers >>>>>>> >>>>>>>> >>>>>>>> Sebastien >>>>>>>> >>>>>>>> >>>>>>>> On Wed, 2014-10-01 at 09:01 +0200, Loic Dachary wrote: >>>>>>>>> Hi Sebastien, >>>>>>>>> >>>>>>>>> In the context of http://tracker.ceph.com/issues/9356 I'm trying to collect as much information as I can in an environment that can be reconstructed to avoid interferences. Could you please let me know which ./configure line you are using ? Or which do_autogen.sh command you are using. >>>>>>>>> >>>>>>>>> Thanks in advance :-) >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>>>>> the body of a message to majordomo@vger.kernel.org >>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>>> the body of a message to majordomo@vger.kernel.org >>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>> >>>>> >>>> >>>> >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>> the body of a message to majordomo@vger.kernel.org >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 263 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: librados AIO problem diagnostic 2014-10-08 7:14 ` Loic Dachary @ 2014-10-08 8:00 ` Sebastien Ponce 2014-10-09 14:06 ` Sebastien Ponce 0 siblings, 1 reply; 15+ messages in thread From: Sebastien Ponce @ 2014-10-08 8:00 UTC (permalink / raw) To: Loic Dachary; +Cc: Ceph Development It vanished again.... I've spent quite some time analysing the helgrind output to try to find it, but so far I got nothing but problems outside this part (BTW, it would be interesting to run tests systematically in helgrind, even if there is a lot of false positive). Now I've just seen the backtrace you've sent and this already helps a lot; I did not even have a backtrace so far. In conclusion, still ongoing.... Sebastien On Wed, 2014-10-08 at 09:14 +0200, Loic Dachary wrote: > Hi Sebastien, > > Have you been able to reproduce it or did it vanish again ? This is a tricky one ;-) > > Cheers > > On 02/10/2014 16:58, Loic Dachary wrote: > > > > > > On 02/10/2014 16:31, Sebastien Ponce wrote: > >> As mentioned in the bug, I was a bit fast in concluding I could > >> reproduce the problem. I can reproduce the original one, but not the > >> remaining issue once the fix was applied. > >> > >> Loic, can you confirm that you are using the wip-9356 branch of repo > >> git@github.com:sponce/ceph.git ? > > > > Yes. I'm able to reproduce the problem with 0daddfbf1164d6ba3f38eee29d2f11acfa62f2b6 from your tree https://github.com/sponce/ceph.git which is the head of wip-9356 > > > > Cheers > > > >> > >> Sebastien > >> > >> > >> On Thu, 2014-10-02 at 11:49 +0200, Sebastien Ponce wrote: > >>> Hurray, I manage to seg fault ! (sounds odd this sentence...) > >>> So indeed, fresh ubuntu, debug mode does it. What I've also changed in > >>> the meantime is the type of machine. It's still virtual, but more > >>> powerful (4 cores). > >>> > >>> Let's debug it now... > >>> > >>> Sebastien > >>> > >>> > >>> On Wed, 2014-10-01 at 15:09 +0200, Sebastien Ponce wrote: > >>>> Ok, we are getting closer then. > >>>> I'll try to reinstall the ubuntu machine and run with -g. > >>>> > >>>> Sebastien > >>>> > >>>> On Wed, 2014-10-01 at 15:08 +0200, Loic Dachary wrote: > >>>>> > >>>>> On 01/10/2014 13:37, Sebastien Ponce wrote: > >>>>>> I've tried it on the RHEL6 like machine, but no luck. Or should I say I > >>>>>> was lucky ? Anyway no crash. > >>>>>> > >>>>>> Discussing with Dan Van Der Ster, he told me that he had once the case > >>>>>> where the crash was there or not depending on the libc version. Just in > >>>>>> case, what are you using ? On the machine I've used this morning, it's > >>>>>> glibc-2.12-1.107.el6.x86_64 > >>>>> > >>>>> I'm using > >>>>> > >>>>> ii libc-bin 2.19-0ubuntu3 > >>>>> > >>>>> and I do not have glibc installed. I'm quite sure you can reproduce the problem on a fresh Ubuntu 14.04 with the following > >>>>> > >>>>> ./autogen.sh ; ./configure --disable-static --with-debug CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" > >>>>> > >>>>> it fails consistently in my container. When I try with ./configure it does not fail. Something is going on when there is -g instead of the default flags. I checked that --disable-static and --with-debug make no difference. > >>>>> > >>>>> Cheers > >>>>> > >>>>>> > >>>>>> Sebastien > >>>>>> > >>>>>> On Wed, 2014-10-01 at 10:31 +0200, Loic Dachary wrote: > >>>>>>> Hi, > >>>>>>> > >>>>>>> On 01/10/2014 09:28, Sebastien Ponce wrote: > >>>>>>>> Thanks for this effort ! > >>>>>>>> I'll look at all that this afternoon (it's mid morning here). > >>>>>>> > >>>>>>> Same here : I'm in Paris ;-) > >>>>>>> > >>>>>>>> Concerning configure and autogen, I do the standard (naive ?) > >>>>>>>> "./autogen.sh; ./configure; make" given by the readme. > >>>>>>> > >>>>>>> Could you try with > >>>>>>> > >>>>>>> ./autogen.sh ; ./configure --disable-static --with-debug CC='ccache gcc' CXX='ccache g++' CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" ; make > >>>>>>> > >>>>>>> and let me know if you see a problem when running > >>>>>>> > >>>>>>> cd src > >>>>>>> rm -fr dev out ; mkdir -p dev ; CEPH_NUM_MON=1 CEPH_NUM_OSD=3 ./vstart.sh -d -n -X -l mon osd > >>>>>>> ./ceph_test_rados_striper_api_aio > >>>>>>> > >>>>>>> I'm trying with your "./configure" on my end to check if this changes something. > >>>>>>> > >>>>>>> It's going to be a fun one ;-) > >>>>>>> > >>>>>>> Cheers > >>>>>>> > >>>>>>>> > >>>>>>>> Sebastien > >>>>>>>> > >>>>>>>> > >>>>>>>> On Wed, 2014-10-01 at 09:01 +0200, Loic Dachary wrote: > >>>>>>>>> Hi Sebastien, > >>>>>>>>> > >>>>>>>>> In the context of http://tracker.ceph.com/issues/9356 I'm trying to collect as much information as I can in an environment that can be reconstructed to avoid interferences. Could you please let me know which ./configure line you are using ? Or which do_autogen.sh command you are using. > >>>>>>>>> > >>>>>>>>> Thanks in advance :-) > >>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> -- > >>>>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > >>>>>>>> the body of a message to majordomo@vger.kernel.org > >>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > >>>>>> the body of a message to majordomo@vger.kernel.org > >>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html > >>>>>> > >>>>> > >>>> > >>>> > >>>> -- > >>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > >>>> the body of a message to majordomo@vger.kernel.org > >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html > >>> > >>> > >>> -- > >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > >>> the body of a message to majordomo@vger.kernel.org > >>> More majordomo info at http://vger.kernel.org/majordomo-info.html > >> > >> > >> -- > >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > >> the body of a message to majordomo@vger.kernel.org > >> More majordomo info at http://vger.kernel.org/majordomo-info.html > >> > > > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: librados AIO problem diagnostic 2014-10-08 8:00 ` Sebastien Ponce @ 2014-10-09 14:06 ` Sebastien Ponce 2014-10-09 14:24 ` Loic Dachary 0 siblings, 1 reply; 15+ messages in thread From: Sebastien Ponce @ 2014-10-09 14:06 UTC (permalink / raw) To: Loic Dachary; +Cc: Ceph Development Finally this should be fixed. Loic, could you retry with the new state of my branch and confirm ? Sebastien On Wed, 2014-10-08 at 10:00 +0200, Sebastien Ponce wrote: > It vanished again.... > I've spent quite some time analysing the helgrind output to try to find > it, but so far I got nothing but problems outside this part (BTW, it > would be interesting to run tests systematically in helgrind, even if > there is a lot of false positive). > Now I've just seen the backtrace you've sent and this already helps a > lot; I did not even have a backtrace so far. > In conclusion, still ongoing.... > > Sebastien > > > On Wed, 2014-10-08 at 09:14 +0200, Loic Dachary wrote: > > Hi Sebastien, > > > > Have you been able to reproduce it or did it vanish again ? This is a tricky one ;-) > > > > Cheers > > > > On 02/10/2014 16:58, Loic Dachary wrote: > > > > > > > > > On 02/10/2014 16:31, Sebastien Ponce wrote: > > >> As mentioned in the bug, I was a bit fast in concluding I could > > >> reproduce the problem. I can reproduce the original one, but not the > > >> remaining issue once the fix was applied. > > >> > > >> Loic, can you confirm that you are using the wip-9356 branch of repo > > >> git@github.com:sponce/ceph.git ? > > > > > > Yes. I'm able to reproduce the problem with 0daddfbf1164d6ba3f38eee29d2f11acfa62f2b6 from your tree https://github.com/sponce/ceph.git which is the head of wip-9356 > > > > > > Cheers > > > > > >> > > >> Sebastien > > >> > > >> > > >> On Thu, 2014-10-02 at 11:49 +0200, Sebastien Ponce wrote: > > >>> Hurray, I manage to seg fault ! (sounds odd this sentence...) > > >>> So indeed, fresh ubuntu, debug mode does it. What I've also changed in > > >>> the meantime is the type of machine. It's still virtual, but more > > >>> powerful (4 cores). > > >>> > > >>> Let's debug it now... > > >>> > > >>> Sebastien > > >>> > > >>> > > >>> On Wed, 2014-10-01 at 15:09 +0200, Sebastien Ponce wrote: > > >>>> Ok, we are getting closer then. > > >>>> I'll try to reinstall the ubuntu machine and run with -g. > > >>>> > > >>>> Sebastien > > >>>> > > >>>> On Wed, 2014-10-01 at 15:08 +0200, Loic Dachary wrote: > > >>>>> > > >>>>> On 01/10/2014 13:37, Sebastien Ponce wrote: > > >>>>>> I've tried it on the RHEL6 like machine, but no luck. Or should I say I > > >>>>>> was lucky ? Anyway no crash. > > >>>>>> > > >>>>>> Discussing with Dan Van Der Ster, he told me that he had once the case > > >>>>>> where the crash was there or not depending on the libc version. Just in > > >>>>>> case, what are you using ? On the machine I've used this morning, it's > > >>>>>> glibc-2.12-1.107.el6.x86_64 > > >>>>> > > >>>>> I'm using > > >>>>> > > >>>>> ii libc-bin 2.19-0ubuntu3 > > >>>>> > > >>>>> and I do not have glibc installed. I'm quite sure you can reproduce the problem on a fresh Ubuntu 14.04 with the following > > >>>>> > > >>>>> ./autogen.sh ; ./configure --disable-static --with-debug CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" > > >>>>> > > >>>>> it fails consistently in my container. When I try with ./configure it does not fail. Something is going on when there is -g instead of the default flags. I checked that --disable-static and --with-debug make no difference. > > >>>>> > > >>>>> Cheers > > >>>>> > > >>>>>> > > >>>>>> Sebastien > > >>>>>> > > >>>>>> On Wed, 2014-10-01 at 10:31 +0200, Loic Dachary wrote: > > >>>>>>> Hi, > > >>>>>>> > > >>>>>>> On 01/10/2014 09:28, Sebastien Ponce wrote: > > >>>>>>>> Thanks for this effort ! > > >>>>>>>> I'll look at all that this afternoon (it's mid morning here). > > >>>>>>> > > >>>>>>> Same here : I'm in Paris ;-) > > >>>>>>> > > >>>>>>>> Concerning configure and autogen, I do the standard (naive ?) > > >>>>>>>> "./autogen.sh; ./configure; make" given by the readme. > > >>>>>>> > > >>>>>>> Could you try with > > >>>>>>> > > >>>>>>> ./autogen.sh ; ./configure --disable-static --with-debug CC='ccache gcc' CXX='ccache g++' CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" ; make > > >>>>>>> > > >>>>>>> and let me know if you see a problem when running > > >>>>>>> > > >>>>>>> cd src > > >>>>>>> rm -fr dev out ; mkdir -p dev ; CEPH_NUM_MON=1 CEPH_NUM_OSD=3 ./vstart.sh -d -n -X -l mon osd > > >>>>>>> ./ceph_test_rados_striper_api_aio > > >>>>>>> > > >>>>>>> I'm trying with your "./configure" on my end to check if this changes something. > > >>>>>>> > > >>>>>>> It's going to be a fun one ;-) > > >>>>>>> > > >>>>>>> Cheers > > >>>>>>> > > >>>>>>>> > > >>>>>>>> Sebastien > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> On Wed, 2014-10-01 at 09:01 +0200, Loic Dachary wrote: > > >>>>>>>>> Hi Sebastien, > > >>>>>>>>> > > >>>>>>>>> In the context of http://tracker.ceph.com/issues/9356 I'm trying to collect as much information as I can in an environment that can be reconstructed to avoid interferences. Could you please let me know which ./configure line you are using ? Or which do_autogen.sh command you are using. > > >>>>>>>>> > > >>>>>>>>> Thanks in advance :-) > > >>>>>>>>> > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> -- > > >>>>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > >>>>>>>> the body of a message to majordomo@vger.kernel.org > > >>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>>> > > >>>>>> -- > > >>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > >>>>>> the body of a message to majordomo@vger.kernel.org > > >>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html > > >>>>>> > > >>>>> > > >>>> > > >>>> > > >>>> -- > > >>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > >>>> the body of a message to majordomo@vger.kernel.org > > >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html > > >>> > > >>> > > >>> -- > > >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > >>> the body of a message to majordomo@vger.kernel.org > > >>> More majordomo info at http://vger.kernel.org/majordomo-info.html > > >> > > >> > > >> -- > > >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > >> the body of a message to majordomo@vger.kernel.org > > >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > >> > > > > > > > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: librados AIO problem diagnostic 2014-10-09 14:06 ` Sebastien Ponce @ 2014-10-09 14:24 ` Loic Dachary 0 siblings, 0 replies; 15+ messages in thread From: Loic Dachary @ 2014-10-09 14:24 UTC (permalink / raw) To: Sebastien Ponce; +Cc: Ceph Development [-- Attachment #1: Type: text/plain, Size: 7011 bytes --] On 09/10/2014 16:06, Sebastien Ponce wrote: > Finally this should be fixed. > Loic, could you retry with the new state of my branch and confirm ? Hi Sebastien, I commented on the pull request and scheduled a run in the gitbuilder. Cheers > > Sebastien > > On Wed, 2014-10-08 at 10:00 +0200, Sebastien Ponce wrote: >> It vanished again.... >> I've spent quite some time analysing the helgrind output to try to find >> it, but so far I got nothing but problems outside this part (BTW, it >> would be interesting to run tests systematically in helgrind, even if >> there is a lot of false positive). >> Now I've just seen the backtrace you've sent and this already helps a >> lot; I did not even have a backtrace so far. >> In conclusion, still ongoing.... >> >> Sebastien >> >> >> On Wed, 2014-10-08 at 09:14 +0200, Loic Dachary wrote: >>> Hi Sebastien, >>> >>> Have you been able to reproduce it or did it vanish again ? This is a tricky one ;-) >>> >>> Cheers >>> >>> On 02/10/2014 16:58, Loic Dachary wrote: >>>> >>>> >>>> On 02/10/2014 16:31, Sebastien Ponce wrote: >>>>> As mentioned in the bug, I was a bit fast in concluding I could >>>>> reproduce the problem. I can reproduce the original one, but not the >>>>> remaining issue once the fix was applied. >>>>> >>>>> Loic, can you confirm that you are using the wip-9356 branch of repo >>>>> git@github.com:sponce/ceph.git ? >>>> >>>> Yes. I'm able to reproduce the problem with 0daddfbf1164d6ba3f38eee29d2f11acfa62f2b6 from your tree https://github.com/sponce/ceph.git which is the head of wip-9356 >>>> >>>> Cheers >>>> >>>>> >>>>> Sebastien >>>>> >>>>> >>>>> On Thu, 2014-10-02 at 11:49 +0200, Sebastien Ponce wrote: >>>>>> Hurray, I manage to seg fault ! (sounds odd this sentence...) >>>>>> So indeed, fresh ubuntu, debug mode does it. What I've also changed in >>>>>> the meantime is the type of machine. It's still virtual, but more >>>>>> powerful (4 cores). >>>>>> >>>>>> Let's debug it now... >>>>>> >>>>>> Sebastien >>>>>> >>>>>> >>>>>> On Wed, 2014-10-01 at 15:09 +0200, Sebastien Ponce wrote: >>>>>>> Ok, we are getting closer then. >>>>>>> I'll try to reinstall the ubuntu machine and run with -g. >>>>>>> >>>>>>> Sebastien >>>>>>> >>>>>>> On Wed, 2014-10-01 at 15:08 +0200, Loic Dachary wrote: >>>>>>>> >>>>>>>> On 01/10/2014 13:37, Sebastien Ponce wrote: >>>>>>>>> I've tried it on the RHEL6 like machine, but no luck. Or should I say I >>>>>>>>> was lucky ? Anyway no crash. >>>>>>>>> >>>>>>>>> Discussing with Dan Van Der Ster, he told me that he had once the case >>>>>>>>> where the crash was there or not depending on the libc version. Just in >>>>>>>>> case, what are you using ? On the machine I've used this morning, it's >>>>>>>>> glibc-2.12-1.107.el6.x86_64 >>>>>>>> >>>>>>>> I'm using >>>>>>>> >>>>>>>> ii libc-bin 2.19-0ubuntu3 >>>>>>>> >>>>>>>> and I do not have glibc installed. I'm quite sure you can reproduce the problem on a fresh Ubuntu 14.04 with the following >>>>>>>> >>>>>>>> ./autogen.sh ; ./configure --disable-static --with-debug CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" >>>>>>>> >>>>>>>> it fails consistently in my container. When I try with ./configure it does not fail. Something is going on when there is -g instead of the default flags. I checked that --disable-static and --with-debug make no difference. >>>>>>>> >>>>>>>> Cheers >>>>>>>> >>>>>>>>> >>>>>>>>> Sebastien >>>>>>>>> >>>>>>>>> On Wed, 2014-10-01 at 10:31 +0200, Loic Dachary wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> On 01/10/2014 09:28, Sebastien Ponce wrote: >>>>>>>>>>> Thanks for this effort ! >>>>>>>>>>> I'll look at all that this afternoon (it's mid morning here). >>>>>>>>>> >>>>>>>>>> Same here : I'm in Paris ;-) >>>>>>>>>> >>>>>>>>>>> Concerning configure and autogen, I do the standard (naive ?) >>>>>>>>>>> "./autogen.sh; ./configure; make" given by the readme. >>>>>>>>>> >>>>>>>>>> Could you try with >>>>>>>>>> >>>>>>>>>> ./autogen.sh ; ./configure --disable-static --with-debug CC='ccache gcc' CXX='ccache g++' CFLAGS="-Wall -g" CXXFLAGS="-Wall -g" ; make >>>>>>>>>> >>>>>>>>>> and let me know if you see a problem when running >>>>>>>>>> >>>>>>>>>> cd src >>>>>>>>>> rm -fr dev out ; mkdir -p dev ; CEPH_NUM_MON=1 CEPH_NUM_OSD=3 ./vstart.sh -d -n -X -l mon osd >>>>>>>>>> ./ceph_test_rados_striper_api_aio >>>>>>>>>> >>>>>>>>>> I'm trying with your "./configure" on my end to check if this changes something. >>>>>>>>>> >>>>>>>>>> It's going to be a fun one ;-) >>>>>>>>>> >>>>>>>>>> Cheers >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Sebastien >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, 2014-10-01 at 09:01 +0200, Loic Dachary wrote: >>>>>>>>>>>> Hi Sebastien, >>>>>>>>>>>> >>>>>>>>>>>> In the context of http://tracker.ceph.com/issues/9356 I'm trying to collect as much information as I can in an environment that can be reconstructed to avoid interferences. Could you please let me know which ./configure line you are using ? Or which do_autogen.sh command you are using. >>>>>>>>>>>> >>>>>>>>>>>> Thanks in advance :-) >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>>>>>>>> the body of a message to majordomo@vger.kernel.org >>>>>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>>>>>> the body of a message to majordomo@vger.kernel.org >>>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>>>> the body of a message to majordomo@vger.kernel.org >>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>> >>>>>> >>>>>> -- >>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>>> the body of a message to majordomo@vger.kernel.org >>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>> >>>>> >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>> the body of a message to majordomo@vger.kernel.org >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>> >>>> >>> >> >> >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 263 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: librados AIO problem diagnostic 2014-10-01 7:28 ` Sebastien Ponce 2014-10-01 8:31 ` Loic Dachary @ 2014-10-01 8:33 ` Loic Dachary 2014-10-01 8:50 ` Sebastien Ponce 1 sibling, 1 reply; 15+ messages in thread From: Loic Dachary @ 2014-10-01 8:33 UTC (permalink / raw) To: Sebastien Ponce; +Cc: Ceph Development [-- Attachment #1: Type: text/plain, Size: 1132 bytes --] Which kernel are you running accord to uname -a ? I have: Linux 0a2ad46b4fb5 3.13.0-23-generic #45-Ubuntu SMP Fri Apr 4 06:58:38 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux On 01/10/2014 09:28, Sebastien Ponce wrote: > Thanks for this effort ! > I'll look at all that this afternoon (it's mid morning here). > Concerning configure and autogen, I do the standard (naive ?) > "./autogen.sh; ./configure; make" given by the readme. > > Sebastien > > > On Wed, 2014-10-01 at 09:01 +0200, Loic Dachary wrote: >> Hi Sebastien, >> >> In the context of http://tracker.ceph.com/issues/9356 I'm trying to collect as much information as I can in an environment that can be reconstructed to avoid interferences. Could you please let me know which ./configure line you are using ? Or which do_autogen.sh command you are using. >> >> Thanks in advance :-) >> > > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 263 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: librados AIO problem diagnostic 2014-10-01 8:33 ` Loic Dachary @ 2014-10-01 8:50 ` Sebastien Ponce 0 siblings, 0 replies; 15+ messages in thread From: Sebastien Ponce @ 2014-10-01 8:50 UTC (permalink / raw) To: Loic Dachary; +Cc: Ceph Development I've tried on 3 machines up to now, with different kernels : - a RHEL6 (or close to it) with kernel 2.6.32-358.2.1.el6.x86_64 - a debian unstable box with kernel 3.12-1-amd64 #1 SMP Debian 3.12.6-2 - an ubuntu one, as you suggested, don't remember the kernel (and box is gone) Sebastien On Wed, 2014-10-01 at 10:33 +0200, Loic Dachary wrote: > Which kernel are you running accord to uname -a ? I have: > > Linux 0a2ad46b4fb5 3.13.0-23-generic #45-Ubuntu SMP Fri Apr 4 06:58:38 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux > > On 01/10/2014 09:28, Sebastien Ponce wrote: > > Thanks for this effort ! > > I'll look at all that this afternoon (it's mid morning here). > > Concerning configure and autogen, I do the standard (naive ?) > > "./autogen.sh; ./configure; make" given by the readme. > > > > Sebastien > > > > > > On Wed, 2014-10-01 at 09:01 +0200, Loic Dachary wrote: > >> Hi Sebastien, > >> > >> In the context of http://tracker.ceph.com/issues/9356 I'm trying to collect as much information as I can in an environment that can be reconstructed to avoid interferences. Could you please let me know which ./configure line you are using ? Or which do_autogen.sh command you are using. > >> > >> Thanks in advance :-) > >> > > > > > > -- > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2014-10-09 14:24 UTC | newest] Thread overview: 15+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-10-01 7:01 librados AIO problem diagnostic Loic Dachary 2014-10-01 7:28 ` Sebastien Ponce 2014-10-01 8:31 ` Loic Dachary 2014-10-01 11:37 ` Sebastien Ponce 2014-10-01 13:08 ` Loic Dachary 2014-10-01 13:09 ` Sebastien Ponce 2014-10-02 9:49 ` Sebastien Ponce 2014-10-02 14:31 ` Sebastien Ponce 2014-10-02 14:58 ` Loic Dachary 2014-10-08 7:14 ` Loic Dachary 2014-10-08 8:00 ` Sebastien Ponce 2014-10-09 14:06 ` Sebastien Ponce 2014-10-09 14:24 ` Loic Dachary 2014-10-01 8:33 ` Loic Dachary 2014-10-01 8:50 ` Sebastien Ponce
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.