* Re: ceph v10.2.4 QE validation status
2016-11-23 10:39 ` Abhishek L
@ 2016-11-23 14:39 ` Gregory Farnum
2016-11-23 14:42 ` Sage Weil
` (2 subsequent siblings)
3 siblings, 0 replies; 26+ messages in thread
From: Gregory Farnum @ 2016-11-23 14:39 UTC (permalink / raw)
To: Abhishek L, Yuri Weinstein; +Cc: Ceph Development, Dachary, Loic, John Spray
On Wed, Nov 23, 2016 at 5:39 AM, Abhishek L <abhishek@suse.com> wrote:
>
> Hi Sage, Greg,
John signed off on the FS stuff and he's the one who does that now.
Looks good to me. ;)
-Greg
>
>
> Yuri Weinstein writes:
>
>> See updated status - http://tracker.ceph.com/issues/17487#note-32
>>
>> Outstanding issues:
>>
>> knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>> pls review/approve, assumed Approved ?)
>>
>> upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>> pls review/approve, seems persistent, but maybe not a showstopper)
>>
>> upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
>> to make it pass, see issues in the tacker summary above)
>>
>> Sage, jewel 10.2.4 can be released as soon as you agree with the
>> findings/summary.
>
> Do you think we're ready to release 10.2.4 yet?
>
>
> Best,
> Abhishek
>>
>> Thx
>> YuriW
>>
>> On Fri, Nov 11, 2016 at 8:47 AM, Yuri Weinstein <yweinste@redhat.com> wrote:
>>> Detailed summary of the QE Validation can be found here
>>> http://tracker.ceph.com/issues/17487#note-32
>>>
>>> The following suites were in scope of this point release validation:
>>>
>>> rados (subset 35/50 297 jobs)
>>> rbd
>>> rgw
>>> fs
>>> krbd
>>> kcephfs
>>> knfs
>>> rest
>>> hadoop
>>> samba
>>> ceph-deploy
>>> ceph-disk
>>> upgrade/client-upgrade
>>> upgrade/hammer-x (jewel)
>>> upgrade/infernalis-x (jewel) - deprecated
>>> powercycle
>>> (please let me know if any suites are missing from this ^ list)
>>>
>>> ==============================
>>> Issues requiring approval/decision:
>>>
>>> fs - http://tracker.ceph.com/issues/17832 (Greg pls review/approve)
>>>
>>> krbd - http://tracker.ceph.com/issues/17221 (Jason, Ilya pls review/approve)
>>>
>>> knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>>> pls review/approve)
>>>
>>> upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>>> pls review/approve)
>>>
>>> upgrade/infernalis-x (jewel) - deprecated, but Nathan is still working
>>> to make it pass (Sage pls review/approve)
>>>
>>> powercycle - in progress
>>>
>>> Thx
>>> YuriW
>
>
> --
> Abhishek Lekshmanan
> SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: ceph v10.2.4 QE validation status
2016-11-23 10:39 ` Abhishek L
2016-11-23 14:39 ` Gregory Farnum
@ 2016-11-23 14:42 ` Sage Weil
2016-11-23 15:06 ` John Spray
2016-11-29 10:46 ` Abhishek L
2016-11-30 21:56 ` Ken Dreyer
2016-12-01 13:26 ` Alfredo Deza
3 siblings, 2 replies; 26+ messages in thread
From: Sage Weil @ 2016-11-23 14:42 UTC (permalink / raw)
To: Abhishek L
Cc: Gregory Farnum, Ceph Development, Abhishek Varshney,
Dachary, Loic, Dachary, Loic, Dillaman, Jason, Nathan Cutler,
Ilya Dryomov, John Spray, Durgin, Josh, Yuri Weinstein
[-- Attachment #1: Type: TEXT/PLAIN, Size: 2636 bytes --]
On Wed, 23 Nov 2016, Abhishek L wrote:
>
> Hi Sage, Greg,
>
>
> Yuri Weinstein writes:
>
> > See updated status - http://tracker.ceph.com/issues/17487#note-32
> >
> > Outstanding issues:
> >
> > knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
> > pls review/approve, assumed Approved ?)
> >
> > upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
> > pls review/approve, seems persistent, but maybe not a showstopper)
> >
> > upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
> > to make it pass, see issues in the tacker summary above)
> >
> > Sage, jewel 10.2.4 can be released as soon as you agree with the
> > findings/summary.
>
> Do you think we're ready to release 10.2.4 yet?
I'm reproducing http://tracker.ceph.com/issues/17847 with logs to make
sure this isn't a regression.
We can ignore the infernalis runs.
I think we can ignore the knfs selinux issue too.. Greg, can you confirm?
Thanks!
sage
>
>
> Best,
> Abhishek
> >
> > Thx
> > YuriW
> >
> > On Fri, Nov 11, 2016 at 8:47 AM, Yuri Weinstein <yweinste@redhat.com> wrote:
> >> Detailed summary of the QE Validation can be found here
> >> http://tracker.ceph.com/issues/17487#note-32
> >>
> >> The following suites were in scope of this point release validation:
> >>
> >> rados (subset 35/50 297 jobs)
> >> rbd
> >> rgw
> >> fs
> >> krbd
> >> kcephfs
> >> knfs
> >> rest
> >> hadoop
> >> samba
> >> ceph-deploy
> >> ceph-disk
> >> upgrade/client-upgrade
> >> upgrade/hammer-x (jewel)
> >> upgrade/infernalis-x (jewel) - deprecated
> >> powercycle
> >> (please let me know if any suites are missing from this ^ list)
> >>
> >> ==============================
> >> Issues requiring approval/decision:
> >>
> >> fs - http://tracker.ceph.com/issues/17832 (Greg pls review/approve)
> >>
> >> krbd - http://tracker.ceph.com/issues/17221 (Jason, Ilya pls review/approve)
> >>
> >> knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
> >> pls review/approve)
> >>
> >> upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
> >> pls review/approve)
> >>
> >> upgrade/infernalis-x (jewel) - deprecated, but Nathan is still working
> >> to make it pass (Sage pls review/approve)
> >>
> >> powercycle - in progress
> >>
> >> Thx
> >> YuriW
>
>
> --
> Abhishek Lekshmanan
> SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-11-23 14:42 ` Sage Weil
@ 2016-11-23 15:06 ` John Spray
2016-11-29 10:46 ` Abhishek L
1 sibling, 0 replies; 26+ messages in thread
From: John Spray @ 2016-11-23 15:06 UTC (permalink / raw)
To: Sage Weil
Cc: Abhishek L, Gregory Farnum, Ceph Development, Abhishek Varshney,
Dachary, Loic, Dachary, Loic, Dillaman, Jason, Nathan Cutler,
Ilya Dryomov, Durgin, Josh, Yuri Weinstein
On Wed, Nov 23, 2016 at 2:42 PM, Sage Weil <sweil@redhat.com> wrote:
> On Wed, 23 Nov 2016, Abhishek L wrote:
>>
>> Hi Sage, Greg,
>>
>>
>> Yuri Weinstein writes:
>>
>> > See updated status - http://tracker.ceph.com/issues/17487#note-32
>> >
>> > Outstanding issues:
>> >
>> > knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>> > pls review/approve, assumed Approved ?)
>> >
>> > upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>> > pls review/approve, seems persistent, but maybe not a showstopper)
>> >
>> > upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
>> > to make it pass, see issues in the tacker summary above)
>> >
>> > Sage, jewel 10.2.4 can be released as soon as you agree with the
>> > findings/summary.
>>
>> Do you think we're ready to release 10.2.4 yet?
>
> I'm reproducing http://tracker.ceph.com/issues/17847 with logs to make
> sure this isn't a regression.
>
> We can ignore the infernalis runs.
>
> I think we can ignore the knfs selinux issue too.. Greg, can you confirm?
The knfs selinux issue can be ignored, yes. Last investigation
indicated that it probably wasn't cephfs-specific.
John
> Thanks!
> sage
>
>>
>>
>> Best,
>> Abhishek
>> >
>> > Thx
>> > YuriW
>> >
>> > On Fri, Nov 11, 2016 at 8:47 AM, Yuri Weinstein <yweinste@redhat.com> wrote:
>> >> Detailed summary of the QE Validation can be found here
>> >> http://tracker.ceph.com/issues/17487#note-32
>> >>
>> >> The following suites were in scope of this point release validation:
>> >>
>> >> rados (subset 35/50 297 jobs)
>> >> rbd
>> >> rgw
>> >> fs
>> >> krbd
>> >> kcephfs
>> >> knfs
>> >> rest
>> >> hadoop
>> >> samba
>> >> ceph-deploy
>> >> ceph-disk
>> >> upgrade/client-upgrade
>> >> upgrade/hammer-x (jewel)
>> >> upgrade/infernalis-x (jewel) - deprecated
>> >> powercycle
>> >> (please let me know if any suites are missing from this ^ list)
>> >>
>> >> ==============================
>> >> Issues requiring approval/decision:
>> >>
>> >> fs - http://tracker.ceph.com/issues/17832 (Greg pls review/approve)
>> >>
>> >> krbd - http://tracker.ceph.com/issues/17221 (Jason, Ilya pls review/approve)
>> >>
>> >> knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>> >> pls review/approve)
>> >>
>> >> upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>> >> pls review/approve)
>> >>
>> >> upgrade/infernalis-x (jewel) - deprecated, but Nathan is still working
>> >> to make it pass (Sage pls review/approve)
>> >>
>> >> powercycle - in progress
>> >>
>> >> Thx
>> >> YuriW
>>
>>
>> --
>> Abhishek Lekshmanan
>> SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-11-23 14:42 ` Sage Weil
2016-11-23 15:06 ` John Spray
@ 2016-11-29 10:46 ` Abhishek L
2016-11-30 14:57 ` Abhishek L
1 sibling, 1 reply; 26+ messages in thread
From: Abhishek L @ 2016-11-29 10:46 UTC (permalink / raw)
To: Sage Weil
Cc: Gregory Farnum, Ceph Development, Abhishek Varshney,
Dachary, Loic, Dachary, Loic, Dillaman, Jason, Nathan Cutler,
Ilya Dryomov, John Spray, Durgin, Josh, Yuri Weinstein
Sage Weil writes:
> On Wed, 23 Nov 2016, Abhishek L wrote:
>>
>> Hi Sage, Greg,
>>
>>
>> Yuri Weinstein writes:
>>
>> > See updated status - http://tracker.ceph.com/issues/17487#note-32
>> >
>> > Outstanding issues:
>> >
>> > knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>> > pls review/approve, assumed Approved ?)
>> >
>> > upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>> > pls review/approve, seems persistent, but maybe not a showstopper)
>> >
>> > upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
>> > to make it pass, see issues in the tacker summary above)
>> >
>> > Sage, jewel 10.2.4 can be released as soon as you agree with the
>> > findings/summary.
>>
>> Do you think we're ready to release 10.2.4 yet?
>
> I'm reproducing http://tracker.ceph.com/issues/17847 with logs to make
> sure this isn't a regression.
>
> We can ignore the infernalis runs.
>
> I think we can ignore the knfs selinux issue too.. Greg, can you confirm?
>
Added the prs 12001 & 12167 on top of the jewel branch and scheduled rados runs
at
http://pulpito.ceph.com/abhi-2016-11-29_10:22:25-rados-wip-jewel-10-2-4-distro-basic-smithi/ &
updated the tracker at http://tracker.ceph.com/issues/17851#note-17,
with details of the test. I'll update the progress once the suite goes
through
Best,
Abhishek
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-11-29 10:46 ` Abhishek L
@ 2016-11-30 14:57 ` Abhishek L
2016-11-30 16:01 ` Sage Weil
2016-11-30 16:09 ` Josh Durgin
0 siblings, 2 replies; 26+ messages in thread
From: Abhishek L @ 2016-11-30 14:57 UTC (permalink / raw)
To: Sage Weil
Cc: Gregory Farnum, Ceph Development, Abhishek Varshney,
Dachary, Loic, Dachary, Loic, Dillaman, Jason, Nathan Cutler,
Ilya Dryomov, John Spray, Durgin, Josh, Yuri Weinstein
Abhishek L writes:
> Sage Weil writes:
>
>> On Wed, 23 Nov 2016, Abhishek L wrote:
>>>
>>> Hi Sage, Greg,
>>>
>>>
>>> Yuri Weinstein writes:
>>>
>>> > See updated status - http://tracker.ceph.com/issues/17487#note-32
>>> >
>>> > Outstanding issues:
>>> >
>>> > knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>>> > pls review/approve, assumed Approved ?)
>>> >
>>> > upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>>> > pls review/approve, seems persistent, but maybe not a showstopper)
>>> >
>>> > upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
>>> > to make it pass, see issues in the tacker summary above)
>>> >
>>> > Sage, jewel 10.2.4 can be released as soon as you agree with the
>>> > findings/summary.
>>>
>>> Do you think we're ready to release 10.2.4 yet?
>>
>> I'm reproducing http://tracker.ceph.com/issues/17847 with logs to make
>> sure this isn't a regression.
>>
>> We can ignore the infernalis runs.
>>
>> I think we can ignore the knfs selinux issue too.. Greg, can you confirm?
>>
> Added the prs 12001 & 12167 on top of the jewel branch and scheduled rados runs
> at
> http://pulpito.ceph.com/abhi-2016-11-29_10:22:25-rados-wip-jewel-10-2-4-distro-basic-smithi/ &
>
> updated the tracker at http://tracker.ceph.com/issues/17851#note-17,
> with details of the test. I'll update the progress once the suite goes
> through
Had around 11 tests fail from 296 scheduled, there were a couple of
valgrind issues on ceph-mon (which were seen at earlier runs on jewel as
well) and an s3test failure, rest of the issues were looking related to
infrastructure as they were failing to get specific version numbers from
gitbuilders.
Reported this issue as:
http://tracker.ceph.com/issues/18089
subsequent re runs are still failing with similar errors. The details
are updated at
http://tracker.ceph.com/issues/17487#note-37
The upgrade suite has also failed with similar errors.
Overall it looks like we're close to cutting the release, as soon as we
can get through these infra errors
Cheers,
Abhishek
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-11-30 14:57 ` Abhishek L
@ 2016-11-30 16:01 ` Sage Weil
2016-11-30 16:21 ` Abhishek L
2016-11-30 16:09 ` Josh Durgin
1 sibling, 1 reply; 26+ messages in thread
From: Sage Weil @ 2016-11-30 16:01 UTC (permalink / raw)
To: Abhishek L
Cc: Gregory Farnum, Ceph Development, Abhishek Varshney,
Dachary, Loic, Dachary, Loic, Dillaman, Jason, Nathan Cutler,
Ilya Dryomov, John Spray, Durgin, Josh, Yuri Weinstein
On Wed, 30 Nov 2016, Abhishek L wrote:
>
> Abhishek L writes:
>
> > Sage Weil writes:
> >
> >> On Wed, 23 Nov 2016, Abhishek L wrote:
> >>>
> >>> Hi Sage, Greg,
> >>>
> >>>
> >>> Yuri Weinstein writes:
> >>>
> >>> > See updated status - http://tracker.ceph.com/issues/17487#note-32
> >>> >
> >>> > Outstanding issues:
> >>> >
> >>> > knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
> >>> > pls review/approve, assumed Approved ?)
> >>> >
> >>> > upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
> >>> > pls review/approve, seems persistent, but maybe not a showstopper)
> >>> >
> >>> > upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
> >>> > to make it pass, see issues in the tacker summary above)
> >>> >
> >>> > Sage, jewel 10.2.4 can be released as soon as you agree with the
> >>> > findings/summary.
> >>>
> >>> Do you think we're ready to release 10.2.4 yet?
> >>
> >> I'm reproducing http://tracker.ceph.com/issues/17847 with logs to make
> >> sure this isn't a regression.
> >>
> >> We can ignore the infernalis runs.
> >>
> >> I think we can ignore the knfs selinux issue too.. Greg, can you confirm?
> >>
> > Added the prs 12001 & 12167 on top of the jewel branch and scheduled rados runs
> > at
> > http://pulpito.ceph.com/abhi-2016-11-29_10:22:25-rados-wip-jewel-10-2-4-distro-basic-smithi/ &
https://github.com/ceph/ceph-qa-suite/pull/1292
avoids xenial for the rados upgrade tests (in jewel branch).
The upgrade tests already explicitly call out trusty, so they should be
fine.
> > updated the tracker at http://tracker.ceph.com/issues/17851#note-17,
> > with details of the test. I'll update the progress once the suite goes
> > through
>
> Had around 11 tests fail from 296 scheduled, there were a couple of
> valgrind issues on ceph-mon (which were seen at earlier runs on jewel as
> well) and an s3test failure, rest of the issues were looking related to
> infrastructure as they were failing to get specific version numbers from
> gitbuilders.
>
> Reported this issue as:
>
> http://tracker.ceph.com/issues/18089
>
> subsequent re runs are still failing with similar errors. The details
> are updated at
>
> http://tracker.ceph.com/issues/17487#note-37
>
> The upgrade suite has also failed with similar errors.
Which one?
Thanks!
sage
>
> Overall it looks like we're close to cutting the release, as soon as we
> can get through these infra errors
>
> Cheers,
> Abhishek
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-11-30 16:01 ` Sage Weil
@ 2016-11-30 16:21 ` Abhishek L
2016-11-30 16:29 ` Abhishek L
0 siblings, 1 reply; 26+ messages in thread
From: Abhishek L @ 2016-11-30 16:21 UTC (permalink / raw)
To: Sage Weil
Cc: Abhishek L, Gregory Farnum, Ceph Development, Abhishek Varshney,
Dachary, Loic, Dachary, Loic, Dillaman, Jason, Nathan Cutler,
Ilya Dryomov, John Spray, Durgin, Josh, Yuri Weinstein
Sage Weil writes:
> On Wed, 30 Nov 2016, Abhishek L wrote:
>>
>> Abhishek L writes:
>>
>> > Sage Weil writes:
>> >
>> >> On Wed, 23 Nov 2016, Abhishek L wrote:
>> >>>
>> >>> Hi Sage, Greg,
>> >>>
>> >>>
>> >>> Yuri Weinstein writes:
>> >>>
>> >>> > See updated status - http://tracker.ceph.com/issues/17487#note-32
>> >>> >
>> >>> > Outstanding issues:
>> >>> >
>> >>> > knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>> >>> > pls review/approve, assumed Approved ?)
>> >>> >
>> >>> > upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>> >>> > pls review/approve, seems persistent, but maybe not a showstopper)
>> >>> >
>> >>> > upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
>> >>> > to make it pass, see issues in the tacker summary above)
>> >>> >
>> >>> > Sage, jewel 10.2.4 can be released as soon as you agree with the
>> >>> > findings/summary.
>> >>>
>> >>> Do you think we're ready to release 10.2.4 yet?
>> >>
>> >> I'm reproducing http://tracker.ceph.com/issues/17847 with logs to make
>> >> sure this isn't a regression.
>> >>
>> >> We can ignore the infernalis runs.
>> >>
>> >> I think we can ignore the knfs selinux issue too.. Greg, can you confirm?
>> >>
>> > Added the prs 12001 & 12167 on top of the jewel branch and scheduled rados runs
>> > at
>> > http://pulpito.ceph.com/abhi-2016-11-29_10:22:25-rados-wip-jewel-10-2-4-distro-basic-smithi/ &
>
> https://github.com/ceph/ceph-qa-suite/pull/1292
>
> avoids xenial for the rados upgrade tests (in jewel branch).
>
> The upgrade tests already explicitly call out trusty, so they should be
> fine.
>
>> > updated the tracker at http://tracker.ceph.com/issues/17851#note-17,
>> > with details of the test. I'll update the progress once the suite goes
>> > through
>>
>> Had around 11 tests fail from 296 scheduled, there were a couple of
>> valgrind issues on ceph-mon (which were seen at earlier runs on jewel as
>> well) and an s3test failure, rest of the issues were looking related to
>> infrastructure as they were failing to get specific version numbers from
>> gitbuilders.
>>
>> Reported this issue as:
>>
>> http://tracker.ceph.com/issues/18089
>>
>> subsequent re runs are still failing with similar errors. The details
>> are updated at
>>
>> http://tracker.ceph.com/issues/17487#note-37
>>
>> The upgrade suite has also failed with similar errors.
>
> Which one?
The upgrade/client-upgrade suite (I hope this is the right upgrade
suite), the errors are the same failed to fetch package version errors
seen for the rados suite, so not actual test run errors yet.
http://pulpito.ceph.com/abhi-2016-11-30_09:58:50-upgrade:client-upgrade-wip-jewel-10-2-4-distro-basic-smithi/
is the run
Best,
Abhishek
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-11-30 16:21 ` Abhishek L
@ 2016-11-30 16:29 ` Abhishek L
2016-11-30 16:57 ` Sage Weil
0 siblings, 1 reply; 26+ messages in thread
From: Abhishek L @ 2016-11-30 16:29 UTC (permalink / raw)
To: Abhishek L
Cc: Sage Weil, Gregory Farnum, Ceph Development, Abhishek Varshney,
Dachary, Loic, Dachary, Loic, Dillaman, Jason, Nathan Cutler,
Ilya Dryomov, John Spray, Durgin, Josh, Yuri Weinstein
Abhishek L writes:
> Sage Weil writes:
>
>> On Wed, 30 Nov 2016, Abhishek L wrote:
>>>
>>> Abhishek L writes:
>>>
>>> > Sage Weil writes:
>>> >
>>> >> On Wed, 23 Nov 2016, Abhishek L wrote:
>>> >>>
>>> >>> Hi Sage, Greg,
>>> >>>
>>> >>>
>>> >>> Yuri Weinstein writes:
>>> >>>
>>> >>> > See updated status - http://tracker.ceph.com/issues/17487#note-32
>>> >>> >
>>> >>> > Outstanding issues:
>>> >>> >
>>> >>> > knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>>> >>> > pls review/approve, assumed Approved ?)
>>> >>> >
>>> >>> > upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>>> >>> > pls review/approve, seems persistent, but maybe not a showstopper)
>>> >>> >
>>> >>> > upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
>>> >>> > to make it pass, see issues in the tacker summary above)
>>> >>> >
>>> >>> > Sage, jewel 10.2.4 can be released as soon as you agree with the
>>> >>> > findings/summary.
>>> >>>
>>> >>> Do you think we're ready to release 10.2.4 yet?
>>> >>
>>> >> I'm reproducing http://tracker.ceph.com/issues/17847 with logs to make
>>> >> sure this isn't a regression.
>>> >>
>>> >> We can ignore the infernalis runs.
>>> >>
>>> >> I think we can ignore the knfs selinux issue too.. Greg, can you confirm?
>>> >>
>>> > Added the prs 12001 & 12167 on top of the jewel branch and scheduled rados runs
>>> > at
>>> > http://pulpito.ceph.com/abhi-2016-11-29_10:22:25-rados-wip-jewel-10-2-4-distro-basic-smithi/ &
>>
>> https://github.com/ceph/ceph-qa-suite/pull/1292
>>
>> avoids xenial for the rados upgrade tests (in jewel branch).
Ah alright, maybe this was the cause after all, let's get this in (or
push a branch to ceph-qa-suite so that I can schedule against that)
>>
>> The upgrade tests already explicitly call out trusty, so they should be
>> fine.
>>
>>> > updated the tracker at http://tracker.ceph.com/issues/17851#note-17,
>>> > with details of the test. I'll update the progress once the suite goes
>>> > through
>>>
>>> Had around 11 tests fail from 296 scheduled, there were a couple of
>>> valgrind issues on ceph-mon (which were seen at earlier runs on jewel as
>>> well) and an s3test failure, rest of the issues were looking related to
>>> infrastructure as they were failing to get specific version numbers from
>>> gitbuilders.
>>>
>>> Reported this issue as:
>>>
>>> http://tracker.ceph.com/issues/18089
>>>
>>> subsequent re runs are still failing with similar errors. The details
>>> are updated at
>>>
>>> http://tracker.ceph.com/issues/17487#note-37
>>>
>>> The upgrade suite has also failed with similar errors.
>>
>> Which one?
> The upgrade/client-upgrade suite (I hope this is the right upgrade
> suite), the errors are the same failed to fetch package version errors
> seen for the rados suite, so not actual test run errors yet.
>
> http://pulpito.ceph.com/abhi-2016-11-30_09:58:50-upgrade:client-upgrade-wip-jewel-10-2-4-distro-basic-smithi/
> is the run
>
> Best,
> Abhishek
--
Abhishek Lekshmanan
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-11-30 16:29 ` Abhishek L
@ 2016-11-30 16:57 ` Sage Weil
2016-12-02 14:00 ` Abhishek L
0 siblings, 1 reply; 26+ messages in thread
From: Sage Weil @ 2016-11-30 16:57 UTC (permalink / raw)
To: Abhishek L
Cc: Gregory Farnum, Ceph Development, Abhishek Varshney,
Dachary, Loic, Dachary, Loic, Dillaman, Jason, Nathan Cutler,
Ilya Dryomov, John Spray, Durgin, Josh, Yuri Weinstein
[-- Attachment #1: Type: TEXT/PLAIN, Size: 3773 bytes --]
On Wed, 30 Nov 2016, Abhishek L wrote:
> Abhishek L writes:
>
> > Sage Weil writes:
> >
> >> On Wed, 30 Nov 2016, Abhishek L wrote:
> >>>
> >>> Abhishek L writes:
> >>>
> >>> > Sage Weil writes:
> >>> >
> >>> >> On Wed, 23 Nov 2016, Abhishek L wrote:
> >>> >>>
> >>> >>> Hi Sage, Greg,
> >>> >>>
> >>> >>>
> >>> >>> Yuri Weinstein writes:
> >>> >>>
> >>> >>> > See updated status - http://tracker.ceph.com/issues/17487#note-32
> >>> >>> >
> >>> >>> > Outstanding issues:
> >>> >>> >
> >>> >>> > knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
> >>> >>> > pls review/approve, assumed Approved ?)
> >>> >>> >
> >>> >>> > upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
> >>> >>> > pls review/approve, seems persistent, but maybe not a showstopper)
> >>> >>> >
> >>> >>> > upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
> >>> >>> > to make it pass, see issues in the tacker summary above)
> >>> >>> >
> >>> >>> > Sage, jewel 10.2.4 can be released as soon as you agree with the
> >>> >>> > findings/summary.
> >>> >>>
> >>> >>> Do you think we're ready to release 10.2.4 yet?
> >>> >>
> >>> >> I'm reproducing http://tracker.ceph.com/issues/17847 with logs to make
> >>> >> sure this isn't a regression.
> >>> >>
> >>> >> We can ignore the infernalis runs.
> >>> >>
> >>> >> I think we can ignore the knfs selinux issue too.. Greg, can you confirm?
> >>> >>
> >>> > Added the prs 12001 & 12167 on top of the jewel branch and scheduled rados runs
> >>> > at
> >>> > http://pulpito.ceph.com/abhi-2016-11-29_10:22:25-rados-wip-jewel-10-2-4-distro-basic-smithi/ &
> >>
> >> https://github.com/ceph/ceph-qa-suite/pull/1292
> >>
> >> avoids xenial for the rados upgrade tests (in jewel branch).
>
> Ah alright, maybe this was the cause after all, let's get this in (or
> push a branch to ceph-qa-suite so that I can schedule against that)
I added a patch for the client-upgrade tests to the same branch. You
should be able to just schedule with --suite-branch jewel-avoid-xenial
instead of --suite-branch jewel (for both rados and
upgrade/client-upgrade).
sage
> >>
> >> The upgrade tests already explicitly call out trusty, so they should be
> >> fine.
> >>
> >>> > updated the tracker at http://tracker.ceph.com/issues/17851#note-17,
> >>> > with details of the test. I'll update the progress once the suite goes
> >>> > through
> >>>
> >>> Had around 11 tests fail from 296 scheduled, there were a couple of
> >>> valgrind issues on ceph-mon (which were seen at earlier runs on jewel as
> >>> well) and an s3test failure, rest of the issues were looking related to
> >>> infrastructure as they were failing to get specific version numbers from
> >>> gitbuilders.
> >>>
> >>> Reported this issue as:
> >>>
> >>> http://tracker.ceph.com/issues/18089
> >>>
> >>> subsequent re runs are still failing with similar errors. The details
> >>> are updated at
> >>>
> >>> http://tracker.ceph.com/issues/17487#note-37
> >>>
> >>> The upgrade suite has also failed with similar errors.
> >>
> >> Which one?
> > The upgrade/client-upgrade suite (I hope this is the right upgrade
> > suite), the errors are the same failed to fetch package version errors
> > seen for the rados suite, so not actual test run errors yet.
> >
> > http://pulpito.ceph.com/abhi-2016-11-30_09:58:50-upgrade:client-upgrade-wip-jewel-10-2-4-distro-basic-smithi/
> > is the run
> >
> > Best,
> > Abhishek
>
>
> --
> Abhishek Lekshmanan
> SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-11-30 16:57 ` Sage Weil
@ 2016-12-02 14:00 ` Abhishek L
2016-12-05 11:53 ` Abhishek L
0 siblings, 1 reply; 26+ messages in thread
From: Abhishek L @ 2016-12-02 14:00 UTC (permalink / raw)
To: Sage Weil, Durgin, Josh, Yuri Weinstein, Dillaman, Jason,
Alfredo Deza, Samuel Just
Cc: Abhishek L, Gregory Farnum, Ceph Development, Abhishek Varshney,
Dachary, Loic, Dachary, Loic, Nathan Cutler, Ilya Dryomov,
John Spray
Sage Weil writes:
> On Wed, 30 Nov 2016, Abhishek L wrote:
>> Abhishek L writes:
>>
>> > Sage Weil writes:
>> >
>> >> On Wed, 30 Nov 2016, Abhishek L wrote:
>> >>>
>> >>> Abhishek L writes:
>> >>>
>> >>> > Sage Weil writes:
>> >>> >
>> >>> >> On Wed, 23 Nov 2016, Abhishek L wrote:
>> >>> >>>
>> >>> >>> Hi Sage, Greg,
>> >>> >>>
>> >>> >>>
>> >>> >>> Yuri Weinstein writes:
>> >>> >>>
>> >>> >>> > See updated status - http://tracker.ceph.com/issues/17487#note-32
>> >>> >>> >
>> >>> >>> > Outstanding issues:
>> >>> >>> >
>> >>> >>> > knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>> >>> >>> > pls review/approve, assumed Approved ?)
>> >>> >>> >
>> >>> >>> > upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>> >>> >>> > pls review/approve, seems persistent, but maybe not a showstopper)
>> >>> >>> >
>> >>> >>> > upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
>> >>> >>> > to make it pass, see issues in the tacker summary above)
>> >>> >>> >
>> >>> >>> > Sage, jewel 10.2.4 can be released as soon as you agree with the
>> >>> >>> > findings/summary.
>> >>> >>>
>> >>> >>> Do you think we're ready to release 10.2.4 yet?
>> >>> >>
>> >>> >> I'm reproducing http://tracker.ceph.com/issues/17847 with logs to make
>> >>> >> sure this isn't a regression.
>> >>> >>
>> >>> >> We can ignore the infernalis runs.
>> >>> >>
>> >>> >> I think we can ignore the knfs selinux issue too.. Greg, can you confirm?
>> >>> >>
>> >>> > Added the prs 12001 & 12167 on top of the jewel branch and scheduled rados runs
>> >>> > at
>> >>> > http://pulpito.ceph.com/abhi-2016-11-29_10:22:25-rados-wip-jewel-10-2-4-distro-basic-smithi/ &
>> >>
>> >> https://github.com/ceph/ceph-qa-suite/pull/1292
>> >>
>> >> avoids xenial for the rados upgrade tests (in jewel branch).
>>
>> Ah alright, maybe this was the cause after all, let's get this in (or
>> push a branch to ceph-qa-suite so that I can schedule against that)
>
> I added a patch for the client-upgrade tests to the same branch. You
> should be able to just schedule with --suite-branch jewel-avoid-xenial
> instead of --suite-branch jewel (for both rados and
> upgrade/client-upgrade).
>
> sage
>
>
>> >>
>> >> The upgrade tests already explicitly call out trusty, so they should be
>> >> fine.
>> >>
>> >>> > updated the tracker at http://tracker.ceph.com/issues/17851#note-17,
>> >>> > with details of the test. I'll update the progress once the suite goes
>> >>> > through
>> >>>
>> >>> Had around 11 tests fail from 296 scheduled, there were a couple of
>> >>> valgrind issues on ceph-mon (which were seen at earlier runs on jewel as
>> >>> well) and an s3test failure, rest of the issues were looking related to
>> >>> infrastructure as they were failing to get specific version numbers from
>> >>> gitbuilders.
>> >>>
>> >>> Reported this issue as:
>> >>>
>> >>> http://tracker.ceph.com/issues/18089
>> >>>
>> >>> subsequent re runs are still failing with similar errors. The details
>> >>> are updated at
>> >>>
>> >>> http://tracker.ceph.com/issues/17487#note-37
>> >>>
>> >>> The upgrade suite has also failed with similar errors.
>> >>
>> >> Which one?
>> > The upgrade/client-upgrade suite (I hope this is the right upgrade
>> > suite), the errors are the same failed to fetch package version errors
>> > seen for the rados suite, so not actual test run errors yet.
>> >
>> > http://pulpito.ceph.com/abhi-2016-11-30_09:58:50-upgrade:client-upgrade-wip-jewel-10-2-4-distro-basic-smithi/
>> > is the run
>> >
Update on the current status,
For prs 12001 & 12167: (sage/Sam)
The upgrade passed with the exception of infernalis/jewel which failed
with RBD (test_librbd_api.sh) (jdillaman please approve)
http://pulpito.ceph.com/abhi-2016-12-02_09:45:37-upgrade:client-upgrade-wip-jewel-10-2-4-distro-basic-smithi/
the rados run (with the exception of the valgrind issue) are still
waiting for completion for 4 of the 296 jobs, this one suite always
seems to die when scheduled, this is the current url of the rados jobs
(still waiting)
http://pulpito.ceph.com/abhi-2016-12-02_10:06:39-rados-wip-jewel-10-2-4-rc-distro-basic-smithi/
for pr 12207: (josh)
the rgw suite also saw the same valgrind issue
(http://pulpito.ceph.com/abhi-2016-12-01_22:19:07-rgw-wip-jewel-10-2-4-rc-distro-basic-smithi/)
https://github.com/ceph/ceph/pull/12267, (mon cleanup) was merged on Sam's approval.
prs 11884 & 12067 (Sam, Alfredo):
the ceph create-keys pr #11884 hasn't been merged yet into jewel, is
there any suite that must be run to validate this. Same applies for
https://github.com/ceph/ceph/pull/12067 (OSDMon: MOSDBoot )
Further the details of all the runs so far are captured at http://tracker.ceph.com/issues/17487#note-37
I'll update soon once the rados jobs complete.
Best,
Abhishek
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-12-02 14:00 ` Abhishek L
@ 2016-12-05 11:53 ` Abhishek L
2016-12-07 13:06 ` Alfredo Deza
0 siblings, 1 reply; 26+ messages in thread
From: Abhishek L @ 2016-12-05 11:53 UTC (permalink / raw)
To: Sage Weil
Cc: Durgin, Josh, Yuri Weinstein, Dillaman, Jason, Alfredo Deza,
Samuel Just, Gregory Farnum, Ceph Development, Abhishek Varshney,
Dachary, Loic, Dachary, Loic, Nathan Cutler, Ilya Dryomov,
John Spray
Abhishek L writes:
> Sage Weil writes:
>
>> On Wed, 30 Nov 2016, Abhishek L wrote:
>>> Abhishek L writes:
>>>
>>> > Sage Weil writes:
>>> >
>>> >> On Wed, 30 Nov 2016, Abhishek L wrote:
>>> >>>
>>> >>> Abhishek L writes:
>>> >>>
>>> >>> > Sage Weil writes:
>>> >>> >
>>> >>> >> On Wed, 23 Nov 2016, Abhishek L wrote:
>>> >>> >>>
>>> >>> >>> Hi Sage, Greg,
>>> >>> >>>
>>> >>> >>>
>>> >>> >>> Yuri Weinstein writes:
>>> >>> >>>
>>> >>> >>> > See updated status - http://tracker.ceph.com/issues/17487#note-32
>>> >>> >>> >
>>> >>> >>> > Outstanding issues:
>>> >>> >>> >
>>> >>> >>> > knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>>> >>> >>> > pls review/approve, assumed Approved ?)
>>> >>> >>> >
>>> >>> >>> > upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>>> >>> >>> > pls review/approve, seems persistent, but maybe not a showstopper)
>>> >>> >>> >
>>> >>> >>> > upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
>>> >>> >>> > to make it pass, see issues in the tacker summary above)
>>> >>> >>> >
>>> >>> >>> > Sage, jewel 10.2.4 can be released as soon as you agree with the
>>> >>> >>> > findings/summary.
>>> >>> >>>
>>> >>> >>> Do you think we're ready to release 10.2.4 yet?
>>> >>> >>
>>> >>> >> I'm reproducing http://tracker.ceph.com/issues/17847 with logs to make
>>> >>> >> sure this isn't a regression.
>>> >>> >>
>>> >>> >> We can ignore the infernalis runs.
>>> >>> >>
>>> >>> >> I think we can ignore the knfs selinux issue too.. Greg, can you confirm?
>>> >>> >>
>>> >>> > Added the prs 12001 & 12167 on top of the jewel branch and scheduled rados runs
>>> >>> > at
>>> >>> > http://pulpito.ceph.com/abhi-2016-11-29_10:22:25-rados-wip-jewel-10-2-4-distro-basic-smithi/ &
>>> >>
>>> >> https://github.com/ceph/ceph-qa-suite/pull/1292
>>> >>
>>> >> avoids xenial for the rados upgrade tests (in jewel branch).
>>>
>>> Ah alright, maybe this was the cause after all, let's get this in (or
>>> push a branch to ceph-qa-suite so that I can schedule against that)
>>
>> I added a patch for the client-upgrade tests to the same branch. You
>> should be able to just schedule with --suite-branch jewel-avoid-xenial
>> instead of --suite-branch jewel (for both rados and
>> upgrade/client-upgrade).
>>
>> sage
>>
>>
>>> >>
>>> >> The upgrade tests already explicitly call out trusty, so they should be
>>> >> fine.
>>> >>
>>> >>> > updated the tracker at http://tracker.ceph.com/issues/17851#note-17,
>>> >>> > with details of the test. I'll update the progress once the suite goes
>>> >>> > through
>>> >>>
>>> >>> Had around 11 tests fail from 296 scheduled, there were a couple of
>>> >>> valgrind issues on ceph-mon (which were seen at earlier runs on jewel as
>>> >>> well) and an s3test failure, rest of the issues were looking related to
>>> >>> infrastructure as they were failing to get specific version numbers from
>>> >>> gitbuilders.
>>> >>>
>>> >>> Reported this issue as:
>>> >>>
>>> >>> http://tracker.ceph.com/issues/18089
>>> >>>
>>> >>> subsequent re runs are still failing with similar errors. The details
>>> >>> are updated at
>>> >>>
>>> >>> http://tracker.ceph.com/issues/17487#note-37
>>> >>>
>>> >>> The upgrade suite has also failed with similar errors.
>>> >>
>>> >> Which one?
>>> > The upgrade/client-upgrade suite (I hope this is the right upgrade
>>> > suite), the errors are the same failed to fetch package version errors
>>> > seen for the rados suite, so not actual test run errors yet.
>>> >
>>> > http://pulpito.ceph.com/abhi-2016-11-30_09:58:50-upgrade:client-upgrade-wip-jewel-10-2-4-distro-basic-smithi/
>>> > is the run
>>> >
> Update on the current status,
> For prs 12001 & 12167: (sage/Sam)
>
> The upgrade passed with the exception of infernalis/jewel which failed
> with RBD (test_librbd_api.sh) (jdillaman please approve)
>
> http://pulpito.ceph.com/abhi-2016-12-02_09:45:37-upgrade:client-upgrade-wip-jewel-10-2-4-distro-basic-smithi/
>
> the rados run (with the exception of the valgrind issue) are still
> waiting for completion for 4 of the 296 jobs, this one suite always
> seems to die when scheduled, this is the current url of the rados jobs
> (still waiting)
> http://pulpito.ceph.com/abhi-2016-12-02_10:06:39-rados-wip-jewel-10-2-4-rc-distro-basic-smithi/
Loic was able to spot the error in this run (which was a regression
introduced by f95ed3e, this was later fixed in
https://github.com/ceph/ceph-qa-suite/pull/1297, so running this suite
on the latest branch came out green,
http://pulpito.front.sepia.ceph.com/abhi-2016-12-05_10:25:26-rados-wip-jewel-10-2-4-rc-distro-basic-smithi/
Which means 12167 is also good to go.
>
>
> for pr 12207: (josh)
> the rgw suite also saw the same valgrind issue
> (http://pulpito.ceph.com/abhi-2016-12-01_22:19:07-rgw-wip-jewel-10-2-4-rc-distro-basic-smithi/)
>
> https://github.com/ceph/ceph/pull/12267, (mon cleanup) was merged on Sam's approval.
>
> prs 11884 & 12067 (Sam, Alfredo):
> the ceph create-keys pr #11884 hasn't been merged yet into jewel, is
> there any suite that must be run to validate this. Same applies for
> https://github.com/ceph/ceph/pull/12067 (OSDMon: MOSDBoot )
12067 was merged on Sam's approval as well. Which only leaves us with
11884.
Best,
Abhishek Lekshmanan
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-12-05 11:53 ` Abhishek L
@ 2016-12-07 13:06 ` Alfredo Deza
0 siblings, 0 replies; 26+ messages in thread
From: Alfredo Deza @ 2016-12-07 13:06 UTC (permalink / raw)
To: Abhishek L
Cc: Sage Weil, Durgin, Josh, Yuri Weinstein, Dillaman, Jason,
Samuel Just, Gregory Farnum, Ceph Development, Abhishek Varshney,
Dachary, Loic, Dachary, Loic, Nathan Cutler, Ilya Dryomov,
John Spray
On Mon, Dec 5, 2016 at 6:53 AM, Abhishek L <abhishek@suse.com> wrote:
>
> Abhishek L writes:
>
>> Sage Weil writes:
>>
>>> On Wed, 30 Nov 2016, Abhishek L wrote:
>>>> Abhishek L writes:
>>>>
>>>> > Sage Weil writes:
>>>> >
>>>> >> On Wed, 30 Nov 2016, Abhishek L wrote:
>>>> >>>
>>>> >>> Abhishek L writes:
>>>> >>>
>>>> >>> > Sage Weil writes:
>>>> >>> >
>>>> >>> >> On Wed, 23 Nov 2016, Abhishek L wrote:
>>>> >>> >>>
>>>> >>> >>> Hi Sage, Greg,
>>>> >>> >>>
>>>> >>> >>>
>>>> >>> >>> Yuri Weinstein writes:
>>>> >>> >>>
>>>> >>> >>> > See updated status - http://tracker.ceph.com/issues/17487#note-32
>>>> >>> >>> >
>>>> >>> >>> > Outstanding issues:
>>>> >>> >>> >
>>>> >>> >>> > knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>>>> >>> >>> > pls review/approve, assumed Approved ?)
>>>> >>> >>> >
>>>> >>> >>> > upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>>>> >>> >>> > pls review/approve, seems persistent, but maybe not a showstopper)
>>>> >>> >>> >
>>>> >>> >>> > upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
>>>> >>> >>> > to make it pass, see issues in the tacker summary above)
>>>> >>> >>> >
>>>> >>> >>> > Sage, jewel 10.2.4 can be released as soon as you agree with the
>>>> >>> >>> > findings/summary.
>>>> >>> >>>
>>>> >>> >>> Do you think we're ready to release 10.2.4 yet?
>>>> >>> >>
>>>> >>> >> I'm reproducing http://tracker.ceph.com/issues/17847 with logs to make
>>>> >>> >> sure this isn't a regression.
>>>> >>> >>
>>>> >>> >> We can ignore the infernalis runs.
>>>> >>> >>
>>>> >>> >> I think we can ignore the knfs selinux issue too.. Greg, can you confirm?
>>>> >>> >>
>>>> >>> > Added the prs 12001 & 12167 on top of the jewel branch and scheduled rados runs
>>>> >>> > at
>>>> >>> > http://pulpito.ceph.com/abhi-2016-11-29_10:22:25-rados-wip-jewel-10-2-4-distro-basic-smithi/ &
>>>> >>
>>>> >> https://github.com/ceph/ceph-qa-suite/pull/1292
>>>> >>
>>>> >> avoids xenial for the rados upgrade tests (in jewel branch).
>>>>
>>>> Ah alright, maybe this was the cause after all, let's get this in (or
>>>> push a branch to ceph-qa-suite so that I can schedule against that)
>>>
>>> I added a patch for the client-upgrade tests to the same branch. You
>>> should be able to just schedule with --suite-branch jewel-avoid-xenial
>>> instead of --suite-branch jewel (for both rados and
>>> upgrade/client-upgrade).
>>>
>>> sage
>>>
>>>
>>>> >>
>>>> >> The upgrade tests already explicitly call out trusty, so they should be
>>>> >> fine.
>>>> >>
>>>> >>> > updated the tracker at http://tracker.ceph.com/issues/17851#note-17,
>>>> >>> > with details of the test. I'll update the progress once the suite goes
>>>> >>> > through
>>>> >>>
>>>> >>> Had around 11 tests fail from 296 scheduled, there were a couple of
>>>> >>> valgrind issues on ceph-mon (which were seen at earlier runs on jewel as
>>>> >>> well) and an s3test failure, rest of the issues were looking related to
>>>> >>> infrastructure as they were failing to get specific version numbers from
>>>> >>> gitbuilders.
>>>> >>>
>>>> >>> Reported this issue as:
>>>> >>>
>>>> >>> http://tracker.ceph.com/issues/18089
>>>> >>>
>>>> >>> subsequent re runs are still failing with similar errors. The details
>>>> >>> are updated at
>>>> >>>
>>>> >>> http://tracker.ceph.com/issues/17487#note-37
>>>> >>>
>>>> >>> The upgrade suite has also failed with similar errors.
>>>> >>
>>>> >> Which one?
>>>> > The upgrade/client-upgrade suite (I hope this is the right upgrade
>>>> > suite), the errors are the same failed to fetch package version errors
>>>> > seen for the rados suite, so not actual test run errors yet.
>>>> >
>>>> > http://pulpito.ceph.com/abhi-2016-11-30_09:58:50-upgrade:client-upgrade-wip-jewel-10-2-4-distro-basic-smithi/
>>>> > is the run
>>>> >
>> Update on the current status,
>> For prs 12001 & 12167: (sage/Sam)
>>
>> The upgrade passed with the exception of infernalis/jewel which failed
>> with RBD (test_librbd_api.sh) (jdillaman please approve)
>>
>> http://pulpito.ceph.com/abhi-2016-12-02_09:45:37-upgrade:client-upgrade-wip-jewel-10-2-4-distro-basic-smithi/
>>
>> the rados run (with the exception of the valgrind issue) are still
>> waiting for completion for 4 of the 296 jobs, this one suite always
>> seems to die when scheduled, this is the current url of the rados jobs
>> (still waiting)
>> http://pulpito.ceph.com/abhi-2016-12-02_10:06:39-rados-wip-jewel-10-2-4-rc-distro-basic-smithi/
>
> Loic was able to spot the error in this run (which was a regression
> introduced by f95ed3e, this was later fixed in
> https://github.com/ceph/ceph-qa-suite/pull/1297, so running this suite
> on the latest branch came out green,
>
> http://pulpito.front.sepia.ceph.com/abhi-2016-12-05_10:25:26-rados-wip-jewel-10-2-4-rc-distro-basic-smithi/
>
> Which means 12167 is also good to go.
>>
>>
>> for pr 12207: (josh)
>> the rgw suite also saw the same valgrind issue
>> (http://pulpito.ceph.com/abhi-2016-12-01_22:19:07-rgw-wip-jewel-10-2-4-rc-distro-basic-smithi/)
>>
>> https://github.com/ceph/ceph/pull/12267, (mon cleanup) was merged on Sam's approval.
>>
>> prs 11884 & 12067 (Sam, Alfredo):
>> the ceph create-keys pr #11884 hasn't been merged yet into jewel, is
>> there any suite that must be run to validate this. Same applies for
>> https://github.com/ceph/ceph/pull/12067 (OSDMon: MOSDBoot )
>
> 12067 was merged on Sam's approval as well. Which only leaves us with
> 11884.
Bummed that 11884 didn't make it. It fixes issues where it hangs for
ever for any tool that tries to bootstrap and encounters problems
during deployment.
There isn't a specific suite that must be run to validate this.
>
> Best,
> Abhishek Lekshmanan
> SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-11-30 14:57 ` Abhishek L
2016-11-30 16:01 ` Sage Weil
@ 2016-11-30 16:09 ` Josh Durgin
2016-11-30 16:23 ` Abhishek L
1 sibling, 1 reply; 26+ messages in thread
From: Josh Durgin @ 2016-11-30 16:09 UTC (permalink / raw)
To: Abhishek L, Sage Weil
Cc: Gregory Farnum, Ceph Development, Abhishek Varshney,
Dachary, Loic, Dachary, Loic, Dillaman, Jason, Nathan Cutler,
Ilya Dryomov, John Spray, Yuri Weinstein
On 11/30/2016 06:57 AM, Abhishek L wrote:
>
> Abhishek L writes:
>
>> Sage Weil writes:
>>
>>> On Wed, 23 Nov 2016, Abhishek L wrote:
>>>>
>>>> Hi Sage, Greg,
>>>>
>>>>
>>>> Yuri Weinstein writes:
>>>>
>>>>> See updated status - http://tracker.ceph.com/issues/17487#note-32
>>>>>
>>>>> Outstanding issues:
>>>>>
>>>>> knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>>>>> pls review/approve, assumed Approved ?)
>>>>>
>>>>> upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>>>>> pls review/approve, seems persistent, but maybe not a showstopper)
>>>>>
>>>>> upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
>>>>> to make it pass, see issues in the tacker summary above)
>>>>>
>>>>> Sage, jewel 10.2.4 can be released as soon as you agree with the
>>>>> findings/summary.
>>>>
>>>> Do you think we're ready to release 10.2.4 yet?
>>>
>>> I'm reproducing http://tracker.ceph.com/issues/17847 with logs to make
>>> sure this isn't a regression.
>>>
>>> We can ignore the infernalis runs.
>>>
>>> I think we can ignore the knfs selinux issue too.. Greg, can you confirm?
>>>
>> Added the prs 12001 & 12167 on top of the jewel branch and scheduled rados runs
>> at
>> http://pulpito.ceph.com/abhi-2016-11-29_10:22:25-rados-wip-jewel-10-2-4-distro-basic-smithi/ &
>>
>> updated the tracker at http://tracker.ceph.com/issues/17851#note-17,
>> with details of the test. I'll update the progress once the suite goes
>> through
>
> Had around 11 tests fail from 296 scheduled, there were a couple of
> valgrind issues on ceph-mon (which were seen at earlier runs on jewel as
> well) and an s3test failure, rest of the issues were looking related to
> infrastructure as they were failing to get specific version numbers from
> gitbuilders.
>
> Reported this issue as:
>
> http://tracker.ceph.com/issues/18089
>
> subsequent re runs are still failing with similar errors. The details
> are updated at
>
> http://tracker.ceph.com/issues/17487#note-37
>
> The upgrade suite has also failed with similar errors.
>
> Overall it looks like we're close to cutting the release, as soon as we
> can get through these infra errors
We should also get https://github.com/ceph/ceph/pull/12207 in once the
master version (https://github.com/ceph/ceph/pull/12206) is merged -
likely later today.
Josh
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-11-30 16:09 ` Josh Durgin
@ 2016-11-30 16:23 ` Abhishek L
2016-11-30 16:35 ` Josh Durgin
0 siblings, 1 reply; 26+ messages in thread
From: Abhishek L @ 2016-11-30 16:23 UTC (permalink / raw)
To: Josh Durgin
Cc: Abhishek L, Sage Weil, Gregory Farnum, Ceph Development,
Abhishek Varshney, Dachary, Loic, Dachary, Loic, Dillaman, Jason,
Nathan Cutler, Ilya Dryomov, John Spray, Yuri Weinstein
Josh Durgin writes:
> On 11/30/2016 06:57 AM, Abhishek L wrote:
>>
>> Abhishek L writes:
>>
>>> Sage Weil writes:
>>>
>>>> On Wed, 23 Nov 2016, Abhishek L wrote:
>>>>>
>>>>> Hi Sage, Greg,
>>>>>
>>>>>
>>>>> Yuri Weinstein writes:
>>>>>
>>>>>> See updated status - http://tracker.ceph.com/issues/17487#note-32
>>>>>>
>>>>>> Outstanding issues:
>>>>>>
>>>>>> knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>>>>>> pls review/approve, assumed Approved ?)
>>>>>>
>>>>>> upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>>>>>> pls review/approve, seems persistent, but maybe not a showstopper)
>>>>>>
>>>>>> upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
>>>>>> to make it pass, see issues in the tacker summary above)
>>>>>>
>>>>>> Sage, jewel 10.2.4 can be released as soon as you agree with the
>>>>>> findings/summary.
>>>>>
>>>>> Do you think we're ready to release 10.2.4 yet?
>>>>
>>>> I'm reproducing http://tracker.ceph.com/issues/17847 with logs to make
>>>> sure this isn't a regression.
>>>>
>>>> We can ignore the infernalis runs.
>>>>
>>>> I think we can ignore the knfs selinux issue too.. Greg, can you confirm?
>>>>
>>> Added the prs 12001 & 12167 on top of the jewel branch and scheduled rados runs
>>> at
>>> http://pulpito.ceph.com/abhi-2016-11-29_10:22:25-rados-wip-jewel-10-2-4-distro-basic-smithi/ &
>>>
>>> updated the tracker at http://tracker.ceph.com/issues/17851#note-17,
>>> with details of the test. I'll update the progress once the suite goes
>>> through
>>
>> Had around 11 tests fail from 296 scheduled, there were a couple of
>> valgrind issues on ceph-mon (which were seen at earlier runs on jewel as
>> well) and an s3test failure, rest of the issues were looking related to
>> infrastructure as they were failing to get specific version numbers from
>> gitbuilders.
>>
>> Reported this issue as:
>>
>> http://tracker.ceph.com/issues/18089
>>
>> subsequent re runs are still failing with similar errors. The details
>> are updated at
>>
>> http://tracker.ceph.com/issues/17487#note-37
>>
>> The upgrade suite has also failed with similar errors.
>>
>> Overall it looks like we're close to cutting the release, as soon as we
>> can get through these infra errors
>
> We should also get https://github.com/ceph/ceph/pull/12207 in once the
> master version (https://github.com/ceph/ceph/pull/12206) is merged -
> likely later today.
Do you want it in 10.2.4 itself? what all qe suites would you recommend if so?
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-11-30 16:23 ` Abhishek L
@ 2016-11-30 16:35 ` Josh Durgin
0 siblings, 0 replies; 26+ messages in thread
From: Josh Durgin @ 2016-11-30 16:35 UTC (permalink / raw)
To: Abhishek L
Cc: Sage Weil, Gregory Farnum, Ceph Development, Abhishek Varshney,
Dachary, Loic, Dachary, Loic, Dillaman, Jason, Nathan Cutler,
Ilya Dryomov, John Spray, Yuri Weinstein
On 11/30/2016 08:23 AM, Abhishek L wrote:
>
> Josh Durgin writes:
>
>> On 11/30/2016 06:57 AM, Abhishek L wrote:
>>>
>>> Abhishek L writes:
>>>
>>>> Sage Weil writes:
>>>>
>>>>> On Wed, 23 Nov 2016, Abhishek L wrote:
>>>>>>
>>>>>> Hi Sage, Greg,
>>>>>>
>>>>>>
>>>>>> Yuri Weinstein writes:
>>>>>>
>>>>>>> See updated status - http://tracker.ceph.com/issues/17487#note-32
>>>>>>>
>>>>>>> Outstanding issues:
>>>>>>>
>>>>>>> knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>>>>>>> pls review/approve, assumed Approved ?)
>>>>>>>
>>>>>>> upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>>>>>>> pls review/approve, seems persistent, but maybe not a showstopper)
>>>>>>>
>>>>>>> upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
>>>>>>> to make it pass, see issues in the tacker summary above)
>>>>>>>
>>>>>>> Sage, jewel 10.2.4 can be released as soon as you agree with the
>>>>>>> findings/summary.
>>>>>>
>>>>>> Do you think we're ready to release 10.2.4 yet?
>>>>>
>>>>> I'm reproducing http://tracker.ceph.com/issues/17847 with logs to make
>>>>> sure this isn't a regression.
>>>>>
>>>>> We can ignore the infernalis runs.
>>>>>
>>>>> I think we can ignore the knfs selinux issue too.. Greg, can you confirm?
>>>>>
>>>> Added the prs 12001 & 12167 on top of the jewel branch and scheduled rados runs
>>>> at
>>>> http://pulpito.ceph.com/abhi-2016-11-29_10:22:25-rados-wip-jewel-10-2-4-distro-basic-smithi/ &
>>>>
>>>> updated the tracker at http://tracker.ceph.com/issues/17851#note-17,
>>>> with details of the test. I'll update the progress once the suite goes
>>>> through
>>>
>>> Had around 11 tests fail from 296 scheduled, there were a couple of
>>> valgrind issues on ceph-mon (which were seen at earlier runs on jewel as
>>> well) and an s3test failure, rest of the issues were looking related to
>>> infrastructure as they were failing to get specific version numbers from
>>> gitbuilders.
>>>
>>> Reported this issue as:
>>>
>>> http://tracker.ceph.com/issues/18089
>>>
>>> subsequent re runs are still failing with similar errors. The details
>>> are updated at
>>>
>>> http://tracker.ceph.com/issues/17487#note-37
>>>
>>> The upgrade suite has also failed with similar errors.
>>>
>>> Overall it looks like we're close to cutting the release, as soon as we
>>> can get through these infra errors
>>
>> We should also get https://github.com/ceph/ceph/pull/12207 in once the
>> master version (https://github.com/ceph/ceph/pull/12206) is merged -
>> likely later today.
>
> Do you want it in 10.2.4 itself? what all qe suites would you recommend if so?
Yes, we need to not break the librados API or ABI in stable series
especially.
The rgw suite would be enough - it's the only thing that really uses
setxattr.
Josh
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-11-23 10:39 ` Abhishek L
2016-11-23 14:39 ` Gregory Farnum
2016-11-23 14:42 ` Sage Weil
@ 2016-11-30 21:56 ` Ken Dreyer
2016-11-30 22:09 ` Abhishek L
2016-12-01 13:26 ` Alfredo Deza
3 siblings, 1 reply; 26+ messages in thread
From: Ken Dreyer @ 2016-11-30 21:56 UTC (permalink / raw)
To: Abhishek L
Cc: Sage Weil, Gregory Farnum, Ceph Development, Abhishek Varshney,
Dachary, Loic, Dachary, Loic, Dillaman, Jason, Nathan Cutler,
Ilya Dryomov, John Spray, Durgin, Josh, Yuri Weinstein
On Wed, Nov 23, 2016 at 3:39 AM, Abhishek L <abhishek@suse.com> wrote:
> Do you think we're ready to release 10.2.4 yet?
It would be great to get https://github.com/ceph/ceph/pull/12001 in
before v10.2.4 as well.
- Ken
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-11-30 21:56 ` Ken Dreyer
@ 2016-11-30 22:09 ` Abhishek L
2016-12-01 8:09 ` Yoann Moulin
0 siblings, 1 reply; 26+ messages in thread
From: Abhishek L @ 2016-11-30 22:09 UTC (permalink / raw)
To: Ken Dreyer
Cc: Abhishek L, Sage Weil, Gregory Farnum, Ceph Development,
Abhishek Varshney, Dachary, Loic, Dachary, Loic, Dillaman, Jason,
Nathan Cutler, Ilya Dryomov, John Spray, Durgin, Josh,
Yuri Weinstein
On Wed, Nov 30, 2016 at 10:56 PM, Ken Dreyer <kdreyer@redhat.com> wrote:
> On Wed, Nov 23, 2016 at 3:39 AM, Abhishek L <abhishek@suse.com> wrote:
>> Do you think we're ready to release 10.2.4 yet?
>
> It would be great to get https://github.com/ceph/ceph/pull/12001 in
> before v10.2.4 as well.
The current test runs already include 12001,
once we have a green run we should be having it in 10.2.4
Best,
Abhishek
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-11-30 22:09 ` Abhishek L
@ 2016-12-01 8:09 ` Yoann Moulin
0 siblings, 0 replies; 26+ messages in thread
From: Yoann Moulin @ 2016-12-01 8:09 UTC (permalink / raw)
To: Abhishek L, Ken Dreyer
Cc: Abhishek L, Sage Weil, Gregory Farnum, Ceph Development,
Abhishek Varshney, Dachary, Loic, Dachary, Loic, Dillaman, Jason,
Nathan Cutler, Ilya Dryomov, John Spray, Durgin, Josh,
Yuri Weinstein
Hello,
>> It would be great to get https://github.com/ceph/ceph/pull/12001 in
>> before v10.2.4 as well.
>
> The current test runs already include 12001,
> once we have a green run we should be having it in 10.2.4
I don't see the pull request 12001 for 10.2.4[1] release, it's in the 10.2.5[2]
1. http://tracker.ceph.com/issues/17487
2. http://tracker.ceph.com/issues/17851
Do you plan to include it in the 10.2.4 ?
Best regards,
--
Yoann Moulin
EPFL IC-IT
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ceph v10.2.4 QE validation status
2016-11-23 10:39 ` Abhishek L
` (2 preceding siblings ...)
2016-11-30 21:56 ` Ken Dreyer
@ 2016-12-01 13:26 ` Alfredo Deza
3 siblings, 0 replies; 26+ messages in thread
From: Alfredo Deza @ 2016-12-01 13:26 UTC (permalink / raw)
To: Abhishek L
Cc: Sage Weil, Gregory Farnum, Ceph Development, Abhishek Varshney,
Dachary, Loic, Dachary, Loic, Dillaman, Jason, Nathan Cutler,
Ilya Dryomov, John Spray, Durgin, Josh, Yuri Weinstein
On Wed, Nov 23, 2016 at 5:39 AM, Abhishek L <abhishek@suse.com> wrote:
>
> Hi Sage, Greg,
>
>
> Yuri Weinstein writes:
>
>> See updated status - http://tracker.ceph.com/issues/17487#note-32
>>
>> Outstanding issues:
>>
>> knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>> pls review/approve, assumed Approved ?)
>>
>> upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>> pls review/approve, seems persistent, but maybe not a showstopper)
>>
>> upgrade/infernalis-x (jewel) - deprecated (Nathan is still working
>> to make it pass, see issues in the tacker summary above)
>>
>> Sage, jewel 10.2.4 can be released as soon as you agree with the
>> findings/summary.
>
> Do you think we're ready to release 10.2.4 yet?
I thought this PR had made it https://github.com/ceph/ceph/pull/11749
but apparently it didn't for Jewel
The backport PR for Jewel is https://github.com/ceph/ceph/pull/11884
It fixes quite a bit of issues we've had for deployment. It is only
relevant when bootstrapping, when keys are needed for the first time.
Hopefully we can include it as well.
>
>
> Best,
> Abhishek
>>
>> Thx
>> YuriW
>>
>> On Fri, Nov 11, 2016 at 8:47 AM, Yuri Weinstein <yweinste@redhat.com> wrote:
>>> Detailed summary of the QE Validation can be found here
>>> http://tracker.ceph.com/issues/17487#note-32
>>>
>>> The following suites were in scope of this point release validation:
>>>
>>> rados (subset 35/50 297 jobs)
>>> rbd
>>> rgw
>>> fs
>>> krbd
>>> kcephfs
>>> knfs
>>> rest
>>> hadoop
>>> samba
>>> ceph-deploy
>>> ceph-disk
>>> upgrade/client-upgrade
>>> upgrade/hammer-x (jewel)
>>> upgrade/infernalis-x (jewel) - deprecated
>>> powercycle
>>> (please let me know if any suites are missing from this ^ list)
>>>
>>> ==============================
>>> Issues requiring approval/decision:
>>>
>>> fs - http://tracker.ceph.com/issues/17832 (Greg pls review/approve)
>>>
>>> krbd - http://tracker.ceph.com/issues/17221 (Jason, Ilya pls review/approve)
>>>
>>> knfs - http://tracker.ceph.com/issues/16397 (same as in v10.2.3, Greg
>>> pls review/approve)
>>>
>>> upgrade/hammer-x (jewel) - http://tracker.ceph.com/issues/17847 ((Sage
>>> pls review/approve)
>>>
>>> upgrade/infernalis-x (jewel) - deprecated, but Nathan is still working
>>> to make it pass (Sage pls review/approve)
>>>
>>> powercycle - in progress
>>>
>>> Thx
>>> YuriW
>
>
> --
> Abhishek Lekshmanan
> SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 26+ messages in thread