From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============2350165778081258723==" MIME-Version: 1.0 From: Harris, James R Subject: Re: [SPDK] Add Jenkins job for periodic (nightly?) test run against upstream DPDK stable? Date: Fri, 15 Feb 2019 19:46:58 +0000 Message-ID: In-Reply-To: 065C38D4-5D57-4E13-9FF2-DFEB62DC21DD@oracle.com List-ID: To: spdk@lists.01.org --===============2350165778081258723== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Hi Lance, In this case, "DPDK master" means the tip of DPDK. So to summarize, we hav= e three cases here: 1) SPDK master v. DPDK master 2) SPDK master v. DPDK last stable release 3) SPDK last stable release v. DPDK master Since #3 is stable from an SPDK point of view, nightly tests here are fine. I think #1 has to be nightly as well. Since DPDK master is constantly in f= lux, we don't want our SPDK patches to get blocked due to DPDK bugs that ge= t checked in between releases. It's #2 that's the question. I talked with Darek about it this morning. H= ere are the pros and cons as I see them: Per-patch testing: Pros: * immediately find incompatibilities between DPDK last stable release and D= PDK submodule Cons: * only a subset of SPDK tests run against DPDK last stable release - runnin= g all SPDK tests (or at least that don't have known current incompatibiliti= es) against both DPDK last stable release and DPDK submodule would require = doubling the CI systems Nightly testing: Pros: * we run all SPDK tests against DPDK last stable release Cons: * patches could potentially get merged to master which expose an incompatib= ility between DPDK stable and the DPDK submodule I think to start, we get the nightly testing in place ASAP. If a nightly t= est fails it's a critical issue that likely leads to an immediate revert. = We can still add per-patch testing on top of it - maybe on just a very limi= ted number of SPDK tests - but we'll still need to rely on the nightly test= ing for full coverage. I'd be interested in hearing other opinions on this. -Jim =EF=BB=BFOn 2/15/19, 11:25 AM, "Lance Hartmann ORACLE" wrote: = Responses interlined below: = = = > On Feb 15, 2019, at 8:26 AM, Latecki, Karol wrote: > = > Hi everyone, > We had a talk about this here and we have following proposal about te= sting vs DPDK. > We'd like to start with 3 nightly job configurations, with e-mail not= ifications enabled: = = Just nightly? I think Jim had initially expressed an interest in doin= g a run per-patch I believe with the hope of identifying and addressing any= issues more quickly. I'll be eager to hear his (and others') feedback on= this. = = > * SPDK master vs DPDK master = When you write "DPDK master" above -- you mean the SPDK's DPDK submodul= e? If so, maybe we might refer to that as "sDPDK". = > * SPDK master vs DPDK last stable release = Again, clarification: By "DPDK last stable release", you certainly mea= n the *upstream* DPDK's most recently cut stable release (suggested notatio= n, "uDPDK"). However, a new uDPDK stable release is cut one month after = each SPDK release (based on the current respective release schedules). So= would someone (or some automated task?) poll for a newly cut uDPDK stable = and then change the Jenkins job so it started building against that? = = = = > * SPDK last stable release vs DPDK master = = By "DPDK master", you mean the current master for the SPDK's DPDK submo= dule (sDPDK)? = > That way we'd know fast enough about new issues between SPDK and DPDK= . = > = > I think that your point Jim still applies for these configurations - = we'd have to configure them not to test things that we know are failing. Th= at also means that we would have to re-evaluate the configs from time to ti= me. = = -- Lance Hartmann = = = > = > Karol > = > -----Original Message----- > From: Harris, James R = > Sent: Thursday, February 14, 2019 3:32 PM > To: Storage Performance Development Kit ; Stojac= zyk, Dariusz ; Latecki, Karol > Subject: Re: [SPDK] Add Jenkins job for periodic (nightly?) test run = against upstream DPDK stable? > = > We could also just switch some of our existing jobs to build/run agai= nst the latest stable DPDK release. We can be careful about picking jobs t= hat don't have dependencies on patches from our DPDK submodule. > = > On 2/14/19, 6:39 AM, "SPDK on behalf of Luse, Paul E" wrote: > = > So we talked a lot about this in last night's community meeting a= nd everyone agreed as a first step we need to: > = > * create new Jenkins subjob for per-patch testing that tests again= st last stable DPDK release > * configure the job to not test things that we know will fail (lik= e crypto due to a patch we pushed upstream that won't land until 19.05). T= here may be others like this well (some secondary process tests, etc) so it= may take a little experimentation and investigation to get the job right. > * We'll need to review job settings at each release to enable/disa= ble features to test based on the contents of both the SPDK and DPDK releas= es at the time. > = > Darek/Karol, can you guys get started on this and keep everyone po= sted? > = > Thanks! = > Paul > = > PS: Thanks Lance for driving this discussion > = > -----Original Message----- > From: Luse, Paul E = > Sent: Tuesday, February 12, 2019 5:10 AM > To: Storage Performance Development Kit > Subject: RE: [SPDK] Add Jenkins job for periodic (nightly?) test r= un against upstream DPDK stable? > = > I agree that would be a great addition! For some reason I thought = that was already being done somewhere by someone, no matter though, setting= up a Jenkins job is the right way to do it now. Karol, can you take a look= at this and let everyone know your thoughts? > = > Thanks > Paul > = > -----Original Message----- > From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Lance= Hartmann ORACLE > Sent: Monday, February 11, 2019 4:44 PM > To: spdk(a)lists.01.org > Subject: [SPDK] Add Jenkins job for periodic (nightly?) test run a= gainst upstream DPDK stable? > = > = > Group, > = > I recently had a sideband chat with Jim regarding our SPDK testing= with the DPDK, and he urged I open this up to the email list for more disc= ussion. > = > When a Linux distro produces SPDK package rpms, they satisfy the D= PDK dependency via a specified stable DPDK release from upstream that has b= een packaged. However, on the run up to each SPDK quarterly release, we are= constantly building and testing against the SPDK's fork'd copy of the DPDK= (as a git-submodule) on which we often (virtually always?) have our own pa= tches. While this works fine for a development perspective as we work tow= ard the next SPDK release, it's problematic when the time arrives to produc= e new SPDK packages because they will be built against a pristine DPDK stab= le release which very likely is not identical with what we have been testin= g against. When I described this with Jim, he agreed that it would be an = excellent idea to augment our SPDK Jenkins setup with some tests which woul= d run against the latest "pristine" (i.e. upstream stable) DPDK release. = With that in place, when the time arrives to produce new SPDK packages, we = will have gained the testing c > onfidence of having validated running with the exact version of t= he DPDK against which the SPDK packages will have been built (and will run = alongside at runtime). > = > Having said all that, I did float another idea on how we could, if= we really needed to, produce SPDK package rpms based on the combination of= a stable DPDK upstream release and our own collection of patches to it. = The rpm packaging system natively supports such a build, but as you might q= uickly deduce it comes at the cost of additional maintenance efforts which = can become very unpalatable, esp. when/if some of our patches are never acc= epted/merged into the official upstream DPDK tree. > = > = > -- > Lance Hartmann > = > = > = > _______________________________________________ > SPDK mailing list > SPDK(a)lists.01.org > https://lists.01.org/mailman/listinfo/spdk > _______________________________________________ > SPDK mailing list > SPDK(a)lists.01.org > https://lists.01.org/mailman/listinfo/spdk > = > = > -------------------------------------------------------------------- > = > Intel Technology Poland sp. z o.o. > ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VI= I Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07= -52-316 | Kapital zakladowy 200.000 PLN. > = > Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego ad= resata i moze zawierac informacje poufne. W razie przypadkowego otrzymania = tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; = jakiekolwiek > przegladanie lub rozpowszechnianie jest zabronione. > This e-mail and any attachments may contain confidential material for= the sole use of the intended recipient(s). If you are not the intended rec= ipient, please contact the sender and delete all copies; any review or dist= ribution by > others is strictly prohibited. > _______________________________________________ > SPDK mailing list > SPDK(a)lists.01.org > https://lists.01.org/mailman/listinfo/spdk = = --===============2350165778081258723==--