Switch to PR server model for PR bumps - 1 week

Switch to PR server model for PR bumps - 1 week

[Richard Purdie]

I've made it clear we want and need to switch to the PR service for
handling PR bumps. I'd now like to put a deadline in place for doing
this, namely that I'm going to stop requiring PR bumps in patches as of
1 week's time. The TSC did discuss this and our conclusion was that we
need to switch early in this release cycle to give time for any issues
to get resolved. Now is a good time to do this as I think we're ready.

There were a number of open bugs related to the PR service[1]. We've
looked through them and they are now resolved with the exception of one
related to the incremental git PV issue for which we have a plan in
place and patches ready.

We've taken on board feedback about documentation and the following wiki
page has been setup which details the current situation:

https://wiki.yoctoproject.org/wiki/PR_Service

I appreciate this is going to be a disruptive change for some but I
believe it is in the best interests of the project longer term and that
we will have a better system as an end result of this. If people do run
into problems, please send email or file bugs. I'm trying to ensure
there are some people with time available to look into these issues as a
priority so we can make a smooth transition.

Cheers,

Richard

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=3310

Re: Switch to PR server model for PR bumps - 1 week

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2803 bytes --]

On Wed, Dec 05, 2012 at 02:13:59PM +0000, Richard Purdie wrote:
> I've made it clear we want and need to switch to the PR service for
> handling PR bumps. I'd now like to put a deadline in place for doing
> this, namely that I'm going to stop requiring PR bumps in patches as of
> 1 week's time. The TSC did discuss this and our conclusion was that we
> need to switch early in this release cycle to give time for any issues
> to get resolved. Now is a good time to do this as I think we're ready.
> 
> There were a number of open bugs related to the PR service[1]. We've
> looked through them and they are now resolved with the exception of one
> related to the incremental git PV issue for which we have a plan in
> place and patches ready.
> 
> We've taken on board feedback about documentation and the following wiki
> page has been setup which details the current situation:
> 
> https://wiki.yoctoproject.org/wiki/PR_Service
> 
> I appreciate this is going to be a disruptive change for some but I
> believe it is in the best interests of the project longer term and that
> we will have a better system as an end result of this. If people do run
> into problems, please send email or file bugs. I'm trying to ensure
> there are some people with time available to look into these issues as a
> priority so we can make a smooth transition.

After reading this wiki page I have few questions (I admit I haven't
tried it yet).

1) How do you limit access to shared prserv instance?
   Can you define some group as RO, some RW?
   For RO access it would be great to export state from remote PRSERV 
   on first connection and then continue to increment locally (with
   PRSERV running on localhost)
2) If you have slightly different builder (e.g. someone with extra BSP
   or different host distro), then he will have different checksums, so
   different AUTOPR values and you cannot share binary feed with such
   builder, right? I know this wasn't possible (or at least safe) to use
   before too. But it should be more clear from documentation.
3) Is it possible to use OEBasic together with PRSERV or should bitbake
   show fatal error when PRSERV is used toghether with OEBasic?
4) Is there plan to import current PR values from recipes, so that
   bitbake can parse all enabled layers, gather checksum-PR values and
   export it for bitbake-prserv-tool to import it as inital state?
5) Is there plan to remove PR from all recipes or are they ignored with
   PRSERV in use and can stay in for some time? If there is functionality 
   for 4) then we should tag all layers just before PR/PRINC are removed, 
   so someone can create own initial export from last known PR state.

Cheers,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Switch to PR server model for PR bumps - 1 week

[Richard Purdie]

On Wed, 2012-12-05 at 15:45 +0100, Martin Jansa wrote:
> On Wed, Dec 05, 2012 at 02:13:59PM +0000, Richard Purdie wrote:
> > I've made it clear we want and need to switch to the PR service for
> > handling PR bumps. I'd now like to put a deadline in place for doing
> > this, namely that I'm going to stop requiring PR bumps in patches as of
> > 1 week's time. The TSC did discuss this and our conclusion was that we
> > need to switch early in this release cycle to give time for any issues
> > to get resolved. Now is a good time to do this as I think we're ready.
> > 
> > There were a number of open bugs related to the PR service[1]. We've
> > looked through them and they are now resolved with the exception of one
> > related to the incremental git PV issue for which we have a plan in
> > place and patches ready.
> > 
> > We've taken on board feedback about documentation and the following wiki
> > page has been setup which details the current situation:
> > 
> > https://wiki.yoctoproject.org/wiki/PR_Service
> > 
> > I appreciate this is going to be a disruptive change for some but I
> > believe it is in the best interests of the project longer term and that
> > we will have a better system as an end result of this. If people do run
> > into problems, please send email or file bugs. I'm trying to ensure
> > there are some people with time available to look into these issues as a
> > priority so we can make a smooth transition.
> 
> After reading this wiki page I have few questions (I admit I haven't
> tried it yet).
> 
> 1) How do you limit access to shared prserv instance?
>    Can you define some group as RO, some RW?
>    For RO access it would be great to export state from remote PRSERV 
>    on first connection and then continue to increment locally (with
>    PRSERV running on localhost)

This isn't implemented as things stand. As you point out, a true RO
connection causes problems...

> 2) If you have slightly different builder (e.g. someone with extra BSP
>    or different host distro), then he will have different checksums, so
>    different AUTOPR values and you cannot share binary feed with such
>    builder, right? I know this wasn't possible (or at least safe) to use
>    before too. But it should be more clear from documentation.

If the extra BSP uses its own package_arch it would probably be ok,
otherwise not. The host distro only affects native/cross packages and
not the checksum itself so again, that should also be ok?

> 3) Is it possible to use OEBasic together with PRSERV or should bitbake
>    show fatal error when PRSERV is used toghether with OEBasic?


Added to the page: "The service works with both OEBasic and OEBasicHash
generators, with the understanding that PR bumps happen when the
checksum changes and the different generator mechanisms change
signatures under different circumstances."

> 4) Is there plan to import current PR values from recipes, so that
>    bitbake can parse all enabled layers, gather checksum-PR values and
>    export it for bitbake-prserv-tool to import it as inital state?
> 5) Is there plan to remove PR from all recipes or are they ignored with
>    PRSERV in use and can stay in for some time? If there is functionality 
>    for 4) then we should tag all layers just before PR/PRINC are removed, 
>    so someone can create own initial export from last known PR state.

Added to the page: "As implemented, values from the PR service are
included into the PR field as an addition of the form ".X" so r0 becomes
r0.1, r0.2 and so on. This allows existing PR values to be used for
whatever reasons allowing manual PR bumps should it be necessary."

(should cover both questions)

As for removing PR, as recipes are upgraded, I'm expecting we will
remove PR values so we should get a smooth transition. We can still have
PR values, it will just become an exception rather than the rule.

Cheers,

Richard

Re: Switch to PR server model for PR bumps - 1 week

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 4351 bytes --]

On Wed, Dec 05, 2012 at 03:19:53PM +0000, Richard Purdie wrote:
> On Wed, 2012-12-05 at 15:45 +0100, Martin Jansa wrote:
> > On Wed, Dec 05, 2012 at 02:13:59PM +0000, Richard Purdie wrote:
> > > I've made it clear we want and need to switch to the PR service for
> > > handling PR bumps. I'd now like to put a deadline in place for doing
> > > this, namely that I'm going to stop requiring PR bumps in patches as of
> > > 1 week's time. The TSC did discuss this and our conclusion was that we
> > > need to switch early in this release cycle to give time for any issues
> > > to get resolved. Now is a good time to do this as I think we're ready.
> > > 
> > > There were a number of open bugs related to the PR service[1]. We've
> > > looked through them and they are now resolved with the exception of one
> > > related to the incremental git PV issue for which we have a plan in
> > > place and patches ready.
> > > 
> > > We've taken on board feedback about documentation and the following wiki
> > > page has been setup which details the current situation:
> > > 
> > > https://wiki.yoctoproject.org/wiki/PR_Service
> > > 
> > > I appreciate this is going to be a disruptive change for some but I
> > > believe it is in the best interests of the project longer term and that
> > > we will have a better system as an end result of this. If people do run
> > > into problems, please send email or file bugs. I'm trying to ensure
> > > there are some people with time available to look into these issues as a
> > > priority so we can make a smooth transition.
> > 
> > After reading this wiki page I have few questions (I admit I haven't
> > tried it yet).
> > 
> > 1) How do you limit access to shared prserv instance?
> >    Can you define some group as RO, some RW?
> >    For RO access it would be great to export state from remote PRSERV 
> >    on first connection and then continue to increment locally (with
> >    PRSERV running on localhost)
> 
> This isn't implemented as things stand. As you point out, a true RO
> connection causes problems...
> 
> > 2) If you have slightly different builder (e.g. someone with extra BSP
> >    or different host distro), then he will have different checksums, so
> >    different AUTOPR values and you cannot share binary feed with such
> >    builder, right? I know this wasn't possible (or at least safe) to use
> >    before too. But it should be more clear from documentation.
> 
> If the extra BSP uses its own package_arch it would probably be ok,
> otherwise not. The host distro only affects native/cross packages and
> not the checksum itself so again, that should also be ok?
> 
> > 3) Is it possible to use OEBasic together with PRSERV or should bitbake
> >    show fatal error when PRSERV is used toghether with OEBasic?
> 
> 
> Added to the page: "The service works with both OEBasic and OEBasicHash
> generators, with the understanding that PR bumps happen when the
> checksum changes and the different generator mechanisms change
> signatures under different circumstances."
> 
> > 4) Is there plan to import current PR values from recipes, so that
> >    bitbake can parse all enabled layers, gather checksum-PR values and
> >    export it for bitbake-prserv-tool to import it as inital state?
> > 5) Is there plan to remove PR from all recipes or are they ignored with
> >    PRSERV in use and can stay in for some time? If there is functionality 
> >    for 4) then we should tag all layers just before PR/PRINC are removed, 
> >    so someone can create own initial export from last known PR state.
> 
> Added to the page: "As implemented, values from the PR service are
> included into the PR field as an addition of the form ".X" so r0 becomes
> r0.1, r0.2 and so on. This allows existing PR values to be used for
> whatever reasons allowing manual PR bumps should it be necessary."
> 
> (should cover both questions)
> 
> As for removing PR, as recipes are upgraded, I'm expecting we will
> remove PR values so we should get a smooth transition. We can still have
> PR values, it will just become an exception rather than the rule.

So default "hidden" r0 is still used in such cases and PRSERV adds again
just .X.

Thanks for answers.

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Switch to PR server model for PR bumps - 1 week

[Richard Purdie]

On Wed, 2012-12-05 at 16:24 +0100, Martin Jansa wrote:
> On Wed, Dec 05, 2012 at 03:19:53PM +0000, Richard Purdie wrote:
> > On Wed, 2012-12-05 at 15:45 +0100, Martin Jansa wrote:
> > Added to the page: "As implemented, values from the PR service are
> > included into the PR field as an addition of the form ".X" so r0 becomes
> > r0.1, r0.2 and so on. This allows existing PR values to be used for
> > whatever reasons allowing manual PR bumps should it be necessary."
> > 
> > (should cover both questions)
> > 
> > As for removing PR, as recipes are upgraded, I'm expecting we will
> > remove PR values so we should get a smooth transition. We can still have
> > PR values, it will just become an exception rather than the rule.
> 
> So default "hidden" r0 is still used in such cases and PRSERV adds again
> just .X.

Exactly.

> Thanks for answers.

They were good questions :)

Cheers,

Richard

Re: Switch to PR server model for PR bumps - 1 week

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1226 bytes --]

Ah and one more:
6) what about export of LOCALCOUNT from bb persistent cache to import it as
initial values of AUTOINC for git recipes?

Cheers,
On Dec 5, 2012 4:37 PM, "Richard Purdie" <richard.purdie@linuxfoundation.org>
wrote:

> On Wed, 2012-12-05 at 16:24 +0100, Martin Jansa wrote:
> > On Wed, Dec 05, 2012 at 03:19:53PM +0000, Richard Purdie wrote:
> > > On Wed, 2012-12-05 at 15:45 +0100, Martin Jansa wrote:
> > > Added to the page: "As implemented, values from the PR service are
> > > included into the PR field as an addition of the form ".X" so r0
> becomes
> > > r0.1, r0.2 and so on. This allows existing PR values to be used for
> > > whatever reasons allowing manual PR bumps should it be necessary."
> > >
> > > (should cover both questions)
> > >
> > > As for removing PR, as recipes are upgraded, I'm expecting we will
> > > remove PR values so we should get a smooth transition. We can still
> have
> > > PR values, it will just become an exception rather than the rule.
> >
> > So default "hidden" r0 is still used in such cases and PRSERV adds again
> > just .X.
>
> Exactly.
>
> > Thanks for answers.
>
> They were good questions :)
>
> Cheers,
>
> Richard
>
>
>

[-- Attachment #2: Type: text/html, Size: 1670 bytes --]

Re: Switch to PR server model for PR bumps - 1 week

[Richard Purdie]

On Thu, 2012-12-06 at 05:20 +0100, Martin Jansa wrote:
> Ah and one more:
> 6) what about export of LOCALCOUNT from bb persistent cache to import
> it as initial values of AUTOINC for git recipes?

Constantin has posted two patches (one bitbake, one OE) which change the
fetcher to use the PR server for this data. It adds functionality so the
import/export works and it becomes a standard part of the PR service.

Once we switch to the PR service, we can merge those and this problem
becomes resolved.

Cheers,

Richard