ANN: kernel git branches and process changes

ANN: kernel git branches and process changes

[Paul Moore]

Hello all,

As of today I'm making some changes to the LSM, SELinux, and audit
kernel git repositories to make it easier to get changes into
linux-next and to provide a more formal approach to dealing with
significant changes that may need some time in a "staging" branch
before inclusion into a main development branch.  Just as before, I've
documented the kernel development process in the README.md located in
the main/default branch of each kernel git repository, but I'm also
including the new process below.  If anyone has any questions please
speak up.

It may take me a day or two to fully setup the new branches in each
repository, but given where we are at in the current development cycle
I don't expect this to have any major impacts.  If this does
significantly impact any work in progress, please let me know and I
promise we'll figure something out.

Thanks for your understanding.

## Kernel Source Branches and Development Process

### Kernel Source Branches

There are four primary git branches associated with the development process:
stable-X.Y, dev, dev-staging, and next.  In addition to these four primary
branches there are also topic specific, work in progress branches that start
with a "working-" prefix; these branches can generally be ignored unless you
happen to be involved in the development of that particular topic.  The
management of these topic branches can vary depending on a number of factors,
but the details of each branch will be communicated in the relevant discussion
threads on the upstream mailing list.

#### stable-X.Y branch

The stable-X.Y branch is intended for stable kernel patches and is based on
Linus' X.Y-rc1 tag, or a later X.Y.Z stable kernel release tag as needed.
If serious problems are identified and a patch is developed during the kernel's
release candidate cycle, it may be a candidate for stable kernel marking and
inclusion into the stable-X.Y branch.  The main Linux kernel's documentation
on stable kernel patches has more information both on what patches may be
stable kernel candidates, and how to mark those patches appropriately; upstream
mailing list discussions on the merits of marking the patch for stable can also
be expected.  Once a patch has been merged into the stable-X.Y branch and spent
a day or two in the next branch (see the next branch notes), it will be sent to
Linus for merging into the next release candidate or final kernel release (see
the notes on pull requests in this document).  If the patch has been properly
marked for stable, the other stable kernel trees will attempt to backport the
patch as soon as it is present in Linus' tree, see the main Linux kernel
documentation for more details.

Unless specifically requested, developers should not base their patches on the
stable-X.Y branch.  Any merge conflicts that arise from merging patches
submitted upstream will be handled by the maintainer, although help and/or may
be requested in extreme cases.

#### dev branch

The dev branch is intended for development patches targeting the upcoming merge
window, and is based on Linus' latest X.Y-rc1 tag, or a later rc tag as needed
to avoid serious bugs, merge conflicts, or other significant problems.  This
branch is the primary development branch where the majority of patches are
merged during the normal kernel development cycle.  Patches merged into the
dev branch will be present in the next branch (see the next branch notes) and
will be sent to Linus during the next merge window.

Developers should use the dev branch a stable basis for their own development
work, only under extreme circumstances will the dev branch be rebased during
the X.Y-rc cycle and the maintainer will be responsible for resolving any
merge conflicts, although help and/or may be requested in extreme cases.

#### dev-staging branch

The dev-staging branch is intended for development patches that are not
targeting a specific merge window.  The dev-staging branch exists as a staging
area for the main dev branch and as such its use will be unpredictable and it
will be rebased as needed.  Patches merged into the dev-staging branch will be
present in the next branch (see the next branch notes) and should find their
way into the primary dev branch at some point in the future, although that is
not guaranteed.

Unless specifically requested, developers should not use the dev-staging branch
as a basis for any development work.

#### next branch

The next branch is a composite branch built by merging the latest stable-X.Y,
dev, and dev-staging branches in that order.  The main focus of the next branch
is to provide a single branch for linux-next integration testing that contains
all of the commits from the component branches.  The next branch will be
updated whenever there is a change to any one of the component branches, but it
will remain frozen during the merge window so as to cooperate with the wishes
of the linux-next team.

While developers can use the next branch as a basis for development, the dev
branch would likely be a more suitable, and stable, base.

### Kernel Development Process

After Linus closes the kernel merge window closes upstream, the stable-X.Y
branch associated with the current kernel release candidate, the dev branch,
and potentially the dev-staging branch (see the dev-staging branch notes) will
be reset to match the latest vX.Y-rc1 tag in Linus' tree.  The next branch, as
a composite branch composed from these branches, will be updated as a result.

During the development cycle that starts with the close of the kernel merge
window and ends with the tagged kernel release, patches will be accepted into
the stable-X.Y and dev branches as described in their respective sections in
this document.  While patches will be accepted into the stable-X.Y branch at
any point in time, significant changes will likely not be accepted into the dev
branch when there are two or less weeks left in the development cycle; this
typically means that only critical bugfixes are accepted once the vX.Y-rc6
kernel is released.  During this time the next branch will be regenerated on an
as needed basis based on changes in the component branches, and pull requests
will be sent as needed to Linus for patches in the stable-X.Y branch.

Once Linus releases the final vX.Y kernel and the merge window opens, two
things will happen.  The first is that the dev branch will be duplicated into
a new stable-X'.Y' branch, representing the new upcoming kernel release, and
the second is that a pull request will be sent from this branch for inclusion
into the current merge window.  During the merge window process the dev and
next branches should be frozen, although there is a possibility that some
patches may be merged merged into dev-staging for testing or process related
reasons.

#### Pull Requests for Linus

In order to send a pull request to Linus, either for a critical bugfix or as
part of the merge window, a signed git tag must be created that points to the
pull request point.  The tag should be named using the "{subsystem}-pr-{date}"
format and can be generated with the following git command:

```
% git tag -s -m "{subsystem}/stable-X'.Y' PR {date}" {subsystem}-pr-{date}
```

Once the signed tag has been created, it should be used as the basis for the
pull request.

-- 
paul-moore.com

Re: ANN: kernel git branches and process changes

[Bagas Sanjaya]

[-- Attachment #1: Type: text/plain, Size: 2679 bytes --]

On Wed, Oct 25, 2023 at 05:11:51PM -0400, Paul Moore wrote:
> #### stable-X.Y branch
> 
> The stable-X.Y branch is intended for stable kernel patches and is based on
> Linus' X.Y-rc1 tag, or a later X.Y.Z stable kernel release tag as needed.
> If serious problems are identified and a patch is developed during the kernel's
> release candidate cycle, it may be a candidate for stable kernel marking and
> inclusion into the stable-X.Y branch.  The main Linux kernel's documentation
> on stable kernel patches has more information both on what patches may be
> stable kernel candidates, and how to mark those patches appropriately; upstream
> mailing list discussions on the merits of marking the patch for stable can also
> be expected.  Once a patch has been merged into the stable-X.Y branch and spent
> a day or two in the next branch (see the next branch notes), it will be sent to
> Linus for merging into the next release candidate or final kernel release (see
> the notes on pull requests in this document).  If the patch has been properly
> marked for stable, the other stable kernel trees will attempt to backport the
> patch as soon as it is present in Linus' tree, see the main Linux kernel
> documentation for more details.
> 
> Unless specifically requested, developers should not base their patches on the
> stable-X.Y branch.  Any merge conflicts that arise from merging patches
> submitted upstream will be handled by the maintainer, although help and/or may
> be requested in extreme cases.
> 
> #### dev branch
> 
> The dev branch is intended for development patches targeting the upcoming merge
> window, and is based on Linus' latest X.Y-rc1 tag, or a later rc tag as needed
> to avoid serious bugs, merge conflicts, or other significant problems.  This
> branch is the primary development branch where the majority of patches are
> merged during the normal kernel development cycle.  Patches merged into the
> dev branch will be present in the next branch (see the next branch notes) and
> will be sent to Linus during the next merge window.
> 
> Developers should use the dev branch a stable basis for their own development
> work, only under extreme circumstances will the dev branch be rebased during
> the X.Y-rc cycle and the maintainer will be responsible for resolving any
> merge conflicts, although help and/or may be requested in extreme cases.
> 

If I have patches targetting current (not next) release cycle, either for
stabilizing that cycle or for stable backports, I have to base it on dev
branch (not stable-X.Y), right?

Confused...

-- 
An old man doll... just what I always wanted! - Clara

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: ANN: kernel git branches and process changes

[Paul Moore]

On Wed, Oct 25, 2023 at 9:25 PM Bagas Sanjaya <bagasdotme@gmail.com> wrote:
> On Wed, Oct 25, 2023 at 05:11:51PM -0400, Paul Moore wrote:
> > #### stable-X.Y branch
> >
> > The stable-X.Y branch is intended for stable kernel patches and is based on
> > Linus' X.Y-rc1 tag, or a later X.Y.Z stable kernel release tag as needed.
> > If serious problems are identified and a patch is developed during the kernel's
> > release candidate cycle, it may be a candidate for stable kernel marking and
> > inclusion into the stable-X.Y branch.  The main Linux kernel's documentation
> > on stable kernel patches has more information both on what patches may be
> > stable kernel candidates, and how to mark those patches appropriately; upstream
> > mailing list discussions on the merits of marking the patch for stable can also
> > be expected.  Once a patch has been merged into the stable-X.Y branch and spent
> > a day or two in the next branch (see the next branch notes), it will be sent to
> > Linus for merging into the next release candidate or final kernel release (see
> > the notes on pull requests in this document).  If the patch has been properly
> > marked for stable, the other stable kernel trees will attempt to backport the
> > patch as soon as it is present in Linus' tree, see the main Linux kernel
> > documentation for more details.
> >
> > Unless specifically requested, developers should not base their patches on the
> > stable-X.Y branch.  Any merge conflicts that arise from merging patches
> > submitted upstream will be handled by the maintainer, although help and/or may
> > be requested in extreme cases.
> >
> > #### dev branch
> >
> > The dev branch is intended for development patches targeting the upcoming merge
> > window, and is based on Linus' latest X.Y-rc1 tag, or a later rc tag as needed
> > to avoid serious bugs, merge conflicts, or other significant problems.  This
> > branch is the primary development branch where the majority of patches are
> > merged during the normal kernel development cycle.  Patches merged into the
> > dev branch will be present in the next branch (see the next branch notes) and
> > will be sent to Linus during the next merge window.
> >
> > Developers should use the dev branch a stable basis for their own development
> > work, only under extreme circumstances will the dev branch be rebased during
> > the X.Y-rc cycle and the maintainer will be responsible for resolving any
> > merge conflicts, although help and/or may be requested in extreme cases.
> >
>
> If I have patches targetting current (not next) release cycle, either for
> stabilizing that cycle or for stable backports, I have to base it on dev
> branch (not stable-X.Y), right?
>
> Confused...

I would prefer that yes.  I know it sounds a little odd, but I'd
rather see folks develop and test against what we believe to be the
latest subsystem code, which is what lives in the dev branch.  If
there are merge conflicts, I'd rather we see them when merging the fix
into the stable-X.Y branch so we are aware of the conflict at
development/submission time rather than waiting for the next merge
window.  Having done this for a number of years at this point, I've
learned to appreciate seeing merge conflicts as early in the
development cycle as possible and I've also learned that they are
often not as scary in practice as we imagine.

Of course all of this is a general rule since you can't specify every
single situation in guidance like the above; if there is something
that you believe is particularly ugly you can always write the mailing
list, or me directly if the issue is sensitive, and we can sort it
out.  If after a few months (year?) this proves to be too painful we
can always revisit this and update the policy, it's definitely not set
in stone.

Hopefully you are less confused now?

-- 
paul-moore.com

Re: ANN: kernel git branches and process changes

[Bagas Sanjaya]

On 26/10/2023 09:16, Paul Moore wrote:
> On Wed, Oct 25, 2023 at 9:25 PM Bagas Sanjaya <bagasdotme@gmail.com> wrote:
>> On Wed, Oct 25, 2023 at 05:11:51PM -0400, Paul Moore wrote:
>>> #### stable-X.Y branch
>>>
>>> The stable-X.Y branch is intended for stable kernel patches and is based on
>>> Linus' X.Y-rc1 tag, or a later X.Y.Z stable kernel release tag as needed.
>>> If serious problems are identified and a patch is developed during the kernel's
>>> release candidate cycle, it may be a candidate for stable kernel marking and
>>> inclusion into the stable-X.Y branch.  The main Linux kernel's documentation
>>> on stable kernel patches has more information both on what patches may be
>>> stable kernel candidates, and how to mark those patches appropriately; upstream
>>> mailing list discussions on the merits of marking the patch for stable can also
>>> be expected.  Once a patch has been merged into the stable-X.Y branch and spent
>>> a day or two in the next branch (see the next branch notes), it will be sent to
>>> Linus for merging into the next release candidate or final kernel release (see
>>> the notes on pull requests in this document).  If the patch has been properly
>>> marked for stable, the other stable kernel trees will attempt to backport the
>>> patch as soon as it is present in Linus' tree, see the main Linux kernel
>>> documentation for more details.
>>>
>>> Unless specifically requested, developers should not base their patches on the
>>> stable-X.Y branch.  Any merge conflicts that arise from merging patches
>>> submitted upstream will be handled by the maintainer, although help and/or may
>>> be requested in extreme cases.
>>>
>>> #### dev branch
>>>
>>> The dev branch is intended for development patches targeting the upcoming merge
>>> window, and is based on Linus' latest X.Y-rc1 tag, or a later rc tag as needed
>>> to avoid serious bugs, merge conflicts, or other significant problems.  This
>>> branch is the primary development branch where the majority of patches are
>>> merged during the normal kernel development cycle.  Patches merged into the
>>> dev branch will be present in the next branch (see the next branch notes) and
>>> will be sent to Linus during the next merge window.
>>>
>>> Developers should use the dev branch a stable basis for their own development
>>> work, only under extreme circumstances will the dev branch be rebased during
>>> the X.Y-rc cycle and the maintainer will be responsible for resolving any
>>> merge conflicts, although help and/or may be requested in extreme cases.
>>>
>>
>> If I have patches targetting current (not next) release cycle, either for
>> stabilizing that cycle or for stable backports, I have to base it on dev
>> branch (not stable-X.Y), right?
>>
>> Confused...
> 
> I would prefer that yes.  I know it sounds a little odd, but I'd
> rather see folks develop and test against what we believe to be the
> latest subsystem code, which is what lives in the dev branch.  If
> there are merge conflicts, I'd rather we see them when merging the fix
> into the stable-X.Y branch so we are aware of the conflict at
> development/submission time rather than waiting for the next merge
> window.  Having done this for a number of years at this point, I've
> learned to appreciate seeing merge conflicts as early in the
> development cycle as possible and I've also learned that they are
> often not as scary in practice as we imagine.
> 
> Of course all of this is a general rule since you can't specify every
> single situation in guidance like the above; if there is something
> that you believe is particularly ugly you can always write the mailing
> list, or me directly if the issue is sensitive, and we can sort it
> out.  If after a few months (year?) this proves to be too painful we
> can always revisit this and update the policy, it's definitely not set
> in stone.
> 
> Hopefully you are less confused now?
> 

OK, thanks for clarification!

-- 
An old man doll... just what I always wanted! - Clara