[GIT PULL] zstd changes for v6.3-rc1

[GIT PULL] zstd changes for v6.3-rc1

[Nick Terrell]

The following changes since commit c9c3395d5e3dcc6daee66c6908354d47bf98cb0c:

 Linux 6.2 (2023-02-19 14:24:22 -0800)

are available in the Git repository at:

 https://github.com/terrelln/linux.git tags/zstd-linus-v6.3

for you to fetch changes up to 9844ed7e93967c0eaa27cab2fc80583e38696a0e:

 Merge tag 'v6.2' into zstd-linus (2023-03-02 22:15:55 -0800)

----------------------------------------------------------------
Zstd fixes for v6.3

A small number of fixes for zstd-v1.5.2.

I'm not pulling in zstd-v1.5.4 from upstream this release because it
didn't have any time to bake in linux-next, but I'm aiming for the next
update in v6.4.

Signed-off-by: Nick Terrell <terrelln@fb.com>

----------------------------------------------------------------
Enrico Scholz (1):
     zstd: fix assert() logic

Kees Cook (1):
     lib: zstd: Fix -Wstringop-overflow warning

Nick Terrell (3):
     Merge branch 'main' into zstd-linus
     lib: zstd: Backport fix for in-place decompression
     Merge tag 'v6.2' into zstd-linus

lib/zstd/common/zstd_deps.h           |  2 +-
lib/zstd/decompress/huf_decompress.c  |  2 +-
lib/zstd/decompress/zstd_decompress.c | 25 ++++++++++++++++++++++---
3 files changed, 24 insertions(+), 5 deletions(-)

Re: [GIT PULL] zstd changes for v6.3-rc1

[Linus Torvalds]

On Thu, Mar 2, 2023 at 10:23 PM Nick Terrell <terrelln@meta.com> wrote:
>
> Zstd fixes for v6.3

Grr.

This tree had five commits in it.

Of the five, two were merges.

And of those two, ABSOLUTELY NONE had any explanation for them AT ALL.

Honestly, I pulled this, and was then *so* fed up with this kind of
garbage that I just decided that I'm better off without this all.

So this got all undone again, and I'm not pulling this kind of sh*t again.

I'm *very* fed up with having to tell people the same thing over and over again.

Just stop doing merges if you can't be bothered to do do them right.

                Linus

Re: [GIT PULL] zstd changes for v6.3-rc1

[Nick Terrell]

> On Mar 3, 2023, at 9:28 AM, Linus Torvalds <torvalds@linux-foundation.org> wrote:
> 
> !-------------------------------------------------------------------|
>  This Message Is From an External Sender
> 
> |-------------------------------------------------------------------!
> 
> On Thu, Mar 2, 2023 at 10:23 PM Nick Terrell <terrelln@meta.com> wrote:
>> 
>> Zstd fixes for v6.3
> 
> Grr.
> 
> This tree had five commits in it.
> 
> Of the five, two were merges.
> 
> And of those two, ABSOLUTELY NONE had any explanation for them AT ALL.
> 
> Honestly, I pulled this, and was then *so* fed up with this kind of
> garbage that I just decided that I'm better off without this all.
> 
> So this got all undone again, and I'm not pulling this kind of sh*t again.
> 
> I'm *very* fed up with having to tell people the same thing over and over again.
> 
> Just stop doing merges if you can't be bothered to do do them right.

I’m sorry, I thought this was standard practice for merging in the mainline branch.
I’ve been following this article [0], which recommended not rebasing my public
trees, so I merged in the mainline kernel instead.

I am a newer maintainer, and either I missed the documentation, or it slipped my
mind. I will search for documentation about how to write better merge requests.

Meanwhile, what do you want me to do with this tree? If the merges are unacceptable,
I can rebase the 3 patches in my tree onto v6.2, and re-send the pull request.

Best,
Nick Terrell

[0] https://lwn.net/Articles/512720/


>                Linus

Re: [GIT PULL] zstd changes for v6.3-rc1

[Linus Torvalds]

On Fri, Mar 3, 2023 at 9:54 AM Nick Terrell <terrelln@meta.com> wrote:
>
> I’m sorry, I thought this was standard practice for merging in the mainline branch.

Absolutely NOT.

I have harped on "DO NOT DO BACK MERGES" for closer to two _decades_ by now.

When you do zstd development, you should normally have absolutely
*ZERO* reason to merge non-zstd work.

> I’ve been following this article [0], which recommended not rebasing my public
> trees, so I merged in the mainline kernel instead.

Half right.

You should not rebase your public trees.

But you should not merge mainline either.

Exactly what relevance does <N> *thousand* driver updates have to zstd?

There are reasons to merge, but they have to be real, explicit, and
MENTIONED IN THE MERGE.

And no, "update to latest" is simply not a reason.

When close to half the commits are pointless merges that have no
explanation, I will not pull (if I notice).

                 Linus

Re: [GIT PULL] zstd changes for v6.3-rc1

[Nick Terrell]

> On Mar 3, 2023, at 9:59 AM, Linus Torvalds <torvalds@linux-foundation.org> wrote:
> 
> > 
> On Fri, Mar 3, 2023 at 9:54 AM Nick Terrell <terrelln@meta.com> wrote:
>> 
>> I’m sorry, I thought this was standard practice for merging in the mainline branch.
> 
> Absolutely NOT.
> 
> I have harped on "DO NOT DO BACK MERGES" for closer to two _decades_ by now.
> 
> When you do zstd development, you should normally have absolutely
> *ZERO* reason to merge non-zstd work.
> 
>> I’ve been following this article [0], which recommended not rebasing my public
>> trees, so I merged in the mainline kernel instead.
> 
> Half right.
> 
> You should not rebase your public trees.
> 
> But you should not merge mainline either.
> 
> Exactly what relevance does <N> *thousand* driver updates have to zstd?
> 
> There are reasons to merge, but they have to be real, explicit, and
> MENTIONED IN THE MERGE.
> 
> And no, "update to latest" is simply not a reason.
> 
> When close to half the commits are pointless merges that have no
> explanation, I will not pull (if I notice).

Thanks for taking the time to explain, I appreciate it, and will not make the same
mistake again in the future.

What do you prefer I do with my current tree? I guess I can either:
- Leave the merges in and keep a stable tree
- Fix up my tree and clean up the merges, but break the stable tree

Best,
Nick Terrell

>                 Linus

Re: [GIT PULL] zstd changes for v6.3-rc1

[Linus Torvalds]

On Fri, Mar 3, 2023 at 10:03 AM Nick Terrell <terrelln@meta.com> wrote:
>
> What do you prefer I do with my current tree? I guess I can either:
> - Leave the merges in and keep a stable tree
> - Fix up my tree and clean up the merges, but break the stable tree

In this case, since I'm not taking it during the merge window anyway,
just reset and rebase and basically start a new fixes branch that can
get pulled next week after it's been in that form in linux-next.

All of the actual real commits (ie the non-merge ones) seem to be
fixes, so let's just treat them as such.

And for sanity reasons, don't start the branch at a "random commit of
the day". Particularly not during the merge window. You want the
starting point to be something that doesn't have random issues that we
may not even know about yet - simply because you want *your* branch to
be as stable as possible, and you should aim to have to worry about
issues with zstd, not some random "oops, that particular base had a
random bug because of some merge window thing that wasn't found until
-rc3".

So start the fixes branch at a reasonable stableish point (in this
case presumably just 6.2).

Of course, the same thing is true of new development branches too, not
just fixes branches.

It's a bad idea to build a house on quick-sand, and it's a bad idea to
start new development on some unstable source base.

(One special case of "start development at a stable point" is to
simply continue off some old point of your previous development. Then
it's stable not because it was some known release, but simply because
it's what you used previously and had no issues with).

That whole "pick a stable point" thing is worth noting also for the
case when you _do_ decide that yes, you do need to rebase or
back-merge, and you have a good reason to do so. Don't merge a random
commit of the day. Merge a _specific_ commit that you can explain why
you picked _that_ point to merge.

Of course, things like tagged releases aren't necessarily stable by
definition - we find things to fix after release too. But at least
they are hopefully "we at least tried to make sure it wasn't a bad
point".

              Linus

Re: [GIT PULL] zstd changes for v6.3-rc1

[Nick Terrell]

> On Mar 3, 2023, at 10:16 AM, Linus Torvalds <torvalds@linux-foundation.org> wrote:
> 
> On Fri, Mar 3, 2023 at 10:03 AM Nick Terrell <terrelln@meta.com> wrote:
>> 
>> What do you prefer I do with my current tree? I guess I can either:
>> - Leave the merges in and keep a stable tree
>> - Fix up my tree and clean up the merges, but break the stable tree
> 
> In this case, since I'm not taking it during the merge window anyway,
> just reset and rebase and basically start a new fixes branch that can
> get pulled next week after it's been in that form in linux-next.

I will go ahead and do that.

> All of the actual real commits (ie the non-merge ones) seem to be
> fixes, so let's just treat them as such.
> 
> And for sanity reasons, don't start the branch at a "random commit of
> the day". Particularly not during the merge window. You want the
> starting point to be something that doesn't have random issues that we
> may not even know about yet - simply because you want *your* branch to
> be as stable as possible, and you should aim to have to worry about
> issues with zstd, not some random "oops, that particular base had a
> random bug because of some merge window thing that wasn't found until
> -rc3".
> 
> So start the fixes branch at a reasonable stableish point (in this
> case presumably just 6.2).
> 
> Of course, the same thing is true of new development branches too, not
> just fixes branches.
> 
> It's a bad idea to build a house on quick-sand, and it's a bad idea to
> start new development on some unstable source base.
> 
> (One special case of "start development at a stable point" is to
> simply continue off some old point of your previous development. Then
> it's stable not because it was some known release, but simply because
> it's what you used previously and had no issues with).
> 
> That whole "pick a stable point" thing is worth noting also for the
> case when you _do_ decide that yes, you do need to rebase or
> back-merge, and you have a good reason to do so. Don't merge a random
> commit of the day. Merge a _specific_ commit that you can explain why
> you picked _that_ point to merge.
> 
> Of course, things like tagged releases aren't necessarily stable by
> definition - we find things to fix after release too. But at least
> they are hopefully "we at least tried to make sure it wasn't a bad
> point".

Thanks for the time you’ve taken helping me. I will also take some
more time to better familiarize myself with the maintainer workflow, so I
can avoid other mistakes that I don’t know I’m making.

Best,
Nick Terrell

>              Linus

Re: [GIT PULL] zstd changes for v6.3-rc1

[Theodore Ts'o]

On Fri, Mar 03, 2023 at 06:03:50PM +0000, Nick Terrell wrote:
> 
> What do you prefer I do with my current tree? I guess I can either:
> - Leave the merges in and keep a stable tree
> - Fix up my tree and clean up the merges, but break the stable tree

Do you have any downstream trees that depend on your tree?  If you
don't anyone who might be using your tree as a base for forther work
(linux-next doesn't count, since it rewinds every working day).

In general, for most "leaf" trees, rewinding your branches is not a
big deal.  There are some people who worship at the altar of "stable
git branches which never be rewound, forever and ever, Amen".  But
that is really a religious belief, and it's one that I don't subscribe
to.

Sure, if someone is depending on your git tree then rewinding the
branch can cause them problems.  But not all subsystem trees are used
by others as a basis for further work!  There are benefits to
rewinding / rebasing patches; sometimes I'll do rewind the ext4 dev
branch to add a "Tested-By", or to drop a patch which I had merged,
but then later on I discovered that it causes regressions.  In that
case, I'll drop the patch using git rebase -i since it can make life
easier who are doing git bisects.

Cheers,

					- Ted

Re: [GIT PULL] zstd changes for v6.3-rc1

[Linus Torvalds]

On Fri, Mar 3, 2023 at 9:59 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> You should not rebase your public trees.
>
> But you should not merge mainline either.

So lwn has several articles about this, the one I found quickly with google was

    https://lwn.net/Articles/791284/

and it's worth noting that the rule is "don't rebase" and "don't merge
from upstream", but as always, those are rules that then have to be
balanced against other rules and concerns.

Sometimes rebasing _is_ the right thing to do. Sometimes the public
history simply ended up being so full of garbage that the only right
thing to do is to just admit that and start again.

And sometimes merging from upstream _is_ the right thing to do. Maybe
there is something really important that made it upstream that is very
relevant.

But both situations need to be things that you really think about, and
have a reason for. And when you do a merge, that reason should very
much go into the merge message.

For rebases, there's no "rebase message", so those you basically have
to explain at pull time ("this was rebased last week to fix bad
problem XYZ").

              Linus