[GIT PULL] Bitmap patches for v6.0-rc1

[GIT PULL] Bitmap patches for v6.0-rc1

[Yury Norov]

The following changes since commit a111daf0c53ae91e71fd2bfe7497862d14132e3e:

  Linux 5.19-rc3 (2022-06-19 15:06:47 -0500)

are available in the Git repository at:

  https://github.com/norov/linux.git/ tags/bitmap-6.0-rc1

for you to fetch changes up to 36d4b36b69590fed99356a4426c940a253a93800:

  lib/nodemask: inline next_node_in() and node_random() (2022-08-01 08:13:21 -0700)

----------------------------------------------------------------
Bitmap patches for v6.0-rc1

This branch includes:

Qu Wenruo:
lib: bitmap: fix the duplicated comments on bitmap_to_arr64()
https://lore.kernel.org/lkml/0d85e1dbad52ad7fb5787c4432bdb36cbd24f632.1656063005.git.wqu@suse.com/

Alexander Lobakin:
bitops: let optimize out non-atomic bitops on compile-time constants
https://lore.kernel.org/lkml/20220624121313.2382500-1-alexandr.lobakin@intel.com/T/

Yury Norov:
lib: cleanup bitmap-related headers
https://lore.kernel.org/linux-arm-kernel/YtCVeOGLiQ4gNPSf@yury-laptop/T/#m305522194c4d38edfdaffa71fcaaf2e2ca00a961

Alexander Lobakin:
x86/olpc: fix 'logical not is only applied to the left hand side'
https://www.spinics.net/lists/kernel/msg4440064.html

Yury Norov:
lib/nodemask: inline wrappers around bitmap
https://lore.kernel.org/all/20220723214537.2054208-1-yury.norov@gmail.com/

----------------------------------------------------------------
Hi Linus,

Resending the pull request from Aug 02. I forgot to add GIT PULL tag, and
that's why it may not pass your filters.

The patch "headers/deps: mm: Split <linux/gfp_types.h> out of <linux/gfp.h>"
conflicts with 70c248aca9e7e ("mm: kasan: Skip unpoisoning of user pages").

The gfp_types patch moves macros from gfp.h to gfp_types.h, but the
70c248aca9e7e modifies original gfp.h, and therefore git can't apply
gfp_types patch cleanly. The solution is to propagate the following 
change from gfp.h to new gfp_types.h:

 #define GFP_DMA32      __GFP_DMA32
 #define GFP_HIGHUSER   (GFP_USER | __GFP_HIGHMEM)
 #define GFP_HIGHUSER_MOVABLE   (GFP_HIGHUSER | __GFP_MOVABLE | \
-                        __GFP_SKIP_KASAN_POISON)
+                        __GFP_SKIP_KASAN_POISON | __GFP_SKIP_KASAN_UNPOISON)
 #define GFP_TRANSHUGE_LIGHT    ((GFP_HIGHUSER_MOVABLE | __GFP_COMP | \
                         __GFP_NOMEMALLOC | __GFP_NOWARN) & ~__GFP_RECLAIM)
 #define GFP_TRANSHUGE  (GFP_TRANSHUGE_LIGHT | __GFP_DIRECT_RECLAIM)

Thanks,
Yury
---

Alexander Lobakin (13):
      ia64, processor: fix -Wincompatible-pointer-types in ia64_get_irr()
      bitops: always define asm-generic non-atomic bitops
      bitops: unify non-atomic bitops prototypes across architectures
      bitops: define const_*() versions of the non-atomics
      bitops: wrap non-atomic bitops with a transparent macro
      bitops: let optimize out non-atomic bitops on compile-time constants
      net/ice: fix initializing the bitmap in the switch code
      bitmap: don't assume compiler evaluates small mem*() builtins calls
      lib: test_bitmap: add compile-time optimization/evaluations assertions
      lib/bitmap: fix off-by-one in bitmap_to_arr64()
      lib/test_bitmap: test the tail after bitmap_to_arr64()
      iommu/vt-d: avoid invalid memory access via node_online(NUMA_NO_NODE)
      x86/olpc: fix 'logical not is only applied to the left hand side'

Ingo Molnar (2):
      headers/deps: mm: Optimize <linux/gfp.h> header dependencies
      headers/deps: mm: Split <linux/gfp_types.h> out of <linux/gfp.h>

Qu Wenruo (1):
      lib: bitmap: fix the duplicated comments on bitmap_to_arr64()

Yury Norov (10):
      arm: align find_bit declarations with generic kernel
      lib/bitmap: change return types to bool where appropriate
      lib/bitmap: change type of bitmap_weight to unsigned long
      cpumask: change return types to bool where appropriate
      lib/cpumask: change return types to unsigned where appropriate
      lib/cpumask: move trivial wrappers around find_bit to the header
      headers/deps: mm: align MANITAINERS and Docs with new gfp.h structure
      lib/cpumask: move some one-line wrappers to header file
      powerpc: drop dependency on <asm/machdep.h> in archrandom.h
      lib/nodemask: inline next_node_in() and node_random()

 Documentation/core-api/mm-api.rst                  |   8 +-
 MAINTAINERS                                        |   2 +-
 arch/alpha/include/asm/bitops.h                    |  32 +-
 arch/arm/include/asm/bitops.h                      |  18 +-
 arch/hexagon/include/asm/bitops.h                  |  24 +-
 arch/ia64/include/asm/bitops.h                     |  42 +--
 arch/ia64/include/asm/processor.h                  |   2 +-
 arch/m68k/include/asm/bitops.h                     |  49 ++-
 arch/powerpc/include/asm/archrandom.h              |   9 +-
 arch/powerpc/kernel/setup-common.c                 |  12 +
 arch/s390/include/asm/bitops.h                     |  61 ++--
 arch/sh/include/asm/bitops-op32.h                  |  34 +-
 arch/sparc/include/asm/bitops_32.h                 |  18 +-
 arch/sparc/lib/atomic32.c                          |  12 +-
 arch/x86/include/asm/bitops.h                      |  22 +-
 arch/x86/platform/olpc/olpc-xo1-sci.c              |   2 +-
 drivers/dma/ti/k3-udma.c                           |   6 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c            |   2 +-
 drivers/gpu/drm/i915/display/intel_display_power.c |   2 +-
 drivers/gpu/drm/msm/disp/mdp5/mdp5_smp.c           |   2 +-
 drivers/iommu/intel/dmar.c                         |   2 +-
 drivers/iommu/intel/iommu.c                        |   2 +-
 drivers/net/ethernet/intel/ice/ice_switch.c        |   2 +-
 drivers/net/ethernet/mellanox/mlx4/fw.c            |   2 +-
 drivers/net/wireless/ath/ath9k/htc_drv_debug.c     |   2 +-
 drivers/net/wireless/ath/carl9170/debug.c          |   2 +-
 include/asm-generic/bitops/generic-non-atomic.h    | 161 ++++++++++
 .../asm-generic/bitops/instrumented-non-atomic.h   |  35 ++-
 include/asm-generic/bitops/non-atomic.h            | 121 +------
 .../bitops/non-instrumented-non-atomic.h           |  16 +
 include/linux/bitmap.h                             |  37 ++-
 include/linux/bitops.h                             |  50 +++
 include/linux/cpumask.h                            | 125 ++++++--
 include/linux/gfp.h                                | 348 +--------------------
 include/linux/gfp_types.h                          | 348 +++++++++++++++++++++
 include/linux/nodemask.h                           |  24 +-
 lib/Makefile                                       |   2 +-
 lib/bitmap.c                                       |  11 +-
 lib/cpumask.c                                      |  97 +-----
 lib/nodemask.c                                     |   8 -
 lib/test_bitmap.c                                  |  68 ++++
 tools/include/asm-generic/bitops/non-atomic.h      |  34 +-
 tools/include/linux/bitmap.h                       |  12 +-
 tools/include/linux/bitops.h                       |  16 +
 tools/lib/bitmap.c                                 |   6 +-
 45 files changed, 1091 insertions(+), 799 deletions(-)
 create mode 100644 include/asm-generic/bitops/generic-non-atomic.h
 create mode 100644 include/asm-generic/bitops/non-instrumented-non-atomic.h
 create mode 100644 include/linux/gfp_types.h

Re: [GIT PULL] Bitmap patches for v6.0-rc1

[Michael Ellerman]

Hi Linus,

This pull request has a conflict with the random tree in some powerpc
code.

The random tree removed CONFIG_ARCH_RANDOM and changed the arguments and
names of the arch_get_random_xxx() calls, meanwhile the commit in the
bitmap tree moved our arch_get_random_seed_long() into a C file.

Yury Norov <yury.norov@gmail.com> writes:
> The following changes since commit a111daf0c53ae91e71fd2bfe7497862d14132e3e:
>
>   Linux 5.19-rc3 (2022-06-19 15:06:47 -0500)
>
> are available in the Git repository at:
>
>   https://github.com/norov/linux.git/ tags/bitmap-6.0-rc1
>
> for you to fetch changes up to 36d4b36b69590fed99356a4426c940a253a93800:
>
>   lib/nodemask: inline next_node_in() and node_random() (2022-08-01 08:13:21 -0700)
...
>  arch/powerpc/include/asm/archrandom.h              |   9 +-
>  arch/powerpc/kernel/setup-common.c                 |  12 +

There is a textual conflict in archrandom.h. But there's also a fixup
needed in setup-common.c.

Mark spotted it in linux-next but the resolution is not quite right, I
describe the correct result here:

  https://lore.kernel.org/all/87h72q9bgg.fsf@mpe.ellerman.id.au/T/#m068009a89d2412dd4ca14ac598a7d6b8288d4568

Or the end state is attached below.

cheers


==== arch/powerpc/kernel/setup-common.c ====
...

size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs)
{
	if (max_longs && ppc_md.get_random_seed && ppc_md.get_random_seed(v))
		return 1;

	return 0;
}
EXPORT_SYMBOL(arch_get_random_seed_longs);

...

==== arch/powerpc/include/asm/archrandom.h ====
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _ASM_POWERPC_ARCHRANDOM_H
#define _ASM_POWERPC_ARCHRANDOM_H

static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs)
{
	return 0;
}

size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs);

#ifdef CONFIG_PPC_POWERNV
int pnv_get_random_long(unsigned long *v);
#endif

#endif /* _ASM_POWERPC_ARCHRANDOM_H */

Re: [GIT PULL] Bitmap patches for v6.0-rc1

[Linus Torvalds]

On Fri, Aug 5, 2022 at 11:11 PM Michael Ellerman <mpe@ellerman.id.au> wrote:
>
> This pull request has a conflict with the random tree in some powerpc
> code.

I noticed.

And I tried to do a cross-compile, but with my update to F36 the
powerpc64 cross-tools seems to be very broken. I get lots of strange
errors like

    Cannot find symbol for section 11: .text.kgdb_arch_pc.

and I have no idea why.  I used to have a working cross-build
environment at one point, but now it just gives me lots of errors.

The gcc plugin situation also seems broken, and I assume it's just
more of the same breakage.

End result: I tried to make sense of the merge conflicts as best I
could, but I might have completely screwed it up and wasn't really
able to even do a test build of it.

           Linus

Re: [GIT PULL] Bitmap patches for v6.0-rc1

[Michael Ellerman]

Linus Torvalds <torvalds@linux-foundation.org> writes:
> On Fri, Aug 5, 2022 at 11:11 PM Michael Ellerman <mpe@ellerman.id.au> wrote:
>>
>> This pull request has a conflict with the random tree in some powerpc
>> code.
>
> I noticed.

Thanks. Your resolution looks fine and builds for me.

> And I tried to do a cross-compile, but with my update to F36 the
> powerpc64 cross-tools seems to be very broken. I get lots of strange
> errors like
>
>     Cannot find symbol for section 11: .text.kgdb_arch_pc.
>
> and I have no idea why.  I used to have a working cross-build
> environment at one point, but now it just gives me lots of errors.

If you just build defconfig rather than allmodconfig it should build
cleanly.

That's the "recordmcount" issue that's been going on for the last ~year.
It's some bad interaction between recordmcount and weak symbols and
newer versions of binutils.

Naveen made an attempt to fix it, but the discussion with Steve fizzled
out inconclusively.

AIUI objtool doesn't have the same problem so hopefully we can switch to
objtool soon and avoid the issue.

cheers

Re: [GIT PULL] Bitmap patches for v6.0-rc1

[Linus Torvalds]

On Thu, Aug 4, 2022 at 7:29 PM Yury Norov <yury.norov@gmail.com> wrote:
>
>   https://github.com/norov/linux.git/ tags/bitmap-6.0-rc1

Grr.

So I delayed this because I was anticipating pain, but it was worse
than expected.

I think I've sorted things out, but part of sorting things out was to
just basically undo this:

>       lib/bitmap: change type of bitmap_weight to unsigned long

which just was pointless and the explanation for it didn't match what it did.

The explanation was "unsigned makes sense". Ok.

But what it *did* was not to make the return value unsigned, but to
also expand it from "int" to "long".

That did *not* make sense, and caused pointless changes to printf strings.

And - surprise surprise - new users had come in since, so there would
have been even *more* pointless changes to printf strings just to
resolve those.

So I basically undid almost all of it. I did leave the "unsigned"
part, but removed the "long" part, so at least the printf strings
noise didn't happen.

That still caused the type clash in the 'min()' in the
mellanox/mlx4/fw.c file, but at least I couldn't find any new cases of
that pattern, so it seemed acceptable.

Can bitmap sizes in theory be "unsigned long" bits and thus have
"weight" not fit in "int"? Yes. But it's not a realistic concern, I
feel, and I really really didn't want to deal with this pointless
churn.

And honestly, this was enough churn that I seriously just considered
throwing the whole pull request away.  And I feel like I will have to
do a few other architecture tests too before I actually push this out,
so I may end up still doing that if that shows something else wrong.

So for any future pull requests, please

 (a) make them more targeted

 (b) avoids things like that return value change that clearly was just
churn and affected random files in the tree

because right now I'm left with a "never again" feeling about this all.

                Linus

Re: [GIT PULL] Bitmap patches for v6.0-rc1

[Yury Norov]

On Sun, Aug 07, 2022 at 05:12:33PM -0700, Linus Torvalds wrote:
> On Thu, Aug 4, 2022 at 7:29 PM Yury Norov <yury.norov@gmail.com> wrote:
> >
> >   https://github.com/norov/linux.git/ tags/bitmap-6.0-rc1
> 
> Grr.
> 
> So I delayed this because I was anticipating pain, but it was worse
> than expected.
> 
> I think I've sorted things out, but part of sorting things out was to
> just basically undo this:
> 
> >       lib/bitmap: change type of bitmap_weight to unsigned long
> 
> which just was pointless and the explanation for it didn't match what it did.
> 
> The explanation was "unsigned makes sense". Ok.
> 
> But what it *did* was not to make the return value unsigned, but to
> also expand it from "int" to "long".
> 
> That did *not* make sense, and caused pointless changes to printf strings.
> 
> And - surprise surprise - new users had come in since, so there would
> have been even *more* pointless changes to printf strings just to
> resolve those.

That's true. I checked the generated code for arm64 and x86, and long
is nothing better than int. I was sure that there was a difference, but
it's obviously a false memory.

I wish I could rebase everything on top of origin/master before
sending a pull request. It would help me make sure that everything
will be OK on your side, but it doesn't work for you for other reasons.
 
> So I basically undid almost all of it. I did leave the "unsigned"
> part, but removed the "long" part, so at least the printf strings
> noise didn't happen.
> 
> That still caused the type clash in the 'min()' in the
> mellanox/mlx4/fw.c file, but at least I couldn't find any new cases of
> that pattern, so it seemed acceptable.
> 
> Can bitmap sizes in theory be "unsigned long" bits and thus have
> "weight" not fit in "int"? Yes. But it's not a realistic concern, I
> feel, and I really really didn't want to deal with this pointless
> churn.

Many bitmap functions use int type to store the length, so currently
bitmaps can't be larger than INT_MAX, even in theory.
 
> And honestly, this was enough churn that I seriously just considered
> throwing the whole pull request away.  And I feel like I will have to
> do a few other architecture tests too before I actually push this out,
> so I may end up still doing that if that shows something else wrong.
> 
> So for any future pull requests, please
> 
>  (a) make them more targeted
> 
>  (b) avoids things like that return value change that clearly was just
> churn and affected random files in the tree
> 
> because right now I'm left with a "never again" feeling about this all.

I understand that. I'm not happy as well. I'll be more careful with
pull requests. Sorry again.

Thanks,
Yury

Re: [GIT PULL] Bitmap patches for v6.0-rc1

[pr-tracker-bot]

The pull request you sent on Thu,  4 Aug 2022 19:29:40 -0700:

> https://github.com/norov/linux.git/ tags/bitmap-6.0-rc1

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/1ab9250751ee91b0ca7bee887894fccfd44ff26a

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html

Re: [GIT PULL] Bitmap patches for v6.0-rc1

[Linus Torvalds]

On Sun, Aug 7, 2022 at 5:48 PM <pr-tracker-bot@kernel.org> wrote:
>
> https://git.kernel.org/torvalds/c/1ab9250751ee91b0ca7bee887894fccfd44ff26a

Oh Christ, and then after all that, I pushed out the wrong merge tree
that didn't have the final fix committed.

I've done a force-push and hope nobody saw that wrong head.

This was even more painful than it needed to be with that
self-inflicted unforced error.

But hopefully people never noticed, because nobody sane is looking at
my tree on a Sunday evening.

            Linus

Re: [GIT PULL] Bitmap patches for v6.0-rc1

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 319 bytes --]

Hi Linus,

On Sun, 7 Aug 2022 17:57:35 -0700 Linus Torvalds <torvalds@linux-foundation.org> wrote:
>
> But hopefully people never noticed, because nobody sane is looking at
> my tree on a Sunday evening.

Well, for some of us it is Monday already :-)  I did not notice though.

-- 
Cheers,
Stephen Rothwell

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]